AWS EC2 Instance Classes and Types

There are different types of EC2 instances which you can choose for your application. There are varying combinations of CPU, memory, storage (EBS or Instance storage), and networking capacity. Broadly, we can classify EC2 instances as:

1. General Purpose
2. Compute Optimized
3. Memory Optimized
4. Storage Optimized
5. Accelerated Computing / GPU Optimized

1. General Purpose

Classes: T2 (T stands for Tiny) - Burstable Performance, M5 (M stands for Medium) - Fixed Performance

Suited for: Development environments, Small and mid-sized databases, Low-traffic web applications, Build servers, Code repositories, Testing and staging environments, Early product experiments etc.

Instance sizes: nano, micro, small, medium, large, xlarge, 2xlarge, 4xlarge, 8xlarge, 10xlarge

CPU Credits: T2 instances accrue CPU credits when they are idle, and use CPU credits when they are active. T2 instances are a good choice for workloads that don’t use the full CPU often or consistently, but occasionally need to burst (e.g. web servers, developer environments and small databases).

2. Compute Optimized

Classes: C5 (C stands for Compute)

3. Memory Optimized

Classes: X1, R4 (R stands for RAM)

Charged Per GB of RAM

4. Storage Optimized

Classes: I3 (I stands for I/O), D2 (D stands for Dense)

Suited for: Data Warehouse, intended for workloads that need greater sequential write and read access to larger data sets

5. Accelerated Computing / GPU Optimized

Classes: P3, G3 (G stands for Graphics), F1

Uses NVIDIA GPU

Suited for: Low Latency, High Throughput, High IOPS, Machine Learning, Deep Learning, Big Data Analytics, In-memory Analytics, Reporting, Graphics, Gaming, Video Encoding, High Performance Databases, Batch Processing, Streaming, Speech Recognition, Ad Serving, 3D Visualization, Distributed Analytics, Computational Finance, Financial Analytics, Computational Fluid Dynamics, Genomics Research, Drug Discovery, Scientific Modelling, Molecular Modeling, Media Transcoding etc.

Difference between EC2 and Lightsail in AWS (EC2 vs Lightsail)

EC2 and Lightsail are compute services offered by AWS. Broadly, you can say Lightsail is the lighter version of EC2 where you don't need to manually configure underlying infrastructure like EBS, EFS, VPC, Subnets, Storage Groups, ACL etc. Following are the basic points about Lightsail to consider:

1. With EC2, you have to manually configure Storage and Networking. But if you don't want to take headache of underlying infrastructure such as Storage and Networking, you can go with Lightsail.

2. Lightsail is a VPS (Virtual Private Server) service in which the above mentioned infrastructure is in-built.

3. Lightsail provides: 

• VPS (Virtual Private Server with Autoscaling)
• Storage (like EBS, EFS for EC2)
• Networking (like VPC, Subnet, SG, NACL for EC2)
• Load Balancing (like ELB)
• API (Application Program Interface)
• Integration with other AWS services via VPC peering

4. Backup: You can easily create snapshot of you Lightsail VPS.

5. In short, Lightsail is the simpler and lighter version of EC2 with limited functionalities. The target market for Lightsail appears to be those who just want a simple VPS without going into the complexities of EC2. Later on, you can easily switch from Lightsail VPS to EC2.

AWS IAM: Identity and Access Management in AWS

Identity and Access Management is a very useful service offering from AWS. IAM is used to authenticate and authorize the users and AWS services to use AWS resources. Below are the basic points to not about AWS IAM:

1. Authentication and Authorization: Control access to AWS resources for your users. For example, developer should only be able to access compute and storage resources, DBA should only be able to access database resources etc.

2. Components: Users, Groups, Policies, Roles

3. Root User/Account: User/Account with which you have created your AWS account. It has all the access. It is advisable not to use root account. Instead, create an Admin account and provide all the access. Keep it for emergency purpose. 

4. User Access Type: You can provide two types of access to user: Programmatic Access (Access Key ID and Secret Access Key), Console Access (Password).

5. Access Key ID and Secret Access Key: You have to note down the Secret Access Key, once lost, you need to regenerate it.

6. User Login URL: https://your_account_name.signin.aws.amazon.com/console

7. Groups and Policies: Instead of assigning policies to individual users, it is recommended to create a group and assign the policies to that group. Now keep adding/removing users to that group. For example, if you want to create 5 developer accounts and want to assign same policies to them, instead of assigning those policies to individual accounts 5 times, better create a group say “Developer_Group”, assign those policies to this group and add all those 5 users to this group. Later, you can add/remove users to/from this group.

8. Roles: Set of permissions. Assigned to AWS services. For example: Create a role of type “Amazon EC2”, assign permission “AmazonS3FullAccess”. Now assign this role to any EC2 instance (Actions -> Instance Settings -> Attach/Replace IAM Role). Now any application deployed on this EC2 instance will be able to communicate with S3. 

Example: Suppose you have created a web application which uploads a file to S3. If you run this web application on the EC2 instance which has above role assigned, your file will be uploaded to S3 successfully. But if you run the same web application to any other EC2 instance which does not have above roles assigned to it, you will get access denied error. 

Now there are two ways to run this web application successfully on this server. Either assign the above role to this EC2 instance or mention Access Key ID and Secret Access Key (of the user who has S3 access) in your web application code.

9. User vs Role: User and Roles are similar components. We need to attach permission to them. “User” is created for people while “Role” is created for AWS resources.

10. Policies: Policies are permissions. You can also create your own policies using “Policy Generator” or using JSON code. If you assign both “Allow” and “Deny” policy to a user, “Deny” will be given priority. 

11. MFA (Multi Factor Authentication): Multi-layer of security. Just like OTP. You need to manage a Virtual MFA Device. To do this, click on to activate a virtual MFA device, a bar code will be displayed, download Google Authenticator App, scan the bar code shown in console, now two authentication codes will be generated, write them to the console and that’s it. Now if the user with MFA logs in to the console, he/she has to provide MFA code also. 

12. Global Service: IAM is not region specific, it is global service.

13. Eventual Consistency: There is eventual consistency when you change any settings like policy/roles/permissions

14. Free to use

Elastic Beanstalk: PaaS offering from Amazon

Elastic Beanstalk is a simple way to deploy your application on AWS. No need to take headache of managing the infrastructure. Below are some basic point to remember about Elastic Beanstalk:

1. PaaS offering from Amazon.

2. Platforms supported: PHP, Java, Python, Ruby, Node.js, .NET, Go and Docker

3. Application Deployment: Just upload your application code (packaged code and libraries) and the service automatically handles all the details such as resource provisioning, load balancing, auto-scaling, and monitoring.

4. Resources used by Elastic Beanstalk: Elastic Beanstalk uses core AWS services such as Amazon EC2, Amazon Elastic Container Service (Amazon ECS), Auto Scaling, and Elastic Load Balancing to support your applications.

5. Monitoring and Logging: You can easily monitor and manage the health of your applications. Logs are created and saved in S3.

6. Application Versioning: You can maintain multiple versions of your application. Application versions are saved in S3.

7. Default EBL URL: Once you deploy your application, a default EBL URL will be created (for example: your_application_name.elasticbeanstalk.com). If ELB is not used, URL will point to your EC2 instance directly.

8. Free of cost: You will be charged only for the resources it launches.

Difference between Internet Gateway and NAT Gateway (Internet Gateway vs NAT Gateway)

There is a minor and confusing difference between Internet Gateway and NAT Gateway. Below are some points to illustrate the difference between both:

1. Internet Gateway is used to connect a VPC to the internet while a NAT Gateway is used to connect the private subnet to the internet (through public subnet and Internet Gateway).

2. NAT Gateway cannot work without Internet Gateway. Your VPC must have Internet Gateway otherwise how NAT Gateway will direct traffic to the internet? NAT Gateway should always be launched in the public subnet where there is a route out Internet Gateway. If there is no route out to the Internet Gateway, NAT Gateway would not be able to connect the instances in the private subnet with internet.

3. NAT Gateway and NAT Instances only support IPv4 addresses while Internet Gateway supports both.

4. NAT Gateway supports only one way communication (from private subnet to internet and not vice-versa) while Internet Gateway supports both inbound and outbound traffic.

I have written a detailed article on NAT Gateway and NAT Instances here. Hope this might help.

Difference between NAT Instances and NAT Gateways (NAT Instances vs NAT Gateways)

NAT stands for Network Address Translation.

We launch many instances in private subnet in the VPC for security reasons. These instances cannot communicate with the internet. But there are many scenarios when these instances need to connect to internet like for patch updates, software installation, connection to Git repository etc. In these scenarios we need to make these instances communicate with the internet. 

NAT Instances and NAT Gateways come handy in these cases. These allow only outbound traffic to internet and restrict inbound traffic from internet. It means our instances in the private subnet can make connection to internet but nobody from the internet can access our instances in private subnet. 

Below are some basic points and differences between NAT Instances and NAT Gateways:

NAT Instances

1. NAT instance is like an EC2 instance and is also launched like an EC2 instance from AWS console.

2. It should always be launched in the public subnet.

3. Once launched, you need to manually disable source/destination check (this option is available under Actions >> Networking). This is because, it will be sending and receiving traffic on behalf of other instances, so the source and/or destination might not be itself.

4. You need to manage this instance yourself like you manage your EC2 instances.

5. NAT instance should be assigned an Elastic IP (but you can also use public IP).

NAT Gateway

1. Managed by AWS (you need to manage NAT instances yourself).

2. Always deploy your NAT Gateway in public subnet.

3. You must allocate Elastic IP to your NAT Gateway (you can allocate public IP to NAT instances).

4. In the main Route Table of your VPC (or the Route Table connected to private subnet), add a route out to this NAT Gateway. Set Destination as 0.0.0.0/0 and set target as NAT Gateway.

5. You cannot assign security groups to NAT Gateway (you can assign security groups to NAT instances).

6. You need one in each AZ since they only operate in a single AZ.

Note: Both NAT Instances and NAT Gateway only support IPv4 traffic (not IPv6).

Documentation: NAT Instances, NAT Gateway, Difference between NAT Instance and NAT Gateway

Difference between CloudSearch and ElasticSearch in AWS (CloudSearch vs ElasticSearch)

Both CloudSearch and ElasticSearch use powerful underlying search engines and enable you to search and analyze the data. Both are listed under the Analytics services in AWS console. Below are the basic differences between CloudSearch and ElasticSearch: 

CloudSearch

1. Custom search service for your website or application.

2. CloudSearch uses open source Apache Solr as the underlying search engine.

3. Supports 34 languages and popular search features such as highlighting, autocomplete, and geospatial search.

4. It requires data to be loaded as documents and is good for full-text search, with an understanding of languages and grammar (example, synonyms, words to ignore etc.).

5. You can create a search domain and upload the data that you want to make searchable, and Amazon CloudSearch will automatically provision the required resources and deploy a highly tuned search index.

ElasticSearch

1. The service offers open-source Elasticsearch APIs, managed Kibana, and integrations with Logstash (ELK stack) and other AWS Services, enabling you to securely ingest data from any source and search, analyze, and visualize it in real time.

2. It is commonly used for near real-time visualizations of logs files and data analytics.

3. The service also offers built-in integrations with other AWS services such as Amazon Kinesis Data Firehose, AWS IoT, and Amazon CloudWatch Logs for data ingestion; AWS CloudTrail for auditing; Amazon VPC, AWS KMS, Amazon Cognito, and AWS IAM for security.