Robust AWS VPC Design for High Availability and Security

Introduction

In today’s cloud-centric world, a secure and scalable network infrastructure is essential. Amazon Web Services (AWS) offers Virtual Private Cloud (VPC), enabling users to create isolated networks tailored to their needs.

This project will guide you through setting up a comprehensive VPC environment on AWS, covering key components such as subnets, gateways, load balancers, and security measures.

By the end of this tutorial, you’ll learn to build and manage a robust VPC architecture, ensuring your cloud resources are secure and highly available.

Feel free to connect with me on LinkedIn to discuss this post, or ask any questions.

Project Overview

This project demonstrates how to create a Virtual Private Cloud (VPC) suitable for hosting servers in a production environment.

To enhance resilience, servers are deployed across two Availability Zones using an Auto Scaling Group and an Application Load Balancer.

For additional security measures, servers are placed in private subnets, while a public subnet allows inbound traffic through the Application Load Balancer.

Servers can access the internet via a NAT Gateway, which is deployed in both Availability Zones for improved resilience.

An Internet Gateway facilitates outbound Internet access for resources within the VPC.

A bastion host is used alongside Security Groups, EC2 instances, and Target Groups to securely manage access to resources within the VPC.

NOTE: Most of these step can also be automated using Terraform, but for the sake of simplicity, we will do it manually here.

Tools and Technologies Used

In this project, we leverage several key tools and technologies to build a robust Virtual Private Cloud (VPC) environment on AWS:

1: Amazon VPC (Virtual Private Cloud): Provides a logically isolated network to deploy AWS resources.

2: Availability Zones: Geographic regions with independent infrastructure to enhance fault tolerance and stability.

3: Subnets:

• Public Subnet: Allows resources to be accessible from the internet.
• Private Subnet: Provides enhanced security by isolating resources from direct internet access.

4: NAT Gateway: Enables instances in private subnets to access the internet while maintaining security.

5: Internet Gateway: Facilitates outbound internet access for resources within the VPC.

6: Application Load Balancer (ALB): Distributes incoming traffic across multiple targets such as EC2 instances, enhancing fault tolerance and scalability.

7: Security Groups: Acts as virtual firewalls to control inbound and outbound traffic at the instance level.

8: EC2 Instances: Virtual servers running in the AWS cloud that provide compute capacity.

9: Auto Scaling Group: Ensures the availability and scalability of EC2 instances by automatically adjusting capacity based on traffic demands.

10: Bastion Host: Acts as a secure gateway to access and manage instances in private subnets.

11: Target Groups: Directs Application Load Balancer traffic to registered targets, such as EC2 instances.

These tools and technologies collectively enable the creation of a resilient, scalable, and secure VPC environment suitable for hosting production applications on AWS.

Creating VPC

Creating a Virtual Private Cloud (VPC) is the first step in setting up a secure and isolated network environment within AWS.

A VPC allows you to launch AWS resources, such as EC2 instances, in a logically isolated virtual network that you define.

This section will walk through the process of creating a VPC, which will serve as the foundation for our entire infrastructure.

This VPC will include public and private subnets across multiple availability zones to ensure high availability and security for our applications.

Note: This step can also be automated using Terraform, but for the sake of simplicity, we will do it manually here.

1: Sign in to AWS Console:

• Go to AWS Management Console and sign in using your credentials.
• After signing in, you’ll land on the AWS Management Console dashboard.

2: Navigate to VPC Dashboard:

• Search for “VPC” in the search bar and select it from the results.

3: Start VPC Wizard:

• In the VPC dashboard, click on the “Create VPC” button to start the VPC creation process.

4: Choose a VPC Configuration:

• AWS provides several VPC configuration options. Let’s choose the “VPC and more” option.
• Add a name tag to identify your VPC.

5: Configure VPC Details:

• Enter the IPv4 CIDR block for your VPC. This defines the range of IP addresses that can be used within your VPC.
• Optionally, you can also specify an IPv6 CIDR block for your VPC.
• Leave other settings as default or configure as needed based on your requirements.

6: Configure Subnets:

• For Number of Availability Zones, choose 2, so that you can launch instances in multiple Availability Zones to improve resiliency.
• For the Number of public subnets, choose 2.
• For the Number of private subnets, choose 2.
• For NAT gateways, choose 1 per AZ to improve resiliency.
• For VPC endpoints, if your instances must access an S3 bucket, keep the S3 Gateway default. Otherwise, instances in your private subnet can’t access Amazon S3. There is no cost for this option, so you can keep the default if you might use an S3 bucket in the future. If you choose None, you can always add a gateway VPC endpoint later on.

• The wizard will automatically create route tables for your subnets.
• Ensure that routes are correctly configured, such as routing internet-bound traffic through an Internet Gateway for public subnets.
• For DNS options, clear Enable DNS hostnames.

7: Review and Create:

• Review your configurations details you’ve chosen for your VPC.
• Choose “Create VPC”.

• AWS will now create your VPC based on the configuration you’ve specified.
• This process usually completes within a few minutes please be patient.

8: Verification:

• After the VPC is created, verify its details in the VPC dashboard.
• Ensure all components (VPC, subnets, route tables) are correctly configured according to your requirements.

Creating Auto Scaling Group

Auto Scaling in AWS empowers applications to automatically adjust compute capacity based on demand, optimizing performance and costs.

This guide explores the straightforward steps to create an Auto Scaling group using the AWS Management Console.

By setting up Auto Scaling, you can ensure your infrastructure scales efficiently to meet fluctuating workload requirements, enhancing the resilience and cost-effectiveness of your AWS deployments.

1: Sign in to AWS Console:

• Go to AWS Management Console and sign in using your credentials.

2: Navigate to Auto Scaling:

• Search for “Auto Scaling” in the search bar and select it from the results.

3: Create an Auto Scaling Group:

• Click on the “Create Auto Scaling group” button to start creating a new Auto Scaling group.

4: Choose Launch Template or Launch Configuration:

• Click on “Create Launch Template” if you don’t have an existing one, or choose an existing Launch Template from the dropdown.

5: Create a Launch Template (if applicable):

If creating a new Launch Template, provide the following details:

• Template name: Enter a name and description for your Template.

• AMI (Amazon Machine Image): Select an AMI for your EC2 instances.

• Instance type: Choose the instance type for your EC2 instances (e.g., t2.micro).
• Key pair (if needed): Specify a key pair for SSH access to instances.

• Configure other instance details such as networking, as per your requirements.
• Ensure you select the previously created VPC, we will be using this throughout the project.

5: Configure Security Group

• Create a Security Group.

• Ensure that the security group allows SSH access and custom TCP port 8000.

• Configure other instance details such as storage as per your requirements.
• Click on “Create launch template” once all details are configured.

• Verify that the Launch template was created successfully.

6: Configure Auto Scaling Group:

After selecting or creating a Launch Template, configure the Auto Scaling group settings:

• Group name: Provide a name for your Auto Scaling group.

• Network: Choose the VPC and subnet where instances will be launched. Select the two private subnets and for the VPC select the one we created previously.

• Group size: Define the initial number of instances (2).
• Scaling Limits: Choose the minimum(1) and maximum(4) desired capacity.

7: Configure Scaling Policies (if applicable):

• Configure scaling policies (optional).
• Click “Next”.

• Add Notifications (optional).

8: Review and Create:

• Review all the configuration details you’ve chosen for your Auto Scaling group.
• Double-check the Launch Template, scaling policies, networking settings, and any other configurations.
• Click on “Create Auto Scaling group” to initiate the creation process.

9: Wait for Auto Scaling Group Creation:

• AWS will now create your Auto Scaling group based on the configuration you’ve specified.
• This process may take a few moments to complete please be patient.

10: Verification:

• After the Auto Scaling group is created, verify its details in the Auto Scaling dashboard.

• Go to the EC2 dashboard and ensure instances are launched according to scaling policies ( 2 instances in our case) and that they are integrated in two availability zones.

Creating Bastion Host

In AWS environments, maintaining secure access to private instances within a Virtual Private Cloud (VPC) is crucial for effective management and troubleshooting.

A bastion host, or jump server, serves as a secure gateway allowing authorized users to securely connect to instances in private subnets.

• Go to AWS Management Console and sign in using your credentials.
• In the AWS Management Console, search for “EC2” in the search bar.
• Click on “Launch Instance” to start the instance creation process.
• Select an appropriate AMI for your bastion host.
• Select an instance type that meets your performance and capacity requirements. A t2.micro instance type is often sufficient for a bastion host.
• Select an existing key pair or create a new key pair to securely access your instance.
• Download the key pair file (.pem) and keep it safe. You cannot download it again after it is created. It will be required to connect to your bastion host via SSH.
• Choose the VPC you created earlier for your project.
• Select a public subnet within your VPC to ensure the bastion host has a public IP address and internet access.
• Auto-assign Public IP: Enable this option to assign a public IP address to the bastion host.
• Configure storage settings as needed. The default settings are typically sufficient for a bastion host.
• Add tags to the instance for easy identification and organization. For example, you can add a tag with Key “Name” and Value “Bastion Host”.
• Create a new security group or use an existing one that allows inbound SSH (port 22) access from your IP address (or IP range) for secure SSH connections.
• Optionally, configure additional rules for any other necessary access (e.g., HTTPS for web-based management).
• Review all the instance details you’ve configured.
• Click on “Launch Instances” to start the bastion host instance.

Access the Bastion Host

• Once the instance state changes to “running”, note down the public IP address of the bastion host.
• Use SSH and the key pair file (.pem) to securely connect to the bastion host from your local machine or terminal:

ssh -i /path/to/your-key.pem ec2-user@bastion-host-public-ip

• Replace /path/to/your-key.pem with the path to your downloaded key pair file, and bastion-host-public-ip with the actual public IP address of your bastion host.
• Verify that you can successfully connect to the bastion host using SSH.

Alternatively for Windows users if you have MobaXterm you have use it to ssh into the bastion host like we have been doing with ec2 instances on our other project demos.

Securely Copying PEM File to Bastion Host

To securely access EC2 instances via SSH through a bastion host, you first need to transfer your SSH key (.pem file) to the bastion host.

This is particularly crucial when the EC2 instance resides in a private subnet without direct internet access.

In such cases, the bastion host serves as a secure gateway, allowing controlled SSH access from your local machine to the private instances within the AWS Virtual Private Cloud (VPC).

Below are the steps for securely copying your SSH key using the scp command.

• Ensure you have the .pem file (your private key) that you downloaded when you created the EC2 instance.
• Before transferring the file, make sure its permissions are set correctly to prevent unauthorized access:

chmod 400 /path/to/your-key.pem

• Open a terminal or command prompt on your local machine.
• Use the scp command to securely copy the .pem file to the bastion host.

scp -i /path/to/your-key.pem /path/to/your-key.pem ec2-user@bastion-host-public-ip:/home/ec2-user/

• Replace /path/to/your-key.pem with the actual path to your .pem file, and bastion-host-public-ip with the public IP address of your bastion host.
• Use /home/ubuntu/ instead of /home/ec2-user/ if you choose an Ubuntu AMI.

Example Sample:

scp -i ~/.ssh/my-key.pem ~/.ssh/my-key.pem ec2-user@123.456.789.0:/home/ec2-user/

• This command securely copies the .pem file to the home directory (/home/ec2-user/) of the ec2-user on the bastion host.
• If you’re prompted to enter a password, use the password associated with the SSH key pair for authentication.

• After the transfer completes, verify that the .pem file is in the correct directory on the bastion host:

ls /home/ec2-user/

• If you used an Ubuntu AMI, use:

ls /home/ubuntu/

SSH into EC2 Instance via Bastion Host

Once the .pem file is securely copied to the bastion host, you can proceed to SSH into your EC2 instance using the bastion host as a gateway:

1: SSH into Bastion Host:

We had done this in the above steps

ssh -i /path/to/your-key.pem ec2-user@bastion-host-public-ip

2: SSH into EC2 Instance from Bastion Host:

Once connected to the bastion host, SSH into the desired EC2 instance within your VPC using its private IP address and the .pem file:

ssh -i /home/ec2-user/your-key.pem ec2-user@private-ip-of-ec2-instance

Example for Ubuntu:

ssh -i /home/ubuntu/your-key.pem ubuntu@private-ip-of-ec2-instance

Creating a Simple HTML Application

After connecting to the EC2 instance via the Bastion host, if you don’t have an application to deploy, you can create a simple HTML application for demonstration purposes.

• On your EC2 instance, use vim to create and edit an HTML file:

vim index.html

• Press i to enter insert mode in vim.
• Paste the following HTML content:

<!DOCTYPE html>
<html>
<head>
<title>Page Title</title>
</head>
<body>

<h1>My First AWS Project to demonstrate apps in private subnet</h1>

</body>
</html>

• Press Esc to exit insert mode.
• Type :wq and press Enter to save the file and exit vim.
• You can list the directory contents to confirm that index.html is created:

ls

Note: We intentionally deployed the application on only one instance to check if the load balancer will distribute 50% of the traffic to one instance, which will receive a response, and 50% to another instance, which will not receive a response.

This concept will be explained in the next part where we will create an Application Load Balancer.

Viewing Your HTML Application

To view your HTML application, you need to serve it through a web server. For simplicity, you can use Python’s built-in HTTP server.

1: Install Python (if not already installed):

sudo yum install python3 -y

For Ubuntu, use:

sudo apt-get install python3 -y

2: Start the HTTP Server:

python3 -m http.server 8000

Creating an Application Load Balancer

An Application Load Balancer (ALB) in AWS distributes incoming application traffic across multiple targets, such as EC2 instances, in different availability zones.

This ensures high availability and fault tolerance for your applications.

1: Sign in to AWS Console:

• Go to the AWS Management Console and sign in using your credentials.

2: Navigate to the EC2 Dashboard:

• In the AWS Management Console, search for “EC2” in the search bar.

3: Access Load Balancers:

• In the left-hand menu, under “Load Balancing,” click on “Load Balancers.”

4: Create Load Balancer:

• Click on the “Create Load Balancer” button.
• Choose “Application Load Balancer” and click on “Create.”

5: Configure Load Balancer Basics:

• Enter a name for your load balancer.
• Choose “internet-facing” to make the load balancer accessible over the internet.
• Choose “ipv4”.

6: Configure Network and Listeners:

• Select the VPC you created earlier.
• Select at least two availability zones and their corresponding public subnets.
• By default, a listener on port 80 (HTTP) is created. You can add more listeners if needed.

7: Configure Security Groups:

• Choose an existing security group or create a new one that allows inbound HTTP traffic (port 80).
• Ensure this security group also allows any other required inbound traffic.

8: Configure Routing:

• Create a new target group.
• Enter a name for your target group.
• Choose “Instance” to register EC2 instances.
• Select “HTTP” enter port 80
• Configure the health check settings (default settings are usually sufficient).

9: Register Targets:

• Select the EC2 instances you want to include in this target group.
• Click “Include as pending below”.

• Click “Create target group”.

10: Review and Create:

• Review all your settings for the Application Load Balancer.
• Click “Create” to launch the load balancer.

11: Verify Load Balancer:

• After the ALB is created, you will see it listed in the Load Balancers section.
• Note the DNS name of the ALB. This is the endpoint you will use to access your application.

Verification

1: Access the Load Balancer DNS:

• Open a web browser and navigate to the DNS name of the load balancer.

• You should see your application, verifying that the ALB is correctly distributing traffic.

2: Check Traffic Distribution:

• As noted earlier, only one instance has the application deployed.
• Access the ALB DNS multiple times to observe if 50% of the requests go to the instance with the application (receiving a response) and 50% go to the instance without the application (not receiving a response).

Clean up Resources

After completing your AWS VPC architecture project, it’s essential to clean up resources to prevent incurring unnecessary charges.

AWS bills for resources such as EC2 instances, NAT Gateways, and Load Balancers, even if they are not actively used.

Follow these steps to delete resources effectively:

1: Terminate EC2 Instances:

• Navigate to the EC2 Dashboard.
• Select the instances you created for the project.
• Click on “Instance State” and choose “Terminate.”
• Confirm the termination.

2: Delete Auto Scaling Groups:

• Go to the Auto Scaling Dashboard.
• Select the Auto Scaling group you created.
• Click on “Actions” and choose “Delete.”
• Confirm the deletion.

3: Remove Load Balancers:

• In the EC2 Dashboard, click on “Load Balancers.”
• Select the Application Load Balancer you created.
• Click on “Actions” and choose “Delete.”
• Confirm the deletion.

4: Delete NAT Gateways:

• Navigate to the VPC Dashboard.
• Click on “NAT Gateways.”
• Select the NAT Gateway you created.
• Click on “Actions” and choose “Delete NAT Gateway.”
• Confirm the deletion.
• Ensure to release the Elastic IP associated with the NAT Gateway to avoid charges.

5: Remove Security Groups:

• Go to the VPC Dashboard.
• Click on “Security Groups.”
• Select the security groups created for the project.
• Ensure no instances are using them, then delete the security groups.

6: Delete Subnets and VPC:

• In the VPC Dashboard, click on “Subnets.”
• Select each subnet created and delete them.
• After deleting all subnets, go back to the VPC Dashboard.
• Select the VPC you created and delete it.

7: Remove Internet Gateways:

• Navigate to the VPC Dashboard.
• Click on “Internet Gateways.”
• Detach the Internet Gateway from your VPC.
• Once detached, delete the Internet Gateway.

8: Release Elastic IP Addresses:

• Go to the EC2 Dashboard.
• Click on “Elastic IPs.”
• Select the Elastic IPs you allocated.
• Click on “Actions” and choose “Release Elastic IP Addresses.”

9: Delete Key Pairs:

• In the EC2 Dashboard, click on “Key Pairs.”
• Select the key pairs created for the project.
• Delete the key pairs.

Conclusion

In this project, we successfully created a robust Virtual Private Cloud (VPC) infrastructure suitable for a production environment.

By deploying resources across multiple availability zones, we ensured high availability and resiliency. Key components included public and private subnets, a bastion host for secure access, an auto scaling group for automated scaling of EC2 instances, and an Application Load Balancer to distribute incoming traffic efficiently.

This setup not only enhances security through the use of private subnets and security groups but also ensures that applications remain highly available and scalable.

This foundational architecture is critical for building resilient, secure, and scalable cloud applications.

Connect with me

For more insights and future projects, feel free to connect with me on LinkedIn , GitHub and Medium.