AWS- Deploying Web Servers in Private Subnets

In this post, I will attempt to explain how to deploy Web Servers in private subnets. The common deployment pattern followed by newbies is to deploy Web Servers in public subnets or edge network and use an Elastic Load Balancer to route traffic to them. However, one may want to deploy such servers in private subnets or core network owing to better security and/or company network policies. So let us skip the fluff and get down to the nitty gritty.

Assumptions
I assume the reader is modestly aware of AWS services like Elastic Compute Cloud (EC2), Elastic Load Balancer (ELB); has knowledge of Virtual Private Cloud (VPC) and can spin up EC2 instances.

Objective
The objective is to present a highly available (in a single region) and secure deployment solution for a set of Web Servers.

Explanation
I shall use the diagram below, to explain the deployment.

Diagram-1

The solution has the following significant components:

• The solution spans two AZs to provide high availability.

• Subnets - Each AZ has a pair of private and public subnets.

• Two private subnets meant to house the Web Servers

• Two public subnets meant to house the ELB nodes, NAT instance/s and bastion host/s

• Internet Gateway

• An Internet Gateway attached to the VPC

• Security Groups

• A security group (demo-public-secgrp) for the NAT instance and ELB nodes in the public subnets which allows HTTP and HTTP(S) traffic from any source

• An internal security group (demo-private-secgrp) for all Web Server instances in the private subnets that allows HTTP and HTTP(S) traffic originating only from the instances behind the public security group. This security group also allows SSH traffic originating only from the instances behind the bastion security group (demo-bastion-secgrp: described below)

• A special security group (demo-basion-secgrp) for the bastion host deployed in the public subnet that allows SSH access (@ port 22). Ideally, SSH/RDP access should be allowed from trusted networks only (like home/office networks) and should not be opened for ALL (0.0.0.0/0). However for the purpose of this example, I have kept the SSH access to the bastion host open to all.

• Route Tables

• A route table for the public subnets to route all VPC local traffic and Internet bound traffic to the Internet Gateway

• A route table for the private subnets to route all VPC local traffic and Internet bound traffic to the NAT instance in the public subnet. The instances in the private subnet cannot access the Internet directly. They have to go via the NAT instance.

• NAT Instance

• A NAT instance deployed in one of the public subnets and placed behind the public security group (demo-public-secgrp). The Web Servers in the private subnet uses this NAT instance to access the Internet.

• Bastion Host

• A bastion host deployed in one of the public subnets and placed behind the special security group (demo-basion-secgrp). The bastion host is used to administer the Web Servers in the private subnets

• EC2 Instances

• One EC2 instance in each AZ, deployed in the private subnet behind the internal security group- demo-private-secgrp

• These instances house the Web Servers (like Apache HTTP Server)

• Elastic Load balancer

• An application load balancer deployed across the public subnets in the two AZs and placed behind the public security group (demo-public-secgrp)

• The target group of the application load balancer has the EC2 instances as the registered targets

Points to ponder

• NAT gateway could be used in place of a NAT instance, in the public subnet. In fact, NAT gateways are recommended as they provide better availability and bandwidth over NAT instances. The NAT gateway service is also a managed service that does not require administration efforts.
• The bastion host could be deployed in a separate subnet to reduce the blast radius of any attack and should be hardened to reduce attack vectors.
• In Production, instead of storing private SSH keys in the bastion hosts which leads to security vulnerability, place them in local computer and use SSH agent forwarding to connect to the EC2 instances in a private subnet through the bastion host (Here's a relevant post by Mike Pope explaining how to establish SSH agent forwarding)
• NAT instance/s and bastion host/s could be deployed in a high availability mode.
• In the diagram above, I have chosen to distribute the ELB traffic across two manually created EC2 instances. An auto-scaling group might be used for higher elasticity.
• I have chosen to use the default NACL that comes along with the custom VPC. This NACL allows all inbound and outbound traffic, by default. However, some rules could be added to the NACL purely based on needs.