How to Gain Real Savings Through a Proper Cloud Setup

The Problem

While reviewing a customer’s AWS bill recently, I noticed line-item entry for $14,298 on their bill for Amazon Elastic Compute Cloud NatGateway. That’s a lot of money being spent on internet traffic to and from their account. The AWS NatGateway has not only an hourly charge of $0.045/hour, but also a usage charge of $0.045/GB Data Processed. As you can see in the illustration above, this customer for the month had generated 317,742 GB of traffic through their NatGateway.
The question I immediately had was, “What type of traffic is this?” After analyzing their VPC Flow Logs, I discovered that almost 200,000 GB of the traffic was talking to the AWS services DynamoDB and S3. See when you communicate with DynamoDB, S3, and some other AWS services, the endpoint for the communication is a public internet address. This allows users both inside and outside AWS’s cloud to use the services. And since this is internet traffic, the NatGateway charges accumulated quickly.

The Solution

So how to you lower these costs? There are multiple answers here.

VPC Endpoint for S3

The quickest and easiest solution was to add a VPC Endpoint for S3 to the customer’s VPCs. This modifies the customer’s VPC routing tables to allow them to access S3 from within AWS. This means that the traffic goes straight to S3, bypassing the internet and most importantly, bypassing the NatGateway. This change takes just a few minutes to make, causes no disruption of services, and, in this case, quickly eliminated 120,000GB of traffic being processed by the NatGateway. That’s a savings of $5,400/month!

Kicking Lambda Functions Out of the VPC

All of the Lambda functions that were configured in the customer’s account were setup to be inside their VPCs. Most of these Lambda functions made no use of resources inside the VPC; instead, they were consuming resources from external data sites and S3. The functions also stored data on S3, DynamoDB, and Kinesis Firehose. And, although configuring the VPC Endpoint for S3 previously had prevented some of this traffic from going throught the NatGateway, traffic destined to external sites, DynamoDB, and Kinesis Firehose were getting through. Unfortunately, AWS has not released VPC Endpoints for DynamoDB and Kinesis yet, although the DynamoDB one is in Public Preview in certain regions, at the time of this writing.
Changing the functions to no longer run inside the VPC took about an hour’s worth of work, devoted to modifying and testing their deployment jobs. This relatively quick change cut 54,000GB of traffic from going through the NatGateway, resulting in an additional savings of $2,430/month.

Moving EC2 Data Vacuums to Public Subnets

The last major consumer of internet traffic were data vacuum worker EC2 instances that pulled data from remote sources. Since the instances are still protected by AWS Security Groups, we decided there was no risk in moving these instances outside the private subnets and into public ones. When an instance is in a public subnet, all internet traffic is processed by the VPC Internet Gateway, which incurs no expense for processing the data. Sure — there’s still an expense for data leaving the VPC onto the internet — but that cost is also added if you’re using a NatGateway also. Usage was reduced by 33,000GB, which resulted in savings of $1,485/month.

Conclusion

By knowing what services were available from AWS, and how the customer’s systems interacted not with just AWS, but also their external vendors systems also, Blue Sentry was able to save the customer over $9,300/month which adds up to almost $112,000/year. The customer had no downtime or system degradation because of these changes.

Want to optimize your AWS Cloud environment to realize real savings to your bottom line? Schedule a 15-minute, free, no-obligation appointment with us today!

Update: In June 2017, we were able to assist our customer in gaining access to the AWS VPC Endpoint for DynamoDB Preview.  After being accepted into the preview, the changes to the customer’s environment took less than 1 hour with zero downtime.  The result was even more savings as reflected in their July 2017 bill, another $2500 in savings.  Since making these adjustments savings have totaled 83%/month.