Director of Engineering Emily Dresner faced a mammoth challenge – to build a scalable travel engine that could ingest and assimilate terabytes of data every day without the need to constantly provision new storage, an industry-changing mobile application, and a rapid-value delivery pipeline for new product features, all in a very short time frame. “For Upside, the name of the game was speed and agility, as we raced toward launch of our much-anticipated new product,” says Dresner. “We needed to learn quickly what worked and what didn’t work, and then instantly adapt,” she added.
For this business travel company, there was really never a doubt that the solution would be deployed on Amazon Web Services (AWS). The maturity, scalability and API-driven nature of the AWS platform made it uniquely matched to their needs. Although they had a very capable internal team, Dresner wanted a partner with deep expertise in DevOps automation, as well as experience deploying and managing environments at scale on AWS, to ensure that the company hit its tight timeline with a minimal learning curve.
This business travel company’s search for a DevOps partner led it to Blue Sentry, an Advanced Tier Consulting Partner in the Amazon Web Services Partner Network (APN). Blue Sentry’s expertise in DevOps, data services, and mobile apps was a perfect fit for the deployment challenge, and its status as an AWS Managed Service Partner gave Dresner the confidence that, once deployed, the solution could be expertly managed at scale.
Their team had taken a micro-services approach to architecting the application, so that development could happen in parallel and new features could be tested and deployed rapidly, independent of a monolithic build process. Static content was to be served via AWS Cloudfront, while dynamic content was provided from backend micro-services running on Lambda. This approach would scale automatically, ensuring it would always be “right-sized,” from the start-up phase up to its emergence as a platform handling millions of visitors per day.
DynamoDB and Aurora managed database solutions were used for their capabilities in providing unlimited capacity. EMR (Elastic Map Reduce) clusters were deployed by multiple data scientists, to facilitate the daily analysis of terabytes of new data along with accumulated historical data.
They also wanted product managers to make a manual decision to push new versions of the product to the customer, but they wanted developers to be as agile as possible in getting these features developed and tested. To fit this need, Blue Sentry designed and implemented a continuous integration of new code using Github, CircleCI, Atlas by Hashicorp, and Terraform Enterprise. Every feature pushed by a developer triggered an automatic “compile and test” of that microservice before merging with the appropriate code branch. Code branches were then merged and tested hierarchically, before the code base was applied to an Amazon Machine Instance (AMI) and registered as a promotable artifact. Artifacts were then automatically promoted to a staging environment, where product managers could make the final decision.
Once Blue Sentry integrated the solution with Slack, their chosen communication platform, product managers could then easily deploy from Slack and see the results of developers’ builds faster than ever, while not leaving that communications channel. This allowed the infrastructure engineers and developers to work together quickly to resolve failed builds, so that a bug never slowed the deployment by more than a few minutes. Turning these build results into organized data also allowed the their team to see trends, and to continuously iterate to improve the quality of the code delivery process.
“Not only is Upside’s solution highly scalable, but Blue Sentry’s complete CI/CD solution has also allowed DevOps to become more self-service to Upside’s product managers,” said Dresner.
Code projects can now go from merged to deployed in less than 15 minutes. And if a rollback is ever necessary, this can be accomplished very rapidly by a product manager, without the involvement of a single engineer
As they continue to grow, they will face new challenges. But a scalable platform, and a partner who can get the most out if it, makes challenges less daunting. “Blue Sentry has risen to every challenge we have thrown at them, and helped us win with excellence,” says Dresner. “We see a long future with Blue Sentry as we continue to grow our business,” she added