Your AWS re:Invent 2016 Keynote re:Cap
The two major keynotes during re:Invent 2016 announced new EC2 instance families and sizes, Organizations, AWS Aurora, Lex, Polly, Rekognition, Snowmobile, X-Ray, CodeBuild, Shield, Pinpoint, and more. Read this article for a quick update on major releases and what they could mean from an AWS cost efficiency perspective.
See recap highlights for a specific keynote:
We’ll discuss all of the interesting bits during our re:Invent re:Cap live discussion the week after the event.
December 1st Keynote Coverage
We captured the major keynote releases from December 1st that sparked immediate thoughts from an AWS cost efficiency and cloud cost management perspective. Check them out below.
During the second major keynote, AWS announced their application and microservices architecture analysis and debugging tool, X-Ray. This service will focus on helping users identify and troubleshoot root causes of performance issues and errors within applications. Through a process of tracing requests, recording traces, analyzing service maps and issues, X-Ray helps AWS users debug and improve app performance.
From a cost management perspective
Pricing begins with a “Perpetual Free Tier” where the first 100,000 traces recorded are free per month. It also includes the first 1,000,000 traces retrieved or scanned at no cost within the same monthly scope. Additional charges begin at $5 per million traces recorded per month. For retrieved and scanned traces, there’s a cost of $0.50 per million per month beyond the free tier.
We’ll be curious to investigate ways to parse and present X-Ray costs in alignment with specific applications within the organization. Keeping up with aligning the right metrics for new tools, like X-Ray, ensures AWS users can deliver the right cost and usage data to the right users, and strengthen optimization conversations (that X-Ray will ultimately produce) with a cost efficiency perspective.
To help centralize build environments for projects of all types and sizes, AWS released CodeBuild, their fully-managed build service. AWS users can now reduce the amount of work in setting up, scaling, and patching build servers by letting AWS handle it through a pay-as-you-use service.
Each managed build service is elastic, scalable, and easy to use. Users need only to identify the following elements to get started: Source Repository, Build Environment, IAM Role, Build Spec, and Compute Type. Each CodeBuild deployment is within an isolated, container-based environment.
As far as pricing goes, CodeBuild bills users by price per build minute based on three sizes of Compute Type:
- Build.general1.small (3GB Memory, 2 vCPUs, $0.005 per build minute)
- Build.general1.medium (7GB Memory, 4 vCPUs, $0.010 per build minute)
- Build.general1.large (15GB Memory, 8 vCPUs, $0.020 per build minute)
There’s a Free Tier that people can start with which includes 100 build minutes of the build.general1.small. Beyond that, users will be charged by their instance type per price per build minute. Check out the specifics on the AWS documentation, including pricing examples.
Watching for the additional costs
According to the AWS docs, additional charges include data transfer, log streams to CloudWatch, S3 for storage, and other services like Key Management and CodePipeline. We’ll be watching how our customers adopt CodeBuild and investigate ways that will best align build service costs with their additional costs and how that spending aligns with specific environments or projects involved.
DDoS protection on AWS is here: AWS Shield
AWS announced Shield, their managed service for protecting web applications against DDoS attacks. Coordinating between Elastic Load Balancing, CloudFront, and Route 53, Shield will protect apps and services from all types of modern attacks. It’s available in two flavors:
- Shield Standard: available at no extra cost, protects from 96% of most common attacks.
- Shield Advanced: provides additional protection against volumetric attacks, intelligent attack detection, and grants users access to the AWS DDoS Response Team (DRT) to protect against bill spikes.
Looking at Shield from a cost management perspective
While Shield Standard is free, Shield Advanced as a few pricing elements to consider. Subscriptions for Shield Advanced requires a 1-year commitment and a monthly fee of $3,000. Data transfer costs for Shield Standard are free, but the Advanced version will incur transfer costs across Elastic Load Balancing and CloudFront at a data transfer price per GB rate. See the official AWS documentation for more details and for pricing examples.
To assist app developers with in-app push messaging, AWS introduced Pinpoint. Targeted messaging can now be deployed throughout the app experience to help inform, educate, or retain users, depending on what kind of messaging the business wants to send. There is an analytics component to this service that will visualize and reveal data trends to users so they can optimize these campaigns. The AWS guest blog article about Pinpoint reveals more details.
Pinpointing costs of the service
Pinpoint uses a pay per month model that includes:
- Monthly Targeted Audience
- Events collected
- Notifications sent
There’s a free tier for each of these components per month. Beyond the free threshold, each component incurs additional charges per month. Users who adopt Pinpoint will want to keep an eye on how these additional push notification charges affect the overall spending, while wanting to find ways to align costs with specific apps and services within the business or enterprise.
See greater Pinpoint pricing detail on the AWS documentation.
AWS Step Functions: Visual workflows for distributed apps
AWS knows that building a complex distributed app, connecting to multiple web services and microservices while tending to a user base at-scale can get pretty crazy. Their solution is the introduction of Step Functions, a visual way to coordinate app components into a series of steps that can be tracked in a visual workflow.
AWS Step Functions now helps users build their apps and services with the tools they know and love, but they can now embark using a visualized path that helps streamline builds, but also identify where errors or bugs may reside. As users move from one step of an app to the next, they are charged a state transition from the start to the end of the workflow. Learn more about Step Functions from Jeff Barr’s thorough coverage, or by taking a closer look on the AWS documentation.
How Step Functions can affect the monthly bill
For the state transitions mentioned briefly above, AWS allows 4,000 state transitions for free per month. Beyond the free monthly tier, users will be charged $0.025 per 1,000 state transitions. Many apps and services are comprised of all types of build orders, components, and connected services. So there’s no surprise that using Step Functions can incur charges for the interaction and data transfer between them all.
According to AWS Step Functions pricing, users should expect Data Transfer, Lambda, and EC2 pricing (including related Reserved Instances or Spot instances if applicable) to affect Step Functions costs.
We’ll be observing how our customers use Step Functions and how to align this additional costs to existing organizations and their environments.
Other service additions to note
While we focused on announcements that provoke immediate thoughts on how users manage and optimize their AWS cost efficiency, we want to ensure folks check out other new features too. These include:
- AWS Blox: An open source collection of container management and orchestration on Amazon ECS.
November 30th Keynote Coverage
Here are the key announcements and releases from the first re:Invent 2016 general keynote during November 30th.
AWS introduced Organizations
AWS’s Organizations feature now allows administrators to arrange AWS accounts into Organizational Units. These could be grouped by teams, development environments, and other units that represent how your business wants to track progress, usage, and costs of its AWS spending.
Administrators can apply service control policies that make it much easier to grant or withhold access to AWS products and features to certain groups. This removes a lot of the previous administrative challenges of having to write custom scripts or wrangle these permissions at a granular level. Get more details from the official AWS blog announcement.
Coupling Organizations with a cloud cost management tool like Cloudability can help clearly allocate costs between multiple accounts and teams within the business or enterprise, leading to better conversations between teams on how to improve AWS efficiency.
New EC2 instances: I3, C5 T2, and R4
Responding to huge demands in in-memory analytics, multimedia processing, advanced machine learning, and scalable storage-intensive workloads, AWS CEO Andy Jassy announced a new EC2 family and instances.
F1: Access to Field-Programmable Gate Array (FGPA)
The new F1 instance family is still in a testing phase (with AWS currently running their developer preview), but the crowd was excited to be able to bring potential large-scale performance improvements to genomics, seismic analysis, big data search, and encryption algorithms.
R4: The next generation of memory-intensive instances with a business intelligence focus
The R4 family will have next-generation processors and much faster RAM than the R3 instances, including a larger L3 cache and overall higher memory speeds. There’s also a network advantage of the R4 instances, with the ability to support up to 20Gbps within a Placement Group and 12Gbps of throughput to EBS storage.
Expanding the T2 family
Previously, T2 users could only scale up to the t2.large. Jassy announced the addition of the t2.xlarge and t2.2xlarge to offer more burstable compute action in larger instance sizes.
Here are additions that are “in the works” by AWS:
Adding the new I3 family
The latest generation of IOPS-intensive instances will feature low-latency, Non Volatile Memory Express (NVMe)-based SSDs. AWS expects this configuration to provide up to 3.3 million random IOPS in 4 KB block sizes with up to 16 GBps of disk throughput. Users can put the I3 to the test in their I/O-intensive relational and NoSQL databases, transactional, and analytics environments.
C5: An upgrade in compute-intensive instances
The new C5 instances will focus on machine learning, multimedia, scientific, and financial operations featuring the latest Intel processors. These new instances will feature Intel’s “Skylake” processors.
Instead of having to provision G2s or P2s to provide GPU support, these new elastic GPUs can be added to existing instances to provide 1 to 8 GB of GPU and compute. See the official release from AWS on elastic GPUs and the other new instances.
New instances from an AWS cost perspective
We’re excited to hear more about new instance types and features that cater to evolving business and enterprise workloads. You can bet on our team working hard on ingesting the cost and usage data of all these new instances to make sure our customers can make the best optimization decisions possible to spend wisely on AWS. Also, expect more EC2 comparisons from the team to find out which are most cost effective!
AI-related additions by AWS
With all the love throughout the year for Alexa, it was no shock that AWS would release parts of the popular Amazon voice recognition kit to developers, including image recognition and text to speech as well.
The releases include:
- Amazon Rekognition: This is AWS’s deep learning-based image detection and recognition with the power to learn faces, scenes, objects, and more. There was a big nod to security functionality with this new service.
- Amazon Lex: AWS released Lex to let developers integrate the Automatic Speech Recognition/Natural Learning Understanding that powers Alexa, but into their own projects.
- Amazon Polly: AWS introduced Polly, the cloud service that converts text to lifelike speech in 47 voices and 24 languages. Polly learns and can translate shorthand and abbreviations into natural language. E.g., “It’s 47 degrees F in W A” becomes an MP3 file that says “It’s 47 degrees Fahrenheit in Washington.”
The Cloudability team will be observing future usage of Rekognition, Lex, and Polly to determine new ways to optimize and save on these services as they make their ways into real business use cases.
Athena is AWS’s serverless query engine that can quickly analyze large amounts out of AWS S3 using Standard SQL. This means running analytics on your data without the need of a data warehouse or multiple serves to interpret and analyze the data.
Athena can run interactive SQL queries on top of CSV, TSV, and Parquet files. It’s built on top of PrestoDB, the open source framework created by Facebook. And since Athena is comprised of S3 objects, it will take advantage of S3’s scalability, flexibility, and resilience. Check out Jeff Barr’s official coverage for more details on AWS Athena.
From a cloud cost management perspective, we’ll investigate how to best analyze, report, and visualize Athena queries in relation to their S3 storage costs.
Introducing AWS Lightsail
AWS Lightsail provides prepackaged Virtual Private Servers (VPS) options on AWS. Interested users can choose from a variety of plans and configurations that are priced by the hour. While the advantage here is to get started with a VPS quickly, there’s not a lot of room to optimize the VPS settings as they come pre-configured with a set amount of compute, memory, storage, and data transfer.
Get more details on Lightsail from the AWS blog.
Some businesses start out cloud native. Some have to migrate from older data center environments. While Snowball does its best, migrating terabytes to petabytes of data can take a long while. It’s time to meet Snowmobile.
AWS literally drove one of these ruggedized vehicles onto the keynote stage. Snowmobile can store up to 100 PB of data and features 1 Tbps (terabytes per second) across a network of 40 Gbps connections. AWS expects to migrate a 100 PB load in about ten (10) days. Get the Snowmobile lowdown from the AWS blog.
We support this out-of-the-box thinking to get more businesses onto the cloud. Know that we’ll be paying attention to how storage evolves and how to make sure people get the right insights to spend as efficiently on cloud storage as possible.
More to come
We covered the big releases with large cost implications on AWS environments from our perspective. There’s still one more keynote on December 1st and plenty of insights to come from the re:Invent 2016 sessions.
You can also join us for our live panel discussion and Q&A next week after the event.
Until then, stay tuned for more details from the next keynote.