I have a long standing dream of learning Drums 😁 I already have made two aborted attempts by enrolling in drums classes in Bangalore, but have not yet gone beyond the basics. One instructor shared an insight about Drums, that a Drum is not a musical instrument, and it needs to work with other instruments to create music. In fact, a single drum is not sufficient, you need a Drum kit or multiple drums, for the sound to be effective and to make sense ..
Similarly, when we talk about Container Orchestration, you need multiple services to work together like VM’s, Load Balancer, Storage and other services, along with a Container Orchestrator, in a manner somewhat similar to my Drum analogy, where a Drum needs other instruments to make music !!
As a person who has been in customer facing roles for the vast majority of my career, we tend to get a lot of requests from customers who would like to get an estimate of what it would cost to run their workload on the AWS platform. In the good old days (before AWS), while I would give them some general guidance, I would pass their requests to the sales folks, who would compute the approximate cost, with some discount added etc, and share it over an official email. They would consult me on what services to add, and other technical inputs, but the ownership of pricing was still with the sales guys/gals ..
Well, with AWS, all the pricing information is public for all services, and there is also a AWS pricing calculator, where anyone can get an estimate by adding various services to create a bill of materials (BOM). There is even an API to get pricing for AWS services !! Public cloud has indeed democratized pricing and empowered IT folks 😀
This slightly long personal blog is specifically focused on getting started with pricing with AWS Container services — Amazon ECS, Amazon EKS , AWS Fargate and Red Hat OpenShift Service on AWS (ROSA) at a high level. I have not added details of pricing with other services which are part of the AWS container services portfolio like AWS App Runner.
Note: PRICING is a HUGE TOPIC !! As always, pricing is subject to your own workload and you should factor in the requirements of your specific workload like the number of servers, required services etc. Also, the supported features keep changing, and hence the official AWS documentation, the AWS pricing calculator are the final authority. This blog is to give a rough idea about how to approach pricing .. As always, your mileage will vary.
Also, if you are a customer, partner or a potential customer of AWS, do reach out to the AWS In-country account teams, they can bring in Solution Architects, Container specialists for a more deep-dive and comprehensive engagement.
AWS Container Services
As per https://aws.amazon.com/containers/services/ , AWS has three managed container orchestration services:
- Amazon ECS — fully managed container orchestration service from AWS
- Amazon EKS — full managed Kubernetes service
- Red Hat OpenShift Service on AWS (ROSA) — managed Open Shift on AWS
The Control Plane is responsible for orchestration of your containers, while the Data Plane consists of the worker nodes, where the containers actually run.
We have two options for the Data Plane or compute, with AWS Container Services:
- Amazon EC2 — Run containers on virtual machines. Supported by ECS, EKS and ROSA
- AWS Fargate — a serverless compute engine for containers. Supported by ECS and EKS, not yet supported in ROSA.
There are also hybrid options like ECS Anywhere, EKS Anywhere as well as AWS Outposts, which I am not covering in this blog.
I am personally biased, and tend to follow a “Fargate First” approach, unless the requirement proves otherwise !!
In addition to the above, you will most likely need Amazon Elastic Container Registry (ECR), a fully-managed container registry, as well as Amazon CloudWatch, Observability services like AWS Container Insights, AMP, AMG and a whole bunch of the other AWS services.
Lets get started with pricing for Amazon ECS and Amazon EKS:
The first thing, always refer to the official pricing pages at https://aws.amazon.com/ecs/pricing/ and https://aws.amazon.com/eks/pricing/ and the AWS service documentation.
Key things to consider:
- Control Plane — ECS Control Plane is free, while for EKS there is $73 per cluster per month.
Note: Each EKS cluster can handle a large number of nodes (and pods) in the data plane, please refer to https://docs.aws.amazon.com/eks/latest/userguide/service-quotas.html and check with your AWS account team for more information on limits and the maximum nodes/pods per EKS cluster, which also depends on the EC2 instance type used in the Data plane. The question of using a single cluster or multiple clusters is also based on many factors ..
- Data Plane — You only pay for the AWS resources you consume in the Data plane, as well as other AWS services that you use.
- EC2 — Supports Linux, Windows
- Fargate — Supports Linux, Windows
A few things to consider:
Operating system:
You should consider the OS of the EC2 and Fargate nodes based on your specific requirement.
- Linux — ECS and EKS support Linux, including Amazon Linux, Bottlerocket and other flavors of Linux. Please refer to the documentation for more details
- Windows — Both Fargate and EC2 support Windows. Again, please refer to the docs for the exact Windows versions and other gotchas
Processor Architecture:
Now, you have a choice of both x86 based servers as well as Graviton (ARM) based servers. Do remember, that while Graviton based servers provide much better price performance, you may need to check if your application and its libraries have been tested on Graviton.
Also, if your workload requires GPU like for ML workloads, you will have to select the EC2 instances which have these specific processors.
Leveraging Savings Plan
Leverage AWS Savings plan to get huge savings:
Look at the following savings when switching between using On-demand pricing and with 1 year and 3 year EC2 savings plan with No-upfront (NURI):
Why would you want to even think of On-demand pricing, when you are planning to run workloads continuously like in production ?
Using EC2 spot instance
Amazon EC2 Spot Instances are spare Amazon EC2 capacity that offers steep discounts off of On-Demand prices. Amazon EC2 Spot Instances can be interrupted with a two-minute interruption notice when EC2 needs the capacity back. EC2 Spot instances with Containers is a match made in heaven and should be leveraged.
Please see the specific documentation for ECS and EKS for more details on using EC2 Spot.
Using Fargate instances:
Fargate has the following pricing levers, please refer to https://aws.amazon.com/fargate/pricing/ for the official details:
- vCPU and memory allocated. These come in preset sizes, make sure you select the right setting. This the power of Fargate, in allocating the right amount of CPU/Memory resources to Fargate. Do not over provision.
- OS — Supports both Linux and Windows
- Processor — supports x86 and ARM
- Storage — How much ephemeral storage is needed for your Fargate instance?
- Spot — Fargate supports Spot only with ECS, as of now.
- Savings Plan — Similar to EC2, it also supports 1 year and 3 year savings plan . Please note that the AWS pricing calculator does not yet support showing Fargate pricing for Fargate Spot and Compute Savings Plans, hence these have to be calculated manually.
From https://aws.amazon.com/savingsplans/compute-pricing/, you can get a savings between 20% to 45% for 1 year and 3 year NURI for Fargate.
Sample pricing links:
A sample pricing for an ECS cluster with two EC2 based worker nodes (t3.large -2 vCPU, 8 GiB memory, 30 GB EBS volume) and some additional services like an ALB, NAT gateway, ECR is shown below. The sample pricing link can be accessed at https://calculator.aws/#/estimate?id=bf71454b65166882192d441acfc2143480c8ee8a
For an EKS cluster, the only additional line item that needs to be added is the Control plane. The sample link can be accessed at https://calculator.aws/#/estimate?id=23a371eb981d394cb7c96a6bee919477bb962938
For Fargate, the pricing calculator currently supports pricing for On Demand only, and you will have to use pricing available at https://aws.amazon.com/fargate/pricing/ to manually calculate pricing with Savings Plan or Spot.
The On Demand pricing link for the above single Fargate instance is available at https://calculator.aws/#/estimate?id=bcd48a7ebe08a841a2db01a7d92cec74dc9e0790
As mentioned before, you can get a savings between 20% to 45% for 1 year and 3 year NURI for Fargate, with Fargate savings plan.
Lets get started with pricing with ROSA :
The official pricing link for ROSA is available at https://aws.amazon.com/rosa/pricing/. The documentation at https://docs.openshift.com/rosa/rosa_install_access_delete_clusters/rosa_getting_started_iam/rosa-aws-prereqs.html has great details on the AWS prerequisites for ROSA.
There are a few important aspects with ROSA, as per my limited understanding:
- Pricing has a ROSA service fee which includes a component for worker nodes and a cluster fee. This service fee has on-demand, 1 year and 3 year options also available as per https://aws.amazon.com/rosa/pricing/
- AWS infrastructure costs covers the resources provisioned for the ROSA cluster which includes EC2, EBS, NAT gateways, Private Link, Load Balancers (Classic and Network) etc. EC2 can be either utilize On-Demand, 1 year and 3 year savings plan for pricing.
- ROSA does not support EC2 Spot and Fargate yet.
- Billing is consolidated for a ROSA cluster and is part of your overall AWS billing, which simplifies things
The AWS ROSA pricing page has two great examples which illustrate the pricing for an on-demand and 1 year scenarios. To get a more detailed estimate which also includes other components like the Load Balancers, NAT gateways and others, please refer to the architecture for ROSA at https://openshift4-on-aws.awsworkshop.io/4-rosa-track/3-rosa-architecture/1_rosa-arch.html
I have created a simple spreadsheet, by plugging in all these numbers to get an estimate for a public facing ROSA cluster, and you can do the same too or check with your AWS account team or AWS container specialists.
Note: This is for a specific number of servers in Master, Worker and Infra nodes
A sample AWS pricing link for the infra piece for the above sample scenario is available at https://calculator.aws/#/estimate?id=199cad49ccf7c070cc9577a130714640cfed6c24
Have a look at this great workshop at https://openshift4-on-aws.awsworkshop.io/4-rosa-track.html to get started on ROSA.
Other (important) stuff
Remember, that these are estimates and your actual billing will depend on your workload, traffic and lots of other factors. A few tips:
- Ensure that you tag resources — https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-using-tags.html and https://docs.aws.amazon.com/eks/latest/userguide/eks-using-tags.html This AWS blog also has tips around Cost optimisation — https://aws.amazon.com/blogs/containers/cost-optimization-checklist-for-ecs-fargate/
- For EKS, Kubecost is now integrated with Amazon EKS, which can help with monitoring costs of your EKS cluster at a more granular level — https://aws.amazon.com/blogs/containers/aws-and-kubecost-collaborate-to-deliver-cost-monitoring-for-eks-customers/
- Leverage monitoring and observability into your container infrastructure like Cloud Watch container insights, Amazon Managed Prometheus and Amazon Managed Grafana.
- Leverage the AWS well architected principles for Cost Optimisation — https://www.wellarchitectedlabs.com/cost/
- If you are part of other Enterprise Discount programs, private pricing agreements, have AWS credits etc, they will also affect your overall pricing
- Monitor your actual usage, when you run your workload in test or production.
- Always, look for ways to optimize, optimize, optimize costs regularly !!
Finally, as always, I am open to feedback and corrections. This is a vast topic, and there can/will be different interpretations.
Please use the AWS official documentation and pricing tools, to get the latest information. But, my sincere hops is that this blog will help you get started with Pricing on AWS Container services.
Best wishes on your journey with AWS Container Services 🙏
