Look ma, I can run containers on Amazon ECS Anywhere — On-premises or on the Cloud !!
TL DR: Amazon ECS Anywhere is a feature of Amazon ECS that enables you to easily run and manage container workloads on customer-managed infrastructure and is now Generally available (GA) — https://aws.amazon.com/about-aws/whats-new/2021/05/amazon-elastic-container-service-anywhere-is-now-generally-available/ !! This is now available across 23 AWS Regions (and growing)where Amazon ECS is available, including AWS India Mumbai region.
Key resources:
- Documentation : https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-anywhere.html
- Make sure you look at the key considerations for Amazon ECS Anywhere: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-anywhere.html#ecs-anywhere-considerations (like supported operating systems, system architectures etc.)
- AWS Blogs : https://aws.amazon.com/blogs/aws/getting-started-with-amazon-ecs-anywhere-now-generally-available/ and https://aws.amazon.com/blogs/containers/building-an-amazon-ecs-anywhere-home-lab-with-amazon-vpc-network-connectivity/
- AWS samples : https://github.com/aws-containers/ecs-anywhere-tutorial (from Massimo Re Ferrè, which has a more realistic use-case for using ECS Anywhere including leveraging AWS managed services)
- Pricing : https://aws.amazon.com/ecs/anywhere/pricing/
- FAQ’s : https://aws.amazon.com/ecs/anywhere/faqs/
We recently had an online event centered around Amazon ECS. You can watch the session on Amazon ECS anywhere, where the key features, demos and service roadmap were explained in-detail:
Demo Architecture:
I needed a quick way to setup an demo of Amazon ECS Anywhere, what better than to run this on my laptop using an Ubuntu image with Vagrant and VirtualBox and connectivity back to the Amazon ECS control Plane via the internet. Obviously, in a real-world scenario, you would want to use a more secure (use VPN), reliable connection (like AWS Direct Connect, redundant VPN etc.) from your data center to AWS.
Note: Nathan Peck in his awesome blog using Raspberry PI’s has used an AWS Site to site VPN for connectivity between his external compute resources and Amazon ECS.
Steps
Setting up External servers
- A variety of OS and system architectures are supported for external servers, please see https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-anywhere.html#ecs-anywhere-considerations for a complete list.
- I used a cheap way of running an Ubuntu VM using Vagrant and VirtualBox on my Mac !!
brew install virtualbox
brew install vagrantNote: I had some issues while starting the VM due to the Virtual Box getting blocked by MacOS in loading some kernel modules. You need to go to System Preferences -> Security & Privacy and allow software from the vendor to load.
Using the AWS Console or CLI
- Create the IAM roles required as mentioned at https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-anywhere-iam.html
- Create an ECS cluster
aws ecs create-cluster --cluster-name $CLUSTER_NAME- Create an ECS anywhere activation key, make a note of the ActivationID and ActivationCode
aws ssm create-activation --iam-role ECSAnywhereRole | tee ssm-activation.json- Login to the Ubuntu VM, and run the ECS anywhere installation script (make sure the environment variables are set correctly)
vagrant upvagrant ssh curl --proto "https" -o "/tmp/ecs-anywhere-install.sh" "https://amazon-ecs-agent.s3.amazonaws.com/ecs-anywhere-install-latest.sh"sudo bash /tmp/ecs-anywhere-install.sh --region $REGION --cluster $CLUSTER_NAME --activation-id $ACTIVATION_ID --activation-code $ACTIVATION_CODE
At this point, unless there are network connectivity issues, the Ubuntu server should show up as an External server in the Amazon ECS console
You can get more details on the AWS System Manager about the server
Deploying a sample service on the External server
- I used a sample image from nginx called nginxdemos:latest to test the deployment
- Follow the steps mentioned to create a Task Definition as per https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-anywhere-runtask.html Please note, you will have to use the
requiresCompatibilitiesparameter and specifyEXTERNAL
Please note at this point of time only bridge, host, or none network modes are supported awsvpc network mode isn't supported
- Using the AWS CLI or the console, register the Task definition and run a ECS service or a Task on the newly formed ECS cluster
- Once the task or service is created, you can ssh into the Ubuntu vagrant box and you will notice both the nginx image as well as the ecs agent image running as containers inside Ubuntu.
We can also access the nginxdemo web page on our local laptop, if we have forwarded the port from Vagrant to the localhost in Vagrantfile.
- From within the ECS console
Finally, we can also stream the logs back to AWS by using Amazon Cloudwatch (https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-anywhere-iam.html#ecs-anywhere-iam-conditional) or using Fluentbit as per the instruction at https://docs.aws.amazon.com/AmazonECS/latest/developerguide/firelens-example-taskdefs.html
That concludes this short demo …
Conclusion:
With the GA of Amazon ECS Anywhere, the next frontier of leveraging existing compute resources in your data center to run managed containers using an orchestrator like Amazon ECS is now possible.
I feel, workloads that require compute to be in the data centers due to Compliance reasons or in scenarios which need close integration with backends (maybe something like a payment switch) or databases, will be great use-cases to leverage Amazon ECS Anywhere. Of course, you can also run ECS anywhere in raspberry PI or other kinds of compute devices too !!
