19 Dec ICYMI – AWS re:Invent Recap
The juggernaut that was Amazon Web Service's re:Invent conference took over Las Vegas at the end of November: about 43,000 attendees were spread across multiple venues along The Strip.
The conference, as expected, had its slew of announcements, starting with a firehose of information during CEO Andy Jassy's Tuesday keynote that I found somewhat overwhelming. AWS is offering so many services that even keeping up with what they are is becoming a challenge.
Announcement highlights among the 60 unleashed during the week included:
- AWS PrivateLink and Inter-Region VPC Peering
- GuardDuty for security via Machine Learning
- Fargate and Kubernetes support for containers
- Amazon Sumerian for VR and AR
- Lots of new Media Services
- Bare Metal Instances
- Updated Lambda deployment options as well as a Serverless Application Repository
- DynamoDB Global Tables and Aurora Serverless
- Cloud9 IDE
- Lots of new language processing with Comprehend, Translate and Transcribe
- DeepLens and SageMaker for machine learning.
With well over 100 services, AWS seems like a comprehensive IT service provider with a catalogue of services. This is certainly true, but if you look at little broader, you may find this misses the point of what AWS is actually building, which is a global cloud operating system. The separate offerings should not be seen as separate items, but rather as parts of a single system in the same way we think of what an operating system provides.
An operating system offers storage via block (AWS has EBS), file (AWS has EFS) or maybe object (AWS has S3), and networking via various means (VPC, ENI, etc.) There may be a built-in DNS server (Route 53), load balancing (ELB), monitoring (CloudWatch), logging (CloudTrail), messaging (SNS) and security (IAM, security groups); even containers (ECS) are a native part of a modern OS. On your OS you can also easily add application platform services such as a web server (S3, API Gateway) or a database (RDS, Aurora, DynamoDB).
You can start to see the bigger picture. AWS is building an OS, but this time it’s a distributed global OS, which is pretty much impossible to do yourself. One of the announcements was an update to an AWS service, DynamoDB, which now has Global Tables. DynamoDB is a key/value store, so it’s a “simple” table database; but now you can use it as a truly globally-distributed, highly-available service. Writes can be written to local copies in any region; you choose and the table self-replicates to wherever else you chose to. Could you conceivably do this yourself?
The “serverless” future was another unmistakable, overarching theme of re:Invent. Serverless is used as a term by AWS to mean not just Functions as a Service with Lambda, but also includes any service when you don't have to manage an OS in an EC2 instance or container.
The serverless ideal is stitching together a multitude of AWS services without ever having to manage any “traditional” infrastructure such as a cluster, OS or having to choose a size. These will be Lambda functions doing the compute functions accessed via Amazon API Gateway, accessing data pulled in from perhaps IoT devices via Kinesis, and then stored in S3, DynamoDB or Aurora. You don't have to manage any underlying instances or scale: AWS does this on your behalf, and you only pay when you use it, never paying for idle. In my opinion, the next decade of IT will be all about serverless.
To delve deeper into how much of this is achievable today, I attended a three-hour workshop: “Build a Multi-Region Serverless Application For Resilience and High Availability”. The premise was creating a globally-distributed and highly-available customer comment-handling website. The components included DynamoDB tables (pre-global tables, so some manual functions were needed to globally sync data), a globally distributed website and API behind a custom domain name with certificate and end-to-end monitoring. If the DynamoDB table in one site failed, the whole front end of the app would fail over automatically to another site in another region to access a working copy of the replicated DynamoDB table. No EC2 instances or containers were used in the workshop; all components were serverless. It showed the powerful simplicity of what's possible with a serverless approach.
Updated architectural thinking was highlighted in AWS CTO Werner Vogels' keynote. In case you are not aware, Werner is one of the world’s distributed systems experts. The keynote was an opportunity for AWS to tell us what IT architecture should look like in a modern-cloud-first world.
Interestingly there wasn't anything revolutionary or a brand new framework for us to digest, but rather building on what we've done before with the advantages cloud gives us. The modern architectural patterns he went through are to be able to build globally-distributed, always-on microservice-based applications with the following tenets: not guessing capacity needs, testing at production scale, extensive automation, evolutionary architecture and data-driven decision making, all improved via game days.
Werner then threw out a vision for the future by saying “The only code you will write in the future is business logic. Everything else will be managed services”. The idea is you will be using various serverless components tied together without having to worry about any underlying infrastructure, and only paying when your applications are being used.
re:Invent shows how AWS is impressive at its scale and pace of innovation; but there are adoption issues. Customers and partners I spoke to had the very real issue of trying to keep up with what AWS actually does. A DIY approach to stitching together the various AWS services is tough, and customers are wanting easier integration between the various services. AWS touts itself as a customer driven organization, so hopefully we'll see a simplification and better inter-service glue.
AWS may be fantastic at managing scale in the clouds, but down to earth at re:Invent, there were some logistical issues. They included sessions in rooms too small; there was no wriggle room to move elsewhere, which created long walk-up queues often full hours in advance. This, of course, caused some deserved grumbling; but other than this, the majority of the conference was extremely well organized. An army of friendly, smiley and helpful information helpers and queue attendants smoothed the crowd flow. For now, at least, the robots aren't taking everything over!
AWS re:Invent was very worthwhile attending, and provided an exhilarating glimpse into the future of IT.