5 AWS Trends and Wishes for early 2019

AWS is the largest public Cloud provider in the world, and it is constantly evolving at a rapid clip, and using the scale of its service to reap the benefits from the economies that can be brought to bear at that scale.

The IT industry is itself evolving, with new patterns, protocols, and approaches being created in and out of the cloud. AWS is well placed to embrace many of these trends; things like WebSockets, IPv6, and more. But not everything is “done”in AWS; it’s all a continuous work-in-progress to stay current; but AWS’s approach (independent Service Teams, loose coupling, well-documented API interfaces)  and track record puts it far ahead of the competition in the race to stay current.

I’ve been using AWS for >10 years now, hold 8 AWS Certifications at this point in time, served nearly 3 years as the only Solution Architect with a “depth” in Security for Australia & New Zealand, have been a Cloud Warrior for 2 years, and now an AWS Ambassador. I’ve developed and delivered critical government solutions in Australia that the entire population depends upon every day, so have a reasonably deep understanding of the requirements that organisations have around their digital systems. With nearly 20 years as a Debian Linux developer, and >20 years delivering online services, my experience puts me in reasonable position to understand the ecosystem.

Here’s a list of things I foresee becoming commonplace in early 2019:

  • Organisation CloudTrail: enforcing company wide API logging standards, leading to better analysis of CloudTrail logs and the activity they expose
  • Enforced patterns around serving static content via S3: blocked public access by default, enabled only by CloudFront and Origin Access Identity to serve content stored in S3. side effect: appropriate TLS Certificates, and TLS Protocol and Cipher enforcement.
  • Virtual Private Cloud: enforced company-wide standards on routing: Transit Gateway from a corporate “production services”account”, once DirectConnect is supported by Transit Gateway
  • CloudFront and ALB set to HTTPS only (possibly with HTTP-> HTTPS redirect), with TLS 1.2 only!

5 Things I’d still like to see in AWS:

  • Improved health checks for Network and Application Load Balancers, similar to the existing ELB (Classic).
  • ECDSA certificates from Amazon Certificate Manager
  • TLS 1.3 on ALB, CloudFront, and the ability to restrict TLS Protocols to TLS 1.2+, or TLS 1.3+.
  • VPC: IPv6-only comms for intra-VPC services (RDS, ElastiCache, ALB/ELB, RedShift, etc.), IPv6-only subnets leading to IPv6-only VPCs, helped by service discounts for adopting IPv6-only
  • In Australia: AWS finally added to the ASD Protected Cloud list, without a Consumer Guide!

None of these are surprises to those who have extensively used AWS and hold those valuable AWS certifications.  These items don’t preclude your immediate extensive usage of the Cloud; they present visibility of the continuing evolution that is required in IT.

AWS Re:Invent: rest of the releases

Well, that was busy week. It was almost impossible to keep up with the announcements; an overwhelming feeling of something akin to playing Tetris as announcements poured down faster than I could read, understand and appreciate them.

So, having got past day 1, here’s the rest of what I think of what happened next:

DynamoDB Transactions and on-demand

ACID Compliance (atomicity, consistency, isolation, and durability) was always one of the constraints that those new to NoSQL were always trying to understand. For some workloads it was OK to move this validation to the user space (app server) for others, not so much.

On-demand DynamoDB removes the need to set sharding requirements, and let DynamoDB scale (and charge) as required from usage patterns.

CloudWatch Logs Insights

When I first saw this console, it just yelled “This is Sumo Logic” at me.


For many years, infrastructure being delivered to a Region was a pre-configured rack with all equipment ready to run. This release effectively shifts the delivery address from “AWS Region” to a customers data centre. It means there is a new channel for delivery of the equipment, and thus produces more scale, and ultimately, drives down cost further.

But who still wants to run data centres? The compliance, maintenance, physical security are all very compelling. Plus, an on-premise deployment has maintenance, and capacity limits that are way lower than the Region.

S3 Glacier Deep Archive

From Glacier with infrequent retrieval, to a deeper retention – Deep Archive requires data retention for half a year, and is a 12 hour restore. But the benefit is a huge price savings: US$1/TB/month (yes, Terabyte). That’s US$0.0009765/GB/month – so about time we changed units of measurement to the TB. Compare to Azure blob storage at US$0.002/GB/month (US$2.048/TB/month) , that’s less than half the cost .

When combined with some sensible data work-flows for backups, you’ll save a ton of money. But the biggest win will be when 3rd party backup solutions can instrument this themselves automatically. For example, the last 7 days of backup may sit on S3 Standard durability, and then get migrated to Glacier for 3 months, and then Deep Glacier after that.

Using the above tiering, lets do a 2 TB full backup once per week, and a 100 GB daily incremental. We’ll take 2.6 TB in S3 Std, then 11 weeks of S3 Glacier at 26 TB, and then 9 months of S3 Glacier Deep Archive for 93.6 TB. Sum total monthly cost is = 2.6 * 1024 * 0.025 + 26 * 1024 * 0.005 + 93.6 * 1 = 66.56 + 133.12 + 93.6 = US$293.28 / month = US$3,519.36/year assuming a one year retention.

If we had kept this all onS3 standard durability, then we would have been looking at US$37,539.84/year.

So, who’s going to make the first move? CommVault? Synology? StoreSimple? Storage Gateway VTL?

Managed Blockchain

Previously AWS had said it didn’t want to run a managed Blockchain service, saying no company should sit at the centre of this, but customer demand wins over this: and now two services filling the space: Blockchain as a Service, and the Quantum ledger database service.

Both of these are interesting to me, and I’ll be speaking with customers to see if they want us to integrate this into their solutions. Neither will replace using a relational database for temporal processing, state, etc. But for point in time authoritative signed data, they look interesting.


This one requires some testing. I’ve previously looked at Mechanical Turk for doing human-intelligence level OCR, but as a service this may be better. Any process that does text extraction should have a multi pronged approach to ensure accuracy; so perhaps a pass of Tectract, followed by a pass of Mech Turk (or other Humans), and then if there is a conflict/mismatch, flag for management inspection….

Security Hub

This is huge for me, and one I am actively getting my head around before recommending into customer environments. Its also enthused me to get back to AWS Config, which I’d previously discounted on cost.

Security Hub united several AWS security services. Each of these have had their own interface, cross-account capabilities, etc. Of course, for me, and my Public Sector customers, the lack of Macie in Australia is still a consideration here.

AWS Organisational CloudTrails

I’ve been a fan of CloudTrail since I first heard of it. The fact that it could always deliver API logs across-account – to a dedicated security account. without any fear or possibility of it being filtered or edited by the source account was a key enabler in enterprise workloads.

Its developed well since its initial launch, with multi-region support, digests files to detect tampering and more. But with all these options came the possibility of inconsistent deployments across a large fleet of accounts.

And while my perception has always been consistency, its only after circling back that you realise that not everything is consistent, with new AWS accounts being added at different times.

It is only with me starting to play with Config and Security Hub (see above) that these inconsistencies have come to light; and the new solution to this is just in time: Organisation Trails, that apply from the Billing/Organisation account, down to all dependent accounts.

An Organisation trial in a dependent account cannot be deleted or modified. They can log cross-account almost the same previous implementation – with the exception of  a few new Permissions required on the destination S3 Bucket policy.

Lambda Ruby, BYO Runtime, and Firecracker

Firecracker is a strong story, but in the end, having a manage environment for it is worth it if I can do so (ie, if latency, sovereignty, etc can be met). What will be interesting is the opportunity for more eyes to review it’s source code.

FSx (Luster & Windows Fileshare)

Managed file shares sound great, but now there’s confusion between EFS and FSx (and to some degree, Storage Gateway as an NFS and CIFS file share).

And much more

I wont go into detail on the large list of other services; my interest is the vast majority of web, security and DevOps-enabling services that continue to incrementally improve. But what happens next is interesting.

Config revisted

When first launched, I got bill shock form turning Config on with just a few rules. But now its much richer, and easier to understand. As it is one of the security tools feeds across into Security Hub, its forced me to circle back to Config and start re-evaluating some of its rules. Its come a long way, and much of the tooling I have written myself in the past to do cross-account checks, which Config also does, can now feed via Security Hub back to a central (organisatoin-wide) interface for alerting and actioning.


With some 50,000 people at re:Invent this year, the pace of innovation continues to put AWS far ahead of its competitors.