Cloud – Page 8 – JEB's Blog

Evolution of Compute: Physical to Serverless

Unless you’ve been under a rock, you’ve seen the impact that Hyperscale Public Cloud has made on the IT industry. Its invention wasn’t to be a thing, but to be a continually evolving, improving thing.

And while many organisations will use SaaS platforms, those platforms themselves often run atop the IaaS and PaaS platforms of a hyperscale cloud platform.

One person’s SaaS is another person’s IaaS.
Me, James Bromberger

But its worth just checking on the evolution of IT service delivery at a low level, for not everyone who is in the IT industry has seen what that looks like at this time.

Evolution of service delivery from Physical servers to Serverless.

Change is hard. Humans are bad at it. I’ve seen many who evolved from column 1 to column 2, and have felt they are “done”. They aren’t on-board for the next wave of the evolution.

I suffer from this too. But three is a short cut that I can offer: try to jump from where you are now, to as far to the right as you can in one step.

Every one of these phases is a monumental shift in the way that services are delivered, requiring training, and experience. There is an overhead knowledge baggage that engineers take with them, trying to work out what functions the same as before, and what is different. This is taxing, stressful, and unpleasant.

So rather than repeat this process in sequence, over years for each change, my recommendation is to see how far to the right you can jump. Some limitations will crop up that prevent you from leap-frogging all the way to Serverless, but that’s OK. Other services will not be thus constrained.

Well Architected, meet Well Maintained

In 2012, the Well Architected concept was born inside AWS. It is a set of principles that helps lead to success in the Cloud; at that time, that was the AWS EC2 environment. It’s well worth a read if you have not seen it. At this time, its also been adopted my Microsoft for the Azure environment as well.

However, I want to move your attention from Architecture time, to operations time.

If you look at the traditional total life-cycle activities, there’s a lot of time and effort spent learning, adjusting, and implementing supporting technologies that are starting to become invisible in the Serverless world.

Lets look at the operational activities done in a physical environment, and compare that to Serverless. I’ll skip the middle phases of evolution as shown above:

Activity	Physical	Serverless
Physical security	Required	Managed
Physical installation	Required	Managed
Capacity Planning	Required	Managed
Network switching	Required	Managed
Hardware power planning	Required	Managed
Physical cooling	Required	Managed
Hardware procurement	Required	Managed
Hardware firmware updates	Required	Managed
OS installation	Required	Managed
OS patching	Required	Managed
OS upgrade	Required	Managed
OS licensing	Often Required	Managed
Runtime selection	Required	Required
Runtime minor patching	Required	Managed
Runtime major version upgrade	Required	Required
App server selection	Required	Managed
App server minor patching	Required	Managed
App server major version upgrade	Required	Managed
Code base maintenance	Required	Required
Code base 3rd party library updates (SDKs)	Required	Required
Network encryption protocol and cipher upgrades (TLS, etc)	Required	Required

As you can see, a large number of activities that should be done regularly to ensure operational excellence. However, I am yet to see a traditional physical environment, or virtualised on-prem environment that actively does all of the above well.

It’s an easy test: wander into any Java environment, and ask what version of the Java runtime is deployed in production. The typical response is “we updated to Java 8 two years ago“. What that means if “we haven’t touched the exact deployed version of Java for two years“.

Likewise, ask what version of Windows Server is deployed? Anything older than 2016 (even that, with 2019 has been out for nearly 2 years at this time is generous) shows a lack of agility and maintenance.

I challenge those in IT operations to think through the above table and check the last time their service updated each row – post project launch. If its a poor show, the change is your in “support mode”, and not “DevOps Operations”.

So what can be done to help do this maintenance?

Take it away. Stop it. While it can be argued to be important, and interesting, you’re possibly better off spending that effort on the smaller list that remains in a Serverless environment.

Evolution Continues

We can’t see where this evolution will go next. We do see that identity, authentication, authorisation, in-flight encryption, remain key elements to be aware of.

What comes next, I can’t predict. I know many ideas will be thrown about, new or recycled, and some will work, while others will wither and disappear again.

The only constant in life is change.
Heraclitus, b. 565 BC

AWS Certification trends (on LinkedIn)

I am always trying to find great talent; it’s part of being a Practice Lead in a large consulting organisation to find and develop talent. I work with a team recruiters who are constantly finding and screening people for the many roles we have.

I’ve been a big proponent of the AWS Certifications for a number of reasons; amongst which are value and confidence to the holder, value to the partner, value to the customer. I helped contribute questions to the AWS Solution Architect Professional certification in 2014 whilst passing through Herndon, Washington DC as an AWS employee, and again in February 2020 in San Francisco as an industry Subject Matter Expert, just before COVID-19 started closing down travel.

Today I took to LinkedIn, and did a search for the various AWS Certifications, and found a tally that looked interesting. These numbers are by no means authoritative, and could just be a reflection of the network of connections that I have.

AWS Certification	Tally	Launch Year	#/Year (to 2020)
Solution Architect Associate*	311,000	2013	44,428
Developer Associate*	189,000	2014	31,500
Cloud Practitioner*	103,000	2017	34,333
Solution Architect Professional*	94,000	2014	15,667
DevOps Engineer Professional*	57,000	2014	9,500
SysOps Associate*	29,000	2017	9,667
Security Specialty*	12,000	2018	6,000
Networking Specialty*	7,800	2018	3,900
Database Specialty*	7,200	2019	7,200
Data Analytics Specialty	6,300	2019	6,300
Big Data Specialty (retired/renamed to Data Analytics)	81,000	2014 – 2019	16,200 #
Machine Learning Specialty	5,300	2019	5,300
Alexa Skill Builder Specialty	546	2019	549

AWS Certifications as found on Linked In, 18/9/2020. * Denotes certifications I hold. # only calculated over the five years this was active.

With such a low number for the Alexa certification, I expect the source numbers is not be complete. Many people in certain industries (eg, intelligence services) will not put their profile online.

But regardless, lets review what we see…

The clear winner is the venerable Solution Architect Associate with the largest number per annum and largest number in total. Its seen as the initial certification in the technical certs, and is regularly reported as one of the most valuable in the industry with respect to salary expectations. Its also the longest cert I have held – being part of the very first cohort to pass this in January 2013.

While the Developer Associate certification is in second place by total number, it is just eclipsed by the number of people who have taken the Cloud Practitioner Foundational certification, on a yearly basis. The Cloud Prac is billed as an entry level, non-technical certification, so its appeal is to an even wider audience – the technical team can obtain it relatively easily, and the non-technical roles involved in total service delivery can achieve this as well.

At the Professional level, it seems the demand for certified Architects outweighs the DevOps Engineers almost 2:1; I suspect this is as a natural progression from that initial SA Associate.

The Data Analytics certification replaced the original Big Data cert last year; this gives us an insight into the change in demand. Over its active lifetime, Big Data drove 16,200 per year – its replacement sites at almost a third the prior demand. Perhaps the data analytics hype is stablising?

The total number of certifications reported above is 903,146; just shy of a million certifications in 7 years (and probably more given the validity of the data) excluding re-certifications (after 3 years, now).

Lets see what this looks like in a year from now. New AWS certifications will likely launch, continuing to help validate and differentiate experienced Cloud engineers.

Writing (some of) the questions for the AWS Solution Architect Professional Certification

Writing the SA Professional questions in San Francisco.

bs the longest certified AWS individuals.

During my time with AWS, I also helped contribute to an early set of questions for the then-in-development Solution Architetc Professional certification. My contributions pulled upon my many years of involvement in Linux and Open Source, as well as my time then as AWS Security Solution architect for Australia and New Zealand.

As time (and I) moved on, I continued to sit more AWS certifications – at this time, I hold 8 AWS Certifications, and am awaiting results of the new Database Specialty certification. I’ve written many times about sitting these certifications, given guidance to friends and colleagues on sitting them. I’ve watched as the value to an individual of these certifications has increased, making them amongst some of the most respected, and best paid certifications in the technology field.

The attention to detail on running the certifications is high. The whole point of a certification is to discriminate fairly based on those who have the required capability to perform a task, and those who do not. If the certification were too easy, then it would undermine the value of the certification to those who are more adept in the topic.

Of course, the certification itself is not based on the same static set of questions. Some questions get invalidated over time as features get released and updated. Some services fall out of fashion, and new services are born that become critical (could you imagine running today without CloudTrail enabled).

The questions for these certifications are in a pool; and each time a candidate sits a certification, a subset of the currently active questions gets presented to them. The order of the questions is not fixed. The likelihood of two people getting the same questions, in the same order is extremely low.

However, over time, the pool runs low. Questions expire. New questions are needed.

In January 2020, I received a request to attend a question-writing workshop as a Subject Matter Expert (SME) for the Solution Architect Professional certification.

These workshops bring together some of the most capable, experienced AWS Cloud engineers on the planet. The goal is not to write questions that none of us could pass, but questions that all of us could pass that would bring more people into this tier.

Travel there

Arriving on Sunday, I managed to make it to my hotel, and then run to dinner with some dear friends and former colleagues from a decade ago who live in and around San Francisco.

Monday was a work day, so I was in the Modis office in San Francisco, talking to our team there about our cloud practice in Australia.

Corey Quin, @QuinnyPig, Cloud Economist, and James Bromberger having a coffee catch up in Unuion Square

I was also lucky enough to cross paths with Corey Quinn, whom I had met when he came to Perth for the Latency conference in 2018. A quick coffee, and we realised we knew a fair number of people in common, across AWS, and the UK and Australia.

Consul-General *Nick* Nichles speaks at the 111 Mimosa Gallery and the Austrade Cybersecurity event

As timing was still working well, there was an AISA and NAB sponsored trade delegation with the Australian Consular General hosting an event in town on Monday evening. Many people were in town for the popular RSA Conference, so I popped along.

Small world it is, running in to Andrew Woodward of ECU, and Graeme Speak of Bankvault, both from Perth. I was also recognised from my AISA presentations over the last few years…

The exam workshop

14 Subject Matter Experts (SMEs) from around the world gathered in San Francisco for the Question writing workshop. The backgrounds were all varied, from end customers of massive national broadcasters, to finance workloads, government, and more.

Much time was spent trying to strike a fair balance of what should be passable, and trying to ensure the expression of the problems, and the answers, were as clear and unequivocal as possible.

The 2020-02-25 to 2020-02-27 SA Professional workshop team (minus Cassandra Hope)

Three days of this was mentally draining. But the team contributed and reviewed over 100 items. These items now go through review, and may eventually turn up in an exam that those aspiring to the professional lavel AWS certification will sit.

Ding ding! A cable car, the easiest way from Van Ness to Sansome Sts (along California, past the Top of the Mark, and more)

Thanks to the AWS team for organising and paying for my travel, and thanks to my team for letting me participate.

AWS Certified Database — Specialty

Today, Monday 25th of November 2019, is the dawn of a new AWS Certification, the “Certified Database — Specialty“, taking the current active AWS certifications to 12:

Cloud Practitioner — Foundational
Solution Architect — Associate
SysOps — Associate
Developer — Associate
Solution Architect — Professional
DevOps Engineer — Professional
Networking — Specialty
Security — Specialty
Big Data — Specialty
Alexa Skills Builder — Specialty
Machine Learning — Specialty
Database — Specialty

I saw my first AWS Certification, the Solution Architect Associate, back in January of 2013 with the initial cohort of AWS staff while in Seattle, and thus am the equal longest AWS-certified person in the world; to which I have continued doing many of these certifications.

I’ve been using databases – primarily open source databases such as MySQL and Postgres, since the mid 1990s. I was certified by MySQL AB back in 2005 in London. Indeed, in 2004 I wrote (and open sourced) an exhaustive MySQL replication check for Nagios, so I have some in-depth knowledge here.

So today, on this first day of the new certification, I went and sat it. Since this is a new beta, there are no immediate pass/fail scores made available — that will be some time in 2020, when enough people have sat this, and grading can be done to determine a fair passing score (as well as review the questions.

Services Covered

As always, three’s an NDA so I can’t go into detail about questions, but I can confirm some of the services covered:

RDS — of course — with Postgres, MySQL, Oracle and SQL Server
DynamoDB – regional and global tables
Aurora – both Postgres and MySQL interfaces
Elasticache Redis
DocumentDB
DMS
Glue

Sadly for Corey Quinn, no Route53 as a database-storage-engine, but DNS as a topic did come up. As did a fair amount of security, of course.

What was interesting was a constant focus on high availability, automated recovery, and minimal downtime when doing certain operations. This plays squarely into the Well-Architected Framework.

Who is this Certification for?

In my opinion, this certification is playing straight into the hands of the existing Database Administrator, who has perhaps long felt threatened by the automation that has replaced much of the undifferentiated heavy lifting of basic database operation (patching, replication and snapshots) with Managed RDS instances.

This gives the humble DBA of yore a pathway to regain legitimacy; for those that don’t will be left behind. It will probably spur many DBAs to undertake architectures and approaches they may have often felt were too hard, or too complicated, when indeed these are quite easy with managed services.

Conclusion

A good outing for a new certification, but the odd typo (the likes of which I produce) were seen (eg: cloud where it should have been could, if you can believe me).

For anyone with a Pro SA and Pro Dev Ops certification, this one shouldn’t be a stretch too hard. Of course, come March I may eat my words.

I know how much work goes into creating these question pools, reviewing the blueprints, questions, and the work yet to be done – grading and then confirming and rejecting. Well done Cert team on another one hitting customers hands!

AWS Partner Ambassador Meetup #1, Seattle, August 2019

The inaugural global meetup of the top partner engineers from around the world.

Another long overdue post from three weeks ago…

On the heel of the AWS Canberra Public Sector Summit 2019, and after some 24 hours at home with my family, I joined my fellow AWS Partner Ambassador at Modis – Steve Kinsman – and we started to wend our way across three flights to get to Seattle, departing a few minutes after midnight on Friday night/Saturday morning.