Category: Cloud

Recently it was that time again in April — the AWS Sydney Summit. This year the client event comprised two days, a Builders Day, and an Innovation Day. For Partners, the prequel is the Partner Summit.

---

The Partner Summit

Held on the Tuesday from noon onwards at the Hilton Hotel, this event was for the partner communicate to hear the positioning that AWS executives want the partner community to hear.

/

The event was around 400 people, up from 200 in 2023; a common theme we were to see is the return to almost pre-pandemic levels of attendance at this event.

The room seemed quite full, with many standing to the rear (and not just AWS staff standing at the rear!).

Julia Chen showed The regional cloud spend by consumers is, according to Gartners’ Q4 2023 research, still tipped to grow by just under 20%.

Meanwhile, global cloud services spend will hi 1.36 trillion by 2027. One other number that was tossed out was the percentage of workloads currently moved to the cloud: this individual pictured said “10%”, which in contrast to the 2023 messaging of 15% causes some confusion. Have workloads regressed? Or have more workloads been born not on the cloud?

---

AWS Client Summit: Builders Day

An early 7am start at the International Convention Centre at Darling Harbour.

Darling Harbour in the early morning.

The exhibitor hall floorplan.

What was obvious form the expo hall was the size was back to using the full area of the ICC; the keynotes were not using a partitioned side of the expo floor, and this time, there were more breakout sessions than in 2023 (more than double) – as shown by the purple boxes around the perimeter of the expo floor.

The expo floor had the usual mix of consulting services partners, training partners, and those ISV partners trying to sell tools to complement AWS services or fill the gaps.

It wouldn’t be 2024 without a disproportionate representation of Generative AI in the topics. These high compute, high storage solutions drive revenue, and hype.

Atlassian spoke of their wide usage of AWS, and mass deployment of micro services.

Large $9B invested by Amazon in the Australian region(s) thus far (10 years), and US$13B planned.

Amazon Bedrock, a service to abstract away the operational activities of running LLMs, is now operating and available from the Sydney Region.

The skills shortage in AI and Cloud seems to now be an issue as well, since the entire Gen Ai exploded in the last 12 months., Strap in, as much of this hype will drop off the way that there was no mention of blockchain this year (versus 5 years ago).

Long term employee Dave Brown soke about the various innovations they have done to accelerate the state of the art in Cloud computing.

Dave Brown also highlighted some of the global industries that are leveraging AWS Cloud.

Dave also touched on the community outside of Amazon that are related to the AWS ecosystems. No mention of the AWS Partner Ambassadors.

What was obvious now is that the builders day was aimed at more of the engineers; the presentation sin the keynote had more technical detail, and the audience was a tad under full capacity.

---

AWS Client Summit: Innovation Day

More mentions of Gen AI

For the Innovation Day, the keynote venue was packed to the rafters: 9,000 people.

Swimming Australia spoke of their performance tracking, analytics, and dashboard applications.

Lots of data of the ambitious approach on sustainability and reaching 100% of Amazon (not just AWS) 5 years earlier than planned, now estimated to happen in 2025, and already covering 90% of Amazon power usage today (April 2024).

A Project Kuiper ground station, with its first launches just having taken place, showed off their three models of receivers, and plans for service readiness.

Dr Carolynn Hogg spoke of storing genetic data in AWS, alongside former NASA CTO Tom Soderstrom (now in an AWS role).

---

Conclusions…

View of Sydney from the air on departure, April 2024.

So what’s the takeaways from the AWS SUmmit in Sydney for this year.

First, the amount of attendees is back to pre-pandemic levels. The interest is there. The partnership ecosystem has expanded; there’s a lot of organisations looking to help clients who don’t have the skills to do this safely.

I didn’t hear much mention of “shared security models”, or operational activities in the sessions I was in – I think the messaging has now bifurcated along the developer/non-developer line.

GenAI was mentioned in line with the current hype cycle. PoCs are popping up everywhere, and those that are successful are looking at the data lifecycle in GenAI.

Presented case studies from Swimming Australia, UNSW Centre for Genomic Studies, Woodside, 7West media, had previously all had some airtime in the last 12 months, so for those actively listening and processing these, there was nothing ground breaking.

All in all, its another year of evolution: rather than revolution. The steady hand of progress continues.

A few days ago, AWS announced Lambda support for their Code Build service.

Code build sits amongst a slew of Code* services, which developers can pick and chose from to get their jobs done. I’m going to concentrate on just three of them:

• Code Commit: a managed Git repository
• Code Build: a service to launch compute* to execute software compile/built/test services
• Code Pipeline: a CI/CD pipeline that helps orchestrate the pattern of build and release actions, across different environments

My common use case is for publishing (mostly) static web sites. Being a web developer since the early 1990s, I’m pretty comfortable with importing some web frameworks, writing some HTML and additional CSS, grabbing some images, and then publishing the content.

That content is typically deployed to an S3 Bucket, with CloudFront sitting in front of it, and Route53 doing its duty for DNS resolution… times two or three environments (dev, test, prod).

CodePipeline can be automatically kicked off when a developer pushes a commit to the repo. For several years I have used the native Code Pipeline service to deploy this artifact, but there’s always been a few niggles.

As a developer, I also like having some pre-commit hooks. I like to ensure my HTML is reasonable, that I haven’t put any credentials in my code, etc. For this I use the pre-commit hooks.

Here’s my “.pre-commit-config.yaml” file, that sits in the base of my content repo:

repos:
 -   repo: https://github.com/pre-commit/pre-commit-hooks
     rev: v4.4.0
     hooks:
     -   id: mixed-line-ending
     -   id: trailing-whitespace
     -   id: detect-aws-credentials
     -   id: detect-private-key
 -   repo: https://github.com/Lucas-C/pre-commit-hooks-nodejs
     rev: v1.1.2
     hooks:
     -   id: htmllint

There’s a few more “dot” files such as “.htmllintrc” that also get created and persisted to the repo, but here’s the catch: I want them there for the developers, but I want them purged when being published.

Using the original CodePipeline with the native S3 Deployer was simple, but didn’t give the opportunity to tidy up. That would require Code Build.

However, until this new announcement, using code build meant defining a whole EC2 instance (and VPC for it to live in) and waiting the 20 – 60 seconds for it to start before running your code. The time, and cost, wasn’t worth it in my opinion.

Until now, with Lambda.

I defined (and committed to the repo) a buildspec.yml file, and the commands in the build section show what I am tidying up:

version: 0.2
phases:
  build:
    commands:
      - rm -f buildspec.yml .htmllintrc .pre-commit-config.yaml package-lock.json package.json
      - rm -rf .git

Yes, the buildspec.yml file is one of the files I don’t want to publish.

Time to change the Pipeline order, and include a Build stage that created a new output artifact based upon that. The above buildspec.yml file then has an additional section at the end:

artifacts:
  files:
    - '**/*'

This the code Build job config, we define a new name for the output artifacts, int his case I called it “TidySource”.However there was an issue with the output artifact from this.

When CodeCommit triggers a build, it makes a single artifact available to the pipeline: the ZIP contents from the repo, in an S3 Bucket for the pipeline. The format of this object’s key (name) is:

s3://codepipeline-${REGION}-{$ID}/${PIPELINENAME}/SourceArti/${BUILDID}

The original S3 Deployer in CodePipeline understood that, and gave you the option to decompress the object (zip file) when it put it in the configured destination bucket.

CodeBuild supports multiple artifacts, and its format for the output object defined from the buildspec is:

s3://codepipeline-${REGION}-{$ID}/${PIPELINENAME}/SourceArti/${BUILDID}/${CODEBUILDID}

As such, S3 Deployer would then look for the object that should match the first syntax, and fails.

Hmmm.

OK, I had one more niggle about the s3 deployer: it doesn’t tidy up. If you delete something from your repo, the S3 deploy does not delete from the deployment location – it just unpacks over the top, leaving previously deployed files in place.

So my last change was to ditch both the output artifact from code build, and the original S3 deployer itself, and use the trusty aws s3 sync command, and a few variables in the code pipeline:

- aws s3 --region $AWS_REGION sync . s3://${S3DeployBucket}/${S3DeployPrefix}/ --delete

So the pipeline now looks like:

You can view the resulting web site at https://cv.jamesbromberger.com/. If you read the footnotes here it will tell you about some of the pipeline. Now I have a new place to play in – automating some of the framework management via NPM during the build phase, and running a few sed commands to update resulting paths in HTML content.

But my big wins are:

1. You can’t hit https://cv.jamesbromberger.com/.htmllintrc any more (not that there was anything secure in there, but I like to be… tidy)
2. Older versions of frameworks (bootstrap) are no longer lying around in /assets/bootstrap-${version}/.
3. Its not costing me more timer or money when doing this tidy up, thanks to Lambda.