Goodbye, iiNet.

I first found my way online around 1991, calling into BBSes in Australia (such as Dialix). When I arrived at the University of Western Australia as an undergraduate in 1994, the ISPs were starting to be born, and I subscribed to Omen Internet ISP, run by Mark Dignam and his brother

In the early years of market acquisition, Omen was consumed by iiNet, and thus I became an iiNet subscriber on ADSL. iiNet was itself founded and staffed by friends across the industry in Perth, and during this period there was plenty of innovation happening in the organisations across ADSL and DSLAMs, connectivity, routing, and speed. They were one of the first to offer naked ADSL – not requiring a telephone number subscription.

I pushed off to the UK in 2003, and upon my 2010 return, I subscribed again to iiNet. They (had been) Perth based and started by friends, and plenty of friends had worked there in engineering and other role.

As I have experimented with IPv6 tunnels back in 1999 from UWA, I looked for this in iiNet, and found only 6RD – tunnelled/encapsulated IPv6 offering from iiNet. The downside of this experiment was that packets were tunnelled from the customer site, to iiNet in Sydney. As I was in Perth, my IPv4 traffic would hit local cache endpoints, but IPv6 would traverse 50ms of Australia before getting the change to peer to cache services. It was… sub-optimal.

However, iiNet started an IPV6 blog, and showed a promise that technical engineering was continuing.

Sadly, that last saw a post in 2013, and since then, crickets.

But that was not all.

It seemed that every year, iiNet would silently introduce a new service plan. This plan would be nearly identical to the one customers were already on, but with slightly larger included data plan, or slightly faster, or slightly cheaper. In any sense, always better. Customers would have to notice this newer plan, and then request to move to it (later, this was self service in the iInet customer Toolbox). But it always took an action by the customer to ensure they continued to get value from iiNet. This meant that customers couldn’t trust iiNet to always be getting the best option. I recently discovered I was paying $10/month more than other customers, for the same speed internet access, and still unlimited downloads.

I don’t think this is overly customer focused. They are looking after their interest, rather than long term customer satisfaction and retention.

So coupled with the decline in engineering, and faced with an impressive price/performance offer from a competitor, I have finally churned away from iiNet.

I contracted Aussie Broadband on Monday 1st June at 9am. By 11am the same day I had a 1GB/sec internet connection with native IPv6 enabled (FTTP). I am in the process of porting over the home phone number I have had with iiNet for a decade.

Am I paying more? Yes.

Is it better? Yes.

I went from a 50/20 NBN unlimited plan, with a VoIP service with all Australian land-line and mobile calls included; it was AU$89/month, while iiNet’s new offering was $79. I ended up on 1000/50 NBN unlimited plan, with a VoIP services with all land-line and mobiles calls included for $169/month. 2x the price, 20x the speed.

Does that make it 10 times better? Hmm….

But more importantly, as they introduced this plan, Aussie indicated their existing customers on their legacy, more expensive yet slower plans would be migrated to this without them having to lift a finger. Proacticely better for their existing customers.

This breed customer trust.

So with two knock-out blows — innovation in engineering, and customer focus — I finally pulled the pin, giving up on the hope that the iiNet of old would engineer its way towards a modern ISP with a strong customer focus.

I have had a number of friends and colleagues move to Aussie Broadband in the last few months, and thus far I haven’t seen any one have any issues that haven’t been resolved quickly and capably. What held me back was my included phone number via iiNet, but that has now ported the phone number across to Aussie as well.

How did you get started in AWS?

Someone posed the question recently: how did you get started in using AWS?

Once upon a time…. I was working in London (2003-2010), and during my time at Vibrant Media running the IT operations team for their contextual advertising platform, I was looking for ways to serve content and process requests efficiently.

Vibrant had thousands of customers, and CommScore reporting indicated our advertising services were seen by some 49% of the US population each month (the platform was world-wide, but the CommScore report was for the US market). It was fairly busy.

In 2008 I stumbled across the then-launched AWS (starting 2006). At that time, the rudimentary controls were basic, and the architectural patterns for VPC at that time did not suit our requirements (all traffic from the VPC had to egress to the customer VPN – no IGW!). So I parked the idea, and moved on.

In 2010 I returned to Australia, and was approached by the team at Netshelter to implement a crawler for forum sites to identify the influencers in the network. Unlike my previous role at Vibrant, Netshelter had no data centres, no infrastructure, just AWS.

It was the words of Richard Brindley who said “we just have AWS, don’t worry about the bill, because anything you do in AWS is going to be vastly cheaper than what we would have done on premises”.

With only myself to architect, implement and operate the solution, I had to find ways to make myself scale. Platform as a service – managed components, was key. Any increase in pricing meant that I didn’t have to deal with the details of operations.

As a Linux developer and System Admin for the 15 years prior to that, I started with the EC2 platform. Finding images, launching them, and configuring them. Then came the automation of installation: scripting the deployment of packages as required for the code I was writing (back then, in Perl).

Pretty quickly, I realised I needed to scale horizontally to get through the work, and I would need some capability to distribute out work. I turned to SQS, and within a day had the epiphany that a reliable queue system was more important than a fleet of processing nodes. Individuals nodes could fail, but a good approach to queuing and message processing could overcome many obstacles.

In storing my results, I needed a database. I had been MySQL certified for years, writing stored procedures, creating schemas, and managing server updates. All of which was fascinating, but time consuming. RDS MySQL was the obvious choice to save me time.

As VPC capability evolved, additional layers of security became easier to implement without introducing Single Points of Failure (SPOFs), or pinch-points and bottlenecks.

From an Australian perspective, this was an interesting era: it was pre-Region in Australia. That meant that, at that time, most organisations dismissed cloud as not being applicable to them. True, some organisations addressing European and US markets were all-in, but latency and fears around the then-relevant Patriot Act kept usage low (this obviously changed in 2012!).

But in essence, the getting started advise of not worrying about the bill with respect to what the equivalent all-in-cost would have been for co-location fees, bandwidth commitments, compute and storage hardware, rack and stack time and costs, and th eoverhead of managing all these activities, meant that the immediacy and control of an AWs environment was far more effective.

I didn’t go wild on cost. Keeping an eye on the individual components mean the total charges remained sensible. As they say, look after the pennies, and the pounds look after themselves.

What was key was the approach to continuously learn. And then relearn something when it changes slightly, or unlearn past behaviours that no longer made sense.

It was also useful to always push the boundaries; reach out and ask service teams to add new capabilities, be they technical, compliance, policy, etc.

How would I start today…well, that’s another article for another day….

Writing (some of) the questions for the AWS Solution Architect Professional Certification

Writing the SA Professional questions in San Francisco.

bs the longest certified AWS individuals.

During my time with AWS, I also helped contribute to an early set of questions for the then-in-development Solution Architetc Professional certification. My contributions pulled upon my many years of involvement in Linux and Open Source, as well as my time then as AWS Security Solution architect for Australia and New Zealand.

As time (and I) moved on, I continued to sit more AWS certifications – at this time, I hold 8 AWS Certifications, and am awaiting results of the new Database Specialty certification. I’ve written many times about sitting these certifications, given guidance to friends and colleagues on sitting them. I’ve watched as the value to an individual of these certifications has increased, making them amongst some of the most respected, and best paid certifications in the technology field.

The attention to detail on running the certifications is high. The whole point of a certification is to discriminate fairly based on those who have the required capability to perform a task, and those who do not. If the certification were too easy, then it would undermine the value of the certification to those who are more adept in the topic.

Of course, the certification itself is not based on the same static set of questions. Some questions get invalidated over time as features get released and updated. Some services fall out of fashion, and new services are born that become critical (could you imagine running today without CloudTrail enabled).

The questions for these certifications are in a pool; and each time a candidate sits a certification, a subset of the currently active questions gets presented to them. The order of the questions is not fixed. The likelihood of two people getting the same questions, in the same order is extremely low.

However, over time, the pool runs low. Questions expire. New questions are needed.

Transamerica Building, San Francisco

In January 2020, I received a request to attend a question-writing workshop as a Subject Matter Expert (SME) for the Solution Architect Professional certification.

These workshops bring together some of the most capable, experienced AWS Cloud engineers on the planet. The goal is not to write questions that none of us could pass, but questions that all of us could pass that would bring more people into this tier.

Travel there

Arriving on Sunday, I managed to make it to my hotel, and then run to dinner with some dear friends and former colleagues from a decade ago who live in and around San Francisco.

Monday was a work day, so I was in the Modis office in San Francisco, talking to our team there about our cloud practice in Australia.

Corey Quin, @QuinnyPig, Cloud Economist, and James Bromberger having a coffee catch up in Unuion Square

I was also lucky enough to cross paths with Corey Quinn, whom I had met when he came to Perth for the Latency conference in 2018. A quick coffee, and we realised we knew a fair number of people in common, across AWS, and the UK and Australia.

Consul-General Nick Nichles speaks at the 111 Mimosa Gallery and the Austrade Cybersecurity event

As timing was still working well, there was an AISA and NAB sponsored trade delegation with the Australian Consular General hosting an event in town on Monday evening. Many people were in town for the popular RSA Conference, so I popped along.

Small world it is, running in to Andrew Woodward of ECU, and Graeme Speak of Bankvault, both from Perth. I was also recognised from my AISA presentations over the last few years…

The Bay Bridge

The exam workshop

14 Subject Matter Experts (SMEs) from around the world gathered in San Francisco for the Question writing workshop. The backgrounds were all varied, from end customers of massive national broadcasters, to finance workloads, government, and more.

Much time was spent trying to strike a fair balance of what should be passable, and trying to ensure the expression of the problems, and the answers, were as clear and unequivocal as possible.

The 2020-02-25 to 2020-02-27 SA Professional workshop team (minus Cassandra Hope)

Three days of this was mentally draining. But the team contributed and reviewed over 100 items. These items now go through review, and may eventually turn up in an exam that those aspiring to the professional lavel AWS certification will sit.

Ding ding! A cable car, the easiest way from Van Ness to Sansome Sts (along California, past the Top of the Mark, and more)

Thanks to the AWS team for organising and paying for my travel, and thanks to my team for letting me participate.


AWS Certified Database — Specialty

Today, Monday 25th of November 2019, is the dawn of a new AWS Certification, the “Certified Database — Specialty“, taking the current active AWS certifications to 12:

  • Cloud Practitioner — Foundational
  • Solution Architect — Associate
  • SysOps — Associate
  • Developer — Associate
  • Solution Architect — Professional
  • DevOps Engineer — Professional
  • Networking — Specialty
  • Security — Specialty
  • Big Data — Specialty
  • Alexa Skills Builder — Specialty
  • Machine Learning — Specialty
  • Database — Specialty

I saw my first AWS Certification, the Solution Architect Associate, back in January of 2013 with the initial cohort of AWS staff while in Seattle, and thus am the equal longest AWS-certified person in the world; to which I have continued doing many of these certifications.

I’ve been using databases – primarily open source databases such as MySQL and Postgres, since the mid 1990s. I was certified by MySQL AB back in 2005 in London. Indeed, in 2004 I wrote (and open sourced) an exhaustive MySQL replication check for Nagios, so I have some in-depth knowledge here.

So today, on this first day of the new certification, I went and sat it. Since this is a new beta, there are no immediate pass/fail scores made available — that will be some time in 2020, when enough people have sat this, and grading can be done to determine a fair passing score (as well as review the questions.

Services Covered

As always, three’s an NDA so I can’t go into detail about questions, but I can confirm some of the services covered:

  • RDS — of course — with Postgres, MySQL, Oracle and SQL Server
  • DynamoDB – regional and global tables
  • Aurora – both Postgres and MySQL interfaces
  • Elasticache Redis
  • DocumentDB
  • DMS
  • Glue

Sadly for Corey Quinn, no Route53 as a database-storage-engine, but DNS as a topic did come up. As did a fair amount of security, of course.

What was interesting was a constant focus on high availability, automated recovery, and minimal downtime when doing certain operations. This plays squarely into the Well-Architected Framework.

Who is this Certification for?

In my opinion, this certification is playing straight into the hands of the existing Database Administrator, who has perhaps long felt threatened by the automation that has replaced much of the undifferentiated heavy lifting of basic database operation (patching, replication and snapshots) with Managed RDS instances.

This gives the humble DBA of yore a pathway to regain legitimacy; for those that don’t will be left behind. It will probably spur many DBAs to undertake architectures and approaches they may have often felt were too hard, or too complicated, when indeed these are quite easy with managed services.

Conclusion

A good outing for a new certification, but the odd typo (the likes of which I produce) were seen (eg: cloud where it should have been could, if you can believe me).

For anyone with a Pro SA and Pro Dev Ops certification, this one shouldn’t be a stretch too hard. Of course, come March I may eat my words.

I know how much work goes into creating these question pools, reviewing the blueprints, questions, and the work yet to be done – grading and then confirming and rejecting. Well done Cert team on another one hitting customers hands!

Death of the Data Centre; long live the Data Centre

The last time I worked inside a data centre co-lo was in2009. From East Coast US to West, from the UK to Europe, and here in Australia I spent many long hours in these windowless hubs of electronic existence.

It’s been 10 years.

I started making a data centre at my fathers manufacturing organisation in the early 1990s. As a small business it had a number of computer systems, a small 100 MBit/sec LAN, and a room with air-conditioning that we sealed off and deployed physical dedicated servers and UPS units. I recall naming every host on the network with a name starting with the letter P:

  • Parrot
  • Pigeon
  • Pardaloot
  • Pootoo
  • Peacock
  • Pacific Black Duck

You get the idea. The company was called Pelican.

By the time I attended The University of Western Australia I was of course gravitating to the University Computer Club, a student association I would end up being Vice President and then President of. During my time there with friends, we furnished a small data centre our of recycled materials in order to contain the cooling for our server farm within the expanse of the vast Cameron Hall building; this structure still stands today (webcams).

In 1997 my interest in networking and digital rights led to help found The Western Australian Internet Association, now known as Internet.asn.au. Thus not being a network.

Despite not creating or working at an ISP in these earlier years of Internet, I was reasonably proficient in the IT physical infrastructure deployment. My professional career saw me spend 20 years within the data centres of banks, education, government, financial services. I used to order millions of dollars of server blade enclosures, remote control power distribution units and dual-power transfer units for reliability, switches, load balancers, remote KVM units; and upon notification of delivery at a data centre in Manhattan (111 8th Avenue, or 6th Ave), Seattle, China Basin in San Francisco, Andover MA, Amsterdam or more, I would organise for myself or my team to parachute in, un-box, unwrap, stack, and then crimp Ethernet leads, power on, and deploy clusters of servers, then kick off initial install server deployment, retreating home to then finish the software install remotely, and bring servers online and into service.

It was all about dependencies; have the right equipment in the right place at the right time to minimise the time spent in the co-lo.

The last one that I worked in was 2009. The last one that I visited was in 2013 – and that was one of the massive halls within the sprawling Amazon Web Services (AWS) US-East-1 complex; a facility that few people ever get to see (no photos).

All that effort, the logistics and physical work of installing equipment, is now largely redundant. I create virtual data centres on cloud providers from templates with more fault tolerance, scalability, and privacy in literally 5 minutes, across the planet without having to spend my time hidden for days (to weeks) crimping Ethernet cables, balancing redundant power usage, and architecting spanning tree powered reliable layer 2 networks.

While some write of the death of the data centre, I think the data centre has changed who its direct customers are. I’m not interested in touring facilities and planning cabinet layouts. I have better things to do. The hyper-scale cloud providers have automated and abstracted so much, that it is not cost effective for me to do any of that manual work any more.

Vivre la Data Centre. You don’t need to market to me any more. Just to those Cloud providers; cut your costs, you’re a commodity, and have been for a decade.