It's Starting To Look A Lot Like... Y2K

And look who's in charge of the country.

Do you remember Y2K, the threat that hung over humanity in the closing days of the twentieth century?

Horror scenarios of planes falling from the sky, electronic payments failing and ATMs refusing to dispense cash. As for your VCR following instructions and recording your favourite show - forget about it.

All because computer systems had been designed to only store two digits of year data. Just like old style cheques had “19” pre printed for the year. It was easy to convert ‘95’ into ‘1995’, but what would happen when the year was ‘00’?

Back in 1999 I was working for Epson, the printer people, in Hemel Hempstead as a computer programmer. A couple of years out from Uni I was responsible for updating their systems and making sure they’d still work once the New Year ticked over.

Image: GoogleMaps.

Around the world highly paid consultants were making extortionate amounts of money protecting companies from what might happen and future proofing their systems. Up in Hertfordshire there was me. Although to be fair the systems I was looking after were for Sales and Marketing and Customer Claims, nary a 747 insight.

Did I make big changes to ensure things would continue to work long into the future? Or kick the can down the road with a number eight wire type solution that would last a bit longer?

Well I was leaving the country before the end of the year, to return to NZ, and none of us there thought these systems were going to be around long term and by the time any of it became an issue we’d be long gone. I imagine there were probably similar conversations had about the time we got into such a problem in the first place.

So instead of making changes to the database which might break a bunch of other systems I made code changes. For example if a date for a business activity ended in a number higher than 50 you could assume it was the 20th century and add 1900 to the two digit number stored, if it was less than 50 then you’d add 2000.

It wasn’t perfect but it would keep things running for another 50 years and avoided support calls that might see someone ring up to complain their new printer wasn’t working only to be told that it wasn’t covered because they’d had it for 100 years.

Come New Year’s Eve while my ex-colleagues were offered vast enticements to be on call or onsite when midnight rolled around, on the biggest party night in a thousand years, I toasted in the new millennia with Fi on her mum’s balcony. Watching the fireworks, the lights from the Sky Tower, and looking out for falling aircraft.

In the event nothing really happened. Would bad things have occurred without the Y2K hysteria? It’s hard to say for sure.

Undoubtedly some took advantage of the hysteria, milking it for all they could. I’m glad I tested the systems I’d been working on, even if it was simply me changing the date on the Test Server and running a few transactions through. Better to spend that effort than find everything goes pear shaped, you haven’t checked it, and there is no way to roll things back and recover.

Sound familiar?

Global Chaos and Inconvenience

Long queues of passengers form at the check-in counters at Ninoy Aquino International Airport in Manila, Philippines on July 19. Ezra Acayan/Getty Images

In case you’re wondering why on earth I’ve chosen to talk about computer system failures, and you didn’t catch up with yesterday’s news - this happened:

A botched software upgrade from US cybersecurity firm CrowdStrike hit Microsoft’s Windows Operating System leaving many facing the infamous blue screen of death (BSOD), indicating a non-recoverable crash. Essentially the death of the computer but with the miracle of “have you tried turning it off and on again?”

(Image credit: Lea Rae / Shutterstock)

As you might imagine there are quite a few such machines around the place and the impact has been significant. Actually it’s been pretty much the worst case scenario that was sold to risk adverse IT Managers quarter of a century ago.

There has been chaos with flights grounded, trains not running, and TV channels like Sky News in the UK going off the air. All manner of systems used for every day transactions from making payments to booking medical treatments have been off line.

There’s quite a fun summary from Australia here:

What’s happening in Aotearoa?

As businesses and governments scrambled to respond to the outage, to understand what happened and to deal with the fallout a thought occurred to me…

Things may appear uncertain, with global system outages, but we could rest assured Acting Prime Minister - Arnold J Rimmer, was on the job.

David Seymour. Image: BeFunky.

I recall writing about the terrible prospect of a major event occurring with Christopher Luxon in charge, rather than Chippy or Jacinda. That seemed bad enough. I hadn’t considered that David Seymour, smeg head first class, would be running things. What could possibly go wrong?

Here in Aotearoa Auckland Transport, St Johns, and banking services were impacted. There were long queues at supermarkets, with checkouts down, and HOP cards weren’t working.

Kiwibank reported problems with online banking on Friday. WAIKATO TIMES

Kiwis took to Social Media, one saying - “The self-checkouts all started running into errors and resetting themselves too. By the time I got to the front of the line there were only two or three that remained working.”

“I went into Chemist Warehouse today and Woolies and both had at least one computer system with a BSOD problem, with the Chemist Warehouse one stuck in a reboot and BSOD loop,” another said.

Blue screens of death are visible on self-checkouts at a Woolworths store in Auckland. (1News)

With the sort of resolve that Aucklanders are loved for by the rest of the country one said, “I have cash and Chardonnay just in case Western civilisation falls over this evening.”

Seymour said he’d launched a “whole of government response”. Which I assume was intended to sound comforting, rather than simply indicate that there was little they could do to help.

The Acting Prime Minister continued, “The Government has worked quickly to understand the impacts of this issue and minimise them wherever possible.

We appreciate the inconvenience this is causing for the public, retailers and businesses. The Government is closely monitoring developments and will continue to provide updates.”

So I hope you’ve smoked plenty of kippers because I imagine “Ace” will have had a very busy night indeed.

Why did this happen?

In a statement with even less content than a Christopher Luxon pronouncement Microsoft said: “We’re aware of an issue affecting Windows devices due to an update from a third-party software platform. We anticipate a resolution is forthcoming.”

If I might translate… “It was not us, it was the other guys, but either way you’re poked. We hope at some point in the future things will be better but your guess is as good as ours.”

CrowdStrike, no I’d never heard of them either - but billions of people around the world sure will have now, confirmed that the outages were down to their software update. Which at least means the conspiracy theorists will be satisfied that this isn’t some sort of cyber-attack but is down to simple, good old-fashioned negligence.

Yes, there might have been a wee bit of sarcasm in that last bit.

The founder and chief executive of CrowdStrike, George Kurtz, said he was “deeply sorry for the impact that we’ve caused to customers”, adding there had been a “negative interaction” between the update and Microsoft’s operating system.

“A negative interaction”, what a lovely euphemism that is for “the largest IT outage in history.”

Here’s our old friend Matt Green, you might remember him from newsletters about the UK election, with a more straight up, and a lot more amusing, summary of things:

What happens next?

Well for starters CrowdStrike’s share price fell by 12%, to be honest I’m surprised it wasn’t more.

Their CEO said the timeline for fixing the issues “could be hours, could be a bit longer” and said many customers would need to manually reboot their machines.

Microsoft said that some customers had seen success by rebooting their PCs as many as 15 times. Which is rather taking the piss with the old “have you tried turning it off and on again?”

Seriously though, this has resulted in wide scale confusion and no doubt many millions in lost productivity and profits.

People have been unable to see GPs in the UK, the Mayor of Portland issued an emergency declaration, at the time of writing 5,078 flights have been cancelled worldwide, and Bloomberg reported impacts to banks and hedge funds saying “Some people had to go home”. The horror.

This is the event that was predicted at the end of the last century, with wide-scale outages and chaos. It’s worrying to consider that it wasn’t the result of any organised, ill intent, but simply a trusted provider making an update. It makes you wonder what could occur if someone actually set out to cause mayhem.

Still, at least we can rest assured that Arnold “Ace” Rimmer is on the job and keeping us safe. In these uncertain times who could ask for more than that?

---

Thanks for reading my kōrero, this one is public so you’re most welcome to share it. This writing is my full time job, and it’s entirely funded by paid subscriptions. So I’d really appreciate it if you’d consider a paid subscription for less than $2 per week.

The headline of this newsletter has certain connotations, and I know people always enjoy an early chance to recognise that Christmas is on it’s way. 🙂

Longer time readers might recall my two favourite Christmas songs, Tim Minchin with his White Wine in the Sun, and the Paul Kelly song about making gravy. Well there’s another one I really like, perhaps you do too.

Right I’m off to make Mrs Snowman a coffee - have a good one all you lovely people.