100 Days of Lockdown

Lockdown started 100 days ago, when my children were sent home from school. Since then we’ve lived within these 4 walls, but it looks like the new normal will start on 4th July. Time to reflect. blog.mindrocketnow.com

Over three months on, and I’m still trying to come to terms with how the world has changed. Some things I thought I knew turned out to be different, and I realised that many things that I’d taken for granted were great privileges. I did also get some things right. Let me know in the comments if any of these ring true for you.

Which predictions I go right

Family time

We have become closer as a family. Even though we’re within these four walls, we have that healthy combination of personal space that we enjoy being alone in, and companionship whenever we want it. Family was always important; now it’s central, and we’re all thriving as a result.

Indulging

The additional time gave me opportunity to indulge my hobbies of playing music, gardening, reading comics, geek cinema. The additional time also gave me the opportunity for physical and mental self-improvement. It feels like an indulgence to be able to spend the time to go for a run or meditate or learn a new drum fill or just sit still and read, without worrying about what I should be doing instead.

Job hunting sucks

I’ve made a career of doing interim roles for the past decade, and I’ve enjoyed the pattern of changing contexts every couple of years. However, the job market slowed to a stop, at least for my niche. The hot opportunities I was pursuing before lockdown became cold. One might return, but the others have just evaporated into silence. And new ones to replace them are few and far between. Job hunting is a test of resilience at the best of times. This isn’t the best of times. This is going to suck even more than normal.

Financial impact

It took a long while for our financial health to recover from the fallout of the banking collapse, because we were fortunate in that our careers remained relevant in all that time. The economic collapse after the pandemic crisis always looked more serious to me, as it was forcing re-factoring of pretty much all aspects of our economy. Rather like bee hive colony collapse, where worker bees are forced to leave hives that look functional, but are actually sick. I’m worried that like bees fleeing a sick hive, we’ll have to flee our careers in the sick part of the economy, to find a healthier part. Do we have enough years in us to re-factor and still retire with all our debts wiped clean? Or will we be bequeathing our debts to our children? If we’re lucky enough to remain in our current careers, then probably yes. If we have to find a new hive, then possibly no.

What surprised me

New skills

I’ve enjoyed learning new skills. I really enjoy cooking, making use of knife skills, combining new flavours, but most of all, seeing my family eat the product of my labours. I enjoy learning Python, thinking within the structure that coding mandates, and can now create web sites, data visualisations, financial analyses. These might even be useful new skills.

Life slowing down

My to do list didn’t disappear. Every day, I look at it and wonder why I haven’t been able to just burn through it. But each day brought a new couple of things to get done, and my capacity to motor through the tasks diminished. My productivity slowed along with the pace of life. I don’t miss the pace, and I find the undiminished to do list an acceptable price.

Smaller world

Video conferencing has brought distant friendships closer. Some of my friendships have drifted due to lack of opportunity to nurture; people are abroad in different time zones, people are busy with their own lives, people had their quota of social interactions at (=quick drink after) work. When that stopped, we turned to video conferences, which turns out to be better. It’s more inclusive as you don’t have to be within physical reach, and it’s more convenient as you don’t have to worry about the last train home. However, it’s no replacement for a large-scale shared experience; I’ve been to a beer fest, a cinema festival and a football match and they were all insipid. However at the personal-scale, it really works.

Nature is uplifting

Sunshine spills into my study. Bird song and the susurration of wind is louder than traffic noise. The rambling rose over the arch is in full bloom of 80s pink. Bees are constantly examining the lavender. I can see a whole world through my study window. I had no idea that these simple quotidien events would have such a profoundly positive effect on my wellbeing.

The new normal

The UK government announced that further easing of lockdown measures will come into force from 4th July. This feels like the moment the country comes out of hibernation. We’re starting to plan again, to think about holidays, to encourage social growth. It feels good to feel positive again.

So what is DevOps anyway?

Those of you in software development probably already know the answer to this. I thought I did too. Then I found out I was wrong. [blog.mindrocketnow.com]

First the academic answer: according to https://aws.amazon.com/devops/what-is-devops/ 

DevOps is the combination of cultural philosophies, practices, and tools that increases an organisation’s ability to deliver applications and services at high velocity

So far so vague. Deliveroo also has a high velocity delivery service. Perhaps a better description of DevOps behaviours is from https://tnw.to/gT0J2  

DevOps aims to close the gap between what the client ordered and what the development team has delivered. There is an emphasis on short release cycles, iterative approach to design, and automation of repetitive steps. 

Which sounds a lot like Agile, doesn’t it? Like many, I thought that Agile was a way of aligning Product and Engineering together into one happily productive team, and DevOps extended this to include Validation and Operations. The promise of DevOps is by having one bigger happy team of Product Owners, Developers, Testers and Infrastructure Operations Engineers, we can get to safely deploying every minute, instantly measuring how much delight we’re giving our customers in real-time.

I have yet to see this harmony in my experience. Instead, I’ve seen that Agile Transformation turns out to be a power struggle between Product and Engineering, and DevOps extends the blast radius to include Operations. I’ve always thought that the conflict arises from a fundamental conflation of people’s roles as contributors to an outcome (“squads”) versus their role as being a subject matter expert (“tribes”). The former is how to build your MacGuffin as best as you can to address the market need, whereas the latter deals with ensuring your resources are what you need. 

Moreover, organisations are normally oriented around the latter, as only senior people are allowed to spend money = hiring and authorising budgets. Delivery teams are formed as a “matrix” with no line management accountability, and executives who should be resource managers instead become points of escalation for the products, because that’s where the action is. This will only work if there’s a strong common vision of what good likes. If not, you have people up and down the food chain trying to balance what’s good for the product versus what’s good for their line management within their own heads, which inevitably causes cognitive dissonance. 

The closest I experienced a true alignment between the organisation functions was when I was running both an ops and an engineering team concurrently, and I myself represented the product (an ops tool for assuring content through the media supply chain). It was a small team, but it achieved remarkable results. Looking back, I think it was effective because the close collaboration between Product, Engineering, Testing and Operations got us to the point that we had a very short distance between defining changes in theory and measuring success in the market (and using the learnings to define the next set of changes).

DevOps defines how that close collaborative culture should work. It’s a mindset, not a toolset. It’s not about deploying automation, but understanding how automation shortens the distance from change to market. It’s not about the Head of Product owning everything, but the product team empowered to own everything. So here’s my current definition:

DevOps is a codified culture of collaboration between Market, Product, Engineering, Testing and Operations.

I’ll continue working on this definition as I continue living it. 

What are your experiences of living the DevOps culture? Write to me in the comments.

A Modern Backup Strategy for Families

A Modern Backup Strategy for Families

If you don’t back up, you should. This post will help you understand what makes up a successful backup strategy. [blog.mindrocketnow.com]

My eldest recently told me, in an embarrassed voice, that she never backs anything up. All of her stuff is on her phone and tablet, so if either breaks, she’ll lose something, perhaps something important. I realised that the backup strategy that I’ve successfully implemented for myself isn’t fit for purpose for a busy family. This is the new, shiny strategy that I’m putting in place, and I hope it’ll give you some ideas for your own strategy.

You need a backup for your whole life

Some things that aren’t currently backed up really need to be, and yet not everything needs to be backed up. 

Most of my family’s transactions with the outside world is online, at least in part. This means things that were purely physical now are either mostly or totally online. As our digital footprint increases, so does the scope of things that could be backed up. This is a short list of data that has recently crossed from analogue to digital:

• All media (photos, music, books, films)

• Recipes

• Passwords

• Credit cards

• Share certificates

• Strava run log

• Passports 

• Health data (blood pressure, ECG scans)

• Chats with friends

Yet not every single thing needs to be backed up, and just to make it a little more complicated, not everyone needs to back the same things up. For example, my wife doesn’t feel the need to back up her business tweets. I do. We place a different value on the very same data set. Our backup strategy takes account of this.

However, backing up costs, in both money and effort. How do you determine what makes sense to back up? Here are 3 rules to help you decide: 

1. After you create that post, that file, the piece of data will you need to refer back to it next month? If the answer is “yes” or “don’t know”, make an online backup.

2. Will it take more time and effort to recreate that data from scratch, than to recover from your backup? If the answer is “no” then go ahead and make that online backup. It’ll take us half an eternity to to reinstall and recreate all the app data that’s on our phones, so that makes it a prime candidate for backing up.

3. Don’t back up online data into physical. If you’re happy with your online recipe service, and have everything you need online, don’t buy another recipe book. If you have a password manager, you really need a password manager, don’t write them down at the back of your diary. Only make an exception if that physical thing itself gives you joy - if that recipe book is an interesting coffee table book, if that online photo deserves a framed print, if that album has a limited vinyl edition. 

So your first task should be to write down a list of all the things to back up, per person in the household. Then go through the list and look out for two things:

1. Do I want to reuse this data item in the future? Repeatedly? Then spend some time now to turn it into an asset that needs zero effort to be reused in the future. For example, ripping a CD (if it’s not available on a streaming service) or writing a “lessons learnt” style white paper.

2. Will this data item take a disproportionate amount of effort to back up? Then you might as well be realistic and cross these off the list because they won’t get done. All those baby photos not yet in frames are priceless, but won’t get backed up unless you prioritise the time to scan them all in. 

Have a backup strategy, not just a single backup

If it’s important enough to be backed up online, then it’s important enough to be securely backed up online.

I signed up for a Dropbox account when it launched in 2009. I’ve now got cloud accounts with Dropbox, Google, Apple, Microsoft and Box, as well as Amazon AWS and IBM cloud business storage. That’s a lot of cloud storage, and it’s a headache to manage effectively. Moreover by themselves, none of them make a backup strategy.

A backup strategy needs to be:

• Frictionless: seamless in your everyday tech life;

• Secure: only allow you to access, or give you tools to manage secure sharing;

• Cost-effective: I don’t want yet another monthly subscription, for something that I hope I don’t have to use.

So my backup strategy makes use of 3 policies using different tiers of storage:

1. Real-time for up-to-date online copies: All files that are regularly used are in folders that sync to the cloud. Nothing is solely on my computer’s (or phone’s) storage. This gives me the convenience of having everything whenever and however I want it, available if anything goes wrong with my hard drive, and also gives me tools to securely share with others.

• Dropbox is used for work files (currently 8GB).

• Google Drive is for family files (currently 17GB).

• Apple iCloud is for backing up devices because we’re an Apple family. This is also the only one I pay for, as we back up all 12 of our Apple devices, which means we need the 200Gb tier.

• Microsoft OneDrive is for the Xbox and OneNote because neither integrate with anything else (currently 5GB).

• Box is used for random stuff that I don’t want to manage. It’s the least used, and the one I could just delete without noticing (currently 50GB).

So that’s about 280GB of cloud storage for £2.49 per month, and it’s much more than enough. The biggest data hogs are the Apple device backups, which is why I pay for iCloud. If you choose not to back up your devices, then you could probably get away with not paying a penny for enough cloud storage. Regardless, I wouldn’t recommend having this many cloud accounts, as I have to remember my internal rules for which files are where. Thankfully, all these cloud services integrate well with my various devices, so I can search across them. Try to stick with one service, if you can - my recommendation is Google Drive (Google One for paid additional storage).

The honorable exceptions are for my media files and disk images. I have so much of them (about 6TB) that cloud storage is too expensive, and they don’t change so they needn’t be backed up. So they’re in a Drobo NAS in my home (see tier 3 Archiving). The irony, of course, is that thanks to Spotify, Kindle and Netflix, we don’t use these media files anyway.

2. Near real-time for recent online copies: This is where the automatic backups are made. As an Apple family, we use Time Machine, and it’s a brilliant piece of software. It automatically uses system downtime to create a compressed backup of the computers (but not yet phones and tablets - those are backed up to iCloud). Its brilliance comes from the fact that you can search for files backwards in time as well as across your folders, from the same Spotlight UI, making recovery easy.

My backup location for Time Machine is on a Drobo NAS in the house. This is a RAID6 array, and supports hot-swapping hard drives of differing sizes. It’s around 10 years old now, but the technology is intrinsically quite reliable, so it’s still going strong. However, if you don’t already have one, I don’t recommend spending on a rock solid RAID6 NAS like I did. If the Drobo ever dies, I’ll send Time Machine to the cloud as 2TB iCloud costs £6.99 which is just about half the cost of owning a NAS for 3 years. To me, there’s no longer any economic reason to back up to equipment that I have to buy and maintain.

3. Archive for older offline copies: Archiving is very different to backing up, and is the essentially the opposite of my Near Real-Time policy. I don’t back up my cloud drive; Time Machine is set to exclude these folders. Instead, I periodically copy those cloud-synced folders to my Archive directory in my Drobo. This way, there is a copy of all of my files that isn’t linked to the cloud. If I accidentally delete a file from Google Drive that I want again, I won’t be able to find it online, but I will be able to find it in my Drobo archive. 

Archiving is not just a backup of your backups, it’s also for other file types that you shouldn’t be backing up in the first place. The key criteria is that these files should not change. Archiving is good for:

• All media (photos, music, books, films)

• Last working version of software and OS

• Full disk images

Unlike tier 2 backups, I recommend that you keep them on USB drives rather than in the cloud. Do a full archive once per year on a USB drive, copy it onto a second drive, and put them both in an airtight container somewhere. There’s no need to buy an expensive NAS for this (as I’m doing), just buy inexpensive external drives. (And don’t bother reusing old ones, buy a couple of new drives every year.) Spin them up once a year just to make sure they still boot and can be read from your computer. This will give you an inexpensive archive that is physically secure.

Some final thoughts

From bitter experience...

You should back up thoughtlessly. If you need to think about backing up your stuff, if it’s not automatic, there’s a danger it won’t get done, and won’t be there when you really need it. We use Time Machine, because… Apple. There are plenty of other solutions out there, mostly for free. Use the one that fits with your workflow with zero maintenance. 

A backup is useless if you can’t find anything. Searching is a panacea, but only works if your backup is properly indexed, versioned and regularly maintained (= checked that it can be restored). That’s why it’s worth paying for backup software. However, if you find yourself searching through your backups regularly (and if you search your archive at all), then it’s a sign that you’ve confused your storage tiers. Remember: files that you use often should be in tier 1 cloud storage, back ups that you restore from infrequently should be in tier 2 cloud storage, and archives that you don’t search at all should be in tier 3 USB drives.

Recently, I found my old CD backups (remember those days?) from when I first started working. For a fleeting moment, I thought about firing them up and extracting the files that I might now need, before I remembered that very old backups are dangerous. Even if the data doesn’t degrade physically (electromagnetically), it will degrade in value. Those project files that were important to retain just in case, become irrelevant as context inexorably changes over time. Knowledge is hard to back up. Unless you’ve taken the time to create reusable assets, such as lessons learnt, then adapting an old file to a new context will take longer than creating something afresh. Restoring from a very old disk image will very likely fail, as the drivers will be out of date, so keeping them gives a false sense of security.

Lastly: don’t back up when you need to - it’ll be too late by then. Instead, back up now.

Let me know how you’ve implemented your backup strategy in the comments.