History of Media Supply Chain Workflows

Over the course of my career, I’ve changed the way I look at media technologies, as the industry itself has changed the way it implements technology. In this blog I look at the major changes of the last couple of decades. [blog.mindrocketnow.com]

When I started out in this industry, as analogue was being replaced by digital, each broadcaster had its highly secure data centre consisting of racks of highly specialised opaque boxes, all connected to each other with thick braids of cables. The role of the broadcast architect was to understand all the configuration possibilities of each of those boxes, and how to connect them into chains that represented the workflow you were trying to implement. 

The first revolution was to replace all of the highly specialised opaque boxes with general purpose computers. These were more occluded boxes, because whilst the hardware was general purpose, the software became even more configurable and required new programming skills to chain into workflows. Fundamentally, those workflows were the same, and built into the same racks, just cheaper to buy.

The second revolution is well underway, and was born of a desire to stop capital spend on new boxes every year, and to prefer annual operational spend on someone else to do that for us. So those broadcaster data centres are now all shuttered and workflows are now in the cloud. Creating the chain now only requires mastery of one or two programming languages, but also of the myriad of APIs for all the software components. And because those APIs can be improved at a much faster rate, the possibilities to do interesting things in those workflows has increased at a much faster rate also.

At its simplest level, the workflows have remained the same throughout all of these three revolutions. For the purposes of this post, I’m going to focus on VOD for TV, and the playback workflows within that. (I’m not going to get into associated and important workflows like marketing, revenue assurance, operations.) Let’s get into it!

Simplified broadcaster workflow

The role of a broadcaster is to market their catalogue as widely as possible. To do this they focus on making programmes applicable to the market:

• Choosing titles that will be popular, then putting the corresponding master assets into the workflow

• Making sure they conform to technical quality standards

• Making sure that they conform to legislative taste and decency standards

• Enriching with metadata (enrichment assets) that makes the asset more attractive, such as creatively written synopses, artfully composed imagery, dubbing in local language by local talent, creatively written subtitles

• They also look at enrichment in a broader sense, such as marketing titles locally in cooperation with service providers

• Insert ads

• Finally, the asset is exported in a form that can be imported by service providers, at negotiated cost, at an agreed schedule

Simplified service provider workflow

Service providers are sometimes the same broadcaster that provided the VOD asset, other times they are MVPD (multichannel video programming distributors), or perhaps content aggregators. All have a similar role to broadcasters, to market their (aggregated) catalogue (to consumers) as widely as possible, with the addition of implementing an attractive business model:

• Making partnerships with broadcasters that are popular

• Making sure the VOD assets that are provided conform to technical quality standards, and are timely

• Further conform the VOD assets to conform to platform technical standards, e.g. different bit rates, packaging

• Insert/ replace ads

• Enrich the metadata to reflect the business model, including: DRM method, tagging for revenue accounting

• As each consuming device might have a different preference for how it consumes the asset, the asset is exported once more

• Ingest the associated metadata (enrichment assets) into a catalogue and making it discoverable

• Promote the title both technically e.g. on the home screen of the app, and also non-technically e.g. in marketing literature

Simplified consumer device workflow

In many respects, the end consumer holds the power in the value chain. They determine the value of assets by asserting their choice of what to watch. They also provide one of the two sources of money in the value chain by buying stuff. To do this, their workflow consists of:

• Discovering content to watch, which is usually done by viewing associated assets, e.g. the next programme in the programme guide, “watch next” recommendations, marketing literature, ads

• Buying the content (even free content goes through the same authentication/ authorisation/ accounting process)

• Playing the content (which encompasses the technical processes of unpacking, decrypting and presenting images)

• And I’ll separate out viewing as oftentimes the viewing is on a different device to the playback, which can also insert/ replace ads i.e. smart TVs

• Watch the ads, watch the content

What next?

This business has changed dramatically over the last couple of decades, unsurprisingly as technology as a whole has changed dramatically. Broadcast generally follows, not leads, and at a comfortable distance, in order to sweat every last drop of amortisation from investments. So it’s relatively easy to make predictions - just look at what’s going on elsewhere. Here are a few of the trends happening in other industries which will wind their way into broadcast at some point.

Intelligence at the edge

A relatively new website architecture pattern is the Jamstack (JavaScript + APIs + Markup). The idea is that the website application itself is static, and as much intelligence as needed is associated with the content. That means the website can be distributed over a CDN and doesn’t need to be built for each visitor individually.

For media workflows, as some of the intelligence can only be implemented when you know the target, it means pushing out some of the conform/ enrich/ export steps from the central cloud to the network edge. This becomes possible as CDN providers implement cloud processing features at cloud prices.

Dumb apps

Putting intelligence at the edge means being able to deploy dumb apps. Maintaining apps fast becomes a profusion problem. The Now TV app is available on Apple, Android, Xbox, Roku, Samsung smart TV, LG smart TV, Amazon Fire TV as well as its own range of hardware. Each of thes hardware platforms comes with their own app store, and each app store has its own foibles. Each release can take 2 weeks to approve, so many companies employ release managers to handle the negotiations with the app stores to assure that they will pass first time. Even so, it takes a lot of resource to manage 8x approvals in parallel.

It’s very attractive to only need to deploy a new release once a year, to reduce the app store submission headache. This requires finding clever ways of making the app dumb. Web browsers are very complicated pieces of software that are almost invisible to surfing, and this is the direction that video apps need to go.

Intelligent content

As the app gets dumber, the content will need to get cleverer. Entire features of the app will need to be delivered as content. Content will self-aggregate as web pages with hyperlinks can, so that enrichment assets can be viewed at consumption time. And content will change after publishing, no longer remaining static. Just like the recent Cats movie, you’ll be able to see version 2.0 of your favourite movie as it reacts to real audience reaction.

What do you think of my predictions? Let me know in the comments!

My road-tested GTD setup using Todoist

A key part of my working from home habit is the Getting Things Done technique. This is how I’ve set it up. How does it differ from your setup? [blog.mindrocketnow.com]

Getting Things Done has revolutionised my productivity. I’m now confident I know what to do and when. More importantly, I’m confident when I don’t need to do anything, so I can enjoy my down time. This post isn’t to extol the virtues of GTD, but to get into the detail of how I’ve implemented it, as the GTD technique leaves the implementation to individual preference. 

Everything is paperless for me (at least as much as I can), so my GTD implementation is totally electronic also. I’m omnivorous in which technology I embrace, so I need my GTD setup to work across Echo devices, Android tablets, iPhones, Apple Watch and on the web. My primary tool of choice is Todoist, and is available across all these platforms.

Capture and Clarify

The first element of a GTD setup is Capture.  I’ve tried to reduce my capture channels as much as possible so that I can be confident that nothing gets lost as it’s passed to me, which is why I funnel everything into my Todoist inbox. All my multiple email addresses are now aggregated into Spark (app only, no desktop client), and anything that needs actioning gets sent to my Todoist inbox. I use Feedly to aggregate my RSS feeds, and anything that needs actioning, or perhaps some dedicated time for reading, also gets sent into my Todoist inbox. Finally, I still have some paper that comes my way, which goes into my IRL inbox on my desk. As this gets processed, I transcribe them into tasks that into my Todoist inbox. 

Inevitably, there are many more capture channels than can be aggregated into Spark, Feedly or my inbox. A couple have integrations with Todoist: Slack, my work Outlook email. Most do not have integrations: LinkedIn messages, WhatsApp messages, my notebook, voicemail, the table by the door, my wallet, telling me stuff - these need to be transcribed into my Todoist inbox.

Then comes processing the captured items. I take each item from my Todist inbox, top to bottom, and Clarify them by adding enough detail that they turn into actions, organise them into Projects, do the actions according to your Context (or what you feel up to doing next).

Sometimes clarifying needs additional contextual information, which GTD calls Project Support. I keep as much electronic as possible, in OneNote. I chose OneNote after a dalliance with Evernote because Microsoft seems disinterested in charging for a pro-level software, at least for now. Any that can be digitised, including photos and web clippings, here with titles that enable cross-referencing to the Todoist action. I also add the URL of the page as a comment to the task, to enable one-click linking of action to context.

But oftentimes you can’t get rid of physical stuff, so I also have an expandable folder comprising of clear plastic A4 wallets to hold this stuff. It’s important to process the stuff before putting into this folder, to avoid it becoming another capture channel, yet another thing to keep track of.

Do the easy tasks first

One of the powers of GTD is that it quickly filters out noise from your inbox by asking some simple questions. The first is: is it actionable? If it isn’t then it’s one of: poorly defined so you need to clarify it some more, not worth doing so you bin it, good to remember so you archive it (again in OneNote), or perhaps it needs to be done at a future date or just sometime in the future perhaps when there’s some more information. For actions to Incubate, Todoist enables adding time and date alarms and integrates with Google calendar so that you can diarise it. If you need to develop further information, that becomes an action into the inbox.

If it is actionable, GTD’s second question is to decide: if you can defer it (use the same technique as Incubating), if you can delegate it (email it with the supporting the OneNote web page if applicable), if it can be done in 2 min then do it now, or if you need to set some time aside to do it.

The 2 minute rule is probably the most powerful tip for me. 2 minute is all it takes to write an acknowledgement email, to look something up in archives, to schedule a meeting. However, it’s not long enough to compose an email with information content, or write something new for the archive, or write a good agenda for a meeting. So the 2 minute rule filters out all the low brainpower tasks which can be done immediately.

Get to work!

The first scan gets to the list of actions that require work to complete. The scan also familiarises yourself with the actions, so you’ll inevitably start prioritising them. Todoist has a day view which presents all the tasks with time dependency, and has tags to describe priority and other categorising features. GTD recommends you do the tasks according to your context. So if you’re at your computer, do your computer-based tasks. However, understanding your context is quite nuanced, and is the area that has seen the most change in my GTD workflow.

I’ve set up the priorities for be:

1. Overriding priority, do it right now - used sparingly

2. Impactful actions - stuff that makes an immediate difference

3. Possible actions - things I can do right now, without waiting for someone else, or need to develop any new information

4. Default

I’ve set up the context labels also: 

• Someday and Soon to filter out the actions that need thinking about at a later date

• Watch List and Reading to classify entertainment recommendations

• Agenda For and Waiting For to capture the actions from other people that will enable me to do my action

• Offline because some stuff needs actual physical labour rather than pressing of keys

• Online because most of my world is online

• Networking because I find this to be really hard, so put some special focus

• Decision because some decisions need consideration even if I have all the information, and this gives me permission to take the time to think

My day is then organised by these tags. Todoist has a powerful query feature which enables me to create views that gives me my work list for each of my contexts. These are (together with the filter):

• Urgent tasks (P1 or those requiring a Decision)

• Impactful tasks (P2)

• Next actions for work (P3 or above and in my work projects, but not in my reading or deferred lists)

• Next actions for personal (P3 or above for actions for stuff for my family, so gets different focus)

• Next actions for errands (P3 or above and in my home projects, but not in my reading or deferred lists)

• Agenda for stuff to talk about with my wife

• Agenda for others

• Entertainment list (reading and watching)

• Final long list for my Weekly Review

Future changes to the workflow

One of the features of work in lockdown is that pretty much all work is online and remote, so the online label no longer provides any help to contextualise better. Others have had success in using 10min, 1h and 1d labels, in order to guide how much effort that will be needed.

Todoist has a gamify element called Karma. You set a number of tasks to complete each day and each month and watch your progress up the Karma ranking. I’m not getting through the same number of tasks now as I did before, so I need to reflect on whether I’m capturing the right actions, or perhaps not binning enough.

However, these are tweaks. Todoist + integrations + GTD has served me extremely well over the last 2 years, and I recommend the approach to everyone.

Giving up on Spotify

Today, I’m taking the big step of migrating from Spotify to Apple Music. Music is a big part of daily lockdown life, so if I get this wrong, the family will be revolting. Let’s see how it goes. [blog.mindrocketnow.com]

We’ve had a premium account for 5 years now. I’ve loved the service from Spotify, and found its music recommendations to be uncannily good. At the time, it was streets ahead in terms of integrations. It was easy to find gadgets that natively integrated with Spotify, and really hard to find native integrations with Apple Music or Google Play.

Over time, most of our device manufacturers announced integrations with all the music services, so that advantage eroded. So when an offer for a free family plan for 4 months comes along, it’s time for a change to save some money. After that, it’s £15 per month, same as everywhere else, so whether we keep it or not will be down to how much we as a family like the service.

We’re an Apple family, so setting up the subscription to Apple Music was very smooth. Because I hadn’t used iTunes in a long while, I had to review and accept new terms and conditions. I actually read them this time, and learnt a few things, like:

• There’s an android app for Apple Music. Strange bedfellows.
• iTunes Match is included in the Apple Music subscription (yes, it still exists, though nobody cares).
• I’ve given blanket consent for Apple and bedfellows to use my “technical data” - and because it’s an undefined term, it could pretty much encompass anything. I wonder how GDPR feels about this? 
• There is a cancellation policy for digital goods as long as written instruction with their forms is given within 14 days.
• Children under the age of majority should be made to read the agreement with a parent to make sure they understand it - fun times!

After setting up the subscription, it’s time to set up the service with all our devices. In order from easy to hard:

• Apple hardware - so easy. Also, ooh now the Apple Watch app works

• Sonos - set new default music in settings

• Waze - change default to Apple Music in settings

• Alexa - download and link new skill, then set default music service in settings

• (Sadly defunct) Triby - no native support for Apple Music (unlike Spotify), but can play music from the iPhone via bluetooth, or use the Alexa integration to stream from 

• Xbox One - we used to stream background music in the native Spotify app, but there’s no native Apple equivalent. Could download a third party airplay receiver for $20, but we decided to just play using the Alexa speaker in the same room.

• Android hardware - as easy as downloading the new-found app, then signing in. Except the app hangs on my shiny thing.

Now it’s the hard task: migrate my playlists. My pro tip is to prune them first! Spotify puts all the liked albums into the liked songs playlist, so making the list bloated. Once that’s done, there are plenty of tools to help move them from one service to another. I was at my desktop, so I used https://www.tunemymusic.com/ and it was very easy. I had 7882 songs so it did take 10 min to process, and 398 songs were lost along the way. But once done, all my playlists were magically available everywhere. Authorisations are via API so no passwords are stored by the website. Just to be sure, you can revoke permissions from Spotify’s web site https://www.spotify.com/uk/account/apps/ (which is a bit flakey) and a setting within the Apple Music app. 

Obviously, there’s also an app for that called Songshift. It’s perhaps more convenient for individual songs as the app can be invoked from the long press contextual menu. Again, pro tip is to revoke permissions once the migration is done.

It’s done! The whole process took about an hour, including a trip for coffee, so really quite painless. First impressions are that the range of music appears to be equivalent, but not identical. Recommended playlists are good, presenting music that I’d never heard before but immediately liked. Beats 1 seems to be exclusively R&B so not very interesting to me. The Android app is really good, better laid out than the Spotify app.

There are some bugs in the Apple Music app which are surprising and annoying, and as a consequence of a higher reliance on connectivity. I can’t actually play any music in the detail page of the iOS app as it times out before fetching the song list. As a result, the detail page is just blank, so I have to long press from the album page. Phone reboot fixed this. The watch app has to try twice to connect to the phone (don’t think that’s due to Apple music). Spotify was a lot better at dealing with lower and flakier bandwidths.

However on reflection, the family agrees it looks like a keeper. Will let you know in 4 months time.

Controlling the Algorithm

Can algorithms be racist? If they can, how can we control them? In this post, I look at the presentation of bigotry in technology and what we can do about it. [blog.mindrocketnow.com]

As part of my Python course, I’ve been learning machine learning techniques, or how bots recognise patterns in data sets by calculating correlations. In other words, how to create a decision-making algorithm. When I looked up from my keyboard, I began to notice that algorithms are getting some very bad press at the moment, which made me think a bit deeper. 

History of bad ideas

Facebook is suffering from more sponsors withholding ad spend. Despite Facebook claiming today “There is no profit to be had in content that is hateful”, the company still cannot stop big brands’ ads being placed next to racist posts. Big brands are responding in a way that’s eye-catching, by withholding ad spend. Eye-catching, but perhaps not ultimately effective as just 6% of Facebook’s revenue are from big brands. The remaining 94% consists of hundreds of thousands of businesses around the world, who cannot afford alternative methods of reaching their audience. The extraordinarily broad success of Facebook’s algorithm emboldens it to be blasé to both government and big business despite civic and corporate activism. And Facebook has some truth on its side, as its algorithm wasn’t designed to promote racism, so how can it be responsible for the racist outcomes?

Perhaps you remember Microsoft’s Tay bot, born in 2016. Within 16 hours it became a staple of future AI courses as a cautionary tale. Unwisely, Microsoft designed Tay to learn language from people on Twitter, as it also learned their values. Trolls targeted the bot and trained it to be a racist conspiracy theorist - presumably just for fun. Which essentially how trolls birth other trolls in their online echo chamber.

Things haven’t improved over time. Let’s try an experiment together right now. Perform a Google image search for unprofessional hair. What do you see? I did that just now, and saw pictures of mostly black women, which infers that most women with unprofessional hair are black. Which is racist. Was the algorithm that presented the results racist?

Enough people labelled pictures of black women with “unprofessional hair” that Google’s algorithm made the correlation and applied that inference to all the photos that it came across. This news story first broke in 2016. Before then, you only saw pictures of black women. Now, you see news stories interspersed with pictures of black women. Which shows that Google’s algorithm can’t distinguish between the two types of results.

It’s worth repeating: the algorithm cannot differentiate between non-discriminatory stories about discrimination, and results that infer discriminatory conclusions. This is because digital footprints never go away. Digital history is only additive. But at least whilst the search algorithm reductively simplifies and generalises, it doesn’t confer moral value, as both types of results are shown together.

It occurs to me that this is no different to people simplifying and generalising. But people normally understand that individual interactions are nuanced and to be judged on their own merits. Algorithms inherently do not. And it’s so much worse when algorithms enable ill-judged conclusions because they’re so impactful when they get it wrong. Algorithms now control all the complex transactions in life:

• Presenting options for what watch next in YouTube;

• Adjusting your insurance premium based on how hard you brake and accelerate;

• Sequencing traffic lights in city centres;

• Filtering your CV for keywords, to see if you’re a good candidate to interview;

• Then assessing you might fit into company in your first video interview by measuring how you fidget;

• Analysing credit card transactions to spot fraudulent activity;

• Predicting crime hot spots based on the wealth of neighbourhoods;

• Spotting the faces of terrorists flagged on watch lists on public transport CCTV.

Correlation is not causality

The core of the problem is that algorithms present correlation, and we interpret them as causality. Which at best is spurious, and at worst is bigoted. Algorithms aren’t inherently problematic, but can become so because people are, and the algorithms learn from people. The results aren’t inherently problematic, but do infer problematic conclusions, if we don’t understand their limitations. This train of logic is how we end up with discriminatory health insurance pricing.

Algorithms can be inspected, whereas humans cannot. But let’s not mistake this transparency for understanding. Even if the algorithm itself is clear and concise, the data sets are often complex, which makes outcomes unpredictable and not understandable. Then because we don’t understand them, yet believe that some other smart person could if they wanted, we over-trust them. There’s no civic demand to examine them, and civic acceptance of the conclusions. Which is how we end up with law enforcement resourcing algorithms over-policing poor neighbourhoods, and becoming part of the problem. Funding according to the algorithm targets the correlation, but not the cause.

So we have seen how spurious correlation and inherently biased data sets are major weaknesses of algorithms. The third major problem is that algorithms use past data to make predictions on likelihoods of decisions. As every investor knows, past performance does not necessarily predict future results. Decisions based on likelihoods are very bad at figuring out what to do in edge cases. When you apply those algorithms to millions of decisions, the number of bad decisions at the edge mount up. And each of those bad decisions changes someone’s life. Each edge case matters to someone.

It’s beneficial to allow people to game algorithms? For instance, part of the duty of hospital administrators is to work the NHS appointments system to enable patients to reorganise treatments for the convenience of the patient. Human intervention is needed because the appointments system optimises hospital resources.

Last Monday, TikTok and K-pop fans claimed responsibility for the lack of supporters at Donald Trump's campaign rally. They understood how the TikTok algorithms boost videos in order to promote them to like-minded activists, and deleted their posts after a day or two to avoid the plan leaking to Trump’s team.

The alternative to not understanding these algorithms is no longer feasible. The world has become too data rich to be navigated without help from artificial intelligence. We can either manipulate them to our benefit, or they will manipulate us to theirs. So what can we do about it?

Legislation and activism

It’s not illegal for a business to prioritise its resources to serve customers that are willing to pay the most. For example, it’s not illegal to prioritise call centre agent pick-up times based upon whether your number matches a list of high-value customers, even if it means you don’t answer calls from low-value customers at all. But it is discriminatory. 

In the EU GDPR legislation gives citizens the right to explanation of automated decision making. On request, companies are obliged to explain how sharing data links to decisions made about customers, and the impact of that decision. This lays important legislative foundation, as it forces companies to understand how data links to decisions, which many do not. However, this legislation falls short of protecting against bad algorithms, data and decisions. 

As we’ve seen, the combination of complex logic, inherently biased data sets and prescriptive application, make algorithms overly blunt instruments. This is now recognised by leading tech, perhaps more so than governments. Amazon notes that technology like Amazon Rekognition should only be used to narrow the field of potential matches, but because legislation still doesn’t understand this, it is implementing a one-year moratorium on police use. Amazon recognises that asking individuals to safeguard themselves against mis-application of its algorithms is unfair because it’s just too complex for end consumers.

This is exactly where we need legislation to protect us. My hope is that parliaments will enact well-considered limits on use of algorithms in industry and government, focusing on public safety and law enforcement. We should then use these limits to hold companies and agencies who do not safeguard their algorithms to account.

Legislation is not the only only tool that we have. As I’ve noted early, I do agree that inspecting algorithms and data sets is out of the reach of all but data scientists, so useless to most agencies. However, specific testing for discriminatory outcomes isn’t. So another important tool is to empower trading standards bodies to test for algorithm bias, the same way that they test for food hygiene.

Finally, there are things that end consumers can do to train the algorithms. We can increase our social connections, because online segregation is as destructive as physical segregation. By exposing ourselves to more opinions, algorithms are exposed to more diversity and present less echo chamber click bait. 

We shouldn’t engage with the click bait, the posts that elicit strong emotion. Imagine if the whole world scrolled past the trolls - then the oxygen would be removed, and the trolls would wither away, because the algorithms would see that bilious content is not clickworthy.

Fundamentally, we should play nice so that algorithms don't make bigots of us all, otherwise we only have ourselves to blame.