Driving Home In A Large Bomb

On the one hand, an outdoor grill basically means you’re 100% Americaon. On the other, there’s the drive home when you realize you’re in a metal cage containing gasoline and propane, all sitting on top of a sizeable bed of lithium. Past a school. That focuses your mind, I’ll say.

Last week’s medical shenanigans got resolved in the “non-expensive, but you’ve screwed up the one treatment for psoriasis that has worked for you” manner. Which is marginally better than “tens of thousands of dollars and you’ve lost the treatment”, but still a bitter pill to swallow as we head into the colder months (which is when it’s worst for me up here in Ohio). Hurrah for the perfect framework that is the US health insurance system! (and 3rd-party HR SaaS operations while I’m here)

It is, however, difficult to remain sad and upset when you have a agent of chaos running through your house, laughing in absolute delight at the toy icemaker she’s been playing with all afternoon, and desperately trying to make friends with Helvetica Black (who is, I think beginning to accept that this is her fate). Or when she finally does fall asleep to the gentle sounds of The Shamen’s Ebeneezer Goode…

Thank Goodness You're Here (@ Lucidworks)

A good first week back at Lucidworks, slightly marred by an ongoing health insurance disaster that still has no conclusion in sight (but a variety of potential outcomes from a shrug of shoulders to a very expensive mistake). I even managed to go from idea to demo for a small (but fun!) research project in a couple of days. Which is not something I’ll be able to do every week as a manager, but it’s good to still be doing active research!

(Don’t worry, there is a long list)

Also, how did I not know that David Peace has a new book coming out??? Another football book to go along with The Damned Utd and Red or Dead. I’m starting to think we might never actually see UKDK…

Thank Goodness You're Here!

You cannot go wrong with a game that uses the IBA advert cue dot as its save icon.

Announcements!

As promised, exciting employment announcements¹!

I’m going to be leaving Bookend.AI this Wednesday. Following that, on August 12th, I’m going to be heading to…checks notes

Lucidworks!

checks notes again

Lucidworks?? Yes, I’m heading back to the company that I left last year, but this time as Senior Manager of Data Science. Although this is, as you might expect from that title, more of a management role, I will still be doing research as part of the job. Things to hopefully look forward to in the near future:

• Fun! And! Games! With! Sparse! Autoencoders!
• Death! To! MS! MARCO!
• The! Rudest! Paper! Title! Discovered! By! Accident!

And more besides. I’ve been missing search; it’s time to get back in…!

Building a website in 2024

As mentioned earlier, it’s the 20th anniversary of You Are The Generation That Bought More Shoes And You Get What You Deserve, and to celebrate, I rebuilt the website, running into all sorts of issues that weren’t really a problem in the context of webdev when I built the original version.

The first iteration of the website had a bunch of clips from The Mayfair Set and a bunch of quotes; the page simply cycled through the clips and quotes at random. Which is fine, but I wanted something a little more interesting to celebrate two decades.

Dance, you fuckers, dance¹

Firstly, I wanted more clips. That was relatively easy — all of Adam Curtis’ series went up on iPlayer a couple of years ago, so I have nice hi-res (and complete!) copies of those. Instead of cycling at random, though, I decided to return to my love of embeddings.

Using a SigLIP multimodal model, I encoded the lyrics of the song, all the quotes (with some new additions) and a random frame from every five seconds across every single episode of Curtis’s documentaries. Yay, a bunch of embeddings! You know I love them.

Once I had this pile of embeddings, I turned back to the original video for the song. I split it up into 38 different five-second fragments, and then every slot is randomly assigned to either:

• A clip of the original video
• A Curtis clip based on the nearest neighbour lookup from the lyrics at that point in the song compared to the video embeddings, picking a random element from top_k
• A Curtis clip based on the nearest neighbour lookup from the current quote compared to the video embeddings, picking a random element from top_k

Obviously, the easiest thing to do at this point would be to bung all the embeddings in a vector database, but I didn’t want the hassle of having to deal with setting up FAISS or a more complicated vector store. Plus I also wanted it to be pretty fast…and given that the videos, the quotes, and the lyrics are fixed, I just precomputed all the embedding lookups for a top_k of 50. torch.topk for the win².

This gave me two very large arrays, which I could have stuck behind a Python API to generate the clips on-demand. But I was feeling like the website should be even more dumb than usual, so I just got Claude 3 Sonnet to generate a bunch of JavaScript and copied the arrays into the HTML page directly. It’s all there, go and peek (and don’t blame me for the terrible JS code. The computer wrote it!).

After that, it was just a matter of dealing with how browsers handle playing audio (when I built the original website, auto-playing was allowed, but that hasn’t been the case for quite some time now), and I also hard-coded the start and end of the song to play from the original video to provide a better ‘playlist’.

My feeling was it was going to be relatively simple to package up and get started on Google Cloud Run. I just had to upload the clips and make a small Docker container for hosting the page (with a tiny FastAPI server to do some mounting and serve the page itself). That was fine, but when I tried to actually get the application running under the proper domain name, everything broke. Mind you, it broke with an obscure Kubernetes error that I have seen a lot in my time, so even though the UI couldn’t tell me what was wrong, I knew instantly.

It seems that Google Cloud Run creates a pod on a random internal Kubernetes cluster and uses the domain name as the pod name. When dealing with sane domain names, that works fine. However, youarethegenerationthatboughtmoreshoesandyougetwhatyoudeserve.com is 65 characters long. And pod names can only be a maximum of 63 characters. Boo. With a bit more access to the system, I could probably have fixed that, but you don’t get that luxury with Google Cloud Run. I ended up having to use an ugly DNS redirect to the bare Cloud Run URL (which is why you see the domain name change when you visit the site).

And there was more to come. I blithely posted out an announcement on Bluesky, but forgot that bare domains links these days default to HTTPS connections. and for some reason³, I didn’t have an SSL cert. No problem! Let’s Encrypt! Except…imagine an entire day of bouncing around SSL providers before coming to the conclusion that no, none of them were going to accept the domain and its inordinate length. So my launch fizzled and spluttered. Oh well. In the meantime, the non-SSL site is fully operational.

See you back here for the 25th?

This Frequency's My Universe

You Are The Generation That Bought More Shoes And You Get What You Deserve by Johnny Boy is twenty years old.

You may turn into dust now.

To celebrate, I’ve updated the youarethegenerationthatboughtmoreshoesandyougetwhatyoudeserve.com website. It’s a complete rebuild that builds a new ‘video’ for the song using clips from every Adam Curtis series¹ and the original song’s video. I’ll be posting a deeper explanation into how it all works by using image and text embeddings in a week or so, but for now, feel free to view and consider the passage of time.

YEAH! YEAH!

Baaaaack Home

Actual news to come later this week:

• 20th anniversary of a thing!
• Important employment details!
• (perhaps) A rant about the maybe inevitable choices of Durham!

Quiet Here, Isn't It?

Going to be a lean few weeks here, sadly. Illness and probably spending much of the day unblocking a sink means not much of a post today. And then I’m spending next weekend visiting friends and former¹ co-workers, so don’t expect a lot then either.

I do have a couple of days off coming up, so maybe I’ll work through the backlog then laughs bitterly…

Peak Dad

Continuing to be a tease, but some exciting (if perhaps a little odd) news this week that I’ll likely be sharing properly in the first week of August. Not exactly all change, but some change.

Other things from the week:

• on recent events: welp

• The happiest baby is apparently a baby with her mouth stuffed full of croissant.

• In my quest to hit Peak Dad, I have ordered a Blackstone Griddle. Maeryn will hopefully learn to love smashburgers.

• Not only am I out of proper brown sauce, I’m almost out of the pretender brown sauce. And I’m down to my last half box of tea! British reserves are low…

• I’m not saying I have a problem, but I actually have a schedule for buying Lego sets for the rest of the year…

• This Frequency’s My Universe is now pretty much done. Two bugs remain, one of which seems to be a limitation of Google’s Cloud Run service and how it maps domain names to Kubernetes pod names (because I remember the cryptic error message from some earlier k8s adventures) — this I can’t fix and so there will be an awkward URL redirect. Things look terrible on mobile right now too, but I think a few CSS directives should fix that. Looking good for August!

• Planning for Thanksgiving has begun! But…this year we’re going to try and scale down our ambitions a little. Check back in a few months to see how that’s worked out. Our record for keeping things contained is…not great.

• It’s 25 years since The Mayfair Set was broadcast. I still want to know if there are more lyrics to the Starship rewrite of We Built This City that was performed for Milken:

We built this city

on high yield bonds

All This And France Kicking Le Pen Too?

A strangely muted election night, considering the circumstances and the absolute scale of the defeat. While seeing the back of Rees-Mogg (until he is regenerated into his inevitable final form as a Lord), the hapless Truss, Corbyn seeing off the Labour challenger, and watching the Paisley dynasty disappear from the night were all great fun, Farage winning handily in Clacton and Starmer’s Labour not exactly being inspiring meant it did not quite bring the 1997 feeling.

But maybe that’s a good thing, given what happened there? I can wince at Starmer’s language cancelling the Rwanda programme, pointing out its failure as a deterrent instead of the immorality of it. But on the other hand, he cancelled it within a few hours of becoming PM. Not even waiting until next week. Unlike Blair, they actually plan on nationalizing the railways - Ed finally gets to execute the plan that seems to have survived his and Corbyn’s stints as Leader of The Opposition.

And the idea of creating New Towns for the first time in decades, planning regulations and the green belt be damned? That’s actually pretty radical, and in the right hands could be amazing. Towns built for the 21st century - you can’t say “15 minutes” out loud, but imagine new areas for living and working where pedestrians, bikes and public transit could be given the same priority as personal cars! EV stations! Houses that aren’t just Barratt Homes extrapolated to their 21st century endpoint with cheap fixtures and fittings! New ways of living!

Of course, I’m likely to be disappointed. It’ll probably be “Poundbury, but we added two charging stations”. But still, the possibility of actually building something in the UK that isn’t in London?¹

So, a little optimistic? Just maybe?