Coding Fatigue

There’s a bunch of articles going around about AI burnout and how this New World of Claude may not be all it’s cracked up to be, which is mostly summed up by this skeet from Camille Fournier, about a manager’s lot:

The part of "everyone becomes a manager" in AI that I didn't really think about until now was the mental fatigue of context switching and keeping many tasks going at once, which of course is one of the hardest parts of being a manager and now you all get to enjoy it too

— Camille Fournier (@skamille.themanagerswrath.com) February 10, 2026 at 8:30 PM

This week, I have run 25 experiments, all trying to both replicate and improve upon one particular embedding paper, sometimes staying up beyond 11pm to get the most recent results and kick off my next idea, have designed a tvOS app that’ll work with my local NAS and an NFC reader in an attempt to reduce the slop that Maeryn is exposed to (the debasement of “We’re Going On A Bear Hunt” on YouTube is what pushed me towards this), and have already formulated new model training plans that will likely take up the rest of February. On top of actual work, and you know, spending time with Tammy and Maeryn. I still haven’t put the earlier projects I did anywhere near deployment, as there’s always something else to be tinkering with.

Token Anxiety

i think i mostly echo this for myself. with so much that can be done, i often feel like i should be doing something, always

[image or embed]

— Tim Kellogg (@timkellogg.me) February 15, 2026 at 6:44 AM

(the important thing is that I am not quite at this point, though I will say I did make time this morning to kick off a new run of a refactored LEAF trainline pipeline this morning while I was having breakfast. But you’ll pry my books from my cold dead hands)

Anyway, aside from a few mountains dotted around here and there, the snow is mostly gone. The solar panels have stopped complaining and started producing power again. Now I just need to see if the retaining wall got damaged even further under the weight of all the snow…but it’s basically beyond salvaging, so I don’t think it’ll really matter in the grand scheme of things.

In-between starting this post and finishing it, the first skeet has become a flashpoint for anti-AI quotes, and even one of my ex-bosses became the main character on BlueSky for a good hour or two. It’s incredibly annoying as I watch people share blatant lies and slip into 9/11 conspiracies, or just infuriating when I read an article about When The Wind Blows that dismisses the characters’ memories about the Blitz as lying to themselves. I don’t know why this bothers me so much, but it does; I have read retelling after retelling of people’s time during WWII…and yes, quite a lot of them had thrilling times. Ignorant Americans trying to rewrite history that isn’t even their own. No, this doesn’t have anything to do with AI per se, just that the people that apparently can’t even read and understand sarcasm also have a tendency to share those things. As well as getting deeper and deeper into a conspiracy mindset that feels like it is going to end up re-inventing The Protocols of the Elders of Zion but from a leftist perspective.

And now I should probably stop subtweeting half of the internet and go to bed.

60-A-Day Cough

Most of the illnesses seem to have faded away now, but I still have a cough that sounds like it is has been sponsored by John Player Special (Number 666).

The temperature will go above zero tomorrow; after two weeks of -20ºC at night and snow everywhere, I am beyond ready for the melting. It has been such a long time since I have seen grass.

Anyway, with things thawing out, I have to start thinking about the rest of the month and also next month…which means it is time to begin planning for Maeryn’s birthday cake! I don’t have much to go on, but our enthusiastic cake expert has demanded “red cake”. Which means this technique will probably be involved at some point but beyond that, I am still trying to think of everything from shape to flavour…still, over a month to go right now.

More exciting updates next week!

End of January

Quite frankly, I am very sick of being sick. I have spent most of January completely wrecked, hanging onto work with my fingernails, shivering in bed at 8pm, running out of breath simply by standing up, waking everybody up with loud uninterrupted coughing, and you know, just generally being miserable. Let us hope that February is slightly better.

I do have a bunch of things to talk about next month, but only because they didn’t get finished this month and so Claude and I have to take them with me to Shrove Tuesday; one of them is a project I’ve been tinkering with for eight years that Claude cut through a Gordian Knot in under five minutes, and the other is something that came out of watching Eno last year. If I can get either of them to a demoable stage, we’ll have another fun walkthrough. If not, then you can all wait until March.

It’s snowing again. It wasn’t content with being -20ºC this morning, it is now snowing once more. Not a fan. Although seeing a half-frozen Ohio River from the bridge you’re driving over is an interesting sight. It’s not going to stay cold enough for it to freeze completely for the first time since 1977-8 , but it’s going to come close…

If you sent me an email in January…you might get a reply in February…

Adding To The List of Cities Where I've Been Sick In Public: Atlanta Edition

Okay, let’s talk about Atlanta. Or as we call it here: HATE WEEK

• I was supposed to arrive in Atlanta around 17:00, having plenty of time to meet up with a co-worker for dinner and some drinks. Due to Charlotte, I arrived at 21:00. Hopping mad, I find a nearby Publix and then go to bed. This, it turns out, is actually the high point of the entire trip.
• Sleep is fleeting; the hotel bedsheets are paper-thin, and I normally just sleep in a t-shirt on work trips. Even messing with the thermostat doesn’t seem to get me warm. foreshadowing
• Monday starts okay enough; others are complaining about the heating and the chilliness of the previous night. Still, after lunch, I’m feeling cold again. As one o’clock turns to two, I reach for my coat, and the sense of doom starts coming down.
• Some time after four, my co-workers are telling me I look terrible and I agree. I am feeling even worse. I know that bad things are going to happen, and I have a limited time to prepare.
• I head back to Publix, because I know there’s a Target nearby, and despite my current avoidance of them, they’re the only place nearby that I know will sell me the things I’m going to need: masks, a thermometer, day/night cold/flu medicine, new pyjamas, crackers, etc.
• I get lost. As I’m walking (it’s only twelve minutes away!), I get confused and turn right too early. I wander around a multi-storey carpark for thirty minutes until I realize it’s the wrong one.
• As time progresses, I am getting worse and worse, and more desperate to find Target.
• Just in time, I enter through the front doors, promptly march into the toilets, take off my coat and jumper…and throw up.
• Mortified, but slightly better than I was ten minutes ago, I buy things in Target. Another customer helpfully intercedes when my pathetic pleas do not enlist help from the assistants when my medicine purchase triggers an ID check.
• I realize that I will not make it back, so I hail a Lyft for the pathetic sub-mile distance back to the hotel. Still: it’s Atlanta, so it still takes me 20 minutes to get back. I throw up again and scramble into my new clothes, turning the thermostat up to 24˚C in an effort to stay warm. My fever spikes at over 40˚C
• I do not go downstairs for two more days. I have little sleep, I remote into the meetings as best I can. Sometimes I pass out and find time is marching on. But obviously also keeping a brave face it on and saying I don’t need anything.
• Wednesday night, I try to go outside to have the first substantial meal since Monday lunchtime. And the first I’d have kept down since Sunday. I end up in a very fancy mall where the benches are made of marble. I discover this after walking into one of them and collapsing over the top. The bruise is quite impressive.
• Thursday, my final day, the fever has gone, and I’m downstairs, masked up, and keeping distance from people, eating separately, and so on. It has been a terrible trip, but I know I’ll be home by the end of the night.
• foreshadowing
• My flight from Atlanta to Charlotte takes off…well it accelerates on the runway and then abruptly stops due to the cockpit door not being closed.
• Still, we take off five minutes later and we land in Charlotte five minutes early; fifty minutes before boarding of my next flight begins, and an hour and a half until takeoff.
• It takes an hour and thirty-five minutes for Charlotte Airport to deign to give us a landing gate. I break down at one hour and twenty, in a rage and tears.
• It turns out that my next flight left five minutes early anyway.
• By the time I get to the agent desk, most of the rage is gone and they take care of me, putting me up in a decent hotel and rebooking me on the first flight out the next morning. They recommend that it might take two hours to get my bags, so I should probably leave them at the airport.
• The hotel ran out of travel toothbrushes two days’ prior.
• I was supposed to have a lie-in on Friday. Surprise Maeryn and get her in the car to go off to daycare with Tammy, then go back to sleep, light shopping, and pick her up in the afternoon. Instead, I’m tired, still suffering from nasty after-effects from the virus/whatever it was, and I don’t get home until something close to 2pm.
• Meanwhile, back home, Tammy has also been getting sicker and sicker, being infected by the world’s cutest plague carrier and bedtime-denier.

So, an almost completely wasted week, I will do almost anything to avoid going through CLT again, but I am now at least home and can suffer in more space and comfort, and continue parenting. I don’t want to leave the city for quite some time.

Oddly, I spent my semi-conscious time watching Peter Cook documentaries, interviews, Mill Hill, of all things…and I’m fairly sure this contributed to some of my comments as the week went on, one of which was described as “straight from “the guide on being British” book”. I will hopefully not continue to rip off Beyond The Fringe sketches in Slack next week.

What an extraordinary episode in the economic progress of man that age was which came to an end in August 1914!

… The inhabitant of London could order by telephone, sipping his morning tea in bed, the various products of the whole Earth, in such quantity as he might see fit, and reasonably expect their early delivery upon his doorstep.

He could at the same moment and by the same means adventure his wealth in the natural resources and new enterprises of any quarter of the world, and share — without exertion or even trouble — in their prospective fruits and advantages.

Or he could decide to couple the security of his fortunes with the good faith of the townspeople of any substantial municipality in any continent that fancy or information might recommend.

He could secure forthwith, if he wished it, cheap and comfortable means of transit to any country or climate without passport or other formality.

He could dispatch his servant to the neighbouring office of a bank for such supply of the precious metals as might seem convenient — and could then proceed abroad to foreign quarters, without knowledge of their religion, language or customs, bearing coined wealth upon his person.

He would consider himself greatly aggrieved and much surprised at the least interference.

But most important of all, he regarded this state of affairs as normal, certain and permanent — except in the direction of further improvement.

Any deviation from it would be seen as aberrant, scandalous and avoidable.

The projects and politics of militarism and imperialism, of racial and cultural rivalries, of monopolies, restrictions and exclusion, which were to play the serpent to this paradise, were little more than the amusements of his daily newspaper.

They appeared to exercise almost no influence at all on the ordinary course of social and economic life, the internationalization of which was nearly complete in practice.

The Economic Consequences of the Peace — J.M. Keynes

for no reason whatsoever

Bees? BEES!

Turns out that a great way of spending a Sunday morning is coming up with different games for coloured wooden bees. So far we have:

• Bee fight for their correct colours in the hive
• Drop bees from big height and see if they get into a hive hole (quite rare)
• Divebombing bees
• Bee Swarm On Daddy
• Bees Kiss

Bees: surprisingly versatile!

Off to Atlanta next week, which means updates here for a week or two might be sparse, but you did get the first tech blog of the year on Friday. I have a vague plan this year for making my blogging setup work remotely again, but part of that involves upgrading Hugo…and that has never really gone well for me in the past, so I am putting it off as a ‘later in the year’ thing.

UPDATE: so, I saw this on Monday

I am more and more convinced that right-wingers are like this because they surround themselves with sociopaths to the point they don't know when they're broken inside because they're the same as the people they are regularly around.

Traumatized people traumatizing their kids.

[image or embed]

— Dave, Aspiring Peasant (@aspiringpeasant.com) January 5, 2026 at 4:01 PM

I remain astounded at these people, who seem to have no joy in their heart at all. You can’t tell me that there was a better way to spend that hour on Sunday than with Maeryn and her wooden bees. What would I be doing instead? What on earth would be better than watching her laugh when I did a bee divebombing run into her tummy, or when she playfully put all the bees in the wrong coloured holes to get a rise out me? I don’t want to be part of their RETVRN world.

A Christmas LEAF

One of the more crazy things I did after Thanksgiving (and I don’t believe the ER visit had much to do with it, but you can read a midlife crisis into things if you want) is buy a DGX Spark¹. Now, my rationalisation of this expense was that my existing deep learning rig is almost nine years old and showing its age (while the graphics card in it is three years old and can still be flogged on eBay for the exact price I bought it in 2023, and might even go up in value in 2026), plus I’m assuming that the big influx in RAM prices means that it is not likely to get any cheaper in the next 24 months. And 128Gb VRAM would be more than I can get in a single A100 deployment at work.

Anyway, it turned up, it has been set up, and it has been sitting around waiting for a project. Hello, Christmas!

Over the past…five? Six? years, a standard pattern for training embedding models has emerged:

1. You get a pretrained model, or train your own model from scratch to be a general language model
2. You then do a ‘weakly-supervised’ fine-tune on top of that model, where you have a large corpus of web data and you basically tune on ‘title / first x tokens of text’
3. A much smaller, highly-supervised fine-tuning pass on high-quality datasets like MSMARCO.

I absolutely despise step 2. It requires huge amounts of data, most of which is extremely low quality and quite often not obviously semantically related, and also needs significant infrastructure to train at the batch sizes required to make it passable. With the improvements in datasets over the past couple of years (things like fineweb-edu), I have been convinced there’s a better way to do it. Something that you could pull off on a single A100…or a DGX Spark.

The first little inkling of an idea came with the Stella/Jasper paper that appeared over the New Year of 2024/5, where a much larger teacher model was used to orient the vector space of a smaller model. At the time, I saw it as a vindication of my approach of using a projection layer to make images searchable from a text-only embedding model, but I also felt that more could be done with the idea. Other things got in the way, I filed it away in my head for the rest of the year, and it wasn’t until I saw the LEAF paper a couple of months ago that it popped back out and I got excited again.

The general idea of both papers is this: we take a lot of data (good data, hopefully), and vectorize it with a very strong teacher model. Maybe it outputs 768-dimensional embeddings. We then have a student model, which is likely to be smaller and outputs a lower dimensionality of embedding, say 256. The student is a standard pre-trained model, which means it can probably produce text, but is absolutely rubbish at tackling embedding similarity. We add a small projection network to the student, which can be as simple as a single linear layer expanding the 256 embedding to 768 dimensions. We then take the embeddings and the texts from the first step of the process and train the student model (plus its projection layer) to match the vectors obtained by the teacher. So if “crazy golf” is vectorized by the teacher to a 768-dimensional vector of [1.0, 1.0,…], we’d keep updating our student model until it said something very similar.

Now, the trick is that the amount of data needed to train this student is much, much smaller than you’d traditionally do with the “fire the entire of the web at the model and hope it works” weakly supervised step that most embedding models are trained with, and you don’t need pairs. Just text. Having said that, my attempts to replicate the LEAF paper met with mixed results. It was definitely improving the ability of the student model, but not to the extent that the paper promised.

But, but…what if we take LEAF and use it to bypass step 2, but still run the supervised pass in step 3? That would likely allow us to get a half-decent model in under 24 hours, if we use some good quality datasets, a reasonable teacher, and…say, had access to a DGX Spark.

The Christmas project in full:

• Vectorize a large chunk of the fineweb with snowflake-arctic-embed-m-v2.0 (a shoutout to Stéphan Tulkens here, for doing a bunch of extraction work that I just re-vectorized), along with basic vocab and word definition vectors that LEAF recommended. Note that I am not doing the full LEAF mix here, as they pull in a lot of the supervised datasets that I will be using in Step 3 and I want to actually fine-tune on them properly.

• Use ettin-encoder-17m as the base model - a ModernBERT-based model that is only 17m parameters, but has a decent context length, and is a very, very strong performing model in its size class

• We’ll use nanoBEIR’s ndcg metric as our benchmark — to start with, how well does the base model work for retrieval just by itself? (spoilers: it will be bobbins)

• Then, run LEAF training on the model using the datasets above - replacing Step 2 in the traditional embedding model training run, and throwing away the projection layer afterwards (based on an observation in this paper that you still get most of the LEAF benefits even without the projection layer after training)²

• Run nanoBEIR on base+LEAF

• Perform Step 3 with a 1m sample of retrieval datasets from F2LLM, using 10 hard negatives per query.³

• Oh, and we’ll also try just training the base on the F2LLM dataset directly, skipping Step 2 altogether, just for a comparison

• The question: can this simple model training recipe out-perform the nanoBEIR benchmark results of the stalwart all-MiniLM-L6-v2?

So, between Christmas and New Year, myself and Claude did some experimenting. I did have some troubles getting a good base Docker image for the Spark, but once I had sorted that⁴, we got some good training runs in.

Victory! Well, just, but at the end of the day, Brian, a win is a win. Now back to the studio…

I think the experiment shows that there really is something to LEAF. The base model is utterly useless by itself, and while training on the curated datasets helps considerably, it is still lagging behind the other variants. And there is still something to be said for the final supervised fine-tuning run. It provides a 10% uplift to the LEAF-only model (and this tracks with my experiments in October when throwing the supervised datasets into the LEAF mix).

Of course, beating all-MiniLM-L6-v2 isn’t all that these days. One of my favourite small models, snowflake-arctic-embed-xs, scores 0.61 on nanoBEIR. But it has 22m parameters versus my model’s 17m, and I’ve barely done any training - 4 million data points, whereas most embedding models break 200m without a sweat.⁵ Follow ups planned for 2026 include: better teachers, more LEAF data, and a look at those hard negatives in F2LLM⁶. But I’m quite pleased at this first experiment run on the Spark.

Goodbye To All That (2025 Edition)

I celebrated the last day of 2025 in the most American way possible: I drove to the local recycling centre with a car full of cardboard in my pyjamas. I am not exaggerating here, either. I literally took off my dressing gown, put on shoes and a coat, and got in the care. Maybe it’s a symbol of 2025 in general.

this is fine

The day before, I went on another traditional American jaunt: lunch at a fast-food restaurant as a little treat. That was…a touch depressing. I don’t go very much these days, so I’m as much to blame as anybody else, but I was the only person in the entire dining area; it was basically just a ghost kitchen with a take-away window and mobile app pickup section. If the numbers are correct that around ~75% of all fast-food meals are now eaten off-premises…at what point do they stop bothering? And why does it make me a little sad except for the weird nostalgia associated with it?

Anyway, the Liminal Week is over, and the New Year is about to begin. Do I have resolutions? No, I long gave up on those, but a few odds and ends. The usual things of ‘exercise a bit more, and eat less terrible lunches’, along with trying to keep my end up data science experiments and whatnot. But mainly, it’s “do the best I can, watch Maeryn grow into being 3 and more, spend a little less, and have as much fun as possible with everybody I know.” Trite, sure, but there are worse ways to be.

voting the bastards out is implied in every post like the destruction of Carthage

Books of 2025

Book list for 2025 is up¹, and a few things stick out:

• Not a lot of non-fiction this year, aside from Bowie and trains
• I did indeed power through the entire Parker series somewhat quickly
• There was a lot of crime. And book series…
• The Woman Who Laughed had an odd surprise in the acknowledgments section
• Love for Lydia was probably the most horrific read of the year, and yes, that includes Black Flame and the Derek Raymond books. Which is not exactly what I expected from the writer of The Darling Buds of May
• I really enjoy how Tessa Hadley writes, but it means I will likely finish everything she’s written early in 2026…
• It is important to re-read the Red Riding Quartet as December breaks
• People are right about GBH, but Ted Lewis also wrote Boldt and I can’t quite bridge the quality gap between them
• (I’m running out of Joel Lane too)
• Mr. Lonely was…not entirely what I was expecting from Eric Morecambe.
• Finally, I run hot and cold on Mick Herron. I think Slow Horses is a slightly more irony-pilled version of Tom Clancy, but I do appreciate the lengths he goes in his writing to misdirect the reader. Also, the Zoë Boehm books are basically set in my Oxford, so when he writes “she got off the bus at Borders”, I know exactly where she is…and that it’s not quite there anymore (but you can get a sandwich at Sainsbury’s instead).

One thing not reflected above is that almost everything is digital. I have a pile of physical books that I need to get through, but making time for those is a trifle harder.

End of Term 2025

It feels like very bad form to post an entry saying “I went to the ER but every thing is fine”…and then not post for over two weeks. It was not intentional, I assure you. I had an entry all plotted out in my head last week, about my first crown, giving the dentist tips on which chocolate couverture to use in her baking before she performed some impressive violence on my back teeth and jaw…and I remembered on Saturday that I never actually wrote it. Which seems to sum up December so far: vague plans that are abandoned almost as soon as they’re formed.

But, family is all here, Maeryn spending all weekend in a constant burst of excitement over Christmas and having GRANDDAD¹ in the house. Also, her new bed, which should last her for a fair few years and stops me from saying “yes, she sleeps on a mattress on the floor…wait, I’m being told I have to refer to it as a Montessori Floor Bed”. Christmas dessert (and other savoury things, I guess) planning and making begins soon…

There will be more posting before the end of the year. I have the 2025 book list to post (this year featuring a script ported from Ruby to Python using Claude Code), and maybe a little tech thing, if it pans out. But otherwise, Happy Christmas!

Thanksgiving: ER Edition

Last Sunday, I decided¹ that the best way to end the holiday weekend was a visit to the ER. It just caps the event off, I think? Anyway, I spent much of Saturday night and Sunday morning believing I was having a heart attack. So, off to the ER, we went…and I have to say, that if you can pick the time to go, Sunday after Thanksgiving seems like an amazing day for it. There was literally nobody else in the reception when I walked in; I was seen within 5 minutes and I walked out just under three hours later, and most of that was waiting for test results that showed that everything was pretty fine and there was no evidence of a heart attack. We all settled on the idea that it was a bad acid reflux flare, which can often present similar symptoms. But it feels like something you get checked out, just in case.

I’d like to say that the week improved from there…and I guess it did to the extent that I did not feel like my heart was about to kick off, but the entire week at work basically had me forming a David-Peace-during-Red-Riding-Quartet monologue in my head as one issue after another piled on top of use. 1st December, 2025. HATE WEEK. Hopefully next week is a touch calmer.

Also, it turns out that my family arrives in under two weeks, and while I understand the linear motion of time, I do not quite accept the fact that I have less than 14 days to get the place ready for them.