You Worked Earlier!!

A week in research adventures:

• Spend a fortnight prior with a paper stuck in your head. There has to be a better way to do good thing
• Take a weekend and fail. But you know it’s in there somewhere
• Carve out time during the week and on a Thursday afternoon, eureka! you get an example working and finally understand the mysteries of the vLLM KV Cache
• Thursday evening: plan out the paper in your head. Oh, this is going to be great!
• Friday: construct a quick eval
• Friday mid-morning. Yeah, well it worked for that example, didn’t it, sucker boy?
• Friday afternoon: deep sigh

I still think I can work on the problem and get something marginally useful, but it’s a bit of a rollercoaster. And reminds me why my good friend Sava spends so much time on his evaluations first. I had tried a few variations of the initial sample, sure, but not sending 5000 different ones through the model and I paid a high price for my hubris.

(I still haven’t given up, mind you. Because it would be quite nice if I can find a way of making it work)

How did the year just…run out? I’m in Chicago this week for an all-hands meeting at New Company (which currently is a lot easier to organize than at Old Company), and the the next week is Thanksgiving, and then it’s December! I don’t even have enough time to put together a proper dessert menu, let alone make it…though that won’t stop me trying, obviously.