Andrej Karpathy sees a widening gap in how people see LLM capabilities. The general public has mostly not interacted with these systems. When they have, it looks like a derpy chatbot. The majority of professionals still see only a modest uplift. And a small sliver of professionals are experiencing the vertigo of the curve going vertical. And it is all happening at the same time.Many of my colleagues are living in a world where LLM capabilities are stunning, but it’s easy to forget that’s not what most people experience. One thing I do is use free-tier LLM chatbots, and I’m still finding that while most of the time they are useful, their unreliability undermines much of that usefulness. Yes, you can argue that I’m not using frontier tools or a forceful enough harness, but that’s true of most regular folks too. To understand their perception, it’s handy to experience it. ❄ ❄ ❄ ❄ ❄That observation particularly resonates with me today, as I’ve just spent the weekend at Foocamp, an open space event run by O’Reilly. It was full of those who are experiencing that vertigo of LLM capabilities, so it felt very much like being in a bubble of those on the bleeding edge. While it’s worth remembering that that’s not a typical view of what’s happening, it’s good to see what those explorers are experiencing. (As I talk about this, I’m applying the Chatham House rule, if anyone wants attribution, let me know.)A couple of folks talked about doing immersion courses to help teach developers how to do agentic engineering. The idea is to get people out of their usual work environment, and spend one or two weeks with experienced coaches, learning how to use these tools well. Such things remind me very much of the Extreme Programming immersion classes I was involved with at the turn of the century. As with that, the experience of building something is important to understand such a new style of work.Unsurprisingly the coaches observed that the biggest impact wasn’t technical, but emotional. New tools are just new tools, but these tools change basic assumptions of what it means to be a software developer, which for many people challenges their identity. One coach said how his main job was to take such people out for a walk to a coffee shop, so they could process what’s happening. (This reminds me of a participant of the Extreme Programming immersion who was so horrified by the use of a non-locking version-control system (CVS) that he walked out of the class on the first day.) ❄ ❄I’m wary of talking about themes to an event where I attend less than one-tenth of the sessions, but a topic I kept coming across was how large enterprises were embracing this technology - or rather not. One participant used a lovely metaphor: corporations have acquired a Swiss-army knife, but are only using the toothpick. I see plenty of decent reasons for this: it’s one thing for a startup or individual to use a hot technology, but there’s a big difference between a website going down and your factories grinding to a halt. Big organizations involve workflows involving lots of people that have built up over many years - they don’t change easily. Security concerns abound, the lethal trifecta still lurks.In particular good use of AI in enterprises relies on good data, and many such organizations don’t have their data in a decent shape. The path to getting agent-ready data isn’t trivial and it’s hard to see how we can get the knife and saw out until enterprises are much further down that path. One thing that struck me is that there was little conversation about this at Foocamp - a sign of its visibility among the people there. ❄ ❄I very much enjoyed Foocamp, but events like this are always a mix of excitement and dread. I knew very few of the people there, so it was a great opportunity to get out of my own social bubble. But I also find it nearly impossible to walk up to people I don’t know (or know slightly) and impose my company on them. The great thing about unconferences is that the nature of the sessions make it plausible for someone like me to engage in conversation during them, and begin some tentative links. Also I thank the many folks there who were friendly to me as I casually walked around trying to hide how much I was panicking. ❄ ❄ ❄ ❄ ❄Laurie Voss says today is 1997 On Saturday, Apollo’s chief economist Torsten Slok published No Signs of AI Payoff Outside the Tech Sector. It’s short and it’s blunt: outside of 7 major tech companies, there is no data yet to support a productivity gain that justifies all the money pouring into tech. My position is that he’s half right: are valuations too high? Absolutely. We’re in a bubble, and it’s absolutely going to pop. But does that mean investing in AI is a waste of money? Absolutely not, and using this early data to try and decide that question would be like trying to decide in 1997 whether the Internet was going to pay off. The data just isn’t there yet, because these things take time.I’m not as sure as he is that we can peg today to 1997 rather than 1999, but I agree that this boom bears a lot of resemblance to the dotcom boom (although I think this one is bigger). While many dotcom companies crashed (I always think of pets.com and webvan) he’s right to point out that anyone who invested in Amazon in 1997 did very well. Where might those good investments be this time? if you can build something that proves you got the answer, AI can get the answer much faster than you can, and there are real productivity gains to be had doing that generally.As we think about this, remember that investment isn’t just buying shares, but also where we choose to invest that precious resource of our time and energy. ❄ ❄ ❄ ❄ ❄Adam Tornhill asks Were All Those Years of Coding Wasted? For me, agentic coding is enjoyable because it lets me take on larger parts of a problem than I would have been able to master manually. Infrastructure updates are no longer a blocker. And I don’t need to spend time learning about a build system I never cared about. My personal goal is to get better at working with higher-level abstractions, and accept that writing code is likely to be a dying skill. My bet on the future is that our problem solving skills, logical mindset, and understanding of systems will be at least as rewarding. In the end, we only need to challenge our identity if we mistake programming for writing code. AI finally allows us to separate the two. And that’s where all those years of experience transfer the best: knowing not only what to build but how to build it well. ❄ ❄Tim Bray looks at this from another direction For me (and I’m not alone), the biggest issue isn’t what GenAI can and can’t do, it’s the ethical and economic discomforts. This technology is being peddled by terrible people who are rich and whose goal is to get richer. It’s bought and deployed by terrible people who want to slash their payrolls by throwing thought workers onto the street. It facilitates the oncoming climate catastrophe and devalues human creative effort. ❄ ❄ ❄ ❄ ❄Annie Vella observes that the change we’re seeing in software development follows a way of thinking about automation that goes back to the 1960’s as the distinction between direct and supervisory control. The main difference between direct and supervisory control is that in the latter, there’s a computer that closes the loop instead of a human operator. Before AI, writing software was much more like direct control - we closed the loop ourselves. We interpreted requirements, made changes to the code, received feedback and acted on it, over and over. Now, coding agents can increasingly close that loop themselves - writing code, running tests and fixing their own mistakes - while we move towards supervisory control: telling the agent what we want, keeping an eye on how it’s going and taking over when we need to.She outlines various ideas that this intellectual thread developed. The five-level model developed by Endsley and Kiris seems apposite: Level Human System In AI-assisted software development 1. None Decide, act - No AI assistance 2. Decision support Decide, act Suggest The early days of AI coding assistants’ autocomplete 3. Consensual AI Concur Decide, act AI coding agents where you must approve the actions they take 4. Monitored AI Veto Decide, act Auto-mode, with the ability to veto the coding agent’s actions 5. Full automation - Decide, act Full auto-mode, such as might be used in a dark software factory Vella cautions that this comes with an important caveat: There is one notable difference between the automation described by this prior research and the type of automation that AI agents give us. Both sets of levels assume that the computer’s account of what it did is accurate and honest. As we all know by now, what an AI agent says it did and what it actually did may not be the same thing. Still, it’s automation, and it still requires supervision - arguably more, since we can’t simply take its word for it.Vella’s conclusion from all this is that: “our next step must be to start redesigning the job and the system, in tandem”. I agree with this, and the Foocampers’ immersion courses I mentioned earlier are part of that exercise. The tricky thing is that we don’t yet know what this new job design looks like, but I think Vella is on the right path when she says: What if we stopped thinking of the SDLC as a process we follow, and instead started thinking of it as a control system we build and supervise? ❄ ❄ ❄ ❄ ❄A while ago I mentioned how much I enjoyed Eric Evans’s keynote at DDD Europe. Now the video is available so those who weren’t in Antwerp can watch it.He opens by looking at our future, saying that rather than fruitlessly trying to predict it, we should instead explore it with “curiosity, fun, and wonder”. He’s been doing this, digging deeper into the theory and fundamentals that underpin AI.He goes into detail on his side project: the Domain Language Surveyor. The idea is to be able to examine different repositories, and analyze the vocabulary used inside those repositories to see if they are related bounded contexts. To do this he built a pipeline of tools, illustrating how a useful system blends both computational and inferential components. He then used this pipeline to analyze the vocabulary of some open-source healthcare codebases, to see how their bounded contexts related. ❄ ❄ ❄ ❄ ❄A topic that regularly comes up when I talk to companies outside the Foocamp world, is the cost of tokens, and the concern about how that cost is going to increase in the future. We’ve been worrying about that at Thoughtworks, and my colleague Andy Nolan describes our token sourcing strategy, which we forecast will reduce our token costs in 2028 from $16-40M down to $6.8-1M. The strategy rents frontier models for the work that needs frontier-level reasoning. It runs open-weight models, on capacity we choose, for the work that carries our IP or runs at a volume where owning the capacity costs less.The strategy involves looking at each LLM workload, and classifying it on two axes: domain differentiation and security & sovereignty. We then place each workload on a nine cell grid. The nine cells run from the cheapest capable API, swapped as prices change, to a fine-tuned open model on our own infrastructure, with full control and full upkeep. For a given workload, the highest score on a given axis determines the right position in the grid. There is no single right place for all workloads. Each organization ends up with its own distribution and mix of workloads across the grid.