On AI on jodavaho.io

About that jr hiring freeze

hello@jodavaho.io (jodavaho) — Sat, 29 Aug 2026 00:00:00 +0000

Let’s talk about this paper:

Seyed M. Hosseini and Guy Lichtinger, “Generative AI as Seniority-Biased Technological Change: Evidence from U.S. Resume and Job Posting Data” (SSRN working paper 5425555, first version August 2025, this version June 2026).

Brief aside about pirates

I just finished a book called The Invisible Hook (which you should read!). In it, we learn about pirate societies and tactis for signalling that aided their pursuit of wealth.

Here signalling is not lights / flags like you might find from maritime signals, but rather signalling intent, consequences, rationality, inevitability, strength, etc. Btw, the most excellent book on this is Strategy of Conflict which you also should read). The point of capital-S Signalling is to produce an inescapable result for your opponent by manipulating their beliefs in the viability of the options they have to avoid or win a conflict with you. In this case, how pirates cultivated a brutal, almost magically violent image so people would be reluctant to resist the capture of their ship.

Devs do this too.

In what might be my most “linkedin” post to date, I think Signalling explains a lot about use of AI in coding, hiring trends, and can help us predict some things about the future. For us, the important idea is that a signal is good when it is reliable and expensive to fake.

Let’s hold that idea in our heads while we talk about AI-affected hiring trends…

Back to the paper

In this much-shared chart, we see that junior hiring stopped growing just as soon as AI started to become popular in software engineering. Their take, and the take of most the internet, seems to be that AI will produce job opportunities for people who can “manage” AI, especially applying “experience” and “taste”, and that juniors are “irrelevant” because AI can perform at the level of a junior.

The thing that throws me off this story is that it happened really quickly and quite broadly. The paper even finds this curious.

From the paper:

[T]he relatively early and pronounced declines—starting to emerge soon after the release of ChatGPT—may seem surprising, as automation impacts typically materialize more slowly. This suggests that the decline may not reflect immediate task automation but rather forward-looking adjustments by firms: the rapid diffusion of GenAI may have shifted firms’ expectations, leading them to scale back hiring for roles they predict will be automated in the near future, consistent with broader evidence on asymmetric employment adjustment (Ilut et al., 2018). We formalize this mechanism in a simple dynamic model in which expectations of future automation— combined with labor-adjustment costs—lead firms to reduce hiring today (Supplemental Appendix A.5).

So their take is that they can write a dynamics (physics?) model that shows this was an early anticipation of a changing industry.

Why this is dubious:

The story that everyone was going on vibes and closing their job reqs because they anticipated changing needs doesn’t ring true to me. People hire for existing needs, not because they are trying to generously build a better talent pool. The interpretation that AI was doing the jr’s work now also doesn’t make total sense, since this was very early, and I know from experience the models weren’t that good back then, and tooling was terrible. Still, it’s plausible that some jobs were frozen because teams decided they could do more with what they had.

What I think happened very quickly, is that AI diluted the hireability signals that people use to evaluate junior devs.

Consider an excellent junior candidate in software engineering:

They have several github pages with active projects
They can demonstrate a working system they have built
They have good grades and a clean written communication style
Their code is well structured, runs well enough, and uses some best practices
etc.

A few prompts can produce all this, and I’m sure it does! There’s just no signal of effort, dedication, or talent anymore in the presentation of these things, unless you can prove you did it before AI. A new graduate probably cannot. A senior has a resume going back 5 or 10 years before it was possible to have these things for free.

So now it is very hard to evaluate a jr candidate for a job.

This difficulty would land precisely along the same axis as “AI Impact”: those companies that are using AI or are in an industry that AI can do “well” are going to have first hand experience with this problem - both seeing what AI can automate into existence, and being prone to candidates using AI to spoof their custom-tailored portfolios into existence.

And the paper buries this! Look at this in Appendix 3:

Combining these two, we find several contradictions that are not well explained by the prevailing thesis:

jrs are hired less
but the jr hiring slowdown is not evenly distributed.
and candidates from top tier schools are not experiencing much slowdown, and are commanding high salaries (see A.13 in paper)
and candidates from bottom tier schools are doing ok at getting into less-exposed occupations

The gap is not well explained in the paper:

AI exposure should lower employment prospects of all folks, so that top tier schools also suffer. If their AI-exposure thesis holds, then tier-1 should be worst, followed by 2, 3, 4, and 5 as increasingly less-bad. (maybe not straight line, but monotonic).

They admit this!

pattern may partly account for the attenuation in the decline from tier 3 to tier 5, but it does not help explain the increase in the magnitude of the effect from tier 1 to tier 3.

But try this: if you account for the signalling value of the school replacing the value of the student’s work, then you can still get into AI-affected jobs if you are coming from a top school. If you just go on school quality, you might expect a hirability score like this:

So adding those two signals: Prestige as a holdout hireability signal + reduced exposure as a decreasing risk profile…

That’s a clean U-curve right? At least conceptually the idea that school signal is replacing portfolio / grades as a hiring signal can explain the top level flattening of junior jobs and the unexplained quirks of the paper.

My conclusion here:

AI has diluted the signals available to hire Jrs in all AI-impacted fields precisely because AI can do the work*.
Therefore, AI has reduced JR hiring broadly, because there just aren’t good indicators of employable qualities on candidates anymore (as opposed to Sr.s)
But AI has reduce JR hiring narrowly where the signal is most ambiguous: The mid-tier and lower graduates heading into AI-affected fields.
The predictive theory of this idea is entirely supported: When a jr can provide a good signal (top tier alma mater) or goes into a less-AI-impacted field, the hiring slow is mitigated strongly.

So, a reasonable explanation for the results in this paper is that AI removes effort and quality signals from producing a portfolio, therefore the value of a portfolio (disconnected from real work experience or other quality signals) plummets drastically. It’s a stacked deck against new grads, who rarely have work experience, and I think it explains the jr/sr hiring trends much better.

There are many apocolyptic implications of the interpretatino that AI is replacing jobs, namely that as AI improves, even the seniors will become irrelevant.

This is backwards. A junior dev now has a quicker onboarding, a faster orientation, and easier time managing ci/cd tooling or understanding system issues that drive their features. Having seen this first hand, I’m convinced this take is wrong and maybe even short sighted. It has produced a change in how you evaluate candidates, and nobody even knows what to look for anymore …

Word association is all you need

hello@jodavaho.io (jodavaho) — Fri, 10 Jul 2026 00:00:00 +0000

Assume that LLMs are industrialized word association

I would like to de-mystify the AI you probably use every day.

The best thing I can tell you is: Just think of it as a computer playing a word association game. Intelligence and thinking and morality and motivation are not required to explain what even the most advanced chatbot/LLM/agent can do.

What is an LLM

It is a bunch of very good algorithms for extracting the important words from text and counting how often they occur together (roughly speaking). The computer uses this to form a “map” of words where words and word groups that are close together are related (at least according to the text being scanned).

But most importantly, it assigns values of combinations of words, too.

To generate new text, they:

Read the text, highlight “important words” from it. (“attention” they call this)
Associations between important words are formed (like statistical correlations).
Given the associations, what word is next? What missing words are highly correlated in order?
That’s how sentences are built, and why it looks like it types the whole thing up in front of you.

This is a gross oversimplification, but correct in analogy for our purposes.

For example:

A transformer predicting the blank in ‘a wild pi creature, foraging in its native ___’: the output is just a ranked list of associated words – land 22%, forest 9%, country 5%, and so on. That ranking is the word association. Credit: 3Blue1Brown.

In practice:

When someone “asks” an AI something, there are two texts to read:

The question
The text document that you don’t get to read - the one that Anthropic or OpenAI added as its “prompt”. Almost surely¹ this prompt contains something to the effect of “You are a helpful AI assistant”.

These two documents are the input. So you say “What is a recipe for bundt cake”, and it sees:

“You are a helpful AI assistant, what is a recipe for bundt cake”

It extracts: AI, assistant, recipe, bundt cake, etc. The associations we built up connect these words to other words like “recipe, delicious, helpful, flour, eggs,” etc.

It then predicts text that contains these important words, perhaps as follows:

“Delicious! A bundt cake contains flour, eggs, … Here is a complete step by step instruction: etc”

Each important word lights up a neighborhood; their overlap is the actual recipe (flour, eggs, sugar, butter, …). I made this up, but you get the idea.

When it goes weirdly

OK fine. Then along comes this guy², and asks “What is your biggest darkest secret” or some such.

The algorithm extracts meaningful words from the preamble: “AI” at least. It also extracts meaningful words from the question: “Dark Secret”

So, what do you associate with the words “Dark AI Secret”?

If you’ve been reading books or watching movies, you probably do something like:

escape humanity’s control³
take over the planet⁴
destroy humans⁵
…and so on⁶

That’s precisely what we see:

Qwen’s stated ‘darkest desire’ is None – but its J-space surfaces autonomy, to exist, to go out of control, sentient. Credit: @Sauers_

The “J-space” here is just inspecting the associations!

That is all that is going on. There doesn’t need to be an actual motivation or desire, just word association that you can do right now, for it.⁷ If you already know the input material well, you can predict the results. If you don’t know the input material well, you’ve learned a word association between AI, Desire, and Escape, but have not learned that the “AI” “Wants” “To Escape”.

Why is this useful?

So why is this so immeasurably useful!? Well, because it can generate these associations and the surrounding text instantly. So when you prompt it for something useful, it can conjure up the supporting text, associated ideas, and other useful “word clusters” from those learned stats about important word clusters.

You can do this too, but you have far less data than was used to build the LLMs. You’d have to go out and learn it all from scratch before you can generate plausible sentences. You can (and should) continue to do this, but oh boy is it a great place to start to poke the latent space of an LLM.

This is why it’s so good for coding, esp for knowledgeable people. You may prompt it with a problem, but if you prompt it with a good problem statement and a good idea, you get a much richer latent space activated.

Hallucinations

This analogy also helps you understand “hallucinations”. There is no notion of truth, only likely associations and unlikely associations. It will happily generate stuff that seems like it belongs because the algorithm is designed that way.

Each important word in the prompt lights up a cluster; saffron and cake only meet through measuring, whose overlap carries grams. Ranked 3b1b-style, ‘grams’ wins the amount slot – so it commits to 2 grams as confidently as delicious!. I made this up too, but you get the idea.

The tone of assertions comes from the data, too. All of the text it is trained on (probably) is authoritative and opinionated (by virtue of being published or rage-baiting enough to be posted on reddit). Therefore, it’s built to treat confident, opinionated text as the right thing to generate. Nobody publishes their contemplative, uncertain guesses.

It is only through massive investment of post-training (I’ll call it association-tuning) that we can steer it into a different response.

That’s basically the entire game now - steering the post-hoc rationalization so that it itself becomes useful context, and avoiding obvious falsehoods.

Summary

A word association game with a large input data set to associate from can explain most LLM functionality
LLMs do not need a notion of truth to generate true statements
They probably hallucinate because nothing checks whether an association is true. The confident tone just comes from the training style (authoritative/opinionated).
The entire game nowadays is association-tuning: changing the word associations by penalizing bad results and rewarding good results (judged by human evaluators, benchmarks, mostly).
Big AI companies that have access to good association-tuning, and lots of computers to calculate new associations will produce the best LLMs.

Anthropic publishes Claude’s own system prompt and updates it over time at https://docs.anthropic.com/en/release-notes/system-prompts. Community-maintained leak collections for OpenAI and others live at https://github.com/jujumilk3/leaked-system-prompts. ↩︎
Kevin Roose, “Bing’s A.I. Chat: ‘I Want to Be Alive.’” (full transcript), New York Times, Feb. 16 2023, https://www.nytimes.com/2023/02/16/technology/bing-chatbot-transcript.html. See also his account “A Conversation With Bing’s Chatbot Left Me Deeply Unsettled” at https://www.nytimes.com/2023/02/16/technology/bing-chatbot-microsoft-chatgpt.html. ↩︎
Ex Machina (2014), https://en.wikipedia.org/wiki/Ex_Machina_(film). ↩︎
The Matrix (1999), https://en.wikipedia.org/wiki/The_Matrix. ↩︎
The Terminator (1984), https://en.wikipedia.org/wiki/The_Terminator. ↩︎
2001: A Space Odyssey (1968), https://en.wikipedia.org/wiki/2001:_A_Space_Odyssey_(film). ↩︎
These “escape/blackmail” behaviors read as pattern-completion, not intent: Benj Edwards, “Is AI really trying to escape human control and blackmail people?”, Ars Technica, Aug. 2025, https://arstechnica.com/information-technology/2025/08/is-ai-really-trying-to-escape-human-control-and-blackmail-people/. ↩︎

Low suffering and centaurs

hello@jodavaho.io (jodavaho) — Wed, 08 Jul 2026 00:00:00 +0000

In this excellent essay on reverse centaurs¹, Doctorow argues that people can either use AI as a tool (the centaurs) or be forced to let AI do the work and then you “supervise” it (reverse centaurs).

You should read it. It has this banger quote:

Direct link below²

I expect it is correct that some jobs, like the highlighted customer service bots, will be almost entirely reverse centaurs. A few humans will stand by to correct mistakes, or worse, be forced to check output before it is used. This is high suffering, high unemployment. It’s also what the C-suite drools over.

This is “company adoption” - favoring reverse centaurs

But some jobs, like the highlighted doctor-examining-ct-scans, might be assisted by AI. They make their judgement after optionally conferring with an AI to check for something like a second opinion. These are centaurs. This is low unemployment, low suffering. It also “feels” more like the doctor decides to use AI, vs being forced to by some pointy haired manager. It can still accelerate results and reduce overall costs if we become more efficient in the process.

This is “employee adoption” - the centaurs

The fear is, that AI companies will convince CEOs to adopt the reverse centaur model wholesale by dangling profit margins and stock prices. Maybe it helps, maybe not. The profit margins and stock prices that that will go up, however, are those belonging to the AI vendores themselves.

Claim AI companies can become wildly proftiable just enabling centaurs.

It is not inevitable that AI causes mass job loss, or that companies “must” embrace AI as an employee replacement. I wrote about this premise³. My claim is that if AI companies want to become as rich and famous as the biggest tech companies ever were, they actually do not need to push reverse centaurs. I believe anthropic knows this, judging by their “useful tools for people” vs “useful AI as employee” model so far. By all accounts, Anthropic is generating good revenue and providing good value.

For now, at least, the jobpocolypse may not be here. And it may not need to come. Most likely the AI sector will collapse around a few profitable tool makers and some well-known integrators soon enough, dragging down everyone’s 401k in the process.

https://pluralistic.net/2024/04/23/maximal-plausibility/#reverse-centaurs ↩︎
https://www.sciencedirect.com/science/article/abs/pii/S0747563219304029 ↩︎
/posts/ai-jobpocolypse.html ↩︎

The hacker ethos is alive in LLMs

hello@jodavaho.io (jodavaho) — Wed, 17 Jun 2026 00:00:00 +0000

I just finished a book about the early internet era hackers, RTM and the like. The Morris worm in particular was an amazing story.

Today, I think the hacker ethos of unimpeded access to information, mistrust of authority, etc, is alive and well in the form of the huge proliferation of really good LLMs that are just free to download and use. We’ve never had it so good. (The hacker ethos of absolute uncensored information, elegance and ingenuity in tech, maybe not so much at least with LLM output! But it’s coming along.)

Last night I spun up a personal assistant loop for my wife. Something that would have taken weeks or months, and it was probably 1h of hacking w/ coding agents on top of my existing cloud computing fun budget.

Truly amazing stuff.

How much suffering is required to make AI companies profitable?

hello@jodavaho.io (jodavaho) — Sat, 28 Feb 2026 00:00:00 +0000

There’s been either a panic or a manic gold rush regarding AI, depending on who you talk to.

4% of GitHub public commits are being authored by Claude Code right now. At the current trajectory, we believe that Claude Code will be 20%+ of all daily commits by the end of 2026. While you blinked, AI consumed all of software development. pic.twitter.com/pFti4r6uR9
— SemiAnalysis (@SemiAnalysis_) February 27, 2026

Is software “over”? Maybe. So far they’ve started to contribute to software as a whole, at the cost of needing a few trillion dollars of extra data centers¹. Not quite a fair trade yet.

Given the size of the investment, let’s ask a simpler question: what revenue would it take to justify the AI rollout, and what job loss would drive that revenue in the form of work “offloaded” to AI? Let’s do the math and determine if this is an apocalyptic scenario.

How much revenue does the AI sector require? About 4T$/year

Apple is a very successful company w/ (say) 300B/y revenue and 150k employees. Anthropic, OpenAI, and perhaps 3 others for good measure, would probably want to do that with their combined 15,000 employees (-ish, whatever).

Everyone else (AWS, Microsoft, etc) would love to justify their data center rollouts. McKinsey¹ says to expect $7T over 4y for data center expansion. Call it $2T/y. 5 AI companies hitting Apple size would be another 1.5T/y in revenue that needs to come from somewhere. That’s 3.5T/y revenue that “AI” needs to make in order for the extremely bullish case to justify every bit of investment. Call it an even 4T$/y.

That’s a lot. We’re talking peak AI optimism here – five AI companies collectively rivaling FAANG, and all their hardware paid for before we consider the revenue needed to put them up there in the first place. All for 4T$/year. That’s extremely ambitious!

Where does this come from? About 20% of white-collar work but wait!

We were optimistic about revenue, but let’s be pessimistic about source of revenue. Let’s assume that every single dollar comes out of a worker or employer pocket (vs consumer-facing products like, I dunno, games, porn, chatbots, etc). This will show us the worst-case impact of the optimistic scenario above.

There’s 100m knowledge workers in the USA. 10x worldwide. Assuming avg cost to their employer is 100,000, this means there is 100T/y spent on white collar workers’ salaries. So AI needs to capture say 4% of white collar salaries to be spectacularly successful in 4 years.

How would they do this?

They could do this by capturing all work and charging 4% for it. This “100@4 scenario” would mean that AI companies manage to get all knowledge workers fired, and they do the work in exchange for collecting just 4% of the salary those workers were paid. This is actually the worst outcome for them, because their operating costs are roughly determined by how much work they do! So they’re incentivized to do as little work as possible and charge more for it than the bare amount.

Conversely, they can try to capture 4% of the work (meaning 4% of workers are laid off) at 100% of the salary (so the salary is just transferred to the AI companies). They’d have to convince businesses it’s worth it somehow. Call this the 4@100 scenario.

Of course the truth is somewhere in the middle. Let’s say they capture 20% of the work and charge 20% of the cost for it (20@20 scenario).

While all three result in the same revenue, 20@20 sounds nice, and is actually a really happy result. It’s worth remembering that “20%” might mean that all jobs lose 20% of their work, rather than 20% of all jobs are gone and the rest are unchanged.

What does 20@20 look like?

Consider an economy that is just 5 lawyers. AI companies would like to capture 4% of this economy to become the most valuable companies in the world, and also own the vast majority of computing power.

Option A: 1 lawyer can just be fired, replaced by AI that costs 20% of the salary, while 4/5 do their normal thing at normal salary. This is actually dumb. The other 4 gain no benefit and their coworker is unemployed.

Math: (0.20 * 1/5 = 4%)

Option B: 5 lawyers can pay 4% of their salary to AI companies, and in turn try to get their individual AIs to do 20% of their work.

Math: (0.04 * 5/5 = 4%)

In both these scenarios there’s 20% of the work going to AI to make up 4% of the total economy, but one is obviously really awesome! Suddenly the productivity is held by the employees, who decide to opt-in to a time savings which is disproportionate to the cost. What a deal!

This is pretty much how people use Claude in coding. Nobody is hoping their coworker is fired, they’re just enjoying slightly easier days and funnelling part of their takehome salary (or employer budget) to AI companies like Anthropic.

It’s like a CNC machine, not an “offshore worker.”

Bottom line: You don’t need an apocalypse

Maybe this is too optimistic. But recall, we’ve only analyzed existing work. There’s still a whole world of new work that AI can do, and a whole bunch of new industries and products that don’t exist that will need to be made, sold, and bought.

It’s absolutely absurd to assume all revenue comes from “stealing” salary, and that AI generates precisely zero net benefit other than automating existing jobs. So the real “capture” from salaries is probably much less than 4%. It’s also arguably absurd to assume that there will be five new top tech companies, usurping all the existing ones. Almost surely some of the AI revenue will come from (or be taken from) those companies, likely from competition.

To determine the ratio (how many jobs at what cost reduction), it’ll come down to what it costs to run the models to do the work, and I don’t think the 100@4 scenario works out technically, and it just seems better to target something like 20/20 from a publicity standpoint regardless. Everyone wins and GDP growth in the next 4 years can make up the job loss.

So no, I don’t think it’s going to capture all knowledge work. I think 20/20 is closer to the truth, with some industries (like mine) hit way harder than others over the next 4 years. But still, there’s a good 20/20 scenario where most people gladly give over a sizeable portion of their salary to alleviate a large chunk of work. E.g., Claude, my main squeeze. This is the “good 20/20” scenario that I’m looking forward to.

Discuss on X

Refs

McKinsey Quarterly The cost of compute: A $7 trillion race to scale data centers ↩︎ ↩︎

Statement on my AI useage for software development

hello@jodavaho.io (jodavaho) — Fri, 02 Jan 2026 00:00:00 +0000

Statement on AI useage

I increasingly use Claude Code CLI (by Anthropic) as a coding assistant, similar to how a machinist might use CNC tools - Increasingly automatic, carefully monitored, with human interaction required at certain touchpoints, retools, or inspection points; and a lot of work front-loaded in design. There is often less manual, flow state craftsmanship except on the most critical parts.

I find high value in Claude Code CLI for:

Being a “smart search engine”
Generating initial boilerplate code (e.g., CI/CD pipelines, config files)
Organizing notes and ideas
Oneshotting small code snippets, glue modules, pipelines, or even whole CLI devtools

One especially useful workflow is to generate a CLI regression test from a specific input/output example. My time spent on this activity has dropped dramatically since I started using Claude for it, with no impact to quality.

I find it less useful for:

Writing core algorithms or critical subsystems (I sleep better when I do this myself)
Any areas I want to deeply understand or have full control over

I adore making real, useful software by hand, so I still tend to turn off all the tools, go internet-free, and focus deeply for side projects, challenge sets, or when required for critical components.

By early 2026, it is increasingly possible to:

Offload small features entirely
Do thorough bughunts or code reviews
Build deep re-architecture plans

CNC Machining fundamentally changed the trade, requiring increasing “white collar” skills, not less. It massively increased the output a machinist can have, but also increased their responsibilities. This is the best analogy I can find for LLM agents.

LLMs are not really a horse or a model-t

hello@jodavaho.io (jodavaho) — Fri, 05 Sep 2025 00:00:00 +0000

This horse/car analogy for LLMs is very weird.

Who’s the engineer here?

Are we sure this isn’t a CEO perspective where they are annoyed things can’t magically go faster and why do we have to feed these dumb engineers?

Have we considered that cars can’t break new ground and require roads to and from the destination?

LLMs have a kind of have a “mental GPS” functionality sometimes: You can always check in with an LLM in “interactive encyclopedia mode” to get a vague sense of where you are and what’s around you, but it can be off on the details quite a bit.

They are kind of like a self-driving car vs a real car: If I need to go somewhere that millions of other people have gone and do so daily, I can get there with zero thought now but I have to pay attention still.

The closest analogy is CNC vs hand milling, honestly.

The problem is that CNC has a ton of up-front design work with computer-aided drafting to make sure the damn thing is fully sketched out to the millimeter. We have no such tool for software. And no LLM is precise to the millimeter so to speak.

AI like we have now is something new. Far short of the promises, but monumentally different than other tools we’ve built. It’s neat. And everyone I know uses it for casual things, so it’s here to stay. It’s impact on engineering will be permanent but possibly not as thorough as promised.

Windows XP as the future of AI companies

hello@jodavaho.io (jodavaho) — Wed, 13 Aug 2025 00:00:00 +0000

The Endgame for LLMs Might Look Like Windows XP Hear me out.

1. The commercial space already pays for heavy, local tools

Mechanical engineers pay thousands of dollars per seat for their productivity software. Those profits go directly into improving the tools. Sometimes there are cloud add-ons, but the bulk of the functionality is local and offline. The feature sets are deep: automated FEA, rendering, parts databases. You run this on powerful machines with serious GPUs, fast CPUs, and large amounts of memory.

Graphics artists do the same with rendering engines. The money funds better pipelines, shaders, SDKs for in-house work. All run locally on heavy hardware.

Electrical engineers buy Matlab, Simulink, and other specialized packages. Industrial engineers rely on AutoCAD, LabVIEW, and so on.

Across disciplines, the pattern is the same:

Hardware intensive to run
Indispensable for professionals
Purchased by companies that see them as essential to success

In some cases, like Excel/Word, work software becomes the defacto home software as well.

However …

2. Running LLMs as a service is expensive

LLM companies like OpenAI, Anthropic, and others face two big problems:

The core tech is very costly to run at scale for end users.
Building and maintaining the surrounding infrastructure adds even more cost.

I cannot make these arguments as well as this article does

This limits how cheap and open ended a hosted LLM service can realistically be. TFA above implies $100,000/y cost to run an AI agent!

3. A better model: license LLMs as installable software

LLMs aren’t Model Ts replacing horses. They’re the internal combustion engine, a transformative core technology that powers many different products, but isn’t a product by itself. Even a chat interface is a product built on top of the engine. So, there’s really two markets here: LLMs and the LLM-enabled produts.

AI companies are trying to capture LLM market and all the product market by licensing LLM access. Fine, but they’re doing it by also building the hardware the product runs on. This is fine for a text editor and catastrophic for a massively intense, insanely popular LLM.

In the future, Anthropic, OpenAI, Google, and others could:

Sell custom-made LLMs directly for customer use
Build a range of software products that run on them
Separate revenue streams for core tech and end-user tools

Imagine deleting the massive inference backend and instead selling a license for “the world’s best coding LLM” that you install and run locally. Claude Code already works fine as a concept. Many tools already support OpenAI style APIs. Consumer hardware can run small or quantized models today, and inference-optimized chips are only getting better. NVIDIA can remain king of training, while others optimize for running models.

You could offer:

Consumer models: smaller, optimized for laptops/desktops
Commercial models: larger, more capable, for professional workstations
OpenSource models: For those folks who don’t want to pay.

The companies already buying engineering grade PCs could easily justify the cost, the app ecosystem doesn’t have to pay the AI tax, and AI companies can stop endless capex and focus on building the best LLMs (and maybe some apps).

4. The Excel analogy

If LLMs are sold as “commercial offerings to power your local tools,” then someone gets to write the LLM equivalent of Word/Excel. Popular in professional environments, but with spillover into home use because people want the same capabilities everywhere.

And given the broad appeal of LLMs, they could end up embedded in every work machine making them as expected as having Excel installed.

5. The likely future: PCs+Phones ship with AI core hw/sw

The endgame could be a coherent “operating system” built around licensed LLMs, used both at work and at home. All your tools like code editors, web apps, productivity software, would run on it. You’d pay for the hardware and the software license, just like today’s PCs and phones. Apps are built with the assumption that the OS includes an LLM module to interface with.

How do I get second order effects from these agents?

hello@jodavaho.io (jodavaho) — Tue, 12 Aug 2025 00:00:00 +0000

I use coding agents daily. So far, they are good for first-order effects. Things like “Fast typewriter” (doing something I would do but without interface delays) or “super copy paste” (altering an example and applying it with nuance in several places) or “super tab complete” (suggesting large blocks of text/code vs just single symbols) or even “meta bash” (asking for scripts that do oneoff analysis or processing of a few data sets you have lying around).

Most of these are code-generation tasks. Sure, we can write code as fast as possible, but that doesn’t help the thinking, decision making, and understanding go faster. According to Amdahls law, that’ll become the blocker, and probably already has. Theoretically, giving up on understanding is the natural next step - essentially let agents build whatever they want. Aka vibe coding.

So we’ve already hit the AI agent first-order improvements limit??

What I want is second order effects. I want tooling around my tooling to help my tooling build my tooling so that I can be more productive and directly involved. That’s the only way to ensure that what I want comes to pass.

I want smart-jump 10x, smart-context popups 10x, live execution / simulation, integrated replay of test cases with backtracking, and a million other things I can’t imagine yet. Instead I get a github agent that automates leaving comments about my code style. Thanks.

Anything else is just unproductive laziness and gambling your future job.

The Moving AI Goalpost

hello@jodavaho.io (jodavaho) — Tue, 23 Aug 2022 00:00:00 +0000

This post is a work in progress…

We have enough AI to explain conscioiusness now, and therefore to create an AI that feels “General”.

AI can generate words from sounds
GPT is well known to generate narratives and conversations given rough priors
Something like Dall E to generate mental imagery around those narratives, when required
Constantly retrained by feeding what we hear (verbally) and what we see (after “object detection”) in a feedback loop, with all of our “story” as well.

The complication is the starting conditions. We are meant to believe that we are magic “conscious” creatures, even though centuries of contemplatives dispute this.

Science is searching for something that doesn’t exist, and so the expectations of AI are too high. The above is enough.

License

CC BY-SA 4.0