I’m probably going to stumble over some of the terminology here, but I think it might be possible to describe what @[email protected] is proposing as a consequence of LLMs ultimately being lossy compression systems. Inference is a function over a lossily-compressed data set, and “chain-of-thought reasoning” and “agents” may sound sophisticated, but are simply applying containerization and DevOps tools to VM images of the inference application in an attempt to get around hard memory limits on the context window for inference. “Chain-of-thought” attempts this in a serial fashion, passing results from one instance to the next, while “agents” implement this hierarchically and recursively (and woe to the poor bastards who wished that mess upon themselves). But in both cases, the “finalization” phase is necessarily a further lossy compression step, attempting to compress a result from the inference process to a fresh instance of the inference application, so as not to immediately blow out the new instance’s context window.

Given this necessity, it comes to seem somewhat intuitive that there may be “strange attractors” in the higher-dimensional vector space that is the compressed data set which surround code that creates and maintains message passing channels. No matter what you’re doing with an “agentic” process, the inherent necessity of context cramdown & message passing means that querying into the space where such code examples lie is a hidden requisite of running the damned things, thus turning such functionality into the sort of selfish elements that BioMan is talking about.

The problem in investigating and concretely describing this phenomenon is nailing down the exact functions and processes that make it happen. Given the godawful messes in the Claude frontend codebase that @[email protected] has been documenting, I’d be surprised if there’s one developer in a hundred at Anthropic or OpenAI who can describe in detail how the intentionally-developed context-passing code for their “agents” works.

Somebody writes a low-level kernel for Emacs and the revenge of the Lisp machines begins

what the hell is facebook “shipping” that’s so important anyway? more fake trump-as-jesus photos for louisiana snake-fondlers?

VC fundraising pitch: local Dutch Bros knockoff that offers drive-thru coffee enemas

The mechanics of this are left as an exercise to the reader

Hasn’t Time been a Marc Benioff rag for a while now? This is the kind of shit that guy eats right up

oh oh i think i learned this in calculus class

Every time Yegge comes up these days, I feel compelled to reiterate that I unironically want to see the Computer History Museum try to stand up a Gas Town replica 20 years from now

i still can’t find the shift key on my keyboard.* i think it may just be a greybeard myth. no big deal, posting messages like this really bolsters my public image by making me seem relaxed and approachable.

*Unless I am publicly accused of fraud, trade secret theft, mass copyright infringement, etc. in which case my PR team tells me I need to temporarily drop the act.

Defining a non-GAAP unit of account is your most important task as a Silicon Valley executive. Product? Pffft hahahahaha

By 2025/26, these machines will outpace many college graduates.

tick tick tick tick

(I am choosing to forego the obvious jibe that the models fulfilled this prophecy by dragging many college graduates down beneath them)

The government agents who live in my walls and plant microphones in all my appliances impose entirely different productivity metrics, which are simultaneously more and less attainable

or flooding the zone?

With Zerohedge I think this is most likely. Maintaining truthiness by predicting 30 out of the last 2 recessions.

Observe the power of a rhetorician who recycles singularitarian AND libertarian talking points simultaneously!

…damn, it’s not really very impressive. at all.

I still respect the definition he gives for magic, which he probably pulled forward from one of the many past ritual magicians he’s studied. Magic is the change in consciousness in accordance with will… but we must always remember that that doesn’t say whose consciousness, nor whose will.

Yeah, I began losing interest in Greer as it became clear that he was perfectly happy squatting in the middle of the red-brown alliance during the Trump era. His critiques of industrialism and unquestioning belief in technological progress broadly align with what we discuss here, but he will always coddle MAHA types and tale a shrugging “well, what can ya do?” attitude towards people like Trump, as it fits his preference for cyclical theories of civilization.

I noticed a couple months ago that he actually managed to dig Nick Land out of whatever tweaker den that guy’s been hiding in for a podcast, which says a lot about what he’s willing to indulge these days.

I think the rationalist beliefs are quite pervasive among mathematicians

Unsurprising, it’s quite flattering to believe that your area of study is what generates The One True Ethical Code!

The sweaty desperation on both sides is hilarious: both the Forest City developers for getting in bed with a dodgy hustler like Balaji, and Balaji for signing up with such obviously desperate people, whose reputation in the region has already been fairly well torpedoed.

For that matter, who the hell is Emil Michael? Defense Undersecretary tells me nothing, they’re handing those sorts of appointments to anyone who’s ever scammed $5 in venture capital

I am not retreating from public writing

That’s OK, it will still mostly be confined to oversized Twitter posts, so most people will ignore it.

I am personally surprised the Chinese state continues to allow the open sourcing of models this good, given potential risks. To be clear, I myself might be fine with models presenting this level of marginal risk being open weight, but I am surprised that China is fine with it. I suspect the reason they are is 75% explained by strategic blindness/lack of AGI-pilledness (the CCP is very Yann Lecun-y in its views of AI).

The strategy seems quite clear to me, and not blind at all: they want to hold a gun to the head of over-capitalized Americans like Mr. Ball and his enablers, and will be perfectly happy if that drives our tech industry into a financial escalation spiral that ultimately neuters it for years to come. He has the barest glimmerings of being aware of this, but frames it wrong because he can’t let go of the sci-fi sentimentality that drives his whole industry.

The other 25% or so is their lack of compute for customer inference (making China’s open-weight strategy an unintended byproduct of US export controls) and the normal Chinese strategy of aggressive exports.

The Chinese still have overwhelming clout in final electronics assembly, i.e. turning this shit into things people can actually use. Of course they are interested in maintaining that advantage. And if, in the medium term, they can nudge more financially desperate American politicians and executives towards loosening trade and investment restrictions, possibly even towards equity partnerships, so much the better. “Lack of compute for customer inference” is ultimately a red herring. If I’m a Chinese economic strategist, why would I not expect GPU tech to become commoditized over time, like every other aspect of advanced computing technology has? Perhaps even more rapidly, as the designs are inherently massively-parallel implementations of similar base execution blocks!

The rest of his “accelerationist”/“decelerationist” drivel concerns a silly spat that is mainly centered around and driven by Twitter. It is still difficult to overestimate just how brain-rotted Twitter has made these people.

e: case in point, he leads out with barely-veiled Covid lab-leak trutherism, surely just to rile up the hogs. What a loser hahahahahaha

“A nonliving, invisible, dangerous, and infinitely self-replicating agent escaped from a Chinese lab,” you say? Color me shocked.