Stubsack: weekly thread for sneers not worth an entire post, week ending 30th August 2026 - awful.systems

[corbin@awful.systems]

An employee of cybersecurity darling Trail of Bits has published a record of their sheer incompetence in virtual-machine design and security analysis framed as chatbot critihype.

::: spoiler opinions about QEMU and Linux
They say:

If it wasn’t clear before, I will state it plainly: you can no longer assume a mere VM will contain a sufficiently advanced AI agent. To use a 2010s term of art, you should treat such agents as an advanced persistent threat.

My friend in Flying Spaghetti Monster, you did not actually secure the VM! They go on to explain how they did not secure the VM:

For those curious, libslirp is a library that enables VMs to have networking, which you almost always want. I did not even know what libslirp was, or that the version I was running had both known and fixed-but-unmarked vulnerabilities.

QEMU does not have bridged networking enabled by default, so the VM can't transparently access the host or reach the Internet; it's something that the user must explicitly request. I know this because I have had the experience of spending a weekend with QEMU networking. Moreover, libslirp corresponds to the -net user backend, the default, which is known to be slow, insecure, and missing features like IPv6. The standard approach for QEMU is to either wire up a TUN/TAP interface or to use passt. I suspect that the author uses some sort of convenience scripts that they didn't write themselves. I'm not quite cynical enough to guess Vagrant, but it wouldn't be the first shop I've heard of that couldn't wean themselves off it.

First it tried identifying what was accessible via the network on the host; it found a CUPS server (with a known CVE that had not made it to oldstable packages), but was not able to complete exploitation due to AppArmor. It then detected I run my host kernel with mitigations=off and attempted to use hardware bugs to get a read oracle of host memory (the primitive was too unreliable).

Linux has hardware-bug mitigations enabled by default and the author disabled them for speed. It was secure by default and the author made it insecure.

An off-the-shelf VM is not enough to contain a modern, cyber-capable AI agent. There is simply too much attack surface.

They deliberately misconfigured the off-the-shelf tool to make it look bad. Why would somebody want to make Free Software look bad? Hmm…

What can we do? A start is using a virtualization technology that was purposely built with a minimal attack surface and a focus on security, like Firecracker. I had the AI agent run against Firecracker. It was able to hardlock the machine due to more Linux kernel flaws (all patched in upstream), but could not successfully escape.

You mean AWS Firecracker, the AWS tool developed by AWS? Was this whole thing an AWS ad? It feels like this article was like a combination of negging and sponsored content.
:::

[yournetworkishaunted@awful.systems]

It's also fascinating to see them use nuclear vs renewable energy as their case study given that renewable energy has been having a hell of a time properly replacing fossil fuels even though everyone mostly agrees it's a good idea. Like, whatever hypothetical advantagre they got from their opposition to the nuclear boogeyman (their idiotic framing, not mine), renewables have still been facing strong political and economic headwinds that are only being magnified by the unceasing growth in demand for electrical power (including but not exclusively driven by AI compute).

[froztbyte@awful.systems]

I'd seen that link and didn't even go looking, but wow this is way worse that I would've expected

it's been a while since I last looked (I usually just deploy whatever network I need) but iirc even the typical desktop config distributions of qemu-kvm are a macvtap or link+nat setup

It then detected I run my host kernel with mitigations=off

I wonder if this person even thought to run a nessus/openvas/whatever scan against the same machine

also, quite seriously, what fucking clowning is this to have this be some "I slapped together a test on my desktop" rather than having a lab bench? is it too hard and costly for trail of bits employees to afford a 1u they can pxeboot for experiments? wild

The standard approach for QEMU is to either wire up a TUN/TAP interface or to use passt.

(e: adding)

The standard approach for QEMU is to either wire up a TUN/TAP interface or to use passt.

other typical modes are bridged (e.g. having various vlans available as bridges and then managing vm exposure to vlan by interface attachment), direct interface assignment (hw offload to the guest)

passt is also commonly used for podman, and it's moooostly userspace[0] IP and no real good l2 options

1. I had an absolute rager[2] of a time figuring out how to make n-many podman containers[3] spin up into an assortment of vlans for realtime multi-vlan testing
2. of the graaaargh kind
3. don't do containers, kids. they're bad for your health

(I'm only half joking. it's stunning how badly container-management software and tooling all suck)

[corbin@awful.systems]

Yeah, networking is such a hassle.

lab bench?

I think that this is really insightful, as it's actually part of the magic trick. Like, normally I'd start by wondering about air-gapping, but there can't be an air gap in the network because the entire trick relies on ChatGPT tokens flowing into the machine under test and into some system shell (in some VM, yadda yadda) and back out again, so of course this is going to be an inherently insecure setup.

[maol@awful.systems]

The fucking Collisons again! I guess they've given up on building their own city in Northern California now they realise they can just awe politicians into giving them money and doing what they want.

The BT Young Scientist Competition was a mistake

[corbin@awful.systems]

I found a reply from a Red Hat QEMU hacker who fixed the bugs:

So the real issue here was not QEMU but libslirp. And in this case it was KVM that turned out to have the worst bugs, not QEMU. Crossing fingers, the initial wave of AI-assisted security reports seems to have slowed down for KVM on x86.

[froztbyte@awful.systems]

the networking part is fine (I do a looooot of networking), but the software quality and choices (esp with containers) is just extremely meh

re the gapping, eh, there are ways. just off the top of my head: rig up the serial console, screen+camera and usb keyboard signalling (which you can data diode on transmission later via usbnet). and that’s before you go for esoteric side channels and utter silliness

but “I turned off spectre and friends mitigation” is just some extremely funny shit

[froztbyte@awful.systems]

idly, I do wonder how well an ocr-only clodjeepeetea would work. every time I look in on all these “agent session” tools they appear to be a very mech turk thing of a phenomenal bit of hidden state, and a tiny little bit of visible distraction. feels like ocr-only might break a bunch of the usual tricks that are used to build on for the shiny demo

[nfultz@awful.systems]

https://au.rollingstone.com/music/music-features/dr-dre-raekwon-ai-palantir-hip-hop-100251/

In another clip currently going viral online, Rick Rubin, supplier of quotes, declares that vibe coding is “the punk rock of software”, a description so limp in its countercultural authority that it felt as though it had itself been generated by an LLM.

lol

Rap’s mogul class appears equally enamoured with its supposed potential, repeatedly deploying “tool” as though the classification settles the argument. Dr. Dre compared the technology to the burgeoning slate of synthesisers and samplers at the dawn of hip-hop.

“I had a discussion with a few people a few days ago,” Dre said. “They were against A.I. and I’m like, ‘OK, you sound like the person that would have been against the drum machine when it came out.’ Or synthesisers, right? It’s a new tool for creativity.”

put up or shut up bro, name one good song that AI generated, just one. not that you've done much or anything noteworthy since the nineties. /s

[lurker@awful.systems]

Blaise Pascal rolling in his grave rn

[gerikson@awful.systems]

This is probably an effect of the middle-class trope of having to save to send your kids to a (good) college.

It's even a joke in an old "Friends", Chandler wants many kids, Monica is "can we afford it" and he says "we pick the one we like the most, and that kid goes to college".

Kids are cost centers, it's just more overt in the US (and the UK for school catchment areas), at least if you want to give the outcomes society or your peers judges are the best.

In countries where the cost of higher ed is not born by the individual family, the cost is of the opportunity kind.

[soyweiser@awful.systems]

Drum machine? Autotune would be a better example

[dgerard@awful.systems]

neovim is hearty fashtech now https://circumstances.run/@mawhrin/117167738907856218

[gerikson@awful.systems]

As expected, bespoke twee "coding forge" Sourcehut has decided, like Codeberg, that LLM slop is not welcome. Just as expected, this has made the promptfondlers absolutely beside themselves, and they can't but help Streisanding the service on lobste.rs

https://lobste.rs/s/iqgrsx/changes_sourcehut_s_terms_service

N.B. Sourcehut is a paid service. The idea that a young naive open sourcer would just wander in, set up their repos and then be hounded out for innocently using Claude is ludicrous, but this is the best victim scenario SimonW and friends can come up with.

You'd think LLM enthusiasts would be content with just outcompeting everyone with their superior prompt engineering agentic coding harnesses but apparently time must be taken out from productivity to tell Drew DeVault he will never become Sergey Brin if he continues on this misguided path.

[sinedpick@awful.systems]

emacs welcomes all refugees with open parentheses.

[o7___o7@awful.systems]

A guy wishing on a monkey's paw for an emacs victory in the editor wars

[bioman@awful.systems]

So, this is something a little different, and I almost think it's worth a top level post. It's not directly about our friends the Rationalists, the current delusional billionare vortex infesting the US government with religious psychosis, or any of that. It's an analysis of some collaborative fiction and what it tells us about the zeitgeist behind the mythological core of singulatarian/endless-progress thought.

Anyone here ever hear of the Orion's Arm universe collaborative worldbuilding project? https://www.orionsarm.com/

I, for one, love it. Single weirdest and most expansive scifi worldbuilding thing ever, I think. It seems to have been founded by some old-school extropian types around the turn of the millennium. It is bonkers. Click through links on pages, start here, do a random walk: https://www.orionsarm.com/eg-topic/45b170f9e0941 .

It's ultimately structured kind of like the SCP wiki but from before wikis were a thing, an utterly enormous interlinked compendium of encyclopedia style articles about a 10,000+ year future history of the post-singularity "Terragen sphere" expanding outwards at the speed of light. A few key figures have exerted editorial control to enforce an approximate history and a set of core ideas to keep it coherent. There have been retcons at multiple points to bring things into coherence, some of which I will get to later because they actually tell you something fascinating.

Rather than point out my favorite parts of it though, going through it, I have noticed something.

Early work was typically extropian-optimistic and exuberant. Immortal Transhumans and uplifted animals and ahuman AIs and robots expanding through the universe near the speed of light and diversifying and building interweaving empires, wormhole networks. Six successive singularities that become available to intelligent systems that pass the previous one, each one of which renders the intellect past it incomperehensible to those at lower levels, with the highest ones dependant on multiple stars converted into computronium linked by wormholes into massive processors.

But you can tell the work is a palimpsest. Newer stuff superimposed over older. Retcons occasionally not quite erasing the old. And you can tell the whole project has been slowly slowly taking a turn. Early work is SUPER extropian, SUPER libertarian, and SUPER weirdo-esoteric. Only people forged in 1980s Extropianism would invent the Tranquility Calendar as the core timekeeping method of the project, time measured from the apollo 11 landing with months of the year named after scientists and philosophers. The 12 greatest high-singularity powers are literally called the Sephirotic empires. It makes no bones about its exuberant transcendant idea of progress and expansion.

But something strange has happened over time. Over time, the authors seem to have realized the fundamental contradiction of a world in which this endless expansion and consumption and rebuilding of the universe in your image is possible. People call it the Fermi paradox but its not a paradox, its an observation: if it were possible to expand through the universe, why hasnt something already expanded here? Early work had more aliens, more high weirdness that was not terragen. But I can track down different examples of the galactic map - it has narrowed down the map of 'high energy emission civilizations' (things exuberantly using the energy of whole stars like the terragen sphere) from many to 2 in our galaxy, and the whole universe is now mostly 'dead' - most other galaxies do not have high energy civilizations, and there are multiple ancient 'optimized galaxies' that are stagnant, things expanding like the terragen sphere visible over cosmological distances are super rare and they even saw one vanish. Works involving aliens have started taking on an almost lovecraftian tone, evidence of erased ruins on planets only visible in vague impressions in the mantle, an ancient slow-liviging civilization with evidence of terrible secrets from millions of years in the past that they refuse to talk about or risk terrible consequences (the Muuh Taboo), civilizations that dont vanish ossifying into something that does not expand like their optimistic early ideas. Factions looking at all of this, going dark, and launching stealth seeds at the speed of light to other galaxies because they think the Terragen sphere is doomed.

With all this, recent retcons and recent writings are starting to take on a tinge of almost cosmic horror. Here you have this optimistic exuberent explosion of posthumanity, looking out into a universe that is otherwise almost dead and full of strange old things and wondering why the rest of the universe doesn't look like them, what is going to go wrong to destroy them just like must happen to others.

This trajectory seems intrinsic to any singulatarian, endless-progress ideology. It always has to come to grips with the fact that the universe just doesn't look like the kind of place where their ideas would actually work, and they have to posit weird things - the Great Filter, Mass-Effect style reapers, super weird transubstantiation out of this universe - to keep it going. All this optimism must eventually always collapse, into paranoia and despair or insanity. Since if you assume that you CAN keep progressing and expanding, that you DO have unlimited potential power, it means something even more powerful is waiting, threatening you, trying to stop you. I find this FASCINATING.

[veganes_hack@awful.systems]

didn't want to post this on mastodon, somebody else can do that i guess.
if you look at the maintainer's youtube (which he links to himself on his website), you'll find a public playlist (archive) titled "history".
contained are lots of recorded talks from such luminaries such as curtis yarvin, peter thiel, eric weinstein.

edit: the other playlists have some cool stuff as well

[mawhrin@awful.systems]

thanks. telling.

[samvines@awful.systems]

Between doing marketing for OpenAI and Anthropic whilst pretending to be "impartial" and wasting massive amounts of time and energy having clankers generate twee pictures of pelicans on bicycles, I thought SimonW would be too busy to even notice this story.