Show HN: DOOM in the kernel, or fibers in eBPF

Posted by ayles 2 days ago

Some time ago my coworker who was working on first version of Perforator (https://github.com/yandex/perforator) kept talking about eBPF, so I got curious about its ISA and restrictions. That was around the time I read about DOOM on pregnancy test, so I thought: "what if someone run DOOM inside of Linux kernel in eBPF? Surely, with some restrictions, this should still be possible?"

I started to play around with it somewhere in 2024 - first stripping DOOM to bare minimum that is needed for showcase, simplifying code and trying to pass first restrictions that I encountered - function limit, recursion, memory access and checks. I tried to do this by hand, but it was, well, tedious labor. So I changed direction and instead started to do some of the things with LLVM-passes - rewriting memory access, simplifying and rewriting loops etc. But there were just so many corners. And given absence of time and all, I forgot about this project.

But couple months ago I thought: "LLMs are quite good nowadays, so why not try again this time with more hands". I tried different approaches on how memory access could be "virtualized", how loops can be made bounded, and in general - how to run unbounded logic on such a machine.

When I got DOOM running, I got carried away. Approach was so "generic", so it would be a crime not to try to run more things. And now we have it - lua running on xdp hot path, llama2 working with softfloats, and even cpython, that did not fit initially into 1M verifier budget, is running there now thanks to freplace.

There is ton of work to do in order to make programs run faster, to make integration easier, but working examples are already there.

Comments

Comment by tptacek 1 day ago

There's so much good content in here, it's a shame you sandblasted your writeup with AI. "To the verifier, it is a number with a biography" is where I veered off the road and into a culvert.

Comment by ayles 1 day ago

Yes, there is a language barrier. And I was in a hurry because of ongoing life events, so it was kinda "today or in a month" for me. Couldn't keep it much longer for myself and wanted to share.

Comment by MisterTea 1 day ago

If what you wrote here are your own words then I would say that you write well enough to not need an LLM.

Comment by ayles 1 day ago

That is actually quite reassuring, thanks

Comment by tptacek 1 day ago

I totally understand where you're coming from. I'd recommend just publishing in your own words even if English isn't your best language; at this point, in 2026, it will read as charming and genuine, not slapdash.

A big problem with current model AI writing is that the topic sentence of every paragraph reads like a magazine headline, and it can be tricky to catch because a lot of magazine headlines are kind of good --- when they're headlines!

Number with a biography, yeesh. :)

This was neat work, though. Thanks for sharing!

Comment by ayles 1 day ago

Noted for the future. I actually wrote a draft myself and mostly used ai to translate it. And there were some "late discoveries" that required me to patch article fast. For instance, for older memory tier (without arenas) I was using 2MB+8B regions with 8B overlap - this allowed unaligned reads to happen without checks, but later I got 4MB non-overlapping regions with llvm read-write alignments to work and that allowed me to remove additional check on stores, make memory fully writable from host side and made atomics correct. So this kind of patch needed to land in article between my "finishing tinkerings".

Comment by kbigdelysh 20 hours ago

Actually, I think you did the right thing to use AI to write the article. As far as it's a clear and interesting story I don't care how or who wrote it.

Comment by tptacek 16 hours ago

Like, I'm not moralizing about it or anything; I'm extraordinarily AI-positive. But I think if you're trying to help people break through the noise to communicate to developers, "it's fine to have AI write your post" is not great advice. You can be fine with it! Good on you! But developers in general seem to recoil from it.

Comment by rvz 1 day ago

Damn. That is a real shame. This whole writeup has some slop in it then.

Comment by markrwilliams 1 day ago

Skip the first half of the article to learn how the project overcame eBPF programs' limited runtime environment:

"On Linux 6.9 and newer (6.10 on arm64, where JIT support for the arena landed later), the window is backed by bpf_arena... On kernels without a usable arena, the same four gigabytes are assembled from 4-MiB pieces... [in] separate global-data maps."

These maps are of type BPF_MAP_TYPE_ARRAY.

See https://lwn.net/Articles/961941/ for a discussion of `bpf_arena`.

Comment by ayles 1 day ago

And compatibility part was most fun =)

Basically, it is better to use maps that can be directly used with load with immediate map index/fd, but we have at most 64 of those, sharing limit with user's data maps and even maps used for (iirc) 7.1 gotox. So for most programs we just use 32 maps or so. If we need more memory - only then array maps (I mean one that has more than 1 element and each element is 4MB) are used as upper part of address space, since they require helper call and are overall slower for our case. But fiber stacks can be allocated from this "slow space", because fiber mostly never reloads its map pointer and fiber stack is aligned so it is always within one such region. (and that is one part of two - memory; second one is about compute and verifier limits and we overcome them by using regions and dispatch-loop)

Comment by morolis 1 day ago

I've been thinking about this question how can I run general-purpose programs in eBPF? I look u idea, It's cool!