A Python Interpreter Written in Python

Posted by xk3 3 days ago

Counter106Comment29OpenOriginal

Comments

Comment by BoppreH 5 hours ago

> Byterun is a Python interpreter written in Python. This may strike you as odd, but it's no more odd than writing a C compiler in C.

I'm not so sure. The difference between a self-hosted compiler and a circular interpreter is that the compiler has a binary artifact that you can store.

With an interpreter, you still need some binary to run your interpreter, which will probably be CPython, making the new interpreter redundant. And if you add a language feature to the custom interpreter, and you want to use that feature in the interpreter itself, you need to run the whole chain at runtime: CPython -> Old Interpreter That Understand New Feature -> New Interpreter That Uses New Feature -> Target Program. And the chain only gets longer, each iteration exponentially slower.

Meanwhile with a self-hosted compiler, each iteration is "cached" in the form a compiled binary. The chain is only in the history of the binary, not part of the runtime.

---

Edit since this is now a top comment: I'm not complaining about the project! Interpreters are cool, and this is genuinely useful for learning and experimentation. It's also nice to demystify our tools.

Comment by gwerbin 3 hours ago

PyPy handled this by implementing PyPy in a restricted minimal subset of Python that they called RPython, and that seemed to work out well for them.

Comment by SJC_Hacker 2 hours ago

This is the case only if the new interpreter does not simply include the layer that the old interpreter has for translating bytecode to native instructions. Once you have that, you can simply bootstrap any new interpreters from previous ones. Even in the case of supporting new architectures, you can still work at the Python level to produce the necessary binary, although the initial build would have to be done on an already supported architechture.

Comment by anitil 8 hours ago

Oooh it's a bytecode interpreter! I was wondering how they'd fit a parser/tokenizer in 500 lines unless the first was `import tokenizer, parser`. And it looks like 1500ish lines according to tokei

I think because python is a stack-based interpreter this is a really great way to get some exposure to how it works if you're not too familiar with C. A nice project!

Comment by blueybingo 6 hours ago

the article glosses over something worth pausing on: the `getattr` trick for dispatching instructions (replacing the big if-elif chain) is actaully a really elegant pattern that shows up in a lot of real interpreters and command dispatchers, not just toy ones -- worth studying that bit specifically if you're building anything with extensible command sets.

Comment by johndough 25 minutes ago

Are you a bot? All your recent comments point out a thing in an article and contain LLM-isms.

Comment by bdangubic 24 minutes ago

you asking a bot if it is a bot? :)

Comment by vachanmn123 5 hours ago

Very well written! Everyone used to tell me during Uni that stacks are used for running programs, never ACTUALLY understood where or how.

Comment by jgbuddy 37 minutes ago

one liner:

eval(str)

Comment by tekknolagi 8 hours ago

See also https://github.com/nedbat/byterun and https://github.com/rocky/x-python

Comment by bjoli 8 hours ago

And, in some ways, PyPy. I still think it is the sanest way to implement Python.

It makes me sad that I have to write C to make any meaningful changes to Python. Same goes for ruby. Rubinius was such a nice project.

Hacking on schemes and lisps made me realize how much more fun it is when the language is implemented in the language itself. It also makes sure you have the right abstractions for solving a bunch of real problems.

Comment by anitil 8 hours ago

> And, in some ways, PyPy

What do you mean by that? I'm not familiar with PyPy

Comment by nxpnsv 8 hours ago

PyPy is python implemented in python. It is fast.

Comment by wyldfire 1 hour ago

The fact that it's written in python is often brought up in order to explain its name. But really, it's much less interesting than the fact that it has a tracing JIT. If it were called PyJIT I'd bet it would be clearer and more obvious that it's fast. And people would prob get less hung up on the distinction between python/rpython.

Comment by notpushkin 8 hours ago

https://pypy.org/

It lags behind CPython in features and currently only supports Python versions up to 3.11. There was a big discussion a month ago: https://news.ycombinator.com/item?id=47293415

But you can help! https://pypy.org/howtohelp.html

https://opencollective.com/pypy

Comment by Doxin 8 hours ago

PyPy is python implemented in RPython, which is technically a python subset. It's so restricted it might as well be a different language though.

Comment by bjoli 6 hours ago

It is restricted in a way that you would restrict yourself to write high speed software in most languages, and I found it is not that restrictive compared to C that you would have to use if you were to write a fast Python library.

Comment by Doxin 5 hours ago

oh for sure, but I still feel like telling people pypy is written in python is misleading. it's written in something significantly like python, but it's not python.

Comment by mjmas 4 hours ago

> technically a python subset

So it can just run under CPython? If so, then that isn't too misleading.

Comment by bjoli 3 hours ago

Yes. It can run under Cpython (2.7).

Comment by actionfromafar 5 hours ago

Well, one could rewrite Python (perhaps piece by piece?) in Shedskin.

Shedskin is very nearly Python compatible, one could say it is an implementation of Python.

Comment by throwpoaster 1 hour ago

How does it define ‘\n’? ;)

cf: https://www.cs.cmu.edu/~rdriley/487/papers/Thompson_1984_Ref...

Comment by gield 3 hours ago

(2012)

Comment by em-bee 18 minutes ago

actually it was published as a chapter in "500 lines or less" in 2016: https://news.ycombinator.com/item?id=11796253

the text is based on python 3.5 which was released in 2015

other discussions:

https://news.ycombinator.com/item?id=16795049

https://news.ycombinator.com/item?id=12455104

https://news.ycombinator.com/item?id=11796253

Comment by woadwarrior01 7 hours ago

aka A Metacircular Interpreter

Comment by mapontosevenths 2 hours ago

Do you think God stays in heaven because he too lives in fear of what he's created?

Comment by andltsemi3 6 hours ago

"Yaw dog I heard you liked python, so I put python in your python so you can interpret python while you interpret python"

Comment by kevinten10 6 hours ago

[dead]

Comment by hcfman 8 hours ago

Just wondering why you stopped there? Why not a python interpreter for a python interpreter for python ?

Comment by dnnddidiej 6 hours ago

It already is that.