Python JIT project was asked to pause development

Posted by kbumsik 3 days ago

Comments

Comment by mwkaufma 3 days ago

>> While the intent is not to call for competing proposals, we believe that now is a good time to discuss and propose alternative proposals as well.

lol

>> For example, rather than proposing one single concrete JIT implementation, it may make more sense for the PEP to describe a JIT infrastructure that can support multiple implementation strategies.

poison-pill requirement

>> We are setting a window of six months for a PEP to be submitted and resolved. If no such PEP is accepted within that window, the JIT code must be removed from the main branch

so it's going to be removed from the main branch

Comment by sidewndr46 3 days ago

Without a doubt, a great way to kill any project is to add unrelated and ambitious technical requirements to the project. This opens it up to an avalanche of discussion and feedback and almost certainly will kill it off

Comment by scott_w 3 days ago

No, it’s a simple request to fully investigate the options before committing a massive piece of work to Python. We’ve seen bad implementations of things land before and now live forever. And frankly, if the team can’t pull together a strong maintenance plan, it can’t be allowed to remain in main.

Comment by pjmlp 3 days ago

So it will join PyPy and GraalPy in the corner.

Python JIT history is full of drama, and no, Smalltalk, Common Lisp, Interlisp-D, SELF are just as dynamic if not more.

Comment by stevefan1999 3 days ago

Honorable mention: https://github.com/tonybaloney/Pyjion

Comment by ck45 2 days ago

Not to forget Unladen Swallow Which even tried to merge into CPython: https://peps.python.org/pep-3146/

Comment by scott_w 3 days ago

JIT in CPython has nothing to do with PyPy or GraalPy: it's its own thing. If they can't get a PEP accepted within 6 months then it's best that the code isn't weighing on the main codebase until an approach can be agreed, at which point work integrating it into main can restart. It's not an all-or-nothing situation.

Comment by pjmlp 3 days ago

> JIT in CPython has nothing to do with PyPy or GraalPy: it's its own thing.

I haven't said otherwise.

Comment by scott_w 3 days ago

So why did you bring them up…

Comment by pjmlp 2 days ago

My words were

> So it will join PyPy and GraalPy in the corner.

If you cannot understand what that means, I am not a English professor.

Comment by scott_w 2 days ago

I understand your words, I don’t understand why you think they’re relevant to the discussion.

Comment by pjmlp 2 days ago

Relegated to niche users, just like the other two.

Comment by scott_w 2 days ago

Except it’s not. The intention of JIT in CPython is to make it into the main branch feature complete. If they can’t get the necessary support then it won’t be relegated to niche uses, it’ll be abandoned and need a new effort to get off the ground.

Not even remotely the same context.

Comment by pjmlp 2 days ago

It certainly is the same context, given the Python culture to ignore JITs, rewrite in C, and call it "Python" libraries.

We have been here several times, versus the other dynamic languages, and it has nothing to do with the usual excuse how dynamic Python happens to be.

Comment by scott_w 2 days ago

I don’t know why you keep bringing up Python being dynamic, it’s not particularly relevant. What matters is the content of the request which is to outline how the feature should work and be maintained over time before it can be accepted.

What’s your position? That we should allow an extremely complex system into Python with:

- No clear design goal?

- No maintenance plan?

- No discussion with the steering committee?

PyPy and Jython, and others are not built by the Python team and there has never been a suggestion that any of these systems would be part of CPython.

So again, I still don’t see how different implementations of Python are relevant to the discussion about CPython.

Comment by pjmlp 2 days ago

They are relevant for the Python culture towards JIT adoption.

Comment by onlyrealcuzzo 3 days ago

And Ruby

Comment by petre 2 days ago

Ruby has not one, two, but three JITs.

Comment by pjmlp 2 days ago

Actually more, are you country JRuby, TruffleRuby and RubyMotion.

Comment by throwaway81523 3 days ago

CL is definitely less dynamic than Python. Dunno about the others.

Comment by pjmlp 2 days ago

Why do you think so?

All of them support changing anything at anytime, killing all JIT assumptions, and forcing it to redo the world.

Stop execution, land into the debugger, change whatever code you feel like during the debugging session and then redo last statement, continuing execution.

There are also ways to do this on fly, without necessarily using the debugger.

Comment by dataflow 3 days ago

> We’ve seen bad implementations of things land before and now live forever.

Er, doesn't that depend on how leaky the abstraction is? How often have you seen a JITted language be unable to swap in a new JITter due to some sort of unintended coupling?

Comment by scott_w 2 days ago

If it’s so easy then writing a PEP and getting it approved in 6 months will be trivial.

Comment by dataflow 2 days ago

"It"? What are you talking about? You didn't answer my question at all. I said if you have a second JIT to swap to in the future, you should be able to switch to that without breaking apps and not be forced to keep this implementation forever -- because JITs don't tend to expose APIs or leaky abstractions to the code, and it's not hard to ensure this one doesn't either. I asked you if you've ever seen another language that actually had another JIT to switch to but wasn't able to. Instead of addressing my point, you replied that it should be easy to write something else (a second JIT, "general infrastructure" for JITs, or whatever suits your fancy) and get it approved? What? No that is not, and I never claimed that would be easy. And if it has to be approved by someone who responds like this, it clearly wouldn't be fun either.

Comment by scott_w 1 day ago

"It" is the PEP that the Steering Committee are asking for. I'm not on the committee and I don't know how to write a JIT to meet the requirements.

What I don't understand is "what is your position?" Is is that writing a PEP is unnecessary because adding a JIT to Python could trivially meet the goals you state?

My position is simple: adding a JIT is a major change to the language, so of course it should undergo a PEP and answer the major questions that the steering committee put forward. If you think those questions are trivial, then that's ok. Just answer them in the PEP.

> And if it has to be approved by someone who responds like this, it clearly wouldn't be fun either.

I really don't see why you'd think I would be representative of the steering committee's opinions or attitudes. Do I share a name with one of the members?

Comment by LtWorf 3 days ago

> and now live forever.

What forever are you talking about? Python removes stuff every single release.

Comment by scott_w 2 days ago

Default list/dict arguments is the one that comes to mind. Yes, I know why it happens, no, it’s still bad because it trips up every beginner.

Comment by tasuki 2 days ago

Why does it happen?

Comment by NekkoDroid 2 days ago

Default arguments are only evaluated once and assigned the same instance to every call that doesn't specify that argument. So when you assign a new list as a default argument and then append to that list, the next call will already have one element in that list. So what you need to do is have it "None-able" and within the function create a new list.

Comment by LtWorf 2 days ago

You can use anything immutable like a string, a number, a tuple of immutables, a frozendict, any dataclass with frozen=True, and so on.

No need to do the annoying if x is None.

Comment by tasuki 2 days ago

Yes I know as much, but why? Is it for speed? Python is slow anyway, so no big deal. Incompetence? Malice perhaps?

Comment by LtWorf 2 days ago

It's just because the `def xxx()` part gets executed once when the module is loaded so the default arguments get created then. It's not really a design choice.

If you declare a function inside a for loop with the default argument set to a list, it will be re-declared at every iteration of the loop and the list of the default argument will be a new instance every time.

Comment by tasuki 9 hours ago

> It's just because the `def xxx()` part gets executed once when the module is loaded so the default arguments get created then.

I understand as much.

> It's not really a design choice.

That explains it then. Pretty damning though, no?

Comment by NekkoDroid 8 hours ago

I don't have any evidence for my theory, but I assume it is because arguments are processed mostly on the function side instead of the call site.

My assumption just makes sense when all the function really gets is "a list of positional arguments and a dict of keyword arguments" (ala `*args, **kwargs`) that is "deconstructed" to the named variables on the function side. Then the function never gets the default value passed and fills it in before executing the body. Therefore it needs some value to assign and that value is determined when parsing the function definition.

So effectively I want to say that I think instead of the default value being passed at the call site (which is how C++ for example does it by inserting the expression inplace of a specified value) it is filled in by the function before executing the body.

In the end this is just a guess, but that is my working hypothesis.

Comment by hanzeweiasa 3 days ago

[flagged]

Comment by Qem 3 days ago

> For that reason, the Steering Council is formally requesting a Standards Track PEP be authored that the community can discuss and the Steering Council can formally accept (or reject), making the case for the JIT as a supported, non-experimental part of CPython: its guarantees, its maintenance commitments, and its impact on redistributors.

I didn't notice the current PEP was a provisional one. Hope the new one gets approved. The experimental JIT was reported to finally breaking even and surpassing the default interpreter just a couple of months ago[1].

[1] https://fidget-spinner.github.io/posts/jit-on-track.html

Comment by ksec 3 days ago

>The experimental JIT was reported to finally breaking even and surpassing the default interpreter just a couple of months ago[1].

Thank You. As someone who don't follow python closely I thought their JIT would be similar to what Ruby has.

Not that Ruby YJIT or ZJIT is anywhere close to what JVM provides, but in this case it seems to be quite far ahead of Python.

Which is surprising given how many major companies are using Python. May be because those using Python are not using it as critical part of work unlike Shopify and Stripe which is their core language?

Comment by bob001 3 days ago

Python software is to a large extent either doing things in not-python (c, c++, rust, etc.) or doing things that are not cpu bound (io bound, async, etc.). If you're cpu bound then you can either take a 2x jit improvement or take a 10x non-python improvement. There's few companies of a scale where the non-hot path cost of 2x cpu is so massive as to be worth caring about.

Comment by thatguysaguy 3 days ago

The python overhead of launching big ML jobs is nontrivial, so I think speeding that up would be meaningful. (I mean the initial tracing and other setup, not things once the GPUs are actually doing the work).

Comment by bob001 2 days ago

That seems more like the tracing overhead than the python overhead. The original jit proposal would I believe not help at all with that since ML workloads basically do their own jit. The post being discussed however pushes for a general framework and good tracing support so might help ML workloads.

Comment by jbvlkt 3 days ago

Also you can use projects like numba https://numba.pydata.org/

Comment by IshKebab 3 days ago

Sure but best case 15% faster clearly isn't worth the complexity of a JIT. It really needs to be at least twice as fast. Pypy pretty much achieves that on average.

Comment by Qem 3 days ago

15% faster on top of a base interpreter that itself got 40%-50% faster on the same timeframe.

Comment by IshKebab 3 days ago

Right... but it's still only 15% faster than a simpler alternative. In a language that is 50x slower than the alternatives. Clearly not worth it.

Of course the counterargument is that they'll improve it and maybe in future it will be 100% faster... But that seems pretty dubious given the progress so far.

Comment by germandiago 2 days ago

When I choose Python for something, it is clearly for its speed.

It is nice to have the speed, always. But in Python, it would be a mistake to do it at the expense of flexibility. Same for typing: it is great to have it. I use it. But Python should be dynamic and the rest and extension that does not compromise everything else.

If a JIT can make Python 2 or 3x faster, or even 10x for some workloads, that is nice. But the language itself should support same idioms, reflection, dynamic typing, etc at the same level.

Comment by germandiago 1 day ago

oh, I meant it is clearly NOT for its speed.

Comment by mike_hock 3 days ago

Kind of a shit move to suddenly pull the rug once they've finally gotten it working. Should have been kept out of main from the start.

Comment by oliwarner 3 days ago

The post clearly says the intention is to get a formal spec for formal integration.

To leave their experimental phase they have to define some goals to meet and that requires making some architectural choices that still aren't decided.

Comment by bob001 3 days ago

I suspect the recent "we updated the GC without a PEP and it went live and caused massive issues and we need an emergency point release revert" pushed for a greater degree of process overall.

Comment by mellosouls 3 days ago

Editorialised title.

Development hasn't been paused (with negative implications).

It's now considered significant enough that they've requested feature freeze in CPython main until governance/process questions are settled.

Comment by wavemode 3 days ago

It's effectively a pause. In a project the size of CPython, and a subproject the complexity of a JIT, you can't continue work on a separate branch/repo without guaranteeing that there will be a massive amount of (both textual and semantic) merge conflicts down the road.

Comment by necovek 2 days ago

But with a project running for as long as they are, is it not already clear what they are going for? Is the architecture not settled?

If it is, is it not documented somewhere? Maybe as a formal PEP?

If it is not and it is still in heavy experimentation phase (which is fine), it should not be part of the mainline CPython no matter how much more effort it is for the team to experiment with.

Comment by baq 3 days ago

Nah, you just merge main back every day or so.

Comment by folkrav 3 days ago

That works, until that inevitable one merge that's harder to fix and takes longer, which in my experience then tends to snowball until it's basically the very merge hell you were trying to avoid. Can't say I've ever had a great experience with long lived feature branches. I can't imagine what it would even look like trying to do this on such a massive project and such an overarching feature.

Comment by simonw 3 days ago

"Asked to pause development" isn't entirely accurate: they were asked to pause landing new features (as opposed to bug fixes) on the cPython main branch.

Comment by mindcrime 3 days ago

... and only until the whole thing can be fully formalized with an approved PEP. I'm no Python insider, but that doesn't sound horribly controversial to me.

And yet, I have a hunch it will piss off a lot of people nonetheless and lead to much outrage and wailing and gnashing of teeth. Hopefully it all works out in the long run.

Comment by 12398761 3 days ago

That was kind of overdue. The project started five years ago while massively overpromising.

They should perhaps have kept it in a separate branch back then, but now is the next best time.

CPython's selling point was that it is simple, fast enough with C extensions and the code was accessible. Complicating the code base for occasional 50% speedups (and regressions ...) just isn't worth it. There are so many other languages that fill that need.

Now, I hope that the PEP does not overpromise again and is accepted because of Instagram pressure. Instagram can keep its own JIT fork or switch to PHP, Go or whatever.

Comment by ameliaquining 3 days ago

This hasn't been true for a very long time. Python's major selling points these days are the accessibility of the language and the extensive ecosystem. A vanishingly small fraction of users ever look at the implementation internals. Telling people "if you don't like it, switch to a different language" is particularly unhelpful because rewrites are rather famously expensive. Making things more complicated for the few maintainers, so that they work better for the millions of users, is easily a worthwhile tradeoff.

Comment by ptx 3 days ago

Who is going to pay for maintaining the massively more complicated implementation though? Microsoft pulled their funding of the Python team, and even if they hadn't I think there's a danger in making Python so complicated that it can no longer be maintained without the backing of some giant corporation.

Comment by kelvinjps10 3 days ago

What a shame it will receive a halt when they where starting to make progress I know that after submitting the pep it will go back to development. But t would have been better to just keep the development and the pep for an actual release or continue and if gets rejected ask them to stop

Comment by Qem 3 days ago

> What a shame it will receive a halt when they where starting to make progress I know that after submitting the pep it will go back to development.

To be fair, the apparent lack of progress of the JIT before was in part due to the same team improving the base interpreter by 40-50% between 3.10 and 3.14. The JIT implementation was pursuing a moving target. It was not some static milestone. Kudos for them.

Comment by oefrha 3 days ago

Sounds reasonable given the recent YOLO GC debacle https://discuss.python.org/t/reverting-the-incremental-gc-in....

Comment by wavemode 3 days ago

That seems entirely unrelated?

Comment by Retr0id 3 days ago

It's related in the sense that it's an example of something that didn't go through the formal PEP process

Comment by ospider 3 days ago

This is really a sad moment, I hope the free-theading feature can be kept in the main line. For me, running my library in real threads has actuall revealed some hidden fatal bugs which are hard to reproduce with the GIL>

As for performance, Python 3.x was not even as fast as Python 2.x in the beginning, but with the correct mindset, it's eventually faster.

Comment by germandiago 2 days ago

I am not sure if this is even possible since I have no experience implementing JITs, but maybe it would be a good idea to make it pluggable so that it does not unstabilize other parts of the interpreter in a way that it can continue development?

Comment by dist-epoch 3 days ago

There is a large graveyard of JITs and JIT-adjacent projects for Python.

By now it should be clear to anybody working on Python JIT that the probability of failure is 90%.

The future is probably rewriting performance critical Python code in Rust instead of trying to fix Python.

Or maybe a future LLM could add a JIT to Python in an effort-run.

Comment by adsharma 2 days ago

The future is to write Rust in a python interpreter compatible way leveraging large parts of the python ecosystem.

This will require minor tweaks to the language and perhaps a forked interpreter (a forked parser already exists).

Then you solve the two language problem with a combination of probabilistic and deterministic translation.

Comment by varispeed 2 days ago

Death by committee.

I hope they are proud of themselves.

JIT should be a top priority given popularity of Python. It should significantly reduce carbon footprint of Python driven applications.

Comment by jhayward 3 days ago

> While the intent is not to call for competing proposals, we believe that now is a good time to discuss and propose alternative proposals as well.

If I were a contributor I would read such language as saying "we have no respect for you or your intelligence, so we'll just straight up gaslight you and expect you to accept it."

The dictum can't be read literally - it has to be read like the manipulative, narcissist-speak that it is. And what it's telling you is - get out.

Comment by _old_dude_ 3 days ago

I agree. And the next section is very clear that they want to kill the project.

  > For example, rather than proposing one single concrete JIT implementation,
  > it may make more sense for the PEP to describe a JIT infrastructure that
  > can support multiple implementation strategies.
  > Since many different and promising JIT tracing approaches continue to be proposed,
  > we believe the infrastructure should make it easy to experiment with and evaluate
  > those approaches within CPython rather than be highly coupled with a single strategy.

Allowing multiple strategies is far harder and as far as I know, JIT tracing is still unproven.

Comment by zem 3 days ago

I'm a big fan of the pluggable jit strategy (I even gave a brief presentation about it at an earlier pycon). the idea is that you identify the seams in the interpreter where the jit interacts with the main interpreter loop, and then provide a clean mechanism for the built-in jit to be replaced by something else that works with the same api surface. it's a bit harder than letting the built-in jit be tightly coupled to and intertwined with the rest of the interpreter but certainly not far harder, and the work to get this sort of clean separation will help keep the jit maintainable even if there is never a second jit.

Comment by Retr0id 3 days ago

I think this is uncharitable, it's not like they're inventing new requirements that weren't there before. The PEP process has existed the whole time.

Comment by bob001 3 days ago

I suspect there were people who had alternative proposals which got implicitly blocked by this 5 year effort. Letting a subgroup run wild without proper process is not good for a project this large.

Comment by KaiserPro 3 days ago

> The dictum can't be read literally

out of curiosity, why not?

I mean it seems like they want to get a full spec of what JIT should look like in main? given the faff that hapened with the GC removal, I can sort see why they'd want to do this properly. Especially now that it seems like its practical.

Comment by simonw 3 days ago

"it has to be read like the manipulative, narcissist-speak that it is"

That's a very strong claim. I'm not seeing that at all. What causes you to interpret it that way?

Comment by mike_hearn 3 days ago

It's because they say they don't want to call for competitors and then immediately do so in the second half of the same sentence. It probably wasn't written with bad intent, but you can see why people might find it a strange choice.

Comment by Fraterkes 3 days ago

People in this thread writing conspiracy theories over the biggest language in the world requiring a bit of bureaucracy lends some credence to the idea that programming is not real engineering.

Comment by knollimar 3 days ago

Is TS bigger?

Comment by 12_throw_away 3 days ago

Right? Every popular language has lots of real problems. Every language has made decisions that people don't like. Every language has varying degrees of imperfect governance. Weirdly enough, most languages even seem to grow at least a small community of dedicated conspiracy theorists. But for whatever reason, it's particularly hard to find level-headed discussions about Python, moreso than js or rust or C++. It's weird.

Comment by sitkack 3 days ago

I have used Python as my main language since the late 90s and it has been over the last decade and half been getting more unserious.

It would be nice if cpython opened up a bit, pluggable GC and JIT would go along way towards reducing this manufactured drama.

It wasn't cool to see PyPy or Stackless getting sidelined.

Comment by woodruffw 3 days ago

This isn't really a substantive comment, and to at least one extent it's trivially falsifiable (15 years is before Python 3 became usable, so that alone is a "serious" change in the language).

> pluggable GC and JIT would go along way

One of the points mentioned in the linked discussion is explicitly about ensuring that the JIT design enables multiple implementations.

Comment by sitkack 3 days ago

Python 2 to 3 transition was also unserious.

The flippant attitude of cpythons wrt the standard library is also unfortunate.

Please with your substantive comment comment.

Comment by woodruffw 3 days ago

It seems to have been serious enough; I don't think Python would have succeeded as a language if they hadn't done Python 3.

> Please with your substantive comment comment.

I think binning things as drama isn't substantive, particularly when noting about the linked conversation seems dramatic. I also think they're actually talking about the thing you want (pluggable JIT), so the objection seems incongruous.

Comment by anticorporate 3 days ago

> I don't think Python would have succeeded as a language if they hadn't done Python 3.

I assumed Python 2 was pretty much ubiquitous and that the world wasn't adopting Python 3 very quickly for a long time, but I do wonder if the applications I was working with a decade and a half ago (ArcGIS, Blender, Civ4, lots of Red Hat system tools, etc.) biased that viewpoint.

Comment by cdavid 2 days ago

That's debatable. We can't go back in history, but if it were not for ML/data science, I believe python 3 would have killed python. At that time web dev / CLI utilities were major use cases, and that was the time golang became mainstream.

Data science, and then ofc DL being done through python just when python 3 was kinda usable (around 3.3/3.4) was a struck of luck timing-wise.

Comment by questiuner 3 days ago

Why was PyPy abandoned and not embraced by Python? NIH?

Comment by kzrdude 3 days ago

The short answer is that CPython didn't want to break compatibility with lots and lots and lots of Python modules implemented in C, so it was never viable to let PyPy seamlessly replace CPython.

Comment by JulianWasTaken 3 days ago

(PyPy was not abandoned.)

I assume you might mean to ask "why wasn't PyPy adopted in some formal way into CPython" rather than a separate project, for which the answer is at least partially likely to be because it's a completely separate implementation.

Comment by anon-3988 1 day ago

Why is Python trying to be fast? Can it please stop messing around with the bytecode to make it "faster". If your user want speed, please for the love of god just use anything else. Just keep Python simple and immortal. I want to keep using Python as is for the next 50 years, please stop breaking shit because you want to fit a square into a rectangular hole.

Comment by IshKebab 3 days ago

Seems reasonable. As I understand it the JIT implementation has not really been successful anyway.

Comment by andrewmcwatters 3 days ago

[dead]

Comment by OutOfHere 3 days ago

Losing development momentum for a beancounting reason like this one is a sure way to kill a project. It works every time. Once development is halted, it is very difficult to pick it back up.

Comment by bob001 3 days ago

Python isn't a side project to yolo on. Updating the GC without a PEP caused massive issues for actual people using Python. If you want to impact software used by millions of developers then you better be willing to handle a bit of process.

Comment by rurban 2 days ago

Killing the incremental GC for synthetic benchmarks, not real ones, caused the issues. If the processes are broken and you get nothing done with it, you better step aside and do it without. Fork it, and drive the blockers away.

Comment by simonw 3 days ago

They're not being asked to halt development, they're being asked to halt landing new features on the cPython main branch.

Comment by 3 days ago

Comment by kzrdude 3 days ago

The JIT project already lost a lot of momentum when the people working on it lost their jobs (at Microsoft)..

Comment by 12_throw_away 3 days ago

What does "beancounting" mean here? I don't see anything about money or budgets in the announcement. Are you referring to their concerns about the maintainability and complexity of the codebase?

Comment by nmstoker 3 days ago

Duplicate of here:

https://news.ycombinator.com/item?id=48421400

Comment by elpocko 3 days ago

Don't bother clicking, that post got 0 attention. Not helpful, mate.

Comment by nmstoker 3 days ago

It is literally the same link, it was the original and it had the correct title rather than the editorialised one here.

At the time I posted it, both were pretty lacking in attention, so it made sense to direct to the earlier of the two.

Comment by elpocko 3 days ago

It would make sense if there was any activity on that older post. A 12 hour old post with no comments is dead, linking to it is a waste of time, especially for those who click on it to find 0 comments.

I did click on your link, it was a waste of time and that wasn't very nice.