Why Emacs Consult async searches feel slow and how to speed them up

Posted by mpweiher 3 days ago

Counter58Comment23OpenOriginal

Comments

Comment by geocar 3 days ago

It's possible to determine good values automatically: Measure your typing speed.

You can do it in the background with:

https://pavpanchekha.com/blog/emacs-keylogger.html

and periodically update the latency parameters to be just over your natural typing speed.

Comment by IceDane 3 days ago

I just chose to use an editor whose very architecture doesn't require nonsense like this so the editor doesn't freeze on me.

Comment by tmtvl 3 days ago

Oh, you use Lazarus? Cool, it's an amazing project. Too bad it's a bit of an unsung hero, it deserves it's day in the limelight.

Comment by 2 days ago

Comment by geocar 3 days ago

Cool story bro.

I've got an emacs uptime measured in years, so I have no idea what you're talking about, but I'm happy that you're happy.

Comment by iLemming 3 days ago

Oh, you're so funny. What you've been "editing" buddy, your gramma's cookie recipes? Just the other day I transcribed a three-hour long video of a meetup, picking the model, setting parameters, running the process; then scrolling through Reddit while waiting for it to finish; then I opened the transcript in subtitle-editing mode that allowed me to "follow" the video - I would scroll through the subs and it would move the player pointer, I'd watch the part, pause, move back and forth, etc. The opposite way is also possible - moving the pointer in the video will automatically move the cursor in the text. I then sent a request to a model asking it to summarize it, extracted pieces for my notes. I explored URLs from the meeting by OCRing them automatically - I didn't have to manually type jackshit, I'd point to a frame, select the region on the screen, and voila - the url appears in the buffer. I then reverse-url-searched those links, I was curious of what people been talking about that stuff on HN. Then I grabbed the links shared in each thread - single key-press for me. All that without leaving Emacs - no context switching, no focus-wandering, no lollygagging. And that's just a regular Wednesday. The crazy part, that I could just ask an LLM do all that - it could actively drive the process, directly opening things, moving between the buffers, gathering data, etc. Or I could ask it to make it into a deterministic script - if I wanted to have it for automation or whatever.

You "chose to use an editor", pfff... bro, my key-switches are dying of laughing.

Comment by iLemming 2 days ago

For anyone got angered by my tone, allow me to explain. The OC clearly has no idea what this post is about, it has nothing to do with "freezing" or whatever, and remarks like that are simply disingenuous, ignorant and presuming. Sure, Emacs has many shortcomings (just like any piece of software in the world), but thinking that people "choose to use it", is a sign of complete unawareness of what Emacs is about. There's simply no existing piece of software today that can replace Emacs for me, and I'm not alone. I wish there was a meaningful alternative for it, alas, for better or worse - it just doesn't fucking exist. Name any number of choices, and I will give you more than dozens of real, practical, honest reasons for why it is not a replacement.

Comment by hansvm 3 days ago

> scrolled reddit

> no focus-wandering, no lollygagging

Comment by iLemming 3 days ago

whisper.cpp is awesome but even it just can't process a three-hour vid in seconds. What do you suggest should I'd be doing? Clench my butt-cheeks and stare intently into its work log, hoping it would finish faster? Ah, I know, you'd be like: "messing with your Emacs config", right? Because of course, the tale of any Emacs user is that they have no free time, no pets, no family - it all is occupied by Emacs.

Comment by KallDrexx 3 days ago

You know what makes Emacs feel slow?

Corporate EDR BS.

I thought Emacs was terrible to use until I realized it was extremely fast on my home computer. All the pauses that EDR does combined with Emacs mostly single threadedness is a bummer. I still daily drive it now for my IDE at work because it's workflows work better for me than others, but it is such a bummer.

Comment by dannyobrien 3 days ago

(I think EDR here means https://en.wikipedia.org/wiki/Endpoint_detection_and_respons... )

Comment by hansvm 3 days ago

I've had good luck disabling the internet when I need my work computer to run faster. Those "security" checks are optional, and it'll let you work offline.

If just logging filesystem interactions and whatnot for future upload is also slow then you're hosed, but at one place I worked every application too an extra several seconds to start up for some sort of remote program inspection, which is fine till the application is git, grep, or cd.

Comment by matthewbauer 3 days ago

Do you notice this on macOS or Linux?

I definitely notice an EDR tax on macOS generally, but I haven't noticed Emacs having issues specifically. It's usually certain I/O operations like Git or tarball extraction that become slow.

Comment by KallDrexx 3 days ago

I notice it on macOS, which I use for work. I use Linux personally but no EDR there :)

There are just so many little operations that cause stuttering on macOS with an unfortunately over-eager crowdstrike configuration. Magit is the worst offender but it causes paper cuts all over :(

Comment by iLemming 3 days ago

> I notice it on macOS

So, on Mac there are two main problems - displaying and fork/exec.

Rendering on Mac is slower because NS port display backend just sucks. Emacs's redisplay emits many small drawing operations per frame - one per glyph run, cursor, or fringe bitmap. X11/cairo and pgtk on Linux batch those very efficiently - Cocoa has nothing comparable here.

You can try installing one of the forks of Mitsuharu's port - they usually have --with-metal flag and some other improvements, but they are typically one-two versions lagging behind, and in my experience they are not always stable. In general, possibility of Emacs dying on Mac is not zero, on Linux is much more rock solid.

And then the forking problem. Emacs has a big heap, so each fork is expensive. On Mac it triggers all sorts of crap - code-signature validation, Gatekeeper/XProtect scanning, TCC checks. That's dozens of milliseconds per subprocess and Linux has no such crap to deal with - shit is just fast. Magit calls git dozens of times per refresh. Now multiply that and then you get usual complains - pestilence on poor Jonas Bernoulli: "I love Magit, but why is it so damn slow..." etc.

Here are some things that I do to mitigate:

- Do these and never touch keyboard rate settings in the UI, just do it, reboot and thank me later.

      # "Set a blazingly fast keyboard repeat rate, "
      defaults write NSGlobalDomain KeyRepeat -int 1
      # "Set a shorter Delay until key repeat"
      defaults write NSGlobalDomain InitialKeyRepeat -int 9
- disable App Nap for Emacs:

      defaults write org.gnu.Emacs NSAppSleepDisabled -bool YES
- I turned off Spotlight indexer completely - I don't search for files through it nor through Alfred.

      sudo mdutil -a -i off
- Set `core.fsmonitor true` in gitconfig. Add `core.untrackedCache true` to go with `status.showuntrackedfiles all`, otherwise the fsmonitor win is partly eaten by the untracked scan.

- Remove sections from Magit you don't need all the time, e.g.:

     ;; who cares if tags not displayed in magit-refs buffer?
     (remove-hook 'magit-refs-sections-hook #'magit-insert-tags)
There are some other things, but that's what I remember off the top of my head, if I recall some more, will update.

Comment by quibono 2 days ago

Great list, thank you. I recently switched to a Macbook for work and I could feel the difference in speed whenever I used Emacs back on my desktop (Linux).

This helps a bit though magit still feels very sluggish.

Comment by KallDrexx 3 days ago

Interesting thanks I'll give some of these a try.

I know at least some of my slowness is crowdstrike, especially since I can A/B test non-Emacs operations with some coworkers with older machines that don't have the bad config.

Comment by ks6g10 3 days ago

For me it can be slow, especially refreshing magit (which probably is blocked by git, blocked by the bs)(on MacOS)

Comment by ireadmevs 3 days ago

Try checking what M-x magit-toggle-verbose-refresh has to say, you may be able to identify what's slow and optimize around it

Comment by ks6g10 3 days ago

I would guess the roundtrip of intercepting every file stat call in the repo, it's very lovely. But will check.

Comment by KallDrexx 3 days ago

magit is brutal when EDR is involved.

Comment by reddit_clone 3 days ago

MS Defender. I will dump it in a landfill if I could.

Comment by beanjuiceII 3 days ago

you know you can give it exclusions?

Comment by reddit_clone 2 days ago

Yes. But it is locked down in our corporate Macs. Still, one named folder is excluded and we shove _everything_ into that folder.

It still pegs the CPU frequently. It is extremely irritating to open the Activity Monitor and see it at the top.

What good is it in a Mac anyway? This is a not a virus ridden OS like back in the day Windows.