WSL 2 is getting faster Windows file system access
Posted by haydenbarnes 5 days ago
Comments
Comment by avaer 5 days ago
WSL singlehandedly stemmed much of tide of developers moving away from Windows, but WSL native filesystem performance gave devs that magical experience when they boot into Linux the first time and see that the filesystem doesn't have to be ass. There's always been hacks around this, but for many devs the easiest hack was to ditch Windows.
They should have moved heaven to fix this on day one, there's really no engineering excuse. Linux is open source.
Comment by kaelwd 4 days ago
Comment by SomeUserName432 4 days ago
By default setting, windows defender will upload "suspicious files" to Microsoft.
A while back I caught it trying to nab my OpenVPN installer which also contains the certificates.
Comment by alyandon 3 days ago
Comment by pjmlp 4 days ago
Comment by thewebguyd 4 days ago
Comment by Max-Limelihood 4 days ago
I don't think there's any company that's done more to support transparent open-source software except Red Hat. Hell, in true Google style, they've built not one, not two, but four separate FOSS operating systems (Android, ChromiumOS, Fuchsia, and at this point it's time to admit Chromium itself has become an OS-within-an-OS), as well as being the second-biggest contributor to Linux.
Comment by Max-Limelihood 4 days ago
I don't think there's any company that's done more to support transparent open-source software except Red Hat. Hell, in true Google style, they've built not one, not two, but four separate FOSS operating systems (Android, ChromiumOS, Fuchsia, and at this point it's time to admit Chrome has become an OS-within-an-OS), as well as being the second-biggest contributor to Linux.
Comment by skissane 4 days ago
This isn't the only issue. I think another big issue is pushing more and more integration with Microsoft cloud services (e.g. Microsoft accounts), advertising, etc, which Microsoft has made increasingly difficult to opt-out of. They could fix every single technical limitation anyone has ever complained about, but if they don't change their corporate culture on forced cloud/advertising/etc, many won't care about those fixes.
Comment by thewebguyd 4 days ago
They could have, instead, made a standalone ISO of Windows with all the crap already stripped out. Instead, they are still treating the crap-filled version as the default with an option to strip it down.
If they wanted to correct their reputation, it needs to be the other way around. Ship the stripped down version as standard, and make the crap opt-in.
Comment by overgard 4 days ago
Comment by keithnz 4 days ago
Comment by jauntywundrkind 4 days ago
There's very little fiddling around or configuring. 30% sounds god awful terrible; my success rate definitely >85%. In the rare case something doesn't work right away, https://www.protondb.com/ usually has advice in the top or second comment that works great.
I don't really think the windows vs Linux native debate is worth pursuing. Windows games run better than they do on Windows 4 times out of 5, and that's more than good enough.
Comment by saint_yossarian 4 days ago
For some reason certain games default to specific older Proton versions. And then of course there are a bunch of older games with a native Linux port that is now unmaintained and stopped working at some point, but they still work just fine with Proton :)
Comment by poink 4 days ago
Comment by thewebguyd 4 days ago
Comment by Fire-Dragon-DoL 4 days ago
Comment by mzajc 4 days ago
Comment by Fire-Dragon-DoL 4 days ago
Comment by suddenlybananas 4 days ago
Comment by Kenji 5 days ago
Comment by evanjrowley 5 days ago
Comment by oofbaroomf 5 days ago
Comment by evanjrowley 4 days ago
Comment by phowat 5 days ago
Comment by ragequittah 5 days ago
I can hear a subset of people cringe saying "but LLMs are BS machines and you aren't learning anything!" I heartily disagree on both fronts. The main thing holding users like me back from linux was always the snarky RTFM community and the fact that everything has 25 different answers (depending on distro, window manager, and many other factors). LLMs take care of all this friction for you very nicely.
Comment by CamperBob2 4 days ago
It is truly a Star Trek-level experience. Nobody who doesn't want to run Windows (and who isn't forced to run it) needs to run Windows anymore.
Comment by markstock 4 days ago
Comment by IshKebab 4 days ago
And don't say "it works for me". We know some people get very lucky with Linux and stuff just works. In my experience the typical experience is that stuff works much better on Windows than Linux.
E.g. something Windows figured out in 2000 - what happens when your system is low on ram or overloaded? Simple! Press ctrl-alt-del, it will pause other apps and allow you to open the task manager and choose one to kill.
On Linux? There's no task manager in the ctrl-alt-del menu (on KDE anyway), and even if there was it isn't a specially privileged UI so it wouldn't respond. Running low on RAM? No problem we'll just kill a random process and if that doesn't work (it usually doesn't), completely freeze and then hard-reboot. Yeay.
Comment by hootz 4 days ago
On Linux, KDE has the plasma System Monitor, and if you can't use it for some reason you can switch your entire session to a TTY with CTRL+ALT+F2 and kill any process you want.
Comment by IshKebab 4 days ago
How intuitive. That's often disabled, and I've definitely had machines that were so frozen even that didn't work. I guess sysrq keys might have worked but if you're seriously going to suggest that you haven't understood the problem.
> The Task Manager that many times also stopped responding or was permanently stuck behind a frozen full screen game?
Hasn't happened once for me for as long as I can remember.
In fairness Linux has been pretty solid for me too after upgrading to 128GB of RAM and 64GB of swap. But I never needed to do that on Windows.
Comment by thewebguyd 4 days ago
I've had it happen to me, but the last time it did was probably around ~2017 or so on spinning rust. Fast SSDs make it a non-issue now.
Comment by 2001zhaozhao 4 days ago
This could mean that at some point, ironically Linux becomes the most user-friendly operating system for regular people as their AIs can diagnose and fix system issues and problems for them more easily than they can with proprietary operating systems.
Comment by hgoel 5 days ago
Comment by kenz0r 5 days ago
Comment by ryanshrott 4 days ago
Comment by avadodin 5 days ago
Hopefully, they will just push it out to everyone asap. We make heavy use of symlinks into Windows drives.
Comment by chris_money202 5 days ago
Comment by alberth 5 days ago
Comment by omcnoe 5 days ago
The mount is fine and speedy enough, but the underlying reads/writes turn into native NTFS reads/writes through Windows. NTFS file API is incredibly slow - high fixed overhead for initial file access.
So patterns like node_modules with many small individual files (or compiling code in general) are much much slower on Windows or WSL2 /mnt/c due to the fixed overhead adding up over a large number of files.
It's a ridiculous problem that has plagued Windows for years.
Comment by 0x1d7 4 days ago
NTFS itself is a fast file system.
Comment by throwa356262 4 days ago
File access is significantly slower on Windows compared to Linux on the same hardware. You can run Linux inside a VM and it will easily beat the host OS in filesystem performance.
If the problem is not NTFS, then it is in how the Windows OS uses it
Comment by tssva 4 days ago
Isn't that exactly what the comment you are replying to is saying?
Comment by 0x1d7 3 days ago
This is why 'DevDrive' exists for ReFS, to reduce that penalty (though ReFS comes with it's own, namely in write performance due to journaling).
The model wasn't a big deal back in the day of IDE and SCSI, but it certainly shows on flash storage.
Comment by NoGravitas 4 days ago
Comment by wahern 4 days ago
Comment by Fire-Dragon-DoL 4 days ago
I am familiar with the issue, doing any sort of ruby development is a nightmare on windows because each require loads a file becoming increasingly slow at boot time
Comment by torginus 4 days ago
Comment by NoGravitas 4 days ago
Comment by torginus 4 days ago
Comment by tonymet 5 days ago
the rule of thumb without the newest features is to copy work to/from /mnt/c into $HOME as needed.
Comment by weird-eye-issue 5 days ago
Also if you try running Next js from files on Windows from WSL it takes minutes for each page to compile to the point that any local development is impossible so you would have to either run the Next JS server on Windows or move the files to WSL
Comment by rahkiin 4 days ago
Comment by DanielHB 4 days ago
It also has a lot of problems with locking files that are open by a process, if you have a rogue process reading your node_modules npm install or rm node_modules can hang until that process finishes.
yarn2 keeps dependencies as tarballs instead of extracting them to disk, imported files get loaded from the tarball at runtime. Makes a massive difference in windows.
Comment by yakz 5 days ago
Comment by zaptheimpaler 5 days ago
Comment by thesis 5 days ago
Comment by weird-eye-issue 5 days ago
Comment by hparadiz 5 days ago
Comment by weird-eye-issue 5 days ago
Comment by markstock 4 days ago
Comment by weird-eye-issue 4 days ago
Comment by pjmlp 4 days ago
Comment by hparadiz 5 days ago
Yea bro totally. Totally. I'm gonna copy 2TB of media into the WSL virtual disk just so ffmpeg can run a little faster but still way slower than simply running linux.
(I beta tested the shit out of WSL1 and 2) before I wised up and just installed Gentoo forever.
Comment by weird-eye-issue 5 days ago
But either way yeah most people aren't dealing with large media libraries that's obviously a little more difficult. But if you are primarily operating on them with WSL then you would just keep them in the WSL file system and you could access them from Windows whenever you need to...
Comment by benatkin 4 days ago
Comment by pjmlp 4 days ago
Comment by hparadiz 4 days ago
This ladies and gentleman is the problem with discussing Windows design patterns on the internets. They say "use this" and you say "well it's broken in X, Y, Z" ways and instead of fixing it they say "you're using it wrong". No. Maybe it's just an inferior architecture.
Comment by pjmlp 4 days ago
Comment by satvikpendem 4 days ago
Comment by hparadiz 4 days ago
Comment by satvikpendem 4 days ago
Comment by ChrisRR 4 days ago
Comment by hparadiz 4 days ago
The entire point of wsl is to be able to run code compiled for Linux on Windows.
ffmpeg underpins so many things these days. It could be used to extract frames in a PHP based website, convert something to a gif, or a demux an mkv. You may as well be asking me why I'm using a computer.
Comment by DanielHB 4 days ago
If you build 100+ megabytes images several times per day it really sucks.
Comment by nozzlegear 5 days ago
Comment by weird-eye-issue 5 days ago
Either you run npm install from Windows if you are operating on the Windows file system or you run it on WSL if you are operating on the WSL file system both cases will be very fast
Comment by lelandbatey 5 days ago
Comment by hparadiz 5 days ago
Comment by weird-eye-issue 5 days ago
Comment by hparadiz 5 days ago
Comment by weird-eye-issue 5 days ago
Comment by k4rli 4 days ago
Even with tiled windows I haven't had any game crash like that once. "alt-tab" equivalent takes 1ms and it just works. I can throw around the game window between workspaces, resize etc.
It's worth giving it a try. Unfortunate if games with certain AC setups are wanted, like GTA:O or LOL, but I can live without them.
Comment by hparadiz 5 days ago
Comment by weird-eye-issue 5 days ago
Also how can drivers be easier to install than on Windows when updating my GPU driver is one click?
Comment by hparadiz 5 days ago
Comment by dwattttt 4 days ago
Comment by hparadiz 4 days ago
Ya'll will complain about anything.
It is a live USB you can game on. You don't even really need to install it.
Comment by thewebguyd 4 days ago
Comment by weird-eye-issue 4 days ago
Comment by vladvasiliu 4 days ago
At least now this sometimes works if I turn the laptop on with the screen plugged-in. If I go to the toilet and the screen turns off, it's back to some low resolution. When my computer was new 5 years ago, it never did work in 5k, so... baby steps, right?.
Worked perfectly OOB on Linux.
Comment by cyberax 4 days ago
Comment by weird-eye-issue 4 days ago
Comment by cyberax 4 days ago
Comment by weird-eye-issue 4 days ago
Comment by torginus 4 days ago
Comment by rounce 4 days ago
Comment by torginus 4 days ago
I don't know the full story, but afaik, AMD/Intel Mesa based drivers are fully open source and are built in a much more Linux-native way. But unfortunately hardware choice is out of my jurisdiction.
Comment by rounce 2 days ago
Comment by lmm 5 days ago
Huh? It's the same driver. It works the same on every platform. There's no consistent difference in performance (at least not between FreeBSD and Windows, it's been a while since I ran Linux).
Comment by satvikpendem 4 days ago
Comment by pjmlp 4 days ago
Comment by flaunf221 5 days ago
Comment by thewebguyd 4 days ago
But yeah sure, no one runs windows these days.
Comment by markstock 4 days ago
Comment by pjmlp 4 days ago
Comment by thewebguyd 4 days ago
And there's still a big niche that Windows is your only choice since the move to Apple silicon. If you need both a dGPU and access to commercial software, its literally your only choice. Game dev especially comes to mind if you're jumping between maya, after effects, etc. Windows is also huge in finance.
Comment by rahkiin 4 days ago
Comment by pjmlp 4 days ago
Comment by hparadiz 4 days ago
The top compositors right now for desktop are KDE Plasma, Hyprland, Gnome/Mutter and all of them implement the Wayland protocol independently.
For game wrappers Valve makes Gamescope (https://github.com/ValveSoftware/gamescope) which is also a Wayland compositor.
When you run a game through a wrapper like GameScope it will draw to the Wayland Server that GameScope is running and then that subsequently writes to the parent display server (which can actually be X or Wayland).
Anyway it's a far superior and more secure protocol than whatever Windows is doing and you should for sure have ChatGPT explain it to you.
Comment by pjmlp 4 days ago
Gemini told me it was a good decision.
Comment by nozzlegear 4 days ago
Comment by weird-eye-issue 4 days ago
Comment by nozzlegear 4 days ago
Comment by asveikau 4 days ago
Comment by josh3736 4 days ago
WSL1's file performance is pretty much as good as it gets on Windows, since open(2), read(2), etc are all translated directly in-kernel from Linux to Windows API calls. It's still slower than a real Linux kernel since Windows' filesystem filter drivers add a lot of overhead to every operation, and Windows Defender and its realtime scanning in particular makes it 10x worse. (NTFS itself is fine.)
WSL1's filesystem situation is now "fixed" by Dev Drive, which is just a new partition with most filters disabled and Defender is put in to a different mode where scanning is asynchronous instead of blocking every open(2).
WSL2's mounts of Windows disks still has to deal with all of the above, plus the overhead of serializing every operation over a VM socket, which is largely fixed by what this article describes. So even if you've enabled virtiofs to speed up WSL2's cross-VM transfers, you're still going to hit the same Windows filesystem caveats that apply to WSL1 and native Windows apps.
On the other hand, the WSL2 in-VM ext filesystem (ie, the / mount) will be the fastest to Linux apps since it never touches the Windows side, but accessing those files from Windows sucks since they're buried in a VHD image, accessed over a slow 9p (I think) network-like mount on \\wsl.localhost\distro\
All of that to say filesystem perf may or may not have been a factor in the switch from WSL1's very cool NT persona architecture to WSL2's decidedly more boring VM design, but it was pretty clear that Docker was the real showstopper.
Devs needed to run containers, and WSL1 couldn't (and still can't) do it.
I imagine Microsoft took a look at what it would take to implement container support in the NT kernel to the point that Docker would work, and decided it was simply too much work when they could just slap a VM in and get the entire Linux kernel API surface for free. So thus we got WSL2.
I still use WSL1 for light work like running ssh, since there's basically no overhead at all (5 MB of RAM total to run ssh), compared to needing to run an entire second OS in WSL2. And as long as you don't need any containers, even heavier work runs nicely on WSL1 since there's no VM overhead or network NAT shenanigans.
But if you need containers, you need WSL2.
Comment by fragmede 4 days ago
Are there a lot of Microsoft operating systems that feature NTFS without filesystem drivers and Windows Defender?
Comment by josh3736 4 days ago
Filter drivers sit a layer above the filesystem driver and allow you to hook file operations to do things like antivirus scanning, transparent encryption and compression, realtime backups, and implement virtual files (à la Dropbox and OneDrive cloud files that are deleted from local storage and JIT downloaded when accessed).
Those are all useful features, but you pay for the extensibility with performance.
To answer your question, obviously no—at least not in a default configuration—but all that stuff can be disabled if you're so inclined, which would leave you with a Microsoft operating system featuring NTFS without the filters and Defender.
But I'm not sure what point you're trying to make. Different operating systems make different trade-offs?
Comment by fragmede 4 days ago
Comment by torginus 4 days ago
It is what it is. My workflow is that I rsync the relevant files from Windows to Linux, do the work, and rsync them back. It's clunky but works well enough.
Comment by asveikau 4 days ago
This contrast is somewhat true of FAT but not NTFS.
Which is to say, in FAT, the direntry defines the first entry in the cluster chain, and the file size. But NTFS is not like this. There is a concept of inodes, which they call file IDs. Hardlinks are also a thing. You can also create a file, then delete its name, but leave the handle to the inode open. You can also open by file id.
Comment by JdeBP 4 days ago
Comment by anamax 4 days ago
However, the shell for WSL2 runs in a window that grabs things, such as ^V.
So, ssh from WSL2 (to AWS for example) is awkward. For exampl, Emacs on the AWS box is almost unusable.
Comment by foepys 4 days ago
https://github.com/microsoft/terminal
winget install --id Microsoft.WindowsTerminal -eComment by anamax 2 days ago
And if it works, I'll wonder why microsoft doesn't make the ubuntu button use terminal.
Comment by themeiguoren 5 days ago
And for what it’s worth, that version isn’t available yet when I try to update WSL.
Comment by tonymet 5 days ago
WSL filesystem = 50s
virtiofs = 75s
it seems a lot faster. I don't use drvfs (windows files) usually. 50% performance gap is good.
And MS Defender kicked in beforehand losing about 10 seconds. with some config you could turn off defender and get this down to 60s
Comment by cheema33 4 days ago
> 50% performance gap is good.
I am somewhat confused by this. How much time did the default DrvFs take? Or did you mean DrvFs when wrote virtiofs?
Comment by tonymet 4 days ago
The results above are wsl vhd vs virtiofs, the new file system for accessing windows files . Plan9 /9p is the old one , not tested .
Comment by tonymet 4 days ago
Comment by tonymet 5 days ago
https://gist.github.com/tonymet/ec7fc4af0eb11c9d5af22c76d056...
Comment by tonymet 5 days ago
Comment by rtpg 4 days ago
I don't get how people are so comfortable with dev tooling being as busted as it is.
Comment by rcleveng 4 days ago
Comment by politelemon 4 days ago
Comment by cmovq 5 days ago
Comment by bitwize 4 days ago
Comment by tonymet 5 days ago
Comment by thot_experiment 4 days ago
Comment by ta8903 4 days ago
Comment by flumpcakes 4 days ago
The new container capability also looks magical, and being able to access that directly from windows apps seems pretty cool too. Not sure it would be worth the time investment vs. docker desktop, but interesting none the less.
Comment by psyclobe 4 days ago
Was really disappointed when WSL2 came along and just virtualized everything; fs performance took a shit.
Anyway I don't care about windows these days, all Linux all the time.
Comment by agentultra 5 days ago
Comment by ranger_danger 4 days ago
Comment by sanp 5 days ago
Comment by tonymet 5 days ago
with the new wsl version, kernel and config from that blog, it will mount virtiofs, or warn you
Comment by mattkevan 5 days ago
Comment by protocolture 5 days ago
Comment by phendrenad2 5 days ago
Comment by bitwize 4 days ago
( ) Shut down the computer?
( ) Restart the computer?
(*) Restart the computer in Linux mode?
( ) Close all programs and log on as a different user?
Comment by charcircuit 4 days ago
Comment by xela79 4 days ago
Comment by tonymet 4 days ago
Comment by tonymet 4 days ago
Comment by thomasswilliams 4 days ago
Comment by p-t 4 days ago
Comment by rdwrrr 4 days ago
Comment by hiAndrewQuinn 4 days ago
Comment by epolanski 4 days ago
Adobe suite (I used Figma and Adobe tools in my webdev career), PDF editing is a huge issue and lackluster on Linux, most of the office suite is not available natively on Linux (no, there's no valid replacements when your company is deep into VBA tooling, etc), PowerBI, but also many specific internal tools that many businesses have and have been developed as Windows-only applications, I had anything ranging from specific VPNs to screen sharing utils. But there's also more dev-focused tooling, from game development frameworks like Unreal Engine (for which I had to develop UIs in the past) and the related Visual Studio to FPGA tooling and USB debugging.
The list is long and generally based on your domain, but in a career I had almost always moments where I could not avoid having a Windows machine.
WSL allows you to generally have both of the words.
I don't dislike Windows in general to be honest, Power Toys, WSL, a great Terminal, it's an okay machine to develop and I like it better than MacOS from which I still have nightmares for being locked out of my hardware for 2 months due to an apple account issue.
Comment by rdwrrr 1 day ago
Comment by beart 4 days ago
Comment by satvikpendem 4 days ago
Comment by pjmlp 4 days ago
WSL saves me to install them.
Additionally, I only care about running Linux containers, everything else I run on native Windows, so the filesystem performance isn't an issue.
Comment by bel8 4 days ago
Comment by pdantix 4 days ago
Comment by NoGravitas 4 days ago
Comment by tonymet 4 days ago
Comment by ernsheong 4 days ago
Comment by jstrebel 4 days ago