What is the purpose of the lost+found folder in Linux and Unix? (2014)

ZeroFS ended up not needing recovery at all through atomic, strictly ordered commits [1], but it was far from trivial (and not just a matter of requiring a WAL).

[0] https://github.com/Barre/ZeroFS

[1] https://github.com/Barre/ZeroFS/blob/main/zerofs/src/fs/writ...

Based on comments in the kernel source, it seems like the userspace fsck for JFS and F2FS will also sometimes create /lost+found. There might be more that do.

  "52 UNIX FOR SUPER-USERS
  ...
  The filestore consistency check is performed by the command fsck (usu-
  ally stored as /etc/fsck, but sometimes in /bin), which should be used to check
  all discs used as file-systems. It defaults to the list of filestore devices given
  in the file /etc/checklist. At this stage, most of the file-systems will not be
  mounted, so will be inactive; only the root file-system will be active. The
  fsck command goes through each one in turn, reports any inconsistencies in
  them, and offers to correct them. The reply to each query is either 'y' for yes
  (correct the inconsistency), or 'n' for no (leave the file-system inconsistent).
  A parameter '-y' to the command assumes 'yes' replies to all questions, so
  that no further interaction is necessary; a parameter '-n' similarly assumes
  all 'no' answers, and therefore needs no write permission to the device. Any
  'yes' reply may involve the loss of information, such as the complete removal
  of a suspect file. Suspect files on the file-system being checked are written to
  a directory lost + found on the device if such a directory exists; this directory
  must have been created, and be sufficiently large already to hold the names
  of all the files involved. This can be ensured by first creating the directory,
  then creating a number of files in the directory, and then removing them.
  The corrected systems will be consistent, and can later be mounted as and
  when required. It may be possible to recover information from deleted files
  by looking at the lost + found directory. There should be a lost + found direc-
  tory at the head of each mountable file-system.
  When checking the root file-system, there are complications, in that it
  will be active (even though, because it is root, it will not be formally mounted
  as such, but is implicitly mounted as root during the booting process). If
  modifications are necessary, they should be completed, and the machine
  rebooted without first performing a sync (see section 4.5 below for the nor-
  mal procedure for taking a system down). This is to ensure that the disc as
  modified by fsck is not overwritten by any in-core information, which may
  have been generated from information read from the original corrupt (incon-
  sistent) version.
  ...
  "

Old time UNIX systems directories were just another structured file, which a 'd' bit (like others you changed with chmod) on them, where each record was 16 bytes, which 2 first were the inode number followed by 14 bytes reserved for filename. IIRC linux also had first same limit first filesystems. You could read directory with any program, common feat was to check any odd stuff that "ls" would not show with could hexdump or "od" with some flags and print 16 bytes lines per row. That way you also could see any deleted or moved files from that directory, because directory entry was not quite long otherwise cleared but just clearing that inode reference two bytes.

A Quick look from other books that I have close me now Maurice J. Bach, The Design of Operating UNIX System (-86) contains much more about fsck and filesystem fixing issues scattered few pages in the book and what methods were used to mitigate loss of data. S.R Bourne The Unix System, no mention of fsck at all, at least by looking book automatically generated index.

It could have been some other quite old book I did read, but did not own or anything since BSD4.3-tahoe documentation I've read over the years. But sure it would be nice to read that exact reasoning again from credible sources.

edit: Oh, and you could preallocate also just by adding entries or copying some data to lost+found enough, and then remove entries. Unix traditionally have not compacted and resized directories. They only grow and can be if have been very large slow to traverse. The way to compact is creating another, moving existing data there and then swapping directories.

fsck on large hard drives was scary on how long it could take to finish.

At one point, I had one where the directory structure was completely broken and had circles in it (broken SSD). To be fair, in that particular case, I did not look for lost+found and just wrote a tool to extract the data manually that I was looking for.

I saw it often. Running fsck on a crashed/failed partition usually does put some files in there. Maybe I kept using old hardware too long...

On the UFS and suchlike filesystems, at the point that fsck is rescuing orphaned i-nodes, it still has not fully gone through the process of checking and correcting free list information, or indeed fully eliminating errors from the i-node table. Creating a directory involves allocating a new i-node from an unused slot, and free blocks off the free block list.

Ironically, because they are slightly or grossly different to Unix filesystem formats, on HPFS and FAT this is less of an issue. (FAT usually has unused slots in the root directory that it is sane to use at that point, for example.) CHKDSK on OS/2 did create its \FOUND.nnn files on the fly.

       mklost+found pre-allocates disk blocks to the lost+found directory
       so that when e2fsck(8) is being run to recover a file system, it
       does not need to allocate blocks in the file system to store a
       large number of unlinked files.  This ensures that e2fsck will not
       have to allocate data blocks in the file system during recovery.

[0] https://man7.org/linux/man-pages/man8/mklost+found.8.html

https://sherwood.news/tech/stack-overflow-forum-dead-thanks-...

https://news.ycombinator.com/item?id=48282709

Though it could be less, cause I also remember recovering files with foremost on a dd copy of a disk.

I also respect human responses over AI ones every day that ends in Y.