Technology
What Is a File Descriptor? The Number Behind Every Open File
What Is a File Descriptor? The Number Behind Every Open File
In Everything Is a File, I kept saying that Unix lets your keyboard, your network connections, and your disk all answer to the same handful of operations. What I skipped over is how your program actually holds on to one of those things once it's open.
The answer is almost disappointingly simple: it's a number. A small, non-negative integer the kernel hands back when you open something. That number is a file descriptor, and every read(), write(), and close() your program ever makes is aimed at one.
The short version
When a program calls open("notes.txt"), the kernel does the real work — finds the file, checks permissions, sets up bookkeeping — and then gives the program back a number like 3. From then on, the program never says "notes.txt" again. It says "read from 3" or "write to 3," and the kernel knows what that means.
You can see this from Python:
f = open("notes.txt")
print(f.fileno()) # probably 3
Why 3 and not 0? Because 0, 1, and 2 are already taken before your program runs a single line.
0, 1, and 2: the ones you get for free
Every Unix process starts life with three file descriptors already open:
- 0 — standard input (stdin): where the program reads input from
- 1 — standard output (stdout): where normal output goes
- 2 — standard error (stderr): where error messages go
When you run a command in a terminal, all three point at that terminal. That's why typing goes into the program and output comes back to your screen — no special terminal API involved, just three file descriptors that happen to point at the same device.
The kernel also has a rule that makes the numbering predictable: open() always returns the lowest unused number. With 0, 1, and 2 taken, the first file you open gets 3, the next gets 4, and so on. If you close 3 and open something new, you get 3 again.
Seeing them for yourself
On Linux, every process has a directory under /proc listing its open file descriptors. $$ is your shell's own process ID, so:
$ ls -l /proc/$$/fd
lrwx------ 1 duhon duhon 64 Oct 21 10:02 0 -> /dev/pts/0
lrwx------ 1 duhon duhon 64 Oct 21 10:02 1 -> /dev/pts/0
lrwx------ 1 duhon duhon 64 Oct 21 10:02 2 -> /dev/pts/0
lrwx------ 1 duhon duhon 64 Oct 21 10:02 255 -> /dev/pts/0
There they are: 0, 1, and 2 all pointing at your terminal (/dev/pts/0). The 255 is Bash keeping its own private copy of the terminal open, out of the way of anything your scripts might use.
On macOS there's no /proc, but lsof -p $$ shows the same information.
You can open your own file descriptors right from the shell, too:
exec 3> log.txt # open log.txt for writing as fd 3
echo "hello" >&3 # write to fd 3
exec 3>&- # close fd 3
That >&3 syntax — "send this to file descriptor 3" — is the same idea behind the famous 2>&1, which deserves a post of its own.
What the number actually points to
The number is just an index. Behind it, the kernel keeps two layers of bookkeeping:
- A per-process table mapping each number to an entry in the kernel's list of open files.
- The open file itself (the kernel calls it an "open file description"), which tracks things like your current position in the file and whether you opened it for reading, writing, or appending.
That split matters more than it looks. Two file descriptors can point at the same open file — and when they do, they share a position. Read 100 bytes through one, and the other has moved forward 100 bytes too.
This happens all the time without you asking for it. When a process calls fork() to create a child, the child gets a copy of the parent's file descriptor table, pointing at the same open files. That's how a shell can launch a command that writes straight to your terminal: the command inherited the shell's fd 1.
Why this explains so much
Once file descriptors click, a lot of everyday Unix stops being magic:
- Pipes:
ls | grep notesworks because the shell creates a pipe and wiresls's fd 1 to one end andgrep's fd 0 to the other. Neither program knows the other exists. - Redirection:
> out.txtjust means "before running this command, make fd 1 point at out.txt." - "Too many open files": there's a cap on how many descriptors a process can hold — check yours with
ulimit -n. A program that opens files or sockets without closing them eventually hits it and crashes with this error. - Disk space that won't come back: delete a huge log file while a process still has it open, and the space isn't freed. The file has no name anymore, but it still has an open descriptor, so it still exists.
ls -l /proc/<pid>/fdwill show it marked(deleted). Restart the process or close the descriptor and the space comes back.
That last one has saved me real time. "I deleted the file and the disk is still full" sounds like a bug until you remember the kernel keeps open files alive as long as someone's holding one.
Where the idea gets messy
File descriptors are simple, but a few sharp edges are worth knowing:
- The numbers mean nothing outside your process. Your fd 3 and another program's fd 3 are completely unrelated. (There is a way to pass a descriptor between processes over a Unix socket, but it's a special mechanism, not something you get by sharing the number.)
- Numbers get reused immediately. Because
open()always hands out the lowest free number, closing a descriptor twice can be dangerous in threaded programs — by the secondclose(), that number may already belong to a different file another thread just opened. - Child processes inherit everything. By default, programs started with
execkeep all of the parent's open descriptors, which can leak files or sockets into processes that shouldn't have them. That's why theO_CLOEXECflag exists: "close this automatically when running a new program." - It's not universal. Windows uses "handles" — a similar idea built differently. Most cross-platform languages hide both behind a file object, which is why you rarely see the raw number.
Why it's worth knowing
You can write code for years without ever touching a raw file descriptor. But the moment something goes wrong at the edges — a stuck pipe, a leaked socket, a disk that won't free up, a "Too many open files" crash in production — the explanation is almost always sitting in that little numbered table.
And it's the missing piece of "everything is a file." Files, terminals, pipes, and sockets can all share one interface because, from your program's side, they all collapse into the same thing: a small number you can read from and write to.