filelife

@amitmund September 10, 2026

Linux `filelife` — Complete Learning Notes & Output Guide

filelife is an eBPF/BCC performance-tracing tool that tracks the lifespan of short-lived (ephemeral) files. It records when a file is created and when it is deleted, calculating the exact time delta (age) to highlight storage thrashing and unnecessary disk metadata churn.

1. What is `filelife`?

filelife stands for:

File Lifespan Tracker

Part of the BCC (BPF Compiler Collection), filelife instruments the Linux virtual filesystem (VFS) layer inside the kernel using eBPF probes.

It answers critical systems questions:

Which processes are creating and immediately deleting files?
How many seconds or milliseconds do temporary files live?
Is our disk subsystem bottlenecked by rapid inode creation and deletion?
Are lock files, scratch buffers, or IPC pipes lingering longer than expected?
Can short-lived files be moved to a memory-backed file system (tmpfs) to avoid SSD write wear and storage latency?

2. Installation

filelife requires root privileges (sudo) and a kernel with eBPF support enabled (Linux 4.4+).

Debian / Ubuntu

sudo apt update
sudo apt install bpfcc-tools linux-headers-$(uname -r)

(On Debian/Ubuntu, BCC tools are often named with a suffix, e.g., filelife-bpfcc or found in /usr/sbin/filelife-bpfcc).

RHEL / Rocky / AlmaLinux / CentOS

sudo dnf install bcc-tools

(Binaries reside in /usr/share/bcc/tools/filelife).

Arch Linux

sudo pacman -S bcc-tools

Verify:

sudo filelife -h || sudo filelife-bpfcc -h

3. Basic Syntax

sudo filelife [options]

Run live system-wide:

sudo filelife

4. Anatomy of the Default Command Output

When you run filelife, it monitors the system and prints a line each time a tracked file is unlinked from the filesystem:

TIME     PID    COMM             AGE(s)  FILE
14:22:01 4102   python3            0.02  test_session.tmp
14:22:03 8912   gcc                0.45  ccX381a.s
14:22:05 10421  gunicorn           0.00  worker_heartbeat.lock
14:22:12 12040  java              14.20  upload_cache_883.part
14:22:18 15210  rm                 0.05  query_result.csv

5. Breakdown of Every Output Heading & Column

+----------+--------+------------------+---------+-----------------------+
| TIME     | PID    | COMM             | AGE(s)  | FILE                  |
+----------+--------+------------------+---------+-----------------------+
| 14:22:01 | 4102   | python3          |    0.02 | test_session.tmp      |
+----------+--------+------------------+---------+-----------------------+

5.1 `TIME`

Format: HH:MM:SS (wall-clock timestamp).
Meaning: The exact time at which the file was unlinked/deleted by the kernel.
Troubleshooting Significance: Allows immediate correlation with system logs, application exceptions, or sudden spikes in disk %util reported by iostat.

5.2 `PID`

Format: Numeric integer (e.g., 4102).
Meaning: The operating system Process ID that called the unlink or deletion system call (unlink(), unlinkat(), or rmdir()).
Troubleshooting Significance: Identifies the exact offending process. In multi-tenant environments or microservice nodes, this immediately distinguishes background maintenance scripts from active web workers.

5.3 `COMM`

Format: String (e.g., python3, gcc, gunicorn).
Meaning: The executable command name of the thread performing the deletion (derived from task->comm in the kernel task structure, truncated to 16 characters).
Troubleshooting Significance: Reveals what technology stack or utility is causing filesystem turnover without having to manually inspect /proc/<PID>.

5.4 `AGE(s)`

Format: Decimal floating-point number in seconds (e.g., 0.02, 14.20).
Meaning: The duration of existence of the file, measured from the exact moment the file was created (via vfs_create) until it was unlinked (via vfs_unlink).
Troubleshooting Significance:
Near-Zero Lifespan (0.00 – 0.10s): Indicates extreme file churn. These are ephemeral files (locks, temporary buffers, intermediate compiler artifacts) that are hitting physical storage unnecessarily and should be moved to RAM (tmpfs).
Longer Lifespan (> 10.00s): Indicates files that lingered for a batch process or transaction before cleanup.

5.5 `FILE`

Format: String filename or path snippet (e.g., worker_heartbeat.lock, ccX381a.s).
Meaning: The filename component of the deleted dentry passed into the kernel VFS layer.
Troubleshooting Significance: Tells you what internal application routine generated the file (e.g., a .lock file vs. a .part download vs. a .csv export).

6. How `filelife` Works Internally

filelife does not poll the filesystem; it injects eBPF programs directly into Linux VFS entry points:

+-------------------------------------------------------------+
| 1. File Creation Hook: kprobe:vfs_create                    |
|    - The kernel allocates a dentry / inode.                 |
|    - filelife stores: Key = Inode Pointer                   |
|                       Value = Current Timestamp (ns)        |
|    - Stored inside an in-kernel eBPF Hash Map               |
+-------------------------------------------------------------+
                               |
                               v
+-------------------------------------------------------------+
| 2. File Deletion Hook: kprobe:vfs_unlink                    |
|    - The kernel executes the unlink operation.              |
|    - filelife retrieves creation timestamp from hash map.   |
|    - Computes: Delta = Current Time - Creation Time         |
|    - Emits: TIME, PID, COMM, Delta / 10^9, FILE             |
|    - Deletes map entry to prevent kernel memory leaks.      |
+-------------------------------------------------------------+

7. Command Options & Filtering

Flag	Purpose	Practical Example
`-p <PID>`	Trace file lifespans strictly for a specific Process ID.	`sudo filelife -p 4102`
`-d <seconds>`	Filter out short files; show only files older than $N$ seconds.	`sudo filelife -d 1`
`-h`	Display the help menu and supported flags.	`sudo filelife -h`

8. Real-World Troubleshooting Scenarios

Scenario A: High Storage IOPS Caused by Micro-Lock Files

iostat -xz 1 shows high write transactions (w/s), but cumulative throughput (wMB/s) is low.

Run filelife to investigate:

sudo filelife

Output:

TIME     PID    COMM             AGE(s)  FILE
11:02:15 1842   celery             0.00  task_4821.lock
11:02:15 1842   celery             0.00  task_4822.lock
11:02:16 1842   celery             0.00  task_4823.lock

Diagnosis: Celery background workers are creating and unlinking lock files in milliseconds on disk. Fix: Redirect the lock directory to /dev/shm (a memory-backed tmpfs) to eliminate physical disk writes.

Scenario B: Diagnosing Compiler Build Directory Churn

A build server compiling C++/Rust applications suffers from disk saturation during CI/CD runs.

Filter by PID of the compiler build runner:

sudo filelife -p 9210

Diagnosis: Identifies whether intermediate .o, .s, or pipe files are living for milliseconds, indicating that mounting the build target directory in tmpfs will dramatically speed up build times.

9. Important Interview Questions & Answers

Q: What are the performance advantages of `filelife` compared to running `fatrace` or `inotifywait`?

Answer: inotifywait requires setting recursive user watches on directories, which consumes memory and fails if watch limits (fs.inotify.max_user_watches) are hit. fatrace emits notifications for all opens, reads, and writes, causing massive user-space context switching overhead under heavy load. In contrast, filelife runs entirely in kernel space via eBPF, calculates the age delta in-kernel, and only emits output when a file is actually unlinked.

Q: Why might `filelife` miss files that existed before the command was started?

Answer: filelife records file creation timestamps in an in-memory eBPF hash map when vfs_create is triggered. If a file was created prior to launching filelife, the tool has no recorded creation timestamp for its inode in the map. When the file is unlinked, filelife finds no matching key and ignores the event.

Linux filelife — Complete Learning Notes & Output Guide

1. What is filelife?