biotop
Linux biotop — Complete Learning Notes & Output Guide
biotopis an eBPF/BCC performance-tracing tool that provides a dynamic, top-like real-time view of block device I/O broken down by process name (command), PID, major/minor device, read/write volume, and throughput.
1. What is biotop?
biotop belongs to the BCC (BPF Compiler Collection) toolkit, originally designed by Brendan Gregg.
While tools like iostat show aggregate device metrics (tps, kB_read/s), and biosnoop lists individual I/O requests as an endless stream, biotop aggregates block-level storage traffic into a refreshed, ranked interactive table (similar to top or htop, but for disk I/O).
It answers critical storage accountability questions:
- Which specific processes or applications are currently driving disk throughput?
- Is a storage performance degradation caused by a database, a backup daemon, a log writer, or an unindexed background script?
- What is the ratio of read bandwidth versus write bandwidth per process?
- How many total I/O operations (IOPS) and Megabytes per second are individual PIDs consuming?
2. Installation
biotop requires root privileges (sudo or CAP_BPF) and kernel development headers matching your running kernel.
Debian / Ubuntu
sudo apt update
sudo apt install bpfcc-tools linux-headers-$(uname -r)
(On Debian/Ubuntu, BCC tools are often named with a -bpfcc suffix: biotop-bpfcc located in /usr/sbin/).
RHEL / Rocky / AlmaLinux / CentOS
sudo dnf install bcc-tools kernel-devel-$(uname -r)
(Executables reside in /usr/share/bcc/tools/biotop).
Arch Linux
sudo pacman -S bcc-tools
Verify the installation:
sudo biotop -h 2>/dev/null || sudo biotop-bpfcc -h
3. Basic Syntax & Flags
sudo biotop [options] [interval] [count]
Essential Command Flags
| Flag | Description | Practical Example |
|---|---|---|
interval |
Refresh interval in seconds (default is 1 second). | sudo biotop 3 |
count |
Number of update cycles before exiting. | sudo biotop 2 10 |
-C, `--noclear`` |
Do not clear the terminal screen between updates (ideal for log piping). | sudo biotop -C 5 |
**-h, --help** |
Display the help message and available options. | sudo biotop -h |
4. Anatomy of the Output Screen
When you run biotop, it clears the terminal and prints a live summary table updated at your specified interval:
sudo biotop 2
Raw Output Example
Tracing block device I/O... Output every 2 secs. Ctrl-C to end.
TIME loadavg: 1.45, 1.20, 1.10 2026-09-11 07:05:12
PID COMM D MAJ MIN DISK I/O KBS WRITE/M LAT(ms)
5410 postgres W 259 2 sda 120 4800 42 2.15
8912 python3 R 259 2 sda 45 1800 0 14.50
1120 rsyslogd W 259 2 sda 8 32 8 42.10
1420 dockerd W 259 3 sdb 2 8 2 1.02
5. Breakdown of Every Output Heading & Field
+------+----------+---+-----+-----+--------+-----+------+---------+---------+
| PID | COMM | D | MAJ | MIN | DISK | I/O | KBS | WRITE/M | LAT(ms) |
+------+----------+---+-----+-----+--------+-----+------+---------+---------+
| 5410 | postgres | W | 259 | 2 | sda | 120 | 4800 | 42 | 2.15 |
+------+----------+---+-----+-----+--------+-----+------+---------+---------+
5.1 Header Metadata
TIME: Current system timestamp of the refresh interval.loadavg: Standard 1-minute, 5-minute, and 15-minute system load averages to correlate disk traffic with CPU demand.
5.2 PID
- Meaning: The operating system Process ID associated with the task responsible for the disk operations. (Note: For kernel background writeback tasks like
jbd2orkswapd, the PID reflects the kernel thread).
5.3 COMM
- Meaning: The short executable command name of the process (derived from
task->comm, truncated to 16 characters).
5.4 D (Direction / Operation Type)
Indicates the dominant block I/O direction during the interval:
R: Read operations dominating.W: Write operations dominating.RW: Mixed read and write activity.
5.5 MAJ & MIN
- Meaning: The Major and Minor device numbers identifying the physical storage partition kernel device object (e.g., Major
259, Minor2corresponds tosda).
5.6 DISK
- Meaning: The human-readable name of the block device handle (e.g.,
sda,sdb,nvme0n1).
5.7 I/O
- Meaning: The total number of block I/O requests (IOPS) issued by this PID during the refresh interval.
5.8 KBS (Kilobytes per Second)
- Meaning: The total data throughput (reads plus writes combined) transferred by the process during the interval, expressed in Kilobytes per second ($\text{KB/s}$).
5.9 WRITE/M (Write Megabytes)
- Meaning: Total volume of data written to disk by this process during the interval, expressed in Megabytes ($\text{MB}$). Helps instantly spot which process is consuming write wear or filling up storage.
5.10 LAT(ms)
- Meaning: The average latency in milliseconds for I/O requests issued by this process during the interval. High latency values highlight processes impacted by storage throttling or slow device queues.
6. How biotop Works Internally
biotop instruments the Linux block layer using eBPF, avoiding the performance penalties of strace or disk polling:
+-------------------------------------------------------------------------+
| KERNEL SPACE |
| |
| 1. Request Issue (kprobe:block_rq_issue): |
| - Intercepts block requests. |
| - Captures: PID, task->comm, device, start timestamp, byte size. |
| |
| 2. Request Complete (kprobe:block_rq_complete): |
| - Calculates request duration delta. |
| - Aggregates metrics per-PID into an in-kernel eBPF Hash Map. |
+-------------------------------------------------------------------------+
|
| Read & Reset Map every N seconds
v
+-------------------------------------------------------------------------+
| USER SPACE |
| Python Script: Ranks PIDs by throughput/IOPS, sorts, clears screen, |
| and formats into a top-like table. |
+-------------------------------------------------------------------------+
7. Real-World Troubleshooting Scenarios
Scenario A: Pinpointing Unknown Disk Write Spikes
iostat reports that disk utilization is pegged at 100%, but you don't know which service is causing it.
Run biotop to rank active processes:
sudo biotop 1
Diagnosis: Watches the live table update every second. The process at the top with the highest KBS or WRITE/M is the culprit. If it's a rogue logging script or an unconstrained database vacuum, you can target it immediately.
Scenario B: Correlating Process Latency with I/O Volume
An application is timing out during heavy batch imports.
Monitor biotop focusing on latency:
sudo biotop 2
Diagnosis: Look at the LAT(ms) column next to the import script's PID. If latency climbs past $50\text{--}100\,\text{ms}$, the import job is saturating the storage controller queue, forcing its own threads into uninterruptible sleep (D state).
8. Important Interview Questions & Answers
Q: What is the primary operational difference between biotop and iotop?
Answer: iotop relies on accounting counters maintained by the kernel (/proc/[pid]/io), which can suffer from update delays, lack fine-grained latency per process, and incur noticeable overhead when scanning all active PIDs on busy systems. biotop uses in-kernel eBPF maps to capture block device I/O events (block_rq_issue and block_rq_complete) directly at the block layer, providing precise IOPS, throughput, and average latency metrics per PID with near-zero overhead.
Q: Can biotop track file paths or individual filenames like opensnoop or ext4slower?
Answer: No. biotop operates at the block device layer (bio), which deals in LBA sectors, disk devices, and raw byte blocks. By the time a write reaches the block layer, pathnames have long been translated into block numbers by the VFS and filesystem layers. To see filenames and paths, use opensnoop, ext4slower, or fatrace.