df and du

Where did all my space go?

Something I find myself doing frequently from the CLI is checking to see how much disk space I have left on my computer. To do this, I use the df command. I pass the -h flag so that the df program will display sizes in “human readable” format, like so:

gabe@xps:~$ df -h
Filesystem     Size    Used   Avail Capacity  Mounted on
/dev/sd0a     1005M   1004M  -49.8M   105%    /
/dev/sd0l     70.1G   24.5G   42.2G    37%    /home
/dev/sd0d      3.9G    4.0M    3.7G     0%    /tmp
/dev/sd0f      2.0G    1.9G   -8.0K   100%    /usr
/dev/sd0g     1005M    232M    723M    24%    /usr/X11R6
/dev/sd0h      9.8G    4.7G    4.6G    51%    /usr/local
/dev/sd0k      2.0G    2.0K    1.9G     0%    /usr/obj
/dev/sd0j      2.0G    837M    1.0G    44%    /usr/src
/dev/sd0e     14.4G   29.9M   13.7G     0%    /var

There’s a lot going on there, lets break it down. The first column gives the “Filesystem”. In this case, those lines represent disk partitions, but they could also be other disks. For example, if I put a usb drive in my computer, that would also show up on the above list. The second column is “Size”, that is the total size of the disk or partition. The third column tells me how much space I am currently using on that disk. The “Avail” column tells me how much free space I have left on the disk (notice in two cases I have negative space!). The “Capacity” column lists how much of the free space I am using. Finally, the “Mounted on” column tells me where in my directory tree I can access this disk or partition.

One thing to notice right away is that I have two partitions that are using more than 100% of the available space. This is a feature of my operating system that we don’t need to worry about just now. The key take away is that I have two partitions that are full.

So, both my / and my /usr partitions are full, is this a big deal? It really depends, in this particular case, for day to day work this won’t really impact me but when it comes time to upgrade my operating system, there is a good chance the upgrade will fail since the /usr partition is where the operating system stores programs and utilities. The fact that / is full is also troubling as this is where the main parts of the operating system are stored. I know from experience with my own computer that 1G is more than enough to hold a full install of the operating system and seeing that it is all used up is very suspicious.

I’d like to figure out what’s using all that space, so I’ll now turn to another program du.

My main concern at the moment is the root filesystem (/) so I’ll check there first. The du command will show me how much space individual files or directories are using, which will allow me to figure out what is causing the problem.

du also has a -h flag, which I pass for “human readable output” as well as the -x flag which tells du to only look at files on the partition or disk that I run it on and not on all disks/partitions.

(As usual, you can see more about the du command by typing man du at the prompt.)

First, I need to change into the directory that is full:

gabe@xps:~$ cd /

Then I execute the du command:

gabe@xps:/$ du -hx
2.0K    ./home
2.0K    ./tmp
2.0K    ./usr
2.0K    ./var
2.0K    ./altroot
5.4M    ./bin
2.0K    ./dev/fd
932M    ./dev
4.0K    ./etc/amd
2.0K    ./etc/authpf
94.0K   ./etc/examples

I’ve trimmed the output, as I can see right away what is causing the problem. The folder ./dev is 932M. That is very strange. Again, from experience I know that the /dev folder is usually very small on my system since there isn’t any data stored there. This directory stores device files which are special files for giving you access to hardware on your machine. In most day to day use you don’t ever have to worry about this folder.

Lets take a closer look at what’s going on. I’ll change into the /dev directory and then use the ls command to list the files there sorted by size, largest files first.

gabe@xps:~$ cd /dev
gabe@xps:/dev$ ls -lS |less
total 1907996
-rw-r--r--  1 root  wheel      529827840 Oct  1 13:24 sd1
-rw-r--r--  1 root  wheel      446758912 Oct  1 17:15 rsd1
-r-xr-xr-x  1 root  wheel          11565 Sep 20 23:09 MAKEDEV
dr-xr-xr-x  2 root  wheel           1024 Sep 21 11:08 fd
lrwxr-xr-x  1 root  wheel              9 May  3 14:10 audioctl -> audioctl0
lrwxr-xr-x  1 root  wheel              6 May  3 14:10 audio -> audio0
lrwxr-xr-x  1 root  wheel              6 May  3 14:10 mixer -> mixer0
lrwxr-xr-x  1 root  wheel              6 May  3 14:10 radio -> radio0
lrwxr-xr-x  1 root  wheel              6 May  3 14:10 sound -> sound0
lrwxr-xr-x  1 root  wheel              6 May  3 14:10 video -> video0
lrwxr-xr-x  1 root  wheel              4 May  3 14:10 pci -> pci0
crw-r--r--  1 root  wheel      83,   0 Sep 21 11:08 apm
crw-r--r--  1 root  wheel      83,   8 Sep 21 11:08 apmctl
crw-r--r--  1 root  wheel      45,   3 Sep 21 11:08 arandom
crw-rw-rw-  1 root  wheel      42,   0 Oct  3 12:38 audio0
crw-rw-rw-  1 root  wheel      42,   1 Sep 21 11:08 audio1
crw-rw-rw-  1 root  wheel      42,   2 Sep 21 11:08 audio2
crw-rw-rw-  1 root  wheel      42, 192 Sep 21 11:08 audioctl0
crw-rw-rw-  1 root  wheel      42, 193 Sep 21 11:08 audioctl1
crw-rw-rw-  1 root  wheel      42, 194 Sep 21 11:08 audioctl2
crw-------  1 root  wheel      79,   0 Sep 21 11:08 bio
crw-r--r--  1 root  wheel      49,   0 Sep 21 11:08 bktr0

Again, I’ve trimmed the output since the relevant information is right on top. I also did something we haven’t seen before, which was to use the | operator. This creates a pipeline between two programs allowing you to send the output of the first program as the input to the next program. I’ll talk more about pipelines next week. In this case, I took the output of the ls program and passed that to the less program so that the output would only be displayed one page at a time instead of scrolling up too fast for me to read.

As a reminder you can page through the output of the less program with the space bar, or move up and down a line using the ‘k’ and ‘j’ keys on your keyboard respectively. To get out of the less program at any time, just hit the ‘q’ key.

So, this output shows me two files that are huge in comparison to everything else in the /dev directory. sd1 is 500M and rsd1 is 400M. There is something very wrong with that. Neither of these files should exist at all, and the fact that they are here tells me that I’ve make a mistake at some point and created a file where I shouldn’t have. Deleting these files should fix the issue I’m having. The only problem is these files aren’t mine, and so the system won’t let me delete them:

gabe@xps:/dev$ rm sd1
override rw-r--r--  root/wheel for sd1? y
rm: sd1: Permission denied

This is a good thing, because if I don’t know what I’m doing I could seriously mess up my computer.

We’ll revisit this issue at a later date when we have a few more tools under our belts. For now, I’ll live with the fact that my / filesystem is full since my computer seems to be working fine otherwise.

What can you find out about your system using the df and du commands?

Can you find out what directory is using the most space in your home folder?

Don’t try to run the rm program on anything just yet, if you delete something using the command line, it’s gone forever.

New Terms

  • partition - A logical slice of a physical disk. Say I have a 1TB drive, I can tell the operating system to divide that drive up into many smaller slices that I can then treat as completely independent disks.
  • less - A program to display the contents of a file on a page by page basis.
  • pipeline - A tool that allows you to send the output of one program as the input to another program.
  • filesystem - An individual formatted partition of a disk.
  • du - A program for displaying disk usage information on a file or directory level
  • df - A program for displaying disk usage information on the filesystem level
  • /dev - A special directory the operating system uses to give you access to your hardware.
  • /usr - A special directory where the operating system installs tools and utilities you can use.