df and du

Where did all my space go?

Something I find myself doing frequently from the CLI is checking to see how much disk space I have left on my computer. To do this, I use the df command. I pass the -h flag so that the df program will display sizes in “human readable” format, like so:

gabe@xps:~$df -h Filesystem Size Used Avail Capacity Mounted on /dev/sd0a 1005M 1004M -49.8M 105% / /dev/sd0l 70.1G 24.5G 42.2G 37% /home /dev/sd0d 3.9G 4.0M 3.7G 0% /tmp /dev/sd0f 2.0G 1.9G -8.0K 100% /usr /dev/sd0g 1005M 232M 723M 24% /usr/X11R6 /dev/sd0h 9.8G 4.7G 4.6G 51% /usr/local /dev/sd0k 2.0G 2.0K 1.9G 0% /usr/obj /dev/sd0j 2.0G 837M 1.0G 44% /usr/src /dev/sd0e 14.4G 29.9M 13.7G 0% /var  There’s a lot going on there, lets break it down. The first column gives the “Filesystem”. In this case, those lines represent disk partitions, but they could also be other disks. For example, if I put a usb drive in my computer, that would also show up on the above list. The second column is “Size”, that is the total size of the disk or partition. The third column tells me how much space I am currently using on that disk. The “Avail” column tells me how much free space I have left on the disk (notice in two cases I have negative space!). The “Capacity” column lists how much of the free space I am using. Finally, the “Mounted on” column tells me where in my directory tree I can access this disk or partition. One thing to notice right away is that I have two partitions that are using more than 100% of the available space. This is a feature of my operating system that we don’t need to worry about just now. The key take away is that I have two partitions that are full. So, both my / and my /usr partitions are full, is this a big deal? It really depends, in this particular case, for day to day work this won’t really impact me but when it comes time to upgrade my operating system, there is a good chance the upgrade will fail since the /usr partition is where the operating system stores programs and utilities. The fact that / is full is also troubling as this is where the main parts of the operating system are stored. I know from experience with my own computer that 1G is more than enough to hold a full install of the operating system and seeing that it is all used up is very suspicious. I’d like to figure out what’s using all that space, so I’ll now turn to another program du. My main concern at the moment is the root filesystem (/) so I’ll check there first. The du command will show me how much space individual files or directories are using, which will allow me to figure out what is causing the problem. du also has a -h flag, which I pass for “human readable output” as well as the -x flag which tells du to only look at files on the partition or disk that I run it on and not on all disks/partitions. (As usual, you can see more about the du command by typing man du at the prompt.) First, I need to change into the directory that is full: gabe@xps:~$ cd /


Then I execute the du command:

gabe@xps:/$du -hx 2.0K ./home 2.0K ./tmp 2.0K ./usr 2.0K ./var 2.0K ./altroot 5.4M ./bin 2.0K ./dev/fd 932M ./dev 4.0K ./etc/amd 2.0K ./etc/authpf 94.0K ./etc/examples  I’ve trimmed the output, as I can see right away what is causing the problem. The folder ./dev is 932M. That is very strange. Again, from experience I know that the /dev folder is usually very small on my system since there isn’t any data stored there. This directory stores device files which are special files for giving you access to hardware on your machine. In most day to day use you don’t ever have to worry about this folder. Lets take a closer look at what’s going on. I’ll change into the /dev directory and then use the ls command to list the files there sorted by size, largest files first. gabe@xps:~$ cd /dev
gabe@xps:/dev$ls -lS |less total 1907996 -rw-r--r-- 1 root wheel 529827840 Oct 1 13:24 sd1 -rw-r--r-- 1 root wheel 446758912 Oct 1 17:15 rsd1 -r-xr-xr-x 1 root wheel 11565 Sep 20 23:09 MAKEDEV dr-xr-xr-x 2 root wheel 1024 Sep 21 11:08 fd lrwxr-xr-x 1 root wheel 9 May 3 14:10 audioctl -> audioctl0 lrwxr-xr-x 1 root wheel 6 May 3 14:10 audio -> audio0 lrwxr-xr-x 1 root wheel 6 May 3 14:10 mixer -> mixer0 lrwxr-xr-x 1 root wheel 6 May 3 14:10 radio -> radio0 lrwxr-xr-x 1 root wheel 6 May 3 14:10 sound -> sound0 lrwxr-xr-x 1 root wheel 6 May 3 14:10 video -> video0 lrwxr-xr-x 1 root wheel 4 May 3 14:10 pci -> pci0 crw-r--r-- 1 root wheel 83, 0 Sep 21 11:08 apm crw-r--r-- 1 root wheel 83, 8 Sep 21 11:08 apmctl crw-r--r-- 1 root wheel 45, 3 Sep 21 11:08 arandom crw-rw-rw- 1 root wheel 42, 0 Oct 3 12:38 audio0 crw-rw-rw- 1 root wheel 42, 1 Sep 21 11:08 audio1 crw-rw-rw- 1 root wheel 42, 2 Sep 21 11:08 audio2 crw-rw-rw- 1 root wheel 42, 192 Sep 21 11:08 audioctl0 crw-rw-rw- 1 root wheel 42, 193 Sep 21 11:08 audioctl1 crw-rw-rw- 1 root wheel 42, 194 Sep 21 11:08 audioctl2 crw------- 1 root wheel 79, 0 Sep 21 11:08 bio crw-r--r-- 1 root wheel 49, 0 Sep 21 11:08 bktr0  Again, I’ve trimmed the output since the relevant information is right on top. I also did something we haven’t seen before, which was to use the | operator. This creates a pipeline between two programs allowing you to send the output of the first program as the input to the next program. I’ll talk more about pipelines next week. In this case, I took the output of the ls program and passed that to the less program so that the output would only be displayed one page at a time instead of scrolling up too fast for me to read. As a reminder you can page through the output of the less program with the space bar, or move up and down a line using the ‘k’ and ‘j’ keys on your keyboard respectively. To get out of the less program at any time, just hit the ‘q’ key. So, this output shows me two files that are huge in comparison to everything else in the /dev directory. sd1 is 500M and rsd1 is 400M. There is something very wrong with that. Neither of these files should exist at all, and the fact that they are here tells me that I’ve make a mistake at some point and created a file where I shouldn’t have. Deleting these files should fix the issue I’m having. The only problem is these files aren’t mine, and so the system won’t let me delete them: gabe@xps:/dev$ rm sd1
override rw-r--r--  root/wheel for sd1? y
rm: sd1: Permission denied


This is a good thing, because if I don’t know what I’m doing I could seriously mess up my computer.

We’ll revisit this issue at a later date when we have a few more tools under our belts. For now, I’ll live with the fact that my / filesystem is full since my computer seems to be working fine otherwise.

What can you find out about your system using the df and du commands?

Can you find out what directory is using the most space in your home folder?

Don’t try to run the rm program on anything just yet, if you delete something using the command line, it’s gone forever.

New Terms

• partition - A logical slice of a physical disk. Say I have a 1TB drive, I can tell the operating system to divide that drive up into many smaller slices that I can then treat as completely independent disks.
• less - A program to display the contents of a file on a page by page basis.
• pipeline - A tool that allows you to send the output of one program as the input to another program.
• filesystem - An individual formatted partition of a disk.
• du - A program for displaying disk usage information on a file or directory level
• df - A program for displaying disk usage information on the filesystem level
• /dev - A special directory the operating system uses to give you access to your hardware.
• /usr - A special directory where the operating system installs tools and utilities you can use.