15

I am writing a small little script to clear space on my linux everyday via CRON if the cache directory grows too large. Since I am really green at bash scripting, I will need a little bit of help from you linux gurus out there.

Here is basically the logic (pseudo-code)

    if ( Drive Space Left < 5GB )
    {
        change directory to '/home/user/lotsa_cache_files/'

        if ( current working directory = '/home/user/lotsa_cache_files/')
        {
            delete files in /home/user/lotsa_cache_files/
        }
    }

Getting drive space left

I plan to get the drive space left from the '/dev/sda5' command. If returns the following value to me for your info :

Filesystem           1K-blocks      Used Available Use% Mounted on<br>
/dev/sda5            225981844 202987200  11330252  95% /

So a little regex might be necessary to get the '11330252' out of the returned value

A little paranoia

The 'if ( current working directory = /home/user/lotsa_cache_files/)' part is just a defensive mechanism for the paranoia within me. I wanna make sure that I am indeed in '/home/user/lotsa_cache_files/' before I proceed with the delete command which is potentially destructive if the current working directory is not present for some reason.

Deleting files

The deletion of files will be done with the command below instead of the usual rm -f:

find . -name "*" -print | xargs rm

This is due to the inherent inability of linux systems to 'rm' a directory if it contains too many files, as I have learnt in the past.

1
  • 3
    "inherent inability of linux systems to 'rm' a directory if it contains too many files" Does that also imply to "rm -rf"? Commented May 6, 2011 at 8:45

5 Answers 5

22

Just another proposal (comments within code):

FILESYSTEM=/dev/sda1 # or whatever filesystem to monitor
CAPACITY=95 # delete if FS is over 95% of usage 
CACHEDIR=/home/user/lotsa_cache_files/

# Proceed if filesystem capacity is over than the value of CAPACITY (using df POSIX syntax)
# using [ instead of [[ for better error handling.
if [ $(df -P $FILESYSTEM | awk '{ gsub("%",""); capacity = $5 }; END { print capacity }') -gt $CAPACITY ]
then
    # lets do some secure removal (if $CACHEDIR is empty or is not a directory find will exit
    # with error which is quite safe for missruns.):
    find "$CACHEDIR" --maxdepth 1 --type f -exec rm -f {} \;
    # remove "maxdepth and type" if you want to do a recursive removal of files and dirs
    find "$CACHEDIR" -exec rm -f {} \;
fi 

Call the script from crontab to do scheduled cleanings

2
  • 3
    I suggest using the -delete flag instead of -exec rm -f {} \;. Also consider adding -xdev as the first flag to stay in the same filesystem. Commented Nov 28, 2017 at 8:07
  • 1
    If you want to keep the latest files, I suggest : find "$CACHEDIR" -type f -mmin +720 -delete to delete only files older than 12 hours (*60 min = 720).
    – Philippe
    Commented Dec 27, 2022 at 7:17
11

I would do it this way:

# get the available space left on the device
size=$(df -k /dev/sda5 | tail -1 | awk '{print $4}')

# check if the available space is smaller than 5GB (5000000kB)
if (($size<5000000)); then
  # find all files under /home/user/lotsa_cache_files and delete them
  find /home/user/lotsa_cache_files -name "*" -delete
fi
1
  • That's what I was going to recommend: Hard code the directory in the find command and don't use xargs because it can be dangerous.
    – David W.
    Commented May 6, 2011 at 18:36
5

Here's the script I use to delete old files in a directory to free up space...

#!/bin/bash
#
#  prune_dir - prune directory by deleting files if we are low on space
#
DIR=$1
CAPACITY_LIMIT=$2

if [ "$DIR" == "" ]
then
    echo "ERROR: directory not specified"
    exit 1
fi

if ! cd $DIR
then
    echo "ERROR: unable to chdir to directory '$DIR'"
    exit 2
fi

if [ "$CAPACITY_LIMIT" == "" ]
then
    CAPACITY_LIMIT=95   # default limit
fi

CAPACITY=$(df -k . | awk '{gsub("%",""); capacity=$5}; END {print capacity}')

if [ $CAPACITY -gt $CAPACITY_LIMIT ]
then
    #
    # Get list of files, oldest first.
    # Delete the oldest files until
    # we are below the limit. Just
    # delete regular files, ignore directories.
    #
    ls -rt | while read FILE
    do
        if [ -f $FILE ]
        then
            if rm -f $FILE
            then
                echo "Deleted $FILE"

                CAPACITY=$(df -k . | awk '{gsub("%",""); capacity=$5}; END {print capacity}')

                if [ $CAPACITY -le $CAPACITY_LIMIT ]
                then
                    # we're below the limit, so stop deleting
                    exit
                fi
            fi
        fi
    done
fi
3
  • Please check again the code! It seems that it will delete files within the current folder.
    – Max
    Commented Jun 3, 2017 at 20:56
  • 1
    @Max it does cd $DIR so it won't.
    – dgpro
    Commented Aug 15, 2017 at 11:18
  • Great script, thanks. One subtle thing though is the exit statement; it doesn't exit the script, but only the subshell that is processing the output of the ls command. I would suggest to replace exit with a break so it is obvious that the script continues after the while loop
    – luca
    Commented Aug 16 at 14:04
3

To detect the occupation of a filesystem, I use this :

df -k $FILESYSTEM | tail -1 | awk '{print $5}'

that gives me the occupation percentage of the filesystem, this way, I don't need to compute it :)

If you use bash, you can use the pushd/popd operation to change directory and be sure to be in.

pushd '/home/user/lotsa_cache_files/'
do the stuff
popd
-2

Here's what I do:

while read f; do rm -rf ${f}; done < movies-to-delete.txt

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.