Thursday, February 3, 2011

What Unix command can I use to enforce a limit of 100 files in a folder?

I would like to delete the oldest files in a directory, after a limit of 100 files. In other words, I want to ensure that no more than 100 files exist in the directory, and if a limit is exceeded, delete the oldest files after the limit. I don't just want to delete files older than x days, since if this was run on a cronjob, eventually all files would be deleted.

I guess if I were to program this, the pseudo code would be:

list = dir.getFiles()
list.sortByDate()
deleteList = list.getSubList(100, end) // from, to
deleteAll(deleteList)

So what would the appropriate Unix command be? I guess find would be involved somehow with the -exec argument, but I'm not sure about the sorting/limiting aspect.

  • find should not be necessary. If you first go to the right dir,

    rm -f `ls -rt | head -n -100`
    

    to specify a path

    rm -f `ls -rt /path/to/my/dir | head -n -100`
    

    and for cron (on Ubuntu!)

    /bin/rm -f `/bin/ls -rt /path/to/my/dir | /usr/bin/head -n -100`
    

    A command path can be determined using which, e.g.

    which ls
    

    Finally, if file names contain spaces, they should be quoted ls -Q then sent to xargs

    /bin/ls -Qrt /path/to/my/dir | /usr/bin/head -n -100 | /usr/bin/xargs /bin/rm -f
    

    (tested on Ubuntu, for your tests, replace rm -f with echo to see what is to be deleted)

    chronos : Small typo: use `head -n 100` instead of `head -n -100`.
    nbolton : Thanks. I'm running this from a cronjob. How do I get full paths?
    ring0 : This is not a typo, the `ls` is `-rt` so we need `-100` to tell `head` to get rid of the 100 newest lines/files! To set full paths, simply put it in the `ls`, for instance `ls -rt /my/path/to/dir | head -n -100`. For `cron`, `ls` ... should better have their full path. Please see my edit above.
    Redmumba : Use `xargs` instead, or files may not be handled as you expect (for example, spaces): `ls -rt | head -n -100 | xargs rm -f`
    ring0 : If you have spaces in file names, `xargs` needs also the `Q` option of `ls`. Edited again.
    Redmumba : Good catch, and +1 for completeness.
    nbolton : `ls` is not *giving* me the full paths, which is required, since I need to run the command from `cron`. It seems to me that `find` is the only command that will *give* me full paths.
    nbolton : I guess I'll just use `cd` in my script since printing full paths isn't possible.
    From ring0
  • Are you reinventing the wheel called log rotation? If so, use logrotate (on linux systems; other systems will have their own equivalent programs).

    nbolton : I'm not sure how I'd use logrotate to do this. Baring in mind that I don't want to rename the files or delete them after each file name has been rotated x number of times. Please give an example of how logrotate could do the same thing as ring0's example.
    From ramruma

0 comments:

Post a Comment