I want to find a bash command that will let me grep every file in a directory and write the output of that grep to a separate file. My guess would have been to do something like this
ls -1 | xargs -I{} "grep ABC '{}' > '{}'.out"
but, as far as I know, xargs doesn't like the double-quotes. If I remove the double-quotes, however, then the command redirects the output of the entire command to a single file called '{}'.out instead of to a series of individual files.
Does anyone know of a way to do this using xargs? I just used this grep scenario as an example to illustrate my problem with xargs so any solutions that don't use xargs aren't as applicable for me.
-
A solution without
xargs
is the following:find . -mindepth 1 -maxdepth 1 -type f -exec sh -c "grep ABC '{}' > '{}.out'" \;
...and the same can be done with
xargs
, it turns out:ls -1 | xargs -I{} sh -c "grep ABC '{}' > '{}.out'"
Edit: single quotes added after remark by lhunath.
Zifre : He said he wants to use xargs. I posted a solution without it too, but deleted once I saw that he needed xargs.Stephan202 : You're right. Reason I posted my answer was that it's better to have an alternative solution to get the job done than none at all. Turns out that it put me on the right track to find the desired answer (that is, the sh -c trick).Zifre : +1 That second solution works, although I'm sure there is a better way... -
Do not make the mistake of doing this:
sh -c "grep ABC {} > {}.out"
This will break under a lot of conditions, including funky filenames and is impossible to quote right. What you need to do, is this:
sh -c 'grep ABC "$1" > "$1.out"' -- {}
Applies to
xargs
as well asfind
.By the way, never use xargs without the
-0
option (unless for very rare and controlled one-time interactive use where you aren't worried about destroying your data).Also don't parse
ls
. Ever. Use globbing orfind
instead: http://mywiki.wooledge.org/ParsingLsUse
find
for everything that needs recursion and a simple loop with a glob for everything else:find /foo -exec sh -c 'grep "$1" > "$1.out"' -- {} \;
or non-recursive:
for file in *; do grep "$file" > "$file.out"; done
Notice the proper use of quotes.
Stephan202 : +1. Indeed I should not have forgotten about the quotes. -
I assume your example is just an example and that you may need > for other things. GNU Parallel http://www.gnu.org/software/parallel/ may be your rescue. It does not need additional quoting as long as your filenames do not contain \n:
ls | parallel "grep ABC {} > {}.out"
If you have filenames with \n in it:
find . -print0 | parallel -0 "grep ABC {} > {}.out"
As an added bonus you get the jobs run in parallel.
0 comments:
Post a Comment