~xjobs/
home of xjobs utility

Description

xjobs reads job descriptions line by line and executes them in parallel. It limits the number of parallel executing jobs and starts new jobs when jobs finish. Therefore, it combines the arguments from every input line with the utility and arguments given on the command line. If no utility is given as an argument to xjobs, then the first argument on every job line will be used as utility. To execute utility xjobs searches the directories given in the PATH environment variable and uses the first file found in these directories.

xjobs is most useful on multi-processor/core machines when one needs to execute several time consuming command several that could possibly be run in parallel. With xjobs this can be achieved easily, and it is possible to limit the load of the machine to a useful value. It works similar to xargs, but starts several processes simultaneously and gives only one line of arguments to each utility call.

Technology/Features

xjobs is written in C and uses the POSIX API. There is nothing special about it, but if one searches a small and useful example for lex integration, its source code may be worth a look.

Current status

xjobs is stable. If you encounter a bug or need an enhancement, please feel free to contact me.

Comment on GNU xargs vs. xjobs

Yes, GNU's xargs has an option -P that allow parallelizing jobs. But you must tell xargs how many arguments to pass to each job, as it doesn't make a difference between a space and a newline charakter. In consequence each job issued by xargs must have the same number of arguments, whereas xjobs can handle different jobs with different commands and different number of arguments.

Additionally, xjobs support I/O redirection, which makes some applications possible that cannot be done with GNU's xargs. xjobs also determines the number of processors automatically, whereas xargs must be told how many processes to start. Finally, non-GNU xargs (e.g. Solaris' and DEC/Compaq/HP Alpha's) don't have the option -P. Try to do the following with GNU's xargs with some files having blanks in their name:

ls -1 *.mp3 | sed 's/\(.*\)\.mp3/"\1.mp3" > "\1.wav"/' | xjobs -- mpg123 -s

If you aren't convinced yet, just compare the output of unziping multiple zip files in parallel with xjobs and GNU xargs. xjobs won't intermix the output of unzip like xargs. Instead it will present the output of each unzip separated clearly from the other jobs.