km3pipe.utils.qrunprocessor¶
Use this batch runner to process a given list of run numbers with a script, which takes a -o to create a summary file, which has the name of the processed file + a given suffix. The actual call will look like this, with a bit of copy safeness:
/abs/path/to/SCRIPT run.root -o /abs/path/to/OUTPUT_PATH+SUFFIX
Before constructing the job scripts, the OUTPUT_PATH will be traversed
to find files which have already been converted to avoid multiple conversions.
Usage:
qrunprocessor [options] DET_ID RUN_LIST OUTPUT_PATH SCRIPT
qrunprocessor (-h | --help)
Options:
DET_ID Detector ID (e.g. 29).
RUN_LIST Path to the file containing the space separated run IDs.
OUTPUT_PATH Path to store the individual summary files.
SCRIPT The script to fire up.
-s SUFFIX The suffix, appended by SCRIPT [default: .summary.h5].
-n N_FILES Number of files to process per job [default: 10].
-e ET Estimated walltime per file in minutes [default: 15].
-f FSIZE Estimated filesystem size for a job [default: 12G].
-m VMEM Estimated vmem for a job [default: 8G].
-j JOBNAME The name of the submitted jobs [default: qrunprocessor].
-l LOG_PATH Path of the job log files [default: qlogs].
-v PYTHONVENV Path to the Python virtual env.
-c CLUSTER Cluster to run on (in2p3, woody, ...) [default: in2p3].
-q Dryrun: don't submit jobs, just print the job script.
-h --help Show this screen.
Module Contents¶
Functions¶
|