Automatic File Pre-Staging Example with Condor-G

To instruct Condor-G to stage various files by copying them to the remote submission system as the first part of your job, you must specify a few extra Condor submission attributes:

executable = /bin/cat
transfer_executable = false
globusscheduler = gatekeeper.purdue.nwicgrid.org/jobmanager-pbs
universe = globus
arguments = my_data
when_to_transfer_output = ON_EXIT
transfer_input_files = my_data
output = my_job.out
log = my_job.log
queue

Note that re-staging your data files for each job may be slow and wasteful, so you may wish to copy them over only once, then run many jobs using the same files.

Then, submit this Condor file to the Purdue Altix (caesar):

condor_submit hostname.condor

Or submit the same to the Notre Dame cluster by changing the globusscheduler line in the file and resubmitting:

globusscheduler = gatekeeper.nd.nwicgrid.org/jobmanager-sge

You can watch the progress of the job by reading the file “my_job.log“:

tail -50f my_job.log

Once complete, you should see “005 (...) mm/dd hh:mm:ss Job terminated.” with some more details about how the job terminated following that. Then you can read the file “my_job.out” to see the results:

more my_job.out