Automatic File Pre-Staging Example with Globus RSL
To instruct Globus to stage various files by copying them to the remote submission system as the first part of your job, you must specify a few extra RSL attributes:
& (jobtype=single)
(environment=(GLOBUS_DUROC_SUBJOB_INDEX 0))
(file_stage_in=($(GLOBUSRUN_GASS_URL)/./my_data remote_my_data))
(file_clean_up=remote_my_data)
(executable=/bin/cat)
(arguments=remote_my_data)
Note that re-staging your data files for each job may be slow and wasteful, so you may wish to copy them over only once, then run many jobs using the same files. You can do this by omitting the file_clean_up attribute until actually done using the file.
Then, submit this RSL file to the Purdue Altix (caesar) or Notre Dame cluster as normal:
globusrun -s -f my_job.rsl -r gatekeeper.purdue.nwicgrid.org/jobmanager-pbs
globusrun -s -f my_job.rsl -r gatekeeper.nd.nwicgrid.org/jobmanager-sge
If you would like to redirect your output and error, you may add the following parameters to the RSL file:
(stdout=my_job.out)
(stderr=my_job.err)
To instruct Globus to stage various files by copying them to the remote submission system as the first part of your job, you must specify a few extra RSL attributes:
& (jobtype=single) (environment=(GLOBUS_DUROC_SUBJOB_INDEX 0)) (file_stage_in=($(GLOBUSRUN_GASS_URL)/./my_data remote_my_data)) (file_clean_up=remote_my_data) (executable=/bin/cat) (arguments=remote_my_data)
Note that re-staging your data files for each job may be slow and wasteful, so you may wish to copy them over only once, then run many jobs using the same files. You can do this by omitting the file_clean_up attribute until actually done using the file.
Then, submit this RSL file to the Purdue Altix (caesar) or Notre Dame cluster as normal:
globusrun -s -f my_job.rsl -r gatekeeper.purdue.nwicgrid.org/jobmanager-pbs globusrun -s -f my_job.rsl -r gatekeeper.nd.nwicgrid.org/jobmanager-sge
If you would like to redirect your output and error, you may add the following parameters to the RSL file:
(stdout=my_job.out) (stderr=my_job.err)
Powered by Joom Prosolution
