.. _end_user_submit: Submit ====== What is submit? --------------- The ``Submit`` process is a tool to help submit a hpc job to a cluster using d3VIEW from the command line instead of the web interface. Not only can jobs be submitted, but postprocessing can be done on completed jobs. Requirements ------------ • Python 2.7 (Latest release of Python 2.7 can be found `here `_) • Lucy • Connection to d3VIEW How to run ---------- The ``Submit`` tool is used by invoking ``Lucy``, and specifying the arguments needed to submit/monitor a job. The most basic command is as follows:: $ lucy submit -d3view-url http://d3VIEW-internal.com -job-json job.json :program:`lucy` :program:`submit` :option:`-d3view-url` ``http://d3VIEW-internal.com`` :option:`-job-json` ``job.json`` If you don't have a job.json file prepared, one will automatically be created after running the command with the keyword arguments outlined below. Without the prepared job.json, the submit command will look like:: $ lucy submit \ -d3view-url http://d3VIEW-internal.com \ -u my_name \ -a b19ed11cee194d0b7cb1f1d5a0a80e890972c06c \ -t lsdyna \ -v r712sp \ -i my_input_file.k \ -inc "key" \ -c 8 \ -q test_queue \ -w 999:00:00 \ -solver-args "precision=sp, binout_to_ascii=no" \ -n my_test_job1 \ -d "Testing my first job" \ -download-attachments "dyna_binout, dyna_ascii" Arguments --------- ``Submit`` comes with a number of arguments to allow flexibility when submitting different kinds of jobs. All arguments are shown directly below, with links to their sections. +--------------------------------+----------------------------------+ | Parameter name | When required | +================================+==================================+ | `Connection arguments`_ | | +--------------------------------+----------------------------------+ |:option:`-d3view-url` | **Always required** | +--------------------------------+----------------------------------+ |:option:`-d3view-port` | **Always required** | +--------------------------------+----------------------------------+ |:option:`-u`, | **Always required** | |:option:`-user` | | +--------------------------------+----------------------------------+ |:option:`-a`, | **Always required** | |:option:`-api-key` | | +--------------------------------+----------------------------------+ +--------------------------------+----------------------------------+ | `Simulation arguments`_ | | +--------------------------------+----------------------------------+ |:option:`-s`, | | |:option:`-hpcserver-name` | | +--------------------------------+----------------------------------+ |:option:`-project-id` | | +--------------------------------+----------------------------------+ |:option:`-loadcase-name` | | +--------------------------------+----------------------------------+ |:option:`-loadcase-id` | | +--------------------------------+----------------------------------+ |:option:`-t`, | **Required for new simulations** | |:option:`-solver-type` | | +--------------------------------+----------------------------------+ |:option:`-v` | **Required for new simulations** | +--------------------------------+----------------------------------+ |:option:`-i` | **Required for new simulations** | +--------------------------------+----------------------------------+ |:option:`-inc`, | | |:option:`-include-files` | | +--------------------------------+----------------------------------+ |:option:`-c`, | **Required for new simulations** | |:option:`-ncpu` | | +--------------------------------+----------------------------------+ |:option:`-q` | **Required for new simulations** | +--------------------------------+----------------------------------+ |:option:`-w`, | **Required for new simulations** | |:option:`-walltime` | | +--------------------------------+----------------------------------+ |:option:`-solver-args` | | +--------------------------------+----------------------------------+ |:option:`-r`, | | |:option:`-response-template` | | +--------------------------------+----------------------------------+ |:option:`-n`, | | |:option:`-name`, | | |:option:`-job-name` | | +--------------------------------+----------------------------------+ |:option:`-d`, | | |:option:`-description`, | | |:option:`-job-desc` | | +--------------------------------+----------------------------------+ |:option:`-vd`, | | |:option:`-variable-definition`, | | +--------------------------------+----------------------------------+ +--------------------------------+----------------------------------+ | `Additional arguments`_ | | +--------------------------------+----------------------------------+ |:option:`-job-json` | | +--------------------------------+----------------------------------+ |:option:`-new-simulation` | | +--------------------------------+----------------------------------+ |:option:`-job-id` | | +--------------------------------+----------------------------------+ |:option:`-simulation-id` | | +--------------------------------+----------------------------------+ |:option:`-download-attachments` | | +--------------------------------+----------------------------------+ Connection arguments ^^^^^^^^^^^^^^^^^^^^ These Arguments are required for the application to be able to log into d3VIEW. .. option:: -d3view-url The url to d3VIEW. Such as ``http://d3VIEW-internal.com`` (Note: do not include port number if shown, that is the next parameter.) .. option:: -d3view-port The port to d3VIEW, if it is not 80, this should be set. .. option:: -u, -user The name of the user submitting the job in d3VIEW, .. option:: -a, -api-key The `API Key`_ found in d3VIEW. API Key +++++++ To find the API Key follow these steps: 1) Log in to d3VIEW. 2) Click on the profile icon, found directly next to the Logout button. 3) In the info section the 4th item down is the API Key Simulation arguments ^^^^^^^^^^^^^^^^^^^^ .. option:: -s, -hpcserver-name The name of the `HPC server`_ (in d3VIEW) that the job is going to be submitted to. .. option:: -project-id The name of the d3VIEW project that the simulation should be part of. .. option:: -loadcase-name The name of the d3VIEW loadcase that will be attached to the simulation. .. option:: -loadcase-id The id of the loadcase in d3VIEW. .. option:: -t, -solver-type The type of solver to be run, such as ``lsdyna`` .. option:: -v `Solver version`_ to run. Supported `versions `_ are shown in d3VIEW on the submit screen. .. option:: -i The path to the main input file for the simulation. If the main input file is not in the current working directory, the full path must be specified. .. option:: -inc, -include-files File types or files to include in the simulation, usually additional input files. If multiple files or file extensions are going to be included, the entire group should be surrounded in double quotes and each file name or extension should be separated by a comma and space. The format to include files is as follows:: -inc "ext1, ext2, ext3, filename.extension" .. option:: -c, -ncpu The number of cpus to run the simulation .. option:: -q The queue for the hpc job. See `Finding the scheduler queue`_ .. option:: -w, -walltime The walltime for the job in format:: "HHH:MM:SS" 'HHH' is hours (000-999) 'MM' is minutes (00-59) 'SS' is seconds (00-59) .. option:: -solver-args key=value For each `d3VIEW solver argument`_ in the format:: "precision=sp, binout_to_ascii=no, example=\"quoted value\", example2=2" .. option:: -r, -response-template The name of the response template to apply .. option:: -n, -name, -job-name The name of the simulation. If set to ``'auto'``, ``Submit`` will name the simulation as a combination of the working directory and the parent directory. If the folder structure is:: root ├ stage1 │ ├ 1.1 (working directory) │ │ ├ main_input_file.k │ │ └ additional_input_file.key The simulation will be named stage1_1.1. This is especially useful for LS-OPT where multiple simulations will be submitted from the same LS-OPT job. .. option:: -d, -description, -job-desc The description of the simulation .. option:: -vd, -variable-definition Path to the variable definition file. HPC server ++++++++++ The name of the HPC server should match the name shown in the ``Select HPC Server`` dropdown when submitting a job in d3VIEW. To navigate to this screen: 1) Log in to d3VIEW. 2) Click on the Simulations tab (if not already there) 3) Click the blue ``+ New Simulation`` button found in the top right of the screen 4) Scroll down to the ``Select HPC Server`` section 5) Click ``Nothing Selected`` to open the dropdown. solver version ++++++++++++++ The supported solver versions can be found easily by using the following steps: 1) Log in to d3VIEW. 2) Click on the Simulations tab (if not already there) 3) Click the blue ``+ New Simulation`` button found in the top right of the screen 4) Scroll down to the ``Select HPC Server`` section, and select the `HPC Server`_ 5) Choose the solver type using the ``Choose Solver`` dropdown. 6) Notice the ``Solver Version`` attribute. It will be either a dropdown or text box. 7) If it is a text box, there is only one solver version. Otherwise open the dropdown to view all supported solver versions. Finding the scheduler queue +++++++++++++++++++++++++++ 1) Log in to d3VIEW. 2) Click on the Simulations tab (if not already there) 3) Click the blue ``+ New Simulation`` button found in the top right of the screen 4) Scroll down to the ``Select HPC Server`` section, and select the `HPC Server`_ 5) Choose the solver type using the ``Choose Solver`` dropdown. 6) The queue dropdown is in the middle of the screen immediately above the ``Add this Configuration`` button. 7) Click to open the dropdown and reveal available queues. d3VIEW solver argument ++++++++++++++++++++++ d3VIEW solver arguments are the arguments in the grey box once the solver type has been chosen on the submit screen. Follow these steps to find the solver arguments for the solver of your choice. 1) Log in to d3VIEW. 2) Click on the Simulations tab (if not already there) 3) Click the blue ``+ New Simulation`` button found in the top right of the screen 4) Scroll down to the ``Select HPC Server`` section, and select the `HPC Server`_ 5) Choose the solver type using the ``Choose Solver`` dropdown. 6) All of the Bold names are the display names of the solver attributes, with their respective values in the field below each one. The format of the fields is as follows:: -solver-args "precision=sp, binout_to_ascii=no" .. Note:: An administrator can get the ID from a solver attribute if requested. Additional arguments ^^^^^^^^^^^^^^^^^^^^ .. option:: -job-json This option allows loading all parameters from a job.json file instead of having a large command. The contents of the job.json file should look like:: { "api_url": "", "api_port": "", "username": "", "api_key": "", "hpcserver_name": "", "solver_type": "", "solver_version": "", "solver_input": "", "include_files": "", "num_cores": "", "job_queue": "", "scheduler_walltime": "", "solver_args": "", "response_template": "", "job_name": "", "job_description": "", "download_types": "" } .. option:: -new-simulation If specified, this will create a new simulation every time the process is run. Otherwise subsequent runs using the same job.json will try connect to the existing simulation. .. option:: -job-id The job id used to connect to an existing simulation (Must also specify ``-simulation-id``) .. option:: -simulation-id The simulation id used to connect to an existing simulation (Must also specify ``-job-id``) .. option:: -download-attachments List of types or files to download. Using the same format as :option:`-inc`. If multiple files or file extensions are going to be downloaded, the entire group should be surrounded in double quotes and each file name or extension should be separated by a comma and space. The format to include files is as follows:: -download-attachments "ext1, ext2, ext3, filename.extension"