[[ucc_workflow-extensions]]
Workflow extensions
-------------------

=== Introduction

UCC supports the UNICORE workflow system and allows to submit workflows to
the workflow engine or single jobs to the service orchestrator (broker).

The workflows are executed server-side, and UCC is used only for submitting, 
managing data and getting results.

=== Command overview

The following commands are provided. More details and examples follow below.
  
  - +workflow-submit+ : submit a workflow file
  
  - +workflow-info+ : list information about workflows
  
  - +workflow-trace+ : gather performance data from the workflow execution
  
  
=== Basic use

To check the availability of workflow services, issue the following command

-------------
ucc system-info -l
-------------

This should show at least an accessible workflow engine and service orchestrator.

The distribution contains some example workflow files in the <<<samples/workflows>>> directory 
that you can edit and submit.
  
-------------
ucc workflow-submit yourworkflow.swf
-------------

which will submit the workflow and print the address of the workflow to standard output.
To get the workflow status,

---------------
ucc workflow-info <workflow_address>
---------------  


To list all your workflows, you can use the <<<workflow-info>>> command without an explicit
workflow address

---------------
ucc workflow-info -l
---------------  
   
  
=== Managing workflow data

During workflow execution, data files will be produced that the workflow system will
move to a location on the Grid that is accessible for the individual jobs.
Usually this location is created automatically by UCC before the workflow is submitted, using
a special UNICORE service called a storage factory. If you want to influence this decision,
UCC allows to select the storage factory to be used via the "-f" option to the workflow-submit 
command:

---------------
ucc workflow-submit -f <factory-url> <workflow-file>
---------------  

You can check the available factories with the system-info command. If not specified, UCC will
use the fist storage factory it finds in the registry.

In general, storages that are dynamically created will be deleted when the workflow is deleted.
To persistently store data, you need to make sure to export important result files
to a persistent location (e.g. your Home on some Grid site, or a persistent storage).

Alternatively you can directly specify a storage URL, either using the convenient "u6://..." 
notation, or as a real network URL:

---------------
ucc workflow-submit -S u6://MY-SITE/Home <workflow-file>

ucc workflow-submit -S https://my-gateway/SITE/services/StorageManagement?res=myuser-Home <workflow-file>
---------------  



==== Importing local data for use by a workflow

If you have local files that need to be imported before starting
the workflow, you have to specify this using a normal UCC job file that 
contains only an "Imports" section:

-----------
{
  #stage-in specification

  Imports: [
    {From: local-file.sh, To: "c9m:${WORKFLOW_ID}/input.sh"}
  ],
}
-----------

When submitting the workflow, add the "-u <filename>" option to specify
the imports file.

This will cause UCC to copy the local file "local-file.sh" to the workflow storage space.
You can refer to this file in your workflow using the "global" name "c9m:...", say in a script
activity:

----------
....
        <jsdl:DataStaging>
        <jsdl:CreationFlag>overwrite</jsdl:CreationFlag>
        <jsdl:FileName>input.sh</jsdl:FileName>
          <jsdl:Source>
            <jsdl:URI>c9m:${WORKFLOW_ID}/input.sh</jsdl:URI>
          </jsdl:Source>
        </jsdl:DataStaging>

....
----------

The workflow system will resolve the name at runtime and your file will be used. 
The "${WORKFLOW_ID}" will be replaced by UCC by the real workflow ID, when submitting the job.
This allows you to group your files by workflow ID. 

==== Downloading output files

You can use the usual +get-file+ command to download files using the "global IDs" used by the workflow engine.
Hint: the +workflow-info+ command will list the files that are produced by the workflow.

=== More

==== Tracing

The trace functionality of the workflow engine allows to retrieve some performance data, try
  
---------------
ucc workflow-trace <your_workflow_address>
---------------
