Title: Airavata Workflow Suite Notice: Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at . http://www.apache.org/licenses/LICENSE-2.0 . Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. Airavata XBaya workflow system provides a programming model that allows the scientist to program experiments using service-oriented architecture that would abstract the complexities of the underlying middleware. Airavata XBaya provides interfaces for composition, execution and monitoring of the workflows as illustrated in the following Figure. Airavata XBaya consist of convenient GUI interface for workflow composition, workflow enactment engine/interface for workflow enactment and workflow monitoring module. ![Airavata Workflow Suite](/architecture/workflow-suite.png "Airavata Workflow Suite") ## Workflow Composition Airavata XBaya by design decouples composition and monitoring from the orchestration of the workflow although it does provide an embedded workflow enactment engine integrated with the workbench. As a scientific workflow suite XBaya is often expected to run long running computations thus it often delegates the workflow execution to a persistent orchestration engine and XBaya workbench can monitor the progress of the workflow asynchronously. The workbench provides a convenient drag and drop GUI for SOA based service composition along with other functionalities like service discovery, registry lookup and workflow experiment management. ## Workflow orchestration XBaya provides a unique pluggable architecture for selecting the orchestration engine. When a user composes a workflow using XBaya workbench it builds an abstract Directed Acyclic Graph (DAG) which is independent of any workflow runtime. There are pluggable compiler modules that are capable of producing workflow execution scripts for target runtimes. The above Figure illustrated how the DAG can compiled into different runtimes. Currently XBaya supports BPEL 2.0 and Apache ODE; SCUFL and Taverna; DAGMAN and Pegasus; Jython scripts and XBaya Interpreter Service. Each of these workflow runtimes have their own strengths, for example, Apache ODE is a robust SOA based workflow orchestration engine well suited for long running applications. Pegasus workflow system is well suited for parametric sweeps whereas XBaya Interpreter engine is strong in dynamic and user interactive workflow execution. It is also capable of generating a Jython script that can be executed in any Jython runtime independent of any of the workflow infrastructure. ## Workflow interpreter XBaya provides a inbuilt workflow enactment engine that provides extremely dynamic and interactive workflow execution. As the name suggests the workflow interpreter provides an interpreted workflow execution framework rather than the compiled workflow execution environments. In this context the interpreted workflow means at that execution framework executed tasks in the XBaya DAG as and when the tasks become runnable. Once the user composes the workflow and launches the workflow the workflow interpreter will start executing the workflow DAG. Topological sort of the DAG provides execution ordering of the tasks which allow the identification of the initial ready task set and incrementally adding new tasks to the ready list as previous tasks finish execution. What is unique about this type of execution model is it allows the user to pause the execution of the workflow as necessary and update the DAG’s execution states or even the DAG itself and resume execution and the changes are immediately picked up by the workflow interpreter. Workflow interpreter can be used as an embedded workflow enactment engine within XBaya GUI or as an interpreter service that may run as a persistent service. The functionality provided by the workflow interpreter allows fine gain control over the workflow execution which translates into following functionalities. * Static workflow interactions * Dynamic change workflow inputs, workflow rerun. * Stop, pause, resume execution of the workflow. Pause workflow at a particular (debug) point * Dynamic change in point of execution, workflow smart rerun. * Fault handling and exception models * Dynamic workflow interactions * Reconfiguration of workflow activity * Dynamic addition of activities to the DAG. * Dynamic remove or replace of activity to the DAG. ## Workflow monitoring: XBaya workbench allows the user to monitor the progress of the workflow in real-time. Monitoring includes the services which are currently executing, those that are done. It also provides state of the job submissions to the batch job queues and status of the data transfer. Workflow monitoring in XBaya workbench work in two modes; synchronous mode and asynchronous mode. The synchronous mode only applies when XBaya interpreter runs embedded in the workbench. This allows the execution engine to be aware of which task are ready, running, finished and waiting and thus accordingly update the DAG execution state. But when the workflow execution is delegated to a persistent service like Apache ODE serve or XBaya interpreter service, it is necessary to have asynchronous monitoring. This is achieved using the WS-Eventing based notification messages published by different components of the workflow system. If a user launches a workflow instance a unique topic will be associated with that instance and all the workflow components like workflow engine, Factory service will publish status notification about the workflow to the event bus using the topic assigned to the workflow instance. This asynchronous monitoring is truly asynchronous where the user can launch the workflow, start monitoring, loose internet connectivity and once reconnected the XBaya workbench will bring back the monitoring to the current execution state of the workflow.