Thrift module: computeResourceModel

Enumeration: ResourceJobManagerType

* Enumeration of local resource job manager types supported by Airavata * * FORK: * Forking of commands without any job manager * * PBS: * Job manager supporting the Portal Batch System (PBS) protocol. Some examples include TORQUE, PBSPro, Grid Engine. * * SLURM: * The Simple Linux Utility for Resource Management is a open source workload manager. * * UGE: * Univa Grid Engine, a variation of PBS implementation. * * LSF: * IBM Platform Load Sharing Facility is dominantly installed on IBM clusters. *

FORK	0
PBS	1
SLURM	2
LSF	3
UGE	4

Enumeration: JobManagerCommand

Enumeration of resource job manager commands

SUBMISSION: Ex: qsub, sbatch

JOBMONITORING: Ex: qstat, squeue

DELETION: Ex: qdel, scancel

CHECK_JOB: Detailed Status about the Job. Ex: checkjob

SHOW_QUEUE: List of Queued Job by the schedular. Ex: showq

SHOW_RESERVATION: List all reservations. Ex:showres, show_res

SHOW_START: Display the start time of the specified job. Ex: showstart

SUBMISSION	0
JOB_MONITORING	1
DELETION	2
CHECK_JOB	3
SHOW_QUEUE	4
SHOW_RESERVATION	5
SHOW_START	6

Enumeration: FileSystems

Enumeration of File Systems on the resource

FORK: Forking of commands without any job manager

PBS: Job manager supporting the Portal Batch System (PBS) protocol. Some examples include TORQUE, PBSPro, Grid Engine.

UGE: Univa Grid Engine, a variation of PBS implementation.

SLURM: The Simple Linux Utility for Resource Management is a open source workload manager.

HOME	0
WORK	1
LOCALTMP	2
SCRATCH	3
ARCHIVE	4

Enumeration: SecurityProtocol

Enumeration of security authentication and authorization mechanisms supported by Airavata. This enumeration just describes the supported mechanism. The corresponding security credentials are registered with Airavata Credential store.

USERNAME_PASSWORD: A User Name.

SSH_KEYS: SSH Keys

FIXME: Change GSI to a more precise generic security protocol - X509

USERNAME_PASSWORD	0
SSH_KEYS	1
GSI	2
KERBEROS	3
OAUTH	4

Enumeration: JobSubmissionProtocol

Enumeration of Airavata supported Job Submission Mechanisms for High Performance Computing Clusters.

SSH: Execute remote job submission commands using via secure shell protocol.

GRAM: Execute remote jobs via Globus GRAM service.

UNICORE: Execute remote jobs via Unicore services

LOCAL	0
SSH	1
GLOBUS	2
UNICORE	3
CLOUD	4

Enumeration: MonitorMode

Monitoring modes

POLL_JOB_MANAGER: GFac need to pull job status changes.

XSEDE_AMQP_SUBSCRIBE: Server will publish job status changes to amqp servert.

POLL_JOB_MANAGER	0
JOB_EMAIL_NOTIFICATION_MONITOR	1
XSEDE_AMQP_SUBSCRIBE	2

Enumeration: DataMovementProtocol

Enumeration of data movement supported by Airavata

SCP: Job manager supporting the Portal Batch System (PBS) protocol. Some examples include TORQUE, PBSPro, Grid Engine.

SFTP: The Simple Linux Utility for Resource Management is a open source workload manager.

GridFTP: Globus File Transfer Protocol

UNICORE_STORAGE_SERVICE: Storage Service Provided by Unicore

LOCAL	0
SCP	1
SFTP	2
GridFTP	3
UNICORE_STORAGE_SERVICE	4

Enumeration: ProviderName

Provider name

EC2	0
AWSEC2	1
RACKSPACE	2

Struct: ResourceJobManager

Key	Field	Type	Description	Requiredness	Default value
1	resourceJobManagerId	string		required	"DO_NOT_SET_AT_CLIENTS"
2	resourceJobManagerType	ResourceJobManagerType		required
3	pushMonitoringEndpoint	string		optional
4	jobManagerBinPath	string		optional
5	jobManagerCommands	map<JobManagerCommand, string>		optional

Resource Job Manager Information

resourceJobManagerType: A typical HPC cluster has a single Job Manager to manage the resources.

pushMonitoringEndpoint: If the job manager pushes out state changes to a database or bus, specify the service endpoint. Ex: Moab Web Service, Moab MongoDB URL, AMQP (GLUE2) Broker

jobManagerBinPath: Path to the Job Manager Installation Binary directory.

jobManagerCommands: An enumeration of commonly used manager commands.

Struct: BatchQueue

Key	Field	Type	Description	Requiredness	Default value
1	queueName	string		required
2	queueDescription	string		optional
3	maxRunTime	i32		optional
4	maxNodes	i32		optional
5	maxProcessors	i32		optional
6	maxJobsInQueue	i32		optional
7	maxMemory	i32		optional

Batch Queue Information on SuperComputers

maxRunTime: Maximum allowed run time in hours.

Struct: SCPDataMovement

Key	Field	Type	Description	Requiredness	Default value
1	dataMovementInterfaceId	string		required	"DO_NOT_SET_AT_CLIENTS"
2	securityProtocol	SecurityProtocol		required
3	alternativeSCPHostName	string		optional
4	sshPort	i32		optional	22

Data Movement through Secured Copy

alternativeSCPHostName: If the login to scp is different than the hostname itself, specify it here

sshPort: If a non-default port needs to used, specify it.

Struct: GridFTPDataMovement

Key	Field	Type	Description	Requiredness	Default value
1	dataMovementInterfaceId	string		required	"DO_NOT_SET_AT_CLIENTS"
2	securityProtocol	SecurityProtocol		required
3	gridFTPEndPoints	list<string>		required

Data Movement through GridFTP

alternativeSCPHostName: If the login to scp is different than the hostname itself, specify it here

sshPort: If a non-default port needs to used, specify it.

Struct: UnicoreDataMovement

Key	Field	Type	Description	Requiredness	Default value
1	dataMovementInterfaceId	string		required	"DO_NOT_SET_AT_CLIENTS"
2	securityProtocol	SecurityProtocol		required
3	unicoreEndPointURL	string		required

Data Movement through UnicoreStorage

unicoreEndPointURL: unicoreGateway End Point. The provider will query this service to fetch required service end points.

Struct: LOCALSubmission

Key	Field	Type	Description	Requiredness	Default value
1	jobSubmissionInterfaceId	string		required	"DO_NOT_SET_AT_CLIENTS"
2	resourceJobManager	ResourceJobManager		required

Locally Fork Jobs as OS processes

alternativeSSHHostName: If the login to ssh is different than the hostname itself, specify it here

sshPort: If a non-default port needs to used, specify it.

Struct: LOCALDataMovement

Key	Field	Type	Description	Requiredness	Default value
1	dataMovementInterfaceId	string		required	"DO_NOT_SET_AT_CLIENTS"

LOCAL

alternativeSCPHostName: If the login to scp is different than the hostname itself, specify it here

sshPort: If a non-defualt port needs to used, specify it.

Struct: SSHJobSubmission

Key	Field	Type	Description	Requiredness	Default value
1	jobSubmissionInterfaceId	string		required	"DO_NOT_SET_AT_CLIENTS"
2	securityProtocol	SecurityProtocol		required
3	resourceJobManager	ResourceJobManager		required
4	alternativeSSHHostName	string		optional
5	sshPort	i32		optional	22
6	monitorMode	MonitorMode		optional

Authenticate using Secured Shell

alternativeSSHHostName: If the login to ssh is different than the hostname itself, specify it here

sshPort: If a non-default port needs to used, specify it.

Struct: GlobusJobSubmission

Key	Field	Type	Description	Requiredness	Default value
1	jobSubmissionInterfaceId	string		required	"DO_NOT_SET_AT_CLIENTS"
2	securityProtocol	SecurityProtocol		required
3	globusGateKeeperEndPoint	list<string>		optional

Struct: UnicoreJobSubmission

Key	Field	Type	Description	Requiredness	Default value
1	jobSubmissionInterfaceId	string		required	"DO_NOT_SET_AT_CLIENTS"
2	securityProtocol	SecurityProtocol		required
3	unicoreEndPointURL	string		required

Unicore Job Submission

unicoreEndPointURL: unicoreGateway End Point. The provider will query this service to fetch required service end points. authenticationMode The authenticationMode defines the way certificate is fetched.

Struct: CloudJobSubmission

Key	Field	Type	Description	Requiredness	Default value
1	jobSubmissionInterfaceId	string		required	"DO_NOT_SET_AT_CLIENTS"
2	securityProtocol	SecurityProtocol		required
3	nodeId	string		required
4	executableType	string		required
5	providerName	ProviderName		required
6	userAccountName	string		required

Cloud Job Submission

Struct: JobSubmissionInterface

Key	Field	Type	Description	Requiredness	Default value
1	jobSubmissionInterfaceId	string		required
2	jobSubmissionProtocol	JobSubmissionProtocol		required
3	priorityOrder	i32		required	0

Job Submission Interfaces

jobSubmissionInterfaceId: The Job Submission Interface has to be previously registered and referenced here.

priorityOrder: For resources with multiple interfaces, the priority order should be selected. Lower the numerical number, higher the priority

Struct: DataMovementInterface

Key	Field	Type	Description	Requiredness	Default value
1	dataMovementInterfaceId	string		required
2	dataMovementProtocol	DataMovementProtocol		required
3	priorityOrder	i32		required	0

Data Movement Interfaces

dataMovementInterfaceId: The Data Movement Interface has to be previously registered and referenced here.

priorityOrder: For resources with multiple interfaces, the priority order should be selected. Lower the numerical number, higher the priority

Struct: ComputeResourceDescription

Key	Field	Type	Description	Requiredness	Default value
1	computeResourceId	string		required	"DO_NOT_SET_AT_CLIENTS"
2	hostName	string		required
3	hostAliases	list<string>		optional
4	ipAddresses	list<string>		optional
5	resourceDescription	string		optional
6	batchQueues	list<BatchQueue>		optional
7	fileSystems	map<FileSystems, string>		optional
8	jobSubmissionInterfaces	list<JobSubmissionInterface>		optional
9	dataMovementInterfaces	list<DataMovementInterface>		optional
10	maxMemoryPerNode	i32		optional

Computational Resource Description

computeResourceId: Airavata Internal Unique Identifier to distinguish Compute Resource.

hostName: Fully Qualified Host Name.

hostAliases: Aliases if any.

ipAddress: IP Addresses of the Resource.

resourceDescription: A user friendly description of the resource.

JobSubmissionProtocols: A computational resources may have one or more ways of submitting Jobs. This structure will hold all available mechanisms to interact with the resource. The key is the priority

DataMovementProtocol: Option to specify a prefered data movement mechanism of the available options.

fileSystems: Map of file systems type and the path.