Thrift module: compute_resource_model

ModuleServicesData typesConstants
compute_resource_model BatchQueue
CloudJobSubmission
ComputeResourceDescription
DMType
FileSystems
GlobusJobSubmission
JobManagerCommand
JobSubmissionInterface
JobSubmissionProtocol
LOCALSubmission
MonitorMode
ProviderName
ResourceJobManager
ResourceJobManagerType
SSHJobSubmission
UnicoreJobSubmission

Enumerations

Enumeration: ResourceJobManagerType

* Enumeration of local resource job manager types supported by Airavata * * FORK: * Forking of commands without any job manager * * PBS: * Job manager supporting the Portal Batch System (PBS) protocol. Some examples include TORQUE, PBSPro, Grid Engine. * * SLURM: * The Simple Linux Utility for Resource Management is a open source workload manager. * * UGE: * Univa Grid Engine, a variation of PBS implementation. * * LSF: * IBM Platform Load Sharing Facility is dominantly installed on IBM clusters. *

FORK0
PBS1
SLURM2
LSF3
UGE4

Enumeration: JobManagerCommand

Enumeration of resource job manager commands SUBMISSION: Ex: qsub, sbatch JOBMONITORING: Ex: qstat, squeue DELETION: Ex: qdel, scancel CHECK_JOB: Detailed Status about the Job. Ex: checkjob SHOW_QUEUE: List of Queued Job by the schedular. Ex: showq SHOW_RESERVATION: List all reservations. Ex:showres, show_res SHOW_START: Display the start time of the specified job. Ex: showstart

SUBMISSION0
JOB_MONITORING1
DELETION2
CHECK_JOB3
SHOW_QUEUE4
SHOW_RESERVATION5
SHOW_START6

Enumeration: FileSystems

Enumeration of File Systems on the resource FORK: Forking of commands without any job manager PBS: Job manager supporting the Portal Batch System (PBS) protocol. Some examples include TORQUE, PBSPro, Grid Engine. UGE: Univa Grid Engine, a variation of PBS implementation. SLURM: The Simple Linux Utility for Resource Management is a open source workload manager.

HOME0
WORK1
LOCALTMP2
SCRATCH3
ARCHIVE4

Enumeration: JobSubmissionProtocol

Enumeration of Airavata supported Job Submission Mechanisms for High Performance Computing Clusters. SSH: Execute remote job submission commands using via secure shell protocol. GRAM: Execute remote jobs via Globus GRAM service. UNICORE: Execute remote jobs via Unicore services

LOCAL0
SSH1
GLOBUS2
UNICORE3
CLOUD4
SSH_FORK5
LOCAL_FORK6

Enumeration: MonitorMode

Monitoring modes POLL_JOB_MANAGER: GFac need to pull job status changes. XSEDE_AMQP_SUBSCRIBE: Server will publish job status changes to amqp servert.

POLL_JOB_MANAGER0
JOB_EMAIL_NOTIFICATION_MONITOR1
XSEDE_AMQP_SUBSCRIBE2
FORK3

Enumeration: DMType


COMPUTE_RESOURCE0
STORAGE_RESOURCE1

Enumeration: ProviderName

Provider name

EC20
AWSEC21
RACKSPACE2

Data structures

Struct: ResourceJobManager

KeyFieldTypeDescriptionRequirednessDefault value
1resourceJobManagerIdstringrequired"DO_NOT_SET_AT_CLIENTS"
2resourceJobManagerTypeResourceJobManagerTyperequired
3pushMonitoringEndpointstringoptional
4jobManagerBinPathstringoptional
5jobManagerCommandsmap<JobManagerCommand, string>optional

Resource Job Manager Information resourceJobManagerType: A typical HPC cluster has a single Job Manager to manage the resources. pushMonitoringEndpoint: If the job manager pushes out state changes to a database or bus, specify the service endpoint. Ex: Moab Web Service, Moab MongoDB URL, AMQP (GLUE2) Broker jobManagerBinPath: Path to the Job Manager Installation Binary directory. jobManagerCommands: An enumeration of commonly used manager commands.

Struct: BatchQueue

KeyFieldTypeDescriptionRequirednessDefault value
1queueNamestringrequired
2queueDescriptionstringoptional
3maxRunTimei32optional
4maxNodesi32optional
5maxProcessorsi32optional
6maxJobsInQueuei32optional
7maxMemoryi32optional

Batch Queue Information on SuperComputers maxRunTime: Maximum allowed run time in hours.

Struct: LOCALSubmission

KeyFieldTypeDescriptionRequirednessDefault value
1jobSubmissionInterfaceIdstringrequired"DO_NOT_SET_AT_CLIENTS"
2securityProtocoldata_movement_models.SecurityProtocolrequired
3resourceJobManagerResourceJobManagerrequired

Locally Fork Jobs as OS processes alternativeSSHHostName: If the login to ssh is different than the hostname itself, specify it here sshPort: If a non-default port needs to used, specify it.

Struct: SSHJobSubmission

KeyFieldTypeDescriptionRequirednessDefault value
1jobSubmissionInterfaceIdstringrequired"DO_NOT_SET_AT_CLIENTS"
2securityProtocoldata_movement_models.SecurityProtocolrequired
3resourceJobManagerResourceJobManagerrequired
4alternativeSSHHostNamestringoptional
5sshPorti32optional22
6monitorModeMonitorModeoptional
7batchQueueEmailSenderslist<string>optional

Authenticate using Secured Shell alternativeSSHHostName: If the login to ssh is different than the hostname itself, specify it here sshPort: If a non-default port needs to used, specify it. batchQueueEmailSenders: If a resource always sends the monitoring from a specific address, specify the full email address. If a resource sends emails from multiple addresses ( example: based on the submitted login node) then use the wildchar * to indicate the same. Example: *@*.example.com or *@example.com

Struct: GlobusJobSubmission

KeyFieldTypeDescriptionRequirednessDefault value
1jobSubmissionInterfaceIdstringrequired"DO_NOT_SET_AT_CLIENTS"
2securityProtocoldata_movement_models.SecurityProtocolrequired
3globusGateKeeperEndPointlist<string>optional

Struct: UnicoreJobSubmission

KeyFieldTypeDescriptionRequirednessDefault value
1jobSubmissionInterfaceIdstringrequired"DO_NOT_SET_AT_CLIENTS"
2securityProtocoldata_movement_models.SecurityProtocolrequired
3unicoreEndPointURLstringrequired

Unicore Job Submission unicoreEndPointURL: unicoreGateway End Point. The provider will query this service to fetch required service end points. authenticationMode The authenticationMode defines the way certificate is fetched.

Struct: CloudJobSubmission

KeyFieldTypeDescriptionRequirednessDefault value
1jobSubmissionInterfaceIdstringrequired"DO_NOT_SET_AT_CLIENTS"
2securityProtocoldata_movement_models.SecurityProtocolrequired
3nodeIdstringrequired
4executableTypestringrequired
5providerNameProviderNamerequired
6userAccountNamestringrequired

Cloud Job Submission

Struct: JobSubmissionInterface

KeyFieldTypeDescriptionRequirednessDefault value
1jobSubmissionInterfaceIdstringrequired
2jobSubmissionProtocolJobSubmissionProtocolrequired
3priorityOrderi32required0

Job Submission Interfaces jobSubmissionInterfaceId: The Job Submission Interface has to be previously registered and referenced here. priorityOrder: For resources with multiple interfaces, the priority order should be selected. Lower the numerical number, higher the priority

Struct: ComputeResourceDescription

KeyFieldTypeDescriptionRequirednessDefault value
1computeResourceIdstringrequired"DO_NOT_SET_AT_CLIENTS"
2hostNamestringrequired
3hostAliaseslist<string>optional
4ipAddresseslist<string>optional
5resourceDescriptionstringoptional
6enabledbooloptional
7batchQueueslist<BatchQueue>optional
8fileSystemsmap<FileSystems, string>optional
9jobSubmissionInterfaceslist<JobSubmissionInterface>optional
10dataMovementInterfaceslist<data_movement_models.DataMovementInterface>optional
11maxMemoryPerNodei32optional

Computational Resource Description computeResourceId: Airavata Internal Unique Identifier to distinguish Compute Resource. hostName: Fully Qualified Host Name. hostAliases: Aliases if any. ipAddress: IP Addresses of the Resource. resourceDescription: A user friendly description of the resource. JobSubmissionProtocols: A computational resources may have one or more ways of submitting Jobs. This structure will hold all available mechanisms to interact with the resource. The key is the priority DataMovementProtocol: Option to specify a prefered data movement mechanism of the available options. fileSystems: Map of file systems type and the path.