Apache Slider Application Definition¶

App definition is a declarative definition of a YARN application describing its content. The AppDefinition is used in conjunction with the AppPackage. Application definition is an XML file and is packaged as metainfo.xml.

Structure¶

Non-mandatory fields are described in italics.

The fields to describe an application is as follows:

name: the name of the application
version: the version of the application. name and version together uniquely identify an application.
type: the type of the application. "YARN-APP" identifies an application definition suitable for YARN.
minHadoopVersion: the minimum version of Apache Hadoop on which the app can run
components: the list of component that the application is comprised of
osSpecifics: OS specific package information for the application
commandScript: application wide commands may also be defined. The command is executed on a component instance that is a client
comment: describe the application
exportedConfigs: what configs to export

An application contains several component. The fields associated with a component are:

name: name of the component
category: type of the component - MASTER, SLAVE, and CLIENT
minInstanceCount: the minimum number of instances required for this component
maxInstanceCount: maximum number of instances allowed for a component
appExports: which application level exports are exported by this component
compExports: which component level exports are exported by this component

A component may contain component specific exports. Refer to Specifying Exports for more details.

name: the name of the export
value: the template that will be populated by Slider and then exported

A component contains command script for the component.

commandScript: the script that implements the commands.
- script: the script location - relative to the AppPackage root
- scriptType: type of the script
- timeout: default timeout of the script

Custom commands may be added to a component.

customCommands: any additional commands available for the component and their implementation

An application definition also includes the package used to install the application. Its typically a tarball or some other form of package that does not require root access to install. The details of what happens during install is captured in the command script.

osSpecific: details on a per OS basis
osType: "any" refers to any OS ~ typical for tarballs
packages: list of packages that needs to be deployed
type: type of package
name: name of the package
location: location of the package (can be a relative folder within the parent AppPackage)

Application can define a an order of activation which dictates if some component activation must follow the successful activation of other components.

command: specifies the component and the command in the form component-command currently, START is the only valid command
requires: specifies component and their state that the command depends on, provided in the form component-state currently, STARTED is the only valid state

Applications can also advertise a set of properties (typically urls) that can only be bound when the application components are active. One such item can be the jmx endpoint. The properties to be advertised are organized as export groups (exportGroup) and each group can export one or more properties organized as a property bag. These values are visible through the registry service.

name: specifies the name of the export group

Each exportGroup contains one or more exports. Refer to Specifying Exports for more details.

name: the name of the export
value: the template that will be populated by Slider and then exported

Sample metainfo.xml

<metainfo>
  <schemaVersion>2.0</schemaVersion>
  <application>
    <name>HBASE</name>
    <version>0.96.0.2.1.1</version>
    <type>YARN-APP</type>
    <minHadoopVersion>2.1.0</minHadoopVersion>
    <components>
      <component>
        <name>HBASE_MASTER</name>
        <category>MASTER</category>
        <minInstanceCount>1</minInstanceCount>
        <maxInstanceCount>2</maxInstanceCount>
        <exportedConfigs>hbase-site</exportedConfigs>
        <commandScript>
          <script>scripts/hbase_master.py</script>
          <scriptType>PYTHON</scriptType>
          <timeout>600</timeout>
        </commandScript>
      </component>

      <component>
        <name>HBASE_REGIONSERVER</name>
        <category>SLAVE</category>
        <minInstanceCount>1</minInstanceCount>
        ...
      </component>

    </components>

    <osSpecifics>
      <osSpecific>
        <osType>any</osType>
        <packages>
          <package>
            <type>tarball</type>
            <name>hbase-0.96.1-tar.gz</name>
            <location>package/files</location>
          </package>
        </packages>
      </osSpecific>
    </osSpecifics>

    <commandOrders>
      <commandOrder>
        <command>HBASE_REGIONSERVER-START</command>
        <requires>HBASE_MASTER-STARTED</requires>
      </commandOrder>
    </commandOrders>

    <exportGroups>
      <exportGroup>
        <name>QuickLinks</name>
          <exports>
            <export>
              <name>JMX_Endpoint</name>
              <value>http://${HBASE_MASTER_HOST}:${site.hbase-site.hbase.master.info.port}/jmx</value>
            </export>
            <export>
              <name>Master_Status</name>
              <value>http://${HBASE_MASTER_HOST}:${site.hbase-site.hbase.master.info.port}/master-status</value>
            </export>
         </exports>
      </exportGroup>
    </exportGroups>

  </application>
</metainfo>

Apache Slider (incubating)

Apache Slider Application Definition¶

Structure¶