Exporting data using Slider app package¶

Slider app package allows an application instance to export data about itself, usually to be discoverable by the clients and IT admins. The main classes of export specified via the application package are as follows:

• Exporting configurations
• Exporting URLs, host:ports, or other information constructed based on the deployment context and applied configuration

All exports are specified in the metadata file, metainfo.xml. An export can be at the level of the application - it means there is only one copy of this data or at the level of a component instance - there is one copy of the data per component instance.

Exporting configurations¶

Slider application packages accept an appConfig.json file for all application configuration supplied by the user. Any property whose name starts with "site" are considered configuration. Specifications InstanceConfiguration describes the naming convention.

Export specific configs¶

By default all configurations are exported (e.g. http://host1:44500/ws/v1/slider/publisher/slider/storm-site). They can be disabled by specifying <exportedConfigs>None</exportedConfigs> under <application>. If you want to explicitly specify what to publish you can use comma separated named such as <exportedConfigs>storm-site,another-site</exportedConfigs>.

Which component is responsible for export¶

By default an arbitrary master is chosen as the master responsible for exporting the config. What this means is that when this master is STARTED the applied config known at that time is exported. Otherwise, you can specify which master component type should export configuration by specifying <publishConfig>true</publishConfig> under <component>.

Exporting formatted data at application instance level¶

At application level one or more exportGroups can be defined where each group can contain one or more exported name-value pairs. The sample export group shown below exports two data items app.monitor and nimbus.host_port for a Storm application instance.

<exportGroups>
  <exportGroup>
    <name>QuickLinks</name>
    <exports>
      <export>
        <name>org.apache.slider.monitor/name>
        <value>http://${STORM_UI_SERVER_HOST}:${site.storm-site.ui.port}</value>
      </export>
      <export>
        <name>nimbus.host_port</name>
        <value>${NIMBUS_HOST}:${site.storm-site.nimbus.thrift.port}</value>
      </export>
    </exports>
  </exportGroup>
</exportGroups>

You also need to specify as to which component type should export which data item. This is specified via <appExports> tag such as <appExports>QuickLinks-nimbus.host_port</appExports> for NIMBUS component type.

<components>
  <component>
    <name>NIMBUS</name>
    <category>MASTER</category>
    <appExports>QuickLinks-org.apache.slider.monitor,QuickLinks-nimbus.host_port</appExports>
    ...

Exporting formatted data at component instance level¶

Occasionally, component instances may need to export data on a per component instance basis. This is true if your application is a set of peers - e.g. memcached daemons, or you may have a need for each slave component instance to export some data. Currently, only the host name and dynamically allocated port values can be exported.

A component export can be declaraed within the component definition or along with application export group. When declared along the exportGroups you also need to specify the component that is responsible.

Add the export.

<exportGroup>
  <name>Servers</name>
  <exports>
    <export>
      <name>host_port</name>
      <value>${MEMCACHED_HOST}:${site.global.listen_port}</value>
    </export>
  </exports>
</exportGroup>

Define the component that exports it.

<components>
  <component>
    <name>MEMCACHED</name>
    <category>MASTER</category>
    <compExports>Servers-host_port</compExports>
    ...

Alternatively specify the export in the components section. For example:

<componentExports>
   <componentExport>
     <name>log_viewer_port</name>
     <value>${MEMCACHED_HOST}:${site.storm-site.logviewer.port}</value>
   </componentExport>
</componentExports>`

is used to specify that all Storm Supervisor component instances export the log viewer port. ${MEMCACHED_HOST} refers to the host of the component instances that is exporting. You can use ${THIS_HOST} as well.

When any component instance comes up the value for the component instance is exported and when a component instance goes down the export is removed. Exports are available @ .../ws/v1/slider/publisher/slider/componentinstancedata Sample export:

http://host1:44500/ws/v1/slider/publisher/slider/componentinstancedata
entries": {
   "container_1407354284611_0059_01_000006.log_viewer_port": "host2:60515"
}

A structured JSON for component exports is available at .../ws/v1/slider/publisher/exports. The container log and work directories are automatically made available.

Sample:

GET http://c6403.ambari.apache.org:8088/proxy/application_1415914904020_0015/ws/v1/slider/publisher/exports

{

  "exports": {
    "servers": {
        "description": "Servers",
        "updated": 1415991500805,
        "updatedTime": "Fri Nov 14 18:58:20 UTC 2014",
        "entries": { },
        "empty": true
    },
    "container_log_dirs": {
        "description": "container_log_dirs",
        "updated": 1415991500807,
        "updatedTime": "Fri Nov 14 18:58:20 UTC 2014",
        "entries": { },
        "empty": true
    },
    "container_work_dirs": {
        "description": "container_work_dirs",
        "updated": 1415991500807,
        "updatedTime": "Fri Nov 14 18:58:20 UTC 2014",
        "entries": { },
        "empty": true
    }
  }
}


GET http://c6403.ambari.apache.org:8088/proxy/application_1415914904020_0016/ws/v1/slider/publisher/exports/servers

{
  "description": "Servers",
  "updated": 1415991750794,
  "updatedTime": "Fri Nov 14 19:02:30 UTC 2014",
  "entries": {
      "host_port": [
          {
              "value": "c6403.ambari.apache.org:33387",
              "containerId": "container_1415914904020_0016_01_000002",
              "tag": "1",
              "level": "component",
              "updatedTime": "Fri Nov 14 19:02:30 UTC 2014"
          }
      ]
  },
  "empty": false
}