Title: New Configuration System
Notice: Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
.
http://www.apache.org/licenses/LICENSE-2.0
.
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
[TOC]
This page describes a new configuration system that will be added to VCL that can be used to dynamically configure deployed systems.
Database schema
---------------
config:
* id - id of record
* name - name of record
* ownerid - owner of this record (reference to user.id)
* configtypeid - type of record (reference to configtype.id)
* data - any data associated with this (ex. puppet manifest)
* optional - 0/1 - when this config is mapped to something, can the user specify to apply it or not
configtype:
* id - id of record
* name - name of record
* moduleid - id of module that handles this type of record (reference to module.id)
> initial types: puppet, subimage, shellcommand, perlfunction, software
configvariable:
* id - id of record
* name - name of record
* configid - config this is associated with (reference to config.id)
* value - default value of this variable
* required - 0/1 - when mapped to something, is this variable always applied
* ask - 0/1 - will the user be prompted for a value for this variable, or will the default value always be used
* key - string in config.data to replace with the value of this variable
* datatype - enum(int, multiint, string) - type of this variable so that the frontend knows how to validate it
There are some initial, special names:
* subimage - used for clusters, specifies a subimage to be deployed; when deployed, configinstancevariable.value will be reservation.id of the subimage
* min - used in conjunction with subimage to specify a minimum number of those subimages
* max - used in conjunction with subimage to specify a maximum number of those subimages
* runbefore - used to relate to other configs to specify that this should be run before the ones specified in 'value'
* runafter - used to relate to other configs to specify that this should be run after the ones specified in 'value'
* getdata - ??
configinstance:
> This is similar to the config table, but is for deployed systems.
* id - id of record
* reservationid - reservation this is associated with (references reservation.id)
* configid - config entry being applied (references config.id)
* status - new/processing/completed??
configinstancevariable:
> This is similar to the configvariable table, but is for deployed systems.
* configinstanceid - config instance this is associated with (references configinstance.id)
* configvariableid - config variable being applied (references configvariable.id)
* value - similar to configvariable.value, but can be set by user to different value if configvariable.ask is set to 1
configmap:
>This is for mapping configs to various resources or other items in VCL.
* configid - config being mapped (references config.id)
* subid - id from specific resource table (ex. image.id)
* configmaptypeid - type of resource or other item in VCL this is mapped to (references configmaptype.id)
* affiliationid - allows configs to only be mapped for a specific affiliation, use the Global affilation to map to all (references affiliation.id)
* disabled - 0/1 (a little complicated, normally 0) allows exceptions to a config being applied; set to 1 if a config would be mapped due to a general mapping, but want to disable for a specific instance (ex. if a config is applied for everything deployed by a certain provisioning module, but you don't want it applied for a certain image, you would have an entry in this table mapping it to the provisioning module with disabled set to 0, then you would also have an entry for the image with disabled set to 1)
* stage - start_load/post_load/(others?) which stage in the provisioning process where this record should be applied
configmaptype:
* id - id of this record
* name - name of this record
Initial types:
* image
* OStype
* provisioning
Examples
--------
### Assigning a VLAN to an image
This example shows how to assign a VLAN to an image.
For this example, we'll use the following values from other tables:
* module.id for handling VLAN config type: 67
* image.id for the image in this example: 524
* affiliation.id for the desired affiliation: 6
* reservation.id for the image: 2748
* user.id that owns the configs: 54
configtype:
config:
| id | name | ownerid | configtypeid | data | optional |
| 77 | VLAN 30 | 54 | 5 | 30 | 0 |
configvariable:
| id | name | configid | value | required | ask | key | datatype |
| 486 | VLAN | 77 | 30 | 1 | 0 | (empty) | int |
configmaptype:
configmap:
| configid | subid | configmaptypeid | affiliationid | disabled | stage |
| 77 | 524 | 1 | 6 | 0 | post_load |
configinstance:
| id | reservationid | configid | status |
| 6854 | 2748 | 77 | new |
configinstancevariable:
| configinstanceid | configvariableid | value |
| 6854 | 486 | 30 |
###Hadoop cluster with variable amount of slave nodes
This example shows how a hadoop cluster can be requested with 5-10 slave nodes.
It can be useful to have the variable amount because 10 nodes may
be desired, but you may want to cluster anyway if only 5 nodes are
available or if 10 are requested, but 2 of them fail at deploy time.
For this example, we'll use the following values from other tables:
* module.id for handling subimage config type: 58
* image.id for the Hadoop master image: 453
* image.id for the Hadoop slave image: 454
* affiliation.id for the desired affiliation: 5
* reservation.id for the Hadoop master image: 2351
* reservation.id for the Hadoop slave images: 2352-2361
* user.id that owns the configs: 9
configtype:
| id | name | moduleid |
| 2 | subimage | 58 |
config:
| id | name | ownerid | configtypeid | data | optional |
| 59 | hadoop cluster | 9 | 2 | (empty) | 0 |
configvariable:
| id | name | configid | value | required | ask | key | datatype |
| 146 | subimage | 59 | 454 | 1 | 0 | (empty) | int |
| 147 | min | 59 | 1 | 1 | 1 | (empty) | int |
| 148 | max | 59 | 50 | 1 | 1 | (empty) | int |
configmaptype:
configmap:
| configid | subid | configmaptypeid | affiliationid | disabled | stage |
| 59 | 453 | 1 | 5 | 0 | start_load |
configinstance:
| id | reservationid | configid | status |
| 5023 | 2351 | 59 | new |
configinstancevariable:
| configinstanceid | configvariableid | value |
| 5023 | 146 | 2352 |
| 5023 | 146 | 2353 |
| 5023 | 146 | 2354 |
| 5023 | 146 | 2355 |
| 5023 | 146 | 2356 |
| 5023 | 146 | 2357 |
| 5023 | 146 | 2358 |
| 5023 | 146 | 2359 |
| 5023 | 146 | 2360 |
| 5023 | 146 | 2361 |
| 5023 | 147 | 5 |
| 5023 | 148 | 10 |
### SAS cluster
This example shows how to configure a SAS cluster of 3 nodes: meta, midtier, and apps so that they are all deployed and then started in the correct order.
For this example, we'll use the following values from other tables:
* module.id for handling puppet config type: 57
* module.id for handling subimage config type: 58
* module.id for handling shellcommand config type: 59
* image.id for the SAS meta image: 728
* image.id for the SAS midtier image: 729
* image.id for the SAS apps image: 730
* affiliation.id for the desired affiliation: 7
* reservation.id for the SAS meta image: 2466
* reservation.id for the SAS midtier image: 2467
* reservation.id for the SAS apps image: 2468
* user.id that owns the configs: 15
configtype:
| id | name | moduleid |
| 1 | puppet | 57 |
| 2 | subimage | 58 |
| 3 | shellcommand | 59 |
config:
| id | name | ownerid | configtypeid | data | optional |
| 76 | SAS apps | 15 | 2 | (empty) | 0 |
| 77 | SAS midtier | 15 | 2 | (empty) | 0 |
| 78 | SAS meta config | 15 | 1 | (puppet manifest) | 0 |
| 79 | SAS apps config | 15 | 1 | (puppet manifest) | 0 |
| 80 | SAS midtier config | 15 | 1 | (puppet manifest) | 0 |
| 81 | SAS meta start | 15 | 3 | (startup commands) | 0 |
| 82 | SAS apps start | 15 | 3 | (startup commands) | 0 |
| 83 | SAS midtier start | 15 | 3 | (startup commands) | 0 |
configvariable:
| id | name | configid | value | required | ask | key | datatype |
| 268 | subimage | 76 | 729 | 1 | 0 | (empty) | int |
| 269 | subimage | 77 | 730 | 1 | 0 | (empty) | int |
| 270 | runafter | 78 | 268,269 | 1 | 0 | (empty) | int |
| 271 | runafter | 79 | 270 | 1 | 0 | (empty) | int |
| 272 | runafter | 80 | 271 | 1 | 0 | (empty) | int |
| 273 | runafter | 81 | 272 | 1 | 0 | (empty) | int |
| 274 | runafter | 82 | 273 | 1 | 0 | (empty) | int |
| 275 | runafter | 83 | 274 | 1 | 0 | (empty) | int |
configmaptype:
configmap:
| configid | subid | configmaptypeid | affiliationid | disabled | stage |
| 76 | 728 | 1 | 7 | 0 | start_load |
| 77 | 728 | 1 | 7 | 0 | start_load |
| 78 | 728 | 1 | 7 | 0 | post_load |
| 79 | 728 | 1 | 7 | 0 | post_load |
| 80 | 728 | 1 | 7 | 0 | post_load |
| 81 | 728 | 1 | 7 | 0 | post_load |
| 82 | 728 | 1 | 7 | 0 | post_load |
| 83 | 728 | 1 | 7 | 0 | post_load |
configinstance:
| id | reservationid | configid | status |
| 6005 | 2466 | 76 | new |
| 6006 | 2466 | 77 | new |
| 6007 | 2466 | 78 | new |
| 6008 | 2466 | 79 | new |
| 6009 | 2466 | 80 | new |
| 6010 | 2466 | 81 | new |
| 6011 | 2466 | 82 | new |
| 6012 | 2466 | 83 | new |
configinstancevariable:
| configinstanceid | configvariableid | value |
| 6005 | 268 | 2467 |
| 6006 | 269 | 2468 |
| 6007 | 270 | 6005,6006 |
| 6008 | 271 | 6007 |
| 6009 | 272 | 6008 |
| 6010 | 273 | 6009 |
| 6011 | 274 | 6010 |
| 6012 | 275 | 6011 |