Tashi: Your Faithful Cluster Manager
Apache Tashi

Setting up Tashi on a single test machine


There are several dependencies you need to setup Tashi, they include:

  • Qemu/KVM or Xen
  • Python 2.6 or greater
  • RPyC (version 3.1)
  • MySQL or sqlite, with their python interfaces, if you wish to use them as the data backend

Please note that the code distribution contains an installation walkthrough, which is more up to date than this document.

Once these are setup and configured, check out the Tashi code:

svn co http://svn.apache.org/repos/asf/incubator/tashi/trunk ./tashi

Then, run make:

mryan3@firsthost:~/scratch/tashi$ make
Generating Python code for 'layoutlocality.thrift'...
Copying generated code to 'tashi.services' package...
Symlinking in clustermanager...
Symlinking in nodemanager...
Symlinking in tashi-client...
Symlinking in primitive...
Building nmd...


Now, you may need to need to modify some configuration parameters to make Tashi work in your environment. If you place a file called Tashi.cfg in etc, it will override settings in TashiDefaults.cfg. Some pertinent options:

  • [ClusterManager]data - This specifies whether you use a pickled data file or the SQL backend
  • [NodeManager]VmControl - This specifies which VMM backend you use

If you haven't installed Tashi in /usr/lib/python2.5/site-packages/ or the equivalent folder, you must set the PYTHONPATH environment variable so that python can find the Tashi code:

mryan3@firsthost:~/scratch/tashi$ export PYTHONPATH=/usr/local/tashi/src

Start the cluster manager (CM) first:

mryan3@firsthost:~/scratch/tashi$ DEBUG=1 ./bin/clustermanager.py 
2009-02-19 11:19:04,276 [./bin/clustermanager.py:INFO] Using configuration file(s) ['./etc/TashiDefaults.cfg']
2009-02-19 11:19:04,276 [./bin/clustermanager.py:INFO] Starting cluster manager

In [1]: 

I recommend running in debug mode if you haven't got it up and running yet.

Then, add your first host and network to Tashi. If your box's hostname was "firsthost", an available network had VLAN ID 272, and you used the pickled data format, you'd do:

In [1]: from tashi.rpycservices.rpyctypes import Host, HostState, Network

In [2]: data.baseDataObject.hosts[1] = Host(d={'id':1,'name':'firsthost','state': HostState.Normal,'up':False})

In: data.baseDataObject.networks[1]=Network(d={'id':272,'name':'default'})

In [3]: data.baseDataObject.save()

In [4]:

The name "default" for the network here is significant; automatic placement will only take place to the network called "default".

If you were using MySQL (or sqlite), you'd have to populate the relevant tables in the database.

clustermanager:/opt/tashi/bin# sqlite3 /var/tmp/cm_sqlite.dat
SQLite version 3.5.9
Enter ".help" for instructions
sqlite> insert into networks values (272, 'default');
sqlite> insert into hosts (name) values ('firsthost');

Then you may either create a host lock or quit the CM and restart it to force the creation of host locks.

In [4]: import threading

In [5]: data.baseDataObject.hosts[1]._lock = threading.Lock()

Now, you can start the node manager (NM). You may have to run this as root if you have trouble getting VMs to start as a normal user:

root@firsthost:/home/mryan3/scratch/tashi$ DEBUG=1 ./bin/nodemanager.py 
2009-02-19 11:25:50,470 [__main__:INFO] Using configuration file(s) ['./etc/TashiDefaults.cfg']
2009-02-19 11:25:50,475 [./src/tashi/nodemanager/vmcontrol/qemu.pyc:INFO] No vm information found in /var/tmp/VmControlQemu/

In [1]: 

After a few seconds, you should see no error messages on the NM and you should be able to run "tashi-client getHosts":

mryan3@firsthost:~/scratch/tashi$ ./bin/tashi-client.py getHosts
 id name      decayed up   state  version                      memory cores
 1  firsthost False   True Normal Wed Feb  4 15:20:15 EST 2009 3894   4    

Now that those are running, you can start the scheduler:

mryan3@firsthost:~/scratch/tashi$ ./bin/primitive.py

VM deployment (XEN)

Most Tashi users and developers currently use the Qemu interface, so the Xen interface may be more buggy. Please report any problems observed.

Have your networking environment set up, so that the virtual machines may be attached to the proper virtual bridges.

In Xen 3, virtual disks are attached using blktap, so make sure that the proper blktap module is loaded, and blktapctrl is running properly.

Place a disk image in /dfs (or wherever you configured it to be).

Have tashi-client start up a new virtual machine.

VM deployment (QEMU)

Make sure an image is placed in /dfs (or wherever you configured it to be).

mryan3@firsthost:~/scratch/tashi$ ls -la /dfs
total 2385696
drwxr-xr-x 2 mryan3 mryan3       4096 2009-02-19 12:21 .
drwxrwxrwt 7 root   root         4096 2009-02-19 12:20 ..
-rw-r--r-- 1 mryan3 mryan3 2440556544 2009-02-19 12:22 CentOS-5.2-i386.qcow2

Also, create a file called /etc/qemu-ifup.1. If you want to add the tap device to a bridge, you can make it contain something like this:


/sbin/ifconfig $1 up
/usr/sbin/brctl addif mybridge $1
exit 0

Otherwise, you can simply make it an empty executable script.

And finally, you can use the client to try to create a VM:

mryan3@firsthost:~/scratch/tashi$ ./bin/tashi-client.py createVm --name foobar --disks CentOS-5.2-i386.qcow2
    hostObj: None
    hostId: None
    name: foobar
    typeObj: {'cores': 1, 'memory': 128, 'id': 1, 'name': 'foo'}
    userId: 13090
    decayed: False
    disks: [
        {'uri': 'CentOS-5.2-i386.qcow2', 'persistent': False}
    vmId: None
    userObj: None
    state: Pending
    nics: [
        {'mac': '52:54:00:e5:1d:ca', 'network': 1}
    type: 1
    id: 1
    hints: {}

Disk images can be created by simply using Qemu/KVM on a workstation and placing the qcow file in the appropriate directory.

The permission problems around the NM are usually related to creating the TAP device. Try running the "QEMU command:" reported by the NM manually if you continue to have problems.

The VM will use a TAP device on your system and will be accessible over it. If you statically configure the guest for this test, that will work. It is also possible to setup a bridge and run a DHCP server on the host to provide IP addresses to the guest. In either case, you should be able to log into your guest shortly after that command completes.

mryan3@firsthost:~/scratch/tashi$ ssh root@
The authenticity of host ' (' can't be established.
RSA key fingerprint is 3d:4b:43:25:05:d8:89:23:ec:9b:6c:1b:42:59:e7:70.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '' (RSA) to the list of known hosts.
root@'s password: 
Last login: Thu Jan  8 10:25:10 2009
[root@localhost ~]# 

It is also possible to access the VM via VNC:

mryan3@firsthost:~/scratch/tashi$ ./bin/tashi-client vmmspecificcall --instance foobar --arg startvnc
VNC started on firsthost:5900
mrayn3@firsthost:~/scratch/tashi$ vncviewer firsthost


Apache Tashi is an effort undergoing incubation at The Apache Software Foundation (ASF), sponsored by the Apache Incubator PMC. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF.