Installing VCL 2.2 Contents: I. Database (MySQL) II. Web (frontend) III. Management Node (backend) IV. Adding extra local accounts V. Adding LDAP authentication I. Database (MySQL) Prerequisites: MySQL 5 installed and running Installation: 1. run the mysql command line client mysql 2. create a database in mysql named for use with VCL CREATE DATABASE vcl; 3. create a user with SELECT, INSERT, UPDATE, and DELETE privileges on the database you just created ** NOTE: Replace vcluserpassword with your own password! GRANT SELECT,INSERT,UPDATE,DELETE ON vcl.* TO 'vcluser'@'localhost' IDENTIFIED BY 'vcluserpassword'; 4. exit from the mysql command line client exit 5. import vcl.sql file into database mysql vcl < mysql/vcl.sql II. Web (frontend) Prerequisites: Your web server should meet the following criteria before installing the VCL Frontend Code: * Apache HTTP Server v1.3 or v2.x with SSL enabled - while VCL may run under another webserver capable of running PHP code, it has only been tested to work with Apache HTTP Server * PHP 5 * php modules that should be installed (depending on your Linux distro, some of these may be compiled in to php instead of being a separate module): - php-gd - php-json - php-mcrypt - php-mysql - php-openssl - php-sysvsem - php-xml - php-xmlrpc - php-ldap (if you will be using ldap authentication) * If you are running RedHat or CentOS, these can be installed with yum install -y httpd mod_ssl php-gd php-mcrypt php-mysql php-xml php-xmlrpc php-ldap * useful to have the server set up to be able to send debugging emails * php-mcrypt requires libmcrypt and mcrypt libraries as dependencies. These may need to be installed first. Installation: 1. move the web directory somewhere that your web server can access it (you'll probably also want to rename it to 'vcl') ex: mv web /var/www/html/vcl 2. copy/rename secrets-default.php to secrets.php 3. modify vcl/.ht-inc/secrets.php * set $vclhost, $vcldb, $vclusername, and $vclpassword to match your database setup * create random passwords for $mcryptkey, $mcryptiv, and $pemkey - $mcryptiv must be 8 hex characters 4. run the genkeys.sh script from within vcl/.ht-inc and give it $pemkey from secrets.php as the passphrase (3 times, copy/paste is a good idea here) 5. copy/rename conf-default.php to conf.php 6. modify vcl/.ht-inc/conf.php to match your site - COOKIEDOMAIN needs to be the domain name your web server is using, or left blank if you are accessing it by IP only ** You really need to modify every entry in the "Things in this section must be modified" part of the file.** 7. *NOTICE* JpGraph 2.x is no longer available. JpGraph 3.x is released under a dual license. QPL 1.0 (Qt Free Licensee). Free for non-commercial, open-source or educational use (JpGraph Professional License for commercial use). If you are planning to use this for commercial use and don't want to pay for JpGraph, you can safely skip this step with the only side effect of not being able to display a few graphs on the statistics page. Download JpGraph from http://www.aditus.nu/jpgraph/jpdownload.php * download the 3.x series, extract it, and copy the src directory from it to vcl/.ht-inc/jpgraph 8. make the .ht-inc/maintenance directory writable by the web server user - i.e. if the httpd process is running as user 'apache' run 'chown apache .ht-inc/maintenance' 9. open a browser and open the testsetup.php page * i.e. if you set up your site to be https://my.server.org/vcl/ open https://my.server.org/vcl/testsetup.php 10. debug any issues reported by testsetup.php 11. now, open the index.php page in your browser 12. select Local Account and use 'admin' as the user and 'adminVc1passw0rd' as the password **Now would be a good time to change the password from the default by going to User Preferences** 13. click the "Management Nodes" link 14. enter the hostname and IP of your management node 15. click Add 16. fill in these additional required fields: * SysAdmin Email Address - error emails will be sent to this address * Install Path - this is parent directory under which image files will be stored - only required if doing bare metal installs or using VMWare with local disks * End Node SSH Identity Key Files - probably just enter "/etc/vcl/vcl.key" 17. optionally, fill in these unrequired fields: * Address for Shadow Emails - End users are sent various emails about the status of their reservations. If this field is configured, copies of all of those emails will be sent to this address. * Public NIC configuration method - this defaults to Dynamic DHCP - if DHCP is not available for the public interface of your nodes, you can set this to Static. Then, the IP configuration on the nodes will be manually set using Public Netmask, Public Gateway, Public DNS Server, and the IP address set for the computer under Manage Computers 18. click "Confirm Management Node" 19. click Submit 20. click the "Management Nodes" link 21. select "Edit Management Node Grouping" 22. click Submit 23. select the checkbox for your management node 24. click Submit If you will only be using VMWare (and not xCAT), skip to ADDING VMHOSTS AND VIRTUAL MACHINES ADDING BARE METAL HOSTS FOR XCAT You can initially add individual computers or multiple computers all together. After you have added at least one computer, you will need to go to Manage Computers -> Edit Computer Information to additional ones. Adding Individual Computers 1. click "Manage Computers" 2. select the "Add Single Computer" radio button 3. click Submit 4. fill in Hostname, IP Address, owner (admin@Local), RAM, Proc Speed, Network Speed, select "blade" for Type, select "xCAT 2.x Provisioning" for "Provisioning Engine", and click the checkbox under "allcomputers", and "newimages" 5. click Confirm Computer 6. click Submit (don't worry about the fact that the computer you just added isn't listed after clicking Submit) 7. after you've configured your image library and your management node has started checking in, you should be able to make a reservation Adding Multiple Computers 1. click "Manage Computers" 2. select the "Add Multiple Computers" radio button 3. click Submit 4. fill in * Hostname - the hostnames of all the computers must have a numerical part that is sequential, use a % as a placeholder where that part would be * Start value - the first number of the numerical part of the hostname * End value - the last number of the numerical part of the hostname * Start IP Address - if using static public addresses, the IP addresses must be sequential; enter the first address here; if using DHCP, just enter something like 1.1.1.1 * End IP address - the last IP address of the sequence; if using DHCP, you'll need to enter something that would work out to the last address relative to Start IP Address (i.e. if adding 3 computers, use 1.1.1.1 for start and 1.1.1.3 for end) * Owner - owner of the computer * RAM * Processor Speed * Network Speed * Type - blade * Provisioning Engine - xCAT 2.x * check allComputers and newimages 5. click Confirm Computer 6. click Submit (don't worry about the fact that the computers you just added aren't listed after clicking Submit) 7. after you've configured your image library and your management node has started checking in, you should be able to make a reservation ADDING VMHOSTS AND VIRTUAL MACHINES If you are using standalone VMWare servers (i.e. ones that VCL did not deploy using xCAT), you first need to add the VMWare servers; then, you need to add the virtual machines. You can either add them individually (Adding Individual VMWare Servers/Virtual Machines), or if they have sequential hostnames and IP addresses, you can add them all at once (Adding Multiple VMWare Servers/Virtual Machines). Once you have added at least one computer, you can get to the "Add Single Computer" page by going to Manage Computers->Edit Computer Information and clicking Add. You can get to the "Add Multiple Computers" page by doing the same thing but checking the "Add Multiple" checkbox. Adding Individual VMWare Servers 1. click "Manage Computers" 2. select the "Add Single Computer" radio button 3. click Submit 4. fill in * Hostname * IP Address * State - vmhostinuse - double check this because you will not be able to change it later * owner (admin@Local) * RAM * Proc Speed * Network Speed * Type - blade * Provisioning Engine - xCAT 2.x Provisioning * click the checkbox under "allcomputers" 5. click Confirm Computer 6. click Submit 7. select a VM Host Profile to use with this host - you can edit the VM Host Profile later if needed 8. click Add Computer (don't worry about the fact that the computer you just added isn't listed after clicking Submit) Adding Individual Virtual Machines 1. click "Manage Computers" 2. select the "Add Single Computer" radio button 3. click Submit 4. fill in * Hostname * IP Address * State - maintenance * owner (admin@Local) * RAM * Proc Speed * Network Speed * Type - virtualmachine * Provisioning Engine - VMware * click the checkbox under "All VM Computers" and "newvmimages" 5. click Confirm Computer 6. click Submit (don't worry about the fact that the computer you just added isn't listed after clicking Submit) Adding Multiple VMWare Servers 1. click "Manage Computers" 2. select the "Add Multiple Computers" radio button 3. click Submit 4. fill in * Hostname - the hostnames of all the computers must have a numerical part that is sequential, use a % as a placeholder where that part would be * Start value - the first number of the numerical part of the hostname * End value - the last number of the numerical part of the hostname * Start IP Address - if using static public addresses, the IP addresses must be sequential; enter the first address here; if using DHCP, just enter something like 1.1.1.1 * End IP address - the last IP address of the sequence; if using DHCP, you'll need to enter something that would work out to the last address relative to Start IP Address (i.e. if adding 3 computers, use 1.1.1.1 for start and 1.1.1.3 for end) * State - vmhostinuse - double check this because you will not be able to change it later * Owner - owner of the computer * RAM * Processor Speed * Network Speed * Type - blade * Provisioning Engine - xCAT 2.x * check allComputers 5. click Confirm Computers 6. click Submit 7. select a VM Host Profile to use with these hosts - you can edit the VM Host Profile later if needed 8. click Add Computers Adding Multiple Virtual Machines 1. click "Manage Computers" 2. select the "Add Multiple Computers" radio button 3. click Submit 4. fill in * Hostname - the hostnames of all the computers must have a numerical part that is sequential, use a % as a placeholder where that part would be * Start value - the first number of the numerical part of the hostname * End value - the last number of the numerical part of the hostname * Start IP Address - if using static public addresses, the IP addresses must be sequential; enter the first address here; if using DHCP, just enter something like 1.1.1.1 * End IP address - the last IP address of the sequence; if using DHCP, you'll need to enter something that would work out to the last address relative to Start IP Address (i.e. if adding 3 computers, use 1.1.1.1 for start and 1.1.1.3 for end) * Start private IP Address - (optional) similar to Start IP Address, but for the private side * End private IP Address - (optional) similar to the End IP Address but for the private side * Start MAC Address - (optional) if mac addresses are sequential, with the first one being the private MAC address for the first computer, the second one being the public MAC address for the first computer, the third one being the private MAC address of the second computer, etc, you can enter the first one here and then have the option of generating data to add to your dhcpd.conf file later in the process. * State - maintenance * Owner - owner of the computer * RAM * Processor Speed * Network Speed * Type - virtualmachine * Provisioning Engine - VMware * check "All VM Computers" and newvmimages 5. click Confirm Computers 6. click Submit 7. If you filled in the private address fields and the Start MAC Address, you can now enter the private IP address of the management node that will be handling these virtual machines to generate information to add to your dhcpd.conf file. III. Management Node (backend) Tested on CentOS5, Red Hat Advanced Server 4,5, RedHat Fedora Core Operating systems. Prerequisites: MySQL 5 client Nmap - security scanner OpenSSH client - All distros usually have this installed by default Perl 5.8.0 or later Perl modules SEE STEP 2 below in Installation: * Class-Data-Inheritable * Compress-Raw-Zlib * Crypt-SSLeay * DBI * Devel-StackTrace * Digest-SHA1 * Exception-Class * HTML-Parser * IO-Compress * libwww-perl * MailTools * Object-InsideOut * RPC-XML * XML-Parser * YAML Installation: 1. Move the managementnode directory to /usr/local/ and rename it to vcl. ex. mv managementnode /usr/local/vcl 2. Install Required Perl modules. A script is provided in the VCL repository called install_perl_libs.pl which will attempt to download and install the required Perl libraries. Run the script: perl /usr/local/vcl/bin/install_perl_libs.pl The script will hang or terminate if it encounters a problem. If this occurs, manually run the last command the script attempted. The command should be listed in the output. You will need to troubleshoot the problem. The most likely cause of the problem is a missing Linux package. Make sure the required packages are installed. 3. Configure vcld.conf 1. Create the /etc/vcl directory: mkdir /etc/vcl 2. Copy the generic vcld.conf file to /etc/vcl: cp /usr/local/vcl/etc/vcl/vcld.conf /etc/vcl 3. Edit the /etc/vcl/vcld.conf file: vi /etc/vcl/vcld.conf The following lines must be configured in order to start the VCL daemon (vcld) and allow it to check in to the database: * FQDN - the fully qualified name of the management node, this should match the name that was configured for the management node in the database * server - the IP address or FQDN of the database server * LockerWrtUser - database user account with write privileges * wrtPass - database user password 4. Save the vcld.conf file 4. Install the VCL Daemon (vcld) Service 1. Copy the vcld service script to /etc/init.d and name it vcld: cp /usr/local/vcl/bin/S99vcld.linux /etc/init.d/vcld 2. Add the vcld service using chkconfig: /sbin/chkconfig --add vcld 3. Configure the vcld service to automatically run at runtime levels 3-5: /sbin/chkconfig --level 345 vcld on 5. Start and Check the vcld Service 1. Start the vcld service: /sbin/service vcld start You should see output similar to the following: Starting vcld daemon: BIN PATH: /usr/local/vcl/bin pre-execution: config file being used: /etc/vcl/vcld.conf FQDN is not listed pre-execution: process name is set to: vcld pre-execution: verbose mode is set to: 1 pre-execution: testing mode is set to: 0 pre-execution: log file being used: /var/log/vcld.log pre-execution: PID file being used: /var/run/vcld.pid Created process 23696 renamed to vcld ... [ OK ] Note: the vcld service can also be started by running the service script directly: /etc/init.d/vcld start 2. Check the vcld service by monitoring the vcld.log file: tail -f /var/log/vcld.log You should see the following being added to the log file every few seconds if the management node is checking in with the database: 2009-06-16 16:57:15|15792|vcld:main(165)|lastcheckin time updated for management node 18: 2009-06-16 16:57:15 6. Download and Configure Windows Dependencies If you plan to capture Windows images, the following dependencies need to be downloaded to the locations specified below. 1. Windows XP and Server 2003 Deployment Tools (Sysprep) The Windows XP and Server 2003 Deployment Tools are available from Microsoft and are required in order for the capture of Windows XP and Server 2003 VCL images to work. The Sysprep.exe utility is included in the Deployment Tools. You do not need to download Sysprep for Windows 7 or Windows Server 2008 because it is included in the operating system. (note: if the following links do not work, search microsoft.com for Sysprep download) Download the Windows XP Service Pack 3 Deployment Tools: http://www.microsoft.com/downloads/details.aspx?FamilyID=673a1019-8e3e-4be0-ac31-70dd21b5afa7&displaylang=en Download the System Preparation tool for Windows Server 2003 Service Pack 2 Deployment: http://www.microsoft.com/downloads/details.aspx?familyid=93F20BB1-97AA-4356-8B43-9584B7E72556&displaylang=en The packages you download are in Microsoft's .cab format and need to be extracted. It is easiest to extract the files on a Windows computer. Windows Explorer is able to open the .cab file and then the files contained within can be copied elsewhere. Copy the extracted Windows XP Sysprep files to the following directory on the management node after they have been extracted: /usr/local/vcl/tools/Windows_XP/Utilities/Sysprep Copy the extracted Windows Server 2003 Sysprep files to the following directory on the management node after they have been extracted: /usr/local/vcl/tools/Windows_Server_2003/Utilities/Sysprep Your Windows product keys and/or KMS server addresses need to be entered into the VCL database in order to capture a Windows image using Sysprep. Enter the information into the database by running the the following command: /usr/local/vcl/bin/vcld -setup Select "Windows OS Module" and follow the prompts. 2. Download Drivers Drivers which aren't included with Windows must be downloaded and saved to the management node. The drivers required will vary greatly depending on the hardware. The only way to know what additional drivers you need is to install Windows on a computer and check for missing drivers. The drivers must be copied to the appropriate directory on the management node. The VCL image capture process copies the driver directories to the computer before an image is captured. Drivers from multiple directories will be copied based on the version of Windows being captured. There are driver directories under tools for each version of Windows (Windows XP, Windows Vista, ...) and for each version group of Windows (5, 6, ...). This allows drivers which are common to multiple versions of Windows to be shared in the management node tools directory structure. For example, if a chipset driver works for all versions of Windows, it can be saved in: tools/Windows/Drivers/Chipset If Windows XP and Windows Server 2003 both use the same network driver, it can be saved in: tools/Windows_Version_5/Drivers/Network If a storage driver only works for Windows XP, it should be saved in: tools/Windows_XP/Drivers/Storage During the image capture process, each Windows version directory is copied to the computer under C:\Cygwin\home\root\VCL. The order in which the Windows version directories are copied goes from most general to most specific. In the example above, the order would be: 1. (Windows) tools/Windows/Drivers/Chipset 2. (Windows Version 5) tools/Windows_Version_5/Drivers/Network 3. (Windows XP) tools/Windows_XP/Drivers/Storage The resulting directory structure on the Windows computer will be: C:\Cygwin\home\root\VCL\Drivers \Chipset - driver works for all versions of windows \Network - driver works for Windows XP and Server 2003 \Storage - driver only works for Windows XP The following list shows which driver files should be saved in the driver directories: - tools/Windows/Drivers - drivers common to all versions of Windows - tools/Windows_Version_5/Drivers - drivers used by Windows XP and Server 2003 - tools/Windows_Version_XP/Drivers - drivers only used by Windows XP - tools/Windows_Version_Server_2003/Drivers - drivers only used by Windows Server 2003 - tools/Windows_Version_6/Drivers - drivers used by Windows Vista and Server 2008 - tools/Windows_Vista/Drivers - drivers only used by Windows Vista - tools/Windows_Server_2008/Drivers - drivers only used by Windows Server 2008 The directory structure under each Drivers directory does not matter, though it is helpful to organize each directory by driver class and to organize each directory using the same theme. For example: tools/Windows_Version_XP/Drivers /Chipset /Network /Storage /Video 7. Provisioning Engines and Hypervisors VCL supports the following, please see the related site for installation and setup. xCAT - Extreme Cluster Administration Tool versions 1.3 and 2.1. http://xcat.sourceforge.net/ VMware - Free server 1.x, ESX standard Server, ESXi http://www.vmware.com VMware toolkit - http://www.vmware.com/support/developer/viperltoolkit/ IV. Adding extra local accounts Additional local accounts can now be added using the backend code. After you have finished the backend install, run vcld -setup select vcl base module option, and follow the prompts. V. Adding LDAP authentication Prerequisites: make sure you have php-ldap installed 1) fill in the necessary information in vcl/.ht-inc/conf.php 2) add an entry to the affiliation table and use the id for that entry as 'affiliationid' for your new entry in vcl/.ht-inc/conf.php 3) uncomment the 'require_once(".ht-inc/authmethods/ldapauth.php");' line in in vcl/.ht-inc/conf.php