Installing VCL 2.1 Contents: I. Database (MySQL) II. Web (frontend) III. Management Node (backend) IV. Adding extra local accounts V. Adding LDAP authentication I. Database (MySQL) Prerequisites: MySQL 5 installed and running Installation: 1. create a database in mysql named for use with VCL CREATE DATABASE vcl; 2. create a user with SELECT, INSERT, UPDATE, and DELETE privileges on the database you just created ** NOTE: Replace vcluserpassword with your own password! GRANT SELECT,INSERT,UPDATE,DELETE ON vcl.* TO 'vcluser'@'localhost' IDENTIFIED BY 'vcluserpassword'; 3. import vcl.sql file into database mysql vcl < vcl.sql II. Web (frontend) Prerequisites: Your web server should meet the following criteria before installing the VCL Frontend Code: * Apache HTTP Server v1.3 or v2.x with SSL enabled - while VCL may run under another webserver capable of running PHP code, it has only been tested to work with Apache HTTP Server * PHP 5 * php modules that should be installed (depending on your Linux distro, some of these may be compiled in to php instead of being a separate module): - php-gd - php-json - php-mcrypt - php-mysql - php-openssl - php-sysvsem - php-xml - php-xmlrpc * useful to have the server set up to be able to send debugging emails * php-mcrypt requires libmcrypt and mcrypt libraries as dependencies. These may need to be installed first. Installation: 1. move the web directory somewhere that your web server can access it (you'll probably also want to rename it to 'vcl') ex: mv web /var/www/html/vcl 2. modify vcl/.ht-inc/secrets.php * set $vclhost, $vcldb, $vclusername, and $vclpassword to match your database setup * create random passwords for $mcryptkey, $mcryptiv, and $pemkey - $mcryptiv must be 8 hex characters 3. run the genkeys.sh script from within vcl/.ht-inc and give it $pemkey from secrets.php as the passphrase (3 times, copy/paste is a good idea here) 4. modify vcl/.ht-inc/conf.php to match your site - COOKIEDOMAIN needs to be the domain name your web server is using, or left blank if you are accessing it by IP only 5. *NOTICE* JpGraph 2.x is no longer available. JpGraph 3.x is released under a dual license. QPL 1.0 (Qt Free Licensee). Free for non-commercial, open-source or educational use (JpGraph Professional License for commercial use). If you are planning to use this for commercial use and don't want to pay for JpGraph, you can safely skip this step with the only side effect of not being able to display a few graphs on the statistics page. Download JpGraph from http://www.aditus.nu/jpgraph/jpdownload.php * For PHP5, download the 3.x series, extract it, and copy the src directory from it to vcl/.ht-inc/jpgraph 6. download version 0.4.0 of Dojo Toolkit: http://download.dojotoolkit.org/release-0.4.0/dojo-0.4.0-ajax.tar.gz * extract it under the vcl directory and rename "dojo-0.4.0-ajax" to "dojoAjax" 7. download version 1.1.0 of Dojo Toolkit: http://download.dojotoolkit.org/release-1.1.0/dojo-release-1.1.0.tar.gz * extract it under the vcl directory and rename "dojo-release-1.1.0" to "dojo" 8. go into the themes directory (vcl/themes) and run "./copydojocss.sh default" to copy parts of dojo's css into the "default" theme 9. if you want to be able to edit any of the documentation that comes bundled with the vcl web code, download fckeditor from http://www.fckeditor.net/download (most people can skip this step) * extract it under the vcl directory 10. open a browser and open the testsetup.php page * i.e. if you set up your site to be https://my.server.org/vcl/ open https://my.server.org/vcl/testsetup.php 11. debug any issues reported by testsetup.php 12. now, open the index.php page in your browser 13. select Local Account and use 'admin' as the user and 'adminVc1passw0rd' as the password 14. click the "Management Nodes" link 15. enter the hostname and IP of your management node 16. click Add 17. fill in "Install Path" - this is parent directory under which image files will be stored 18. enter "/etc/vcl/vcl.key" for "End Node SSH Identity Key Files" 19. click "Confirm Management Node" 20. click Submit 21. click the "Management Nodes" link 22. select "Edit Management Node Grouping" 23. click Submit 24. select the checkbox for your management node 25. click Submit 26. click "Manage Computers" 27. select the "Add Single Computer" radio button 28. click the Submit 29. fill in Hostname, IP Address, owner (admin@Local), RAM, Proc Speed, Network Speed, select "blade" for Type, select "xCAT 1.x Provisioning" for "Provisioning Engine", and click the checkbox under "allcomputers", and "newimages" Note: if using using VMware, select "virtualmachine" for Type and "VMWare Server Provisioning" for "Provisioning Engine" 30. click Confirm Computer 31. click Submit (don't worry about the fact that the computer you just added isn't listed after clicking Submit) 32. after you've configured your image library and your management node has started checking in, you should be able to make a reservation III. Management Node (backend) Tested on CentOS5, Red Hat Advanced Server 4,5, RedHat Fedora Core Operating systems. Prerequisites: MySQL 5 client Nmap - security scanner OpenSSH client - All distros usually have this installed by default Perl 5.8.0 or later Perl modules SEE STEP 2 below in Installation (some of these may be built in for your distro): - MailTools 2.04 - Class-Data-Inheritable 0.08 - Devel-StackTrace 1.20 - Exception-Class 1.26 - Object-InsideOut 3.52 - Module-Build 0.30 - Net-XMPP 1.02 - GSSAPI 0.26 - Digest-SHA1 2.12 - Digest-HMAC 1.01 - GBARR/Authen-SASL 2.12 - XML-Stream 1.22 - Net-Jabber 2.0 - YAML 0.68 - RPC-XML 0.64 - XML-Parser 2.36 - Crypt-SSLeay 0.57 - Compress-Raw-Zlib 2.020 - IO-Compress 2.020 - DBI 1.609 - libwww-perl 5.827 - HTTP-Headers Installation: 1. Move the managementnode directory to /usr/local/ and rename it to vcl. ex. mv managementnode /usr/local/vcl 2. Install Required Perl modules. A script is provided in the VCL repository called install_perl_libs.pl which will attempt to download and install the required Perl libraries. Run the script: perl /usr/local/vcl/bin/install_perl_libs.pl A large amount of output will be displayed on the console the first time the script is run. It will pause if any of the module installations ask for configuration information. Accept all of the defaults by pressing enter when this happens. Run the script a 2nd time to check if all of the modules the script is configured to install were successfully installed. Output similar to the following should be displayed for each module: ============================================================================== URL: http://search.cpan.org/CPAN/authors/id/T/TI/TIMB/DBI-1.609.tar.gz Module filename: DBI-1.609.tar.gz Module name: DBI-1.609 Module package: DBI Checking if DBI is installed Module is already installed: DBI ============================================================================== Additional output will be displayed if a module has not been successfully installed. You will need to troubleshoot if any modules were not installed successfully. 3. Configure vcld.conf 1. Create the /etc/vcl directory: mkdir /etc/vcl 2. Copy the generic vcld.conf file to /etc/vcl: cp /usr/local/vcl/etc/vcl/vcld.conf /etc/vcl 3. Edit the /etc/vcl/vcld.conf file: vi /etc/vcl/vcld.conf The following lines must be configured in order to start the VCL daemon (vcld) and allow it to check in to the database: * FQDN - the fully qualified name of the management node, this should match the name that was configured for the management node in the database * server - the IP address or FQDN of the database server * LockerWrtUser - database user account with write privileges * wrtPass - database user password 4. Save the vcld.conf file 4. Install the VCL Daemon (vcld) Service 1. Copy the vcld service script to /etc/init.d and name it vcld: cp /usr/local/vcl/bin/S99vcld.linux /etc/init.d/vcld 2. Add the vcld service using chkconfig: /sbin/chkconfig --add vcld 3. Configure the vcld service to automatically run at runtime levels 3-5: /sbin/chkconfig --level 345 vcld on 5. Start and Check the vcld Service 1. Start the vcld service: /sbin/service vcld start You should see output similar to the following: Starting vcld daemon: BIN PATH: /usr/local/vcl/bin pre-execution: config file being used: /etc/vcl/vcld.conf FQDN is not listed pre-execution: process name is set to: vcld pre-execution: verbose mode is set to: 1 pre-execution: testing mode is set to: 0 pre-execution: log file being used: /var/log/vcld.log pre-execution: PID file being used: /var/run/vcld.pid Created process 23696 renamed to vcld ... [ OK ] Note: the vcld service can also be started by running the service script directly: /etc/init.d/vcld start 2. Check the vcld service by monitoring the vcld.log file: tail -f /var/log/vcld.log You should see the following being added to the log file every few seconds if the management node is checking in with the database: 2009-06-16 16:57:15|15792|vcld:main(165)|lastcheckin time updated for management node 18: 2009-06-16 16:57:15 6. Download and Configure Windows Dependencies If you plan to capture Windows images, the following dependencies need to be downloaded to the locations specified below and the sysprep.inf files need to be configured. 1. Windows XP and Server 2003 Deployment Tools (Sysprep) The Windows XP and Server 2003 Deployment Tools are available from Microsoft and are required in order for the capture of Windows XP and Server 2003 VCL images to work. The Sysprep.exe utility is included in the Deployment Tools. You do not need to download Sysprep for Windows Vista or Server 2008 because it is included in the operating system. (note: if the following links do not work, search microsoft.com for Sysprep download) Download the Windows XP Service Pack 3 Deployment Tools: http://www.microsoft.com/downloads/details.aspx?FamilyID=673a1019-8e3e-4be0-ac31-70dd21b5afa7&displaylang=en Download the System Preparation tool for Windows Server 2003 Service Pack 2 Deployment: http://www.microsoft.com/downloads/details.aspx?familyid=93F20BB1-97AA-4356-8B43-9584B7E72556&displaylang=en The packages you download are in Microsoft's .cab format and need to be extracted. It is easiest to extract the files on a Windows computer. Windows Explorer is able to open the .cab file and then the files contained within can be copied elsewhere. (Note: The Sysprep directories mentioned below should already exist on the management node and should contain a sysprep.inf file. Copy the downloaded files into the Sysprep directories leaving the sysprep.inf file in place.) Copy the extracted Windows XP Sysprep files to the following directory on the management node after they have been extracted: /usr/local/vcl/tools/Windows_XP/Utilities/Sysprep Copy the extracted Windows Server 2003 Sysprep files to the following directory on the management node after they have been extracted: /usr/local/vcl/tools/Windows_Server_2003/Utilities/Sysprep Your Windows product keys need to be entered into the sysprep.inf files: Find the ProductKey line in the following file and replace WIN_XP_PRO_KEY with your product key: /usr/local/vcl/tools/Windows_XP/Utilities/Sysprep/sysprep.inf Find the ProductKey line in the following file and replace WIN_2003_ENT_KEY with your product key: /usr/local/vcl/tools/Windows_Server_2003/Utilities/Sysprep/sysprep.inf 2. NewSID - Windows SID Changing Utility NewSID.exe is used to change the SID of a Windows computer if Sysprep is not used. VCL currently uses NewSID.exe during the load process for Windows Vista images. Download the NewSID.exe utility: http://technet.microsoft.com/en-us/sysinternals/bb897418.aspx Save the NewSID.exe utility in the following location on the management node: /usr/local/vcl/tools/Windows/Utilities/NewSID/newsid.exe 3. SPDrvScn.exe - Windows Driver Scanning Utility SPDrvScn.exe is used before an image is captured to enter the paths of drivers to the Windows registry so that they are loaded when Sysprep attempts to install devices. Download the SPDrvScn.exe utility from the link on the following webpage: http://vernalex.com/tools/spdrvscn/ Save SPDrvScn.exe in the following location on the management node: /usr/local/vcl/tools/Windows/Utilities/SPDrvScn/spdrvscn.exe 4. Download Drivers Drivers which aren't included with Windows must be downloaded and saved to the management node. The drivers required will vary greatly depending on the hardware. The only way to know what additional drivers you need is to install Windows on a computer and check for missing drivers. The drivers must be copied to the appropriate directory on the management node. The VCL image capture process copies the driver directories to the computer before an image is captured. Drivers from multiple directories will be copied based on the version of Windows being captured. There are driver directories under tools for each version of Windows (Windows XP, Windows Vista, ...) and for each version group of Windows (5, 6, ...). This allows drivers which are common to multiple versions of Windows to be shared in the management node tools directory structure. For example, if a chipset driver works for all versions of Windows, it can be saved in: tools/Windows/Drivers/Chipset If Windows XP and Windows Server 2003 both use the same network driver, it can be saved in: tools/Windows_Version_5/Drivers/Network If a storage driver only works for Windows XP, it should be saved in: tools/Windows_XP/Drivers/Storage During the image capture process, each Windows version directory is copied to the computer under C:\Cygwin\home\root\VCL. The order in which the Windows version directories are copied goes from most general to most specific. In the example above, the order would be: 1. (Windows) tools/Windows/Drivers/Chipset 2. (Windows Version 5) tools/Windows_Version_5/Drivers/Network 3. (Windows XP) tools/Windows_XP/Drivers/Storage The resulting directory structure on the Windows computer will be: C:\Cygwin\home\root\VCL\Drivers \Chipset - driver works for all versions of windows \Network - driver works for Windows XP and Server 2003 \Storage - driver only works for Windows XP The following list shows which driver files should be saved in the driver directories: - tools/Windows/Drivers - drivers common to all versions of Windows - tools/Windows_Version_5/Drivers - drivers used by Windows XP and Server 2003 - tools/Windows_Version_XP/Drivers - drivers only used by Windows XP - tools/Windows_Version_Server_2003/Drivers - drivers only used by Windows Server 2003 - tools/Windows_Version_6/Drivers - drivers used by Windows Vista and Server 2008 - tools/Windows_Vista/Drivers - drivers only used by Windows Vista - tools/Windows_Server_2008/Drivers - drivers only used by Windows Server 2008 The directory structure under each Drivers directory does not matter, though it is helpful to organize each directory by driver class and to organize each directory using the same theme. For example: tools/Windows_Version_XP/Drivers /Chipset /Network /Storage /Video 3rd party mass storage hardware IDs and driver .inf file paths must be added to the SysprepMassStorage section in sysprep.inf for Windows XP and Windows Server 2003 in order for the saved image to boot properly on different hardware. - Identify the mass storage drivers required for your hardware which aren't native to Windows - Download drivers for your hardware - Each driver will have 1 or more .inf files. Examine the .inf files. Find all lines in this format containing a PnP device ID: %DevDescD1% = SYMMPI_Inst, PCI\VEN_1000&DEV_0054&SUBSYS_1F041028 The PnP device ID in the example above is: PCI\VEN_1000&DEV_0054&SUBSYS_1F041028 - Each PnP device ID must be added to the sysprep.inf file under the [SysprepMassStorage] section using the following format: ID = "C:\Sysprep\Drivers\\<.inf file path>" Example: LSI SAS drivers commonly need to be downloaded and the hardware IDs need to be added to the sysprep.inf files in order for computers with LSI SAS controllers to boot. - Download the LSI SAS driver from ibm.com: ibm_dd_mptsas_1.30.02.00_windows_32-64.exe - Extract the ZIP file (it's a self-extracting zip; you can unzip it with whatever unzip tool you prefer) - Copy the files from the 32 bit XP directory (image/xp-32) to the appropriate directory on the management node: tools/Windows/Drivers/Storage/LSI-SAS - Locate the .inf file included with the driver is: tools/Windows/Drivers/Storage/LSI-SAS/symmpi.inf - Locate the PnP ID lines in the .inf file: [LSI] %DevDesc2% = SYMMPI_Inst, PCI\VEN_1000&DEV_0622 %DevDesc3% = SYMMPI_Inst, PCI\VEN_1000&DEV_0624 %DevDesc4% = SYMMPI_Inst, PCI\VEN_1000&DEV_0626 %DevDesc5% = SYMMPI_Inst, PCI\VEN_1000&DEV_0628 %DevDesc6% = SYMMPI_Inst, PCI\VEN_1000&DEV_0030 %DevDesc7% = SYMMPI_Inst, PCI\VEN_1000&DEV_0032 %DevDesc8% = SYMMPI_Inst, PCI\VEN_1000&DEV_0050 %DevDesc9% = SYMMPI_Inst, PCI\VEN_1000&DEV_0054 %DevDesc10% = SYMMPI_Inst, PCI\VEN_1000&DEV_0058 %DevDesc11% = SYMMPI_Inst, PCI\VEN_1000&DEV_0056 %DevDesc12% = SYMMPI_Inst, PCI\VEN_1000&DEV_0640 %DevDesc13% = SYMMPI_Inst, PCI\VEN_1000&DEV_0646 %DevDesc14% = SYMMPI_Inst, PCI\VEN_1000&DEV_0062 [DELL] %DevDescD1% = SYMMPI_Inst, PCI\VEN_1000&DEV_0054&SUBSYS_1F041028 %DevDescD2% = SYMMPI_Inst, PCI\VEN_1000&DEV_0054&SUBSYS_1F051028 %DevDescD3% = SYMMPI_Inst, PCI\VEN_1000&DEV_0054&SUBSYS_1F061028 %DevDescD4% = SYMMPI_Inst, PCI\VEN_1000&DEV_0054&SUBSYS_1F071028 %DevDescD5% = SYMMPI_Inst, PCI\VEN_1000&DEV_0054&SUBSYS_1F081028 %DevDescD6% = SYMMPI_Inst, PCI\VEN_1000&DEV_0054&SUBSYS_1F091028 %DevDescD7% = SYMMPI_Inst, PCI\VEN_1000&DEV_0058&SUBSYS_1F0E1028 %DevDescD8% = SYMMPI_Inst, PCI\VEN_1000&DEV_0058&SUBSYS_1F0F1028 %DevDescD9% = SYMMPI_Inst, PCI\VEN_1000&DEV_0058&SUBSYS_1F101028 - Based on the contents of the .inf file, added the following to the Windows XP and Windows Server 2003 sysprep.inf files under [SysprepMassStorage]: PCI\VEN_1000&DEV_0622 = "C:\Sysprep\Drivers\Storage\LSI-SAS\symmpi.inf" PCI\VEN_1000&DEV_0624 = "C:\Sysprep\Drivers\Storage\LSI-SAS\symmpi.inf" PCI\VEN_1000&DEV_0626 = "C:\Sysprep\Drivers\Storage\LSI-SAS\symmpi.inf" PCI\VEN_1000&DEV_0628 = "C:\Sysprep\Drivers\Storage\LSI-SAS\symmpi.inf" PCI\VEN_1000&DEV_0030 = "C:\Sysprep\Drivers\Storage\LSI-SAS\symmpi.inf" PCI\VEN_1000&DEV_0032 = "C:\Sysprep\Drivers\Storage\LSI-SAS\symmpi.inf" PCI\VEN_1000&DEV_0050 = "C:\Sysprep\Drivers\Storage\LSI-SAS\symmpi.inf" PCI\VEN_1000&DEV_0054 = "C:\Sysprep\Drivers\Storage\LSI-SAS\symmpi.inf" PCI\VEN_1000&DEV_0058 = "C:\Sysprep\Drivers\Storage\LSI-SAS\symmpi.inf" PCI\VEN_1000&DEV_0056 = "C:\Sysprep\Drivers\Storage\LSI-SAS\symmpi.inf" PCI\VEN_1000&DEV_0640 = "C:\Sysprep\Drivers\Storage\LSI-SAS\symmpi.inf" PCI\VEN_1000&DEV_0646 = "C:\Sysprep\Drivers\Storage\LSI-SAS\symmpi.inf" PCI\VEN_1000&DEV_0062 = "C:\Sysprep\Drivers\Storage\LSI-SAS\symmpi.inf" PCI\VEN_1000&DEV_0054&SUBSYS_1F041028 = "C:\Sysprep\Drivers\Storage\LSI-SAS\symmpi.inf" PCI\VEN_1000&DEV_0054&SUBSYS_1F051028 = "C:\Sysprep\Drivers\Storage\LSI-SAS\symmpi.inf" PCI\VEN_1000&DEV_0054&SUBSYS_1F061028 = "C:\Sysprep\Drivers\Storage\LSI-SAS\symmpi.inf" PCI\VEN_1000&DEV_0054&SUBSYS_1F071028 = "C:\Sysprep\Drivers\Storage\LSI-SAS\symmpi.inf" PCI\VEN_1000&DEV_0054&SUBSYS_1F081028 = "C:\Sysprep\Drivers\Storage\LSI-SAS\symmpi.inf" PCI\VEN_1000&DEV_0054&SUBSYS_1F091028 = "C:\Sysprep\Drivers\Storage\LSI-SAS\symmpi.inf" PCI\VEN_1000&DEV_0058&SUBSYS_1F0E1028 = "C:\Sysprep\Drivers\Storage\LSI-SAS\symmpi.inf" PCI\VEN_1000&DEV_0058&SUBSYS_1F0F1028 = "C:\Sysprep\Drivers\Storage\LSI-SAS\symmpi.inf" PCI\VEN_1000&DEV_0058&SUBSYS_1F101028 = "C:\Sysprep\Drivers\Storage\LSI-SAS\symmpi.inf" If you have hardware using an LSI SAS controller (IBM HS21 blades), the section above can be copied and pasted into your sysprep.inf files: /usr/local/vcl/tools/Windows_XP/Utilities/Sysprep/sysprep.inf /usr/local/vcl/tools/Windows_Server_2003/Utilities/Sysprep/sysprep.inf 7. Provisioning Engines and Hypervisors VCL supports the following, please see the related site for installation and setup. xCAT - Extreme Cluster Administration Tool versions 1.3 and 2.1. http://xcat.sourceforge.net/ VMware - Free server 1.x, ESX standard Server, ESXi http://www.vmware.com VMware toolkit - http://www.vmware.com/support/developer/viperltoolkit/ IV. Adding extra local accounts There's not currently a tool for this. You will need to add entries directly to the database. 1) add entry to user table INSERT INTO user (unityid, firstname, lastname, email, lastupdated) VALUES ('myusername', 'myfirstname', 'mylastname', 'myemailaddr', NOW()); 2) find out the id generated for that user SELECT id, unityid FROM user WHERE unityid = 'myusername'; 3) add entry to the localauth table INSERT INTO localauth (userid, salt, passhash, lastupdated) VALUES ('place1', 'place2', 'place3', NOW()) with place1 = id from step 2 place2 = an 8 char random string place3 = sha1sum( desired password with place2 stuck on the end ) this can be generated under linux like this (using 'thedog' as the password and 11111111 as place2): echo -n 'thedog11111111' | sha1sum Once a user has been added, the user can go to User Preferences to change his/her password V. Adding LDAP authentication 1) fill in the necessary information in vcl/.ht-inc/conf.php 2) add an entry to the affiliation table and use the id for that entry as 'affiliationid' for your new entry in vcl/.ht-inc/conf.php 3) uncomment the 'require_once(".ht-inc/authmethods/ldapauth.php");' line in in vcl/.ht-inc/conf.php