= Welcome to XDOK, a Super Simple Site Generation Tool = This is a tool for the generation step of documentation management at the ASF. It should be usable in many other contexts. == WARNING == See WARNING.txt. == If you know what you're doing... == svn checkout \ https://svn.apache.org/repos/asf/infrastructure/site-tools/trunk/xdok cd xdok vi xdocs/*.html ./xdok svn commit -m "Blah" == Get started == You need to have subversion and python 2.2 or later installed. If you don't know how to make that happen, don't bother reading on. This tool is not for you. You also need to have 'libxml2' installed, in particular you need the xmllint tool. If you have a recent version of GNOME, it should be installed already. Many linux and UNIX distributions provided it by default. If you don't have it yet, get it from http://www.xmlsoft.org/downloads.html. Step 1: checkout or update. Run cd / svn checkout \ https://svn.apache.org/repos/asf/infrastructure/site-tools/trunk/xdok cd xdok If you already have xdok checked out, run cd / cd xdok svn update instead. Note you can replace the "/" path with anything you like. Step 1.1: if you are using xdok for the first time, run python xdok -h to view basic command line help, and to make sure the tool is properly set up. Step 2: change stuff. Edit the contents of the xdocs/ directory. Use simple and valid XHTML only. See the xdok10.dtd file for formal rules. Step 3: build. cd into the directory containing this file, then type python xdok On platforms other than windows, usually you can also type ./xdok Step 4: view changes. Open target/index.html (or the path(s) corresponding to the file(s) you changed) with your web browser and verify things look ok. Step 5: commit. Type (Of course with a different commit message) svn commit -m "Meaningful commit message goes here." Step 6: view changes online. Wait a few hours, then open http://www.apache.org/ in your web browser (or the path(s) corresponding to the file(s) you changed) and verify things look ok. == Validating only == (This does not work on windows unless you are using the cygwin environment). (This does not work with the default 'find' command provided under Solaris). While doc generation is generally really fast, if you want to just check that your changes are in line with the DTD, you can run ./validate to run your changes through xmllint without invoking the overhead of a full build. You should always do a full build before a commit. == Using as an installed tool == (These instructions are not for windows unless you're using cygwin, though a roughly equivalent approach should be available.) In addition to the above commands, run export PATH=$PATH:/xdok replacing "/xdok" with wherever you checked out xdok. Now, run xdok using the -b, -s and -t switches, pointing them to wherever your docs are / should go: xdok -b /xdok \ -s ~/svn/infrastructure/site/trunk/xdocs \ -t ~/svn/infrastructure/site/trunk/docs == Advanced features == * use a custom template cp /xdok/templates/site.kid.html \ ~/svn/infrastructure/site/trunk/xdok.template.kid.html vi ~/svn/infrastructure/site/trunk/xdok.template.kid.html xdok -b /xdok \ -s ~/svn/infrastructure/site/trunk/xdocs \ -t ~/svn/infrastructure/site/trunk/docs \ --template=~/svn/infrastructure/site/trunk/xdok.template.kid.html See http://kid.lesscode.org/ for the template language reference. You can do pretty much anything you can do in python, velocity and/or XSLT within the template. Available variables are: * doc: ElementTree Element for the source xml file currently being processed * ET: convenience access to the elementtree.ElementTree module (or the one from cElementTree if its available) * basedir: the "basedir" for the xdok install (either the current working directory or the basedir specified using the -b commandline option) * spath: the filename of the source xml file currently being processed * tpath: the target filename of the source xml file currently being processed * rpath: the "relative" path (relative to the sourcedir) of the source xml file currently being processed * customize the files to process. Use the "--include" and "--exclude" options. This command turns *.xml and *.doc into *.html, skipping processing of any directory named 'private' completely: xdok -b /xdok \ -s ~/svn/infrastructure/site/trunk/xdocs \ -t ~/svn/infrastructure/site/trunk/docs \ --template=~/svn/infrastructure/site/trunk/xdok.template.kid.html \ --include='*\.xml$' --include='*\.doc$' \ --exclude='\/private$' * don't customize what output files are named. This command turns *.xml into *.xml: xdok -b /xdok \ -s ~/svn/infrastructure/site/trunk/xdocs \ -t ~/svn/infrastructure/site/trunk/docs \ --template=~/svn/infrastructure/site/trunk/xdok.template-xml.kid.html \ --include='*\.xml$' \ --no-renames * disable validation. This may increase transformation speed on large trees: xdok -b /xdok \ -s ~/svn/infrastructure/site/trunk/xdocs \ -t ~/svn/infrastructure/site/trunk/docs \ --no-validate * allow validation against custom DTDs: xdok -b /xdok \ -s ~/svn/infrastructure/site/trunk/xdocs \ -t ~/svn/infrastructure/site/trunk/docs \ --catalog=~/svn/infrastructure/site/trunk/dtds/catalog.xml See http://xmlsoft.org/catalog.html for more on DTD catalogs. * integrate with other tools like Anakia. Here's an example where the output of xdok is used as input for Anakia: kdocs -- files for processing with xdok xdocs -- output directory for xdok input directory for anakia docs -- output directory for anakia cd ~/svn/infrastructure/site/trunk xdok -b /xdok \ -s ~/svn/infrastructure/site/trunk/kdocs \ -t ~/svn/infrastructure/site/trunk/xdocs \ --template=~/svn/infrastructure/site/trunk/xdok.template-anakia-xml.kid.html \ --include='*\.xml$' \ --no-renames ./build.sh The reverse setup obviously looks similar: xdocs -- files for processing with anakia docs -- output directory for anakia input directory for xdok target -- output directory for xdok cd ~/svn/infrastructure/site/trunk ./build.sh xdok -b /xdok \ -s ~/svn/infrastructure/site/trunk/docs \ -t ~/svn/infrastructure/site/trunk/target \ --template=~/svn/infrastructure/site/trunk/xdok.template-anakia-html.kid.html \ --no-renames Basically you can build simple "xml tree pipelines" this way. The tricky bit is restricted to making sure the transformation steps align (eg getting the Kid template right as well as the velocity template or XSLT spec for the other tools). == Tested platforms == * Mac OS X 10.4 with standard python2.3 * Mac OS X 10.4 with python 2.2 from DarwinPorts * Mac OS X 10.4 with python 2.3 from DarwinPorts * Mac OS X 10.4 with python 2.4.2 from DarwinPorts * Windows XP SP2 with python 2.4.2 * Windows XP SP2 with cygwin and its python 2.4.1 * Ubuntu Linux 5.10 with python 2.2 * Ubuntu Linux 5.10 with python 2.3 * Ubuntu Linux 5.10 with python 2.4.2 * FreeBSD 5.4-STABLE with python 2.4.2 * Solaris SunOS 5.10 with python 2.3.3 == Help with development == Just read this file once or twice, read the inline comments in the xdok script, and start hacking. Some guidelines... * lets do agile test-first development. Write the test before you write the code. * write tests by writing shell scripts which go into test/. Feel free to call other stuff from those shell scripts. Script names should end with a "sh" extension, and start with a XX integer identifier. * run tests as follows cd xdok bash selftest * KISS. KISS. Thank you very much. This is not a web framework. It is a very simple and straightforward tool. * Please maintain multi-platform compatibility. Write portable python. Make sure to at least test with 2.2 and 2.4 on your own machine. * Try to avoid dependencies. Just copy them into the main tree or duplicate effort and write code yourself. * patches should go to site-dev at apache dot org. Subscription to that list is open to all ASF committers. * write karma for the SVN repository is automatic for all ASF members and also available to anyone in the apsite group. If you feel you should be added to the apsite group, ask on the infrastructure mailing list. * if you're not an ASF committer but would like to use this code, feel free. We're not very likely to give you SVN access just to work on the xdok scripts, so if you want to go wild with them its probably easiest to just fork the codebase. * there is no "owner" for this code. Modify it to your hearts' content. Please try not to break anything though, people use this stuff. * if you contribute, add your name to NOTICE.txt if you want. * see TODO.txt for ideas of what could be done. * please write meaningful commit messages. See http://subversion.tigris.org/hacking.html#log-messages for a great guide to writing meaningful commit messages.