::Go back to Oozie Documentation Index::
JDK commands (java, javac) must be in the command path.
The Maven command (mvn) must be in the command path.
To generate the documentation, Oozie uses a patched Doxia plugin for Maven with improved twiki support.
This plugin is available from Yahoo GitHub Maven repository and it is automatically downloaded by Maven when building Oozie.
The source of the modified plugin is available in the Oozie GitHub repository, in the ydoxia branch.
To build and install it locally run the following command in the ydoxia branch:
NOTE: SSH actions are deprecated in Oozie 2.
To run SSH Testcases and for easier Hadoop start/stop configure SSH to localhost to be passphrase-less.
Create your SSH keys without a passphrase and add the public key to the authorized file:
$ ssh-keygen -t dsa $ cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys2
Test that you can ssh without password:
$ ssh localhost
The JARs for the specified Hadoop and Pig versions must be available in one of the Maven repositories defined in Oozie main 'pom.xml' file. Or they must be installed in the local Maven cache.
Using embedded Hadoop minicluster with 'simple' authentication:
$ mvn clean test
Using a Hadoop cluster with 'simple' authentication:
$ mvn clean test -Doozie.test.hadoop.minicluster=false
Using embedded Hadoop minicluster with 'simple' authentication and Derby database:
$ mvn clean test -Doozie.test.hadoop.minicluster=false -Doozie.test.db=derby
Using a Hadoop cluster with 'kerberos' authentication:
$ mvn clean test -Doozie.test.hadoop.minicluster=false -Doozie.test.hadoop.security=kerberos
NOTE: The embedded minicluster cannot be used when testing with 'kerberos' authentication.
Using a custom Oozie configuration for testcases (for example, to use a MySQL DB):
$ mvn clean test -Dmysql -Doozie.test.config.file=/home/tucu/oozie-site-mysql.xml
All these options can be set using -D .
Except for the options marked with (*) , the options can be specified in the test.properties in the root of the Oozie project. The options marked with (*) are used in Maven POMs, thus they don't take effect if specified in the test.properties file (which is loaded by the XTestCase class at class initialization time).
includeHadoopJars (*): includes Hadoop JARs and its transitive dependencies in the Oozie WAR file, default is undefined (Hadoop JARs are not included).
generateSite (*): generates Oozie documentation, default is undefined (no documentation is generated)
mysql (*): Includes MySQL JDBC driver in the distro, default is undefined, MySQL JDBC driver is not included.
oracle (*): Includes Oracle JDBC driver in the distro, default is undefined, Oracle JDBC driver is not included.
skipTests (*): skips the execution of all testcases, no value required, default is undefined
test = (*): runs a single test case, to run a test give the test class name without package and extension, no default
hadoop20 = (*) : indicates the build/test should not include classes for Hadoop 20S, default is 'false'
oozie.test.db (*): indicates the database to use for running the testcases, supported values are 'hsqldb', 'derby' and 'other'; default value is 'hsqldb'. IMPORTANT, when using 'derby' a Maven profile is activated to run all the testcases in 'always' fork mode (otherwise Derby driver goes bonker after 50 testcases or so due to repeated initializations). Use 'other' when using MySQL, Oracle or PostgreSQL databases, if using 'other' a '-Doozie.test.config.file ' must be used with the correct JDBC information must be provided (the core/src/test/resources/ directory has samples for the different databases).
oozie.test.properties (*): indicates the file to load the test properties from, by default is test.properties . Having this option allows having different test properties sets, for example: minicluster, simple & kerberos.
oozie.test.waitfor.ratio : multiplication factor for testcases using waitfor, the ratio is used to adjust the effective time out. For slow machines the ratio should be increased. The default value is =1 .
oozie.test.config.file = : indicates a custom Oozie configuration file for running the testcases. The specified file must be an absolute path. For example, it can be useful to specify different database than HSQL for running the testcases.
oozie.test.hadoop.minicluster = : indicates if Hadoop minicluster should be started for testcases, default value 'true'
oozie.test.job.tracker = : indicates the URI of the JobTracker when using a Hadoop cluster for testing, default value 'localhost:9001'
oozie.test.name.node = : indicates the URI of the NameNode when using a Hadoop cluster for testing, default value 'hdfs://localhost:9000'
oozie.test.hadoop.pipes = : indicates if Hadoop pipes test case should be run or not, default value 'false'
oozie.test.hadoop.security = : indicates the type of Hadoop authentication for testing, valid values are 'simple' or 'kerberos, default value 'simple'
oozie.test.kerberos.keytab.file = : indicates the location of the keytab file, default value '${user.home}/oozie.keytab'
oozie.test.kerberos.realm = : indicates the Kerberos real, default value 'LOCALHOST'
oozie.test.kerberos.oozie.principal = : indicates the Kerberos principal for oozie, default value '${user.name}/localhost'
oozie.test.kerberos.jobtracker.principal = : indicates the Kerberos principal for the JobTracker, default value 'mapred/localhost'
oozie.test.kerberos.namenode.principal = : indicates the Kerberos principal for the NameNode, default value 'hdfs/localhost'
oozie.test.user.oozie : specifies the user ID used to start Oozie server in testcases, default value is =${user.name} .
oozie.test.user.test : specifies primary user ID used as the user submitting jobs to Oozie Server in testcases, default value is =test .
oozie.test.user.test2 : specifies secondary user ID used as the user submitting jobs to Oozie Server in testcases, default value is =test2 .
oozie.test.user.test3 : specifies secondary user ID used as the user submitting jobs to Oozie Server in testcases, default value is =test3 .
oozie.test.group : specifies group ID used as group when submitting jobs to Oozie Server in testcases, default value is =testg .
NOTE: The users/group specified in oozie.test.user.test2 , oozie.test.user.test3 = and oozie.test.user.group = are used for the authorization testcases only.
oozie.test.dir : specifies the directory where the =oozietests directory will be created, default value is /tmp . The oozietests directory is used by testcases when they need a local filesystem directory.
hadoop.log.dir : specifies the directory where Hadoop minicluster will write its logs during testcases, default value is =/tmp .
test.exclude : specifies a testcase class (just the class name) to exclude for the tests run, for example =TestSubmitCommand .
test.exclude.pattern : specifies one or more patterns for testcases to exclude, for example =**/Test*Command.java .
An Oozie distribution bundles an embedded Tomcat server. The Oozie distro module downloads Tomcat TAR.GZ from Apache once (in the distro/downloads/ directory) and uses it when creating the distro.
The following Maven invocation builds an Oozie distribution:
$ mvn clean package assembly:single
The following properties should be specified when building a release:
The provided bin/mkdistro.sh script runs the above Maven invocation setting all these properties to the right values (the 'vc.*' properties are obtained from the local git repository).
Eclipse and IntelliJ can use directly Oozie Maven project files.
The only special consideration is that the following source directories from the client module must be added to the core module source path: