How to run MR on TEZ ======================= Tez provides an ApplicationMaster that can run MR or MRR jobs. There is a translation layer implemented ( may have bugs so please file JIRAs if you come across any issues ) that allows a user to run an MR job against the TEZ DAG ApplicationMaster. NOTE: Due to some quirks, all jobs will show an IOException and a failure at the end. Please look at the YARN ResourceManager UI to check the correct result of the job. Install/Deploy Instructions =========================== 1) Deploy Apache Hadoop using the 3.0.0-SNAPSHOT from trunk. 2) Copy the tez jars and their dependencies from $TEZ_SRC/tez-dist/target/tez-0.2.0-SNAPSHOT/ either into $HADOOP_PREFIX/share/hadoop/common/lib/ or modify HADOOP_CLASSPATH in hadoop-env.sh to point to the location where the TEZ jars and dependencies can be found. 3) Modify mapred-site.xml to change "mapreduce.framework.name" property from its default value of "yarn" to "yarn-tez" 4) Submit a MR job as you normally would using something like: $HADOOP_PREFIX/bin/hadoop jar hadoop-mapreduce-client-jobclient-3.0.0-SNAPSHOT-tests.jar sleep -mt 1 -rt 1 -m 1 -r 1 This will use the TEZ DAG ApplicationMaster to run the MR job. This can be verified by looking at the AM's logs from the YARN ResourceManager UI. 5) There is a basic example of using an MRR job in the tez-mapreduce-examples.jar. Refer to WordCountMRRTest.java in the source code. To run this example: $HADOOP_PREFIX/bin/hadoop jar tez-mapreduce-examples-0.2.0-SNAPSHOT.jar wordcountmrrtest