[[index][::Go back to Oozie Documentation Index::]] ---+!! Command Line Interface Utilities %TOC% ---++ Introduction Oozie provides a command line utility, =oozie=, to perform job and admin tasks. All operations are done via sub-commands of the =oozie= CLI. The =oozie= CLI interacts with Oozie via its WS API. ---++ Oozie Command Line Usage usage: the env variable 'OOZIE_URL' is used as default value for the '-oozie' option custom headers for Oozie web services can be specified using '-Dheader:NAME=VALUE' oozie help : display usage oozie version : show client version oozie job : job operations -action coordinator rerun on action ids (requires -rerun) -change change a coordinator job -config job configuration file '.xml' or '.properties' -date coordinator rerun on action dates (requires -rerun) -definition job definition -dryrun Supported in Oozie-2.0 or later versions ONLY - dryrun or test run a coordinator job (requires -config) - job is not queued -info info of a job -kill kill a job -len number of actions (default TOTAL ACTIONS, requires -info) -localtime use local time (default GMT) -log job log -nocleanup do not clean up output-events of the coordiantor rerun actions (requires -rerun) -offset job info offset of actions (default '1', requires -info) -oozie Oozie URL -refresh re-materialize the coordinator rerun actions (requires -rerun) -rerun rerun a job (workflow requires -config, coordinator requires -action or -date) -resume resume a job -run run a job (requires -config) -start start a job -submit submit a job (requires -config) -suspend suspend a job -value new endtime/concurrency/pausetime value for changing a coordinator job -verbose verbose mode oozie jobs : jobs status -filter user=;name=;group=;status=;... -jobtype job type ('Supported in Oozie-2.0 or later versions ONLY - coordinator' or 'wf' (default)) -len number of jobs (default '100') -localtime use local time (default GMT) -offset jobs offset (default '1') -oozie Oozie URL -verbose verbose mode oozie admin : admin operations -oozie Oozie URL -queuedump show Oozie server queue elements -status show the current system status -systemmode Supported in Oozie-2.0 or later versions ONLY. Change oozie system mode [NORMAL|NOWEBSERVICE|SAFEMODE] -version show Oozie server build version oozie validate : validate a workflow XML file oozie sla : sla operations (Supported in Oozie-2.0 or later) -len number of results (default '100') -offset start offset (default '0') -oozie Oozie URL oozie pig -X : submit a pig job, everything after '-X' are pass-through parameters to pig -config job configuration file '.properties' -file Pig script -oozie Oozie URL ---++ Common CLI Options ---+++ Oozie URL All =oozie= CLI sub-commands expect the -oozie OOZIE_URL option indicating the URL of the Oozie system to run the command against. If the -oozie option is not specified, the =oozie= CLI will look for the =OOZIE_URL= environment variable and uses it if set. If the option is not provided and the environment variable is not set, the =oozie= CLI will fail. ---++ Job Operations ---+++ Submitting a Workflow or Coordinator Job Example: $ oozie job -oozie http://localhost:8080/oozie -config job.properties -submit . job: 14-20090525161321-oozie-tucu The parameters for the job must be provided in a file, either a Java Properties file (.properties) or a Hadoop XML Configuration file (.xml). This file must be specified with the -config option. The workflow application path must be specified in the file with the =oozie.wf.application.path= property. The coordinator application path must be specified in the file with the =oozie.coord.application.path= property. Specified path must be an HDFS path. The job will be created, but it will not be started, it will be in =PREP= status. ---+++ Starting a Workflow Job or Coordinator Job Example: $ oozie job -oozie http://localhost:8080/oozie -start 14-20090525161321-oozie-tucu The =start= option starts a previously submitted workflow job or coordinator job that is in =PREP= status. After the command is executed the workflow job will be in =RUNNING= status and coordinator job will be in =PREMATER= status. ---+++ Running a Workflow Job or Coordinator Job Example: $ oozie job -oozie http://localhost:8080/oozie -config job.properties -run . job: 15-20090525161321-oozie-tucu The =run= option creates and starts a workflow job or coordinator job. The parameters for the job must be provided in a file, either a Java Properties file (.properties) or a Hadoop XML Configuration file (.xml). This file must be specified with the -config option. The workflow application path must be specified in the file with the =oozie.wf.application.path= property. The coordinator application path must be specified in the file with the =oozie.coord.application.path= property. The specified path must be an HDFS path. The job will be created and it will started, the job will be in =RUNNING= status. ---+++ Suspending a Workflow Job or Coordinator Job Example: $ oozie job -oozie http://localhost:8080/oozie -suspend 14-20090525161321-oozie-tucu The =suspend= option suspends a job in =RUNNING= status. After the command is executed the job will be in =SUSPENDED= status. When the coordinator job is suspended, running coordinator actions will stay in running and the workflows will be suspended. ---+++ Resuming a Workflow Job or Coordinator Job Example: $ oozie job -oozie http://localhost:8080/oozie -resume 14-20090525161321-oozie-tucu The =resume= option resumes a job in =SUSPENDED= status. After the command is executed the job will be in =RUNNING= status. When a coordinator job is resumed, it will transition to the status from which it was suspended, if it was RUNNING it cannot transition to PREP . When the coordinator job is resumed it will create all the coordinator actions that should have been created during the time it was suspended, actions will not be lost, they will delayed. ---+++ Killing a Workflow Job or Coordinator Job Example: $ oozie job -oozie http://localhost:8080/oozie -kill 14-20090525161321-oozie-tucu The =kill= option kills a workflow job in =PREP=, =SUSPENDED= or =RUNNING= status and a coordinator job in =PREP=, =PREMATER=, =SUSPENDED= or =RUNNING= status. After the command is executed the job will be in =KILLED= status. ---+++ Changing endtime/concurrency/pausetime of a Coordinator Job Example: $ oozie job -oozie http://localhost:8080/oozie -change 14-20090525161321-oozie-tucu -value endtime=2011-12-01T05:00Z\;concurrency=100\;2011-10-01T05:00Z The =change= option changes a coordinator job that is not in =KILLED= status. Valid value names are: * endtime: the end time of the coordinator job. * concurrency: the concurrency of the coordinator job. * pausetime: the pause time of the coordinator job. Repeated value names are not allowed. New end time must not be before job's start time and last action time. New concurrency value has to be a valid integer. New pause time must not be before job's start time and last action time and must be before job's end time. Also empty string "" can be used to reset pause time to none. After the command is executed the job's end time, concurrency or pause time should be changed. If an already-succeeded job changes its end time, its status will become running. ---+++ Rerunning a Workflow Job Example: $ oozie job -oozie http://localhost:8080/oozie -config job.properties -rerun 14-20090525161321-oozie-tucu The =rerun= option reruns a completed ( =SUCCCEDED=, =FAILED= or =KILLED= ) job skipping the specified nodes. The parameters for the job must be provided in a file, either a Java Properties file (.properties) or a Hadoop XML Configuration file (.xml). This file must be specified with the -config option. The workflow application path must be specified in the file with the =oozie.wf.application.path= property. The specified path must be an HDFS path. The list of nodes to skipped must be provided in the =oozie.wf.rerun.skip.nodes= property separated by commas. After the command is executed the job will be in =RUNNING= status. Refer to the [[DG_WorkflowReRun][Rerunning Workflow Jobs]] for details on rerun. ---+++ Rerunning a Coordinator Action or Multiple Actions Example: $oozie job -rerun [-nocleanup] [-refresh] [-action 1, 3-4, 7-40] (-action or -date is required to rerun.) [-date 2009-01-01T01:00Z::2009-05-31T23:59Z, 2009-11-10T01:00Z, 2009-12-31T22:00Z] (if neither -action nor -date is given, the exception will be thrown.) The =rerun= option reruns a terminated (=TIMEDOUT=, =SUCCEEDED=, =KILLED=, =FAILED=) coordiantor action when coordiator job is not in =FAILED= or =KILLED= state. After the command is executed the rerun coordiator action will be in =WAITING= status. Refer to the [[DG_CoordinatorRerun][Rerunning Coordinator Actions]] for details on rerun. ---+++ Checking the Status of a Workflow Job or Coordinator Job or Coordinator Action Example: $ oozie job -oozie http://localhost:8080/oozie -info 14-20090525161321-oozie-tucu . .---------------------------------------------------------------------------------------------------------------------------------------------------------------- Workflow Name : map-reduce-wf App Path : hdfs://localhost:9000/user/tucu/workflows/map-reduce Status : SUCCEEDED Run : 0 User : tucu Group : users Created : 2009-05-26 05:01 +0000 Started : 2009-05-26 05:01 +0000 Ended : 2009-05-26 05:01 +0000 Actions .---------------------------------------------------------------------------------------------------------------------------------------------------------------- Action Name Type Status Transition External Id External Status Error Code Start End .---------------------------------------------------------------------------------------------------------------------------------------------------------------- hadoop1 map-reduce OK end job_200904281535_0254 SUCCEEDED - 2009-05-26 05:01 +0000 2009-05-26 05:01 +0000 .---------------------------------------------------------------------------------------------------------------------------------------------------------------- The =info= option can display information about a workflow job or coordinator job or coordinator action. The =offset= and =len= option specified the offset and number of actions to display, if checking a workflow job or coordinator job. The =localtime= option displays times in local time, if not specified times are displayed in GMT. The =verbose= option gives more detailed information for all the actions, if checking for workflow job or coordinator job. ---+++ Checking the xml definition of a Workflow Job or Coordinator Job Example: $ oozie job -oozie http://localhost:8080/oozie -definition 14-20090525161321-oozie-tucu ---+++ Checking the server logs of a Workflow Job or Coordinator Job Example: $ oozie job -oozie http://localhost:8080/oozie -log 14-20090525161321-oozie-tucu ---+++ Dryrun of Coordinator Job * This feature is only supported in Oozie 2.0 or later. Example: $ oozie job -oozie http://localhost:8080/oozie job -dryrun -config job.properties ***coordJob after parsing: *** hdfs://localhost:9000/user/angeloh/coord_examples/${YEAR}/${MONTH}/${DAY} ${coord:current(0)} ***actions for instance*** ***total coord actions is 1 *** ------------------------------------------------------------------------------------------------------------------------------------ coordAction instance: 1: hdfs://localhost:9000/user/angeloh/coord_examples/2009/03/06 hdfs://localhost:9000/user/angeloh/coord_examples/${YEAR}/${MONTH}/${DAY} ------------------------------------------------------------------------------------------------------------------------------------ The =dryrun= option tests running a coordinator job with given job properties and does not create the job. The parameters for the job must be provided in a file, either a Java Properties file (.properties) or a Hadoop XML Configuration file (.xml). This file must be specified with the -config option. The coordinator application path must be specified in the file with the =oozie.coord.application.path= property. The specified path must be an HDFS path. ---++ Jobs Operations ---+++ Checking the Status of multiple Workflow Jobs Example: $ oozie jobs -oozie http://localhost:8080/oozie -localtime -len 2 -fliter status=RUNNING . Job Id Workflow Name Status Run User Group Created Started Ended .---------------------------------------------------------------------------------------------------------------------------------------------------------------- 4-20090527151008-oozie-tucu hadoopel-wf RUNNING 0 tucu other 2009-05-27 15:34 +0530 2009-05-27 15:34 +0530 - 0-20090527151008-oozie-tucu hadoopel-wf RUNNING 0 tucu other 2009-05-27 15:15 +0530 2009-05-27 15:15 +0530 - .---------------------------------------------------------------------------------------------------------------------------------------------------------------- The =jobs= sub-command will display information about multiple jobs. The =offset= and =len= option specified the offset and number of jobs to display, default values are =1= and =100= respectively. The =localtime= option displays times in local time, if not specified times are displayed in GMT. The =verbose= option gives more detailed information for each job. A filter can be specified after all options. The =filter=option syntax is: [NAME=VALUE][;NAME=VALUE]*. Valid filter names are: * name: the workflow application name from the workflow definition. * user: the user that submitted the job. * group: the group for the job. * status: the status of the job. The query will do an AND among all the filter names. The query will do an OR among all the filter values for the same name. Multiple values must be specified as different name value pairs. ---+++ Checking the Status of multiple Coordinator Jobs * This feature is only supported in Oozie 2.0 or later. Example: $ oozie jobs -oozie http://localhost:8080/oozie -jobtype coordinator . Job ID App Name Status Freq Unit Started Next Materialized .---------------------------------------------------------------------------------------------------------------------------------------------------------------- 0004165-100531045722929-oozie-wrkf-C smaggs-xaggsptechno-coordinator SUCCEEDED 1440 MINUTE 2010-05-27 00:00 2010-05-29 00:00 .---------------------------------------------------------------------------------------------------------------------------------------------------------------- 0003823-100531045722929-oozie-wrkf-C coordcal2880minutescurrent SUCCEEDED 2880 MINUTE 2010-02-01 16:30 2010-02-05 16:30 .---------------------------------------------------------------------------------------------------------------------------------------------------------------- The =jobtype= option specified the job type to display, default value is 'wf'. To see the coordinator jobs, value is 'coordinator'. ---++ Admin Operations ---+++ Checking the Status of the Oozie System Example: $ oozie admin -oozie http://localhost:8080/oozie -status . Safemode: OFF It returns the current status of the Oozie system. ---+++ Changing the Status of the Oozie System * This feature is only supported in Oozie 2.0 or later. Example: $ oozie admin -oozie http://localhost:8080/oozie -systemmode [NORMAL|NOWEBSERVICE|SAFEMODE] . Safemode: ON It returns the current status of the Oozie system. ---+++ Displaying the Build Version of the Oozie System Example: $ oozie admin -oozie http://localhost:8080/oozie -version . Oozie server build version: 2.0.2.1-0.20.1.3092118008-- It returns the Oozie server build version. ---+++ Displaying the queue dump of the Oozie System * This feature is for administrator debugging purpose Example: $ oozie admin -oozie http://localhost:8080/oozie -queuedump [Server Queue Dump]: (coord_action_start,1),(coord_action_start,1),(coord_action_start,1) (coord_action_ready,1) (action.check,2) It returns the Oozie server current queued commands. ---++ Validate Operations ---+++ Validating a Workflow XML Example: $ oozie validate myApp/workflow.xml . Error: Invalid workflow-app, org.xml.sax.SAXParseException: cvc-complex-type.2.4.a: Invalid content was found starting with element 'xend'. One of '{"uri:oozie:workflow:0.1":decision, "uri:oozie:workflow:0.1":fork, "uri:oozie:workflow:0.1":join, "uri:oozie:workflow:0.1":kill, "uri:oozie:workflow:0.1":action, "uri:oozie:workflow:0.1":end}' is expected. It performs an XML Schema validation on the specified workflow XML file. ---++ SLA Operations ---+++ Getting the records of SLA events * This feature is only supported in Oozie 2.0 or later. Example: $ oozie sla -oozie http://localhost:8080/oozie -len 1 . 1091 0000004-100521225917912-oozie-wrkf-C@1 2010-05-22T00:41Z STARTED 1091 The =offset= and =len= option specified the offset and number of sla events to display, default values are =1= and =100= respectively. The return message is XML format that can be easily consumed by SLA users. ---++ Pig Operations ---+++ Submitting a pig job through HTTP Example: $ oozie pig -oozie http://localhost:8080/oozie -file pigScriptFile -config job.properties -X -param_file params . job: 14-20090525161321-oozie-tucu-W . $cat job.properties fs.default.name=hdfs://localhost:9000 mapreduce.jobtracker.kerberos.principal=ccc dfs.namenode.kerberos.principal=ddd oozie.libpath=hdfs://localhost:9000/user/oozie/pig/lib/ The parameters for the job must be provided in a Java Properties file (.properties). jobtracker, namenode, libpath must be specified in this file. pigScriptFile is a local file. All jar files (including pig jar file) and all other files needed by the pig job (e.g., parameter file in above example) need to be uploaded onto HDFS under libpath beforehand. The workflow.xml will be created in Oozie server internally. Users can get the workflow.xml from console or command line(-definition). The job will be created and run right away. [[index][::Go back to Oozie Documentation Index::]]