| 123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340 |
- <h1><a name="OozieEditorandDashboard-IntroducingOozie%26nbsp%3BEditor%2FDashboard"></a>Introducing Oozie Editor/Dashboard</h1>
- <p>The Oozie Editor/Dashboard application allows you to define Oozie workflow and coordinator applications, run workflow and coordinator jobs, and view the status of jobs. </p>
- <p>A workflow application is a collection of actions arranged in a directed acyclic graph (DAG). It includes control flow nodes (start, end, fork, join, and kill) and action nodes (MapReduce, streaming, Java, Pig, Hive, Sqoop, Shell, and ssh actions). The current release does not support the decision control flow node and the fs and Oozie sub-workflow action nodes.</p>
- <p>A coordinator application allows you to define and execute recurrent and interdependent workflow jobs. The coordinator application defines the conditions under which the execution of workflows can occur.</p>
- <h2><a name="OozieEditorandDashboard-Contents"></a>Contents</h2>
- <style type='text/css'>/*<![CDATA[*/
- div.rbtoc1348526886180 {margin-left: 1.5em;padding: 0px;}
- div.rbtoc1348526886180 ul {margin-left: 0px;padding-left: 20px;}
- div.rbtoc1348526886180 li {margin-left: 0px;padding-left: 0px;}
- /*]]>*/</style><div class='rbtoc1348526886180'>
- <ul>
- <li><a href='#OozieEditorandDashboard-IntroducingOozie%26nbsp%3BEditor%2FDashboard'>Introducing Oozie Editor/Dashboard</a></li>
- <li><a href='#OozieEditorandDashboard-Oozie%26nbsp%3BEditor%2FDashboard%26nbsp%3BInstallationandConfiguration'>Oozie Editor/Dashboard Installation and Configuration</a></li>
- <li><a href='#OozieEditorandDashboard-StartingOozie%26nbsp%3BEditor%2FDashboard'>Starting Oozie Editor/Dashboard</a></li>
- <li><a href='#OozieEditorandDashboard-FilteringListsin%26nbsp%3BOozie%26nbsp%3BEditor%2FDashboard'>Filtering Lists in Oozie Editor/Dashboard</a></li>
- <li><a href='#OozieEditorandDashboard-PermissionsinOozieEditor%2FDashboard'>Permissions in Oozie Editor/Dashboard</a></li>
- <li><a href='#OozieEditorandDashboard-OozieDashboard'>Oozie Dashboard</a></li>
- <ul>
- <li><a href='#OozieEditorandDashboard-Workflows'>Workflows</a></li>
- <li><a href='#OozieEditorandDashboard-Coordinators'>Coordinators</a></li>
- </ul>
- <li><a href='#OozieEditorandDashboard-WorkflowEditor'>Workflow Editor</a></li>
- <ul>
- <li><a href='#OozieEditorandDashboard-OpeningaWorkflow'>Opening a Workflow</a></li>
- <li><a href='#OozieEditorandDashboard-CreatingaWorkflow'>Creating a Workflow</a></li>
- <li><a href='#OozieEditorandDashboard-EditingaWorkflow'>Editing a Workflow</a></li>
- <li><a href='#OozieEditorandDashboard-EditingWorkflowProperties'>Editing Workflow Properties</a></li>
- <li><a href='#OozieEditorandDashboard-SubmittingaWorkflow'>Submitting a Workflow</a></li>
- <li><a href='#OozieEditorandDashboard-SchedulingaWorkflow'>Scheduling a Workflow</a></li>
- </ul>
- <li><a href='#OozieEditorandDashboard-CoordinatorEditor'>Coordinator Editor</a></li>
- <ul>
- <li><a href='#OozieEditorandDashboard-Openinga%26nbsp%3BCoordinator'>Opening a Coordinator</a></li>
- <li><a href='#OozieEditorandDashboard-Creatinga%26nbsp%3BCoordinator'>Creating a Coordinator</a></li>
- <li><a href='#OozieEditorandDashboard-Editinga%26nbsp%3BCoordinator'>Editing a Coordinator</a></li>
- <li><a href='#OozieEditorandDashboard-CreatingaDataset'>Creating a Dataset</a></li>
- <li><a href='#OozieEditorandDashboard-Submittinga%26nbsp%3BCoordinator'>Submitting a Coordinator</a></li>
- </ul>
- <li><a href='#OozieEditorandDashboard-SubmissionsHistory'>Submissions History</a></li>
- </ul></div>
- <h1><a name="OozieEditorandDashboard-Oozie%26nbsp%3BEditor%2FDashboard%26nbsp%3BInstallationandConfiguration"></a>Oozie Editor/Dashboard Installation and Configuration</h1>
- <p>Oozie Editor/Dashboard is one of the applications that is installed as part of Hue. For more information about installing Hue, see <a href="https://ccp.cloudera.com/display/CDH4DOC/Hue+Installation">Hue Installation</a>. For information about Oozie, see <a href="http://archive.cloudera.com/cdh4/cdh/4/oozie/">Oozie Documentation</a>.</p>
- <div class='panelMacro'><table class='noteMacro'><colgroup><col width='24'><col></colgroup><tr><td valign='top'><img src="/static/art/help/warning.gif" width="16" height="16" align="absmiddle" alt="" border="0"></td><td><b>Note:</b> In order to run streaming or Pig jobs as part of a workflow, Oozie must be configured to use the Oozie ShareLib. If this is not the case, Pig and streaming actions will not run. See <a href="https://ccp.cloudera.com/display/CDH4DOC/Hue+Installation">Hue Installation</a> for more information.</td></tr></table></div>
- <h1><a name="OozieEditorandDashboard-StartingOozie%26nbsp%3BEditor%2FDashboard"></a>Starting Oozie Editor/Dashboard</h1>
- <p>To start Oozie Editor/Dashboard, click the <b>Oozie Editor/Dashboard</b> icon ( <span class="image-wrap" style=""><img src="/oozie/static/art/icon.png" width="24" style="border: 0px solid black"/></span>) in the navigation bar at the top of the Hue browser page. <b>Oozie Editor/Dashboard</b> opens with the following screens:</p>
- <ul>
- <li>Dashboad - shows the running and completed workflow and coordinator jobs. The screen is selected and opened to the Workflows page.</li>
- <li>Workflows - shows available workflows.</li>
- <li>Coordinators - shows available coordinators.</li>
- <li>History - shows a list of submitted jobs.</li>
- </ul>
- <h1><a name="OozieEditorandDashboard-FilteringListsin%26nbsp%3BOozie%26nbsp%3BEditor%2FDashboard"></a>Filtering Lists in Oozie Editor/Dashboard</h1>
- <p>The Dashboard, Workflows, Coordinators, and History screens contain lists of workflows, coordinators, and jobs. When you type in the Filter field on these screens, the lists are dynamically filtered to display only those rows containing text that matches the specified substring.</p>
- <h1><a name="OozieEditorandDashboard-PermissionsinOozieEditor%2FDashboard"></a>Permissions in Oozie Editor/Dashboard</h1>
- <p>In the Dashboard workflows and coordinators can only be viewed, submitted, and modified by its owner or a superuser.</p>
- <p>Editor permissions for performing actions on workflows and coordinators are summarized in the following table:</p>
- <style type="text/css">
- td.confluenceTd {
- border: #dddddd solid 1px;
- height: 36px;
- padding: 5px 10px;
- vertical-align: top;
- }
- th.confluenceTh {
- border: #dddddd solid 1px;
- height: 40px;
- padding: 5px 10px;
- vertical-align: center;
- }
- </style>
- <div class='table-wrap'>
- <table class='confluenceTable'><tbody>
- <tr>
- <th class='confluenceTh'> Action </th>
- <th class='confluenceTh'> Superuser or Owner <br class="atl-forced-newline" /> </th>
- <th class='confluenceTh'> All </th>
- </tr>
- <tr>
- <td class='confluenceTd'> View </td>
- <td class='confluenceTd'> Y </td>
- <td class='confluenceTd'> Only if "Is shared" is set <br class="atl-forced-newline" /> </td>
- </tr>
- <tr>
- <td class='confluenceTd'> Submit </td>
- <td class='confluenceTd'> Y </td>
- <td class='confluenceTd'> Only if "Is shared" is set <br class="atl-forced-newline" /> </td>
- </tr>
- <tr>
- <td class='confluenceTd'> Modify </td>
- <td class='confluenceTd'> Y </td>
- <td class='confluenceTd'> N </td>
- </tr>
- </tbody></table>
- </div>
- <h1><a name="OozieEditorandDashboard-OozieDashboard"></a>Oozie Dashboard</h1>
- <p>Oozie Dashboard shows a summary of the running and completed workflow and coordinator jobs.</p>
- <p>You can view jobs for a period up to the last 30 days.</p>
- <p>You can filter the list by date (1, 7, 15, or 30 days) or status (Succeeded, Running, or Killed). The date and status buttons are toggles.</p>
- <h2><a name="OozieEditorandDashboard-Workflows"></a>Workflows</h2>
- <p>Click the <b>Workflows</b> tab to view the running and completed workflows for the filters you have specified.</p>
- <p>Click a workflow row in the Running or Completed table to view detailed information about that workflow.</p>
- <p>For the selected workflow, the following tabs and information is available.</p>
- <ul>
- <li>The <b>Graph</b> tab shows the workflow DAG.</li>
- <li>The <b>Actions</b> tab shows you details about the actions that make up this workflow.
- <ul>
- <li>Click the <b>Id</b> link to see additional details about this action.</li>
- <li>Click the <b>External Id</b> link to view this action in the Hue Job Browser.</li>
- </ul>
- </li>
- <li>The <b>Details</b> tab shows job statistics including start and end times, and provides a link to the actual workflow definition in the File Browser.</li>
- <li>The <b>Configuration</b> tab shows selected job configuration settings.</li>
- <li>The <b>Logs</b> tab shows log output generated by the workflow.</li>
- <li>The <b>Definition</b> tab shows the Oozie workflow definition, as it appears in the <tt>workflow.xml</tt> file (also linked under the application path properties in the <b>Details</b> tab and the <b>Configuration</b> tab).</li>
- </ul>
- <h2><a name="OozieEditorandDashboard-Coordinators"></a>Coordinators</h2>
- <p>Click the <b>Coordinators</b> tab to view the running and completed coordinator jobs for the filters you have specified.</p>
- <p>For the selected coordinator, the following tabs and information is available.</p>
- <ul>
- <li>The <b>Calendar</b> tab shows the timestamp of the job. Click the timestamp to open the workflow DAG. </li>
- <li>The <b>Actions</b> tab shows you details about the actions that make up this coordinator.
- <ul>
- <li>Click the <b>Id</b> link to see additional details about this action.</li>
- <li>Click the <b>External Id</b> link to view this action in the Hue Job Browser.</li>
- </ul>
- </li>
- <li>The <b>Configuration</b> tab shows selected job configuration settings.</li>
- <li>The <b>Logs</b> tab shows log output generated by the coordinator.</li>
- <li>The <b>Definition</b> tab shows the Oozie coordinator definition, as it appears in the <tt>coordinator.xml</tt> file (also linked under the oozie.coord.application.path property in the <b>Configuration</b> tab).</li>
- </ul>
- <h1><a name="OozieEditorandDashboard-WorkflowEditor"></a>Workflow Editor</h1>
- <p>The Workflow Editor is where you create or edit Oozie workflows and submit them for execution. The Workflow Editor comes with several preinstalled sample workflows.</p>
- <p>In Workflow Editor, you can create workflows that include MapReduce, streaming, Java, Pig, Hive, Sqoop, Shell, and ssh actions. You can create these actions in the Workflow Editor, or you can import job designs from Job Designer to be used as actions in your workflow.</p>
- <p>Click the <b>Workflows</b> tab to open the Workflow editor.</p>
- <p>The main page of the workflow editor shows the current set of workflow designs.</p>
- <p>Each row shows a workflow design: its name, description, timestamp of its last modification. It also shows:</p>
- <ul>
- <li><b>Steps</b>: the number of steps in the workflow execution path. This is the number of execution steps between the start and end of the workflow. This will not necessarily be the same as the number of actions in the workflow, if there are control flow nodes in the control path.</li>
- <li><b>Status</b>: who can run the workflow. <b>shared</b> means users other than the owner can access the workflow. <b>personal</b> means only the owner can modify or submit the workflow. The default is personal.</li>
- <li><b>Owner</b>: the user that created the workflow.</li>
- </ul>
- <h2><a name="OozieEditorandDashboard-OpeningaWorkflow"></a>Opening a Workflow</h2>
- <p>To open a workflow, click the workflow. Proceed with <a href="#OozieEditorandDashboard-EditingaWorkflow">Editing a Workflow</a>.</p>
- <h2><a name="OozieEditorandDashboard-CreatingaWorkflow"></a>Creating a Workflow</h2>
- <p>To create a workflow:</p>
- <ol>
- <li>Click the <b>Create</b> button at the top right of the Action Chooser. </li>
- <li>In the Name field, type a name.</li>
- <li>To specify the HDFS deployment directory and Oozie schema version click <b>advanced</b>.</li>
- <li>Click <b>Save</b>. The workflow editor opens. Proceed with <a href="#OozieEditorandDashboard-EditingaWorkflow">Editing a Workflow</a>.</li>
- </ol>
- <p><a name="OozieEditorandDashboard-EditingaWorkflow"></a></p>
- <h2><a name="OozieEditorandDashboard-EditingaWorkflow"></a>Editing a Workflow</h2>
- <p>In the workflow editor you can add and delete actions, clone actions, create and remove fork and join control nodes, and move actions as follows:</p>
- <ul>
- <li>Add actions to the workflow by doing one of the following:
- <ul>
- <li>Click the <b>Add</b> tab.
- <ol>
- <li>Click a <b>+ </b><b><em>action</em></b> button,<b><em> </em></b>where <em>action</em> is MapReduce, Streaming, Java, Pig, Hive, Sqoop, Shell, or Ssh.</li>
- <li>Set the action properties and click <b>Save</b>.</li>
- </ol>
- </li>
- <li>Click the <b>Import</b> tab.
- <ol>
- <li>Click the <b>+ Job Design</b> button.</li>
- <li>Click a radio button next to a job design and click <b>Import</b>.</li>
- </ol>
- </li>
- </ul>
- </li>
- <li>Clone actions by clicking the <span class="image-wrap" style=""><img src="/oozie/static/help/retweet.png" style="border: 0px solid black"/></span> button.</li>
- <li>Create or remove a fork and join by moving an action up or down with the <span class="image-wrap" style=""><img src="/oozie/static/help/updown.png" style="border: 0px solid black"/></span> buttons.</li>
- <li>Change the position of an action by clicking the <span class="image-wrap" style=""><img src="/oozie/static/help/updown.png" style="border: 0px solid black"/></span> buttons in the same direction twice.</li>
- <li><font color="#222222">To upload a file (for example,</font> <font color="#222222">a Pig script) </font><font color="#222222">to the folder containing the files referenced by the workflow, click the</font> <font color="#222222"><b>Upload</b></font><font color="#222222"> button.</font></li>
- </ul>
- <div class='panelMacro'><table class='noteMacro'><colgroup><col width='24'><col></colgroup><tr><td valign='top'><img src="/static/art/help/warning.gif" width="16" height="16" align="absmiddle" alt="" border="0"></td><td><font color="#222222">Each action must have a unique name.</font></td></tr></table></div>
- <h2><a name="OozieEditorandDashboard-EditingWorkflowProperties"></a>Editing Workflow Properties</h2>
- <ol>
- <li>In the workflow editor, click the <b>Properties</b> tab.</li>
- <li>To share the workflow with all users, check the <b>Is shared</b> checkbox.</li>
- <li>To set advanced execution options, click <b>advanced</b> and edit the deployment directory, add parameters and job properties, or Oozie schema version, .</li>
- <li>Click <b>Save</b>.</li>
- </ol>
- <h2><a name="OozieEditorandDashboard-SubmittingaWorkflow"></a>Submitting a Workflow</h2>
- <p>To submit a workflow for execution, click the radio button next to the workflow and click the <b>Submit</b> button.</p>
- <h2><a name="OozieEditorandDashboard-SchedulingaWorkflow"></a>Scheduling a Workflow</h2>
- <p>To schedule a workflow for recurring execution, click the radio button next to the workflow and click the <b>Schedule</b> button. A coordinator is created and opened in the coordinator editor.</p>
- <h1><a name="OozieEditorandDashboard-CoordinatorEditor"></a>Coordinator Editor</h1>
- <p>The Coordinator Editor is where you create or edit Oozie coordinator applications and submit them for execution. The Workflow Editor contains one pre-installed sample coordinator.</p>
- <h2><a name="OozieEditorandDashboard-Openinga%26nbsp%3BCoordinator"></a>Opening a Coordinator</h2>
- <p>To open a coordinator, click the coordinator. Proceed with <a href="#OozieEditorandDashboard-EditingaCoordinator">Editing a Coordinator</a>.</p>
- <h2><a name="OozieEditorandDashboard-Creatinga%26nbsp%3BCoordinator"></a>Creating a Coordinator</h2>
- <p>To create a coordinator:</p>
- <ol>
- <li>Click the <b>Create</b> button at the top right of the Action Chooser. </li>
- <li>In the Name field, type a name.</li>
- <li>In the Workflow drop-down list, choose a workflow that the coordinator will schedule.</li>
- <li>In the Frequency area, specify how often the workflow will be scheduled and how many times it will run.</li>
- <li>Click <b>Save</b>. The coordinator editor opens. Proceed with <a href="#OozieEditorandDashboard-EditingaCoordinator">Editing a Coordinator</a>.</li>
- </ol>
- <p><a name="OozieEditorandDashboard-EditingaCoordinator"></a></p>
- <h2><a name="OozieEditorandDashboard-Editinga%26nbsp%3BCoordinator"></a>Editing a Coordinator</h2>
- <div class='panelMacro'><table class='noteMacro'><colgroup><col width='24'><col></colgroup><tr><td valign='top'><img src="/static/art/help/warning.gif" width="16" height="16" align="absmiddle" alt="" border="0"></td><td><b>Note:</b> Most workflows require either an input dataset, an output dataset, or both.</td></tr></table></div>
- <p>In the coordinator editor you specify coordinator properties and the datasets on which the workflow scheduled by the coordinator will operate as follows:</p>
- <ol>
- <li>To share the coordinator with all users, check the <b>Is shared</b> checkbox.</li>
- <li>To set advanced execution options, click <b>advanced</b> and fill in properties that determine how long a coordinator will wait before timing out, how many coordinators can wait and run concurrently, the coordinator scheduling policy, and the coordinator schema version.</li>
- <li>In the Frequency area, set how many time thes communicator will run for each specified unit, the start and end time of the coordinator, and the timezone of the start and end times.</li>
- <li><font color="#222222">The inputs and outputs of the workflow must be mapped to some data. Click</font> <font color="#222222"><b>Add</b></font> <font color="#222222">and select a dataset from the Dataset drop-down menu and map it to one variable of your workflow.</font><br/>
- <font color="#222222">If no datasets exist, follow the procedure in</font> <a href="#OozieEditorandDashboard-CreatingaDataset">Creating a Dataset</a>.</li>
- <li>Select a dataset from the <b>Dataset</b> drop-down menu.</li>
- <li>Click <b>Save</b>.</li>
- </ol>
- <p><a name="OozieEditorandDashboard-CreatingaDataset"></a></p>
- <h2><a name="OozieEditorandDashboard-CreatingaDataset"></a>Creating a Dataset</h2>
- <ol>
- <li>In the coordinator editor, do one of the following:
- <ul>
- <li>Click the <b>Datasets</b> tab at the top of the editor.</li>
- <li>In the Data area, click the <b>Datasets</b> button.</li>
- </ul>
- </li>
- <li>Click <b>Create</b>.</li>
- <li>In the Start and Frequency fields, specify when and how often input datasets will be available.</li>
- <li>In the Uri field, specify a URI template for the location of input and output datasets. You can specify the variables
- <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent">
- <pre>${YEAR},${MONTH},${DAY},${HOUR},${MINUTE}</pre>
- </div></div>
- <p> to construct URIs and URI paths containing dates and timestamps. For example: </p>
- <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent">
- <pre>hdfs://foo:9000/usr/app/stats/${YEAR}/${MONTH}/data</pre>
- </div></div></li>
- <li>Specify the timezone of the start date.</li>
- <li>In the Done flag field, specify the flag that identifies when input datasets are no longer ready.</li>
- </ol>
- <h2><a name="OozieEditorandDashboard-Submittinga%26nbsp%3BCoordinator"></a>Submitting a Coordinator</h2>
- <p>To submit a coordinator for execution, click the radio button next to the coordinator and click the <b>Submit</b> button.</p>
- <h1><a name="OozieEditorandDashboard-SubmissionsHistory"></a>Submissions History</h1>
- <p>The Submissions History is where you view the history of workflow and coordinator jobs. Clicking a link in the Name column opens the workflow or coordinator in an editor. Clicking a link in the Submission Id column opens the job in the Dashboard.</p>
- <br/>
- </td>
- </tr>
- </table>
- </body>
- </html>
|