|
|
@@ -1,167 +1,172 @@
|
|
|
-<h1><a name="JobDesigner-IntroducingHueJobDesigner"></a>Introducing Hue Job Designer</h1>
|
|
|
+<h1><a name="JobDesigner-IntroducingJobDesigner"></a>Introducing Job Designer</h1>
|
|
|
|
|
|
-<p>The Job Designer application enables you to create and submit Hadoop Map/Reduce jobs to the Hadoop cluster. You can include variables with your jobs to enable you and other users to enter values for the variables when they run your job. The Job Designer supports streaming, MapReduce, and Java jobs. </p>
|
|
|
+<p>The Job Designer application enables you to create and submit Hadoop MapReduce jobs to the Hadoop cluster. You can include variables with your jobs to enable you and other users to enter values for the variables when they run your job. The Job Designer supports MapReduce, streaming, and Java jobs. For more information about Hadoop MapReduce, see the <a href="https://ccp.cloudera.com/display/SUPPORT/Hadoop+Tutorial">Hadoop Tutorial</a>.</p>
|
|
|
|
|
|
-
|
|
|
-<div class='panelMacro'><table class='noteMacro'><colgroup><col width='24'><col></colgroup><tr><td valign='top'><img src="/static/art/help/warning.gif" width="16" height="16" align="absmiddle" alt="" border="0"></td><td><b>Note</b><br /><ul>
|
|
|
- <li>The Job Designer uses Oozie to submit MapReduce jobs. Therefore, Oozie must be installed and configured before you can use JobDesigner.</li>
|
|
|
+<div class='panelMacro'><table class='noteMacro'><colgroup><col width='24'><col></colgroup><tr><td valign='top'><img src="/static/art/help/warning.gif" width="16" height="16" align="absmiddle" alt="" border="0"></td><td><b>Note:</b>
|
|
|
+<ul>
|
|
|
+ <li>Job Designer uses Oozie to submit MapReduce jobs. Therefore, Oozie must be installed and configured before you can use JobDesigner.</li>
|
|
|
+ <li>In order to run streaming jobs as part of a workflow, Oozie must be configured to use the Oozie ShareLib. See <a href="https://ccp.cloudera.com/display/CDH4DOC/Hue+Installation">Hue Installation</a> for more information.</li>
|
|
|
<li>A job's input files must be uploaded to the cluster before you can submit the job.
|
|
|
<br class="atl-forced-newline" /></li>
|
|
|
</ul>
|
|
|
</td></tr></table></div>
|
|
|
|
|
|
|
|
|
+
|
|
|
<h2><a name="JobDesigner-Contents"></a>Contents</h2>
|
|
|
|
|
|
<style type='text/css'>/*<![CDATA[*/
|
|
|
-div.rbtoc1334776075479 {margin-left: 1.5em;padding: 0px;}
|
|
|
-div.rbtoc1334776075479 ul {margin-left: 0px;padding-left: 20px;}
|
|
|
-div.rbtoc1334776075479 li {margin-left: 0px;padding-left: 0px;}
|
|
|
+div.rbtoc1347644622311 {margin-left: 1.5em;padding: 0px;}
|
|
|
+div.rbtoc1347644622311 ul {margin-left: 0px;padding-left: 20px;}
|
|
|
+div.rbtoc1347644622311 li {margin-left: 0px;padding-left: 0px;}
|
|
|
|
|
|
-/*]]>*/</style><div class='rbtoc1334776075479'>
|
|
|
+/*]]>*/</style><div class='rbtoc1347644622311'>
|
|
|
<ul>
|
|
|
- <li><a href='#JobDesigner-IntroducingHueJobDesigner'>Introducing Hue Job Designer</a></li>
|
|
|
+ <li><a href='#JobDesigner-IntroducingJobDesigner'>Introducing Job Designer</a></li>
|
|
|
<li><a href='#JobDesigner-JobDesignerInstallationandConfiguration'>Job Designer Installation and Configuration</a></li>
|
|
|
<li><a href='#JobDesigner-StartingJobDesigner'>Starting Job Designer</a></li>
|
|
|
- <li><a href='#JobDesigner-InstallingtheJobDesignerSamples'>Installing the Job Designer Samples</a></li>
|
|
|
- <li><a href='#JobDesigner-WorkingwithJobDesigns'>Working with Job Designs</a></li>
|
|
|
+ <li><a href='#JobDesigner-InstallingtheSampleJobDesigns'>Installing the Sample Job Designs</a></li>
|
|
|
+ <li><a href='#JobDesigner-JobDesigns'>Job Designs</a></li>
|
|
|
<ul>
|
|
|
+ <li><a href='#JobDesigner-FilteringJobDesigns'>Filtering Job Designs</a></li>
|
|
|
+ <li><a href='#JobDesigner-JobDesignSettings'>Job Design Settings</a></li>
|
|
|
<ul>
|
|
|
+ <li><a href='#JobDesigner-CommonSettings'>Common Settings</a></li>
|
|
|
+</ul>
|
|
|
<li><a href='#JobDesigner-CreatingaMapReduceJobDesign'>Creating a MapReduce Job Design</a></li>
|
|
|
<li><a href='#JobDesigner-CreatingaStreamingJobDesign'>Creating a Streaming Job Design</a></li>
|
|
|
<li><a href='#JobDesigner-CreatingaJavaJobDesign'>Creating a Java Job Design</a></li>
|
|
|
-</ul>
|
|
|
-</ul>
|
|
|
- <li><a href='#JobDesigner-SubmittingaJobtoaCluster'>Submitting a Job to a Cluster</a></li>
|
|
|
-<ul>
|
|
|
-<ul>
|
|
|
+ <li><a href='#JobDesigner-SubmittingaJobDesign'>Submitting a Job Design</a></li>
|
|
|
<li><a href='#JobDesigner-Copying%2CEditing%2CandDeletingaJobDesign'>Copying, Editing, and Deleting a Job Design</a></li>
|
|
|
- <li><a href='#JobDesigner-FilteringtheJobDesigns'>Filtering the Job Designs</a></li>
|
|
|
</ul>
|
|
|
-</ul>
|
|
|
- <li><a href='#JobDesigner-DisplayingJobResults%28theJobSubmissionHistory%29'>Displaying Job Results (the Job Submission History)</a></li>
|
|
|
+ <li><a href='#JobDesigner-DisplayingResultsofSubmittingaJob'>Displaying Results of Submitting a Job</a></li>
|
|
|
</ul></div>
|
|
|
<p><br class="atl-forced-newline" /></p>
|
|
|
|
|
|
<p><a name="JobDesigner-JobDesignerInstallationandConfiguration"></a></p>
|
|
|
+
|
|
|
<h1><a name="JobDesigner-JobDesignerInstallationandConfiguration"></a>Job Designer Installation and Configuration</h1>
|
|
|
|
|
|
-<p>Job Designer is one of the applications that is installed as part of Hue. For more information about installing Hue, see <a href="https://ccp.cloudera.com/display/CDH4B2/Hue+Installation">Hue Installation</a>. For information about installing Oozie, see <a href="https://ccp.cloudera.com/display/CDH4B2/Oozie+Installation">Oozie installation</a>.</p>
|
|
|
+<p>Job Designer is one of the applications that is installed as part of Hue. For more information about installing Hue, see <a href="https://ccp.cloudera.com/display/CDH4DOC/Hue+Installation">Hue Installation</a>. For information about installing Oozie, see <a href="https://ccp.cloudera.com/display/CDH4DOC/Oozie+Installation">Oozie Installation</a>.</p>
|
|
|
|
|
|
<p><a name="JobDesigner-StartingJobDesigner"></a></p>
|
|
|
+
|
|
|
<h1><a name="JobDesigner-StartingJobDesigner"></a>Starting Job Designer</h1>
|
|
|
|
|
|
-<p>To start Job Designer, click the <b>Job Designer</b> tab in the navigation bar at the top of the Hue web page. The <b>Job Designs</b> page opens in the browser.
|
|
|
+<p>To start Job Designer, click the <b>Job Designer</b> icon ( <span class="image-wrap" style=""><img src="/jobsub/static/help/images/21103324.png" height="30" style="border: 0px solid black"/></span>) in the navigation bar at the top of the Hue web page. The <b>Job Designs</b> page opens in the browser.
|
|
|
<br class="atl-forced-newline" /></p>
|
|
|
|
|
|
<p><a name="JobDesigner-InstallingtheJobDesignerSamples"></a></p>
|
|
|
-<h1><a name="JobDesigner-InstallingtheJobDesignerSamples"></a>Installing the Job Designer Samples</h1>
|
|
|
|
|
|
-<p>The Job Designer sample jobs can help you learn how to use Job Designer. </p>
|
|
|
+<h1><a name="JobDesigner-InstallingtheSampleJobDesigns"></a>Installing the Sample Job Designs</h1>
|
|
|
|
|
|
-<p>To install the Job Designer samples, click <b>Install Samples</b> in the <b>Job Designs</b> window and then click <b>Yes</b>. The sample jobs are displayed in the <b>Job Designs</b> window. Job Designer removes the <b>Install Samples</b> button after the samples are installed so you can only install the samples once.
|
|
|
-<br class="atl-forced-newline" /></p>
|
|
|
+<p>The Job Designer sample job designs can help you learn how to use Job Designer. To install the sample job designs, click <b>Install Samples</b> in the <b>Job Designs</b> window and then click <b>Yes</b>. The sample job designs are displayed in the <b>Job Designs</b> window. Job Designer removes the <b>Install Samples</b> button after the samples are installed so you can only install the samples once. </p>
|
|
|
|
|
|
<p><a name="JobDesigner-WorkingwithJobDesigns"></a></p>
|
|
|
-<h1><a name="JobDesigner-WorkingwithJobDesigns"></a>Working with Job Designs</h1>
|
|
|
|
|
|
-<p>In the Job Designer, a job design specifies several meta-level properties of a MapReduce job, including the job design name, description, the MapReduce executable scripts or classes, and any parameters for those scripts or classes. You can create three types of job designs: a <a href="#JobDesigner-CreatingaMapreducedesign">MapReduce Job</a>, a <a href="#JobDesigner-CreatingaStreamingJobDesign">streaming job</a> and a <a href="#JobDesigner-CreatingaJavaJobDesign">Java job design</a>.</p>
|
|
|
+<h1><a name="JobDesigner-JobDesigns"></a>Job Designs</h1>
|
|
|
|
|
|
-<p><a name="JobDesigner-CreatingaMapreducedesign"></a></p>
|
|
|
-<h3><a name="JobDesigner-CreatingaMapReduceJobDesign"></a>Creating a MapReduce Job Design</h3>
|
|
|
-
|
|
|
-<p>A Hadoop JAR consists of MapReduce functions written in Java.</p>
|
|
|
+<p>A job design specifies several meta-level properties of a MapReduce job, including the job design name, description, the MapReduce executable scripts or classes, and any parameters for those scripts or classes. You can create three types of job designs: <a href="#JobDesigner-CreatingaMapreducedesign">MapReduce</a>, <a href="#JobDesigner-CreatingaStreamingJobDesign">streaming</a>, and <a href="#JobDesigner-CreatingaJavaJobDesign">Java</a>.</p>
|
|
|
|
|
|
+<h2><a name="JobDesigner-FilteringJobDesigns"></a>Filtering Job Designs</h2>
|
|
|
|
|
|
-<p><b>To create a MapReduce job design:</b></p>
|
|
|
+<p>You can filter the job designs that appear in the list by owner, name, type, and description. </p>
|
|
|
|
|
|
+<p><b>To filter the Job Designs list:</b></p>
|
|
|
<ol>
|
|
|
- <li>In the <b>Job Designs</b> window, click <b>Create MapReduce Design</b>.
|
|
|
-<br class="atl-forced-newline" /> <br class="atl-forced-newline" />
|
|
|
-The <b>Job Design (MapReduce type)</b> window opens where you can specify information about the MapReduce job.
|
|
|
-<br class="atl-forced-newline" /> <br class="atl-forced-newline" /></li>
|
|
|
- <li>In the <b>Job Design (MapReduce type)</b> window, specify the following information.
|
|
|
-<br class="atl-forced-newline" /></li>
|
|
|
+ <li>In the <b>Job Designs</b> window, click <b>Designs</b>.</li>
|
|
|
+ <li>Enter text in the Filter text box at the top of the <b>Job Designs</b> window. When you type in the Filter field, the designs are dynamically filtered to display only those rows containing text that matches the specified substring.<br/>
|
|
|
+<a name="JobDesigner-DisplayingJobResults"></a></li>
|
|
|
</ol>
|
|
|
|
|
|
|
|
|
+<h2><a name="JobDesigner-JobDesignSettings"></a>Job Design Settings</h2>
|
|
|
+
|
|
|
+<p>All job design settings except Name and Description support the use of variables of the form <tt>$</tt><tt><em>variable_name</em></tt>. When you run the job, a dialog box will appear to enable you to specify the values of the variables.</p>
|
|
|
+
|
|
|
+<h3><a name="JobDesigner-CommonSettings"></a>Common Settings</h3>
|
|
|
+
|
|
|
+<p>All job design types support the settings listed in the following table. </p>
|
|
|
|
|
|
-
|
|
|
-<style type="text/css">
|
|
|
-td {
|
|
|
-border: #dddddd solid 1px;
|
|
|
-height: 36px;
|
|
|
-padding: 5px 10px;
|
|
|
-vertical-align: top;
|
|
|
-}
|
|
|
-th {
|
|
|
-border: #dddddd solid 1px;
|
|
|
-height: 40px;
|
|
|
-padding: 5px 10px;
|
|
|
-vertical-align: center;
|
|
|
-}
|
|
|
-</style>
|
|
|
|
|
|
-<p><br class="atl-forced-newline" /></p>
|
|
|
<div class='table-wrap'>
|
|
|
<table class='confluenceTable'><tbody>
|
|
|
<tr>
|
|
|
-<th class='confluenceTh'> <b>Setting</b> </th>
|
|
|
-<th class='confluenceTh'> <b>Description</b> </th>
|
|
|
+<th class='confluenceTh'> Setting </th>
|
|
|
+<th class='confluenceTh'> Description </th>
|
|
|
</tr>
|
|
|
<tr>
|
|
|
<td class='confluenceTd'> <b>Name</b> </td>
|
|
|
-<td class='confluenceTd'> The <b>Name</b> identifies the MapReduce job and it's collection of parameters. </td>
|
|
|
+<td class='confluenceTd'> Identifies the job and its collection of properties and parameters. </td>
|
|
|
</tr>
|
|
|
<tr>
|
|
|
<td class='confluenceTd'> <b>Description</b> </td>
|
|
|
-<td class='confluenceTd'> Specify a description of the MapReduce job. The description is displayed in the dialog box that appears if you specify variables for the job. </td>
|
|
|
-</tr>
|
|
|
-<tr>
|
|
|
-<td class='confluenceTd'> <b>Jar path</b> </td>
|
|
|
-<td class='confluenceTd'> Specify the name of the JAR file, including the path. </td>
|
|
|
+<td class='confluenceTd'> A description of the job. The description is displayed in the dialog box that appears if you specify variables for the job. <br class="atl-forced-newline" /> </td>
|
|
|
</tr>
|
|
|
<tr>
|
|
|
<td class='confluenceTd'> <b>Job Properties</b> </td>
|
|
|
-<td class='confluenceTd'> Click <b>Add Property</b>, to set a property value: <br/>
|
|
|
-» For <b>Property name</b>, enter a configuration variable name. This field provides auto-completion, so you can type the first few characters of a property name and then select the one you want from a list.<br/>
|
|
|
-» For <b>Value</b>, enter the value you want to use for the variable. </td>
|
|
|
+<td class='confluenceTd'> Job properties. To set a property value, click <b>Add Property</b>. <br class="atl-forced-newline" />
|
|
|
+<ul>
|
|
|
+ <li><b>Property name -</b> a configuration property name. This field provides auto-completion, so you can type the first few characters of a property name and then select the one you want from the drop-down list. </li>
|
|
|
+ <li><b>Value -</b> the property value.</li>
|
|
|
+</ul>
|
|
|
+</td>
|
|
|
</tr>
|
|
|
<tr>
|
|
|
<td class='confluenceTd'> <b>Files</b> </td>
|
|
|
-<td class='confluenceTd'> Click <b>Add File</b>, then enter a file name or browse for a file to upload. </td>
|
|
|
+<td class='confluenceTd'> Files to pass to the job. Equivalent to the Hadoop <tt>-files</tt> option. <br class="atl-forced-newline" /> </td>
|
|
|
</tr>
|
|
|
<tr>
|
|
|
<td class='confluenceTd'> <b>Archives</b> </td>
|
|
|
-<td class='confluenceTd'> Click <b>Add Archive</b>, then enter an archive name or browse for a </td>
|
|
|
+<td class='confluenceTd'> Archives to pass to the job. Equivalent to the Hadoop <tt>-archives</tt> option. <br class="atl-forced-newline" /> </td>
|
|
|
</tr>
|
|
|
</tbody></table>
|
|
|
</div>
|
|
|
|
|
|
-<p><br class="atl-forced-newline" /></p>
|
|
|
+
|
|
|
+
|
|
|
+<p><a name="JobDesigner-CreatingaMapreducedesign"></a></p>
|
|
|
+
|
|
|
+<h2><a name="JobDesigner-CreatingaMapReduceJobDesign"></a>Creating a MapReduce Job Design</h2>
|
|
|
+
|
|
|
+<p>A MapReduce job design consists of MapReduce functions written in Java. You can create a<font color="#000000"> MapReduce job design from existing mapper and reducer classes without having to write a main Java class. You must specify the mapper and reducer classes as well as other MapReduce properties in the Job Properties setting.</font></p>
|
|
|
+
|
|
|
+
|
|
|
+<p><b>To create a MapReduce job design:</b></p>
|
|
|
|
|
|
<ol>
|
|
|
+ <li>In the <b>Job Designs</b> window, click <b>Create MapReduce Design</b>.
|
|
|
+<br class="atl-forced-newline" /></li>
|
|
|
+ <li>In the <b>Job Design (MapReduce type)</b> window, specify the common and job specific information.
|
|
|
+<div class='table-wrap'>
|
|
|
+<table class='confluenceTable'><tbody>
|
|
|
+<tr>
|
|
|
+<th class='confluenceTh'> <b>Setting</b> </th>
|
|
|
+<th class='confluenceTh'> <b>Description</b> </th>
|
|
|
+</tr>
|
|
|
+<tr>
|
|
|
+<td class='confluenceTd'> <b>Jar path</b> </td>
|
|
|
+<td class='confluenceTd'> The fully-qualified path to a JAR file containing the classes that implement the Mapper and Reducer functions. </td>
|
|
|
+</tr>
|
|
|
+</tbody></table>
|
|
|
+</div>
|
|
|
+</li>
|
|
|
<li>Click <b>Save</b> to save the job settings.
|
|
|
<br class="atl-forced-newline" /></li>
|
|
|
</ol>
|
|
|
|
|
|
|
|
|
<p><a name="JobDesigner-CreatingaStreamingJobDesign"></a></p>
|
|
|
-<h3><a name="JobDesigner-CreatingaStreamingJobDesign"></a>Creating a Streaming Job Design</h3>
|
|
|
|
|
|
-<p>Hadoop streaming jobs enable you to create MapReduce functions in any non-Java language that reads standard Unix input and writes standard Unix output. For more information about Hadoop streaming jobs, see <a href="http://archive.cloudera.com/cdh/3/hadoop-0.20.2+320/streaming.html">needs updated link to info</a>.</p>
|
|
|
+<h2><a name="JobDesigner-CreatingaStreamingJobDesign"></a>Creating a Streaming Job Design</h2>
|
|
|
+
|
|
|
+<p>Hadoop streaming jobs enable you to create MapReduce functions in any non-Java language that reads standard Unix input and writes standard Unix output. For more information about Hadoop streaming jobs, see <a href="http://archive.cloudera.com/cdh/3/hadoop-0.20.2+320/streaming.html">Hadoop Streaming</a>.</p>
|
|
|
|
|
|
<p><b>To create a streaming job design:</b></p>
|
|
|
|
|
|
<ol>
|
|
|
- <li>In the <b>Job Designs</b> window, click <b>Create Streaming Design</b>.
|
|
|
-<br class="atl-forced-newline" /> <br class="atl-forced-newline" />
|
|
|
-The <b>Job Design (streaming type)</b> window opens to enable you to specify information about the streaming job.
|
|
|
-<br class="atl-forced-newline" /> <br class="atl-forced-newline" /></li>
|
|
|
- <li>In the <b>Job Design (streaming type)</b> window, specify the following information.
|
|
|
-<br class="atl-forced-newline" /> <br class="atl-forced-newline" />
|
|
|
-<div class='panelMacro'><table class='noteMacro'><colgroup><col width='24'><col></colgroup><tr><td valign='top'><img src="/static/art/help/warning.gif" width="16" height="16" align="absmiddle" alt="" border="0"></td><td><b>Note:</b> You can use variables of the form <tt>$variable_name</tt> for the <b>Mapper</b>, and <b>Reducer</b> settings described in the following table. When the streaming job is run, a dialog box will appear to enable you or users to specify the values of the variables.</td></tr></table></div>
|
|
|
-<p><br class="atl-forced-newline" /></p>
|
|
|
+ <li>In the <b>Job Designs</b> window, click <b>Create Streaming Design</b>.</li>
|
|
|
+ <li>In the <b>Job Design (streaming type)</b> window, specify the common and job specific information.
|
|
|
<div class='table-wrap'>
|
|
|
<table class='confluenceTable'><tbody>
|
|
|
<tr>
|
|
|
@@ -169,69 +174,32 @@ The <b>Job Design (streaming type)</b> window opens to enable you to specify inf
|
|
|
<th class='confluenceTh'> Description </th>
|
|
|
</tr>
|
|
|
<tr>
|
|
|
-<td class='confluenceTd'> <b>Name</b> </td>
|
|
|
-<td class='confluenceTd'> The <b>Name</b> identifies the streaming job design including the associated properties and parameters. </td>
|
|
|
-</tr>
|
|
|
-<tr>
|
|
|
-<td class='confluenceTd'> <b>Description</b> </td>
|
|
|
-<td class='confluenceTd'> Specify a description of the streaming job design. The description is displayed in the dialog box that appears if you specify variables for the job. </td>
|
|
|
-</tr>
|
|
|
-<tr>
|
|
|
<td class='confluenceTd'> <b>Mapper</b> </td>
|
|
|
-<td class='confluenceTd'> Specify the path to the mapper script or class. If the mapper file is not on the machines on the cluster, use the <b>Files</b> option to pack it as a part of job submission. Equivalent to the Hadoop <tt>-mapper</tt> option. </td>
|
|
|
+<td class='confluenceTd'> The path to the mapper script or class. If the mapper file is not on the machines on the cluster, use the <b>Files</b> option to pass it as a part of job submission. Equivalent to the Hadoop <tt>-mapper</tt> option. </td>
|
|
|
</tr>
|
|
|
<tr>
|
|
|
<td class='confluenceTd'> <b>Reducer</b> </td>
|
|
|
-<td class='confluenceTd'> Specify the path to the reducer script or class. If the reducer file is not on the machines on the cluster, use the <b>Files</b> option to pack it as a part of job submission. Equivalent to the Hadoop <tt>-reducer</tt> option. </td>
|
|
|
-</tr>
|
|
|
-<tr>
|
|
|
-<td class='confluenceTd'> <b>Job Properties</b> </td>
|
|
|
-<td class='confluenceTd'> Click <b>Add Property</b>, to set a property value: <br/>
|
|
|
-» For <b>Property name</b>, enter a configuration variable name. <br/>
|
|
|
-» For <b>Value</b>, enter the value you want to use for the variable. </td>
|
|
|
-</tr>
|
|
|
-<tr>
|
|
|
-<td class='confluenceTd'> <b>Files</b> </td>
|
|
|
-<td class='confluenceTd'> Click <b>Add File</b>, then enter a file name</td>
|
|
|
-</tr>
|
|
|
-<tr>
|
|
|
-<td class='confluenceTd'> <b>Archives</b> </td>
|
|
|
-<td class='confluenceTd'> Click <b>Add Archive</b>, then enter an archive name</td>
|
|
|
+<td class='confluenceTd'> The path to the reducer script or class. If the reducer file is not on the machines on the cluster, use the <b>Files</b> option to pass it as a part of job submission. Equivalent to the Hadoop <tt>-reducer</tt> option. </td>
|
|
|
</tr>
|
|
|
</tbody></table>
|
|
|
</div>
|
|
|
-
|
|
|
-<p><br class="atl-forced-newline" /></p></li>
|
|
|
+</li>
|
|
|
+ <li>Click <b>Save</b> to save the job settings.</li>
|
|
|
</ol>
|
|
|
|
|
|
|
|
|
-<p><br class="atl-forced-newline" /></p>
|
|
|
-
|
|
|
-<ol>
|
|
|
- <li>Click <b>Save</b> to save the job settings.
|
|
|
-<br class="atl-forced-newline" /></li>
|
|
|
-</ol>
|
|
|
-
|
|
|
|
|
|
<p><a name="JobDesigner-CreatingaJavaJobDesign"></a></p>
|
|
|
-<h3><a name="JobDesigner-CreatingaJavaJobDesign"></a>Creating a Java Job Design</h3>
|
|
|
|
|
|
-<p>A Hadoop JAR consists of MapReduce functions written in Java.<br/></p>
|
|
|
+<h2><a name="JobDesigner-CreatingaJavaJobDesign"></a>Creating a Java Job Design</h2>
|
|
|
+
|
|
|
+<p>A Java job design consists of MapReduce functions and a main class written in Java.</p>
|
|
|
|
|
|
<p><b>To create a Java job design:</b></p>
|
|
|
|
|
|
<ol>
|
|
|
- <li>In the <b>Job Designs</b> window, click <b>Create Java Design</b>.
|
|
|
-<br class="atl-forced-newline" />
|
|
|
-<br class="atl-forced-newline" />
|
|
|
-The <b>Job Design (java type)</b> window opens where you can specify information about the JAR job.
|
|
|
-<br class="atl-forced-newline" />
|
|
|
-<br class="atl-forced-newline" /></li>
|
|
|
+ <li>In the <b>Job Designs</b> window, click <b>Create Java Design</b>.</li>
|
|
|
<li>In the <b>Job Design (java type)</b> window, specify the following information.
|
|
|
-<br class="atl-forced-newline" />
|
|
|
-<br class="atl-forced-newline" />
|
|
|
-<div class='panelMacro'><table class='noteMacro'><colgroup><col width='24'><col></colgroup><tr><td valign='top'><img src="/static/art/help/warning.gif" width="16" height="16" align="absmiddle" alt="" border="0"></td><td><b>Note:</b> You can use variables of the form <tt>$variable_name</tt> for the <b>Args</b> setting described in the following table. When the JAR job is run, a dialog box will appear to enable you or users to specify the values of the variables.</td></tr></table></div>
|
|
|
-<p><br class="atl-forced-newline" /></p>
|
|
|
<div class='table-wrap'>
|
|
|
<table class='confluenceTable'><tbody>
|
|
|
<tr>
|
|
|
@@ -239,95 +207,59 @@ The <b>Job Design (java type)</b> window opens where you can specify information
|
|
|
<th class='confluenceTh'> <b>Description</b> </th>
|
|
|
</tr>
|
|
|
<tr>
|
|
|
-<td class='confluenceTd'> <b>Name</b> </td>
|
|
|
-<td class='confluenceTd'> The <b>Name</b> identifies the Java job and it's collection of parameters. </td>
|
|
|
-</tr>
|
|
|
-<tr>
|
|
|
-<td class='confluenceTd'> <b>Description</b> </td>
|
|
|
-<td class='confluenceTd'> Specify a description of the java job. The description is displayed in the dialog box that appears if you specify variables for the job. </td>
|
|
|
-</tr>
|
|
|
-<tr>
|
|
|
<td class='confluenceTd'> <b>Jar path</b> </td>
|
|
|
-<td class='confluenceTd'> Specify the name of the JAR file, including the path. </td>
|
|
|
+<td class='confluenceTd'> The fully-qualified path to a JAR file containing the main class and mapper and reducer classes. </td>
|
|
|
</tr>
|
|
|
<tr>
|
|
|
<td class='confluenceTd'> <b>Main class</b> </td>
|
|
|
-<td class='confluenceTd'> Specify the (name of?) the Main class to invoke the program. </td>
|
|
|
+<td class='confluenceTd'> The main class to invoke the program. </td>
|
|
|
</tr>
|
|
|
<tr>
|
|
|
<td class='confluenceTd'> <b>Args</b> </td>
|
|
|
-<td class='confluenceTd'> Specify the arguments you want to pass to the running JAR job. </td>
|
|
|
+<td class='confluenceTd'> The arguments to pass to the main class. </td>
|
|
|
</tr>
|
|
|
<tr>
|
|
|
<td class='confluenceTd'> <b>Java opts</b> </td>
|
|
|
-<td class='confluenceTd'> Specify any Java options you want to specify as part of the job. </td>
|
|
|
-</tr>
|
|
|
-<tr>
|
|
|
-<td class='confluenceTd'> <b>Job Properties</b> </td>
|
|
|
-<td class='confluenceTd'> Click <b>Add Property</b>, to set a property value: <br/>
|
|
|
-» For <b>Property name</b>, enter a configuration variable name. <br/>
|
|
|
-» For <b>Value</b>, enter the value you want to use for the variable. </td>
|
|
|
-</tr>
|
|
|
-<tr>
|
|
|
-<td class='confluenceTd'> <b>Files</b> </td>
|
|
|
-<td class='confluenceTd'> Click <b>Add File</b>, then enter a file name</td>
|
|
|
-</tr>
|
|
|
-<tr>
|
|
|
-<td class='confluenceTd'> <b>Archives</b> </td>
|
|
|
-<td class='confluenceTd'> Click <b>Add Archive</b>, then enter an archive name</td>
|
|
|
+<td class='confluenceTd'> Options to pass to the JVM. </td>
|
|
|
</tr>
|
|
|
</tbody></table>
|
|
|
</div>
|
|
|
-
|
|
|
-<p><br class="atl-forced-newline" /></p></li>
|
|
|
-</ol>
|
|
|
-
|
|
|
-
|
|
|
-<ol>
|
|
|
+</li>
|
|
|
<li>Click <b>Save</b> to save the job settings.</li>
|
|
|
</ol>
|
|
|
|
|
|
|
|
|
<p><a name="JobDesigner-SubmittingaJobtoaCluster"></a></p>
|
|
|
|
|
|
-<h1><a name="JobDesigner-SubmittingaJobtoaCluster"></a>Submitting a Job to a Cluster</h1>
|
|
|
+<h2><a name="JobDesigner-SubmittingaJobDesign"></a>Submitting a Job Design</h2>
|
|
|
|
|
|
-<p><b>To submit a job to a cluster:</b></p>
|
|
|
+<p><b>To submit a job design:</b></p>
|
|
|
|
|
|
<ol>
|
|
|
- <li>In the <b>Job Designs</b> window, click <b>Designs</b> in the upper left corner.
|
|
|
-<br class="atl-forced-newline" />
|
|
|
-Your jobs and other users' jobs are displayed in the <b>Job Designs</b> window.
|
|
|
-<br class="atl-forced-newline" /> <br class="atl-forced-newline" /></li>
|
|
|
- <li>In the <b>Job Design</b> window, click the <b>Submit</b> button for the job you want to submit.
|
|
|
-<br class="atl-forced-newline" /> <br class="atl-forced-newline" /></li>
|
|
|
- <li>If the job contains variables, enter the information requested in the dialog box that appears.
|
|
|
-<br class="atl-forced-newline" />
|
|
|
-For example, the sample streaming PI Calculator job displays the following dialog box to enable you to specify the settings for <b>Iterations per Mapper</b> and <b>Num of mappers</b>.
|
|
|
-<br class="atl-forced-newline" /> <br class="atl-forced-newline" /></li>
|
|
|
- <li>Click <b>Submit</b> to submit the job.
|
|
|
-<br class="atl-forced-newline" /> <br class="atl-forced-newline" />
|
|
|
-After the job is complete, the Job Designer displays the results of the job including the last 10 KB of <tt>stdout</tt> and <tt>stderr</tt> for a streaming job. For example, after the sample streaming PI Calculator job is complete, the following results appear.
|
|
|
-<br class="atl-forced-newline" /> <br class="atl-forced-newline" />
|
|
|
-For information about displaying job results, see <a href="#JobDesigner-DisplayingJobResults">Displaying Job Results</a>.
|
|
|
-<br class="atl-forced-newline" /></li>
|
|
|
+ <li>In the <b>Job Designs</b> window, click <b>Designs</b> in the upper left corner.<br/>
|
|
|
+Your jobs and other users' jobs are displayed in the <b>Job Designs</b> window.</li>
|
|
|
+ <li>Click the <b>Submit</b> button to the right of the job you want to submit.
|
|
|
+ <ol>
|
|
|
+ <li>If the job contains variables, enter the information requested in the dialog box that appears. For example, the sample grep MapReduce design displays a dialog where you specify the output directory.</li>
|
|
|
+ <li>Click <b>Submit</b> to submit the job.</li>
|
|
|
+ </ol>
|
|
|
+ </li>
|
|
|
</ol>
|
|
|
|
|
|
|
|
|
+<p>After the job is complete, the Job Designer displays the results of the job. For information about displaying job results, see <a href="#JobDesigner-DisplayingJobResults">Displaying Job Results</a>.</p>
|
|
|
+
|
|
|
<p><a name="JobDesigner-Copying%2CEditing%2CandDeletingaJobDesign"></a></p>
|
|
|
-<h3><a name="JobDesigner-Copying%2CEditing%2CandDeletingaJobDesign"></a>Copying, Editing, and Deleting a Job Design</h3>
|
|
|
+
|
|
|
+<h2><a name="JobDesigner-Copying%2CEditing%2CandDeletingaJobDesign"></a>Copying, Editing, and Deleting a Job Design</h2>
|
|
|
|
|
|
<p>If you want to edit and use a job but you don't own it, you can make a copy of it and then edit and use the copied job.</p>
|
|
|
|
|
|
<p><b>To copy a job design:</b></p>
|
|
|
|
|
|
<ol>
|
|
|
- <li>In the <b>Job Designs</b> window, click <b>Designs</b>.
|
|
|
-<br class="atl-forced-newline" /> <br class="atl-forced-newline" />
|
|
|
-The jobs are displayed in the <b>Job Designs</b> window.
|
|
|
-<br class="atl-forced-newline" /> <br class="atl-forced-newline" /></li>
|
|
|
- <li>Click the <b>Clone</b> button to the right of the job you want to copy.
|
|
|
-<br class="atl-forced-newline" /> <br class="atl-forced-newline" /></li>
|
|
|
+ <li>In the <b>Job Designs</b> window, click <b>Designs</b>. The jobs are displayed in the <b>Job Designs</b> window.</li>
|
|
|
+ <li>Click the <b>Clone</b> button to the right of the job you want to copy.</li>
|
|
|
<li>In the <b>Job Design Editor</b> window, change the settings and then click <b>Save</b> to save the job settings.</li>
|
|
|
</ol>
|
|
|
|
|
|
@@ -335,12 +267,8 @@ The jobs are displayed in the <b>Job Designs</b> window.
|
|
|
<p><b>To edit a job design:</b></p>
|
|
|
|
|
|
<ol>
|
|
|
- <li>In the <b>Job Designs</b> window, click <b>Designs</b>.
|
|
|
-<br class="atl-forced-newline" /> <br class="atl-forced-newline" />
|
|
|
-The jobs are displayed in the <b>Job Designs</b> window.
|
|
|
-<br class="atl-forced-newline" /> <br class="atl-forced-newline" /></li>
|
|
|
- <li>Click the <b>Edit</b> button to the right of the job you want to copy.
|
|
|
-<br class="atl-forced-newline" /> <br class="atl-forced-newline" /></li>
|
|
|
+ <li>In the <b>Job Designs</b> window, click <b>Designs</b>. The jobs are displayed in the <b>Job Designs</b> window.</li>
|
|
|
+ <li>Click the <b>Edit</b> button to the right of the job you want to copy.</li>
|
|
|
<li>In the <b>Job Design</b> window, change the settings and then click <b>Save</b> to save the job settings.</li>
|
|
|
</ol>
|
|
|
|
|
|
@@ -348,74 +276,44 @@ The jobs are displayed in the <b>Job Designs</b> window.
|
|
|
<p><b>To delete a job design:</b></p>
|
|
|
|
|
|
<ol>
|
|
|
- <li>In the <b>Job Designs</b> window, click <b>job designs</b>.
|
|
|
-<br class="atl-forced-newline" /> <br class="atl-forced-newline" />
|
|
|
-The jobs are displayed in the <b>Job Designs</b> window.
|
|
|
-<br class="atl-forced-newline" /> <br class="atl-forced-newline" /></li>
|
|
|
- <li>In the <b>Job Designs</b> window, select the job, right click, and choose <b>Delete</b> from the context menu.
|
|
|
-<br class="atl-forced-newline" /> <br class="atl-forced-newline" /></li>
|
|
|
+ <li>In the <b>Job Designs</b> window, click <b>Designs</b>. The jobs are displayed in the <b>Job Designs</b> window.</li>
|
|
|
+ <li>In the <b>Job Designs</b> window, select the job, right-click, and choose <b>Delete</b> from the context menu.</li>
|
|
|
<li>Click <b>Ok</b> to confirm the deletion.</li>
|
|
|
</ol>
|
|
|
|
|
|
|
|
|
<p><a name="JobDesigner-FilteringtheJobDesigns"></a></p>
|
|
|
-<h3><a name="JobDesigner-FilteringtheJobDesigns"></a>Filtering the Job Designs</h3>
|
|
|
|
|
|
-<p>You can filter the <b>Job Designs</b> by owner, by job name, or both.</p>
|
|
|
+<h1><a name="JobDesigner-DisplayingResultsofSubmittingaJob"></a>Displaying Results of Submitting a Job</h1>
|
|
|
|
|
|
-<p><b>To filter the Job Designs list:</b></p>
|
|
|
+<p><b>To display the Job Submission History:</b></p>
|
|
|
|
|
|
-<ol>
|
|
|
- <li>In the <b>Job Designs</b> window, click <b>job designs</b>.
|
|
|
-<br class="atl-forced-newline" /> <br class="atl-forced-newline" /></li>
|
|
|
- <li>Enter the name of the owner, job name, or both in the query search boxes at the top of the <b>Job Designs</b> window.
|
|
|
-<br class="atl-forced-newline" /> <br class="atl-forced-newline" />
|
|
|
-<b>Job Designer</b> displays the jobs that match the filter criteria.
|
|
|
-<br class="atl-forced-newline" /> <br class="atl-forced-newline" /></li>
|
|
|
-</ol>
|
|
|
+<p>In the <b>Job Designs</b> window, click the <b>History</b> tab. The jobs are displayed in the <b>Job Submissions History</b> listed by Oozie job ID.</p>
|
|
|
+
|
|
|
+<p><b>To display Job Details:</b></p>
|
|
|
+
|
|
|
+
|
|
|
+<p>In the <b>Job Submission History</b> window, double-click a Oozie Job ID. The results of the job display:</p>
|
|
|
+
|
|
|
+<ul>
|
|
|
+ <li>Actions - a list of actions in the job.
|
|
|
+ <ul>
|
|
|
+ <li>Click the <b>External Id</b> link to display the <a href="Job Browser.html" title="Job Browser">Job Browser</a> application opened to that job.</li>
|
|
|
+ <li>Click <span class="image-wrap" style=""><img src="/jobsub/static/help/images/21627568.png" style="border: 0px solid black"/></span> to display the action configuration. In the action configuration for a MapReduce action, click the value of the mapred.output.dir property to display the job output.</li>
|
|
|
+ </ul>
|
|
|
+ </li>
|
|
|
+ <li>Details - the job details. Click <span class="image-wrap" style=""><img src="/jobsub/static/help/images/21627568.png" style="border: 0px solid black"/></span> to display the Oozie application configuration.</li>
|
|
|
+ <li>Definition - the Oozie application definition.</li>
|
|
|
+ <li>Log - the output log.</li>
|
|
|
+</ul>
|
|
|
|
|
|
|
|
|
-<p><a name="JobDesigner-DisplayingJobResults"></a></p>
|
|
|
-<h1><a name="JobDesigner-DisplayingJobResults%28theJobSubmissionHistory%29"></a>Displaying Job Results (the Job Submission History)</h1>
|
|
|
|
|
|
-<p><b>To display the Job Submission History</b></p>
|
|
|
|
|
|
-<ol>
|
|
|
- <li>In the <b>Job Designs</b> window, click the <b>History</b> tab.
|
|
|
-<br class="atl-forced-newline" />
|
|
|
-<br class="atl-forced-newline" />
|
|
|
-The jobs are displayed in the <b>Job Submissions History</b> list by Oozie job ID.
|
|
|
-<br class="atl-forced-newline" />
|
|
|
-<br class="atl-forced-newline" /></li>
|
|
|
- <li>To display details about a job, double-click the Oozie Job ID.
|
|
|
-<br class="atl-forced-newline" /> <br class="atl-forced-newline" />
|
|
|
-The results of the job appear.
|
|
|
-<br class="atl-forced-newline" />
|
|
|
-<br class="atl-forced-newline" /></li>
|
|
|
- <li>To display details about the job, click the job name under the <b>Launched Jobs</b> heading at the top of the results window.
|
|
|
-<br class="atl-forced-newline" />
|
|
|
-<br class="atl-forced-newline" />
|
|
|
-The following Job Browser screen appears with details about the job.
|
|
|
-<br class="atl-forced-newline" />
|
|
|
-<br class="atl-forced-newline" /></li>
|
|
|
- <li>To display details about a task, click on the view link of task under <b>Recent Tasks</b>.
|
|
|
-<br class="atl-forced-newline" />
|
|
|
-<br class="atl-forced-newline" /></li>
|
|
|
- <li>To view other types of job information, click the <b>Metadata</b> and <b>Counters</b> tabs.
|
|
|
-<br class="atl-forced-newline" />
|
|
|
-<br class="atl-forced-newline" /></li>
|
|
|
- <li>To view the output of the job, click the link next to <b>Output</b> at the top of the screen.
|
|
|
-<br class="atl-forced-newline" />
|
|
|
-<br class="atl-forced-newline" /></li>
|
|
|
- <li>To view status information of all jobs, click the <b>view all tasks</b> link.
|
|
|
-<br class="atl-forced-newline" />
|
|
|
-<br class="atl-forced-newline" />
|
|
|
-Click here for more information about using Job Browser.</li>
|
|
|
-</ol>
|
|
|
|
|
|
|
|
|
-
|
|
|
- </td>
|
|
|
+ <br/>
|
|
|
+ </td>
|
|
|
</tr>
|
|
|
</table>
|
|
|
|