Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS, mail # user - Question regarding hadoop jar command usage


+
KayVajj 2013-03-13, 16:01
+
bejoy.hadoop@... 2013-03-13, 17:16
+
Sandy Ryza 2013-03-13, 17:14
+
KayVajj 2013-03-13, 18:07
Copy link to this message
-
Re: Question regarding hadoop jar command usage
Sandy Ryza 2013-03-13, 18:34
Hi Kay,

Yeah, that line does set your jar as the job jar.  "hadoop jar" expects
java code to configure and submit your job.  "mapred job" takes in a
job.xml configuration file and runs the job based on that.

-Sandy

On Wed, Mar 13, 2013 at 11:07 AM, KayVajj <[EMAIL PROTECTED]> wrote:

> Hi Sandy,
>
> I was going through the RunJar source code and the jar executes locally.
> When the jar fires a mapreduce job,
>
> the way I create JobConf is
>
> JobConf conf = new JobConf(MyJob.class);
>>
>
> Does this set MyJar as the job jar?
>
> Can you explain what is the difference between running an MR job using the
> jar command vs mapred job command (looks like the hadoop job command is
> deprecated).
>
> Thanks
> Kay
>
>
> On Wed, Mar 13, 2013 at 10:14 AM, Sandy Ryza <[EMAIL PROTECTED]>wrote:
>
>> Hi Kay,
>>
>> The jar is just executed locally.  If the jar fires up a mapreduce job
>> and sets itself as the job jar, then mapreduce will handle copying it to
>> the nodes that will use it.
>>
>> -Sandy
>>
>>
>> On Wed, Mar 13, 2013 at 9:01 AM, KayVajj <[EMAIL PROTECTED]> wrote:
>>
>>> I have a question regarding the hadoop jar command. In a cluster of say
>>> nodes n1,n2...n100
>>>
>>> the node n1 has jar Myjar on its local file system.
>>>
>>> If I run the command
>>> hadoop jar local/path/to/Myjar Myclass other-args
>>>
>>> Is the MR job executed just on n1 or any arbitrary node n1..n100?
>>>
>>> If it is any arbitrary node, since Myjar exists only on n1. Is the jar
>>> copied across all nodes on the cluster to run the MR code.
>>>
>>>
>>>
>>>
>>>
>>
>