Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> Question regarding hadoop jar command usage


Copy link to this message
-
Re: Question regarding hadoop jar command usage
Hi Kay,

Yeah, that line does set your jar as the job jar.  "hadoop jar" expects
java code to configure and submit your job.  "mapred job" takes in a
job.xml configuration file and runs the job based on that.

-Sandy

On Wed, Mar 13, 2013 at 11:07 AM, KayVajj <[EMAIL PROTECTED]> wrote:

> Hi Sandy,
>
> I was going through the RunJar source code and the jar executes locally.
> When the jar fires a mapreduce job,
>
> the way I create JobConf is
>
> JobConf conf = new JobConf(MyJob.class);
>>
>
> Does this set MyJar as the job jar?
>
> Can you explain what is the difference between running an MR job using the
> jar command vs mapred job command (looks like the hadoop job command is
> deprecated).
>
> Thanks
> Kay
>
>
> On Wed, Mar 13, 2013 at 10:14 AM, Sandy Ryza <[EMAIL PROTECTED]>wrote:
>
>> Hi Kay,
>>
>> The jar is just executed locally.  If the jar fires up a mapreduce job
>> and sets itself as the job jar, then mapreduce will handle copying it to
>> the nodes that will use it.
>>
>> -Sandy
>>
>>
>> On Wed, Mar 13, 2013 at 9:01 AM, KayVajj <[EMAIL PROTECTED]> wrote:
>>
>>> I have a question regarding the hadoop jar command. In a cluster of say
>>> nodes n1,n2...n100
>>>
>>> the node n1 has jar Myjar on its local file system.
>>>
>>> If I run the command
>>> hadoop jar local/path/to/Myjar Myclass other-args
>>>
>>> Is the MR job executed just on n1 or any arbitrary node n1..n100?
>>>
>>> If it is any arbitrary node, since Myjar exists only on n1. Is the jar
>>> copied across all nodes on the cluster to run the MR code.
>>>
>>>
>>>
>>>
>>>
>>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB