Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Hive >> mail # user >> HADOOP_HOME requirement


+
kulkarni.swarnim@...) 2012-07-18, 16:31
+
Nitin Pawar 2012-07-18, 16:59
+
kulkarni.swarnim@...) 2012-07-18, 17:03
Copy link to this message
-
Re: HADOOP_HOME requirement
this also depends on what version of hive you are using

On Wed, Jul 18, 2012 at 10:33 PM, [EMAIL PROTECTED] <
[EMAIL PROTECTED]> wrote:

> Thanks for your reply nitin.
>
> Ok. So you mean we always need to set HADOOP_HOME irrespective of "hadoop"
> is on the path or not. Correct?
>
> Little confused because that contradicts what's mentioned here[1].
>
> [1]
> https://cwiki.apache.org/confluence/display/Hive/GettingStarted#GettingStarted-RunningHive
>
>
> Thanks,
>
> On Wed, Jul 18, 2012 at 11:59 AM, Nitin Pawar <[EMAIL PROTECTED]>wrote:
>
>> This is not a bug.
>>
>> even if hadoop was path, hive does not use it.
>> Hive internally uses HADOOP_HOME in the code base. So you will always
>> need to set that for hive.
>> Where as for HADOOP clusters, HADOOP_HOME is deprecated but hive still
>> needs it.
>>
>> Don't know if that answers your question
>>
>> Thanks,
>> Nitin
>>
>>
>> On Wed, Jul 18, 2012 at 10:01 PM, [EMAIL PROTECTED] <
>> [EMAIL PROTECTED]> wrote:
>>
>>> Hello,
>>>
>>> The hive documentation states that either HADOOP_HOME should be set or
>>> hadoop should be on the path. However for some cases, where HADOOP_HOME was
>>> not set but hadoop was on path, I have seen this error pop up:
>>>
>>> java.io.IOException: *Cannot run program "null/bin/hadoop" *(in
>>> directory "/root/swarnim/hive-0.9.0-cern1-SNAPSHOT"): java.io.IOException:
>>> error=2, No such file or directory
>>>  at java.lang.ProcessBuilder.start(ProcessBuilder.java:460)
>>> at java.lang.Runtime.exec(Runtime.java:593)
>>>  at java.lang.Runtime.exec(Runtime.java:431)
>>> at org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:268)
>>>  at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:134)
>>> at
>>> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
>>>  at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1326)
>>> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1118)
>>>  at org.apache.hadoop.hive.ql.Driver.run(Driver.java:951)
>>> at
>>> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:258)
>>>  at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:215)
>>> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:406)
>>>  at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:689)
>>> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:557)
>>>  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>> at
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>  at
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>  at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
>>>
>>> Digging into the code in MapRedTask.java, I found the following
>>> (simplified):
>>>
>>> String *hadoopExec* = conf.getVar(System.getenv("HADOOP_HOME") +
>>> "/bin/hadoop");
>>> ...
>>>
>>> Runtime.getRuntime().exec(*hadoopExec*, env, new File(workDir));
>>>
>>> Clearly, if HADOOP_HOME is not set, the command that it would try to
>>> execute is "null/bin/hadoop" which is exactly the exception I am getting.
>>>
>>> Has anyone else run into this before? Is this a bug?
>>>
>>> Thanks,
>>> --
>>> Swarnim
>>>
>>
>>
>>
>> --
>> Nitin Pawar
>>
>>
>
>
> --
> Swarnim
>

--
Nitin Pawar
+
kulkarni.swarnim@...) 2012-07-18, 17:08
+
Nitin Pawar 2012-07-18, 17:18
+
Vinod Singh 2012-07-18, 17:34
+
kulkarni.swarnim@...) 2012-07-18, 17:50
+
kulkarni.swarnim@...) 2012-07-18, 18:11
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB