Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> HADOOP_HOME requirement


Copy link to this message
-
Re: HADOOP_HOME requirement
from hive trunk i can only see this
I am not sure I am 100% sure but I remember setting up HADOOP_HOME always

http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/MapRedTask.java
      String hadoopExec = conf.getVar(HiveConf.ConfVars.HADOOPBIN);

this change was introduced in 0.8

from http://svn.apache.org/repos/asf/hive/branches/branch-0.9/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
<http://svn.apache.org/repos/asf/hive/branches/branch-0.8/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java>

HADOOPBIN("hadoop.bin.path", System.getenv("HADOOP_HOME") + "/bin/hadoop"),

On Wed, Jul 18, 2012 at 10:38 PM, [EMAIL PROTECTED] <
[EMAIL PROTECTED]> wrote:

> 0.9
>
>
> On Wed, Jul 18, 2012 at 12:04 PM, Nitin Pawar <[EMAIL PROTECTED]>wrote:
>
>> this also depends on what version of hive you are using
>>
>>
>> On Wed, Jul 18, 2012 at 10:33 PM, [EMAIL PROTECTED] <
>> [EMAIL PROTECTED]> wrote:
>>
>>> Thanks for your reply nitin.
>>>
>>> Ok. So you mean we always need to set HADOOP_HOME irrespective of
>>> "hadoop" is on the path or not. Correct?
>>>
>>> Little confused because that contradicts what's mentioned here[1].
>>>
>>> [1]
>>> https://cwiki.apache.org/confluence/display/Hive/GettingStarted#GettingStarted-RunningHive
>>>
>>>
>>> Thanks,
>>>
>>> On Wed, Jul 18, 2012 at 11:59 AM, Nitin Pawar <[EMAIL PROTECTED]>wrote:
>>>
>>>> This is not a bug.
>>>>
>>>> even if hadoop was path, hive does not use it.
>>>> Hive internally uses HADOOP_HOME in the code base. So you will always
>>>> need to set that for hive.
>>>> Where as for HADOOP clusters, HADOOP_HOME is deprecated but hive still
>>>> needs it.
>>>>
>>>> Don't know if that answers your question
>>>>
>>>> Thanks,
>>>> Nitin
>>>>
>>>>
>>>> On Wed, Jul 18, 2012 at 10:01 PM, [EMAIL PROTECTED] <
>>>> [EMAIL PROTECTED]> wrote:
>>>>
>>>>> Hello,
>>>>>
>>>>> The hive documentation states that either HADOOP_HOME should be set or
>>>>> hadoop should be on the path. However for some cases, where HADOOP_HOME was
>>>>> not set but hadoop was on path, I have seen this error pop up:
>>>>>
>>>>> java.io.IOException: *Cannot run program "null/bin/hadoop" *(in
>>>>> directory "/root/swarnim/hive-0.9.0-cern1-SNAPSHOT"): java.io.IOException:
>>>>> error=2, No such file or directory
>>>>>  at java.lang.ProcessBuilder.start(ProcessBuilder.java:460)
>>>>> at java.lang.Runtime.exec(Runtime.java:593)
>>>>>  at java.lang.Runtime.exec(Runtime.java:431)
>>>>> at
>>>>> org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:268)
>>>>>  at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:134)
>>>>> at
>>>>> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
>>>>>  at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1326)
>>>>> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1118)
>>>>>  at org.apache.hadoop.hive.ql.Driver.run(Driver.java:951)
>>>>> at
>>>>> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:258)
>>>>>  at
>>>>> org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:215)
>>>>> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:406)
>>>>>  at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:689)
>>>>> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:557)
>>>>>  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>> at
>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>>>  at
>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>>>  at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
>>>>>
>>>>> Digging into the code in MapRedTask.java, I found the following
>>>>> (simplified):
>>>>>
>>>>> String *hadoopExec* = conf.getVar(System.getenv("HADOOP_HOME") +
>>>>> "/bin/hadoop");
>>>>> ...
>>>>>
>>>>> Runtime.getRuntime().exec(*hadoopExec*, env, new File(workDir));

Nitin Pawar
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB