Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> HADOOP_HOME requirement


Copy link to this message
-
Re: HADOOP_HOME requirement
My main concern here was that HADOOP_HOME is deprecated since hadoop 0.23.
So I was hoping it could actually function as documented.

FWIW, I found this bug[1] that addresses exactly this issue. The attached
patch makes HADOOP_HOME not required and auto-detects hadoop from the path.
This seems to have ben patched to 0.10.0.

[1] https://issues.apache.org/jira/browse/HIVE-2757

On Wed, Jul 18, 2012 at 12:50 PM, [EMAIL PROTECTED] <
[EMAIL PROTECTED]> wrote:

> Hm. Yeah I tried out with a few version 0.7 -> 0.9 and seems like they all
> do. May be we should just update the documentation then?
>
>
> On Wed, Jul 18, 2012 at 12:34 PM, Vinod Singh <[EMAIL PROTECTED]>wrote:
>
>> We are using Hive 0.7.1 and there  HADOOP_HOME must be exported so that
>> it is available as environment variable.
>>
>> Thanks,
>> Vinod
>>
>>
>> On Wed, Jul 18, 2012 at 10:48 PM, Nitin Pawar <[EMAIL PROTECTED]>wrote:
>>
>>> from hive trunk i can only see this
>>> I am not sure I am 100% sure but I remember setting up HADOOP_HOME always
>>>
>>>
>>> http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/MapRedTask.java
>>>
>>>
>>>       String hadoopExec = conf.getVar(HiveConf.ConfVars.HADOOPBIN);
>>>
>>> this change was introduced in 0.8
>>>
>>> from http://svn.apache.org/repos/asf/hive/branches/branch-0.9/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java <http://svn.apache.org/repos/asf/hive/branches/branch-0.8/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java>
>>>
>>> HADOOPBIN("hadoop.bin.path", System.getenv("HADOOP_HOME") + "/bin/hadoop"),
>>>
>>> On Wed, Jul 18, 2012 at 10:38 PM, [EMAIL PROTECTED] <
>>> [EMAIL PROTECTED]> wrote:
>>>
>>>> 0.9
>>>>
>>>>
>>>> On Wed, Jul 18, 2012 at 12:04 PM, Nitin Pawar <[EMAIL PROTECTED]>wrote:
>>>>
>>>>> this also depends on what version of hive you are using
>>>>>
>>>>>
>>>>> On Wed, Jul 18, 2012 at 10:33 PM, [EMAIL PROTECTED] <
>>>>> [EMAIL PROTECTED]> wrote:
>>>>>
>>>>>> Thanks for your reply nitin.
>>>>>>
>>>>>> Ok. So you mean we always need to set HADOOP_HOME irrespective of
>>>>>> "hadoop" is on the path or not. Correct?
>>>>>>
>>>>>> Little confused because that contradicts what's mentioned here[1].
>>>>>>
>>>>>> [1]
>>>>>> https://cwiki.apache.org/confluence/display/Hive/GettingStarted#GettingStarted-RunningHive
>>>>>>
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> On Wed, Jul 18, 2012 at 11:59 AM, Nitin Pawar <
>>>>>> [EMAIL PROTECTED]> wrote:
>>>>>>
>>>>>>> This is not a bug.
>>>>>>>
>>>>>>> even if hadoop was path, hive does not use it.
>>>>>>> Hive internally uses HADOOP_HOME in the code base. So you will
>>>>>>> always need to set that for hive.
>>>>>>> Where as for HADOOP clusters, HADOOP_HOME is deprecated but hive
>>>>>>> still needs it.
>>>>>>>
>>>>>>> Don't know if that answers your question
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Nitin
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Jul 18, 2012 at 10:01 PM, [EMAIL PROTECTED] <
>>>>>>> [EMAIL PROTECTED]> wrote:
>>>>>>>
>>>>>>>> Hello,
>>>>>>>>
>>>>>>>> The hive documentation states that either HADOOP_HOME should be set
>>>>>>>> or hadoop should be on the path. However for some cases, where HADOOP_HOME
>>>>>>>> was not set but hadoop was on path, I have seen this error pop up:
>>>>>>>>
>>>>>>>> java.io.IOException: *Cannot run program "null/bin/hadoop" *(in
>>>>>>>> directory "/root/swarnim/hive-0.9.0-cern1-SNAPSHOT"): java.io.IOException:
>>>>>>>> error=2, No such file or directory
>>>>>>>>  at java.lang.ProcessBuilder.start(ProcessBuilder.java:460)
>>>>>>>> at java.lang.Runtime.exec(Runtime.java:593)
>>>>>>>>  at java.lang.Runtime.exec(Runtime.java:431)
>>>>>>>> at
>>>>>>>> org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:268)
>>>>>>>>  at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:134)
>>>>>>>> at
>>>>>>>> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
>>>>>>>>  at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1326)

Swarnim