Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> Compiling Hive against a different version of Hadoop 20


Copy link to this message
-
RE: Compiling Hive against a different version of Hadoop 20
Hi Edward,

 Thanks for your help. I am compiling against a version of Hadoop which
has changed and removed some core classes. I call it Hadoop 20.9, but it
is different from the one available in the public hadoop repository.

Viraj

 

________________________________

From: Edward Capriolo [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, June 15, 2010 7:33 AM
To: [EMAIL PROTECTED]
Subject: Re: Compiling Hive against a different version of Hadoop 20

 

 

On Tue, Jun 15, 2010 at 1:24 AM, Aaron Kimball <[EMAIL PROTECTED]>
wrote:

Also try supplying this flag to ant: -Doffline=true

 

On Mon, Jun 14, 2010 at 9:06 PM, Carl Steinbach <[EMAIL PROTECTED]>
wrote:

Hi Viraj,

 

You can put your hadoop tarball in the cache directory that ivy uses
(probably ~/.ant/cache/hadoop/core/sources). This should prevent Ivy
from failing when it looks for version 0.20.9.

 

Thanks.

 

Carl

 

On Mon, Jun 14, 2010 at 8:33 PM, Viraj Bhat <[EMAIL PROTECTED]> wrote:

Hi all,

 I have a specific version of Hadoop 20, which I need to compile with
Hive.

I tried the following steps

 

http://wiki.apache.org/hadoop/Hive/DeveloperGuide

 

$ant -Dhadoop.root=~/myhome/hadoop-0.20.9 -Dhadoop.version=0.20.9

 

Is there a way to disable the ivy downloads.

 

Even though the source tree for hadoop-0.20.9 is located on my home
directory, I get the following errors.

 

ivy-resolve:

[ivy:resolve] :: Ivy 2.1.0 - 20090925235825 ::
http://ant.apache.org/ivy/ ::

[ivy:resolve] :: loading settings :: file /projects/hive-svn/trunk/ivy/ivysettings.xml

[ivy:resolve]

[ivy:resolve] :: problems summary ::

[ivy:resolve] :::: WARNINGS

[ivy:resolve]           module not found: hadoop#core;0.20.9

[ivy:resolve]   ==== hadoop-source: tried

[ivy:resolve]     -- artifact hadoop#core;0.20.9!hadoop.tar.gz(source):

[ivy:resolve]
http://mirror.facebook.net/facebook/hive-deps/hadoop/core/hadoop-0.20.9/
hadoop-0.20.9.tar.gz

[ivy:resolve]   ==== apache-snapshot: tried

[ivy:resolve]
https://repository.apache.org/content/repositories/snapshots/hadoop/core
/0.20.9/core-0.20.9.pom

[ivy:resolve]     -- artifact hadoop#core;0.20.9.!hadoop.tar.gz(source):

[ivy:resolve]
https://repository.apache.org/content/repositories/snapshots/hadoop/core
/0.20.9/hadoop-0.20.9.tar.gz

[ivy:resolve]   ==== maven2: tried

[ivy:resolve]
http://repo1.maven.org/maven2/hadoop/core/0.20.9/core-0.20.9.pom

[ivy:resolve]     -- artifact hadoop#core;0.20.9.hadoop.tar.gz(source):

[ivy:resolve]
http://repo1.maven.org/maven2/hadoop/core/0.20.9/core-0.20.9.tar.gz

[ivy:resolve]           ::::::::::::::::::::::::::::::::::::::::::::::

[ivy:resolve]           ::          UNRESOLVED DEPENDENCIES         ::

[ivy:resolve]           ::::::::::::::::::::::::::::::::::::::::::::::

[ivy:resolve]           :: hadoop#core;0.20.9: not found

[ivy:resolve]           ::::::::::::::::::::::::::::::::::::::::::::::

 

[ivy:resolve] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS

 

BUILD FAILED

/projects/hive-svn/trunk/build.xml:160: The following error occurred
while executing this line:

/projects/hive-svn/trunk/build.xml:103: The following error occurred
while executing this line:

/projects/hive-svn/trunk/build-common.xml:151: impossible to resolve
dependencies:

        resolve failed - see output for details

 

 

Thanks Viraj

 

 
Also, if I understand correctly, Hive should build the same result
regardless of 0.20.9 or 0.20.1 as everything in 0.20.X is API
compatable. That being true building with 0.20.1 works, so you may want
to stick with that.

NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB