Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> Compiling Hive against a different version of Hadoop 20


Copy link to this message
-
RE: Compiling Hive against a different version of Hadoop 20 - Attempt 2
Hi Carl,

 I worked on your suggestion and it seems to fail at another point.

ant -Dhadoop.root=~/projects/newhadoop/hadoop-0.20.10/
-Dhadoop.version=0.20.10 package -Doffline=true

 

 

install-hadoopcore-internal:

    [untar] Expanding:
/homes/viraj/projects//hive-svn/trunk/build/hadoopcore/hadoop-0.17.2.1.t
ar.gz into /homes/viraj/projects/hive-svn/trunk/build/hadoopcore

 

 

BUILD FAILED

/homes/viraj/projects/hive-svn/trunk/build.xml:160: The following error
occurred while executing this line:

/homes/viraj/projects/hive-svn/trunk/build.xml:103: The following error
occurred while executing this line:

/homes/viraj/projects/hive-svn/trunk/shims/build.xml:56: The following
error occurred while executing this line:

/homes/viraj/projects/hive-svn/trunk/build-common.xml:186:
/homes/viraj/projects/hive-svn/trunk/shims/~/projects/hadoop-0.20.10/bin
not found.

Thanks Viraj

 

________________________________

From: Carl Steinbach [mailto:[EMAIL PROTECTED]]
Sent: Monday, June 21, 2010 9:14 AM
To: [EMAIL PROTECTED]
Subject: Re: Compiling Hive against a different version of Hadoop 20 -
Attempt 2

 

Hi Viraj,

 

Everything should work if you remove the "-Doffline=true" option.

 

The problem you're running into is that the Hive build depends on
ivy-2.1.0.jar which gets downloaded to build/ivy/lib/ivy-2.1.0.jar as
one of the first steps in the build, but only if offline=false. This
copy of ivy gets blown away whenever you run 'ant clean', so you will
need to either leave the '-Doffline=true' switch out, or else run 'ant
ivy-download' before running 'ant package -Doffline=true'.

 

Thanks.

 

Carl

On Sun, Jun 20, 2010 at 11:21 PM, Viraj Bhat <[EMAIL PROTECTED]>
wrote:

Hi all,

 I tried compiling this time by running ant clean first.

 

Then I tried running

ant -Dhadoop.root=~/projects/newhadoop/hadoop-0.20.10/
-Dhadoop.version=0.20.10 package -Doffline=true

 

BUILD FAILED

/homes/viraj/projects/hive-svn/trunk/build.xml:160: The following error
occurred while executing this line:

/homes/viraj/projects/hive-svn/trunk/build.xml:103: The following error
occurred while executing this line:

/homes/viraj/projects/hive-svn/trunk/shims/build.xml:56: The following
error occurred while executing this line:

/homes/viraj/projects/hive-svn/trunk/build-common.xml:127: You need
Apache Ivy 2.1 or later from http://ant.apache.org/

      It could not be loaded from
http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.1.0/ivy-2.1.0.jar

 

Viraj

________________________________

From: Viraj Bhat [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, June 15, 2010 11:27 AM
To: [EMAIL PROTECTED]; [EMAIL PROTECTED]
Subject: RE: Compiling Hive against a different version of Hadoop 20

 

Hi Edward,

 Thanks for your help. I am compiling against a version of Hadoop which
has changed and removed some core classes. I call it Hadoop 20.9, but it
is different from the one available in the public hadoop repository.

Viraj

 

________________________________

From: Edward Capriolo [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, June 15, 2010 7:33 AM
To: [EMAIL PROTECTED]
Subject: Re: Compiling Hive against a different version of Hadoop 20

 

 

On Tue, Jun 15, 2010 at 1:24 AM, Aaron Kimball <[EMAIL PROTECTED]>
wrote:

Also try supplying this flag to ant: -Doffline=true

 

On Mon, Jun 14, 2010 at 9:06 PM, Carl Steinbach <[EMAIL PROTECTED]>
wrote:

Hi Viraj,

 

You can put your hadoop tarball in the cache directory that ivy uses
(probably ~/.ant/cache/hadoop/core/sources). This should prevent Ivy
from failing when it looks for version 0.20.9.

 

Thanks.

 

Carl

 

On Mon, Jun 14, 2010 at 8:33 PM, Viraj Bhat <[EMAIL PROTECTED]> wrote:

Hi all,

 I have a specific version of Hadoop 20, which I need to compile with
Hive.

I tried the following steps

 

http://wiki.apache.org/hadoop/Hive/DeveloperGuide

 

$ant -Dhadoop.root=~/myhome/hadoop-0.20.9 -Dhadoop.version=0.20.9

 

Is there a way to disable the ivy downloads.

 

Even though the source tree for hadoop-0.20.9 is located on my home
directory, I get the following errors.

 

ivy-resolve:

[ivy:resolve] :: Ivy 2.1.0 - 20090925235825 ::
http://ant.apache.org/ivy/ ::

[ivy:resolve] :: loading settings :: file /projects/hive-svn/trunk/ivy/ivysettings.xml

[ivy:resolve]

[ivy:resolve] :: problems summary ::

[ivy:resolve] :::: WARNINGS

[ivy:resolve]           module not found: hadoop#core;0.20.9

[ivy:resolve]   ==== hadoop-source: tried

[ivy:resolve]     -- artifact hadoop#core;0.20.9!hadoop.tar.gz(source):

[ivy:resolve]
http://mirror.facebook.net/facebook/hive-deps/hadoop/core/hadoop-0.20.9/
hadoop-0.20.9.tar.gz

[ivy:resolve]   ==== apache-snapshot: tried

[ivy:resolve]
https://repository.apache.org/content/repositories/snapshots/hadoop/core
/0.20.9/core-0.20.9.pom

[ivy:resolve]     -- artifact hadoop#core;0.20.9.!hadoop.tar.gz(source):

[ivy:resolve]
https://repository.apache.org/content/repositories/snapshots/hadoop/core
/0.20.9/hadoop-0.20.9.tar.gz

[ivy:resolve]   ==== maven2: tried

[ivy:resolve]
http://repo1.maven.org/maven2/hadoop/core/0.20.9/core-0.20.9.pom

[ivy:resolve]     -- artifact hadoop#core;0.20.9.hadoop.tar.gz(source):

[ivy:resolve]
http://repo1.maven.org/maven2/hadoop/core/0.20.9/core-0.20.9.tar.gz

[ivy:resolve]           ::::::::::::::::::::::::::::::::::::::::::::::

[ivy:resolve]           ::          UNRESOLVED DEPENDENCIES         ::

[ivy:resolve]           ::::::::::::::::::::::::::::::::::::::::::::::

[ivy:resolve]           :: hadoop#core;0.20.9: not found

[ivy:resolve]           ::::::::::::::::::::::::::::::::::::::::::::::

 

[ivy:resolve] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS

 

BUILD FAILED

/projects/hive-svn/trunk/build.xml:160: The following error occurred
while executing this line:

/projects/hive-svn/trunk/build.xml:103: The following error occurred
while executing this line:

/projects/hive-svn/trunk/build-common.xml:151: impossible to resolve
depend
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB