|
Viraj Bhat
2010-06-15, 03:33
Carl Steinbach
2010-06-15, 04:06
Aaron Kimball
2010-06-15, 05:24
Edward Capriolo
2010-06-15, 14:32
Viraj Bhat
2010-06-15, 18:22
Viraj Bhat
2010-06-15, 18:26
Viraj Bhat
2010-06-21, 06:21
Omer, Farah
2010-06-21, 15:52
Carl Steinbach
2010-06-21, 16:01
Carl Steinbach
2010-06-21, 16:13
Viraj Bhat
2010-06-21, 23:20
Carl Steinbach
2010-06-22, 00:51
|
-
Compiling Hive against a different version of Hadoop 20Viraj Bhat 2010-06-15, 03:33
Hi all,
I have a specific version of Hadoop 20, which I need to compile with Hive. I tried the following steps http://wiki.apache.org/hadoop/Hive/DeveloperGuide $ant -Dhadoop.root=~/myhome/hadoop-0.20.9 -Dhadoop.version=0.20.9 Is there a way to disable the ivy downloads. Even though the source tree for hadoop-0.20.9 is located on my home directory, I get the following errors. ivy-resolve: [ivy:resolve] :: Ivy 2.1.0 - 20090925235825 :: http://ant.apache.org/ivy/ :: [ivy:resolve] :: loading settings :: file /projects/hive-svn/trunk/ivy/ivysettings.xml [ivy:resolve] [ivy:resolve] :: problems summary :: [ivy:resolve] :::: WARNINGS [ivy:resolve] module not found: hadoop#core;0.20.9 [ivy:resolve] ==== hadoop-source: tried [ivy:resolve] -- artifact hadoop#core;0.20.9!hadoop.tar.gz(source): [ivy:resolve] http://mirror.facebook.net/facebook/hive-deps/hadoop/core/hadoop-0.20.9/ hadoop-0.20.9.tar.gz [ivy:resolve] ==== apache-snapshot: tried [ivy:resolve] https://repository.apache.org/content/repositories/snapshots/hadoop/core /0.20.9/core-0.20.9.pom [ivy:resolve] -- artifact hadoop#core;0.20.9.!hadoop.tar.gz(source): [ivy:resolve] https://repository.apache.org/content/repositories/snapshots/hadoop/core /0.20.9/hadoop-0.20.9.tar.gz [ivy:resolve] ==== maven2: tried [ivy:resolve] http://repo1.maven.org/maven2/hadoop/core/0.20.9/core-0.20.9.pom [ivy:resolve] -- artifact hadoop#core;0.20.9.hadoop.tar.gz(source): [ivy:resolve] http://repo1.maven.org/maven2/hadoop/core/0.20.9/core-0.20.9.tar.gz [ivy:resolve] :::::::::::::::::::::::::::::::::::::::::::::: [ivy:resolve] :: UNRESOLVED DEPENDENCIES :: [ivy:resolve] :::::::::::::::::::::::::::::::::::::::::::::: [ivy:resolve] :: hadoop#core;0.20.9: not found [ivy:resolve] :::::::::::::::::::::::::::::::::::::::::::::: [ivy:resolve] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS BUILD FAILED /projects/hive-svn/trunk/build.xml:160: The following error occurred while executing this line: /projects/hive-svn/trunk/build.xml:103: The following error occurred while executing this line: /projects/hive-svn/trunk/build-common.xml:151: impossible to resolve dependencies: resolve failed - see output for details Thanks Viraj
-
Re: Compiling Hive against a different version of Hadoop 20Carl Steinbach 2010-06-15, 04:06
Hi Viraj,
You can put your hadoop tarball in the cache directory that ivy uses (probably ~/.ant/cache/hadoop/core/sources). This should prevent Ivy from failing when it looks for version 0.20.9. Thanks. Carl On Mon, Jun 14, 2010 at 8:33 PM, Viraj Bhat <[EMAIL PROTECTED]> wrote: > Hi all, > > I have a specific version of Hadoop 20, which I need to compile with Hive. > > I tried the following steps > > > > http://wiki.apache.org/hadoop/Hive/DeveloperGuide > > > > $ant -Dhadoop.root=~/myhome/hadoop-0.20.9 -Dhadoop.version=0.20.9 > > > > Is there a way to disable the ivy downloads. > > > > Even though the source tree for hadoop-0.20.9 is located on my home > directory, I get the following errors. > > > > ivy-resolve: > > [ivy:resolve] :: Ivy 2.1.0 - 20090925235825 :: http://ant.apache.org/ivy/:: > > [ivy:resolve] :: loading settings :: file > /projects/hive-svn/trunk/ivy/ivysettings.xml > > [ivy:resolve] > > [ivy:resolve] :: problems summary :: > > [ivy:resolve] :::: WARNINGS > > [ivy:resolve] module not found: hadoop#core;0.20.9 > > [ivy:resolve] ==== hadoop-source: tried > > [ivy:resolve] -- artifact hadoop#core;0.20.9!hadoop.tar.gz(source): > > [ivy:resolve] > http://mirror.facebook.net/facebook/hive-deps/hadoop/core/hadoop-0.20.9/hadoop-0.20.9.tar.gz > > [ivy:resolve] ==== apache-snapshot: tried > > [ivy:resolve] > https://repository.apache.org/content/repositories/snapshots/hadoop/core/0.20.9/core-0.20.9.pom > > [ivy:resolve] -- artifact hadoop#core;0.20.9.!hadoop.tar.gz(source): > > [ivy:resolve] > https://repository.apache.org/content/repositories/snapshots/hadoop/core/0.20.9/hadoop-0.20.9.tar.gz > > [ivy:resolve] ==== maven2: tried > > [ivy:resolve] > http://repo1.maven.org/maven2/hadoop/core/0.20.9/core-0.20.9.pom > > [ivy:resolve] -- artifact hadoop#core;0.20.9.hadoop.tar.gz(source): > > [ivy:resolve] > http://repo1.maven.org/maven2/hadoop/core/0.20.9/core-0.20.9.tar.gz > > [ivy:resolve] :::::::::::::::::::::::::::::::::::::::::::::: > > [ivy:resolve] :: UNRESOLVED DEPENDENCIES :: > > [ivy:resolve] :::::::::::::::::::::::::::::::::::::::::::::: > > [ivy:resolve] :: hadoop#core;0.20.9: not found > > [ivy:resolve] :::::::::::::::::::::::::::::::::::::::::::::: > > > > [ivy:resolve] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS > > > > BUILD FAILED > > /projects/hive-svn/trunk/build.xml:160: The following error occurred while > executing this line: > > /projects/hive-svn/trunk/build.xml:103: The following error occurred while > executing this line: > > /projects/hive-svn/trunk/build-common.xml:151: impossible to resolve > dependencies: > > resolve failed - see output for details > > > > > > Thanks Viraj >
-
Re: Compiling Hive against a different version of Hadoop 20Aaron Kimball 2010-06-15, 05:24
Also try supplying this flag to ant: -Doffline=true
On Mon, Jun 14, 2010 at 9:06 PM, Carl Steinbach <[EMAIL PROTECTED]> wrote: > Hi Viraj, > > You can put your hadoop tarball in the cache directory that ivy uses > (probably ~/.ant/cache/hadoop/core/sources). This should prevent Ivy from > failing when it looks for version 0.20.9. > > Thanks. > > Carl > > > On Mon, Jun 14, 2010 at 8:33 PM, Viraj Bhat <[EMAIL PROTECTED]> wrote: > >> Hi all, >> >> I have a specific version of Hadoop 20, which I need to compile with >> Hive. >> >> I tried the following steps >> >> >> >> http://wiki.apache.org/hadoop/Hive/DeveloperGuide >> >> >> >> $ant -Dhadoop.root=~/myhome/hadoop-0.20.9 -Dhadoop.version=0.20.9 >> >> >> >> Is there a way to disable the ivy downloads. >> >> >> >> Even though the source tree for hadoop-0.20.9 is located on my home >> directory, I get the following errors. >> >> >> >> ivy-resolve: >> >> [ivy:resolve] :: Ivy 2.1.0 - 20090925235825 :: http://ant.apache.org/ivy/:: >> >> [ivy:resolve] :: loading settings :: file >> /projects/hive-svn/trunk/ivy/ivysettings.xml >> >> [ivy:resolve] >> >> [ivy:resolve] :: problems summary :: >> >> [ivy:resolve] :::: WARNINGS >> >> [ivy:resolve] module not found: hadoop#core;0.20.9 >> >> [ivy:resolve] ==== hadoop-source: tried >> >> [ivy:resolve] -- artifact hadoop#core;0.20.9!hadoop.tar.gz(source): >> >> [ivy:resolve] >> http://mirror.facebook.net/facebook/hive-deps/hadoop/core/hadoop-0.20.9/hadoop-0.20.9.tar.gz >> >> [ivy:resolve] ==== apache-snapshot: tried >> >> [ivy:resolve] >> https://repository.apache.org/content/repositories/snapshots/hadoop/core/0.20.9/core-0.20.9.pom >> >> [ivy:resolve] -- artifact hadoop#core;0.20.9.!hadoop.tar.gz(source): >> >> [ivy:resolve] >> https://repository.apache.org/content/repositories/snapshots/hadoop/core/0.20.9/hadoop-0.20.9.tar.gz >> >> [ivy:resolve] ==== maven2: tried >> >> [ivy:resolve] >> http://repo1.maven.org/maven2/hadoop/core/0.20.9/core-0.20.9.pom >> >> [ivy:resolve] -- artifact hadoop#core;0.20.9.hadoop.tar.gz(source): >> >> [ivy:resolve] >> http://repo1.maven.org/maven2/hadoop/core/0.20.9/core-0.20.9.tar.gz >> >> [ivy:resolve] :::::::::::::::::::::::::::::::::::::::::::::: >> >> [ivy:resolve] :: UNRESOLVED DEPENDENCIES :: >> >> [ivy:resolve] :::::::::::::::::::::::::::::::::::::::::::::: >> >> [ivy:resolve] :: hadoop#core;0.20.9: not found >> >> [ivy:resolve] :::::::::::::::::::::::::::::::::::::::::::::: >> >> >> >> [ivy:resolve] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS >> >> >> >> BUILD FAILED >> >> /projects/hive-svn/trunk/build.xml:160: The following error occurred while >> executing this line: >> >> /projects/hive-svn/trunk/build.xml:103: The following error occurred while >> executing this line: >> >> /projects/hive-svn/trunk/build-common.xml:151: impossible to resolve >> dependencies: >> >> resolve failed - see output for details >> >> >> >> >> >> Thanks Viraj >> > >
-
Re: Compiling Hive against a different version of Hadoop 20Edward Capriolo 2010-06-15, 14:32
On Tue, Jun 15, 2010 at 1:24 AM, Aaron Kimball <[EMAIL PROTECTED]> wrote:
> Also try supplying this flag to ant: -Doffline=true > > > On Mon, Jun 14, 2010 at 9:06 PM, Carl Steinbach <[EMAIL PROTECTED]> wrote: > >> Hi Viraj, >> >> You can put your hadoop tarball in the cache directory that ivy uses >> (probably ~/.ant/cache/hadoop/core/sources). This should prevent Ivy from >> failing when it looks for version 0.20.9. >> >> Thanks. >> >> Carl >> >> >> On Mon, Jun 14, 2010 at 8:33 PM, Viraj Bhat <[EMAIL PROTECTED]> wrote: >> >>> Hi all, >>> >>> I have a specific version of Hadoop 20, which I need to compile with >>> Hive. >>> >>> I tried the following steps >>> >>> >>> >>> http://wiki.apache.org/hadoop/Hive/DeveloperGuide >>> >>> >>> >>> $ant -Dhadoop.root=~/myhome/hadoop-0.20.9 -Dhadoop.version=0.20.9 >>> >>> >>> >>> Is there a way to disable the ivy downloads. >>> >>> >>> >>> Even though the source tree for hadoop-0.20.9 is located on my home >>> directory, I get the following errors. >>> >>> >>> >>> ivy-resolve: >>> >>> [ivy:resolve] :: Ivy 2.1.0 - 20090925235825 :: >>> http://ant.apache.org/ivy/ :: >>> >>> [ivy:resolve] :: loading settings :: file >>> /projects/hive-svn/trunk/ivy/ivysettings.xml >>> >>> [ivy:resolve] >>> >>> [ivy:resolve] :: problems summary :: >>> >>> [ivy:resolve] :::: WARNINGS >>> >>> [ivy:resolve] module not found: hadoop#core;0.20.9 >>> >>> [ivy:resolve] ==== hadoop-source: tried >>> >>> [ivy:resolve] -- artifact hadoop#core;0.20.9!hadoop.tar.gz(source): >>> >>> [ivy:resolve] >>> http://mirror.facebook.net/facebook/hive-deps/hadoop/core/hadoop-0.20.9/hadoop-0.20.9.tar.gz >>> >>> [ivy:resolve] ==== apache-snapshot: tried >>> >>> [ivy:resolve] >>> https://repository.apache.org/content/repositories/snapshots/hadoop/core/0.20.9/core-0.20.9.pom >>> >>> [ivy:resolve] -- artifact hadoop#core;0.20.9.!hadoop.tar.gz(source): >>> >>> [ivy:resolve] >>> https://repository.apache.org/content/repositories/snapshots/hadoop/core/0.20.9/hadoop-0.20.9.tar.gz >>> >>> [ivy:resolve] ==== maven2: tried >>> >>> [ivy:resolve] >>> http://repo1.maven.org/maven2/hadoop/core/0.20.9/core-0.20.9.pom >>> >>> [ivy:resolve] -- artifact hadoop#core;0.20.9.hadoop.tar.gz(source): >>> >>> [ivy:resolve] >>> http://repo1.maven.org/maven2/hadoop/core/0.20.9/core-0.20.9.tar.gz >>> >>> [ivy:resolve] :::::::::::::::::::::::::::::::::::::::::::::: >>> >>> [ivy:resolve] :: UNRESOLVED DEPENDENCIES :: >>> >>> [ivy:resolve] :::::::::::::::::::::::::::::::::::::::::::::: >>> >>> [ivy:resolve] :: hadoop#core;0.20.9: not found >>> >>> [ivy:resolve] :::::::::::::::::::::::::::::::::::::::::::::: >>> >>> >>> >>> [ivy:resolve] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS >>> >>> >>> >>> BUILD FAILED >>> >>> /projects/hive-svn/trunk/build.xml:160: The following error occurred >>> while executing this line: >>> >>> /projects/hive-svn/trunk/build.xml:103: The following error occurred >>> while executing this line: >>> >>> /projects/hive-svn/trunk/build-common.xml:151: impossible to resolve >>> dependencies: >>> >>> resolve failed - see output for details >>> >>> >>> >>> >>> >>> Thanks Viraj >>> >> >> > Also, if I understand correctly, Hive should build the same result regardless of 0.20.9 or 0.20.1 as everything in 0.20.X is API compatable. That being true building with 0.20.1 works, so you may want to stick with that.
-
RE: Compiling Hive against a different version of Hadoop 20Viraj Bhat 2010-06-15, 18:22
Hi Aaron,
I think your suggestion is valuable, but somehow it still tries to contact the: hadoop.mirror=http://mirror.facebook.net/facebook/hive-deps $ant -Dhadoop.root=~/myhome/hadoop-0.20.9 -Dhadoop.version=0.20.9 -Doffline=true ivy-resolve: [ivy:resolve] :: Ivy 2.1.0 - 20090925235825 :: http://ant.apache.org/ivy/ :: [ivy:resolve] :: loading settings :: file /projects/hive-svn/trunk/ivy/ivysettings.xml [ivy:resolve] [ivy:resolve] :: problems summary :: [ivy:resolve] :::: WARNINGS [ivy:resolve] module not found: hadoop#core;0.20.9 [ivy:resolve] ==== hadoop-source: tried [ivy:resolve] -- artifact hadoop#core;0.20.9!hadoop.tar.gz(source): [ivy:resolve] http://mirror.facebook.net/facebook/hive-deps/hadoop/core/hadoop-0.20.9/ hadoop-0.20.9.tar.gz [ivy:resolve] ==== apache-snapshot: tried [ivy:resolve] https://repository.apache.org/content/repositories/snapshots/hadoop/core /0.20.9/core-0.20.9.pom [ivy:resolve] -- artifact hadoop#core;0.20.9!hadoop.tar.gz(source): [ivy:resolve] https://repository.apache.org/content/repositories/snapshots/hadoop/core /0.20.9/hadoop-0.20.9.tar.gz [ivy:resolve] ==== maven2: tried [ivy:resolve] http://repo1.maven.org/maven2/hadoop/core/0.20.9/core-0.20.9.pom [ivy:resolve] -- artifact hadoop#core;0.20.9!hadoop.tar.gz(source): [ivy:resolve] http://repo1.maven.org/maven2/hadoop/core/0.20.9/core-0.20.9.tar.gz [ivy:resolve] :::::::::::::::::::::::::::::::::::::::::::::: [ivy:resolve] :: UNRESOLVED DEPENDENCIES :: [ivy:resolve] :::::::::::::::::::::::::::::::::::::::::::::: [ivy:resolve] :: hadoop#core;0.20.9: not found [ivy:resolve] :::::::::::::::::::::::::::::::::::::::::::::: [ivy:resolve] Viraj ________________________________ From: Aaron Kimball [mailto:[EMAIL PROTECTED]] Sent: Monday, June 14, 2010 10:25 PM To: [EMAIL PROTECTED] Subject: Re: Compiling Hive against a different version of Hadoop 20 Also try supplying this flag to ant: -Doffline=true On Mon, Jun 14, 2010 at 9:06 PM, Carl Steinbach <[EMAIL PROTECTED]> wrote: Hi Viraj, You can put your hadoop tarball in the cache directory that ivy uses (probably ~/.ant/cache/hadoop/core/sources). This should prevent Ivy from failing when it looks for version 0.20.9. Thanks. Carl On Mon, Jun 14, 2010 at 8:33 PM, Viraj Bhat <[EMAIL PROTECTED]> wrote: Hi all, I have a specific version of Hadoop 20, which I need to compile with Hive. I tried the following steps http://wiki.apache.org/hadoop/Hive/DeveloperGuide $ant -Dhadoop.root=~/myhome/hadoop-0.20.9 -Dhadoop.version=0.20.9 Is there a way to disable the ivy downloads. Even though the source tree for hadoop-0.20.9 is located on my home directory, I get the following errors. ivy-resolve: [ivy:resolve] :: Ivy 2.1.0 - 20090925235825 :: http://ant.apache.org/ivy/ :: [ivy:resolve] :: loading settings :: file /projects/hive-svn/trunk/ivy/ivysettings.xml [ivy:resolve] [ivy:resolve] :: problems summary :: [ivy:resolve] :::: WARNINGS [ivy:resolve] module not found: hadoop#core;0.20.9 [ivy:resolve] ==== hadoop-source: tried [ivy:resolve] -- artifact hadoop#core;0.20.9!hadoop.tar.gz(source): [ivy:resolve] http://mirror.facebook.net/facebook/hive-deps/hadoop/core/hadoop-0.20.9/ hadoop-0.20.9.tar.gz [ivy:resolve] ==== apache-snapshot: tried [ivy:resolve] https://repository.apache.org/content/repositories/snapshots/hadoop/core /0.20.9/core-0.20.9.pom [ivy:resolve] -- artifact hadoop#core;0.20.9.!hadoop.tar.gz(source): [ivy:resolve] https://repository.apache.org/content/repositories/snapshots/hadoop/core /0.20.9/hadoop-0.20.9.tar.gz [ivy:resolve] ==== maven2: tried [ivy:resolve] http://repo1.maven.org/maven2/hadoop/core/0.20.9/core-0.20.9.pom [ivy:resolve] -- artifact hadoop#core;0.20.9.hadoop.tar.gz(source): [ivy:resolve] http://repo1.maven.org/maven2/hadoop/core/0.20.9/core-0.20.9.tar.gz [ivy:resolve] :::::::::::::::::::::::::::::::::::::::::::::: [ivy:resolve] :: UNRESOLVED DEPENDENCIES :: [ivy:resolve] :::::::::::::::::::::::::::::::::::::::::::::: [ivy:resolve] :: hadoop#core;0.20.9: not found [ivy:resolve] :::::::::::::::::::::::::::::::::::::::::::::: [ivy:resolve] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS BUILD FAILED /projects/hive-svn/trunk/build.xml:160: The following error occurred while executing this line: /projects/hive-svn/trunk/build.xml:103: The following error occurred while executing this line: /projects/hive-svn/trunk/build-common.xml:151: impossible to resolve dependencies: resolve failed - see output for details Thanks Viraj
-
RE: Compiling Hive against a different version of Hadoop 20Viraj Bhat 2010-06-15, 18:26
Hi Edward,
Thanks for your help. I am compiling against a version of Hadoop which has changed and removed some core classes. I call it Hadoop 20.9, but it is different from the one available in the public hadoop repository. Viraj ________________________________ From: Edward Capriolo [mailto:[EMAIL PROTECTED]] Sent: Tuesday, June 15, 2010 7:33 AM To: [EMAIL PROTECTED] Subject: Re: Compiling Hive against a different version of Hadoop 20 On Tue, Jun 15, 2010 at 1:24 AM, Aaron Kimball <[EMAIL PROTECTED]> wrote: Also try supplying this flag to ant: -Doffline=true On Mon, Jun 14, 2010 at 9:06 PM, Carl Steinbach <[EMAIL PROTECTED]> wrote: Hi Viraj, You can put your hadoop tarball in the cache directory that ivy uses (probably ~/.ant/cache/hadoop/core/sources). This should prevent Ivy from failing when it looks for version 0.20.9. Thanks. Carl On Mon, Jun 14, 2010 at 8:33 PM, Viraj Bhat <[EMAIL PROTECTED]> wrote: Hi all, I have a specific version of Hadoop 20, which I need to compile with Hive. I tried the following steps http://wiki.apache.org/hadoop/Hive/DeveloperGuide $ant -Dhadoop.root=~/myhome/hadoop-0.20.9 -Dhadoop.version=0.20.9 Is there a way to disable the ivy downloads. Even though the source tree for hadoop-0.20.9 is located on my home directory, I get the following errors. ivy-resolve: [ivy:resolve] :: Ivy 2.1.0 - 20090925235825 :: http://ant.apache.org/ivy/ :: [ivy:resolve] :: loading settings :: file /projects/hive-svn/trunk/ivy/ivysettings.xml [ivy:resolve] [ivy:resolve] :: problems summary :: [ivy:resolve] :::: WARNINGS [ivy:resolve] module not found: hadoop#core;0.20.9 [ivy:resolve] ==== hadoop-source: tried [ivy:resolve] -- artifact hadoop#core;0.20.9!hadoop.tar.gz(source): [ivy:resolve] http://mirror.facebook.net/facebook/hive-deps/hadoop/core/hadoop-0.20.9/ hadoop-0.20.9.tar.gz [ivy:resolve] ==== apache-snapshot: tried [ivy:resolve] https://repository.apache.org/content/repositories/snapshots/hadoop/core /0.20.9/core-0.20.9.pom [ivy:resolve] -- artifact hadoop#core;0.20.9.!hadoop.tar.gz(source): [ivy:resolve] https://repository.apache.org/content/repositories/snapshots/hadoop/core /0.20.9/hadoop-0.20.9.tar.gz [ivy:resolve] ==== maven2: tried [ivy:resolve] http://repo1.maven.org/maven2/hadoop/core/0.20.9/core-0.20.9.pom [ivy:resolve] -- artifact hadoop#core;0.20.9.hadoop.tar.gz(source): [ivy:resolve] http://repo1.maven.org/maven2/hadoop/core/0.20.9/core-0.20.9.tar.gz [ivy:resolve] :::::::::::::::::::::::::::::::::::::::::::::: [ivy:resolve] :: UNRESOLVED DEPENDENCIES :: [ivy:resolve] :::::::::::::::::::::::::::::::::::::::::::::: [ivy:resolve] :: hadoop#core;0.20.9: not found [ivy:resolve] :::::::::::::::::::::::::::::::::::::::::::::: [ivy:resolve] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS BUILD FAILED /projects/hive-svn/trunk/build.xml:160: The following error occurred while executing this line: /projects/hive-svn/trunk/build.xml:103: The following error occurred while executing this line: /projects/hive-svn/trunk/build-common.xml:151: impossible to resolve dependencies: resolve failed - see output for details Thanks Viraj Also, if I understand correctly, Hive should build the same result regardless of 0.20.9 or 0.20.1 as everything in 0.20.X is API compatable. That being true building with 0.20.1 works, so you may want to stick with that.
-
Compiling Hive against a different version of Hadoop 20 - Attempt 2Viraj Bhat 2010-06-21, 06:21
Hi all,
I tried compiling this time by running ant clean first. Then I tried running ant -Dhadoop.root=~/projects/newhadoop/hadoop-0.20.10/ -Dhadoop.version=0.20.10 package -Doffline=true BUILD FAILED /homes/viraj/projects/hive-svn/trunk/build.xml:160: The following error occurred while executing this line: /homes/viraj/projects/hive-svn/trunk/build.xml:103: The following error occurred while executing this line: /homes/viraj/projects/hive-svn/trunk/shims/build.xml:56: The following error occurred while executing this line: /homes/viraj/projects/hive-svn/trunk/build-common.xml:127: You need Apache Ivy 2.1 or later from http://ant.apache.org/ It could not be loaded from http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.1.0/ivy-2.1.0.jar Viraj ________________________________ From: Viraj Bhat [mailto:[EMAIL PROTECTED]] Sent: Tuesday, June 15, 2010 11:27 AM To: [EMAIL PROTECTED]; [EMAIL PROTECTED] Subject: RE: Compiling Hive against a different version of Hadoop 20 Hi Edward, Thanks for your help. I am compiling against a version of Hadoop which has changed and removed some core classes. I call it Hadoop 20.9, but it is different from the one available in the public hadoop repository. Viraj ________________________________ From: Edward Capriolo [mailto:[EMAIL PROTECTED]] Sent: Tuesday, June 15, 2010 7:33 AM To: [EMAIL PROTECTED] Subject: Re: Compiling Hive against a different version of Hadoop 20 On Tue, Jun 15, 2010 at 1:24 AM, Aaron Kimball <[EMAIL PROTECTED]> wrote: Also try supplying this flag to ant: -Doffline=true On Mon, Jun 14, 2010 at 9:06 PM, Carl Steinbach <[EMAIL PROTECTED]> wrote: Hi Viraj, You can put your hadoop tarball in the cache directory that ivy uses (probably ~/.ant/cache/hadoop/core/sources). This should prevent Ivy from failing when it looks for version 0.20.9. Thanks. Carl On Mon, Jun 14, 2010 at 8:33 PM, Viraj Bhat <[EMAIL PROTECTED]> wrote: Hi all, I have a specific version of Hadoop 20, which I need to compile with Hive. I tried the following steps http://wiki.apache.org/hadoop/Hive/DeveloperGuide $ant -Dhadoop.root=~/myhome/hadoop-0.20.9 -Dhadoop.version=0.20.9 Is there a way to disable the ivy downloads. Even though the source tree for hadoop-0.20.9 is located on my home directory, I get the following errors. ivy-resolve: [ivy:resolve] :: Ivy 2.1.0 - 20090925235825 :: http://ant.apache.org/ivy/ :: [ivy:resolve] :: loading settings :: file /projects/hive-svn/trunk/ivy/ivysettings.xml [ivy:resolve] [ivy:resolve] :: problems summary :: [ivy:resolve] :::: WARNINGS [ivy:resolve] module not found: hadoop#core;0.20.9 [ivy:resolve] ==== hadoop-source: tried [ivy:resolve] -- artifact hadoop#core;0.20.9!hadoop.tar.gz(source): [ivy:resolve] http://mirror.facebook.net/facebook/hive-deps/hadoop/core/hadoop-0.20.9/ hadoop-0.20.9.tar.gz [ivy:resolve] ==== apache-snapshot: tried [ivy:resolve] https://repository.apache.org/content/repositories/snapshots/hadoop/core /0.20.9/core-0.20.9.pom [ivy:resolve] -- artifact hadoop#core;0.20.9.!hadoop.tar.gz(source): [ivy:resolve] https://repository.apache.org/content/repositories/snapshots/hadoop/core /0.20.9/hadoop-0.20.9.tar.gz [ivy:resolve] ==== maven2: tried [ivy:resolve] http://repo1.maven.org/maven2/hadoop/core/0.20.9/core-0.20.9.pom [ivy:resolve] -- artifact hadoop#core;0.20.9.hadoop.tar.gz(source): [ivy:resolve] http://repo1.maven.org/maven2/hadoop/core/0.20.9/core-0.20.9.tar.gz [ivy:resolve] :::::::::::::::::::::::::::::::::::::::::::::: [ivy:resolve] :: UNRESOLVED DEPENDENCIES :: [ivy:resolve] :::::::::::::::::::::::::::::::::::::::::::::: [ivy:resolve] :: hadoop#core;0.20.9: not found [ivy:resolve] :::::::::::::::::::::::::::::::::::::::::::::: [ivy:resolve] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS BUILD FAILED /projects/hive-svn/trunk/build.xml:160: The following error occurred while executing this line: /projects/hive-svn/trunk/build.xml:103: The following error occurred while executing this line: /projects/hive-svn/trunk/build-common.xml:151: impossible to resolve dependencies: resolve failed - see output for details Thanks Viraj Also, if I understand correctly, Hive should build the same result regardless of 0.20.9 or 0.20.1 as everything in 0.20.X is API compatable. That being true building with 0.20.1 works, so you may want to stick with that.
-
RE: Compiling Hive against a different version of Hadoop 20 - Attempt 2Omer, Farah 2010-06-21, 15:52
I am actually getting a similar error message related to IVY download. I
am basically trying to upgrade the Hive version that I got from the Cloudera HIVE VM, to be upgraded to Hive trunk version. Anybody has an idea on how to fix this? Thanks very much. BUILD FAILED /home/training/hivetrunk/hive/build.xml:160: The following error occurred while executing this line: /home/training/hivetrunk/hive/build.xml:103: The following error occurred while executing this line: /home/training/hivetrunk/hive/shims/build.xml:56: The following error occurred while executing this line: /home/training/hivetrunk/hive/build-common.xml:178: impossible to configure ivy:settings with given file: /home/training/hivetrunk/hive/ivy/ivysettings.xml : java.text.ParseException: failed to load settings from file:/home/training/hivetrunk/hive/ivy/ivysettings.xml: impossible to set defaultTTL to eternal on class org.apache.ivy.core.cache.DefaultRepositoryCacheManager - Farah ________________________________ From: Viraj Bhat [mailto:[EMAIL PROTECTED]] Sent: Monday, June 21, 2010 2:21 AM To: [EMAIL PROTECTED]; [EMAIL PROTECTED] Subject: Compiling Hive against a different version of Hadoop 20 - Attempt 2 Hi all, I tried compiling this time by running ant clean first. Then I tried running ant -Dhadoop.root=~/projects/newhadoop/hadoop-0.20.10/ -Dhadoop.version=0.20.10 package -Doffline=true BUILD FAILED /homes/viraj/projects/hive-svn/trunk/build.xml:160: The following error occurred while executing this line: /homes/viraj/projects/hive-svn/trunk/build.xml:103: The following error occurred while executing this line: /homes/viraj/projects/hive-svn/trunk/shims/build.xml:56: The following error occurred while executing this line: /homes/viraj/projects/hive-svn/trunk/build-common.xml:127: You need Apache Ivy 2.1 or later from http://ant.apache.org/ It could not be loaded from http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.1.0/ivy-2.1.0.jar Viraj ________________________________ From: Viraj Bhat [mailto:[EMAIL PROTECTED]] Sent: Tuesday, June 15, 2010 11:27 AM To: [EMAIL PROTECTED]; [EMAIL PROTECTED] Subject: RE: Compiling Hive against a different version of Hadoop 20 Hi Edward, Thanks for your help. I am compiling against a version of Hadoop which has changed and removed some core classes. I call it Hadoop 20.9, but it is different from the one available in the public hadoop repository. Viraj ________________________________ From: Edward Capriolo [mailto:[EMAIL PROTECTED]] Sent: Tuesday, June 15, 2010 7:33 AM To: [EMAIL PROTECTED] Subject: Re: Compiling Hive against a different version of Hadoop 20 On Tue, Jun 15, 2010 at 1:24 AM, Aaron Kimball <[EMAIL PROTECTED]> wrote: Also try supplying this flag to ant: -Doffline=true On Mon, Jun 14, 2010 at 9:06 PM, Carl Steinbach <[EMAIL PROTECTED]> wrote: Hi Viraj, You can put your hadoop tarball in the cache directory that ivy uses (probably ~/.ant/cache/hadoop/core/sources). This should prevent Ivy from failing when it looks for version 0.20.9. Thanks. Carl On Mon, Jun 14, 2010 at 8:33 PM, Viraj Bhat <[EMAIL PROTECTED]> wrote: Hi all, I have a specific version of Hadoop 20, which I need to compile with Hive. I tried the following steps http://wiki.apache.org/hadoop/Hive/DeveloperGuide $ant -Dhadoop.root=~/myhome/hadoop-0.20.9 -Dhadoop.version=0.20.9 Is there a way to disable the ivy downloads. Even though the source tree for hadoop-0.20.9 is located on my home directory, I get the following errors. ivy-resolve: [ivy:resolve] :: Ivy 2.1.0 - 20090925235825 :: http://ant.apache.org/ivy/ :: [ivy:resolve] :: loading settings :: file /projects/hive-svn/trunk/ivy/ivysettings.xml [ivy:resolve] [ivy:resolve] :: problems summary :: [ivy:resolve] :::: WARNINGS [ivy:resolve] module not found: hadoop#core;0.20.9 [ivy:resolve] ==== hadoop-source: tried [ivy:resolve] -- artifact hadoop#core;0.20.9!hadoop.tar.gz(source): [ivy:resolve] http://mirror.facebook.net/facebook/hive-deps/hadoop/core/hadoop-0.20.9/ hadoop-0.20.9.tar.gz [ivy:resolve] ==== apache-snapshot: tried [ivy:resolve] https://repository.apache.org/content/repositories/snapshots/hadoop/core /0.20.9/core-0.20.9.pom [ivy:resolve] -- artifact hadoop#core;0.20.9.!hadoop.tar.gz(source): [ivy:resolve] https://repository.apache.org/content/repositories/snapshots/hadoop/core /0.20.9/hadoop-0.20.9.tar.gz [ivy:resolve] ==== maven2: tried [ivy:resolve] http://repo1.maven.org/maven2/hadoop/core/0.20.9/core-0.20.9.pom [ivy:resolve] -- artifact hadoop#core;0.20.9.hadoop.tar.gz(source): [ivy:resolve] http://repo1.maven.org/maven2/hadoop/core/0.20.9/core-0.20.9.tar.gz [ivy:resolve] :::::::::::::::::::::::::::::::::::::::::::::: [ivy:resolve] :: UNRESOLVED DEPENDENCIES :: [ivy:resolve] :::::::::::::::::::::::::::::::::::::::::::::: [ivy:resolve] :: hadoop#core;0.20.9: not found [ivy:resolve] :::::::::::::::::::::::::::::::::::::::::::::: [ivy:resolve] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS BUILD FAILED /projects/hive-svn/trunk/build.xml:160: The following error occurred while executing this line: /projects/hive-svn/trunk/build.xml:103: The following error occurred while executing this line: /projects/hive-svn/trunk/build-common.xml:151: impossible to resolve dependencies: resolve failed - see output for details Thanks Viraj Also, if I understand correctly, Hive should build the same result regardless of 0.20.9 or 0.20.1 as everything in 0.20.X is API compatable. That being true building with 0.20.1 works, so you may want to stick with that.
-
Re: Compiling Hive against a different version of Hadoop 20 - Attempt 2Carl Steinbach 2010-06-21, 16:01
Hi Farah,
Looks like you have an older copy of Ivy hanging around in $ANT_HOME/lib (probably ~/.ant/lib/ivy*.jar). You need to delete this older copy of Ivy and try building again. See http://issues.apache.org/jira/browse/HIVE-1120 for more background on this issue. Thanks. Carl On Mon, Jun 21, 2010 at 8:52 AM, Omer, Farah <[EMAIL PROTECTED]>wrote: > I am actually getting a similar error message related to IVY download. I > am basically trying to upgrade the Hive version that I got from the Cloudera > HIVE VM, to be upgraded to Hive trunk version. > > Anybody has an idea on how to fix this? > > Thanks very much. > > > > > > BUILD FAILED > > /home/training/hivetrunk/hive/build.xml:160: The following error occurred > while executing this line: > > /home/training/hivetrunk/hive/build.xml:103: The following error occurred > while executing this line: > > /home/training/hivetrunk/hive/shims/build.xml:56: The following error > occurred while executing this line: > > /home/training/hivetrunk/hive/build-common.xml:178: impossible to configure > ivy:settings with given file: > /home/training/hivetrunk/hive/ivy/ivysettings.xml : > java.text.ParseException: failed to load settings from > file:/home/training/hivetrunk/hive/ivy/ivysettings.xml: impossible to set > defaultTTL to eternal on class > org.apache.ivy.core.cache.DefaultRepositoryCacheManager > > > > *- Farah* > ------------------------------ > > *From:* Viraj Bhat [mailto:[EMAIL PROTECTED]] > *Sent:* Monday, June 21, 2010 2:21 AM > > *To:* [EMAIL PROTECTED]; [EMAIL PROTECTED] > *Subject:* Compiling Hive against a different version of Hadoop 20 - > Attempt 2 > > > > Hi all, > > I tried compiling this time by running ant clean first. > > > > Then I tried running > > ant -Dhadoop.root=~/projects/newhadoop/hadoop-0.20.10/ > -Dhadoop.version=0.20.10 package -Doffline=true > > > > BUILD FAILED > > /homes/viraj/projects/hive-svn/trunk/build.xml:160: The following error > occurred while executing this line: > > /homes/viraj/projects/hive-svn/trunk/build.xml:103: The following error > occurred while executing this line: > > /homes/viraj/projects/hive-svn/trunk/shims/build.xml:56: The following > error occurred while executing this line: > > /homes/viraj/projects/hive-svn/trunk/build-common.xml:127: You need Apache > Ivy 2.1 or later from http://ant.apache.org/ > > It could not be loaded from > http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.1.0/ivy-2.1.0.jar > > > > Viraj > ------------------------------ > > *From:* Viraj Bhat [mailto:[EMAIL PROTECTED]] > *Sent:* Tuesday, June 15, 2010 11:27 AM > *To:* [EMAIL PROTECTED]; [EMAIL PROTECTED] > *Subject:* RE: Compiling Hive against a different version of Hadoop 20 > > > > Hi Edward, > > Thanks for your help. I am compiling against a version of Hadoop which has > changed and removed some core classes. I call it Hadoop 20.9, but it is > different from the one available in the public hadoop repository. > > Viraj > > > ------------------------------ > > *From:* Edward Capriolo [mailto:[EMAIL PROTECTED]] > *Sent:* Tuesday, June 15, 2010 7:33 AM > *To:* [EMAIL PROTECTED] > *Subject:* Re: Compiling Hive against a different version of Hadoop 20 > > > > > > On Tue, Jun 15, 2010 at 1:24 AM, Aaron Kimball <[EMAIL PROTECTED]> wrote: > > Also try supplying this flag to ant: -Doffline=true > > > > On Mon, Jun 14, 2010 at 9:06 PM, Carl Steinbach <[EMAIL PROTECTED]> wrote: > > Hi Viraj, > > > > You can put your hadoop tarball in the cache directory that ivy uses > (probably ~/.ant/cache/hadoop/core/sources). This should prevent Ivy from > failing when it looks for version 0.20.9. > > > > Thanks. > > > > Carl > > > > On Mon, Jun 14, 2010 at 8:33 PM, Viraj Bhat <[EMAIL PROTECTED]> wrote: > > Hi all, > > I have a specific version of Hadoop 20, which I need to compile with Hive. > > I tried the following steps > > > > http://wiki.apache.org/hadoop/Hive/DeveloperGuide > > > > $ant -Dhadoop.root=~/myhome/hadoop-0.20.9 -Dhadoop.version=0.20.9
-
Re: Compiling Hive against a different version of Hadoop 20 - Attempt 2Carl Steinbach 2010-06-21, 16:13
Hi Viraj,
Everything should work if you remove the "-Doffline=true" option. The problem you're running into is that the Hive build depends on ivy-2.1.0.jar which gets downloaded to build/ivy/lib/ivy-2.1.0.jar as one of the first steps in the build, but only if offline=false. This copy of ivy gets blown away whenever you run 'ant clean', so you will need to either leave the '-Doffline=true' switch out, or else run 'ant ivy-download' before running 'ant package -Doffline=true'. Thanks. Carl On Sun, Jun 20, 2010 at 11:21 PM, Viraj Bhat <[EMAIL PROTECTED]> wrote: > Hi all, > > I tried compiling this time by running ant clean first. > > > > Then I tried running > > ant -Dhadoop.root=~/projects/newhadoop/hadoop-0.20.10/ > -Dhadoop.version=0.20.10 package -Doffline=true > > > > BUILD FAILED > > /homes/viraj/projects/hive-svn/trunk/build.xml:160: The following error > occurred while executing this line: > > /homes/viraj/projects/hive-svn/trunk/build.xml:103: The following error > occurred while executing this line: > > /homes/viraj/projects/hive-svn/trunk/shims/build.xml:56: The following > error occurred while executing this line: > > /homes/viraj/projects/hive-svn/trunk/build-common.xml:127: You need Apache > Ivy 2.1 or later from http://ant.apache.org/ > > It could not be loaded from > http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.1.0/ivy-2.1.0.jar > > > > Viraj > ------------------------------ > > *From:* Viraj Bhat [mailto:[EMAIL PROTECTED]] > *Sent:* Tuesday, June 15, 2010 11:27 AM > *To:* [EMAIL PROTECTED]; [EMAIL PROTECTED] > *Subject:* RE: Compiling Hive against a different version of Hadoop 20 > > > > Hi Edward, > > Thanks for your help. I am compiling against a version of Hadoop which has > changed and removed some core classes. I call it Hadoop 20.9, but it is > different from the one available in the public hadoop repository. > > Viraj > > > ------------------------------ > > *From:* Edward Capriolo [mailto:[EMAIL PROTECTED]] > *Sent:* Tuesday, June 15, 2010 7:33 AM > *To:* [EMAIL PROTECTED] > *Subject:* Re: Compiling Hive against a different version of Hadoop 20 > > > > > > On Tue, Jun 15, 2010 at 1:24 AM, Aaron Kimball <[EMAIL PROTECTED]> wrote: > > Also try supplying this flag to ant: -Doffline=true > > > > On Mon, Jun 14, 2010 at 9:06 PM, Carl Steinbach <[EMAIL PROTECTED]> wrote: > > Hi Viraj, > > > > You can put your hadoop tarball in the cache directory that ivy uses > (probably ~/.ant/cache/hadoop/core/sources). This should prevent Ivy from > failing when it looks for version 0.20.9. > > > > Thanks. > > > > Carl > > > > On Mon, Jun 14, 2010 at 8:33 PM, Viraj Bhat <[EMAIL PROTECTED]> wrote: > > Hi all, > > I have a specific version of Hadoop 20, which I need to compile with Hive. > > I tried the following steps > > > > http://wiki.apache.org/hadoop/Hive/DeveloperGuide > > > > $ant -Dhadoop.root=~/myhome/hadoop-0.20.9 -Dhadoop.version=0.20.9 > > > > Is there a way to disable the ivy downloads. > > > > Even though the source tree for hadoop-0.20.9 is located on my home > directory, I get the following errors. > > > > ivy-resolve: > > [ivy:resolve] :: Ivy 2.1.0 - 20090925235825 :: http://ant.apache.org/ivy/:: > > [ivy:resolve] :: loading settings :: file > /projects/hive-svn/trunk/ivy/ivysettings.xml > > [ivy:resolve] > > [ivy:resolve] :: problems summary :: > > [ivy:resolve] :::: WARNINGS > > [ivy:resolve] module not found: hadoop#core;0.20.9 > > [ivy:resolve] ==== hadoop-source: tried > > [ivy:resolve] -- artifact hadoop#core;0.20.9!hadoop.tar.gz(source): > > [ivy:resolve] > http://mirror.facebook.net/facebook/hive-deps/hadoop/core/hadoop-0.20.9/hadoop-0.20.9.tar.gz > > [ivy:resolve] ==== apache-snapshot: tried > > [ivy:resolve] > https://repository.apache.org/content/repositories/snapshots/hadoop/core/0.20.9/core-0.20.9.pom > > [ivy:resolve] -- artifact hadoop#core;0.20.9.!hadoop.tar.gz(source): > > [ivy:resolve] > https://repository.apache.org/content/repositories/snapshots/hadoop/core/0.20.9/hadoop-0.20.9.tar.gz
-
RE: Compiling Hive against a different version of Hadoop 20 - Attempt 2Viraj Bhat 2010-06-21, 23:20
Hi Carl,
I worked on your suggestion and it seems to fail at another point. ant -Dhadoop.root=~/projects/newhadoop/hadoop-0.20.10/ -Dhadoop.version=0.20.10 package -Doffline=true install-hadoopcore-internal: [untar] Expanding: /homes/viraj/projects//hive-svn/trunk/build/hadoopcore/hadoop-0.17.2.1.t ar.gz into /homes/viraj/projects/hive-svn/trunk/build/hadoopcore BUILD FAILED /homes/viraj/projects/hive-svn/trunk/build.xml:160: The following error occurred while executing this line: /homes/viraj/projects/hive-svn/trunk/build.xml:103: The following error occurred while executing this line: /homes/viraj/projects/hive-svn/trunk/shims/build.xml:56: The following error occurred while executing this line: /homes/viraj/projects/hive-svn/trunk/build-common.xml:186: /homes/viraj/projects/hive-svn/trunk/shims/~/projects/hadoop-0.20.10/bin not found. Thanks Viraj ________________________________ From: Carl Steinbach [mailto:[EMAIL PROTECTED]] Sent: Monday, June 21, 2010 9:14 AM To: [EMAIL PROTECTED] Subject: Re: Compiling Hive against a different version of Hadoop 20 - Attempt 2 Hi Viraj, Everything should work if you remove the "-Doffline=true" option. The problem you're running into is that the Hive build depends on ivy-2.1.0.jar which gets downloaded to build/ivy/lib/ivy-2.1.0.jar as one of the first steps in the build, but only if offline=false. This copy of ivy gets blown away whenever you run 'ant clean', so you will need to either leave the '-Doffline=true' switch out, or else run 'ant ivy-download' before running 'ant package -Doffline=true'. Thanks. Carl On Sun, Jun 20, 2010 at 11:21 PM, Viraj Bhat <[EMAIL PROTECTED]> wrote: Hi all, I tried compiling this time by running ant clean first. Then I tried running ant -Dhadoop.root=~/projects/newhadoop/hadoop-0.20.10/ -Dhadoop.version=0.20.10 package -Doffline=true BUILD FAILED /homes/viraj/projects/hive-svn/trunk/build.xml:160: The following error occurred while executing this line: /homes/viraj/projects/hive-svn/trunk/build.xml:103: The following error occurred while executing this line: /homes/viraj/projects/hive-svn/trunk/shims/build.xml:56: The following error occurred while executing this line: /homes/viraj/projects/hive-svn/trunk/build-common.xml:127: You need Apache Ivy 2.1 or later from http://ant.apache.org/ It could not be loaded from http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.1.0/ivy-2.1.0.jar Viraj ________________________________ From: Viraj Bhat [mailto:[EMAIL PROTECTED]] Sent: Tuesday, June 15, 2010 11:27 AM To: [EMAIL PROTECTED]; [EMAIL PROTECTED] Subject: RE: Compiling Hive against a different version of Hadoop 20 Hi Edward, Thanks for your help. I am compiling against a version of Hadoop which has changed and removed some core classes. I call it Hadoop 20.9, but it is different from the one available in the public hadoop repository. Viraj ________________________________ From: Edward Capriolo [mailto:[EMAIL PROTECTED]] Sent: Tuesday, June 15, 2010 7:33 AM To: [EMAIL PROTECTED] Subject: Re: Compiling Hive against a different version of Hadoop 20 On Tue, Jun 15, 2010 at 1:24 AM, Aaron Kimball <[EMAIL PROTECTED]> wrote: Also try supplying this flag to ant: -Doffline=true On Mon, Jun 14, 2010 at 9:06 PM, Carl Steinbach <[EMAIL PROTECTED]> wrote: Hi Viraj, You can put your hadoop tarball in the cache directory that ivy uses (probably ~/.ant/cache/hadoop/core/sources). This should prevent Ivy from failing when it looks for version 0.20.9. Thanks. Carl On Mon, Jun 14, 2010 at 8:33 PM, Viraj Bhat <[EMAIL PROTECTED]> wrote: Hi all, I have a specific version of Hadoop 20, which I need to compile with Hive. I tried the following steps http://wiki.apache.org/hadoop/Hive/DeveloperGuide $ant -Dhadoop.root=~/myhome/hadoop-0.20.9 -Dhadoop.version=0.20.9 Is there a way to disable the ivy downloads. Even though the source tree for hadoop-0.20.9 is located on my home directory, I get the following errors. ivy-resolve: [ivy:resolve] :: Ivy 2.1.0 - 20090925235825 :: http://ant.apache.org/ivy/ :: [ivy:resolve] :: loading settings :: file /projects/hive-svn/trunk/ivy/ivysettings.xml [ivy:resolve] [ivy:resolve] :: problems summary :: [ivy:resolve] :::: WARNINGS [ivy:resolve] module not found: hadoop#core;0.20.9 [ivy:resolve] ==== hadoop-source: tried [ivy:resolve] -- artifact hadoop#core;0.20.9!hadoop.tar.gz(source): [ivy:resolve] http://mirror.facebook.net/facebook/hive-deps/hadoop/core/hadoop-0.20.9/ hadoop-0.20.9.tar.gz [ivy:resolve] ==== apache-snapshot: tried [ivy:resolve] https://repository.apache.org/content/repositories/snapshots/hadoop/core /0.20.9/core-0.20.9.pom [ivy:resolve] -- artifact hadoop#core;0.20.9.!hadoop.tar.gz(source): [ivy:resolve] https://repository.apache.org/content/repositories/snapshots/hadoop/core /0.20.9/hadoop-0.20.9.tar.gz [ivy:resolve] ==== maven2: tried [ivy:resolve] http://repo1.maven.org/maven2/hadoop/core/0.20.9/core-0.20.9.pom [ivy:resolve] -- artifact hadoop#core;0.20.9.hadoop.tar.gz(source): [ivy:resolve] http://repo1.maven.org/maven2/hadoop/core/0.20.9/core-0.20.9.tar.gz [ivy:resolve] :::::::::::::::::::::::::::::::::::::::::::::: [ivy:resolve] :: UNRESOLVED DEPENDENCIES :: [ivy:resolve] :::::::::::::::::::::::::::::::::::::::::::::: [ivy:resolve] :: hadoop#core;0.20.9: not found [ivy:resolve] :::::::::::::::::::::::::::::::::::::::::::::: [ivy:resolve] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS BUILD FAILED /projects/hive-svn/trunk/build.xml:160: The following error occurred while executing this line: /projects/hive-svn/trunk/build.xml:103: The following error occurred while executing this line: /projects/hive-svn/trunk/build-common.xml:151: impossible to resolve depend
-
Re: Compiling Hive against a different version of Hadoop 20 - Attempt 2Carl Steinbach 2010-06-22, 00:51
Hi Viraj,
Your best bet is to take your modified hadoop tar ball (hadoop-0.20.10.tar.gz) and place it in .ant/cache/hadoop/core/sources/ along with the other Hadoop tar balls. Then run "ant -Dhadoop.version=0.20.10 package". Specifying hadoop.root manually is likely to cause problems. The build script will automatically grab hadoop-0.20.10.tar.gz from the ant cache, copy it to build/hadoopcore, unarchive the contents in this directory, and then set hadoop.root to point to build/hadoopcore/hadoop-0.20.10/ Carl On Mon, Jun 21, 2010 at 4:20 PM, Viraj Bhat <[EMAIL PROTECTED]> wrote: > Hi Carl, > > I worked on your suggestion and it seems to fail at another point. > > ant -Dhadoop.root=~/projects/newhadoop/hadoop-0.20.10/ > -Dhadoop.version=0.20.10 package -Doffline=true > > > > > > install-hadoopcore-internal: > > [untar] Expanding: > /homes/viraj/projects//hive-svn/trunk/build/hadoopcore/hadoop-0.17.2.1.tar.gz > into /homes/viraj/projects/hive-svn/trunk/build/hadoopcore > > > > > > BUILD FAILED > > /homes/viraj/projects/hive-svn/trunk/build.xml:160: The following error > occurred while executing this line: > > /homes/viraj/projects/hive-svn/trunk/build.xml:103: The following error > occurred while executing this line: > > /homes/viraj/projects/hive-svn/trunk/shims/build.xml:56: The following > error occurred while executing this line: > > /homes/viraj/projects/hive-svn/trunk/build-common.xml:186: > /homes/viraj/projects/hive-svn/trunk/shims/~/projects/hadoop-0.20.10/bin not > found. > > Thanks Viraj > > > ------------------------------ > > *From:* Carl Steinbach [mailto:[EMAIL PROTECTED]] > *Sent:* Monday, June 21, 2010 9:14 AM > > *To:* [EMAIL PROTECTED] > *Subject:* Re: Compiling Hive against a different version of Hadoop 20 - > Attempt 2 > > > > Hi Viraj, > > > > Everything should work if you remove the "-Doffline=true" option. > > > > The problem you're running into is that the Hive build depends on > ivy-2.1.0.jar which gets downloaded to build/ivy/lib/ivy-2.1.0.jar as one of > the first steps in the build, but only if offline=false. This copy of ivy > gets blown away whenever you run 'ant clean', so you will need to either > leave the '-Doffline=true' switch out, or else run 'ant ivy-download' before > running 'ant package -Doffline=true'. > > > > Thanks. > > > > Carl > > On Sun, Jun 20, 2010 at 11:21 PM, Viraj Bhat <[EMAIL PROTECTED]> wrote: > > Hi all, > > I tried compiling this time by running ant clean first. > > > > Then I tried running > > ant -Dhadoop.root=~/projects/newhadoop/hadoop-0.20.10/ > -Dhadoop.version=0.20.10 package -Doffline=true > > > > BUILD FAILED > > /homes/viraj/projects/hive-svn/trunk/build.xml:160: The following error > occurred while executing this line: > > /homes/viraj/projects/hive-svn/trunk/build.xml:103: The following error > occurred while executing this line: > > /homes/viraj/projects/hive-svn/trunk/shims/build.xml:56: The following > error occurred while executing this line: > > /homes/viraj/projects/hive-svn/trunk/build-common.xml:127: You need Apache > Ivy 2.1 or later from http://ant.apache.org/ > > It could not be loaded from > http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.1.0/ivy-2.1.0.jar > > > > Viraj > ------------------------------ > > *From:* Viraj Bhat [mailto:[EMAIL PROTECTED]] > *Sent:* Tuesday, June 15, 2010 11:27 AM > *To:* [EMAIL PROTECTED]; [EMAIL PROTECTED] > *Subject:* RE: Compiling Hive against a different version of Hadoop 20 > > > > Hi Edward, > > Thanks for your help. I am compiling against a version of Hadoop which has > changed and removed some core classes. I call it Hadoop 20.9, but it is > different from the one available in the public hadoop repository. > > Viraj > > > ------------------------------ > > *From:* Edward Capriolo [mailto:[EMAIL PROTECTED]] > *Sent:* Tuesday, June 15, 2010 7:33 AM > *To:* [EMAIL PROTECTED] > *Subject:* Re: Compiling Hive against a different version of Hadoop 20 > |