Hi,

 

   I am getting this below errors when I try to generate eclipse files using
eclipse-files target. Can anybody help me?

 

 

Buildfile: D:\svn\nextgenmapreduce\mapreduce\build.xml

ivy-download:
      [get] Getting:
<http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.2.0/ivy-2.2.0.jar>
http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.2.0/ivy-2.2.0.jar
      [get] To: D:\svn\nextgenmapreduce\mapreduce\ivy\ivy-2.2.0.jar
      [get] Not modified - so not downloaded

ivy-init-dirs:

ivy-probe-antlib:

ivy-init-antlib:

ivy-init:
[ivy:configure] :: Ivy non official version -  ::
<http://ant.apache.org/ivy/> http://ant.apache.org/ivy/ ::
[ivy:configure] :: loading settings :: file D:\svn\nextgenmapreduce\mapreduce\ivy\ivysettings.xml

ivy-resolve-common:
[ivy:resolve]
[ivy:resolve] :: problems summary ::
[ivy:resolve] :::: WARNINGS
[ivy:resolve]                 module not found:
org.apache.hadoop#yarn-server-common;1.0-SNAPSHOT
[ivy:resolve]         ==== apache-snapshot: tried
[ivy:resolve]
<https://repository.apache.org/content/repositories/snapshots/org/apache/had
oop/yarn-server-common/1.0-SNAPSHOT/yarn-server-common-1.0-SNAPSHOT.pom>
https://repository.apache.org/content/repositories/snapshots/org/apache/hado
op/yarn-server-common/1.0-SNAPSHOT/yarn-server-common-1.0-SNAPSHOT.pom
[ivy:resolve]           -- artifact
org.apache.hadoop#yarn-server-common;1.0-SNAPSHOT!yarn-server-common.jar:
[ivy:resolve]
<https://repository.apache.org/content/repositories/snapshots/org/apache/had
oop/yarn-server-common/1.0-SNAPSHOT/yarn-server-common-1.0-SNAPSHOT.jar>
https://repository.apache.org/content/repositories/snapshots/org/apache/hado
op/yarn-server-common/1.0-SNAPSHOT/yarn-server-common-1.0-SNAPSHOT.jar
[ivy:resolve]         ==== maven2: tried
[ivy:resolve]
<http://repo1.maven.org/maven2/org/apache/hadoop/yarn-server-common/1.0-SNAP
SHOT/yarn-server-common-1.0-SNAPSHOT.pom>
http://repo1.maven.org/maven2/org/apache/hadoop/yarn-server-common/1.0-SNAPS
HOT/yarn-server-common-1.0-SNAPSHOT.pom
[ivy:resolve]           -- artifact
org.apache.hadoop#yarn-server-common;1.0-SNAPSHOT!yarn-server-common.jar:
[ivy:resolve]
<http://repo1.maven.org/maven2/org/apache/hadoop/yarn-server-common/1.0-SNAP
SHOT/yarn-server-common-1.0-SNAPSHOT.jar>
http://repo1.maven.org/maven2/org/apache/hadoop/yarn-server-common/1.0-SNAPS
HOT/yarn-server-common-1.0-SNAPSHOT.jar
[ivy:resolve]                 module not found:
org.apache.hadoop#hadoop-mapreduce-client-core;1.0-SNAPSHOT
[ivy:resolve]         ==== apache-snapshot: tried
[ivy:resolve]
<https://repository.apache.org/content/repositories/snapshots/org/apache/had
oop/hadoop-mapreduce-client-core/1.0-SNAPSHOT/hadoop-mapreduce-client-core-1
.0-SNAPSHOT.pom>
https://repository.apache.org/content/repositories/snapshots/org/apache/hado
op/hadoop-mapreduce-client-core/1.0-SNAPSHOT/hadoop-mapreduce-client-core-1.
0-SNAPSHOT.pom
[ivy:resolve]           -- artifact
org.apache.hadoop#hadoop-mapreduce-client-core;1.0-SNAPSHOT!hadoop-mapreduce
-client-core.jar:
[ivy:resolve]
<https://repository.apache.org/content/repositories/snapshots/org/apache/had
oop/hadoop-mapreduce-client-core/1.0-SNAPSHOT/hadoop-mapreduce-client-core-1
.0-SNAPSHOT.jar>
https://repository.apache.org/content/repositories/snapshots/org/apache/hado
op/hadoop-mapreduce-client-core/1.0-SNAPSHOT/hadoop-mapreduce-client-core-1.
0-SNAPSHOT.jar
[ivy:resolve]         ==== maven2: tried
[ivy:resolve]
<http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-mapreduce-client-cor
e/1.0-SNAPSHOT/hadoop-mapreduce-client-core-1.0-SNAPSHOT.pom>
http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-mapreduce-client-core
/1.0-SNAPSHOT/hadoop-mapreduce-client-core-1.0-SNAPSHOT.pom
[ivy:resolve]           -- artifact
org.apache.hadoop#hadoop-mapreduce-client-core;1.0-SNAPSHOT!hadoop-mapreduce
-client-core.jar:
[ivy:resolve]
<http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-mapreduce-client-cor
e/1.0-SNAPSHOT/hadoop-mapreduce-client-core-1.0-SNAPSHOT.jar>
http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-mapreduce-client-core
/1.0-SNAPSHOT/hadoop-mapreduce-client-core-1.0-SNAPSHOT.jar
[ivy:resolve]                 module not found:
org.apache.hadoop#yarn-common;1.0-SNAPSHOT
[ivy:resolve]         ==== apache-snapshot: tried
[ivy:resolve]
<https://repository.apache.org/content/repositories/snapshots/org/apache/had
oop/yarn-common/1.0-SNAPSHOT/yarn-common-1.0-SNAPSHOT.pom>
https://repository.apache.org/content/repositories/snapshots/org/apache/hado
op/yarn-common/1.0-SNAPSHOT/yarn-common-1.0-SNAPSHOT.pom
[ivy:resolve]           -- artifact
org.apache.hadoop#yarn-common;1.0-SNAPSHOT!yarn-common.jar:
[ivy:resolve]
<https://repository.apache.org/content/repositories/snapshots/org/apache/had
oop/yarn-common/1.0-SNAPSHOT/yarn-common-1.0-SNAPSHOT.jar>
https://repository.apache.org/content/repositories/snapshots/org/apache/hado
op/yarn-common/1.0-SNAPSHOT/yarn-common-1.0-SNAPSHOT.jar
[ivy:resolve]         ==== maven2: tried
[ivy:resolve]
<http://repo1.maven.org/maven2/org/apache/hadoop/yarn-common/1.0-SNAPSHOT/ya
rn-common-1.0-SNAPSHOT.pom>
http://repo1.maven.org/maven2/org/apache/hadoop/yarn-common/1.0-SNAPSHOT/yar
n-common-1.0-SNAPSHOT.pom
[ivy:resolve]           -- artifact
org.apache.hadoop#yarn-common;1.0-SNAPSHOT!yarn-common.jar:
[ivy:resolve]
<http://repo1.maven.org/maven2/org/apache/hadoop/yarn-common/1.0-SNAPSHOT/ya
rn-common-1.0-SNAPSHOT.jar>
http://repo1.maven.org/maven2/org/apache/hadoop/yarn-common/1.0-SNAPSHOT/yar
n-common-1.0-SNAPSHOT.jar
[ivy:resolve]                 ::::::::::::::::::::::::::::::::::::::::::::::

[ivy:resolve]                 ::          UNRESOLVED DEPENDENCIES         ::

[ivy:resolve]                 ::::::::::::::::::::::::::::::::::::::::::::::

[ivy:resolve]                 ::
org.apache.hadoop#yarn-server-common;1.0-SNAPSHOT: not found
[ivy:resolve]                 ::
org.apache.hadoop#hadoop-mapreduce-client-core;1.0-SNAPSHOT: not found
[ivy:r

You want to generate them using mvn instead.  See the mapreduce/yarn/README
file for how to do it.

On Fri, Jul 8, 2011 at 7:00 AM, Devaraj K <[EMAIL PROTECTED]> wrote:

> Hi,
>
>
>
>   I am getting this below errors when I try to generate eclipse files using
> eclipse-files target. Can anybody help me?
>
>
>
>
>
> Buildfile: D:\svn\nextgenmapreduce\mapreduce\build.xml
>
> ivy-download:
>      [get] Getting:
> <http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.2.0/ivy-2.2.0.jar>
> http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.2.0/ivy-2.2.0.jar
>      [get] To: D:\svn\nextgenmapreduce\mapreduce\ivy\ivy-2.2.0.jar
>      [get] Not modified - so not downloaded
>
> ivy-init-dirs:
>
> ivy-probe-antlib:
>
> ivy-init-antlib:
>
> ivy-init:
> [ivy:configure] :: Ivy non official version -  ::
> <http://ant.apache.org/ivy/> http://ant.apache.org/ivy/ ::
> [ivy:configure] :: loading settings :: file > D:\svn\nextgenmapreduce\mapreduce\ivy\ivysettings.xml
>
> ivy-resolve-common:
> [ivy:resolve]
> [ivy:resolve] :: problems summary ::
> [ivy:resolve] :::: WARNINGS
> [ivy:resolve]                 module not found:
> org.apache.hadoop#yarn-server-common;1.0-SNAPSHOT
> [ivy:resolve]         ==== apache-snapshot: tried
> [ivy:resolve]
> <
> https://repository.apache.org/content/repositories/snapshots/org/apache/had
> oop/yarn-server-common/1.0-SNAPSHOT/yarn-server-common-1.0-SNAPSHOT.pom>
>
> https://repository.apache.org/content/repositories/snapshots/org/apache/hado
> op/yarn-server-common/1.0-SNAPSHOT/yarn-server-common-1.0-SNAPSHOT.pom
> [ivy:resolve]           -- artifact
> org.apache.hadoop#yarn-server-common;1.0-SNAPSHOT!yarn-server-common.jar:
> [ivy:resolve]
> <
> https://repository.apache.org/content/repositories/snapshots/org/apache/had
> oop/yarn-server-common/1.0-SNAPSHOT/yarn-server-common-1.0-SNAPSHOT.jar>
>
> https://repository.apache.org/content/repositories/snapshots/org/apache/hado
> op/yarn-server-common/1.0-SNAPSHOT/yarn-server-common-1.0-SNAPSHOT.jar
> [ivy:resolve]         ==== maven2: tried
> [ivy:resolve]
> <
> http://repo1.maven.org/maven2/org/apache/hadoop/yarn-server-common/1.0-SNAP
> SHOT/yarn-server-common-1.0-SNAPSHOT.pom>
>
> http://repo1.maven.org/maven2/org/apache/hadoop/yarn-server-common/1.0-SNAPS
> HOT/yarn-server-common-1.0-SNAPSHOT.pom
> [ivy:resolve]           -- artifact
> org.apache.hadoop#yarn-server-common;1.0-SNAPSHOT!yarn-server-common.jar:
> [ivy:resolve]
> <
> http://repo1.maven.org/maven2/org/apache/hadoop/yarn-server-common/1.0-SNAP
> SHOT/yarn-server-common-1.0-SNAPSHOT.jar>
>
> http://repo1.maven.org/maven2/org/apache/hadoop/yarn-server-common/1.0-SNAPS
> HOT/yarn-server-common-1.0-SNAPSHOT.jar
> [ivy:resolve]                 module not found:
> org.apache.hadoop#hadoop-mapreduce-client-core;1.0-SNAPSHOT
> [ivy:resolve]         ==== apache-snapshot: tried
> [ivy:resolve]
> <
> https://repository.apache.org/content/repositories/snapshots/org/apache/had
>
> oop/hadoop-mapreduce-client-core/1.0-SNAPSHOT/hadoop-mapreduce-client-core-1
> .0-SNAPSHOT.pom>
>
> https://repository.apache.org/content/repositories/snapshots/org/apache/hado
>
> op/hadoop-mapreduce-client-core/1.0-SNAPSHOT/hadoop-mapreduce-client-core-1.
> 0-SNAPSHOT.pom
> [ivy:resolve]           -- artifact
>
> org.apache.hadoop#hadoop-mapreduce-client-core;1.0-SNAPSHOT!hadoop-mapreduce
> -client-core.jar:
> [ivy:resolve]
> <
> https://repository.apache.org/content/repositories/snapshots/org/apache/had
>
> oop/hadoop-mapreduce-client-core/1.0-SNAPSHOT/hadoop-mapreduce-client-core-1
> .0-SNAPSHOT.jar>
>
> https://repository.apache.org/content/repositories/snapshots/org/apache/hado
>
> op/hadoop-mapreduce-client-core/1.0-SNAPSHOT/hadoop-mapreduce-client-core-1.
> 0-SNAPSHOT.jar
> [ivy:resolve]         ==== maven2: tried
> [ivy:resolve]
> <
> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-mapreduce-client-cor
> e/1.0-SNAPSHOT/hadoop-mapreduce-client-core-1.0-SNAPSHOT.pom>
>
> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-mapreduce-client-core

I mapreduce/INSTALL also has some important information in it, and be aware that you do not have to install the avro plugin any more.  Maven can download it and install it automatically now, but the README was never updated.  Also be sure to install protocol buffers.  The build will fail without it.

--Bobby

On 7/8/11 9:04 AM, "Josh Wills" <[EMAIL PROTECTED]> wrote:

You want to generate them using mvn instead.  See the mapreduce/yarn/README
file for how to do it.

On Fri, Jul 8, 2011 at 7:00 AM, Devaraj K <[EMAIL PROTECTED]> wrote:

> Hi,
>
>
>
>   I am getting this below errors when I try to generate eclipse files using
> eclipse-files target. Can anybody help me?
>
>
>
>
>
> Buildfile: D:\svn\nextgenmapreduce\mapreduce\build.xml
>
> ivy-download:
>      [get] Getting:
> <http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.2.0/ivy-2.2.0.jar>
> http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.2.0/ivy-2.2.0.jar
>      [get] To: D:\svn\nextgenmapreduce\mapreduce\ivy\ivy-2.2.0.jar
>      [get] Not modified - so not downloaded
>
> ivy-init-dirs:
>
> ivy-probe-antlib:
>
> ivy-init-antlib:
>
> ivy-init:
> [ivy:configure] :: Ivy non official version -  ::
> <http://ant.apache.org/ivy/> http://ant.apache.org/ivy/ ::
> [ivy:configure] :: loading settings :: file > D:\svn\nextgenmapreduce\mapreduce\ivy\ivysettings.xml
>
> ivy-resolve-common:
> [ivy:resolve]
> [ivy:resolve] :: problems summary ::
> [ivy:resolve] :::: WARNINGS
> [ivy:resolve]                 module not found:
> org.apache.hadoop#yarn-server-common;1.0-SNAPSHOT
> [ivy:resolve]         ==== apache-snapshot: tried
> [ivy:resolve]
> <
> https://repository.apache.org/content/repositories/snapshots/org/apache/had
> oop/yarn-server-common/1.0-SNAPSHOT/yarn-server-common-1.0-SNAPSHOT.pom>
>
> https://repository.apache.org/content/repositories/snapshots/org/apache/hado
> op/yarn-server-common/1.0-SNAPSHOT/yarn-server-common-1.0-SNAPSHOT.pom
> [ivy:resolve]           -- artifact
> org.apache.hadoop#yarn-server-common;1.0-SNAPSHOT!yarn-server-common.jar:
> [ivy:resolve]
> <
> https://repository.apache.org/content/repositories/snapshots/org/apache/had
> oop/yarn-server-common/1.0-SNAPSHOT/yarn-server-common-1.0-SNAPSHOT.jar>
>
> https://repository.apache.org/content/repositories/snapshots/org/apache/hado
> op/yarn-server-common/1.0-SNAPSHOT/yarn-server-common-1.0-SNAPSHOT.jar
> [ivy:resolve]         ==== maven2: tried
> [ivy:resolve]
> <
> http://repo1.maven.org/maven2/org/apache/hadoop/yarn-server-common/1.0-SNAP
> SHOT/yarn-server-common-1.0-SNAPSHOT.pom>
>
> http://repo1.maven.org/maven2/org/apache/hadoop/yarn-server-common/1.0-SNAPS
> HOT/yarn-server-common-1.0-SNAPSHOT.pom
> [ivy:resolve]           -- artifact
> org.apache.hadoop#yarn-server-common;1.0-SNAPSHOT!yarn-server-common.jar:
> [ivy:resolve]
> <
> http://repo1.maven.org/maven2/org/apache/hadoop/yarn-server-common/1.0-SNAP
> SHOT/yarn-server-common-1.0-SNAPSHOT.jar>
>
> http://repo1.maven.org/maven2/org/apache/hadoop/yarn-server-common/1.0-SNAPS
> HOT/yarn-server-common-1.0-SNAPSHOT.jar
> [ivy:resolve]                 module not found:
> org.apache.hadoop#hadoop-mapreduce-client-core;1.0-SNAPSHOT
> [ivy:resolve]         ==== apache-snapshot: tried
> [ivy:resolve]
> <
> https://repository.apache.org/content/repositories/snapshots/org/apache/had
>
> oop/hadoop-mapreduce-client-core/1.0-SNAPSHOT/hadoop-mapreduce-client-core-1
> .0-SNAPSHOT.pom>
>
> https://repository.apache.org/content/repositories/snapshots/org/apache/hado
>
> op/hadoop-mapreduce-client-core/1.0-SNAPSHOT/hadoop-mapreduce-client-core-1.
> 0-SNAPSHOT.pom
> [ivy:resolve]           -- artifact
>
> org.apache.hadoop#hadoop-mapreduce-client-core;1.0-SNAPSHOT!hadoop-mapreduce
> -client-core.jar:
> [ivy:resolve]
> <
> https://repository.apache.org/content/repositories/snapshots/org/apache/had
>
> oop/hadoop-mapreduce-client-core/1.0-SNAPSHOT/hadoop-mapreduce-client-core-1
> .0-SNAPSHOT.jar>
>
> https://repository.apache.org/content/repositories/snapshots/org/apache/hado

Hey Bobby-

Vinod and I did a cleanup pass over INSTALL and yarn/README in
https://issues.apache.org/jira/browse/MAPREDUCE-2645. Please take a look and
check if there's anything else we need to add/update.

Josh

On Fri, Jul 8, 2011 at 7:25 AM, Robert Evans <[EMAIL PROTECTED]> wrote:

> I mapreduce/INSTALL also has some important information in it, and be aware
> that you do not have to install the avro plugin any more.  Maven can
> download it and install it automatically now, but the README was never
> updated.  Also be sure to install protocol buffers.  The build will fail
> without it.
>
> --Bobby
>
> On 7/8/11 9:04 AM, "Josh Wills" <[EMAIL PROTECTED]> wrote:
>
> You want to generate them using mvn instead.  See the mapreduce/yarn/README
> file for how to do it.
>
> On Fri, Jul 8, 2011 at 7:00 AM, Devaraj K <[EMAIL PROTECTED]> wrote:
>
> > Hi,
> >
> >
> >
> >   I am getting this below errors when I try to generate eclipse files
> using
> > eclipse-files target. Can anybody help me?
> >
> >
> >
> >
> >
> > Buildfile: D:\svn\nextgenmapreduce\mapreduce\build.xml
> >
> > ivy-download:
> >      [get] Getting:
> > <http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.2.0/ivy-2.2.0.jar>
> > http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.2.0/ivy-2.2.0.jar
> >      [get] To: D:\svn\nextgenmapreduce\mapreduce\ivy\ivy-2.2.0.jar
> >      [get] Not modified - so not downloaded
> >
> > ivy-init-dirs:
> >
> > ivy-probe-antlib:
> >
> > ivy-init-antlib:
> >
> > ivy-init:
> > [ivy:configure] :: Ivy non official version -  ::
> > <http://ant.apache.org/ivy/> http://ant.apache.org/ivy/ ::
> > [ivy:configure] :: loading settings :: file > > D:\svn\nextgenmapreduce\mapreduce\ivy\ivysettings.xml
> >
> > ivy-resolve-common:
> > [ivy:resolve]
> > [ivy:resolve] :: problems summary ::
> > [ivy:resolve] :::: WARNINGS
> > [ivy:resolve]                 module not found:
> > org.apache.hadoop#yarn-server-common;1.0-SNAPSHOT
> > [ivy:resolve]         ==== apache-snapshot: tried
> > [ivy:resolve]
> > <
> >
> https://repository.apache.org/content/repositories/snapshots/org/apache/had
> > oop/yarn-server-common/1.0-SNAPSHOT/yarn-server-common-1.0-SNAPSHOT.pom>
> >
> >
> https://repository.apache.org/content/repositories/snapshots/org/apache/hado
> > op/yarn-server-common/1.0-SNAPSHOT/yarn-server-common-1.0-SNAPSHOT.pom
> > [ivy:resolve]           -- artifact
> > org.apache.hadoop#yarn-server-common;1.0-SNAPSHOT!yarn-server-common.jar:
> > [ivy:resolve]
> > <
> >
> https://repository.apache.org/content/repositories/snapshots/org/apache/had
> > oop/yarn-server-common/1.0-SNAPSHOT/yarn-server-common-1.0-SNAPSHOT.jar>
> >
> >
> https://repository.apache.org/content/repositories/snapshots/org/apache/hado
> > op/yarn-server-common/1.0-SNAPSHOT/yarn-server-common-1.0-SNAPSHOT.jar
> > [ivy:resolve]         ==== maven2: tried
> > [ivy:resolve]
> > <
> >
> http://repo1.maven.org/maven2/org/apache/hadoop/yarn-server-common/1.0-SNAP
> > SHOT/yarn-server-common-1.0-SNAPSHOT.pom>
> >
> >
> http://repo1.maven.org/maven2/org/apache/hadoop/yarn-server-common/1.0-SNAPS
> > HOT/yarn-server-common-1.0-SNAPSHOT.pom
> > [ivy:resolve]           -- artifact
> > org.apache.hadoop#yarn-server-common;1.0-SNAPSHOT!yarn-server-common.jar:
> > [ivy:resolve]
> > <
> >
> http://repo1.maven.org/maven2/org/apache/hadoop/yarn-server-common/1.0-SNAP
> > SHOT/yarn-server-common-1.0-SNAPSHOT.jar>
> >
> >
> http://repo1.maven.org/maven2/org/apache/hadoop/yarn-server-common/1.0-SNAPS
> > HOT/yarn-server-common-1.0-SNAPSHOT.jar
> > [ivy:resolve]                 module not found:
> > org.apache.hadoop#hadoop-mapreduce-client-core;1.0-SNAPSHOT
> > [ivy:resolve]         ==== apache-snapshot: tried
> > [ivy:resolve]
> > <
> >
> https://repository.apache.org/content/repositories/snapshots/org/apache/had
> >
> >
> oop/hadoop-mapreduce-client-core/1.0-SNAPSHOT/hadoop-mapreduce-client-core-1
> > .0-SNAPSHOT.pom>
> >
> >
> https://repository.apache.org/content/repositories/snapshots/org/apache/hado

Hi Josh,

Which configuration file could tell Yarn the location of HDFS? This could be
found on
http://svn.apache.org/repos/asf/hadoop/common/branches/MR-279/mapreduce/INSTALL

Thank you.

Best,

Haoyuan

On Fri, Jul 8, 2011 at 10:58 PM, Josh Wills <[EMAIL PROTECTED]> wrote:

> Hey Bobby-
>
> Vinod and I did a cleanup pass over INSTALL and yarn/README in
> https://issues.apache.org/jira/browse/MAPREDUCE-2645. Please take a look
> and
> check if there's anything else we need to add/update.
>
> Josh
>
> On Fri, Jul 8, 2011 at 7:25 AM, Robert Evans <[EMAIL PROTECTED]> wrote:
>
> > I mapreduce/INSTALL also has some important information in it, and be
> aware
> > that you do not have to install the avro plugin any more.  Maven can
> > download it and install it automatically now, but the README was never
> > updated.  Also be sure to install protocol buffers.  The build will fail
> > without it.
> >
> > --Bobby
> >
> > On 7/8/11 9:04 AM, "Josh Wills" <[EMAIL PROTECTED]> wrote:
> >
> > You want to generate them using mvn instead.  See the
> mapreduce/yarn/README
> > file for how to do it.
> >
> > On Fri, Jul 8, 2011 at 7:00 AM, Devaraj K <[EMAIL PROTECTED]> wrote:
> >
> > > Hi,
> > >
> > >
> > >
> > >   I am getting this below errors when I try to generate eclipse files
> > using
> > > eclipse-files target. Can anybody help me?
> > >
> > >
> > >
> > >
> > >
> > > Buildfile: D:\svn\nextgenmapreduce\mapreduce\build.xml
> > >
> > > ivy-download:
> > >      [get] Getting:
> > > <http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.2.0/ivy-2.2.0.jar>
> > > http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.2.0/ivy-2.2.0.jar
> > >      [get] To: D:\svn\nextgenmapreduce\mapreduce\ivy\ivy-2.2.0.jar
> > >      [get] Not modified - so not downloaded
> > >
> > > ivy-init-dirs:
> > >
> > > ivy-probe-antlib:
> > >
> > > ivy-init-antlib:
> > >
> > > ivy-init:
> > > [ivy:configure] :: Ivy non official version -  ::
> > > <http://ant.apache.org/ivy/> http://ant.apache.org/ivy/ ::
> > > [ivy:configure] :: loading settings :: file > > > D:\svn\nextgenmapreduce\mapreduce\ivy\ivysettings.xml
> > >
> > > ivy-resolve-common:
> > > [ivy:resolve]
> > > [ivy:resolve] :: problems summary ::
> > > [ivy:resolve] :::: WARNINGS
> > > [ivy:resolve]                 module not found:
> > > org.apache.hadoop#yarn-server-common;1.0-SNAPSHOT
> > > [ivy:resolve]         ==== apache-snapshot: tried
> > > [ivy:resolve]
> > > <
> > >
> >
> https://repository.apache.org/content/repositories/snapshots/org/apache/had
> > >
> oop/yarn-server-common/1.0-SNAPSHOT/yarn-server-common-1.0-SNAPSHOT.pom>
> > >
> > >
> >
> https://repository.apache.org/content/repositories/snapshots/org/apache/hado
> > > op/yarn-server-common/1.0-SNAPSHOT/yarn-server-common-1.0-SNAPSHOT.pom
> > > [ivy:resolve]           -- artifact
> > >
> org.apache.hadoop#yarn-server-common;1.0-SNAPSHOT!yarn-server-common.jar:
> > > [ivy:resolve]
> > > <
> > >
> >
> https://repository.apache.org/content/repositories/snapshots/org/apache/had
> > >
> oop/yarn-server-common/1.0-SNAPSHOT/yarn-server-common-1.0-SNAPSHOT.jar>
> > >
> > >
> >
> https://repository.apache.org/content/repositories/snapshots/org/apache/hado
> > > op/yarn-server-common/1.0-SNAPSHOT/yarn-server-common-1.0-SNAPSHOT.jar
> > > [ivy:resolve]         ==== maven2: tried
> > > [ivy:resolve]
> > > <
> > >
> >
> http://repo1.maven.org/maven2/org/apache/hadoop/yarn-server-common/1.0-SNAP
> > > SHOT/yarn-server-common-1.0-SNAPSHOT.pom>
> > >
> > >
> >
> http://repo1.maven.org/maven2/org/apache/hadoop/yarn-server-common/1.0-SNAPS
> > > HOT/yarn-server-common-1.0-SNAPSHOT.pom
> > > [ivy:resolve]           -- artifact
> > >
> org.apache.hadoop#yarn-server-common;1.0-SNAPSHOT!yarn-server-common.jar:
> > > [ivy:resolve]
> > > <
> > >
> >
> http://repo1.maven.org/maven2/org/apache/hadoop/yarn-server-common/1.0-SNAP
> > > SHOT/yarn-server-common-1.0-SNAPSHOT.jar>
> > >
> > >
> >
> http://repo1.maven.org/maven2/org/apache/hadoop/yarn-server-common/1.0-SNAPS

Haoyuan,

Did you specify HADOOP_CONF_DIR, YARN_CONF_DIR, etc. as per Step 8 of the
INSTALL doc? I've been running my mrv2 cluster in pseudo-distributed mode on
my linux box- happy to send you my configs if you'd like to play. There are
also a couple of changes you need to make to the yarn config files to get
real MR running, as Bobby alluded to in his email.

To run the example in the INSTALL file, you'll need to patch in this change:
https://issues.apache.org/jira/browse/MAPREDUCE-2644

Josh

On Fri, Jul 8, 2011 at 8:07 AM, Haoyuan Li <[EMAIL PROTECTED]> wrote:

> Hi Josh,
>
> Which configuration file could tell Yarn the location of HDFS? This could
> be
> found on
>
> http://svn.apache.org/repos/asf/hadoop/common/branches/MR-279/mapreduce/INSTALL
>
> Thank you.
>
> Best,
>
> Haoyuan
>
> On Fri, Jul 8, 2011 at 10:58 PM, Josh Wills <[EMAIL PROTECTED]> wrote:
>
> > Hey Bobby-
> >
> > Vinod and I did a cleanup pass over INSTALL and yarn/README in
> > https://issues.apache.org/jira/browse/MAPREDUCE-2645. Please take a look
> > and
> > check if there's anything else we need to add/update.
> >
> > Josh
> >
> > On Fri, Jul 8, 2011 at 7:25 AM, Robert Evans <[EMAIL PROTECTED]>
> wrote:
> >
> > > I mapreduce/INSTALL also has some important information in it, and be
> > aware
> > > that you do not have to install the avro plugin any more.  Maven can
> > > download it and install it automatically now, but the README was never
> > > updated.  Also be sure to install protocol buffers.  The build will
> fail
> > > without it.
> > >
> > > --Bobby
> > >
> > > On 7/8/11 9:04 AM, "Josh Wills" <[EMAIL PROTECTED]> wrote:
> > >
> > > You want to generate them using mvn instead.  See the
> > mapreduce/yarn/README
> > > file for how to do it.
> > >
> > > On Fri, Jul 8, 2011 at 7:00 AM, Devaraj K <[EMAIL PROTECTED]>
> wrote:
> > >
> > > > Hi,
> > > >
> > > >
> > > >
> > > >   I am getting this below errors when I try to generate eclipse files
> > > using
> > > > eclipse-files target. Can anybody help me?
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > Buildfile: D:\svn\nextgenmapreduce\mapreduce\build.xml
> > > >
> > > > ivy-download:
> > > >      [get] Getting:
> > > > <
> http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.2.0/ivy-2.2.0.jar>
> > > > http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.2.0/ivy-2.2.0.jar
> > > >      [get] To: D:\svn\nextgenmapreduce\mapreduce\ivy\ivy-2.2.0.jar
> > > >      [get] Not modified - so not downloaded
> > > >
> > > > ivy-init-dirs:
> > > >
> > > > ivy-probe-antlib:
> > > >
> > > > ivy-init-antlib:
> > > >
> > > > ivy-init:
> > > > [ivy:configure] :: Ivy non official version -  ::
> > > > <http://ant.apache.org/ivy/> http://ant.apache.org/ivy/ ::
> > > > [ivy:configure] :: loading settings :: file > > > > D:\svn\nextgenmapreduce\mapreduce\ivy\ivysettings.xml
> > > >
> > > > ivy-resolve-common:
> > > > [ivy:resolve]
> > > > [ivy:resolve] :: problems summary ::
> > > > [ivy:resolve] :::: WARNINGS
> > > > [ivy:resolve]                 module not found:
> > > > org.apache.hadoop#yarn-server-common;1.0-SNAPSHOT
> > > > [ivy:resolve]         ==== apache-snapshot: tried
> > > > [ivy:resolve]
> > > > <
> > > >
> > >
> >
> https://repository.apache.org/content/repositories/snapshots/org/apache/had
> > > >
> > oop/yarn-server-common/1.0-SNAPSHOT/yarn-server-common-1.0-SNAPSHOT.pom>
> > > >
> > > >
> > >
> >
> https://repository.apache.org/content/repositories/snapshots/org/apache/hado
> > > >
> op/yarn-server-common/1.0-SNAPSHOT/yarn-server-common-1.0-SNAPSHOT.pom
> > > > [ivy:resolve]           -- artifact
> > > >
> > org.apache.hadoop#yarn-server-common;1.0-SNAPSHOT!yarn-server-common.jar:
> > > > [ivy:resolve]
> > > > <
> > > >
> > >
> >
> https://repository.apache.org/content/repositories/snapshots/org/apache/had
> > > >
> > oop/yarn-server-common/1.0-SNAPSHOT/yarn-server-common-1.0-SNAPSHOT.jar>
> > > >
> > > >
> > >
> >
> https://repository.apache.org/content/repositories/snapshots/org/apache/hado
> > > >

Thank you Josh. It will be great if you could send me your configs for
pseudo-distributed mode.

Best,

Haoyuan

On Sat, Jul 9, 2011 at 12:25 AM, Josh Wills <[EMAIL PROTECTED]> wrote:

> Haoyuan,
>
> Did you specify HADOOP_CONF_DIR, YARN_CONF_DIR, etc. as per Step 8 of the
> INSTALL doc? I've been running my mrv2 cluster in pseudo-distributed mode
> on
> my linux box- happy to send you my configs if you'd like to play. There are
> also a couple of changes you need to make to the yarn config files to get
> real MR running, as Bobby alluded to in his email.
>
> To run the example in the INSTALL file, you'll need to patch in this
> change:
> https://issues.apache.org/jira/browse/MAPREDUCE-2644
>
> Josh
>
> On Fri, Jul 8, 2011 at 8:07 AM, Haoyuan Li <[EMAIL PROTECTED]> wrote:
>
> > Hi Josh,
> >
> > Which configuration file could tell Yarn the location of HDFS? This could
> > be
> > found on
> >
> >
> http://svn.apache.org/repos/asf/hadoop/common/branches/MR-279/mapreduce/INSTALL
> >
> > Thank you.
> >
> > Best,
> >
> > Haoyuan
> >
> > On Fri, Jul 8, 2011 at 10:58 PM, Josh Wills <[EMAIL PROTECTED]> wrote:
> >
> > > Hey Bobby-
> > >
> > > Vinod and I did a cleanup pass over INSTALL and yarn/README in
> > > https://issues.apache.org/jira/browse/MAPREDUCE-2645. Please take a
> look
> > > and
> > > check if there's anything else we need to add/update.
> > >
> > > Josh
> > >
> > > On Fri, Jul 8, 2011 at 7:25 AM, Robert Evans <[EMAIL PROTECTED]>
> > wrote:
> > >
> > > > I mapreduce/INSTALL also has some important information in it, and be
> > > aware
> > > > that you do not have to install the avro plugin any more.  Maven can
> > > > download it and install it automatically now, but the README was
> never
> > > > updated.  Also be sure to install protocol buffers.  The build will
> > fail
> > > > without it.
> > > >
> > > > --Bobby
> > > >
> > > > On 7/8/11 9:04 AM, "Josh Wills" <[EMAIL PROTECTED]> wrote:
> > > >
> > > > You want to generate them using mvn instead.  See the
> > > mapreduce/yarn/README
> > > > file for how to do it.
> > > >
> > > > On Fri, Jul 8, 2011 at 7:00 AM, Devaraj K <[EMAIL PROTECTED]>
> > wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > >
> > > > >
> > > > >   I am getting this below errors when I try to generate eclipse
> files
> > > > using
> > > > > eclipse-files target. Can anybody help me?
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > Buildfile: D:\svn\nextgenmapreduce\mapreduce\build.xml
> > > > >
> > > > > ivy-download:
> > > > >      [get] Getting:
> > > > > <
> > http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.2.0/ivy-2.2.0.jar>
> > > > >
> http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.2.0/ivy-2.2.0.jar
> > > > >      [get] To: D:\svn\nextgenmapreduce\mapreduce\ivy\ivy-2.2.0.jar
> > > > >      [get] Not modified - so not downloaded
> > > > >
> > > > > ivy-init-dirs:
> > > > >
> > > > > ivy-probe-antlib:
> > > > >
> > > > > ivy-init-antlib:
> > > > >
> > > > > ivy-init:
> > > > > [ivy:configure] :: Ivy non official version -  ::
> > > > > <http://ant.apache.org/ivy/> http://ant.apache.org/ivy/ ::
> > > > > [ivy:configure] :: loading settings :: file > > > > > D:\svn\nextgenmapreduce\mapreduce\ivy\ivysettings.xml
> > > > >
> > > > > ivy-resolve-common:
> > > > > [ivy:resolve]
> > > > > [ivy:resolve] :: problems summary ::
> > > > > [ivy:resolve] :::: WARNINGS
> > > > > [ivy:resolve]                 module not found:
> > > > > org.apache.hadoop#yarn-server-common;1.0-SNAPSHOT
> > > > > [ivy:resolve]         ==== apache-snapshot: tried
> > > > > [ivy:resolve]
> > > > > <
> > > > >
> > > >
> > >
> >
> https://repository.apache.org/content/repositories/snapshots/org/apache/had
> > > > >
> > >
> oop/yarn-server-common/1.0-SNAPSHOT/yarn-server-common-1.0-SNAPSHOT.pom>
> > > > >
> > > > >
> > > >
> > >
> >
> https://repository.apache.org/content/repositories/snapshots/org/apache/hado
> > > > >
> > op/yarn-server-common/1.0-SNAPSHOT/yarn-server-common-1.0-SNAPSHOT.pom
> > > > > [ivy:resolve]           -- artifact

Hi Josh,

Which configuration file could tell Yarn the location of HDFS? This could
not be found on
http://svn.apache.org/repos/asf/hadoop/common/branches/MR-279/mapreduce/INSTALL

Thank you.

Best,

Haoyuan

On Fri, Jul 8, 2011 at 10:58 PM, Josh Wills <[EMAIL PROTECTED]> wrote:

> Hey Bobby-
>
> Vinod and I did a cleanup pass over INSTALL and yarn/README in
> https://issues.apache.org/jira/browse/MAPREDUCE-2645. Please take a look
> and
> check if there's anything else we need to add/update.
>
> Josh
>
> On Fri, Jul 8, 2011 at 7:25 AM, Robert Evans <[EMAIL PROTECTED]> wrote:
>
> > I mapreduce/INSTALL also has some important information in it, and be
> aware
> > that you do not have to install the avro plugin any more.  Maven can
> > download it and install it automatically now, but the README was never
> > updated.  Also be sure to install protocol buffers.  The build will fail
> > without it.
> >
> > --Bobby
> >
> > On 7/8/11 9:04 AM, "Josh Wills" <[EMAIL PROTECTED]> wrote:
> >
> > You want to generate them using mvn instead.  See the
> mapreduce/yarn/README
> > file for how to do it.
> >
> > On Fri, Jul 8, 2011 at 7:00 AM, Devaraj K <[EMAIL PROTECTED]> wrote:
> >
> > > Hi,
> > >
> > >
> > >
> > >   I am getting this below errors when I try to generate eclipse files
> > using
> > > eclipse-files target. Can anybody help me?
> > >
> > >
> > >
> > >
> > >
> > > Buildfile: D:\svn\nextgenmapreduce\mapreduce\build.xml
> > >
> > > ivy-download:
> > >      [get] Getting:
> > > <http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.2.0/ivy-2.2.0.jar>
> > > http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.2.0/ivy-2.2.0.jar
> > >      [get] To: D:\svn\nextgenmapreduce\mapreduce\ivy\ivy-2.2.0.jar
> > >      [get] Not modified - so not downloaded
> > >
> > > ivy-init-dirs:
> > >
> > > ivy-probe-antlib:
> > >
> > > ivy-init-antlib:
> > >
> > > ivy-init:
> > > [ivy:configure] :: Ivy non official version -  ::
> > > <http://ant.apache.org/ivy/> http://ant.apache.org/ivy/ ::
> > > [ivy:configure] :: loading settings :: file > > > D:\svn\nextgenmapreduce\mapreduce\ivy\ivysettings.xml
> > >
> > > ivy-resolve-common:
> > > [ivy:resolve]
> > > [ivy:resolve] :: problems summary ::
> > > [ivy:resolve] :::: WARNINGS
> > > [ivy:resolve]                 module not found:
> > > org.apache.hadoop#yarn-server-common;1.0-SNAPSHOT
> > > [ivy:resolve]         ==== apache-snapshot: tried
> > > [ivy:resolve]
> > > <
> > >
> >
> https://repository.apache.org/content/repositories/snapshots/org/apache/had
> > >
> oop/yarn-server-common/1.0-SNAPSHOT/yarn-server-common-1.0-SNAPSHOT.pom>
> > >
> > >
> >
> https://repository.apache.org/content/repositories/snapshots/org/apache/hado
> > > op/yarn-server-common/1.0-SNAPSHOT/yarn-server-common-1.0-SNAPSHOT.pom
> > > [ivy:resolve]           -- artifact
> > >
> org.apache.hadoop#yarn-server-common;1.0-SNAPSHOT!yarn-server-common.jar:
> > > [ivy:resolve]
> > > <
> > >
> >
> https://repository.apache.org/content/repositories/snapshots/org/apache/had
> > >
> oop/yarn-server-common/1.0-SNAPSHOT/yarn-server-common-1.0-SNAPSHOT.jar>
> > >
> > >
> >
> https://repository.apache.org/content/repositories/snapshots/org/apache/hado
> > > op/yarn-server-common/1.0-SNAPSHOT/yarn-server-common-1.0-SNAPSHOT.jar
> > > [ivy:resolve]         ==== maven2: tried
> > > [ivy:resolve]
> > > <
> > >
> >
> http://repo1.maven.org/maven2/org/apache/hadoop/yarn-server-common/1.0-SNAP
> > > SHOT/yarn-server-common-1.0-SNAPSHOT.pom>
> > >
> > >
> >
> http://repo1.maven.org/maven2/org/apache/hadoop/yarn-server-common/1.0-SNAPS
> > > HOT/yarn-server-common-1.0-SNAPSHOT.pom
> > > [ivy:resolve]           -- artifact
> > >
> org.apache.hadoop#yarn-server-common;1.0-SNAPSHOT!yarn-server-common.jar:
> > > [ivy:resolve]
> > > <
> > >
> >
> http://repo1.maven.org/maven2/org/apache/hadoop/yarn-server-common/1.0-SNAP
> > > SHOT/yarn-server-common-1.0-SNAPSHOT.jar>
> > >
> > >
> >
> http://repo1.maven.org/maven2/org/apache/hadoop/yarn-server-common/1.0-SNAPS

Hey Josh,

I just updated now, and did not realize that it had been fixed.  Everything mostly looks good.

Then name of the tar ball is now -all.tar.gz not -bin.tar.gz

It also doesn't talk about how you need to install the MAPRED, COMMON, and HDFS tar balls as well to get things to work, although it is implied in the exports.

The biggest issue I have is that the default configs are not good enough to actually make map/reduce work with the Resource Manager.  I realize that a lot of this is likely going to be fixed before mrv2 is released, but as it stands you can only run in local mode with the INSTALL and README files which fall back on the MRV1 code to do it.  If that is the intent of the documentation then it is fine, but it would be nice some where, even if it is just on twiki, to indicate what configs are required to setup a single node cluster.

If you want the configs I have been using I am happy to send it to you.

--Bobby Evans
On 7/8/11 9:58 AM, "Josh Wills" <[EMAIL PROTECTED]> wrote:

Hey Bobby-

Vinod and I did a cleanup pass over INSTALL and yarn/README in
https://issues.apache.org/jira/browse/MAPREDUCE-2645. Please take a look and
check if there's anything else we need to add/update.

Josh

On Fri, Jul 8, 2011 at 7:25 AM, Robert Evans <[EMAIL PROTECTED]> wrote:

> I mapreduce/INSTALL also has some important information in it, and be aware
> that you do not have to install the avro plugin any more.  Maven can
> download it and install it automatically now, but the README was never
> updated.  Also be sure to install protocol buffers.  The build will fail
> without it.
>
> --Bobby
>
> On 7/8/11 9:04 AM, "Josh Wills" <[EMAIL PROTECTED]> wrote:
>
> You want to generate them using mvn instead.  See the mapreduce/yarn/README
> file for how to do it.
>
> On Fri, Jul 8, 2011 at 7:00 AM, Devaraj K <[EMAIL PROTECTED]> wrote:
>
> > Hi,
> >
> >
> >
> >   I am getting this below errors when I try to generate eclipse files
> using
> > eclipse-files target. Can anybody help me?
> >
> >
> >
> >
> >
> > Buildfile: D:\svn\nextgenmapreduce\mapreduce\build.xml
> >
> > ivy-download:
> >      [get] Getting:
> > <http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.2.0/ivy-2.2.0.jar>
> > http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.2.0/ivy-2.2.0.jar
> >      [get] To: D:\svn\nextgenmapreduce\mapreduce\ivy\ivy-2.2.0.jar
> >      [get] Not modified - so not downloaded
> >
> > ivy-init-dirs:
> >
> > ivy-probe-antlib:
> >
> > ivy-init-antlib:
> >
> > ivy-init:
> > [ivy:configure] :: Ivy non official version -  ::
> > <http://ant.apache.org/ivy/> http://ant.apache.org/ivy/ ::
> > [ivy:configure] :: loading settings :: file > > D:\svn\nextgenmapreduce\mapreduce\ivy\ivysettings.xml
> >
> > ivy-resolve-common:
> > [ivy:resolve]
> > [ivy:resolve] :: problems summary ::
> > [ivy:resolve] :::: WARNINGS
> > [ivy:resolve]                 module not found:
> > org.apache.hadoop#yarn-server-common;1.0-SNAPSHOT
> > [ivy:resolve]         ==== apache-snapshot: tried
> > [ivy:resolve]
> > <
> >
> https://repository.apache.org/content/repositories/snapshots/org/apache/had
> > oop/yarn-server-common/1.0-SNAPSHOT/yarn-server-common-1.0-SNAPSHOT.pom>
> >
> >
> https://repository.apache.org/content/repositories/snapshots/org/apache/hado
> > op/yarn-server-common/1.0-SNAPSHOT/yarn-server-common-1.0-SNAPSHOT.pom
> > [ivy:resolve]           -- artifact
> > org.apache.hadoop#yarn-server-common;1.0-SNAPSHOT!yarn-server-common.jar:
> > [ivy:resolve]
> > <
> >
> https://repository.apache.org/content/repositories/snapshots/org/apache/had
> > oop/yarn-server-common/1.0-SNAPSHOT/yarn-server-common-1.0-SNAPSHOT.jar>
> >
> >
> https://repository.apache.org/content/repositories/snapshots/org/apache/hado
> > op/yarn-server-common/1.0-SNAPSHOT/yarn-server-common-1.0-SNAPSHOT.jar
> > [ivy:resolve]         ==== maven2: tried
> > [ivy:resolve]
> > <
> >
> http://repo1.maven.org/maven2/org/apache/hadoop/yarn-server-common/1.0-SNAP
> > SHOT/yarn-server-common-1.0-SNAPSHOT.pom>