Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce, mail # dev - [VOTE] Release Apache Hadoop 2.1.0-beta


Copy link to this message
-
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Steve Loughran 2013-07-04, 08:17
On 26 June 2013 09:17, Arun C Murthy <[EMAIL PROTECTED]> wrote:

> Folks,
>
> I've created a release candidate (rc0) for hadoop-2.1.0-beta that I would
> like to get released.
>
> This release represents a *huge* amount of work done by the community (639
> fixes) which includes several major advances including:
> # HDFS Snapshots
> # Windows support
> # YARN API stabilization
> # MapReduce Binary Compatibility with hadoop-1.x
> # Substantial amount of integration testing with rest of projects in the
> ecosystem
>
> The RC is available at:
> http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/
> The RC tag in svn is here:
> http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc0
>
> The maven artifacts are available via repository.apache.org.
>
> Please try the release and vote; the vote will run for the usual 7 days.
>
>
>
>
I'm going to +1 here because
 -we should get the beta out and get feedback on it
 -once it is in the maven repositories, it is effectively released: the
artifacts are in the wild. Even if something with the same version name is
pushed out, maven won't update client machines that have pulled it down:
new artifacts with new versions need to go out.

Now, those maven artifacts don't come with complete Poms, meaning I can't
use them in my builds. The Jars come down, but without the POM files the
build breaks
Downloading:
http://repo.maven.apache.org/maven2/org/apache/hadoop/hadoop-common/2.1.0-beta/hadoop-common-2.1.0-beta.pom
[WARNING] The POM for org.apache.hadoop:hadoop-common:jar:2.1.0-beta is
missing, no dependency information available
Downloading:
http://repo.maven.apache.org/maven2/org/apache/hadoop/hadoop-minicluster/2.1.0-beta/hadoop-minicluster-2.1.0-beta.pom
[WARNING] The POM for org.apache.hadoop:hadoop-minicluster:jar:2.1.0-beta
is missing, no dependency information available
Downloading:
http://repo.maven.apache.org/maven2/org/apache/hadoop/hadoop-hdfs/2.1.0-beta/hadoop-hdfs-2.1.0-beta.pom
[WARNING] The POM for org.apache.hadoop:hadoop-hdfs:jar:2.1.0-beta is
missing, no dependency information available
[WARNING] The POM for org.apache.hadoop:hadoop-common:jar:tests:2.1.0-beta
is missing, no dependency information available
Downloading:
http://repo.maven.apache.org/maven2/org/apache/hadoop/hadoop-yarn-server-common/2.1.0-beta/hadoop-yarn-server-common-2.1.0-beta.pom
[WARNING] The POM for
org.apache.hadoop:hadoop-yarn-server-common:jar:2.1.0-beta is missing, no
dependency information available
Downloading:
http://repo.maven.apache.org/maven2/org/apache/hadoop/hadoop-common/2.1.0-beta/hadoop-common-2.1.0-beta.jar
Downloading:
http://repo.maven.apache.org/maven2/org/apache/hadoop/hadoop-minicluster/2.1.0-beta/hadoop-minicluster-2.1.0-beta.jar
Downloading:
http://repo.maven.apache.org/maven2/org/apache/hadoop/hadoop-hdfs/2.1.0-beta/hadoop-hdfs-2.1.0-beta.jar
Downloading:
http://repo.maven.apache.org/maven2/org/apache/hadoop/hadoop-common/2.1.0-beta/hadoop-common-2.1.0-beta-tests.jar
Downloading:
http://repo.maven.apache.org/maven2/org/apache/hadoop/hadoop-yarn-server-common/2.1.0-beta/hadoop-yarn-server-common-2.1.0-beta.jar
[INFO]
------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO]
------------------------------------------------------------------------

As a result, I can't actually use it -I have to make my own 2.1.1-SNAPSHOT
release not because I want anything newer than the beta, but because those
missing POM files stop me coding against the beta.

I have to view this as a bug in the maven-based release process: somehow
the POM files didn't get published to the apache repos.
   1. We need to know what happened here so that the next beta can be used
   in downstream builds -and eventually, the final release. Because if those
   artifacts don't go up in a way that can be used in maven and ant/ivy
   builds, then I do consider it to be broken.
   2. That said, I'm not sure that artifacts go to the repo until vote
   time, precisely because it is impossible to revoke them once it's happened.
   If there is a way of doing a build of the release with the -SNAPSHOT tag,
   we could push it out to the apache snapshot repo and make sure it works
   there. That is, unless there is some different in -SNAPSHOT releases and
   non snapshot ones.

Arun,

   1. do you have the log of the bits of the build where the Maven
   artifacts were published?
   2. do you still have the POM files locally? If you do, we could try to
   publish them again.

Finally, who understands that step of the build that pushes the artifacts
to the apache release repo? Does it work for the 0.27.x and 0.20.20x
releases?