Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # dev >> [UPDATE 2] Upgrade to protobuf 2.5.0 for the 2.1.0 release, HADOOP-9845


Copy link to this message
-
[UPDATE 2] Upgrade to protobuf 2.5.0 for the 2.1.0 release, HADOOP-9845
I've filed https://issues.apache.org/jira/browse/HADOOP-9872 addressing the
following:

---------

   - handles protoc version correctly independently of the exit code
   - if HADOOP_PROTOC_PATH env var is defined, it uses it as the protoc
   executable * if HADOOP_PROTOC_PATH is not defined, it picks protoc from the
   PATH
   - documentation updated to reflect 2.5.0 is required
   - enforces the version of protoc and protobuf JAR are the same
   - Added to VersionInfo the protoc version used (sooner or later this
   will be useful for in a troubleshooting situation).

Luke Lu <https://issues.apache.org/jira/secure/ViewProfile.jspa?name=vicaya>
suggested
to make the version check for protoc lax (i.e. 2.5.*). While working on the
patch I've thought about that. But that would introduce a potential
mismatch between protoc and protobuff JAR.

Still If you want to use different version of protoc/protobuff from the one
defined in the POM, you can use the -Dprotobuf.version=#### to specify your
alternate version. But I would recommend not to do this, because if you
publish the artifacts to a Maven repo, the fact you used
-Dprotobuf.version=#### will be lost and the version defined in the POM
properties will be used (IMO Maven should use the effective POM on deploy,
but they don't).

---------

It would be great if a few people test the patch locally.

Once this is committed to trunk I'll bacport HADOOP-9845 & HADOOP-9872 to
all the 2 branches.

Thx.

On Tue, Aug 13, 2013 at 1:09 PM, Alejandro Abdelnur <[EMAIL PROTECTED]>wrote:

>
> There is no indication that protoc 2.5.0 is breaking anything.
>
> Hadoop-trunk builds have been failing way before 1/2 way with:
>
> -------
>
>
> [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.12.3:test (default-test) on project hadoop-yarn-client: ExecutionException; nested exception is java.util.concurrent.ExecutionException: java.lang.RuntimeException: The forked VM terminated without saying properly goodbye. VM crash or System.exit called ? -> [Help 1]
> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.12.3:test (default-test) on project hadoop-yarn-client: ExecutionException; nested exception is java.util.concurrent.ExecutionException: java.lang.RuntimeException: The forked VM terminated without saying properly goodbye. VM crash or System.exit called ?
>
> -------
>
>
> The Hadoop-trunk #480 build failed with a JVM abort in a testcase towards
> the end of mapreduce tests.
>
> Until then there were no failures at all.
>
> I've increased heap size and tried a second run and the failure was
> earlier.
>
> I've looked a Hadoop-trunk builds prior to the HADOOP-9845 and it has been
> failing the same way in all the kept builds.
>
> We need to fix Hadoop-trunk builds independently of this.
>
> Any objection to commit HADOOP-9845 to branch-2 and the 2.1.0-beta
> branches to get all the other jenkins jobs working?
>
> I'll wait till tomorrow morning before proceeding.
>
> Thx
>
>
>
>
> On Mon, Aug 12, 2013 at 8:35 PM, Alejandro Abdelnur <[EMAIL PROTECTED]>wrote:
>
>> Jenkins is running a full test run on trunk using protoc 2.5.0.
>>
>>   https://builds.apache.org/job/Hadoop-trunk/480
>>
>> And it seems go be going just fine.
>>
>> If everything looks OK, I'm planing to backport HADOOP-9845 to the
>> 2.1.0-beta branch midday PST tomorrow. This will normalize all builds
>> failures do the protoc mismatch.
>>
>> Thanks.
>>
>> Alejandro
>>
>>
>> On Mon, Aug 12, 2013 at 5:53 PM, Alejandro Abdelnur <[EMAIL PROTECTED]>wrote:
>>
>>> shooting to get it i n for 2.1.0.
>>>
>>> at moment is in trunk till the nightly finishes. then we'll decide
>>>
>>> in the mean time, you can have multiple versions installed in diff dirs
>>> and set the right one in the path
>>>
>>> thx
>>>
>>> Alejandro
>>> (phone typing)
>>>
>>> On Aug 12, 2013, at 17:47, Konstantin Shvachko <[EMAIL PROTECTED]>
>>> wrote:
>>>
>>> > Ok. After installing protobuf 2.5.0 I can compile trunk.

Alejandro
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB