Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # dev >> [UPDATE 2] Upgrade to protobuf 2.5.0 for the 2.1.0 release, HADOOP-9845


Copy link to this message
-
[UPDATE 2] Upgrade to protobuf 2.5.0 for the 2.1.0 release, HADOOP-9845
I've filed https://issues.apache.org/jira/browse/HADOOP-9872 addressing the
following:

---------

   - handles protoc version correctly independently of the exit code
   - if HADOOP_PROTOC_PATH env var is defined, it uses it as the protoc
   executable * if HADOOP_PROTOC_PATH is not defined, it picks protoc from the
   PATH
   - documentation updated to reflect 2.5.0 is required
   - enforces the version of protoc and protobuf JAR are the same
   - Added to VersionInfo the protoc version used (sooner or later this
   will be useful for in a troubleshooting situation).

Luke Lu <https://issues.apache.org/jira/secure/ViewProfile.jspa?name=vicaya>
suggested
to make the version check for protoc lax (i.e. 2.5.*). While working on the
patch I've thought about that. But that would introduce a potential
mismatch between protoc and protobuff JAR.

Still If you want to use different version of protoc/protobuff from the one
defined in the POM, you can use the -Dprotobuf.version=#### to specify your
alternate version. But I would recommend not to do this, because if you
publish the artifacts to a Maven repo, the fact you used
-Dprotobuf.version=#### will be lost and the version defined in the POM
properties will be used (IMO Maven should use the effective POM on deploy,
but they don't).

---------

It would be great if a few people test the patch locally.

Once this is committed to trunk I'll bacport HADOOP-9845 & HADOOP-9872 to
all the 2 branches.

Thx.

On Tue, Aug 13, 2013 at 1:09 PM, Alejandro Abdelnur <[EMAIL PROTECTED]>wrote:

>
> There is no indication that protoc 2.5.0 is breaking anything.
>
> Hadoop-trunk builds have been failing way before 1/2 way with:
>
> -------
>
>
> [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.12.3:test (default-test) on project hadoop-yarn-client: ExecutionException; nested exception is java.util.concurrent.ExecutionException: java.lang.RuntimeException: The forked VM terminated without saying properly goodbye. VM crash or System.exit called ? -> [Help 1]
> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.12.3:test (default-test) on project hadoop-yarn-client: ExecutionException; nested exception is java.util.concurrent.ExecutionException: java.lang.RuntimeException: The forked VM terminated without saying properly goodbye. VM crash or System.exit called ?
>
> -------
>
>
> The Hadoop-trunk #480 build failed with a JVM abort in a testcase towards
> the end of mapreduce tests.
>
> Until then there were no failures at all.
>
> I've increased heap size and tried a second run and the failure was
> earlier.
>
> I've looked a Hadoop-trunk builds prior to the HADOOP-9845 and it has been
> failing the same way in all the kept builds.
>
> We need to fix Hadoop-trunk builds independently of this.
>
> Any objection to commit HADOOP-9845 to branch-2 and the 2.1.0-beta
> branches to get all the other jenkins jobs working?
>
> I'll wait till tomorrow morning before proceeding.
>
> Thx
>
>
>
>
> On Mon, Aug 12, 2013 at 8:35 PM, Alejandro Abdelnur <[EMAIL PROTECTED]>wrote:
>
>> Jenkins is running a full test run on trunk using protoc 2.5.0.
>>
>>   https://builds.apache.org/job/Hadoop-trunk/480
>>
>> And it seems go be going just fine.
>>
>> If everything looks OK, I'm planing to backport HADOOP-9845 to the
>> 2.1.0-beta branch midday PST tomorrow. This will normalize all builds
>> failures do the protoc mismatch.
>>
>> Thanks.
>>
>> Alejandro
>>
>>
>> On Mon, Aug 12, 2013 at 5:53 PM, Alejandro Abdelnur <[EMAIL PROTECTED]>wrote:
>>
>>> shooting to get it i n for 2.1.0.
>>>
>>> at moment is in trunk till the nightly finishes. then we'll decide
>>>
>>> in the mean time, you can have multiple versions installed in diff dirs
>>> and set the right one in the path
>>>
>>> thx
>>>
>>> Alejandro
>>> (phone typing)
>>>
>>> On Aug 12, 2013, at 17:47, Konstantin Shvachko <[EMAIL PROTECTED]>
>>> wrote:
>>>
>>> > Ok. After installing protobuf 2.5.0 I can compile trunk.

Alejandro