Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # dev >> Upgrade to protobuf 2.5.0 for the 2.1.0 release, HADOOP-9845


+
Alejandro Abdelnur 2013-08-07, 16:49
+
Tsuyoshi OZAWA 2013-08-08, 06:59
+
Kihwal Lee 2013-08-08, 15:18
+
Alejandro Abdelnur 2013-08-08, 21:29
+
Alejandro Abdelnur 2013-08-09, 20:15
+
Roman Shaposhnik 2013-08-09, 20:53
+
Doug Cutting 2013-08-09, 21:28
+
Giridharan Kesavan 2013-08-10, 05:56
+
Giridharan Kesavan 2013-08-10, 06:10
Copy link to this message
-
Re: Upgrade to protobuf 2.5.0 for the 2.1.0 release, HADOOP-9845
thanks giri, how do we set 2.4 or 2.5., what is the path to both so we can use and env to set it in the jobs?

thx

Alejandro
(phone typing)

On Aug 9, 2013, at 23:10, Giridharan Kesavan <[EMAIL PROTECTED]> wrote:

> build slaves hadoop1-hadoop9 now has libprotoc 2.5.0
>
>
>
> -Giri
>
>
> On Fri, Aug 9, 2013 at 10:56 PM, Giridharan Kesavan <
> [EMAIL PROTECTED]> wrote:
>
>> Alejandro,
>>
>> I'm upgrading protobuf on slaves hadoop1-hadoop9.
>>
>> -Giri
>>
>>
>> On Fri, Aug 9, 2013 at 1:15 PM, Alejandro Abdelnur <[EMAIL PROTECTED]>wrote:
>>
>>> pinging again, I need help from somebody with sudo access to the hadoop
>>> jenkins boxes to do this or to get sudo access for a couple of hours to
>>> set
>>> up myself.
>>>
>>> Please!!!
>>>
>>> thx
>>>
>>>
>>> On Thu, Aug 8, 2013 at 2:29 PM, Alejandro Abdelnur <[EMAIL PROTECTED]
>>>> wrote:
>>>
>>>> To move forward with this we need protoc 2.5.0 in the apache hadoop
>>>> jenkins boxes.
>>>>
>>>> Who can help with this? I assume somebody at Y!, right?
>>>>
>>>> Thx
>>>>
>>>>
>>>> On Thu, Aug 8, 2013 at 2:24 PM, Elliott Clark <[EMAIL PROTECTED]>
>>> wrote:
>>>>
>>>>> In HBase land we've pretty well discovered that we'll need to have the
>>>>> same version of protobuf that the HDFS/Yarn/MR servers are running.
>>>>> That is to say there are issues with ever having 2.4.x and 2.5.x on
>>>>> the same class path.
>>>>>
>>>>> Upgrading to 2.5.x would be great, as it brings some new classes we
>>>>> could use.  With that said HBase is getting pretty close to a rather
>>>>> large release (0.96.0 aka The Singularity) so getting this in sooner
>>>>> rather than later would be great.  If we could get this into 2.1.0 it
>>>>> would be great as that would allow us to have a pretty easy story to
>>>>> users with regards to protobuf version.
>>>>>
>>>>> On Thu, Aug 8, 2013 at 8:18 AM, Kihwal Lee <[EMAIL PROTECTED]>
>>> wrote:
>>>>>> Sorry to hijack the thread but, I also wanted to mention Avro. See
>>>>> HADOOP-9672.
>>>>>> The version we are using has memory leak and inefficiency issues.
>>> We've
>>>>> seen users running into it.
>>>>>>
>>>>>> Kihwal
>>>>>>
>>>>>>
>>>>>> ________________________________
>>>>>> From: Tsuyoshi OZAWA <[EMAIL PROTECTED]>
>>>>>> To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
>>>>>> Cc: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>; "
>>>>> [EMAIL PROTECTED]" <[EMAIL PROTECTED]>; "
>>>>> [EMAIL PROTECTED]" <[EMAIL PROTECTED]>
>>>>>> Sent: Thursday, August 8, 2013 1:59 AM
>>>>>> Subject: Re: Upgrade to protobuf 2.5.0 for the 2.1.0 release,
>>>>> HADOOP-9845
>>>>>>
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> About Hadoop, Harsh is dealing with this problem in HADOOP-9346.
>>>>>> For more detail, please see the JIRA ticket:
>>>>>> https://issues.apache.org/jira/browse/HADOOP-9346
>>>>>>
>>>>>> - Tsuyoshi
>>>>>>
>>>>>> On Thu, Aug 8, 2013 at 1:49 AM, Alejandro Abdelnur <
>>> [EMAIL PROTECTED]>
>>>>> wrote:
>>>>>>> I' like to upgrade to protobuf 2.5.0 for the 2.1.0 release.
>>>>>>>
>>>>>>> As mentioned in HADOOP-9845, Protobuf 2.5 has significant benefits
>>> to
>>>>>>> justify the upgrade.
>>>>>>>
>>>>>>> Doing the upgrade now, with the first beta, will make things easier
>>> for
>>>>>>> downstream projects (like HBase) using protobuf and adopting Hadoop
>>> 2.
>>>>> If
>>>>>>> we do the upgrade later, downstream projects will have to support 2
>>>>>>> different versions and they my get in nasty waters due to classpath
>>>>> issues.
>>>>>>>
>>>>>>> I've locally tested the patch in a pseudo deployment of 2.1.0-beta
>>>>> branch
>>>>>>> and it works fine (something is broken in trunk in the RPC layer
>>>>> YARN-885).
>>>>>>>
>>>>>>> Now, to do this it will require a few things:
>>>>>>>
>>>>>>> * Make sure protobuf 2.5.0 is available in the jenkins box
>>>>>>> * A follow up email to dev@ aliases indicating developers should
>>>>> install
>>>>>>> locally protobuf 2.5.0
+
Giridharan Kesavan 2013-08-12, 18:30
+
Alejandro Abdelnur 2013-08-12, 18:37
+
Giridharan Kesavan 2013-08-12, 18:47
+
Alejandro Abdelnur 2013-08-12, 21:57
+
Alejandro Abdelnur 2013-08-12, 22:19
+
Konstantin Shvachko 2013-08-13, 00:47
+
Alejandro Abdelnur 2013-08-13, 00:53
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB