|
|
-
VOTE: Committing HADOOP-6949 to 0.22 branch
Konstantin Shvachko 2011-03-29, 04:39
HADOOP-6949 introduced a very important optimization to the RPC layer. Based
on the benchmarks presented in HDFS-1583 this provides an order of magnitude
improvement of current RPC implementation.
RPC is a common component of Hadoop projects. Many of them should benefit
from this change. But since this is an incompatible change it requires a
vote to be included into a previous branch.
Please vote for inclusion of this change into branch 0.22.
+1 from me.
Thanks,
--Konstantin
+
Konstantin Shvachko 2011-03-29, 04:39
-
Re: VOTE: Committing HADOOP-6949 to 0.22 branch
Dhruba Borthakur 2011-03-29, 05:19
This is a very effective optimization, +1 on pulling it to 0.22. -dhruba On Mon, Mar 28, 2011 at 9:39 PM, Konstantin Shvachko <[EMAIL PROTECTED]>wrote: > HADOOP-6949 introduced a very important optimization to the RPC layer. > Based > on the benchmarks presented in HDFS-1583 this provides an order of > magnitude > improvement of current RPC implementation. > RPC is a common component of Hadoop projects. Many of them should benefit > from this change. But since this is an incompatible change it requires a > vote to be included into a previous branch. > Please vote for inclusion of this change into branch 0.22. > > +1 from me. > > Thanks, > --Konstantin > -- Connect to me at http://www.facebook.com/dhruba
+
Dhruba Borthakur 2011-03-29, 05:19
-
Re: VOTE: Committing HADOOP-6949 to 0.22 branch
Nigel Daley 2011-03-29, 15:02
[bcc'ing hdfs-dev and mapreduce-dev] Konstantin, can you outline the effects of this incompatibility? Thx, Nige On Mar 28, 2011, at 10:19 PM, Dhruba Borthakur wrote: > This is a very effective optimization, +1 on pulling it to 0.22. > > -dhruba > > > On Mon, Mar 28, 2011 at 9:39 PM, Konstantin Shvachko > <[EMAIL PROTECTED]>wrote: > >> HADOOP-6949 introduced a very important optimization to the RPC layer. >> Based >> on the benchmarks presented in HDFS-1583 this provides an order of >> magnitude >> improvement of current RPC implementation. >> RPC is a common component of Hadoop projects. Many of them should benefit >> from this change. But since this is an incompatible change it requires a >> vote to be included into a previous branch. >> Please vote for inclusion of this change into branch 0.22. >> >> +1 from me. >> >> Thanks, >> --Konstantin >> > > > > -- > Connect to me at http://www.facebook.com/dhruba
+
Nigel Daley 2011-03-29, 15:02
-
Re: VOTE: Committing HADOOP-6949 to 0.22 branch
Eli Collins 2011-03-29, 05:26
+1
On Mon, Mar 28, 2011 at 9:39 PM, Konstantin Shvachko
<[EMAIL PROTECTED]> wrote:
> HADOOP-6949 introduced a very important optimization to the RPC layer. Based
> on the benchmarks presented in HDFS-1583 this provides an order of magnitude
> improvement of current RPC implementation.
> RPC is a common component of Hadoop projects. Many of them should benefit
> from this change. But since this is an incompatible change it requires a
> vote to be included into a previous branch.
> Please vote for inclusion of this change into branch 0.22.
>
> +1 from me.
>
> Thanks,
> --Konstantin
>
+
Eli Collins 2011-03-29, 05:26
-
Re: VOTE: Committing HADOOP-6949 to 0.22 branch
Doug Cutting 2011-03-29, 15:56
+1
I don't think this creates an incompatibility. It changes the RPC wire
format, but we already require that clients and servers run identical
builds. No application that ran with a prior version of Hadoop would be
broken by this change when it upgrades to this version of Hadoop.
Doug
On 03/28/2011 09:39 PM, Konstantin Shvachko wrote:
> HADOOP-6949 introduced a very important optimization to the RPC layer. Based
> on the benchmarks presented in HDFS-1583 this provides an order of magnitude
> improvement of current RPC implementation.
> RPC is a common component of Hadoop projects. Many of them should benefit
> from this change. But since this is an incompatible change it requires a
> vote to be included into a previous branch.
> Please vote for inclusion of this change into branch 0.22.
>
> +1 from me.
>
> Thanks,
> --Konstantin
>
+
Doug Cutting 2011-03-29, 15:56
-
Re: VOTE: Committing HADOOP-6949 to 0.22 branch
Stephen Boesch 2011-03-29, 16:04
+1
2011/3/29 Doug Cutting <[EMAIL PROTECTED]>
> +1
>
> I don't think this creates an incompatibility. It changes the RPC wire
> format, but we already require that clients and servers run identical
> builds. No application that ran with a prior version of Hadoop would be
> broken by this change when it upgrades to this version of Hadoop.
>
> Doug
>
> On 03/28/2011 09:39 PM, Konstantin Shvachko wrote:
> > HADOOP-6949 introduced a very important optimization to the RPC layer.
> Based
> > on the benchmarks presented in HDFS-1583 this provides an order of
> magnitude
> > improvement of current RPC implementation.
> > RPC is a common component of Hadoop projects. Many of them should benefit
> > from this change. But since this is an incompatible change it requires a
> > vote to be included into a previous branch.
> > Please vote for inclusion of this change into branch 0.22.
> >
> > +1 from me.
> >
> > Thanks,
> > --Konstantin
> >
>
+
Stephen Boesch 2011-03-29, 16:04
-
Re: VOTE: Committing HADOOP-6949 to 0.22 branch
Amit Sangroya 2011-03-29, 16:18
+1
On Tue, Mar 29, 2011 at 6:04 PM, Stephen Boesch <[EMAIL PROTECTED]> wrote:
> +1
>
> 2011/3/29 Doug Cutting <[EMAIL PROTECTED]>
>
>> +1
>>
>> I don't think this creates an incompatibility. It changes the RPC wire
>> format, but we already require that clients and servers run identical
>> builds. No application that ran with a prior version of Hadoop would be
>> broken by this change when it upgrades to this version of Hadoop.
>>
>> Doug
>>
>> On 03/28/2011 09:39 PM, Konstantin Shvachko wrote:
>> > HADOOP-6949 introduced a very important optimization to the RPC layer.
>> Based
>> > on the benchmarks presented in HDFS-1583 this provides an order of
>> magnitude
>> > improvement of current RPC implementation.
>> > RPC is a common component of Hadoop projects. Many of them should benefit
>> > from this change. But since this is an incompatible change it requires a
>> > vote to be included into a previous branch.
>> > Please vote for inclusion of this change into branch 0.22.
>> >
>> > +1 from me.
>> >
>> > Thanks,
>> > --Konstantin
>> >
>>
>
+
Amit Sangroya 2011-03-29, 16:18
-
Re: VOTE: Committing HADOOP-6949 to 0.22 branch
Jakob Homan 2011-03-29, 18:04
+1
n.b. that the vote lost hdfs and common dev at some point. I've added
them back.
On Tue, Mar 29, 2011 at 9:18 AM, Amit Sangroya <[EMAIL PROTECTED]> wrote:
> +1
>
> On Tue, Mar 29, 2011 at 6:04 PM, Stephen Boesch <[EMAIL PROTECTED]> wrote:
>> +1
>>
>> 2011/3/29 Doug Cutting <[EMAIL PROTECTED]>
>>
>>> +1
>>>
>>> I don't think this creates an incompatibility. It changes the RPC wire
>>> format, but we already require that clients and servers run identical
>>> builds. No application that ran with a prior version of Hadoop would be
>>> broken by this change when it upgrades to this version of Hadoop.
>>>
>>> Doug
>>>
>>> On 03/28/2011 09:39 PM, Konstantin Shvachko wrote:
>>> > HADOOP-6949 introduced a very important optimization to the RPC layer.
>>> Based
>>> > on the benchmarks presented in HDFS-1583 this provides an order of
>>> magnitude
>>> > improvement of current RPC implementation.
>>> > RPC is a common component of Hadoop projects. Many of them should benefit
>>> > from this change. But since this is an incompatible change it requires a
>>> > vote to be included into a previous branch.
>>> > Please vote for inclusion of this change into branch 0.22.
>>> >
>>> > +1 from me.
>>> >
>>> > Thanks,
>>> > --Konstantin
>>> >
>>>
>>
>
+
Jakob Homan 2011-03-29, 18:04
-
Re: VOTE: Committing HADOOP-6949 to 0.22 branch
Konstantin Shvachko 2011-03-30, 01:08
Nigel,
The nature of incompatibility is that the RPC version is changing, which
means
VersionedProtocol-s become incompatible all at once. As opposed to say
only DatanodeProtocol or mr.ClientProtocol.
Doug is right because of our strict requirements for protocol compatibility
this
will not affect users upgrading to 0.22.
Matt, thanks for checking and testing.
--Konstantin
On Tue, Mar 29, 2011 at 11:04 AM, Jakob Homan <[EMAIL PROTECTED]> wrote:
> +1
> n.b. that the vote lost hdfs and common dev at some point. I've added
> them back.
>
> On Tue, Mar 29, 2011 at 9:18 AM, Amit Sangroya <[EMAIL PROTECTED]>
> wrote:
> > +1
> >
> > On Tue, Mar 29, 2011 at 6:04 PM, Stephen Boesch <[EMAIL PROTECTED]>
> wrote:
> >> +1
> >>
> >> 2011/3/29 Doug Cutting <[EMAIL PROTECTED]>
> >>
> >>> +1
> >>>
> >>> I don't think this creates an incompatibility. It changes the RPC wire
> >>> format, but we already require that clients and servers run identical
> >>> builds. No application that ran with a prior version of Hadoop would
> be
> >>> broken by this change when it upgrades to this version of Hadoop.
> >>>
> >>> Doug
> >>>
> >>> On 03/28/2011 09:39 PM, Konstantin Shvachko wrote:
> >>> > HADOOP-6949 introduced a very important optimization to the RPC
> layer.
> >>> Based
> >>> > on the benchmarks presented in HDFS-1583 this provides an order of
> >>> magnitude
> >>> > improvement of current RPC implementation.
> >>> > RPC is a common component of Hadoop projects. Many of them should
> benefit
> >>> > from this change. But since this is an incompatible change it
> requires a
> >>> > vote to be included into a previous branch.
> >>> > Please vote for inclusion of this change into branch 0.22.
> >>> >
> >>> > +1 from me.
> >>> >
> >>> > Thanks,
> >>> > --Konstantin
> >>> >
> >>>
> >>
> >
>
+
Konstantin Shvachko 2011-03-30, 01:08
-
Re: VOTE: Committing HADOOP-6949 to 0.22 branch
Steve Loughran 2011-03-30, 12:47
+1
-steve
+
Steve Loughran 2011-03-30, 12:47
|
|
