Folks,

Some Hadoop deployments have upgraded to 0.19.0.  Clearly, the 0.19  
branch has issues and a 0.19.1 release is needed.

Quality issues in the changes made for the file append feature have  
prevented some from deploying Hadoop 0.19.  One of these changes  
(sync) has now been "fixed" by reducing its semantics in Hadoop 0.18.3  
(HADOOP-4997).  This was necessary to stabilize the 0.18 branch.

I would like to propose that we apply this same "fix" to sync in  
0.19.1 and 0.20.0.  Since append requires the full semantics of sync,  
I propose we also disable append (perhaps throw  
UnsupportedOperationException from API?).  Yes, this would  
unfortunately be an incompatible change between 0.19.0 and 0.19.1.  We  
can then take the time needed to fix append properly in 0.21.0.

I will call a vote for 0.19.1 and 0.20.0 when blockers are fixed.

Nigel

Nigel Daley wrote:
> Folks,
>
> Some Hadoop deployments have upgraded to 0.19.0.  Clearly, the 0.19
> branch has issues and a 0.19.1 release is needed.
>
> Quality issues in the changes made for the file append feature have
> prevented some from deploying Hadoop 0.19.  One of these changes (sync)
> has now been "fixed" by reducing its semantics in Hadoop 0.18.3
> (HADOOP-4997).  This was necessary to stabilize the 0.18 branch.
>
> I would like to propose that we apply this same "fix" to sync in 0.19.1
> and 0.20.0.  Since append requires the full semantics of sync, I propose
> we also disable append (perhaps throw UnsupportedOperationException from
> API?).  Yes, this would unfortunately be an incompatible change between
> 0.19.0 and 0.19.1.  We can then take the time needed to fix append
> properly in 0.21.0.

I can see some people being unhappy about this, but giving them a choice
between having the filesystem work or not, hopefully they will see the
merits of the change. And I am +1 to taking time to fix things; fast
fixes often create new problems

The below proposes pushing a working append out 6 months or so.

Can we have pointers as to what is meant by 'quality issues' in the below so
we can make a more informed vote or is it just the hdfs issue posted against
0.19.1?  Is it there also that we would find what is involved making append
work in 0.19.1?

Thanks,
St.Ack

On Fri, Jan 30, 2009 at 2:36 AM, Steve Loughran <[EMAIL PROTECTED]> wrote:

> Nigel Daley wrote:
>
>> Folks,
>>
>> Some Hadoop deployments have upgraded to 0.19.0.  Clearly, the 0.19 branch
>> has issues and a 0.19.1 release is needed.
>>
>> Quality issues in the changes made for the file append feature have
>> prevented some from deploying Hadoop 0.19.  One of these changes (sync) has
>> now been "fixed" by reducing its semantics in Hadoop 0.18.3 (HADOOP-4997).
>>  This was necessary to stabilize the 0.18 branch.
>>
>> I would like to propose that we apply this same "fix" to sync in 0.19.1
>> and 0.20.0.  Since append requires the full semantics of sync, I propose we
>> also disable append (perhaps throw UnsupportedOperationException from API?).
>>  Yes, this would unfortunately be an incompatible change between 0.19.0 and
>> 0.19.1.  We can then take the time needed to fix append properly in 0.21.0.
>>
>
> I can see some people being unhappy about this, but giving them a choice
> between having the filesystem work or not, hopefully they will see the
> merits of the change. And I am +1 to taking time to fix things; fast fixes
> often create new problems
>

Are proposing disabling both append and sync?

While HBase does not use the full semantics of append, we
do depend on sync.

There is a patch for trunk (HADOOP-4379) that we are testing
on the 0.19 branch, 0.20 branch and trunk.

-1 if sync does not work until 0.21.1 recovery from server
crashes depends entirely on sync working.

---
Jim Kellerman, Powerset (Live Search, Microsoft Corporation)
> -----Original Message-----
> From: Nigel Daley [mailto:[EMAIL PROTECTED]]
> Sent: Thursday, January 29, 2009 4:16 PM
> To: [EMAIL PROTECTED]
> Subject: Hadoop 0.19.1
>
> Folks,
>
> Some Hadoop deployments have upgraded to 0.19.0.  Clearly, the 0.19
> branch has issues and a 0.19.1 release is needed.
>
> Quality issues in the changes made for the file append feature have
> prevented some from deploying Hadoop 0.19.  One of these changes
> (sync) has now been "fixed" by reducing its semantics in Hadoop 0.18.3
> (HADOOP-4997).  This was necessary to stabilize the 0.18 branch.
>
> I would like to propose that we apply this same "fix" to sync in
> 0.19.1 and 0.20.0.  Since append requires the full semantics of sync,
> I propose we also disable append (perhaps throw
> UnsupportedOperationException from API?).  Yes, this would
> unfortunately be an incompatible change between 0.19.0 and 0.19.1.  We
> can then take the time needed to fix append properly in 0.21.0.
>
> I will call a vote for 0.19.1 and 0.20.0 when blockers are fixed.
>
> Nigel

stack wrote:
> The below proposes pushing a working append out 6 months or so.
>
> Can we have pointers as to what is meant by 'quality issues' in the below so
> we can make a more informed vote or is it just the hdfs issue posted against
> 0.19.1?  

> Is it there also that we would find what is involved making append
> work in 0.19.1?

If one knew what is enough to fix properly, it would be easy. But over
last couple of months, there have been many fixes (some of these jiras
are listed in one Konstantins HADOOP-4663). The discussions are still
bringing up more cases where the implementation or algorithm should
change. But these are improvements for sure. But doubt if I would be
ready to call it is 'completely fixed'. It needs time and a lot of
testing in large clusters.

Personally I am +1 for getting these into 0.19 branch. Most importantly
even clusters and application not using append or sync were also
affected, thats why extra caution.

my 2 cents. hope this does not digress too much from the main topic.

Raghu.

> Thanks,
> St.Ack
>
>
>
> On Fri, Jan 30, 2009 at 2:36 AM, Steve Loughran <[EMAIL PROTECTED]> wrote:
>
>> Nigel Daley wrote:
>>
>>> Folks,
>>>
>>> Some Hadoop deployments have upgraded to 0.19.0.  Clearly, the 0.19 branch
>>> has issues and a 0.19.1 release is needed.
>>>
>>> Quality issues in the changes made for the file append feature have
>>> prevented some from deploying Hadoop 0.19.  One of these changes (sync) has
>>> now been "fixed" by reducing its semantics in Hadoop 0.18.3 (HADOOP-4997).
>>>  This was necessary to stabilize the 0.18 branch.
>>>
>>> I would like to propose that we apply this same "fix" to sync in 0.19.1
>>> and 0.20.0.  Since append requires the full semantics of sync, I propose we
>>> also disable append (perhaps throw UnsupportedOperationException from API?).
>>>  Yes, this would unfortunately be an incompatible change between 0.19.0 and
>>> 0.19.1.  We can then take the time needed to fix append properly in 0.21.0.
>>>
>> I can see some people being unhappy about this, but giving them a choice
>> between having the filesystem work or not, hopefully they will see the
>> merits of the change. And I am +1 to taking time to fix things; fast fixes
>> often create new problems
>>
>

Raghu Angadi wrote:
>> Is it there also that we would find what is involved making append
>> work in 0.19.1?
>
> If one knew what is enough to fix properly, it would be easy. But over
> last couple of months, there have been many fixes (some of these jiras
> are listed in one Konstantins HADOOP-4663).

Konstantin's comment I referred to (it was also linked from HADOOP-4663,
but harder to find).

https://issues.apache.org/jira/browse/HADOOP-5027#action_12668136

Raghu.

> The discussions are still
> bringing up more cases where the implementation or algorithm should
> change. But these are improvements for sure. But doubt if I would be
> ready to call it is 'completely fixed'. It needs time and a lot of
> testing in large clusters.
>
> Personally I am +1 for getting these into 0.19 branch. Most importantly
> even clusters and application not using append or sync were also
> affected, thats why extra caution.
>
> my 2 cents. hope this does not digress too much from the main topic.
>
> Raghu.
>
>> Thanks,
>> St.Ack
>>
>>
>>
>> On Fri, Jan 30, 2009 at 2:36 AM, Steve Loughran <[EMAIL PROTECTED]>
>> wrote:
>>
>>> Nigel Daley wrote:
>>>
>>>> Folks,
>>>>
>>>> Some Hadoop deployments have upgraded to 0.19.0.  Clearly, the 0.19
>>>> branch
>>>> has issues and a 0.19.1 release is needed.
>>>>
>>>> Quality issues in the changes made for the file append feature have
>>>> prevented some from deploying Hadoop 0.19.  One of these changes
>>>> (sync) has
>>>> now been "fixed" by reducing its semantics in Hadoop 0.18.3
>>>> (HADOOP-4997).
>>>>  This was necessary to stabilize the 0.18 branch.
>>>>
>>>> I would like to propose that we apply this same "fix" to sync in 0.19.1
>>>> and 0.20.0.  Since append requires the full semantics of sync, I
>>>> propose we
>>>> also disable append (perhaps throw UnsupportedOperationException
>>>> from API?).
>>>>  Yes, this would unfortunately be an incompatible change between
>>>> 0.19.0 and
>>>> 0.19.1.  We can then take the time needed to fix append properly in
>>>> 0.21.0.
>>>>
>>> I can see some people being unhappy about this, but giving them a choice
>>> between having the filesystem work or not, hopefully they will see the
>>> merits of the change. And I am +1 to taking time to fix things; fast
>>> fixes
>>> often create new problems
>>>
>>
>

Raghu, thanks for providing the link.

Jim> Are proposing disabling both append and sync?

Jim, this statement is probably too strong.
sync is not disabled per se, you will be able to use it, although
its full semantic is not guaranteed in some failure scenarios.
See more here
https://issues.apache.org/jira/browse/HADOOP-4663#action_12661802

We will have to really disable append (throw UnsupportedOperationException)
because otherwise current solution may lead to a loss of previously existed data.

I agree with Nigel that there is a need for an urgent 0.19.1 release
because a lot of bugs were fixed since 0.18.2 and 0.19.0.
The system is now stable on our clusters with 0.18.3 same fixes went into 0.19.1.

If we try to rush fixing the bugs for append (listed in my comment) we
risk to destabilize the system again, and this is my main concern.

Formally we should not release until a feature is fixed, but I think it
is better to let people use a stable release with limited functionality
rather than having full functionality with a risk of data loss.

Hope this will work for everybody.
--Konstantin
Raghu Angadi wrote:
> Raghu Angadi wrote:
>>> Is it there also that we would find what is involved making append
>>> work in 0.19.1?
>>
>> If one knew what is enough to fix properly, it would be easy. But over
>> last couple of months, there have been many fixes (some of these jiras
>> are listed in one Konstantins HADOOP-4663).
>
> Konstantin's comment I referred to (it was also linked from HADOOP-4663,
> but harder to find).
>
> https://issues.apache.org/jira/browse/HADOOP-5027#action_12668136
>
> Raghu.
>
>> The discussions are still bringing up more cases where the
>> implementation or algorithm should change. But these are improvements
>> for sure. But doubt if I would be ready to call it is 'completely
>> fixed'. It needs time and a lot of testing in large clusters.
>>
>> Personally I am +1 for getting these into 0.19 branch. Most
>> importantly even clusters and application not using append or sync
>> were also affected, thats why extra caution.
>>
>> my 2 cents. hope this does not digress too much from the main topic.
>>
>> Raghu.
>>
>>> Thanks,
>>> St.Ack
>>>
>>>
>>>
>>> On Fri, Jan 30, 2009 at 2:36 AM, Steve Loughran <[EMAIL PROTECTED]>
>>> wrote:
>>>
>>>> Nigel Daley wrote:
>>>>
>>>>> Folks,
>>>>>
>>>>> Some Hadoop deployments have upgraded to 0.19.0.  Clearly, the 0.19
>>>>> branch
>>>>> has issues and a 0.19.1 release is needed.
>>>>>
>>>>> Quality issues in the changes made for the file append feature have
>>>>> prevented some from deploying Hadoop 0.19.  One of these changes
>>>>> (sync) has
>>>>> now been "fixed" by reducing its semantics in Hadoop 0.18.3
>>>>> (HADOOP-4997).
>>>>>  This was necessary to stabilize the 0.18 branch.
>>>>>
>>>>> I would like to propose that we apply this same "fix" to sync in
>>>>> 0.19.1
>>>>> and 0.20.0.  Since append requires the full semantics of sync, I
>>>>> propose we
>>>>> also disable append (perhaps throw UnsupportedOperationException
>>>>> from API?).
>>>>>  Yes, this would unfortunately be an incompatible change between
>>>>> 0.19.0 and
>>>>> 0.19.1.  We can then take the time needed to fix append properly in
>>>>> 0.21.0.
>>>>>
>>>> I can see some people being unhappy about this, but giving them a
>>>> choice
>>>> between having the filesystem work or not, hopefully they will see the
>>>> merits of the change. And I am +1 to taking time to fix things; fast
>>>> fixes
>>>> often create new problems
>>>>
>>>
>>
>
>

Hi Konstantin,

We are also heavy users of fsync().  I've been working with Dhruba on
Hadoop-4379 <https://issues.apache.org/jira/browse/HADOOP-4379>.  His most
recent patch appears to work for Jim's situation.  However, there are still
a couple of problems that need to be resolved before we can start heavily
using it:

1. After application crash/restart, the file length (as returned by
getFileStatus) is incorrect since the length at the namenode is stale.
Ideally getFileStatus() would return the accurate file length by fetching
the size of the last block from the primary datanode.  If that's not
feasible, there should be some other way to obtain the actual file length.

2. When an application comes up after a crash, it seems to hang for about 60
seconds waiting for lease recovery.  Our database cannot go offline for a
whole minute doing nothing.  In our case, when we come up after a crash and
try to re-open the log file, we know for certain that we are the exclusive
owner of that file.  There should be a way to tell the system to forcibly
take over the lease and recover immediately

What do you recommend?  Is there anyway we could get these two issues fixed
for 0.19.1, or should I file issues for them and get them on the schedule
for 0.19.2?

- Doug

On Mon, Feb 2, 2009 at 11:59 AM, Konstantin Shvachko <[EMAIL PROTECTED]>wrote:

> Raghu, thanks for providing the link.
>
> Jim> Are proposing disabling both append and sync?
>
> Jim, this statement is probably too strong.
> sync is not disabled per se, you will be able to use it, although
> its full semantic is not guaranteed in some failure scenarios.
> See more here
> https://issues.apache.org/jira/browse/HADOOP-4663#action_12661802
>
> We will have to really disable append (throw UnsupportedOperationException)
> because otherwise current solution may lead to a loss of previously existed
> data.
>
> I agree with Nigel that there is a need for an urgent 0.19.1 release
> because a lot of bugs were fixed since 0.18.2 and 0.19.0.
> The system is now stable on our clusters with 0.18.3 same fixes went into
> 0.19.1.
>
> If we try to rush fixing the bugs for append (listed in my comment) we
> risk to destabilize the system again, and this is my main concern.
>
> Formally we should not release until a feature is fixed, but I think it
> is better to let people use a stable release with limited functionality
> rather than having full functionality with a risk of data loss.
>
> Hope this will work for everybody.
> --Konstantin
>
>
> Raghu Angadi wrote:
>
>> Raghu Angadi wrote:
>>
>>> Is it there also that we would find what is involved making append
>>>> work in 0.19.1?
>>>>
>>>
>>> If one knew what is enough to fix properly, it would be easy. But over
>>> last couple of months, there have been many fixes (some of these jiras are
>>> listed in one Konstantins HADOOP-4663).
>>>
>>
>> Konstantin's comment I referred to (it was also linked from HADOOP-4663,
>> but harder to find).
>>
>> https://issues.apache.org/jira/browse/HADOOP-5027#action_12668136
>>
>> Raghu.
>>
>>  The discussions are still bringing up more cases where the implementation
>>> or algorithm should change. But these are improvements for sure. But doubt
>>> if I would be ready to call it is 'completely fixed'. It needs time and a
>>> lot of testing in large clusters.
>>>
>>> Personally I am +1 for getting these into 0.19 branch. Most importantly
>>> even clusters and application not using append or sync were also affected,
>>> thats why extra caution.
>>>
>>> my 2 cents. hope this does not digress too much from the main topic.
>>>
>>> Raghu.
>>>
>>>  Thanks,
>>>> St.Ack
>>>>
>>>>
>>>>
>>>> On Fri, Jan 30, 2009 at 2:36 AM, Steve Loughran <[EMAIL PROTECTED]>
>>>> wrote:
>>>>
>>>>  Nigel Daley wrote:
>>>>>
>>>>>  Folks,
>>>>>>
>>>>>> Some Hadoop deployments have upgraded to 0.19.0.  Clearly, the 0.19
>>>>>> branch
>>>>>> has issues and a 0.19.1 release is needed.
>>>>>>
>>>>>> Quality issues in the changes made for the file append feature have

On Feb 2, 2009, at 12:51 PM, Doug Judd wrote:

> What do you recommend?  Is there anyway we could get these two  
> issues fixed
> for 0.19.1, or should I file issues for them and get them on the  
> schedule
> for 0.19.2?

Given the outstanding problems and general level of uncertainty, I'd  
favor releasing a 0.19.1 with the equivalent of the 0.18.3 disable on  
fsync and append. Let's get them fixed in 0.20 first and then we can  
debate whether the rewards of pushing them back into an 0.19.2 would  
make sense. I'm pretty uncomfortable at the moment with how the entire  
functional complex seems to cause a continuous stream of problems.

-- Owen

HBase could live with sync() in 0.20.0 (and preferably 0.19.2),
but 0.20.0 is an absolute blocker.

---
Jim Kellerman, Powerset (Live Search, Microsoft Corporation)
> -----Original Message-----
> From: Owen O'Malley [mailto:[EMAIL PROTECTED]] On Behalf Of Owen
> O'Malley
> Sent: Monday, February 02, 2009 1:51 PM
> To: [EMAIL PROTECTED]
> Subject: Re: Hadoop 0.19.1
>
> On Feb 2, 2009, at 12:51 PM, Doug Judd wrote:
>
> > What do you recommend?  Is there anyway we could get these two
> > issues fixed
> > for 0.19.1, or should I file issues for them and get them on the
> > schedule
> > for 0.19.2?
>
> Given the outstanding problems and general level of uncertainty, I'd
> favor releasing a 0.19.1 with the equivalent of the 0.18.3 disable on
> fsync and append. Let's get them fixed in 0.20 first and then we can
> debate whether the rewards of pushing them back into an 0.19.2 would
> make sense. I'm pretty uncomfortable at the moment with how the entire
> functional complex seems to cause a continuous stream of problems.
>
> -- Owen

Sounds good.  I would much rather wait and have fsync() done correctly in
0.20 than get some sort of hacked version in 0.19.  I'll create a couple of
issues and mark them for 0.20  Thanks.

- Doug

On Mon, Feb 2, 2009 at 1:51 PM, Owen O'Malley <[EMAIL PROTECTED]> wrote:

> On Feb 2, 2009, at 12:51 PM, Doug Judd wrote:
>
>  What do you recommend?  Is there anyway we could get these two issues
>> fixed
>> for 0.19.1, or should I file issues for them and get them on the schedule
>> for 0.19.2?
>>
>
> Given the outstanding problems and general level of uncertainty, I'd favor
> releasing a 0.19.1 with the equivalent of the 0.18.3 disable on fsync and
> append. Let's get them fixed in 0.20 first and then we can debate whether
> the rewards of pushing them back into an 0.19.2 would make sense. I'm pretty
> uncomfortable at the moment with how the entire functional complex seems to
> cause a continuous stream of problems.
>
> -- Owen
>

>  What do you recommend?

In general. There may be people/organizations, which will not compromise
on the reduced functionality in favor of the stability, this is understandable.
I would propose to create a separate (unofficial experimental) branch, which
would track changes like HADOOP-4379. The branch may later either die when the
main stream is fixed or be merged with the trunk if the changes proved to be stable.

 >1. the file length (as returned by getFileStatus) is incorrect

May be the following work around will be useful.
If you read from a file you always try to read more data than the length reported
by the name-node. How much more? The size of one block would be enough, or
even to the next (ceiling) block boundary.

 >2. When an application comes up after a crash, it seems to hang for about 60

Don't have enough context on that, sorry.

Thanks,
--Konstantin

Doug Judd wrote:
> Sounds good.  I would much rather wait and have fsync() done correctly in
> 0.20 than get some sort of hacked version in 0.19.  I'll create a couple of
> issues and mark them for 0.20  Thanks.
>
> - Doug
>
> On Mon, Feb 2, 2009 at 1:51 PM, Owen O'Malley <[EMAIL PROTECTED]> wrote:
>
>> On Feb 2, 2009, at 12:51 PM, Doug Judd wrote:
>>
>>  What do you recommend?  Is there anyway we could get these two issues
>>> fixed
>>> for 0.19.1, or should I file issues for them and get them on the schedule
>>> for 0.19.2?
>>>
>> Given the outstanding problems and general level of uncertainty, I'd favor
>> releasing a 0.19.1 with the equivalent of the 0.18.3 disable on fsync and
>> append. Let's get them fixed in 0.20 first and then we can debate whether
>> the rewards of pushing them back into an 0.19.2 would make sense. I'm pretty
>> uncomfortable at the moment with how the entire functional complex seems to
>> cause a continuous stream of problems.
>>
>> -- Owen
>>
>

Comments inline ...

On Mon, Feb 2, 2009 at 4:23 PM, Konstantin Shvachko <[EMAIL PROTECTED]>wrote:

> >  What do you recommend?
>
> In general. There may be people/organizations, which will not compromise
> on the reduced functionality in favor of the stability, this is
> understandable.
> I would propose to create a separate (unofficial experimental) branch,
> which
> would track changes like HADOOP-4379. The branch may later either die when
> the
> main stream is fixed or be merged with the trunk if the changes proved to
> be stable.
Sure, that sounds reasonable.  One thing I would caution against is spending
a lot of time doing incremental patchwork on something that needs a
ground-up overhaul.  I would much rather wait a couple of months longer and
get software that is based on a well thought out design that is
fundamentally sound.  Ultimately that will be the fastest path to stability.
>
> >1. the file length (as returned by getFileStatus) is incorrect

> May be the following work around will be useful.
> If you read from a file you always try to read more data than the length
> reported
> by the name-node. How much more? The size of one block would be enough, or
> even to the next (ceiling) block boundary.
I could certainly implement a workaround, however, from an API standpoint,
the filesystem (IMHO) should always give you a way to obtain the real length
of the file.  The semantics of the current getFileStatus() make it difficult
to reason about the state of your filesystem.  It basically returns a
"possibly stale" version of the length.  I would prefer to wait for an
implementation that gives an accurate answer and spend my time and energy
helping to test that one, rather than spending a bunch of time implementing
a workaround for the current version.

>2. When an application comes up after a crash, it seems to hang for about
> 60
>
> Don't have enough context on that, sorry.
I spoke too soon on this.  The reason that HDFS was hanging on lease
recovery was because I was opening the file in append mode to force lease
recovery (at Dhruba's suggestion) so that it would update the NameNode with
the proper length.  If I had a method of obtaining the accurate length of
the file, I wouldn't need to do this.  Hence, I didn't bother filing an
issue on this.

- Doug
> Thanks,
> --Konstantin
>
> Doug Judd wrote:
>
>> Sounds good.  I would much rather wait and have fsync() done correctly in
>> 0.20 than get some sort of hacked version in 0.19.  I'll create a couple
>> of
>> issues and mark them for 0.20  Thanks.
>>
>> - Doug
>>
>> On Mon, Feb 2, 2009 at 1:51 PM, Owen O'Malley <[EMAIL PROTECTED]> wrote:
>>
>>  On Feb 2, 2009, at 12:51 PM, Doug Judd wrote:
>>>
>>>  What do you recommend?  Is there anyway we could get these two issues
>>>
>>>> fixed
>>>> for 0.19.1, or should I file issues for them and get them on the
>>>> schedule
>>>> for 0.19.2?
>>>>
>>>>  Given the outstanding problems and general level of uncertainty, I'd
>>> favor
>>> releasing a 0.19.1 with the equivalent of the 0.18.3 disable on fsync and
>>> append. Let's get them fixed in 0.20 first and then we can debate whether
>>> the rewards of pushing them back into an 0.19.2 would make sense. I'm
>>> pretty
>>> uncomfortable at the moment with how the entire functional complex seems
>>> to
>>> cause a continuous stream of problems.
>>>
>>> -- Owen
>>>
>>>
>>

On Feb 2, 2009, at 1:51 PM, Owen O'Malley wrote:

> On Feb 2, 2009, at 12:51 PM, Doug Judd wrote:
>
> > What do you recommend?  Is there anyway we could get these two
> > issues fixed
> > for 0.19.1, or should I file issues for them and get them on the
> > schedule
> > for 0.19.2?
>
> Given the outstanding problems and general level of uncertainty, I'd
> favor releasing a 0.19.1 with the equivalent of the 0.18.3 disable on
> fsync and append. Let's get them fixed in 0.20 first and then we can
> debate whether the rewards of pushing them back into an 0.19.2 would
> make sense. I'm pretty uncomfortable at the moment with how the entire
> functional complex seems to cause a continuous stream of problems.
>
> -- Owen
>
+1
Append/sync code has been very problematic and bug fixes have not  
always fixed things completely.
Hence getting a quick  release of 0.19.1 that deal with the data loss  
(ie 0.18.3 functionality) is important.

sanjay

On Feb 2, 2009, at 6:18 PM, Doug Judd wrote:

> Comments inline ...
>
> On Mon, Feb 2, 2009 at 4:23 PM, Konstantin Shvachko <shv@yahoo-
> inc.com>wrote:
>
> > >  What do you recommend?
> >
> > In general. There may be people/organizations, which will not  
> compromise
> > on the reduced functionality in favor of the stability, this is
> > understandable.
> > I would propose to create a separate (unofficial experimental)  
> branch,
> > which
> > would track changes like HADOOP-4379. The branch may later either  
> die when
> > the
> > main stream is fixed or be merged with the trunk if the changes  
> proved to
> > be stable.
>
>
> Sure, that sounds reasonable.  One thing I would caution against is  
> spending
> a lot of time doing incremental patchwork on something that needs a
> ground-up overhaul.  I would much rather wait a couple of months  
> longer and
> get software that is based on a well thought out design that is
> fundamentally sound.  Ultimately that will be the fastest path to  
> stability.
>

Agree.
sanjay
>

On Feb 2, 2009, at 4:23 PM, Konstantin Shvachko wrote:

>
>  >  What do you recommend?
>
> In general. There may be people/organizations, which will not  
> compromise
> on the reduced functionality in favor of the stability, this is  
> understandable.
> I would propose to create a separate (unofficial experimental)  
> branch, which
> would track changes like HADOOP-4379. The branch may later either  
> die when the
> main stream is fixed or be merged with the trunk if the changes  
> proved to be stable.
>
This is very a interesting suggestion.
Many in the team  have come to the conclusion that complex projects  
like append should be done on a separate branch in the first place and  
integrated with trunk when the project is stable.

sanjay
>
>
>  >1. the file length (as returned by getFileStatus) is incorrect
>
> May be the following work around will be useful.
> If you read from a file you always try to read more data than the  
> length reported
> by the name-node. How much more? The size of one block would be  
> enough, or
> even to the next (ceiling) block boundary.
>
>  >2. When an application comes up after a crash, it seems to hang  
> for about 60
>
> Don't have enough context on that, sorry.
>
> Thanks,
> --Konstantin
>
> Doug Judd wrote:
> > Sounds good.  I would much rather wait and have fsync() done  
> correctly in
> > 0.20 than get some sort of hacked version in 0.19.  I'll create a  
> couple of
> > issues and mark them for 0.20  Thanks.
> >
> > - Doug
> >
> > On Mon, Feb 2, 2009 at 1:51 PM, Owen O'Malley <[EMAIL PROTECTED]>  
> wrote:
> >
> >> On Feb 2, 2009, at 12:51 PM, Doug Judd wrote:
> >>
> >>  What do you recommend?  Is there anyway we could get these two  
> issues
> >>> fixed
> >>> for 0.19.1, or should I file issues for them and get them on the  
> schedule
> >>> for 0.19.2?
> >>>
> >> Given the outstanding problems and general level of uncertainty,  
> I'd favor
> >> releasing a 0.19.1 with the equivalent of the 0.18.3 disable on  
> fsync and
> >> append. Let's get them fixed in 0.20 first and then we can debate  
> whether
> >> the rewards of pushing them back into an 0.19.2 would make sense.  
> I'm pretty
> >> uncomfortable at the moment with how the entire functional  
> complex seems to
> >> cause a continuous stream of problems.
> >>
> >> -- Owen
> >>
> >
>

Sanjay Radia wrote:
>
> On Feb 2, 2009, at 4:23 PM, Konstantin Shvachko wrote:
>
>>
>>  >  What do you recommend?
>>
>> In general. There may be people/organizations, which will not compromise
>> on the reduced functionality in favor of the stability, this is
>> understandable.
>> I would propose to create a separate (unofficial experimental) branch,
>> which
>> would track changes like HADOOP-4379. The branch may later either die
>> when the
>> main stream is fixed or be merged with the trunk if the changes proved
>> to be stable.
>>
>
>
> This is very a interesting suggestion.
> Many in the team  have come to the conclusion that complex projects like
> append should be done on a separate branch in the first place and
> integrated with trunk when the project is stable.
>

There's a lot to be said for branching; I'm also looking at git so I can
do my service lifecycle stuff under SCM properly.

but the cost of merging can be high. I'd estimate 1 morning/week is
spent updating my local SVN and then seeing that everything still works.
If hudson could both test the branches and test any merged branches,
life would be better

The other problem is incompatible branches: the more branches you have
live, the higher the merge cost.

That said, Git promises wonderful things, and we ought to be able to set
up Apache support for git for people wanting to do their own branches
-svn would still be the official SCM tool

On Tue, Feb 3, 2009 at 7:02 PM, Sanjay Radia <[EMAIL PROTECTED]> wrote:

>
> Many in the team  have come to the conclusion that complex projects like
> append should be done on a separate branch in the first place and integrated
> with trunk when the project is stable.
>

How do we determine when a feature on the branch is 'stable'?  Is there a
test suite to run beyond hudson's running of unit tests?

St.Ack

On Thu, Jan 29, 2009 at 4:16 PM, Nigel Daley <[EMAIL PROTECTED]> wrote:
>
> I would like to propose that we apply this same "fix" to sync in 0.19.1 and
> 0.20.0.  Since append requires the full semantics of sync, I propose we also
> disable append (perhaps throw UnsupportedOperationException from API?).
>  Yes, this would unfortunately be an incompatible change between 0.19.0 and
> 0.19.1.  We can then take the time needed to fix append properly in 0.21.0.
>
+1 on a 0.19.1 release with a 0.18.3-like solution removing some of the sync
symantics.  A stable hdfs takes precedence.

Please count us hbasistas in when testing of append is needed.
Applications' like hbase are plain broke without it (You may have heard us
mention this fact once or twice at times in the past -- smile).

Good stuff,
St.Ack

On Feb 4, 2009, at 3:38 AM, Steve Loughran wrote:

> Sanjay Radia wrote:
> >
> > On Feb 2, 2009, at 4:23 PM, Konstantin Shvachko wrote:
> >
> >>
> >>  >  What do you recommend?
> >>
> >> In general. There may be people/organizations, which will not  
> compromise
> >> on the reduced functionality in favor of the stability, this is
> >> understandable.
> >> I would propose to create a separate (unofficial experimental)  
> branch,
> >> which
> >> would track changes like HADOOP-4379. The branch may later either  
> die
> >> when the
> >> main stream is fixed or be merged with the trunk if the changes  
> proved
> >> to be stable.
> >>
> >
> >
> > This is very a interesting suggestion.
> > Many in the team  have come to the conclusion that complex  
> projects like
> > append should be done on a separate branch in the first place and
> > integrated with trunk when the project is stable.
> >
>
> There's a lot to be said for branching; I'm also looking at git so I  
> can
> do my service lifecycle stuff under SCM properly.
>
> but the cost of merging can be high. I'd estimate 1 morning/week is
> spent updating my local SVN and then seeing that everything still  
> works.
> If hudson could both test the branches and test any merged branches,
> life would be better
>

I agree on the cost of merging.
When a project is branched,  after a while one can spend as much as  
30% of cycles merging
in changes.
But when a system is used in production to store data we cannot afford  
to have users loose their data.
The team at Yahoo had to scramble to recover the lost data, put in  
several emergency patches to deal with
the append code.

I am all for extending hudson testing for branches, but hudson  
testing, while helpful, will not be sufficient  for big
projects because hudson does not have a comprehensive set of tests.  
Each new release is tested significantly beyond the hudson tests.

For me the lesson is that large complex projects should be branched.
(This is how commercial software products are engineered).
There will increased cost to the project team, but over all, the  
community  will have more solid releases and the total cost to the  
community  in delivering the techology will be smaller.

sanjay
>
>
> The other problem is incompatible branches: the more branches you have
> live, the higher the merge cost.
>
> That said, Git promises wonderful things, and we ought to be able to  
> set
> up Apache support for git for people wanting to do their own branches
> -svn would still be the official SCM tool
>

Sanjay Radia wrote:
> For me the lesson is that large complex projects should be branched.

We already maintain release branches.  What's under discussion is the
maintenance of feature branches.  We do this today through patch files,
merging each time they are applied.  The proposal is to use a source
code management tool to manage feature branches, which would be merged
less often, but using better merge tools.

FWIW, Nutch's transition to mapreduce (in the nascent days of Hadoop)
was managed as a feature branch.  Mike & I worked in a branch for about
six months before merging back to the trunk.  For a change of that
magnitude, we found this to be much simpler than updating a patch.

So we need concrete proposals of features that deserve a branches.  In
retrospect, it seems like append may have been better handled in a
branch than as a patch, but that's hindsight.  What future features do
we feel demand this?

One possibility is to manage the project split in a branch.  We could
start new repos for the hdfs and mapred sub-projects, but branch the
core repo.  Then changes could continue to be applied to trunk while the
details of the split are worked out.  Core changes would be merged to
the new subprojects and the branch while this is in progress. Once we
feel the split is solid, we can merge the core branch to trunk and open
the new subprojects for business.

If we change the RPC system, or switch DFS client to use RPC instead of
sockets, etc., we might want to do these in a branch since they'll touch
a lot of code and will require extensive testing before we release them.

I don't think this is fundamentally a policy issue.  We still want to
demand that things are well tested before we commit them to trunk.  The
append code was committed to trunk prematurely, perhaps since managing
as a patch was awkward.  So this is a praxis issue.  For features that
take a long-time to develop, that we do not want to be forced to
prematurely commit, a branch is perhaps a better mechanism than a patch.

So I think, if a committer feels a feature requires a branch, then they
can propose that, and if no one objects, they can create it and maintain
it.  The final commit to trunk is what we should watch most closely,
since that's the event that corresponds to a commit today.  Commits to a
feature branch should not require reviews, since these are equivalent to
updating a patch.

Does this sound reasonable?

Doug

On Feb 6, 2009, at 10:35 AM, Doug Cutting wrote:

> Sanjay Radia wrote:
> > For me the lesson is that large complex projects should be branched.
>
> We already maintain release branches.  What's under discussion is the
> maintenance of feature branches.  We do this today through patch  
> files,
> merging each time they are applied.  The proposal is to use a source
> code management tool to manage feature branches, which would be merged
> less often, but using better merge tools.
>
> FWIW, Nutch's transition to mapreduce (in the nascent days of Hadoop)
> was managed as a feature branch.  Mike & I worked in a branch for  
> about
> six months before merging back to the trunk.  For a change of that
> magnitude, we found this to be much simpler than updating a patch.
>
> So we need concrete proposals of features that deserve a branches.  In
> retrospect, it seems like append may have been better handled in a
> branch than as a patch, but that's hindsight.  What future features do
> we feel demand this?
>
> One possibility is to manage the project split in a branch.  We could
> start new repos for the hdfs and mapred sub-projects, but branch the
> core repo.  Then changes could continue to be applied to trunk while  
> the
> details of the split are worked out.  Core changes would be merged to
> the new subprojects and the branch while this is in progress. Once we
> feel the split is solid, we can merge the core branch to trunk and  
> open
> the new subprojects for business.
>
> If we change the RPC system, or switch DFS client to use RPC instead  
> of
> sockets, etc., we might want to do these in a branch since they'll  
> touch
> a lot of code and will require extensive testing before we release  
> them.
>
> I don't think this is fundamentally a policy issue.  We still want to
> demand that things are well tested before we commit them to trunk.  
> The
> append code was committed to trunk prematurely, perhaps since managing
> as a patch was awkward.  So this is a praxis issue.  For features that
> take a long-time to develop, that we do not want to be forced to
> prematurely commit, a branch is perhaps a better mechanism than a  
> patch.
>
> So I think, if a committer feels a feature requires a branch, then  
> they
> can propose that, and if no one objects, they can create it and  
> maintain
> it.  The final commit to trunk is what we should watch most closely,
> since that's the event that corresponds to a commit today.  Commits  
> to a
> feature branch should not require reviews, since these are  
> equivalent to
> updating a patch.
>
> Does this sound reasonable?
>

yes.

>  Commits to a
> feature branch should not require reviews, since these are  
> equivalent to
> updating a patch.

Agree,  but it would be wise for the community to get their feedback  
to the project team
earlier rather than later when the big patch to the trunk occurs.
Can Jiras be used to discuss a branch? Or would the project team  
discuss this via another mechanism
(such as email).
sanjay

>
>
> Doug
>

Sanjay Radia wrote:
> On Feb 6, 2009, at 10:35 AM, Doug Cutting wrote:
>>  Commits to a
>> feature branch should not require reviews, since these are equivalent to
>> updating a patch.
>
> Agree,  but it would be wise for the community to get their feedback to
> the project team
> earlier rather than later when the big patch to the trunk occurs.
> Can Jiras be used to discuss a branch? Or would the project team discuss
> this via another mechanism
> (such as email).

Commits to a feature branch will send a message to the dev list, like
any other commit.  And when folks commit to a feature branch, they
should reference the Jira issue id, as in any other commit, so that
folks browsing Jira can see the commits.

When someone starts a feature branch they should note it in the Jira
issue, so that folks know to browse the "Subversion Commits" tab to see
the patch history.  I'd expect this to proceed as follows:

   1. A comment proposing that a feature branch be added.

   2. A comment by a different committer, endorsing the feature branch,
and no comments objecting.

   3. A comment stating that the feature branch has been added, what
it's url is, and that folks should henceforth use the "Subversion
Commits" or "All" tab to track the issue.

   4. Committers can commit directly to the feature branch, without
reviews.  Since committers must have a CLA on file, Apache license is
assumed.

   5. Non-committers can submit patches against the feature branch to
the issue in Jira.  These would require the license signoff as usual.

Everything would still be in public, on the dev list, as now.

Note that we do *not* want feature branches in external repositories,
since commits there would not generate commit messages to the dev list
nor would they generate links in Jira, etc.

Doug

Sanjay,

> The team at Yahoo had to scramble to recover the lost data,
> put in several emergency patches to deal with
> the append code.
>
This makes it sounds like all of our lost blocks were due to append.
Yes, we did lose many blocks/files in the past few months, but I'd say
it was more of a combination of bugs and not solely from append.

Some that I remember:

HADOOP-4556: in 0.17.2. Nothing to do with append.
HADOOP-4935: Probably more from the new feature of replicating corrupted
blocks (that I requested...)
and append related issues.

I have no opinion about branching, but can we assume that data loss
could happen again and think about a slightly better snapshot?

Current '-upgrade' almost works as long as I have a 'renew!' button
instead of requiring a dfs restart.

Koji
-----Original Message-----
From: Sanjay Radia [mailto:[EMAIL PROTECTED]]
Sent: Thursday, February 05, 2009 4:25 PM
To: [EMAIL PROTECTED]
Subject: Re: Hadoop 0.19.1
On Feb 4, 2009, at 3:38 AM, Steve Loughran wrote:

> Sanjay Radia wrote:
> >
> > On Feb 2, 2009, at 4:23 PM, Konstantin Shvachko wrote:
> >
> >>
> >>  >  What do you recommend?
> >>
> >> In general. There may be people/organizations, which will not  
> compromise
> >> on the reduced functionality in favor of the stability, this is
> >> understandable.
> >> I would propose to create a separate (unofficial experimental)  
> branch,
> >> which
> >> would track changes like HADOOP-4379. The branch may later either  
> die
> >> when the
> >> main stream is fixed or be merged with the trunk if the changes  
> proved
> >> to be stable.
> >>
> >
> >
> > This is very a interesting suggestion.
> > Many in the team  have come to the conclusion that complex  
> projects like
> > append should be done on a separate branch in the first place and
> > integrated with trunk when the project is stable.
> >
>
> There's a lot to be said for branching; I'm also looking at git so I  
> can
> do my service lifecycle stuff under SCM properly.
>
> but the cost of merging can be high. I'd estimate 1 morning/week is
> spent updating my local SVN and then seeing that everything still  
> works.
> If hudson could both test the branches and test any merged branches,
> life would be better
>

I agree on the cost of merging.
When a project is branched,  after a while one can spend as much as  
30% of cycles merging
in changes.
But when a system is used in production to store data we cannot afford  
to have users loose their data.
The team at Yahoo had to scramble to recover the lost data, put in  
several emergency patches to deal with
the append code.

I am all for extending hudson testing for branches, but hudson  
testing, while helpful, will not be sufficient  for big
projects because hudson does not have a comprehensive set of tests.  
Each new release is tested significantly beyond the hudson tests.

For me the lesson is that large complex projects should be branched.
(This is how commercial software products are engineered).
There will increased cost to the project team, but over all, the  
community  will have more solid releases and the total cost to the  
community  in delivering the techology will be smaller.

sanjay
>
>
> The other problem is incompatible branches: the more branches you have
> live, the higher the merge cost.
>
> That said, Git promises wonderful things, and we ought to be able to  
> set
> up Apache support for git for people wanting to do their own branches
> -svn would still be the official SCM tool
>

Doug Cutting wrote:
> Commits to a feature branch will send a message to the dev list, like
> any other commit.  And when folks commit to a feature branch, they
> should reference the Jira issue id, as in any other commit, so that
> folks browsing Jira can see the commits.

What would be sweet is if commit messages contained a link to the Jira
issue.  This would require enhancing the mailer.py script that Apache's
subversion repo uses.

http://subversion.tigris.org/tools_contrib.html#mailer_py

Any Python hacking committers want to take this on?

Doug

Doug Cutting wrote:
> Commits to a feature branch will send a message to the dev list, like
> any other commit.  And when folks commit to a feature branch, they
> should reference the Jira issue id, as in any other commit, so that
> folks browsing Jira can see the commits.
>
> When someone starts a feature branch they should note it in the Jira
> issue, so that folks know to browse the "Subversion Commits" tab to see
> the patch history.  I'd expect this to proceed as follows:
>
>   1. A comment proposing that a feature branch be added.
>
>   2. A comment by a different committer, endorsing the feature branch,
> and no comments objecting.
>
>   3. A comment stating that the feature branch has been added, what it's
> url is, and that folks should henceforth use the "Subversion Commits" or
> "All" tab to track the issue.
>
>   4. Committers can commit directly to the feature branch, without
> reviews.  Since committers must have a CLA on file, Apache license is
> assumed.
>
>   5. Non-committers can submit patches against the feature branch to the
> issue in Jira.  These would require the license signoff as usual.

+1. I agree: no review requirement for feature branches, and 1-5.
I would add to this (6) merging a feature branch to an official branch
goes through regular patch process, that is, a new jira is created with
the patch attachment, which now goes through the review process.

> Everything would still be in public, on the dev list, as now.
>
> Note that we do *not* want feature branches in external repositories,
> since commits there would not generate commit messages to the dev list
> nor would they generate links in Jira, etc.

Didn't understand this: external to what? What is internal repository?

--Konstantin

Konstantin Shvachko wrote:
> +1. I agree: no review requirement for feature branches, and 1-5.
> I would add to this (6) merging a feature branch to an official branch
> goes through regular patch process, that is, a new jira is created with
> the patch attachment, which now goes through the review process.

Yes, I forgot (6).  But I don't think a new Jira is required.  The
branch is equivalent to a patch.  If folks review and +1 it, then it can
be merged to trunk.  An branch-backed issue shouldn't be marked as
"patch available" until it's ready to be reviewed.  Once it has been
merged to trunk then the branch can be removed.

>> Note that we do *not* want feature branches in external repositories,
>> since commits there would not generate commit messages to the dev list
>> nor would they generate links in Jira, etc.
>
> Didn't understand this: external to what? What is internal repository?

External to Apache's SVN, i.e., a Git repo on your own server.

Doug

+1 to branching described above. Good to see experimental branches
considered not-too-evil.

7) If something is not covered in 1-6 common sense should prevail :-)

Raghu.

Doug Cutting wrote:
> Konstantin Shvachko wrote:
>> +1. I agree: no review requirement for feature branches, and 1-5.
>> I would add to this (6) merging a feature branch to an official branch
>> goes through regular patch process, that is, a new jira is created with
>> the patch attachment, which now goes through the review process.
>
> Yes, I forgot (6).  But I don't think a new Jira is required.  The
> branch is equivalent to a patch.  If folks review and +1 it, then it can
> be merged to trunk.  An branch-backed issue shouldn't be marked as
> "patch available" until it's ready to be reviewed.  Once it has been
> merged to trunk then the branch can be removed.
>
>>> Note that we do *not* want feature branches in external repositories,
>>> since commits there would not generate commit messages to the dev
>>> list nor would they generate links in Jira, etc.
>>
>> Didn't understand this: external to what? What is internal repository?
>
> External to Apache's SVN, i.e., a Git repo on your own server.
>
> Doug

Doug Cutting wrote:
> Sanjay Radia wrote:
>> For me the lesson is that large complex projects should be branched.

I currently estimate my merge costs of 20% -one day a week- though a lot
of that is test run time
> If we change the RPC system, or switch DFS client to use RPC instead of
> sockets, etc., we might want to do these in a branch since they'll touch
> a lot of code and will require extensive testing before we release them.
>
> I don't think this is fundamentally a policy issue.  We still want to
> demand that things are well tested before we commit them to trunk.  The
> append code was committed to trunk prematurely, perhaps since managing
> as a patch was awkward.  So this is a praxis issue.  For features that
> take a long-time to develop, that we do not want to be forced to
> prematurely commit, a branch is perhaps a better mechanism than a patch.
>
> So I think, if a committer feels a feature requires a branch, then they
> can propose that, and if no one objects, they can create it and maintain
> it.  The final commit to trunk is what we should watch most closely,
> since that's the event that corresponds to a commit today.  Commits to a
> feature branch should not require reviews, since these are equivalent to
> updating a patch.
>
> Does this sound reasonable?

this sounds good, I would propose my lifecycle changes as an example
  -changes bits at the bottom
  -adds lots of tests
  -needs/encourages review
  -lets me do my work under SCM
  -let's me hook our work Hudson servers up to it

Putting the stuff in a branch makes it easier for people to work on it,
to check it out and build it etc.

-steve

Konstantin Shvachko wrote:
> Doug Cutting wrote:
>> Commits to a feature branch will send a message to the dev list, like
>> any other commit.  And when folks commit to a feature branch, they
>> should reference the Jira issue id, as in any other commit, so that
>> folks browsing Jira can see the commits.
>>
>> When someone starts a feature branch they should note it in the Jira
>> issue, so that folks know to browse the "Subversion Commits" tab to
>> see the patch history.  I'd expect this to proceed as follows:
>>
>>   1. A comment proposing that a feature branch be added.
>>
>>   2. A comment by a different committer, endorsing the feature branch,
>> and no comments objecting.
>>
>>   3. A comment stating that the feature branch has been added, what
>> it's url is, and that folks should henceforth use the "Subversion
>> Commits" or "All" tab to track the issue.
>>
>>   4. Committers can commit directly to the feature branch, without
>> reviews.  Since committers must have a CLA on file, Apache license is
>> assumed.
>>
>>   5. Non-committers can submit patches against the feature branch to
>> the issue in Jira.  These would require the license signoff as usual.
>
> +1. I agree: no review requirement for feature branches, and 1-5.
> I would add to this (6) merging a feature branch to an official branch
> goes through regular patch process, that is, a new jira is created with
> the patch attachment, which now goes through the review process.

I'm not sure about the merge, but it is possible. What is important is
that the branches get reviewed regularly before the big commit day. For
those active branches, that means that we may want some oversight/review
process, and an action plan to deal with unmaintained branches (turn to
jira patch, remove the branch?)
>
>> Everything would still be in public, on the dev list, as now.
>>
>> Note that we do *not* want feature branches in external repositories,
>> since commits there would not generate commit messages to the dev list
>> nor would they generate links in Jira, etc.
>
I'm guessing Doug means Git repos and the like.

Now. apache is starting to set up some Git/SVN sync via github, for
efficient branching . That's the alternate place for me to go with my
code, though I quite like SVN myself.

Also, I am not above using HADOOP- issue tags in the commits to our
sourceforge hosted repository...Jira does have the ability to scan other
repositories if needed.

It sounds like we agree that we should release 0.19.1 with append  
throwing a UnsupportedOperationException and sync getting the same  
reduced semantics as 0.18.3.

I have filed 2 Jira's to get this done:
http://issues.apache.org/jira/browse/HADOOP-5224
http://issues.apache.org/jira/browse/HADOOP-5225

When these are committed, I will roll a Hadoop 0.19.1 candidate release.

Nige

On Feb 2, 2009, at 2:02 PM, Doug Judd wrote:

> Sounds good.  I would much rather wait and have fsync() done  
> correctly in
> 0.20 than get some sort of hacked version in 0.19.  I'll create a  
> couple of
> issues and mark them for 0.20  Thanks.
>
> - Doug
>
> On Mon, Feb 2, 2009 at 1:51 PM, Owen O'Malley <[EMAIL PROTECTED]>  
> wrote:
>
>> On Feb 2, 2009, at 12:51 PM, Doug Judd wrote:
>>
>> What do you recommend?  Is there anyway we could get these two issues
>>> fixed
>>> for 0.19.1, or should I file issues for them and get them on the  
>>> schedule
>>> for 0.19.2?
>>>
>>
>> Given the outstanding problems and general level of uncertainty,  
>> I'd favor
>> releasing a 0.19.1 with the equivalent of the 0.18.3 disable on  
>> fsync and
>> append. Let's get them fixed in 0.20 first and then we can debate  
>> whether
>> the rewards of pushing them back into an 0.19.2 would make sense.  
>> I'm pretty
>> uncomfortable at the moment with how the entire functional complex  
>> seems to
>> cause a continuous stream of problems.
>>
>> -- Owen
>>

Looks good to me ( I saw this just now). Thanks for rolling 0.19.1.

dhruba

On Wed, Feb 11, 2009 at 12:47 PM, Nigel Daley <[EMAIL PROTECTED]> wrote:

> It sounds like we agree that we should release 0.19.1 with append throwing
> a UnsupportedOperationException and sync getting the same reduced semantics
> as 0.18.3.
>
> I have filed 2 Jira's to get this done:
> http://issues.apache.org/jira/browse/HADOOP-5224
> http://issues.apache.org/jira/browse/HADOOP-5225
>
> When these are committed, I will roll a Hadoop 0.19.1 candidate release.
>
> Nige
>
>
> On Feb 2, 2009, at 2:02 PM, Doug Judd wrote:
>
>  Sounds good.  I would much rather wait and have fsync() done correctly in
>> 0.20 than get some sort of hacked version in 0.19.  I'll create a couple
>> of
>> issues and mark them for 0.20  Thanks.
>>
>> - Doug
>>
>> On Mon, Feb 2, 2009 at 1:51 PM, Owen O'Malley <[EMAIL PROTECTED]> wrote:
>>
>>  On Feb 2, 2009, at 12:51 PM, Doug Judd wrote:
>>>
>>> What do you recommend?  Is there anyway we could get these two issues
>>>
>>>> fixed
>>>> for 0.19.1, or should I file issues for them and get them on the
>>>> schedule
>>>> for 0.19.2?
>>>>
>>>>
>>> Given the outstanding problems and general level of uncertainty, I'd
>>> favor
>>> releasing a 0.19.1 with the equivalent of the 0.18.3 disable on fsync and
>>> append. Let's get them fixed in 0.20 first and then we can debate whether
>>> the rewards of pushing them back into an 0.19.2 would make sense. I'm
>>> pretty
>>> uncomfortable at the moment with how the entire functional complex seems
>>> to
>>> cause a continuous stream of problems.
>>>
>>> -- Owen
>>>
>>>
>

                         

----- Original Message -----
From: Steve Loughran <[EMAIL PROTECTED]>
To: [EMAIL PROTECTED] <[EMAIL PROTECTED]>
Sent: Mon Feb 09 08:39:27 2009
Subject: Re: Hadoop 0.19.1

Konstantin Shvachko wrote:
> Doug Cutting wrote:
>> Commits to a feature branch will send a message to the dev list, like
>> any other commit.  And when folks commit to a feature branch, they
>> should reference the Jira issue id, as in any other commit, so that
>> folks browsing Jira can see the commits.
>>
>> When someone starts a feature branch they should note it in the Jira
>> issue, so that folks know to browse the "Subversion Commits" tab to
>> see the patch history.  I'd expect this to proceed as follows:
>>
>>   1. A comment proposing that a feature branch be added.
>>
>>   2. A comment by a different committer, endorsing the feature branch,
>> and no comments objecting.
>>
>>   3. A comment stating that the feature branch has been added, what
>> it's url is, and that folks should henceforth use the "Subversion
>> Commits" or "All" tab to track the issue.
>>
>>   4. Committers can commit directly to the feature branch, without
>> reviews.  Since committers must have a CLA on file, Apache license is
>> assumed.
>>
>>   5. Non-committers can submit patches against the feature branch to
>> the issue in Jira.  These would require the license signoff as usual.
>
> +1. I agree: no review requirement for feature branches, and 1-5.
> I would add to this (6) merging a feature branch to an official branch
> goes through regular patch process, that is, a new jira is created with
> the patch attachment, which now goes through the review process.

I'm not sure about the merge, but it is possible. What is important is
that the branches get reviewed regularly before the big commit day. For
those active branches, that means that we may want some oversight/review
process, and an action plan to deal with unmaintained branches (turn to
jira patch, remove the branch?)
>
>> Everything would still be in public, on the dev list, as now.
>>
>> Note that we do *not* want feature branches in external repositories,
>> since commits there would not generate commit messages to the dev list
>> nor would they generate links in Jira, etc.
>
I'm guessing Doug means Git repos and the like.

Now. apache is starting to set up some Git/SVN sync via github, for
efficient branching . That's the alternate place for me to go with my
code, though I quite like SVN myself.

Also, I am not above using HADOOP- issue tags in the commits to our
sourceforge hosted repository...Jira does have the ability to scan other
repositories if needed.

----- Original Message -----
From: Steve Loughran <[EMAIL PROTECTED]>
To: [EMAIL PROTECTED] <[EMAIL PROTECTED]>
Sent: Mon Feb 09 08:39:27 2009
Subject: Re: Hadoop 0.19.1

Konstantin Shvachko wrote:
> Doug Cutting wrote:
>> Commits to a feature branch will send a message to the dev list, like
>> any other commit.  And when folks commit to a feature branch, they
>> should reference the Jira issue id, as in any other commit, so that
>> folks browsing Jira can see the commits.
>>
>> When someone starts a feature branch they should note it in the Jira
>> issue, so that folks know to browse the "Subversion Commits" tab to
>> see the patch history.  I'd expect this to proceed as follows:
>>
>>   1. A comment proposing that a feature branch be added.
>>
>>   2. A comment by a different committer, endorsing the feature branch,
>> and no comments objecting.
>>
>>   3. A comment stating that the feature branch has been added, what
>> it's url is, and that folks should henceforth use the "Subversion
>> Commits" or "All" tab to track the issue.
>>
>>   4. Committers can commit directly to the feature branch, without
>> reviews.  Since committers must have a CLA on file, Apache license is
>> assumed.
>>
>>   5. Non-committers can submit patches against the feature branch to
>> the issue in Jira.  These would require the license signoff as usual.
>
> +1. I agree: no review requirement for feature branches, and 1-5.
> I would add to this (6) merging a feature branch to an official branch
> goes through regular patch process, that is, a new jira is created with
> the patch attachment, which now goes through the review process.

I'm not sure about the merge, but it is possible. What is important is
that the branches get reviewed regularly before the big commit day. For
those active branches, that means that we may want some oversight/review
process, and an action plan to deal with unmaintained branches (turn to
jira patch, remove the branch?)
>
>> Everything would still be in public, on the dev list, as now.
>>
>> Note that we do *not* want feature branches in external repositories,
>> since commits there would not generate commit messages to the dev list
>> nor would they generate links in Jira, etc.
>
I'm guessing Doug means Git repos and the like.

Now. apache is starting to set up some Git/SVN sync via github, for
efficient branching . That's the alternate place for me to go with my
code, though I quite like SVN myself.

Also, I am not above using HADOOP- issue tags in the commits to our
sourceforge hosted repository...Jira does have the ability to scan other
repositories if needed.

 

----- Original Message -----
From: [EMAIL PROTECTED] <[EMAIL PROTECTED]>
To: [EMAIL PROTECTED] <[EMAIL PROTECTED]>
Sent: Wed Feb 04 10:47:45 2009
Subject: Re: Hadoop 0.19.1

On Tue, Feb 3, 2009 at 7:02 PM, Sanjay Radia <[EMAIL PROTECTED]> wrote:

>
> Many in the team  have come to the conclusion that complex projects like
> append should be done on a separate branch in the first place and integrated
> with trunk when the project is stable.
>

How do we determine when a feature on the branch is 'stable'?  Is there a
test suite to run beyond hudson's running of unit tests?

St.Ack

----- Original Message -----
From: [EMAIL PROTECTED] <[EMAIL PROTECTED]>
To: [EMAIL PROTECTED] <[EMAIL PROTECTED]>
Sent: Wed Feb 04 10:47:45 2009
Subject: Re: Hadoop 0.19.1

On Tue, Feb 3, 2009 at 7:02 PM, Sanjay Radia <[EMAIL PROTECTED]> wrote:

>
> Many in the team  have come to the conclusion that complex projects like
> append should be done on a separate branch in the first place and integrated
> with trunk when the project is stable.
>

How do we determine when a feature on the branch is 'stable'?  Is there a
test suite to run beyond hudson's running of unit tests?

St.Ack

Sanjay Radia wrote:
>  
>
> ----- Original Message -----
> From: Steve Loughran <[EMAIL PROTECTED]>
> To: [EMAIL PROTECTED] <[EMAIL PROTECTED]>
> Sent: Mon Feb 09 08:39:27 2009
> Subject: Re: Hadoop 0.19.1
>
> Konstantin Shvachko wrote:
>> Doug Cutting wrote:
>>> Commits to a feature branch will send a message to the dev list, like
>>> any other commit.  And when folks commit to a feature branch, they
>>> should reference the Jira issue id, as in any other commit, so that
>>> folks browsing Jira can see the commits.
>>>
>>> When someone starts a feature branch they should note it in the Jira
>>> issue, so that folks know to browse the "Subversion Commits" tab to
>>> see the patch history.  I'd expect this to proceed as follows:
>>>
>>>   1. A comment proposing that a feature branch be added.
>>>
>>>   2. A comment by a different committer, endorsing the feature branch,
>>> and no comments objecting.
>>>
>>>   3. A comment stating that the feature branch has been added, what
>>> it's url is, and that folks should henceforth use the "Subversion
>>> Commits" or "All" tab to track the issue.
>>>
>>>   4. Committers can commit directly to the feature branch, without
>>> reviews.  Since committers must have a CLA on file, Apache license is
>>> assumed.
>>>
>>>   5. Non-committers can submit patches against the feature branch to
>>> the issue in Jira.  These would require the license signoff as usual.
>> +1. I agree: no review requirement for feature branches, and 1-5.
>> I would add to this (6) merging a feature branch to an official branch
>> goes through regular patch process, that is, a new jira is created with
>> the patch attachment, which now goes through the review process.
>
> I'm not sure about the merge, but it is possible. What is important is
> that the branches get reviewed regularly before the big commit day. For
> those active branches, that means that we may want some oversight/review
> process, and an action plan to deal with unmaintained branches (turn to
> jira patch, remove the branch?)

+1

>
>
>>> Everything would still be in public, on the dev list, as now.
>>>
>>> Note that we do *not* want feature branches in external repositories,
>>> since commits there would not generate commit messages to the dev list
>>> nor would they generate links in Jira, etc.
>
>
> I'm guessing Doug means Git repos and the like.
>
> Now. apache is starting to set up some Git/SVN sync via github, for
> efficient branching . That's the alternate place for me to go with my
> code, though I quite like SVN myself.
>
> Also, I am not above using HADOOP- issue tags in the commits to our
> sourceforge hosted repository...Jira does have the ability to scan other
> repositories if needed.

I don't know about Github; IDEA 8.1 is adding git support though. That's
the other place I would put stuff, as it would let me share it while the
lawyers tick things off their lists.

One thing about Git is that it has a more laid back notion of what is
"trunk"; you'd end up with more a blurred distro, with -maybe- the Y!
production scheme, the Cloudera branch, the steves-modified-branch, etc,
etc -with people able to pick and choose which to merge in. There's less
of a trunk+branches, more just a set of branches.

I dont know how well things like major refactorings with directories
being moved about goes down in this world, something I'd be reluctant to
play with.

That leads to the problem that Doug was talking about.  If you stay with the
Apache SVN, then JIRA can see the changes as they happen.  Likewise, hudson
would have a harder time knowing what to base the patch on.

On Mon, Feb 16, 2009 at 8:23 AM, Steve Loughran <[EMAIL PROTECTED]> wrote:

> I don't know about Github; IDEA 8.1 is adding git support though. That's
> the other place I would put stuff, as it would let me share it while the
> lawyers tick things off their lists.
>
> One thing about Git is that it has a more laid back notion of what is
> "trunk"; you'd end up with more a blurred distro, with -maybe- the Y!
> production scheme, the Cloudera branch, the steves-modified-branch, etc, etc
> -with people able to pick and choose which to merge in. There's less of a
> trunk+branches, more just a set of branches.
>

--
Ted Dunning, CTO
DeepDyve
4600 Bohannon Drive, Suite 220
Menlo Park, CA 94025
www.deepdyve.com
650-324-0110, ext. 738
858-414-0013 (m)

Steve Loughran wrote:
> One thing about Git is that it has a more laid back notion of what is
> "trunk"; you'd end up with more a blurred distro, with -maybe- the Y!
> production scheme, the Cloudera branch, the steves-modified-branch, etc,
> etc -with people able to pick and choose which to merge in. There's less
> of a trunk+branches, more just a set of branches.

Yes, but trunk is what we share as a community, our consensus reality.
We should be careful not to overly fragment our community.  We can't
vote on a blurred release.  The ASF development style will change,
better embracing tools like Git, but we should change it slowly and
cautiously lest we lose what we value.

Doug