Hi.

Any idea if RandomAccessFile is going to be supported in HDFS?

Regards.

RandomAccessFile isn't supported directly, but you can seek when
reading from files in HDFS (see FSDataInputStream's seek() method).
Writing at an arbitrary offset in an HDFS file is not supported
however.

Cheers,
Tom

On Sun, May 24, 2009 at 1:33 PM, Stas Oskin <[EMAIL PROTECTED]> wrote:
> Hi.
>
> Any idea if RandomAccessFile is going to be supported in HDFS?
>
> Regards.
>

Hi.

I understand that the "append files" functionality is going to provide
Random Access support?

Any idea which version this expected to be?

Regards.

2009/5/25 Tom White <[EMAIL PROTECTED]>

> RandomAccessFile isn't supported directly, but you can seek when
> reading from files in HDFS (see FSDataInputStream's seek() method).
> Writing at an arbitrary offset in an HDFS file is not supported
> however.
>
> Cheers,
> Tom
>
> On Sun, May 24, 2009 at 1:33 PM, Stas Oskin <[EMAIL PROTECTED]> wrote:
> > Hi.
> >
> > Any idea if RandomAccessFile is going to be supported in HDFS?
> >
> > Regards.
> >
>

Hi.

Any idea when the "append" functionality is expected?

Regards.

On Sep 13, 2009, at 3:08 AM, Stas Oskin wrote:

> Hi.
>
> Any idea when the "append" functionality is expected?

A working append is a blocker on HDFS 0.21.0.

The code for append is expected to be complete in a few weeks.  
Meanwhile, the rest of Common, HDFS, and MapReduce have feature-frozen  
and need to be stabilized and all of the critical bugs fixed. I'd  
expect the first releases of 0.21.0 in early November.

-- Owen

Hi.

Just to understand the road-map, 0.21 will be the first stable "append"
implementation?

Regards.

2009/9/20 Owen O'Malley <[EMAIL PROTECTED]>

>
> On Sep 13, 2009, at 3:08 AM, Stas Oskin wrote:
>
>  Hi.
>>
>> Any idea when the "append" functionality is expected?
>>
>
> A working append is a blocker on HDFS 0.21.0.
>
> The code for append is expected to be complete in a few weeks. Meanwhile,
> the rest of Common, HDFS, and MapReduce have feature-frozen and need to be
> stabilized and all of the critical bugs fixed. I'd expect the first releases
> of 0.21.0 in early November.
>
> -- Owen
>

Or maybe more pessimistically, the second "stable" append implementation.

It's not like HADOOP-1700 wasn't intended to work. It was just found not to
after the fact. Hopefully this reimplementation will succeed. If you're
running a cluster that contains mission-critical data that cannot tolerate
corruption or loss, you shouldn't jump on the new-feature bandwagon until
it's had time to prove itself in the wild.

But yes, we hope that appends will really-truly work in 0.21.
Experimental/R&D projects should be able to plan on having a working append
function in 0.21.

- Aaron

On Sun, Sep 20, 2009 at 3:58 PM, Stas Oskin <[EMAIL PROTECTED]> wrote:

> Hi.
>
> Just to understand the road-map, 0.21 will be the first stable "append"
> implementation?
>
> Regards.
>
> 2009/9/20 Owen O'Malley <[EMAIL PROTECTED]>
>
> >
> > On Sep 13, 2009, at 3:08 AM, Stas Oskin wrote:
> >
> >  Hi.
> >>
> >> Any idea when the "append" functionality is expected?
> >>
> >
> > A working append is a blocker on HDFS 0.21.0.
> >
> > The code for append is expected to be complete in a few weeks. Meanwhile,
> > the rest of Common, HDFS, and MapReduce have feature-frozen and need to
> be
> > stabilized and all of the critical bugs fixed. I'd expect the first
> releases
> > of 0.21.0 in early November.
> >
> > -- Owen
> >
>

Thanks, I exactly wanted to ask this for our R&D roadmap.

2009/9/23 Aaron Kimball <[EMAIL PROTECTED]>

> Or maybe more pessimistically, the second "stable" append implementation.
>
> It's not like HADOOP-1700 wasn't intended to work. It was just found not to
> after the fact. Hopefully this reimplementation will succeed. If you're
> running a cluster that contains mission-critical data that cannot tolerate
> corruption or loss, you shouldn't jump on the new-feature bandwagon until
> it's had time to prove itself in the wild.
>
> But yes, we hope that appends will really-truly work in 0.21.
> Experimental/R&D projects should be able to plan on having a working append
> function in 0.21.
>
> - Aaron
>
> On Sun, Sep 20, 2009 at 3:58 PM, Stas Oskin <[EMAIL PROTECTED]> wrote:
>
> > Hi.
> >
> > Just to understand the road-map, 0.21 will be the first stable "append"
> > implementation?
> >
> > Regards.
> >
> > 2009/9/20 Owen O'Malley <[EMAIL PROTECTED]>
> >
> > >
> > > On Sep 13, 2009, at 3:08 AM, Stas Oskin wrote:
> > >
> > >  Hi.
> > >>
> > >> Any idea when the "append" functionality is expected?
> > >>
> > >
> > > A working append is a blocker on HDFS 0.21.0.
> > >
> > > The code for append is expected to be complete in a few weeks.
> Meanwhile,
> > > the rest of Common, HDFS, and MapReduce have feature-frozen and need to
> > be
> > > stabilized and all of the critical bugs fixed. I'd expect the first
> > releases
> > > of 0.21.0 in early November.
> > >
> > > -- Owen
> > >
> >
>

By the way, something that I forgot to ask before:

Will the append implementation be as fast, as the write/copy functions?

Regards.

2009/9/23 Stas Oskin <[EMAIL PROTECTED]>

> Thanks, I exactly wanted to ask this for our R&D roadmap.
>
> 2009/9/23 Aaron Kimball <[EMAIL PROTECTED]>
>
> Or maybe more pessimistically, the second "stable" append implementation.
>>
>> It's not like HADOOP-1700 wasn't intended to work. It was just found not
>> to
>> after the fact. Hopefully this reimplementation will succeed. If you're
>> running a cluster that contains mission-critical data that cannot tolerate
>> corruption or loss, you shouldn't jump on the new-feature bandwagon until
>> it's had time to prove itself in the wild.
>>
>> But yes, we hope that appends will really-truly work in 0.21.
>> Experimental/R&D projects should be able to plan on having a working
>> append
>> function in 0.21.
>>
>> - Aaron
>>
>> On Sun, Sep 20, 2009 at 3:58 PM, Stas Oskin <[EMAIL PROTECTED]> wrote:
>>
>> > Hi.
>> >
>> > Just to understand the road-map, 0.21 will be the first stable "append"
>> > implementation?
>> >
>> > Regards.
>> >
>> > 2009/9/20 Owen O'Malley <[EMAIL PROTECTED]>
>> >
>> > >
>> > > On Sep 13, 2009, at 3:08 AM, Stas Oskin wrote:
>> > >
>> > >  Hi.
>> > >>
>> > >> Any idea when the "append" functionality is expected?
>> > >>
>> > >
>> > > A working append is a blocker on HDFS 0.21.0.
>> > >
>> > > The code for append is expected to be complete in a few weeks.
>> Meanwhile,
>> > > the rest of Common, HDFS, and MapReduce have feature-frozen and need
>> to
>> > be
>> > > stabilized and all of the critical bugs fixed. I'd expect the first
>> > releases
>> > > of 0.21.0 in early November.
>> > >
>> > > -- Owen
>> > >
>> >
>>
>

On Wed, Sep 23, 2009 at 3:28 AM, Stas Oskin <[EMAIL PROTECTED]> wrote:
> By the way, something that I forgot to ask before:
>
> Will the append implementation be as fast, as the write/copy functions?
>
> Regards.
>
> 2009/9/23 Stas Oskin <[EMAIL PROTECTED]>
>
>> Thanks, I exactly wanted to ask this for our R&D roadmap.
>>
>> 2009/9/23 Aaron Kimball <[EMAIL PROTECTED]>
>>
>> Or maybe more pessimistically, the second "stable" append implementation.
>>>
>>> It's not like HADOOP-1700 wasn't intended to work. It was just found not
>>> to
>>> after the fact. Hopefully this reimplementation will succeed. If you're
>>> running a cluster that contains mission-critical data that cannot tolerate
>>> corruption or loss, you shouldn't jump on the new-feature bandwagon until
>>> it's had time to prove itself in the wild.
>>>
>>> But yes, we hope that appends will really-truly work in 0.21.
>>> Experimental/R&D projects should be able to plan on having a working
>>> append
>>> function in 0.21.
>>>
>>> - Aaron
>>>
>>> On Sun, Sep 20, 2009 at 3:58 PM, Stas Oskin <[EMAIL PROTECTED]> wrote:
>>>
>>> > Hi.
>>> >
>>> > Just to understand the road-map, 0.21 will be the first stable "append"
>>> > implementation?
>>> >
>>> > Regards.
>>> >
>>> > 2009/9/20 Owen O'Malley <[EMAIL PROTECTED]>
>>> >
>>> > >
>>> > > On Sep 13, 2009, at 3:08 AM, Stas Oskin wrote:
>>> > >
>>> > >  Hi.
>>> > >>
>>> > >> Any idea when the "append" functionality is expected?
>>> > >>
>>> > >
>>> > > A working append is a blocker on HDFS 0.21.0.
>>> > >
>>> > > The code for append is expected to be complete in a few weeks.
>>> Meanwhile,
>>> > > the rest of Common, HDFS, and MapReduce have feature-frozen and need
>>> to
>>> > be
>>> > > stabilized and all of the critical bugs fixed. I'd expect the first
>>> > releases
>>> > > of 0.21.0 in early November.
>>> > >
>>> > > -- Owen
>>> > >
>>> >
>>>
>>
>

Also at "hadoop world nyc" http://cloudera.com/hadoop-world-nyc their
is going to be a presentation:

Low Latency, Random Reads from HDFS, Jay Booth, Elastic Platforms

That might be of interest to you.

Hi.

Hopefully they will put the presentation online.

Regards.

2009/9/23 Edward Capriolo <[EMAIL PROTECTED]>

> On Wed, Sep 23, 2009 at 3:28 AM, Stas Oskin <[EMAIL PROTECTED]> wrote:
> > By the way, something that I forgot to ask before:
> >
> > Will the append implementation be as fast, as the write/copy functions?
> >
> > Regards.
> >
> > 2009/9/23 Stas Oskin <[EMAIL PROTECTED]>
> >
> >> Thanks, I exactly wanted to ask this for our R&D roadmap.
> >>
> >> 2009/9/23 Aaron Kimball <[EMAIL PROTECTED]>
> >>
> >> Or maybe more pessimistically, the second "stable" append
> implementation.
> >>>
> >>> It's not like HADOOP-1700 wasn't intended to work. It was just found
> not
> >>> to
> >>> after the fact. Hopefully this reimplementation will succeed. If you're
> >>> running a cluster that contains mission-critical data that cannot
> tolerate
> >>> corruption or loss, you shouldn't jump on the new-feature bandwagon
> until
> >>> it's had time to prove itself in the wild.
> >>>
> >>> But yes, we hope that appends will really-truly work in 0.21.
> >>> Experimental/R&D projects should be able to plan on having a working
> >>> append
> >>> function in 0.21.
> >>>
> >>> - Aaron
> >>>
> >>> On Sun, Sep 20, 2009 at 3:58 PM, Stas Oskin <[EMAIL PROTECTED]>
> wrote:
> >>>
> >>> > Hi.
> >>> >
> >>> > Just to understand the road-map, 0.21 will be the first stable
> "append"
> >>> > implementation?
> >>> >
> >>> > Regards.
> >>> >
> >>> > 2009/9/20 Owen O'Malley <[EMAIL PROTECTED]>
> >>> >
> >>> > >
> >>> > > On Sep 13, 2009, at 3:08 AM, Stas Oskin wrote:
> >>> > >
> >>> > >  Hi.
> >>> > >>
> >>> > >> Any idea when the "append" functionality is expected?
> >>> > >>
> >>> > >
> >>> > > A working append is a blocker on HDFS 0.21.0.
> >>> > >
> >>> > > The code for append is expected to be complete in a few weeks.
> >>> Meanwhile,
> >>> > > the rest of Common, HDFS, and MapReduce have feature-frozen and
> need
> >>> to
> >>> > be
> >>> > > stabilized and all of the critical bugs fixed. I'd expect the first
> >>> > releases
> >>> > > of 0.21.0 in early November.
> >>> > >
> >>> > > -- Owen
> >>> > >
> >>> >
> >>>
> >>
> >
>
> Also at "hadoop world nyc" http://cloudera.com/hadoop-world-nyc their
> is going to be a presentation:
>
> Low Latency, Random Reads from HDFS, Jay Booth, Elastic Platforms
>
> That might be of interest to you.
>

Stas,

you can track the status of append here:
http://issues.apache.org/jira/browse/HDFS-265

The latest status is:
http://issues.apache.org/jira/browse/HDFS-265?focusedCommentId=12757764&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#action_12757764

Thanks for your interest,
--Konstantin

Stas Oskin wrote:
> Hi.
>
> Hopefully they will put the presentation online.
>
> Regards.
>
> 2009/9/23 Edward Capriolo <[EMAIL PROTECTED]>
>
>> On Wed, Sep 23, 2009 at 3:28 AM, Stas Oskin <[EMAIL PROTECTED]> wrote:
>>> By the way, something that I forgot to ask before:
>>>
>>> Will the append implementation be as fast, as the write/copy functions?
>>>
>>> Regards.
>>>
>>> 2009/9/23 Stas Oskin <[EMAIL PROTECTED]>
>>>
>>>> Thanks, I exactly wanted to ask this for our R&D roadmap.
>>>>
>>>> 2009/9/23 Aaron Kimball <[EMAIL PROTECTED]>
>>>>
>>>> Or maybe more pessimistically, the second "stable" append
>> implementation.
>>>>> It's not like HADOOP-1700 wasn't intended to work. It was just found
>> not
>>>>> to
>>>>> after the fact. Hopefully this reimplementation will succeed. If you're
>>>>> running a cluster that contains mission-critical data that cannot
>> tolerate
>>>>> corruption or loss, you shouldn't jump on the new-feature bandwagon
>> until
>>>>> it's had time to prove itself in the wild.
>>>>>
>>>>> But yes, we hope that appends will really-truly work in 0.21.
>>>>> Experimental/R&D projects should be able to plan on having a working
>>>>> append
>>>>> function in 0.21.
>>>>>
>>>>> - Aaron
>>>>>
>>>>> On Sun, Sep 20, 2009 at 3:58 PM, Stas Oskin <[EMAIL PROTECTED]>
>> wrote:
>>>>>> Hi.
>>>>>>
>>>>>> Just to understand the road-map, 0.21 will be the first stable
>> "append"
>>>>>> implementation?
>>>>>>
>>>>>> Regards.
>>>>>>
>>>>>> 2009/9/20 Owen O'Malley <[EMAIL PROTECTED]>
>>>>>>
>>>>>>> On Sep 13, 2009, at 3:08 AM, Stas Oskin wrote:
>>>>>>>
>>>>>>>  Hi.
>>>>>>>> Any idea when the "append" functionality is expected?
>>>>>>>>
>>>>>>> A working append is a blocker on HDFS 0.21.0.
>>>>>>>
>>>>>>> The code for append is expected to be complete in a few weeks.
>>>>> Meanwhile,
>>>>>>> the rest of Common, HDFS, and MapReduce have feature-frozen and
>> need
>>>>> to
>>>>>> be
>>>>>>> stabilized and all of the critical bugs fixed. I'd expect the first
>>>>>> releases
>>>>>>> of 0.21.0 in early November.
>>>>>>>
>>>>>>> -- Owen
>>>>>>>
>> Also at "hadoop world nyc" http://cloudera.com/hadoop-world-nyc their
>> is going to be a presentation:
>>
>> Low Latency, Random Reads from HDFS, Jay Booth, Elastic Platforms
>>
>> That might be of interest to you.
>>
>