Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # dev >> Re: [VOTE] Merge HDFS-2832 Heterogeneous Storage Phase 1 to trunk

Copy link to this message
Re: [VOTE] Merge HDFS-2832 Heterogeneous Storage Phase 1 to trunk


On Tuesday, December 3, 2013 10:32 AM, Jun Ping Du <[EMAIL PROTECTED]> wrote:
+1. Good to see HDFS can support different storage tiers.
>I have been involved with minor development & bug fixing effort and I agree it is ready to merge too.
>----- Original Message -----
>From: "Suresh Srinivas" <[EMAIL PROTECTED]>
>Sent: Tuesday, December 3, 2013 8:15:26 AM
>Subject: Re: [VOTE] Merge HDFS-2832 Heterogeneous Storage Phase 1 to trunk
>Great work Arpit and Nicholas!
>+1. I have been part of design. I have been following the changes closely.
>This is ready to be merged into trunk.
>On Mon, Dec 2, 2013 at 4:06 PM, Arpit Agarwal <[EMAIL PROTECTED]>wrote:
>> Hello all,
>> I would like to call a vote to merge phase 1 of the Heterogeneous Storage
>> feature into trunk.
>> *Scope of the changes:*
>> The changes allow exposing the DataNode as a collection of storages and set
>> the foundation for subsequent work to present Heterogeneous Storages to
>> applications. This allows DataNodes to send block and storage reports
>> per-storage. In addition this change introduces the ability to add a
>> 'storage type' tag to the storage directories. This enables supporting
>> different types of storages in addition to disk storage.
>> Development of the feature is tracked in the jira
>> https://urldefense.proofpoint.com/v1/url?u=https://issues.apache.org/jira/browse/HDFS-2832&k=oIvRg1%2BdGAgOoM1BIlLLqw%3D%3D%0A&r=Mw3izENeqbMFnOzHo594vQ%3D%3D%0A&m=Z%2F4r%2B%2FNk9yYikPHTUsPHx9kGN2a1jV0DGMDT3uJYLqw%3D%0A&s=1d962b101464bff32d5f6339d6616e1dc05d7ece8890923c322549257b696ecf.
>> *Details of development and testing:*
>> Development has been done in a separate branch -
>> https://urldefense.proofpoint.com/v1/url?u=https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-2832&k=oIvRg1%2BdGAgOoM1BIlLLqw%3D%3D%0A&r=Mw3izENeqbMFnOzHo594vQ%3D%3D%0A&m=Z%2F4r%2B%2FNk9yYikPHTUsPHx9kGN2a1jV0DGMDT3uJYLqw%3D%0A&s=09ee6285bc4abc6bcac8ba4f05f9ef65d66cf1e94d8e705b356bd429659eb689. The
>> updated design is posted at -
>> https://urldefense.proofpoint.com/v1/url?u=https://issues.apache.org/jira/secure/attachment/12615761/20131125-HeterogeneousStorage.pdf&k=oIvRg1%2BdGAgOoM1BIlLLqw%3D%3D%0A&r=Mw3izENeqbMFnOzHo594vQ%3D%3D%0A&m=Z%2F4r%2B%2FNk9yYikPHTUsPHx9kGN2a1jV0DGMDT3uJYLqw%3D%0A&s=9ca29507f2c8e258ab7d4ddcdae82926bf66e4052c31368b2abead87c7df110f
>> .
>> The changes involve ~6K changed lines of code, with a third of those
>> changes being to tests.
>> Please see the test plan
>> https://urldefense.proofpoint.com/v1/url?u=https://issues.apache.org/jira/secure/attachment/12616642/20131202-HeterogeneousStorage-TestPlan.pdffor&k=oIvRg1%2BdGAgOoM1BIlLLqw%3D%3D%0A&r=Mw3izENeqbMFnOzHo594vQ%3D%3D%0A&m=Z%2F4r%2B%2FNk9yYikPHTUsPHx9kGN2a1jV0DGMDT3uJYLqw%3D%0A&s=f1d595e4b4cb6ee14cd92093cc1d7f173671f606606528931cd34cab4a08fabd
>> the details. Once the feature is
>> merged into trunk, we will continue to test and fix any bugs that may be
>> found on trunk as well as add further tests as outlined in the test plan.
>> The bulk of the design and implementation was done by Suresh Srinivas,
>> Sanjay Radia, Nicholas Sze, Junping Du and me. Also, thanks to Eric
>> Sirianni, Chris Nauroth, Steve Loughran, Bikas Saha, Andrew Wang and Todd
>> Lipcon for providing feedback on the Jiras and in discussions.
>> This vote runs for a week and closes on 12/9/2013 at 11:59 pm PT.
>> Thanks,
>> Arpit
>> --
>> NOTICE: This message is intended for the use of the individual or entity to
>> which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have