Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # dev >> silent data loss during append


Copy link to this message
-
Re: silent data loss during append
What version are you using?

On Thu, Apr 14, 2011 at 3:55 PM, Thanh Do <[EMAIL PROTECTED]> wrote:

> Hi all,
>
> I have recently seen silent data loss in our system.
> Here is the case:
>
> 1. client appends to some block
> 2. for some reason, commitBlockSynchronization
>     returns successfully with synclist = [] (i.e empty)
> 3. in the client code, NO exception is thrown, and
>    client appends successfully.
> 4. However, the block replicas are then removed from
>    datanodes, causing data loss.
>
> Have any one seen this before?
> Is this behavior by design or a bug?
>
> Many thanks,
> Thanh
>
>
>
>