Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase, mail # dev - Let's discuss Snapshots Feature Testing


+
Aleksandr Shulman 2013-01-14, 18:32
+
Ted Yu 2013-01-14, 19:01
+
Aleksandr Shulman 2013-01-14, 19:15
+
Andrew Purtell 2013-01-14, 23:15
+
Jonathan Hsieh 2013-01-15, 01:27
+
Andrew Purtell 2013-01-15, 02:47
+
Jonathan Hsieh 2013-01-15, 08:55
+
Nicolas Liochon 2013-01-15, 09:25
Copy link to this message
-
Re: Let's discuss Snapshots Feature Testing
Jean-Marc Spaggiari 2013-01-15, 19:01
That's a long list of tests. Impressive.

I will add this one:
> -Meanness tests:
> 1. Killing the master
> 2. Performing a compaction
> 3. Table enable/disable
4. Moving a/some region/s while snapshot is running.

Killing the master or a RS will occur in some regions moved, but also,
some bulk imports then load balancing can induct massive regions moves
too...

JM

2013/1/15, Nicolas Liochon <[EMAIL PROTECTED]>:
> I would be +1 on killing datanodes during the tests. I think we tend to
> under analyze the impact on an HDFS error in HBase.
> See for example
> HBASE-6738<https://issues.apache.org/jira/browse/HBASE-6738>:
> in the distributed log, we were considering a task as dead if the split was
> not done in 25s. If you were going to the dead DN to read the WAL, 25s was
> far from enough, and we were ending up doing the same split on multiple
> computers.
>
> HDFS is a nice buddy, but it can't hide everything.
>
>
> On Tue, Jan 15, 2013 at 9:55 AM, Jonathan Hsieh <[EMAIL PROTECTED]> wrote:
>
>> My counter-argument here is that this would be a bug in HDFS as
>> opposed to HBase. It is good to know, but ideally shouldn't be exposed
>> at the HBase level.   This test won't really make sense if there was a
>> different FS underneath.
>>
>> That said, if you insist we can add and and report on this (lower
>> priority than the hbase-level problems though).
>>
>> Jon.
>>
>> On Mon, Jan 14, 2013 at 6:47 PM, Andrew Purtell <[EMAIL PROTECTED]>
>> wrote:
>> > If a datanode goes down and it has an indirect bad effect on snapshots,
>> > this would be useful to know.
>> >
>> > For the HA NN item, I threw that in there for completeness sake.
>> > Ideally
>> a
>> > client like HBase wouldn't notice.
>> >
>> > On Mon, Jan 14, 2013 at 5:27 PM, Jonathan Hsieh <[EMAIL PROTECTED]>
>> wrote:
>> >
>> >> I think the killing data nodes and killing HA NN is out of scope form
>> >> an HBase point of view.
>> >>
>> >
>> >
>> > --
>> > Best regards,
>> >
>> >    - Andy
>> >
>> > Problems worthy of attack prove their worth by hitting back. - Piet
>> > Hein
>> > (via Tom White)
>>
>>
>>
>> --
>> // Jonathan Hsieh (shay)
>> // Software Engineer, Cloudera
>> // [EMAIL PROTECTED]
>>
>