Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> delete rows without writing HLog may be appear in the future?


+
Bing Jiang 2012-11-21, 12:37
+
Michael Segel 2012-11-21, 15:09
+
Bing Jiang 2012-11-21, 15:20
+
lars hofhansl 2012-11-21, 17:27
+
Bing Jiang 2012-11-22, 02:10
Copy link to this message
-
Re: delete rows without writing HLog may be appear in the future?
In our hbase cluster, I test if delete records with hlog or without.
Attachment is my my test.
The result of test can testify why I make a decision of delete rows without
hlog .
2012/11/22 Bing Jiang <[EMAIL PROTECTED]>

> Thanks for all your suggestion and talk.
> One idea occurs to me why not check or restore wal when compaction
> executes. If it does, hbase can drop some unused hlog, I think that will be
> effective to the issue.
> please correct me if I am wrong.
>
> ---Bing
>
> 2012/11/22 lars hofhansl <[EMAIL PROTECTED]>
>
>> I have it on my list of things to do to allow deferred WAL flush as a per
>> operation option (right now it's a CF option).
>> You really do not want to do anything with the WAL off. If you use
>> deferred flush there is still a chance that this might happen (the RS could
>> die in the few seconds after a Delete before it is flushed to the WAL), but
>> it should be a rare occurrance.
>>
>>
>> -- Lars
>>
>>
>>
>> ________________________________
>>  From: Bing Jiang <[EMAIL PROTECTED]>
>> To: [EMAIL PROTECTED]
>> Sent: Wednesday, November 21, 2012 7:20 AM
>> Subject: Re: delete rows without writing HLog may be appear in the future?
>>
>> we need to confirm that put must be safe,but deletes must be quick and
>> low-latency.
>> On Nov 21, 2012 11:10 PM, "Michael Segel" <[EMAIL PROTECTED]>
>> wrote:
>>
>> > Some time later?
>> >
>> > Time of course is relative, so I have to ask what occurred between the
>> > write and the delete?
>> > How much time? Did you have any compactions in between the write and the
>> > delete?
>> >
>> > Why are you not consistent in your use of the WAL ?
>> >
>> >
>> > On Nov 21, 2012, at 6:37 AM, Bing Jiang <[EMAIL PROTECTED]>
>> wrote:
>> >
>> > > hi,all.
>> > > I want to describe a phenomenon that happens to our hbase cluster.
>> > > I use puts(List<Put>) to insert many records with writing hlog enable,
>> > > and some time later I delete all of these records with writing hlog
>> > disable.
>> > > When one week later, i scan the table, I found some records I have
>> delete
>> > > reappear again.
>> > > It is an interesting case. In my opinion, if we delete data without
>> > enable
>> > > writing hlog, when regionserver fails, the log will replay in another
>> > > regionserver.
>> > > Can anyone tell me if I persist on deleting records without enable
>> > writing
>> > > hlog, is there a way to prevent these records from reappearing again
>> some
>> > > time later?
>> > >
>> > > Cheers!
>> > > --
>> > > Bing Jiang
>> > > weibo: http://weibo.com/jiangbinglover
>> > > BLOG: http://blog.sina.com.cn/jiangbinglover
>> > > BLOG: http://www.binospace.com
>> > > National Research Center for Intelligent Computing Systems
>> > > Institute of Computing technology
>> > > Graduate University of Chinese Academy of Science
>> >
>> >
>>
>
>
>
> --
> Bing Jiang
> Tel:(86)134-2619-1361
>
> weibo: http://weibo.com/jiangbinglover
> BLOG: http://blog.sina.com.cn/jiangbinglover
> National Research Center for Intelligent Computing Systems
> Institute of Computing technology
> Graduate University of Chinese Academy of Science
>
>
--
Bing Jiang
Tel:(86)134-2619-1361
weibo: http://weibo.com/jiangbinglover
BLOG: http://blog.sina.com.cn/jiangbinglover
National Research Center for Intelligent Computing Systems
Institute of Computing technology
Graduate University of Chinese Academy of Science
+
ramkrishna vasudevan 2012-11-22, 03:56
+
Bing Jiang 2012-11-22, 04:36
+
lars hofhansl 2012-11-22, 05:01
+
Bing Jiang 2012-11-22, 06:50
+
ramkrishna vasudevan 2012-11-21, 15:24
+
Bing Jiang 2012-11-21, 15:17
+
Kevin Odell 2012-11-21, 15:18
+
Bing Jiang 2012-11-21, 15:37
+
Michael Segel 2012-11-21, 16:18
+
Kevin Odell 2012-11-21, 15:43
+
ramkrishna vasudevan 2012-11-21, 16:25
+
ramkrishna vasudevan 2012-11-21, 17:00
+
ramkrishna vasudevan 2012-11-21, 15:00
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB