Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> delete rows without writing HLog may be appear in the future?


+
Bing Jiang 2012-11-21, 12:37
+
Michael Segel 2012-11-21, 15:09
+
Bing Jiang 2012-11-21, 15:20
+
lars hofhansl 2012-11-21, 17:27
+
Bing Jiang 2012-11-22, 02:10
Copy link to this message
-
Re: delete rows without writing HLog may be appear in the future?
In our hbase cluster, I test if delete records with hlog or without.
Attachment is my my test.
The result of test can testify why I make a decision of delete rows without
hlog .
2012/11/22 Bing Jiang <[EMAIL PROTECTED]>

> Thanks for all your suggestion and talk.
> One idea occurs to me why not check or restore wal when compaction
> executes. If it does, hbase can drop some unused hlog, I think that will be
> effective to the issue.
> please correct me if I am wrong.
>
> ---Bing
>
> 2012/11/22 lars hofhansl <[EMAIL PROTECTED]>
>
>> I have it on my list of things to do to allow deferred WAL flush as a per
>> operation option (right now it's a CF option).
>> You really do not want to do anything with the WAL off. If you use
>> deferred flush there is still a chance that this might happen (the RS could
>> die in the few seconds after a Delete before it is flushed to the WAL), but
>> it should be a rare occurrance.
>>
>>
>> -- Lars
>>
>>
>>
>> ________________________________
>>  From: Bing Jiang <[EMAIL PROTECTED]>
>> To: [EMAIL PROTECTED]
>> Sent: Wednesday, November 21, 2012 7:20 AM
>> Subject: Re: delete rows without writing HLog may be appear in the future?
>>
>> we need to confirm that put must be safe,but deletes must be quick and
>> low-latency.
>> On Nov 21, 2012 11:10 PM, "Michael Segel" <[EMAIL PROTECTED]>
>> wrote:
>>
>> > Some time later?
>> >
>> > Time of course is relative, so I have to ask what occurred between the
>> > write and the delete?
>> > How much time? Did you have any compactions in between the write and the
>> > delete?
>> >
>> > Why are you not consistent in your use of the WAL ?
>> >
>> >
>> > On Nov 21, 2012, at 6:37 AM, Bing Jiang <[EMAIL PROTECTED]>
>> wrote:
>> >
>> > > hi,all.
>> > > I want to describe a phenomenon that happens to our hbase cluster.
>> > > I use puts(List<Put>) to insert many records with writing hlog enable,
>> > > and some time later I delete all of these records with writing hlog
>> > disable.
>> > > When one week later, i scan the table, I found some records I have
>> delete
>> > > reappear again.
>> > > It is an interesting case. In my opinion, if we delete data without
>> > enable
>> > > writing hlog, when regionserver fails, the log will replay in another
>> > > regionserver.
>> > > Can anyone tell me if I persist on deleting records without enable
>> > writing
>> > > hlog, is there a way to prevent these records from reappearing again
>> some
>> > > time later?
>> > >
>> > > Cheers!
>> > > --
>> > > Bing Jiang
>> > > weibo: http://weibo.com/jiangbinglover
>> > > BLOG: http://blog.sina.com.cn/jiangbinglover
>> > > BLOG: http://www.binospace.com
>> > > National Research Center for Intelligent Computing Systems
>> > > Institute of Computing technology
>> > > Graduate University of Chinese Academy of Science
>> >
>> >
>>
>
>
>
> --
> Bing Jiang
> Tel:(86)134-2619-1361
>
> weibo: http://weibo.com/jiangbinglover
> BLOG: http://blog.sina.com.cn/jiangbinglover
> National Research Center for Intelligent Computing Systems
> Institute of Computing technology
> Graduate University of Chinese Academy of Science
>
>
--
Bing Jiang
Tel:(86)134-2619-1361
weibo: http://weibo.com/jiangbinglover
BLOG: http://blog.sina.com.cn/jiangbinglover
National Research Center for Intelligent Computing Systems
Institute of Computing technology
Graduate University of Chinese Academy of Science
+
ramkrishna vasudevan 2012-11-22, 03:56
+
Bing Jiang 2012-11-22, 04:36
+
lars hofhansl 2012-11-22, 05:01
+
Bing Jiang 2012-11-22, 06:50
+
ramkrishna vasudevan 2012-11-21, 15:24
+
Bing Jiang 2012-11-21, 15:17
+
Kevin Odell 2012-11-21, 15:18
+
Bing Jiang 2012-11-21, 15:37
+
Michael Segel 2012-11-21, 16:18
+
Kevin Odell 2012-11-21, 15:43
+
ramkrishna vasudevan 2012-11-21, 16:25
+
ramkrishna vasudevan 2012-11-21, 17:00
+
ramkrishna vasudevan 2012-11-21, 15:00