Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> RegionServer shutdown with ScanWildcardColumnTracker exception


+
Amit Sela 2013-04-17, 08:30
+
ramkrishna vasudevan 2013-04-17, 08:47
+
Amit Sela 2013-04-17, 09:06
+
ramkrishna vasudevan 2013-04-17, 10:42
+
Amit Sela 2013-04-17, 11:47
+
ramkrishna vasudevan 2013-04-17, 15:56
+
Amit Sela 2013-04-17, 16:10
+
ramkrishna vasudevan 2013-04-17, 16:17
Copy link to this message
-
Re: RegionServer shutdown with ScanWildcardColumnTracker exception
Is there any testcases that tries to reproduce your issue?

Regards
Ram
On Wed, Apr 17, 2013 at 9:47 PM, ramkrishna vasudevan <
[EMAIL PROTECTED]> wrote:

> There is a hint mechanism available when scanning happens.  But i dont
> think there should be much of difference between a scan that happens during
> flush and the normal scan.
>
> Will look thro the code and come back on this.
>
> Regards
> RAm
>
>
> On Wed, Apr 17, 2013 at 9:40 PM, Amit Sela <[EMAIL PROTECTED]> wrote:
>
>> No, no encoding.
>>
>>
>> On Wed, Apr 17, 2013 at 6:56 PM, ramkrishna vasudevan <
>> [EMAIL PROTECTED]> wrote:
>>
>> > @Lars
>> > You have any suggestions on this?
>> >
>> > @Amit
>> > You have any Encoder enabled like the Prefix Encoding stuff?
>> > There was one optimization added recently but that is not in 0.94.2
>> >
>> > Regards
>> > Ram
>> >
>> >
>> > On Wed, Apr 17, 2013 at 5:17 PM, Amit Sela <[EMAIL PROTECTED]> wrote:
>> >
>> > > I scanned over this counter with and without column specification and
>> all
>> > > looks OK now.
>> > > I have no CPs in this table.
>> > > Is there some kind of a hint mechanism in HBase' internal scan ?
>> because
>> > > it's weird that ScanWildcardColumnTracker.checkColumn says that
>> column is
>> > > smaller than previous column: *imprersions_ALL_2013041617*. there is
>> no
>> > > imprersions only impressions and r is indeed smaller than s, could it
>> be
>> > > some kind of hint bug ? I don't think I know enough of HBase
>> internals to
>> > > fully understand that...
>> > >
>> > >
>> > >
>> > > On Wed, Apr 17, 2013 at 1:42 PM, ramkrishna vasudevan <
>> > > [EMAIL PROTECTED]> wrote:
>> > >
>> > > > Hi Amit
>> > > >
>> > > > Checking the code this is possible when the qualifiers are not
>> sorted.
>> > >  Do
>> > > > you have any CPs in your path which tries to play with the KVs?
>> > > >
>> > > > Seems to be a very weird thing.
>> > > > Can you try doing a scan on the KV just before this happens.  That
>> will
>> > > tel
>> > > > you the existing kvs that are present.
>> > > >
>> > > > Even now if you can have the cluster you can try scanning for the
>> > region
>> > > > for which the flush happened.  That will give us some more info.
>> > > >
>> > > > Regards
>> > > > Ram
>> > > >
>> > > >
>> > > > On Wed, Apr 17, 2013 at 2:36 PM, Amit Sela <[EMAIL PROTECTED]>
>> > wrote:
>> > > >
>> > > > > The cluster runs Hadoop 1.0.4 and HBase 0.94.2
>> > > > >
>> > > > > I have three families in this table: weekly, daily, hourly. each
>> > family
>> > > > has
>> > > > > the following qualifiers:
>> > > > > Weekly - impressions_{countrycode}_{week#} - country code is 0, 1
>> or
>> > > ALL
>> > > > > (aggregation of both 0 and 1)
>> > > > > Daily and hourly are the same but with yyyyMMdd and yyyyMMddhh
>> > > > > respectively.
>> > > > >
>> > > > > Just before the exception the regionserver StoreFile executes the
>> > > > > following:
>> > > > >
>> > > > > 2013-04-16 17:56:06,769 [regionserver8041.cacheFlusher] INFO
>> > > > > org.apache.hadoop.hbase.regionserver.StoreFile: Delete Family
>> Bloom
>> > > > filter
>> > > > > type for hdfs://
>> > > > > hadoop-master.infolinks.com:8000/hbase/URL_COUNTERS/af2760e
>> > > > > 4d04a9e3025d1fb53bdba8acf/.tmp/dc4ce516887f4e0bbaf6201d69ba90bc:
>> > > > > CompoundBloomFilterWriter
>> > > > > 2013-04-16 17:56:07,331 [regionserver8041.cacheFlusher] INFO
>> > > > > org.apache.hadoop.hbase.regionserver.StoreFile: NO General Bloom
>> and
>> > NO
>> > > > > DeleteFamily was added to HFile
>> > (hdfs://hbase-master-address:8000/hbase
>> > > > > /URL_COUNTERS/*af2760e4d04a9e3025d1fb53bdba8acf*
>> > > > > /.tmp/dc4ce516887f4e0bbaf6201d69ba90bc)
>> > > > > 2013-04-16 17:56:07,331 [regionserver8041.cacheFlusher] INFO
>> > > > > org.apache.hadoop.hbase.regionserver.Store: Flushed ,
>> > > > sequenceid=210517246,
>> > > > > memsize=39.3m, into tmp file hdfs://hbase-master:8000/hbase
>> > > > > /URL_COUNTERS/*af2760e4d04a9e3025d1fb53bdba8acf*
>> > > > > /.tmp/dc4ce516887f4e0bbaf6201d69ba90bc
+
Amit Sela 2013-04-17, 18:34