Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - Improvement: Provide better feedback on Put to unknown CF


Copy link to this message
-
Re: Improvement: Provide better feedback on Put to unknown CF
Jean-Marc Spaggiari 2012-07-10, 12:28
Hi Michael,

I agree that in the code we have access to all the information to
access the right column.

However, let's imagine the column family name is dynamically retrieved
from a property file, and there is a typo. Or, another process removed
the column family. Or there is a bug in the code, and so on.

There is many possibilities why an application might try to access a
CF which, at the end, doesn't exist in the table.  I agree it should
have been checked from the meta before, but skeeping that step might
be required to improve performances.

Adding such exception will not have any negative impact on perfs,
readability, etc. It will simply help a lot the defect tracking when
someone will face the issue and see the stack trace.

JM

2012/7/9, Michael Segel <[EMAIL PROTECTED]>:
> Jean-Marc,
>
> I think you mis understood.
> At run time, you can query HBase to find out the table schema and its column
> families.
>
> While I agree that you are seeing poorly written exceptions, IMHO its easier
> to avoid the problem in the first place.
>
> In a Map/Reduce in side the mapper class, you have everything you need to
> get the table's schema.
> From that you can see the column families.
>
>
> HTH
>
> -Mike
>
> On Jul 9, 2012, at 8:42 AM, Jean-Marc Spaggiari wrote:
>
>> In my case it was a codding issue. Used the wrong final byte array to
>> access the CF. So I agree, the CF is well known since you create the
>> table based on them. But maybe you have added some other CFs later and
>> something went wrong?
>>
>> It's just that based on the exception received, there is no indication
>> that there might be some issues with the CF. So you might end trying
>> to figure what the issue is far from where it's really.
>>
>> 2012/7/9, Michael Segel <[EMAIL PROTECTED]>:
>>> This may beg the question ...
>>> Why do you not know the CF?
>>>
>>> Your table schemas only consist of tables and CFs. So you should know
>>> them
>>> at the start of your job or m/r Mapper.setup();
>>>
>>>
>>> On Jul 9, 2012, at 7:25 AM, Jean-Marc Spaggiari wrote:
>>>
>>>> Hi,
>>>>
>>>> When we try to add a value to a CF which does not exist on a table, we
>>>> are getting the error below. I think this is not really giving the
>>>> right information about the issue.
>>>>
>>>> Should it not be better to provide an exception like
>>>> UnknownColumnFamillyException?
>>>>
>>>> JM
>>>>
>>>> org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException:
>>>> Failed 1 action: DoNotRetryIOException: 1 time, servers with issues:
>>>> phenom:60020,
>>>> at
>>>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1591)
>>>> at
>>>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1367)
>>>> at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:945)
>>>> at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:801)
>>>> at org.apache.hadoop.hbase.client.HTable.put(HTable.java:776)
>>>> at org.myapp.app.Integrator.main(Integrator.java:162)
>>>>
>>>
>>>
>>
>
>