Hi,

When we try to add a value to a CF which does not exist on a table, we
are getting the error below. I think this is not really giving the
right information about the issue.

Should it not be better to provide an exception like
UnknownColumnFamillyException?

JM

org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException:
Failed 1 action: DoNotRetryIOException: 1 time, servers with issues:
phenom:60020,
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1591)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1367)
at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:945)
at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:801)
at org.apache.hadoop.hbase.client.HTable.put(HTable.java:776)
at org.myapp.app.Integrator.main(Integrator.java:162)

I agree.

On Jul 9, 2012, at 5:25 AM, Jean-Marc Spaggiari <[EMAIL PROTECTED]> wrote:

> Hi,
>
> When we try to add a value to a CF which does not exist on a table, we
> are getting the error below. I think this is not really giving the
> right information about the issue.
>
> Should it not be better to provide an exception like
> UnknownColumnFamillyException?
>
> JM
>
> org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException:
> Failed 1 action: DoNotRetryIOException: 1 time, servers with issues:
> phenom:60020,
>    at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1591)
>    at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1367)
>    at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:945)
>    at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:801)
>    at org.apache.hadoop.hbase.client.HTable.put(HTable.java:776)
>    at org.myapp.app.Integrator.main(Integrator.java:162)

This may beg the question ...
Why do you not know the CF?

Your table schemas only consist of tables and CFs. So you should know them at the start of your job or m/r Mapper.setup();
On Jul 9, 2012, at 7:25 AM, Jean-Marc Spaggiari wrote:

> Hi,
>
> When we try to add a value to a CF which does not exist on a table, we
> are getting the error below. I think this is not really giving the
> right information about the issue.
>
> Should it not be better to provide an exception like
> UnknownColumnFamillyException?
>
> JM
>
> org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException:
> Failed 1 action: DoNotRetryIOException: 1 time, servers with issues:
> phenom:60020,
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1591)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1367)
> at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:945)
> at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:801)
> at org.apache.hadoop.hbase.client.HTable.put(HTable.java:776)
> at org.myapp.app.Integrator.main(Integrator.java:162)
>

In my case it was a codding issue. Used the wrong final byte array to
access the CF. So I agree, the CF is well known since you create the
table based on them. But maybe you have added some other CFs later and
something went wrong?

It's just that based on the exception received, there is no indication
that there might be some issues with the CF. So you might end trying
to figure what the issue is far from where it's really.

2012/7/9, Michael Segel <[EMAIL PROTECTED]>:
> This may beg the question ...
> Why do you not know the CF?
>
> Your table schemas only consist of tables and CFs. So you should know them
> at the start of your job or m/r Mapper.setup();
>
>
> On Jul 9, 2012, at 7:25 AM, Jean-Marc Spaggiari wrote:
>
>> Hi,
>>
>> When we try to add a value to a CF which does not exist on a table, we
>> are getting the error below. I think this is not really giving the
>> right information about the issue.
>>
>> Should it not be better to provide an exception like
>> UnknownColumnFamillyException?
>>
>> JM
>>
>> org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException:
>> Failed 1 action: DoNotRetryIOException: 1 time, servers with issues:
>> phenom:60020,
>> at
>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1591)
>> at
>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1367)
>> at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:945)
>> at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:801)
>> at org.apache.hadoop.hbase.client.HTable.put(HTable.java:776)
>> at org.myapp.app.Integrator.main(Integrator.java:162)
>>
>
>

Jean-Marc,

I think you mis understood.
At run time, you can query HBase to find out the table schema and its column families.

While I agree that you are seeing poorly written exceptions, IMHO its easier to avoid the problem in the first place.

In a Map/Reduce in side the mapper class, you have everything you need to get the table's schema.
From that you can see the column families.
HTH

-Mike

On Jul 9, 2012, at 8:42 AM, Jean-Marc Spaggiari wrote:

> In my case it was a codding issue. Used the wrong final byte array to
> access the CF. So I agree, the CF is well known since you create the
> table based on them. But maybe you have added some other CFs later and
> something went wrong?
>
> It's just that based on the exception received, there is no indication
> that there might be some issues with the CF. So you might end trying
> to figure what the issue is far from where it's really.
>
> 2012/7/9, Michael Segel <[EMAIL PROTECTED]>:
>> This may beg the question ...
>> Why do you not know the CF?
>>
>> Your table schemas only consist of tables and CFs. So you should know them
>> at the start of your job or m/r Mapper.setup();
>>
>>
>> On Jul 9, 2012, at 7:25 AM, Jean-Marc Spaggiari wrote:
>>
>>> Hi,
>>>
>>> When we try to add a value to a CF which does not exist on a table, we
>>> are getting the error below. I think this is not really giving the
>>> right information about the issue.
>>>
>>> Should it not be better to provide an exception like
>>> UnknownColumnFamillyException?
>>>
>>> JM
>>>
>>> org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException:
>>> Failed 1 action: DoNotRetryIOException: 1 time, servers with issues:
>>> phenom:60020,
>>> at
>>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1591)
>>> at
>>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1367)
>>> at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:945)
>>> at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:801)
>>> at org.apache.hadoop.hbase.client.HTable.put(HTable.java:776)
>>> at org.myapp.app.Integrator.main(Integrator.java:162)
>>>
>>
>>
>

Hi Michael,

I agree that in the code we have access to all the information to
access the right column.

However, let's imagine the column family name is dynamically retrieved
from a property file, and there is a typo. Or, another process removed
the column family. Or there is a bug in the code, and so on.

There is many possibilities why an application might try to access a
CF which, at the end, doesn't exist in the table.  I agree it should
have been checked from the meta before, but skeeping that step might
be required to improve performances.

Adding such exception will not have any negative impact on perfs,
readability, etc. It will simply help a lot the defect tracking when
someone will face the issue and see the stack trace.

JM

2012/7/9, Michael Segel <[EMAIL PROTECTED]>:
> Jean-Marc,
>
> I think you mis understood.
> At run time, you can query HBase to find out the table schema and its column
> families.
>
> While I agree that you are seeing poorly written exceptions, IMHO its easier
> to avoid the problem in the first place.
>
> In a Map/Reduce in side the mapper class, you have everything you need to
> get the table's schema.
> From that you can see the column families.
>
>
> HTH
>
> -Mike
>
> On Jul 9, 2012, at 8:42 AM, Jean-Marc Spaggiari wrote:
>
>> In my case it was a codding issue. Used the wrong final byte array to
>> access the CF. So I agree, the CF is well known since you create the
>> table based on them. But maybe you have added some other CFs later and
>> something went wrong?
>>
>> It's just that based on the exception received, there is no indication
>> that there might be some issues with the CF. So you might end trying
>> to figure what the issue is far from where it's really.
>>
>> 2012/7/9, Michael Segel <[EMAIL PROTECTED]>:
>>> This may beg the question ...
>>> Why do you not know the CF?
>>>
>>> Your table schemas only consist of tables and CFs. So you should know
>>> them
>>> at the start of your job or m/r Mapper.setup();
>>>
>>>
>>> On Jul 9, 2012, at 7:25 AM, Jean-Marc Spaggiari wrote:
>>>
>>>> Hi,
>>>>
>>>> When we try to add a value to a CF which does not exist on a table, we
>>>> are getting the error below. I think this is not really giving the
>>>> right information about the issue.
>>>>
>>>> Should it not be better to provide an exception like
>>>> UnknownColumnFamillyException?
>>>>
>>>> JM
>>>>
>>>> org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException:
>>>> Failed 1 action: DoNotRetryIOException: 1 time, servers with issues:
>>>> phenom:60020,
>>>> at
>>>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1591)
>>>> at
>>>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1367)
>>>> at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:945)
>>>> at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:801)
>>>> at org.apache.hadoop.hbase.client.HTable.put(HTable.java:776)
>>>> at org.myapp.app.Integrator.main(Integrator.java:162)
>>>>
>>>
>>>
>>
>
>

+1 a proper error message always helps IMHO

------------------------------
On Tue 10 Jul, 2012 5:58 PM IST Jean-Marc Spaggiari wrote:

>Hi Michael,
>
>I agree that in the code we have access to all the information to
>access the right column.
>
>However, let's imagine the column family name is dynamically retrieved
>from a property file, and there is a typo. Or, another process removed
>the column family. Or there is a bug in the code, and so on.
>
>There is many possibilities why an application might try to access a
>CF which, at the end, doesn't exist in the table.  I agree it should
>have been checked from the meta before, but skeeping that step might
>be required to improve performances.
>
>Adding such exception will not have any negative impact on perfs,
>readability, etc. It will simply help a lot the defect tracking when
>someone will face the issue and see the stack trace.
>
>JM
>
>2012/7/9, Michael Segel <[EMAIL PROTECTED]>:
>> Jean-Marc,
>>
>> I think you mis understood.
>> At run time, you can query HBase to find out the table schema and its column
>> families.
>>
>> While I agree that you are seeing poorly written exceptions, IMHO its easier
>> to avoid the problem in the first place.
>>
>> In a Map/Reduce in side the mapper class, you have everything you need to
>> get the table's schema.
>> From that you can see the column families.
>>
>>
>> HTH
>>
>> -Mike
>>
>> On Jul 9, 2012, at 8:42 AM, Jean-Marc Spaggiari wrote:
>>
>> In my case it was a codding issue. Used the wrong final byte array to
>> access the CF. So I agree, the CF is well known since you create the
>> table based on them. But maybe you have added some other CFs later and
>> something went wrong?
>>
>> It's just that based on the exception received, there is no indication
>> that there might be some issues with the CF. So you might end trying
>> to figure what the issue is far from where it's really.
>>
>> 2012/7/9, Michael Segel <[EMAIL PROTECTED]>:
>>> This may beg the question ...
>>> Why do you not know the CF?
>>>
>>> Your table schemas only consist of tables and CFs. So you should know
>>> them
>>> at the start of your job or m/r Mapper.setup();
>>>
>>>
>>> On Jul 9, 2012, at 7:25 AM, Jean-Marc Spaggiari wrote:
>>>
>>> Hi,
>>>
>>> When we try to add a value to a CF which does not exist on a table, we
>>> are getting the error below. I think this is not really giving the
>>> right information about the issue.
>>>
>>> Should it not be better to provide an exception like
>>> UnknownColumnFamillyException?
>>>
>>> JM
>>>
>>> org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException:
>>> Failed 1 action: DoNotRetryIOException: 1 time, servers with issues:
>>> phenom:60020,
>>> at
>>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1591)
>>> at
>>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1367)
>>> at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:945)
>>> at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:801)
>>> at org.apache.hadoop.hbase.client.HTable.put(HTable.java:776)
>>> at org.myapp.app.Integrator.main(Integrator.java:162)
>>>
>>>
>>>
>>
>>
>>

Nobody is arguing that the exception message doesn't make sense. +1 to making better error messages.

What I am suggesting is to take advantage of HBase's meta data. That is, at run time, you can query HBase to determine what tables exist, and see what column families exist.
So you can avoid the error in the first place.
On Jul 10, 2012, at 8:07 AM, Dhaval Shah wrote:

>
> +1 a proper error message always helps IMHO
>
>
>
> ------------------------------
> On Tue 10 Jul, 2012 5:58 PM IST Jean-Marc Spaggiari wrote:
>
>> Hi Michael,
>>
>> I agree that in the code we have access to all the information to
>> access the right column.
>>
>> However, let's imagine the column family name is dynamically retrieved
>> from a property file, and there is a typo. Or, another process removed
>> the column family. Or there is a bug in the code, and so on.
>>
>> There is many possibilities why an application might try to access a
>> CF which, at the end, doesn't exist in the table.  I agree it should
>> have been checked from the meta before, but skeeping that step might
>> be required to improve performances.
>>
>> Adding such exception will not have any negative impact on perfs,
>> readability, etc. It will simply help a lot the defect tracking when
>> someone will face the issue and see the stack trace.
>>
>> JM
>>
>> 2012/7/9, Michael Segel <[EMAIL PROTECTED]>:
>>> Jean-Marc,
>>>
>>> I think you mis understood.
>>> At run time, you can query HBase to find out the table schema and its column
>>> families.
>>>
>>> While I agree that you are seeing poorly written exceptions, IMHO its easier
>>> to avoid the problem in the first place.
>>>
>>> In a Map/Reduce in side the mapper class, you have everything you need to
>>> get the table's schema.
>>> From that you can see the column families.
>>>
>>>
>>> HTH
>>>
>>> -Mike
>>>
>>> On Jul 9, 2012, at 8:42 AM, Jean-Marc Spaggiari wrote:
>>>
>>> In my case it was a codding issue. Used the wrong final byte array to
>>> access the CF. So I agree, the CF is well known since you create the
>>> table based on them. But maybe you have added some other CFs later and
>>> something went wrong?
>>>
>>> It's just that based on the exception received, there is no indication
>>> that there might be some issues with the CF. So you might end trying
>>> to figure what the issue is far from where it's really.
>>>
>>> 2012/7/9, Michael Segel <[EMAIL PROTECTED]>:
>>>> This may beg the question ...
>>>> Why do you not know the CF?
>>>>
>>>> Your table schemas only consist of tables and CFs. So you should know
>>>> them
>>>> at the start of your job or m/r Mapper.setup();
>>>>
>>>>
>>>> On Jul 9, 2012, at 7:25 AM, Jean-Marc Spaggiari wrote:
>>>>
>>>> Hi,
>>>>
>>>> When we try to add a value to a CF which does not exist on a table, we
>>>> are getting the error below. I think this is not really giving the
>>>> right information about the issue.
>>>>
>>>> Should it not be better to provide an exception like
>>>> UnknownColumnFamillyException?
>>>>
>>>> JM
>>>>
>>>> org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException:
>>>> Failed 1 action: DoNotRetryIOException: 1 time, servers with issues:
>>>> phenom:60020,
>>>> at
>>>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1591)
>>>> at
>>>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1367)
>>>> at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:945)
>>>> at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:801)
>>>> at org.apache.hadoop.hbase.client.HTable.put(HTable.java:776)
>>>> at org.myapp.app.Integrator.main(Integrator.java:162)
>>>>
>>>>
>>>>
>>>
>>>
>>>
>
>

Nice discussion here.

Jean-Marc:
Do you mind logging a JIRA ?

Thanks

On Tue, Jul 10, 2012 at 3:15 PM, Michael Segel <[EMAIL PROTECTED]>wrote:

> Nobody is arguing that the exception message doesn't make sense. +1 to
> making better error messages.
>
> What I am suggesting is to take advantage of HBase's meta data. That is,
> at run time, you can query HBase to determine what tables exist, and see
> what column families exist.
> So you can avoid the error in the first place.
>
>
> On Jul 10, 2012, at 8:07 AM, Dhaval Shah wrote:
>
> >
> > +1 a proper error message always helps IMHO
> >
> >
> >
> > ------------------------------
> > On Tue 10 Jul, 2012 5:58 PM IST Jean-Marc Spaggiari wrote:
> >
> >> Hi Michael,
> >>
> >> I agree that in the code we have access to all the information to
> >> access the right column.
> >>
> >> However, let's imagine the column family name is dynamically retrieved
> >> from a property file, and there is a typo. Or, another process removed
> >> the column family. Or there is a bug in the code, and so on.
> >>
> >> There is many possibilities why an application might try to access a
> >> CF which, at the end, doesn't exist in the table.  I agree it should
> >> have been checked from the meta before, but skeeping that step might
> >> be required to improve performances.
> >>
> >> Adding such exception will not have any negative impact on perfs,
> >> readability, etc. It will simply help a lot the defect tracking when
> >> someone will face the issue and see the stack trace.
> >>
> >> JM
> >>
> >> 2012/7/9, Michael Segel <[EMAIL PROTECTED]>:
> >>> Jean-Marc,
> >>>
> >>> I think you mis understood.
> >>> At run time, you can query HBase to find out the table schema and its
> column
> >>> families.
> >>>
> >>> While I agree that you are seeing poorly written exceptions, IMHO its
> easier
> >>> to avoid the problem in the first place.
> >>>
> >>> In a Map/Reduce in side the mapper class, you have everything you need
> to
> >>> get the table's schema.
> >>> From that you can see the column families.
> >>>
> >>>
> >>> HTH
> >>>
> >>> -Mike
> >>>
> >>> On Jul 9, 2012, at 8:42 AM, Jean-Marc Spaggiari wrote:
> >>>
> >>> In my case it was a codding issue. Used the wrong final byte array to
> >>> access the CF. So I agree, the CF is well known since you create the
> >>> table based on them. But maybe you have added some other CFs later and
> >>> something went wrong?
> >>>
> >>> It's just that based on the exception received, there is no indication
> >>> that there might be some issues with the CF. So you might end trying
> >>> to figure what the issue is far from where it's really.
> >>>
> >>> 2012/7/9, Michael Segel <[EMAIL PROTECTED]>:
> >>>> This may beg the question ...
> >>>> Why do you not know the CF?
> >>>>
> >>>> Your table schemas only consist of tables and CFs. So you should know
> >>>> them
> >>>> at the start of your job or m/r Mapper.setup();
> >>>>
> >>>>
> >>>> On Jul 9, 2012, at 7:25 AM, Jean-Marc Spaggiari wrote:
> >>>>
> >>>> Hi,
> >>>>
> >>>> When we try to add a value to a CF which does not exist on a table, we
> >>>> are getting the error below. I think this is not really giving the
> >>>> right information about the issue.
> >>>>
> >>>> Should it not be better to provide an exception like
> >>>> UnknownColumnFamillyException?
> >>>>
> >>>> JM
> >>>>
> >>>> org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException:
> >>>> Failed 1 action: DoNotRetryIOException: 1 time, servers with issues:
> >>>> phenom:60020,
> >>>>    at
> >>>>
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1591)
> >>>>    at
> >>>>
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1367)
> >>>>    at
> org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:945)
> >>>>    at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:801)
> >>>