Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> HBase Writes With Large Number of Columns


Copy link to this message
-
Re: HBase Writes With Large Number of Columns
Final clarification:
bq. I am writing 1.5 KB of data per row across 40 columns.

So your schema is not sparse - you were writing to (all) 40 columns in
second case.

Thanks

On Mon, Mar 25, 2013 at 11:03 AM, Pankaj Misra
<[EMAIL PROTECTED]>wrote:

> Yes Ted, you are right, we are having table regions pre-split, and we see
> that both regions are almost evenly filled in both the tests.
>
> This does not seem to be a regression though, since we were getting good
> write rates when we had lesser number of columns.
>
> Thanks and Regards
> Pankaj Misra
>
>
> ________________________________________
> From: Ted Yu [[EMAIL PROTECTED]]
> Sent: Monday, March 25, 2013 11:15 PM
> To: [EMAIL PROTECTED]
> Cc: [EMAIL PROTECTED]
> Subject: Re: HBase Writes With Large Number of Columns
>
> Copying Ankit who raised the same question soon after Pankaj's initial
> question.
>
> On one hand I wonder if this was a regression in 0.94.5 (though unlikely).
>
> Did the region servers receive (relatively) same write load for the second
> test case ? I assume you have pre-split your tables in both cases.
>
> Cheers
>
> On Mon, Mar 25, 2013 at 10:18 AM, Pankaj Misra
> <[EMAIL PROTECTED]>wrote:
>
> > Hi Ted,
> >
> > Sorry for missing that detail, we are using HBase version 0.94.5
> >
> > Regards
> > Pankaj Misra
> >
> >
> > ________________________________________
> > From: Ted Yu [[EMAIL PROTECTED]]
> > Sent: Monday, March 25, 2013 10:29 PM
> > To: [EMAIL PROTECTED]
> > Subject: Re: HBase Writes With Large Number of Columns
> >
> > If you give us the version of HBase you're using, that would give us some
> > more information to help you.
> >
> > Cheers
> >
> > On Mon, Mar 25, 2013 at 9:55 AM, Pankaj Misra <
> [EMAIL PROTECTED]
> > >wrote:
> >
> > > Hi,
> > >
> > > The issue that I am facing is around the performance drop of Hbase,
> when
> > I
> > > was having 20 columns in a column family Vs now when I am having 40
> > columns
> > > in a column family. The number of columns have doubled and the
> > > ingestion/write speed has also dropped by half. I am writing 1.5 KB of
> > data
> > > per row across 40 columns.
> > >
> > > Are there any settings that I should look into for tweaking Hbase to
> > write
> > > higher number of columns faster?
> > >
> > > I would request community's help to let me know how can I write to a
> > > column family with large number of columns efficiently.
> > >
> > > Would greatly appreciate any help /clues around this issue.
> > >
> > > Thanks and Regards
> > > Pankaj Misra
> > >
> > > ________________________________
> > >
> > >
> > >
> > >
> > >
> > >
> > > NOTE: This message may contain information that is confidential,
> > > proprietary, privileged or otherwise protected by law. The message is
> > > intended solely for the named addressee. If received in error, please
> > > destroy and notify the sender. Any use of this email is prohibited when
> > > received in error. Impetus does not represent, warrant and/or
> guarantee,
> > > that the integrity of this communication has been maintained nor that
> the
> > > communication is free of errors, virus, interception or interference.
> > >
> >
> > ________________________________
> >
> >
> >
> >
> >
> >
> > NOTE: This message may contain information that is confidential,
> > proprietary, privileged or otherwise protected by law. The message is
> > intended solely for the named addressee. If received in error, please
> > destroy and notify the sender. Any use of this email is prohibited when
> > received in error. Impetus does not represent, warrant and/or guarantee,
> > that the integrity of this communication has been maintained nor that the
> > communication is free of errors, virus, interception or interference.
> >
>
> ________________________________
>
>
>
>
>
>
> NOTE: This message may contain information that is confidential,
> proprietary, privileged or otherwise protected by law. The message is
> intended solely for the named addressee. If received in error, please