Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Limits on HBase


Copy link to this message
-
Re: Limits on HBase
but yes you will not be having different versions of those objects as they
are not stored as such in a table. So, that's the down side. In case your
objects are write once read multi types, I think it should work.

Let's see what others say :)

~Himanshu
On Tue, Sep 7, 2010 at 12:49 AM, Himanshu Vashishtha <[EMAIL PROTECTED]
> wrote:

> Assuming you will be using hdfs as the file system: wouldn't saving those
> large objects in the fs and keeping a pointer to them in a hbase table serve
> the purpose.
>
> [I haven't done it myself but I can't see it not working. In fact, I
> remember reading it somewhere in the list.]
>
> ~Himanshu
>
>
> On Mon, Sep 6, 2010 at 11:40 PM, William Kang <[EMAIL PROTECTED]>wrote:
>
>> Hi JG,
>> Thanks for your reply. As far as I have read in Hbase's documentation and
>> wiki, the cell size is not supposed to be larger than 10 MB. For the row,
>> I
>> am not quite sure, but it looks like 256 MB is the upper limit. I am
>> considering store some binary data used to be stored in RDBM blob field.
>> The
>> size of those binary objects may vary from hundreds of KB to hundreds of
>> MB.
>> What would be a good way to use Hbase for it? We really want to use hbase
>> to
>> avoid that scaling problem.
>> Many thanks.
>>
>>
>> William
>>
>> On Mon, Sep 6, 2010 at 7:10 PM, Jonathan Gray <[EMAIL PROTECTED]> wrote:
>>
>> > I'm not sure what you mean by "optimized cell size" or whether you're
>> just
>> > asking about practical limits?
>> >
>> > HBase is generally used with cells in the range of tens of bytes to
>> > hundreds of kilobytes.  However, I have used it with cells that are
>> several
>> > megabytes, up to about 50MB.  Up at that level, I have seen some weird
>> > performance issues.
>> >
>> > The most important thing is to be sure to tweak all of your settings.
>>  If
>> > you have 20MB cells, you need to be sure to increase the flush size
>> beyond
>> > 64MB and the split size beyond 256MB.  You also need enough memory to
>> > support all this large object allocation.
>> >
>> > And of course, test test test.  That's the easiest way to see if what
>> you
>> > want to do will work :)
>> >
>> > When you run into problems, e-mail the list.
>> >
>> > As far as row size is concerned, the only issue is that a row can never
>> > span multiple regions so a given row can only be in one region and thus
>> be
>> > hosted on one server at a time.
>> >
>> > JG
>> >
>> > > -----Original Message-----
>> > > From: William Kang [mailto:[EMAIL PROTECTED]]
>> > > Sent: Monday, September 06, 2010 1:57 PM
>> > > To: hbase-user
>> > > Subject: Limits on HBase
>> > >
>> > > Hi folks,
>> > > I know this question may have been asked many times, but I am
>> wondering
>> > > if
>> > > there is any update on the optimized cell size (in megabytes) and row
>> > > size
>> > > (in megabytes)? Many thanks.
>> > >
>> > >
>> > > William
>> >
>>
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB