Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> HTable.put(List<Put> puts) perform batch insert?


Copy link to this message
-
Re: HTable.put(List<Put> puts) perform batch insert?
Thank you, I will look into these packages :)

On Sun, Jan 16, 2011 at 4:17 AM, Otis Gospodnetic <
[EMAIL PROTECTED]> wrote:

> Hi,
>
> Re HBaseHUT - Alex didn't mention it, but he did a really nice and clear
> writeup
> of it in this post:
>
> http://blog.sematext.com/2010/12/16/deferring-processing-updates-to-increase-hbase-write-performance/
>
>
> Otis
> ----
> Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch
> Lucene ecosystem search :: http://search-lucene.com/
>
>
>
> ----- Original Message ----
> > From: Alex Baranau <[EMAIL PROTECTED]>
> > To: [EMAIL PROTECTED]
> > Sent: Tue, January 11, 2011 10:51:28 AM
> > Subject: Re: HTable.put(List<Put> puts) perform batch insert?
> >
> > Re HBaseHUT J-D was correct: you will gain speed with it in case you need
> > Get  & Put operation to perform your updates.
> >
> > Don't forget to play with  writeToWAL, writeBuffer (with autoFlush=false)
> > attributes!
> >
> > Alex  Baranau
> > ----
> > Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch - Hadoop -
> HBase
> >
> > On  Mon, Jan 10, 2011 at 10:45 PM, Weishung Chung <[EMAIL PROTECTED]>
> wrote:
> >
> > >  Ok, i will test it, thanks again :)
> > >
> > > On Mon, Jan 10, 2011 at 1:53  PM, Jean-Daniel Cryans <
> [EMAIL PROTECTED]
> > >  >wrote:
> > >
> > > > Depending on the level of super fastness you  need, it may or may not
> > > > be fast enough. Better to test it, as  usual.
> > > >
> > > > J-D
> > > >
> > > > On Mon, Jan 10,  2011 at 11:12 AM, Weishung Chung <
> [EMAIL PROTECTED]>
> > > >  wrote:
> > > > > Multiple batches of 10k *new/updated* rows at any time  to
> different
> > > > tables
> > > > > by different clients  simultaneously. I want these multiple batches
> of
> > > > > insertions to  be done super fast. At the same time, I would like
> to be
> > > >  able
> > > > > to scale up to 100k rows at a time (the goal).  Now,  I am building
> a
> > > > cluster
> > > > > of size 6 to 7  nodes.
> > > > >
> > > > > On Mon, Jan 10, 2011 at 1:03 PM,  Jean-Daniel Cryans <
> > > [EMAIL PROTECTED]
> > > >  >wrote:
> > > > >
> > > > >> lotsa rows? That's 1k or 1B?  Inside a OLTP system or OLAP?
> > > > >>
> > > > >>  J-D
> > > > >>
> > > > >> On Mon, Jan 10, 2011 at 10:58  AM, Weishung Chung <
> [EMAIL PROTECTED]>
> > > >  >> wrote:
> > > > >> > Jonathan, awesome, best of breed  APIs!
> > > > >> > Jean, I would like to insert lotsa new rows  with many columns
> in a
> > > > >> > particular column family*  **programmatically in batch just like
> the
> > > > jdbc
> > > >  >> > addBatch method.*
> > > > >> > *Thanks  again.*
> > > > >> >
> > > > >> >
> > > >  >>
> > > > >
> > > >
> > >
> >
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB