Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - Tune MapReduce over HBase to insert data


Copy link to this message
-
Re: Tune MapReduce over HBase to insert data
Ted Yu 2013-01-08, 05:48
Have you tuned the JVM parameter of hbase ?
If you have Ganglia, did you observe high variation in network latency on
the 6 nodes ?

HBase 0.92.2 has been released. Do you plan to upgrade to 0.92.2 or 0.94.3 ?

Cheers

On Mon, Jan 7, 2013 at 9:38 PM, Farrokh Shahriari <
[EMAIL PROTECTED]> wrote:

> Hi again,
> I'm using HBase 0.92.1-cdh4.0.0.
> I have two server machine with 48Gb RAM,12 physical core & 24 logical core
> that contain 12 nodes(6 nodes on each server). Each node has 8Gb RAM & 2
> VCPU.
> I've set some parameter that get better result like set WAL=off on put,but
> some parameters like Heap-size,Deferred log flush don't help me.
> Beside that I have another question,why each time I've run mapreduce,I've
> got different result time while all the config & hardware are same & not
> change ?
>
> Tnx you guys
>
> On Tue, Jan 8, 2013 at 8:42 AM, Ted Yu <[EMAIL PROTECTED]> wrote:
>
> > Have you read through http://hbase.apache.org/book.html#performance ?
> >
> > What version of HBase are you using ?
> >
> > Cheers
> >
> > On Mon, Jan 7, 2013 at 9:05 PM, Farrokh Shahriari <
> > [EMAIL PROTECTED]> wrote:
> >
> > > Hi there
> > > I have a cluster with 12 nodes that each of them has 2 core of CPU.
> Now,I
> > > want insert large data about 2Gb in 80 sec ( or 6Gb in 240sec ). I've
> > used
> > > Map-Reduce over hbase,but I can't achieve proper result .
> > > I'd be glad if you tell me what I can do to get better result or which
> > > parameters should I config or tune to improve Map-Reduce/Hbase
> > performance
> > > ?
> > >
> > > Tnx
> > >
> >
>