Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase, mail # user - Re: Tune MapReduce over HBase to insert data


+
Gerrit Jansen van Vuuren 2013-01-08, 08:56
+
Ted Yu 2013-01-08, 06:36
+
Farrokh Shahriari 2013-01-08, 05:05
+
Ted Yu 2013-01-08, 05:12
+
Farrokh Shahriari 2013-01-08, 05:38
+
lars hofhansl 2013-01-08, 08:02
+
Bing Jiang 2013-01-08, 15:28
+
Asaf Mesika 2013-01-08, 19:01
+
Farrokh Shahriari 2013-01-13, 05:29
+
Anoop John 2013-01-13, 06:45
Copy link to this message
-
Re: Tune MapReduce over HBase to insert data
Ted Yu 2013-01-08, 05:48
Have you tuned the JVM parameter of hbase ?
If you have Ganglia, did you observe high variation in network latency on
the 6 nodes ?

HBase 0.92.2 has been released. Do you plan to upgrade to 0.92.2 or 0.94.3 ?

Cheers

On Mon, Jan 7, 2013 at 9:38 PM, Farrokh Shahriari <
[EMAIL PROTECTED]> wrote:

> Hi again,
> I'm using HBase 0.92.1-cdh4.0.0.
> I have two server machine with 48Gb RAM,12 physical core & 24 logical core
> that contain 12 nodes(6 nodes on each server). Each node has 8Gb RAM & 2
> VCPU.
> I've set some parameter that get better result like set WAL=off on put,but
> some parameters like Heap-size,Deferred log flush don't help me.
> Beside that I have another question,why each time I've run mapreduce,I've
> got different result time while all the config & hardware are same & not
> change ?
>
> Tnx you guys
>
> On Tue, Jan 8, 2013 at 8:42 AM, Ted Yu <[EMAIL PROTECTED]> wrote:
>
> > Have you read through http://hbase.apache.org/book.html#performance ?
> >
> > What version of HBase are you using ?
> >
> > Cheers
> >
> > On Mon, Jan 7, 2013 at 9:05 PM, Farrokh Shahriari <
> > [EMAIL PROTECTED]> wrote:
> >
> > > Hi there
> > > I have a cluster with 12 nodes that each of them has 2 core of CPU.
> Now,I
> > > want insert large data about 2Gb in 80 sec ( or 6Gb in 240sec ). I've
> > used
> > > Map-Reduce over hbase,but I can't achieve proper result .
> > > I'd be glad if you tell me what I can do to get better result or which
> > > parameters should I config or tune to improve Map-Reduce/Hbase
> > performance
> > > ?
> > >
> > > Tnx
> > >
> >
>
+
Bing Jiang 2013-01-13, 09:31
+
Ted Yu 2013-01-13, 15:30
+
Farrokh Shahriari 2013-01-14, 05:58
+
Bing Jiang 2013-01-15, 01:33
+
Farrokh Shahriari 2013-01-16, 11:20
+
Adrien Mogenet 2013-02-04, 19:53