Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> HDFS read/write speeds, and read optimization


Copy link to this message
-
Re: HDFS read/write speeds, and read optimization
Also, It would be interesting to know "data.replication" setting you have
for this benchmark?

On Sun, Jan 3, 2010 at 8:51 AM, Andreas Kostyrka <[EMAIL PROTECTED]>wrote:

> Well, that all depends on many details, but:
>
> -) are you really using 4 discs (configured correctly as data
> directories?)
>
> -) What hdd/connection technology?
>
> -) And 77MB/s would match up curiously well with 1Gbit networking cards?
> So you sure that you are testing a completely local setup? Where's your
> name node running then?
>
> Andreas
>
>
> Am Sonntag, den 03.01.2010, 00:54 +0200 schrieb Stas Oskin:
> > Hi.
> >
> > Can anyone advice on the subject below?
> >
> > Thanks!
> >
> > On Mon, Dec 28, 2009 at 9:01 PM, Stas Oskin <[EMAIL PROTECTED]>
> wrote:
> >
> > > Hi.
> > >
> > > Going back to the subject, has anyone ever bench-marked small (10 - 20
> > > node) HDFS clusters?
> > >
> > > I did my own speed checks, and it seems I can reach ~77Mbps, on a
> quad-disk
> > > node. This comes to ~19Mbps per disk, which seems quite low in my
> opinion.
> > >
> > > Can anyone advice about this?
> > >
> > > Thanks.
> > >
>
>
--
~Rajesh.B
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB