Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # user - HDFS read/write speeds, and read optimization


Copy link to this message
-
Re: HDFS read/write speeds, and read optimization
Rajesh Balamohan 2010-01-03, 13:18
Also, It would be interesting to know "data.replication" setting you have
for this benchmark?

On Sun, Jan 3, 2010 at 8:51 AM, Andreas Kostyrka <[EMAIL PROTECTED]>wrote:

> Well, that all depends on many details, but:
>
> -) are you really using 4 discs (configured correctly as data
> directories?)
>
> -) What hdd/connection technology?
>
> -) And 77MB/s would match up curiously well with 1Gbit networking cards?
> So you sure that you are testing a completely local setup? Where's your
> name node running then?
>
> Andreas
>
>
> Am Sonntag, den 03.01.2010, 00:54 +0200 schrieb Stas Oskin:
> > Hi.
> >
> > Can anyone advice on the subject below?
> >
> > Thanks!
> >
> > On Mon, Dec 28, 2009 at 9:01 PM, Stas Oskin <[EMAIL PROTECTED]>
> wrote:
> >
> > > Hi.
> > >
> > > Going back to the subject, has anyone ever bench-marked small (10 - 20
> > > node) HDFS clusters?
> > >
> > > I did my own speed checks, and it seems I can reach ~77Mbps, on a
> quad-disk
> > > node. This comes to ~19Mbps per disk, which seems quite low in my
> opinion.
> > >
> > > Can anyone advice about this?
> > >
> > > Thanks.
> > >
>
>
--
~Rajesh.B