Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> the performance of HDFS


Copy link to this message
-
Re: the performance of HDFS
Perhaps lshw would help you.

ubuntu:~$ sudo lshw
   ...
        *-storage
             description: RAID bus controller
             product: SB700/SB800 SATA Controller [Non-RAID5 mode]
             vendor: ATI Technologies Inc
             physical id: 11
             bus info: pci@0000:00:11.0
             logical name: scsi0
             logical name: scsi1
             version: 00
             width: 32 bits
             clock: 66MHz
             capabilities: storage pm bus_master cap_list emulated
             configuration: driver=ahci latency=64
             resources: irq:22 ioport:b000(size=8) ioport:a000(size=4)
ioport:9000(size=8) ioport:8000(size=4) ioport:7000(size=16)
memory:fe7ffc00-fe7fffff
           *-disk
                description: ATA Disk
                product: ST3750528AS
                vendor: Seagate
                physical id: 0
                bus info: scsi@0:0.0.0
   ...

On Tue, Jan 25, 2011 at 6:29 PM, Da Zheng <[EMAIL PROTECTED]> wrote:

> No, each node in the cluster is powerful server. I was told the nodes are
> Dell
> Poweredge SC1435, but I cannot figure out the configuration of hard drives.
> Dell
> provides several possible hard drives for this model.
>
> On 1/25/11 7:59 PM, Ted Dunning wrote:
> > This is a really slow drive or controller.
> >
> > Consumer grade 3.5 inch 2TB drives typically can handle 100MB/s.  I would
> > suspect in the absence of real information that your controller is more
> > likely to be deficient than your drive.  If this is on a laptop or
> > something, then I withdraw my thought.
> >
> > On Tue, Jan 25, 2011 at 4:50 PM, Da Zheng <[EMAIL PROTECTED]> wrote:
> >
> >> The aggregate write-rate can get much higher if you use more drives, but
> a
> >>> single stream throughput is limited to the speed of one disk spindle.
> >>>
> >>>  You are right. I measure the performance of the hard drive. It seems
> the
> >> bottleneck is the hard drive, but the hard drive is a little too slow.
> The
> >> average writing rate is 50MB/s.
> >
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB