Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> the performance of HDFS

Copy link to this message
Re: the performance of HDFS
Perhaps lshw would help you.

ubuntu:~$ sudo lshw
             description: RAID bus controller
             product: SB700/SB800 SATA Controller [Non-RAID5 mode]
             vendor: ATI Technologies Inc
             physical id: 11
             bus info: pci@0000:00:11.0
             logical name: scsi0
             logical name: scsi1
             version: 00
             width: 32 bits
             clock: 66MHz
             capabilities: storage pm bus_master cap_list emulated
             configuration: driver=ahci latency=64
             resources: irq:22 ioport:b000(size=8) ioport:a000(size=4)
ioport:9000(size=8) ioport:8000(size=4) ioport:7000(size=16)
                description: ATA Disk
                product: ST3750528AS
                vendor: Seagate
                physical id: 0
                bus info: scsi@0:0.0.0

On Tue, Jan 25, 2011 at 6:29 PM, Da Zheng <[EMAIL PROTECTED]> wrote:

> No, each node in the cluster is powerful server. I was told the nodes are
> Dell
> Poweredge SC1435, but I cannot figure out the configuration of hard drives.
> Dell
> provides several possible hard drives for this model.
> On 1/25/11 7:59 PM, Ted Dunning wrote:
> > This is a really slow drive or controller.
> >
> > Consumer grade 3.5 inch 2TB drives typically can handle 100MB/s.  I would
> > suspect in the absence of real information that your controller is more
> > likely to be deficient than your drive.  If this is on a laptop or
> > something, then I withdraw my thought.
> >
> > On Tue, Jan 25, 2011 at 4:50 PM, Da Zheng <[EMAIL PROTECTED]> wrote:
> >
> >> The aggregate write-rate can get much higher if you use more drives, but
> a
> >>> single stream throughput is limited to the speed of one disk spindle.
> >>>
> >>>  You are right. I measure the performance of the hard drive. It seems
> the
> >> bottleneck is the hard drive, but the hard drive is a little too slow.
> The
> >> average writing rate is 50MB/s.
> >