Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> FW: NNbench and MRBench


Copy link to this message
-
Re: FW: NNbench and MRBench
El 5/7/2011 8:53 PM, [EMAIL PROTECTED] escribió:
> Hi guys,
>
> I have a cluster of 16 machines running Hadoop. Now I want to do some benchmark on this cluster with the "nnbench" and "mrbench".
> I'm new to the hadoop thing and have no one to refer to. I don't know what the supposed result should I have?
> Now for mrbench, I have an average time of 22sec for a one map job. Is this too bad? What the supposed results might be?
>
> For nnbench, what's the supposed results? Below is my result.
> ===============>                             Date&  time: 2011-05-05 20:40:25,459
>
>                          Test Operation: rename
>                              Start time: 2011-05-05 20:40:03,820
>                             Maps to run: 1
>                          Reduces to run: 1
>                      Block Size (bytes): 1
>                          Bytes to write: 0
>                      Bytes per checksum: 1
>                         Number of files: 10000
>                      Replication factor: 1
>              Successful file operations: 10000
>
>          # maps that missed the barrier: 0
>                            # exceptions: 0
>
>                             TPS: Rename: 1763
>              Avg Exec time (ms): Rename: 0.5672
>                    Avg Lat (ms): Rename: 0.4844
> null
>
>                   RAW DATA: AL Total #1: 4844
>                   RAW DATA: AL Total #2: 0
>                RAW DATA: TPS Total (ms): 5672
>         RAW DATA: Longest Map Time (ms): 5672.0
>                     RAW DATA: Late maps: 0
>               RAW DATA: # of exceptions: 0
> ============================> One more question, when I set maps number to bigger, I get all zeros results:
> ============================> Test Operation: create_write
>                              Start time: 2011-05-03 23:22:39,239
>                             Maps to run: 160
>                          Reduces to run: 160
>                      Block Size (bytes): 1
>                          Bytes to write: 0
>                      Bytes per checksum: 1
>                         Number of files: 1
>                      Replication factor: 1
>              Successful file operations: 0
>
>          # maps that missed the barrier: 0
>                            # exceptions: 0
>
>                 TPS: Create/Write/Close: 0
> Avg exec time (ms): Create/Write/Close: 0.0
>              Avg Lat (ms): Create/Write: NaN
>                     Avg Lat (ms): Close: NaN
>
>                   RAW DATA: AL Total #1: 0
>                   RAW DATA: AL Total #2: 0
>                RAW DATA: TPS Total (ms): 0
>         RAW DATA: Longest Map Time (ms): 0.0
>                     RAW DATA: Late maps: 0
>               RAW DATA: # of exceptions: 0
> ====================>
> Can anyone point me to some documents?
> I really appreciate your help :)
>
> Thanks,
> stanley
>    
You can use these resources:
http://www.michael-noll.com/blog/2011/04/09/benchmarking-and-stress-testing-an-hadoop-cluster-with-terasort-testdfsio-nnbench-mrbench/
http://answers.oreilly.com/topic/460-how-to-benchmark-a-hadoop-cluster/
http://wiki.apache.org/hadoop/HardwareBenchmarks
http://www.quora.com/Apache-Hadoop/Are-there-any-good-Hadoop-benchmark-problems

Regards

--
Marcos Luís Ortíz Valmaseda
  Software Engineer (Large-Scaled Distributed Systems)
  University of Information Sciences,
  La Habana, Cuba
  Linux User # 418229
  http://about.me/marcosortiz
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB