Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Monitor network traffic in hadoop


Copy link to this message
-
Re: Monitor network traffic in hadoop
Try something like:
http://tutorialgenius.blogspot.ca/2012/03/tcpdump-localhost.html

After you launch tcpdump it mentions "eth0", while your mapred-site.xml is
referring to "localhost" or "lo".  Perhaps...

(Sorry, I don't have a VM running and can't remember if it's "lo" "elle-o".)
On Sun, Dec 8, 2013 at 2:57 PM, navaz <[EMAIL PROTECTED]> wrote:

> Thanks. I have tried with sudo command and command executed successfully.
> But nothing is captured.
>
> hduser@ubuntu:~/Downloads$ sudo tcpdump port 13562
> [sudo] password for hduser:
> tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
> listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
>
> I am running the mapreduce program and this is the output.
>
> hduser@ubuntu:~/Downloads$ hadoop jar chiu-wordcount2.jar WordCount
> /user/hduser/getty/gettysburg.txt /user/hduser/getty/out8
> Warning: $HADOOP_HOME is deprecated.
>
> 13/12/08 10:43:47 WARN mapred.JobClient: Use GenericOptionsParser for
> parsing the arguments. Applications should implement Tool for the same.
> 13/12/08 10:43:47 INFO input.FileInputFormat: Total input paths to process
> : 1
> 13/12/08 10:43:47 INFO util.NativeCodeLoader: Loaded the native-hadoop
> library
> 13/12/08 10:43:47 WARN snappy.LoadSnappy: Snappy native library not loaded
> 13/12/08 10:43:48 INFO mapred.JobClient: Running job: job_201312081040_0001
> 13/12/08 10:43:49 INFO mapred.JobClient:  map 0% reduce 0%
> 13/12/08 10:43:54 INFO mapred.JobClient:  map 100% reduce 0%
> 13/12/08 10:44:02 INFO mapred.JobClient:  map 100% reduce 33%
> 13/12/08 10:44:03 INFO mapred.JobClient:  map 100% reduce 100%
> 13/12/08 10:44:04 INFO mapred.JobClient: Job complete:
> job_201312081040_0001
> 13/12/08 10:44:04 INFO mapred.JobClient: Counters: 29
> 13/12/08 10:44:04 INFO mapred.JobClient:   Job Counters
> 13/12/08 10:44:04 INFO mapred.JobClient:     Launched reduce tasks=1
> 13/12/08 10:44:04 INFO mapred.JobClient:     SLOTS_MILLIS_MAPS=5478
> 13/12/08 10:44:04 INFO mapred.JobClient:     Total time spent by all
> reduces waiting after reserving slots (ms)=0
> 13/12/08 10:44:04 INFO mapred.JobClient:     Total time spent by all maps
> waiting after reserving slots (ms)=0
> 13/12/08 10:44:04 INFO mapred.JobClient:     Launched map tasks=1
> 13/12/08 10:44:04 INFO mapred.JobClient:     Data-local map tasks=1
> 13/12/08 10:44:04 INFO mapred.JobClient:     SLOTS_MILLIS_REDUCES=8804
> 13/12/08 10:44:04 INFO mapred.JobClient:   File Output Format Counters
> 13/12/08 10:44:04 INFO mapred.JobClient:     Bytes Written=1106
> 13/12/08 10:44:04 INFO mapred.JobClient:   FileSystemCounters
> 13/12/08 10:44:04 INFO mapred.JobClient:     FILE_BYTES_READ=3059
> 13/12/08 10:44:04 INFO mapred.JobClient:     HDFS_BYTES_READ=1606
> 13/12/08 10:44:04 INFO mapred.JobClient:     FILE_BYTES_WRITTEN=108770
> 13/12/08 10:44:04 INFO mapred.JobClient:     HDFS_BYTES_WRITTEN=1106
> 13/12/08 10:44:04 INFO mapred.JobClient:   File Input Format Counters
> 13/12/08 10:44:04 INFO mapred.JobClient:     Bytes Read=1486
> 13/12/08 10:44:04 INFO mapred.JobClient:   Map-Reduce Framework
> 13/12/08 10:44:04 INFO mapred.JobClient:     Map output materialized
> bytes=3059
> 13/12/08 10:44:04 INFO mapred.JobClient:     Map input records=6
> 13/12/08 10:44:04 INFO mapred.JobClient:     Reduce shuffle bytes=3059
> 13/12/08 10:44:04 INFO mapred.JobClient:     Spilled Records=544
> 13/12/08 10:44:04 INFO mapred.JobClient:     Map output bytes=2509
> 13/12/08 10:44:04 INFO mapred.JobClient:     Total committed heap usage
> (bytes)=131600384
> 13/12/08 10:44:04 INFO mapred.JobClient:     CPU time spent (ms)=1010
> 13/12/08 10:44:04 INFO mapred.JobClient:     Combine input records=0
> 13/12/08 10:44:04 INFO mapred.JobClient:     SPLIT_RAW_BYTES=120
> 13/12/08 10:44:04 INFO mapred.JobClient:     Reduce input records=272
> 13/12/08 10:44:04 INFO mapred.JobClient:     Reduce input groups=138
> 13/12/08 10:44:04 INFO mapred.JobClient:     Combine output records=0
> 13/12/08 10:44:04 INFO mapred.JobClient:     Physical memory (bytes)
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB