Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> RE: size of input files


Copy link to this message
-
RE: size of input files
Do the counters provide the input file size ? I mean is bytes read equal to input file size ? Is there any log where I could find input file size submitted to each job. I believed that bytes read from fs is different from the input file size to the job.

*------------------------*

Cheers !!!

Siddharth Tiwari

Have a refreshing day !!!
"Every duty is holy, and devotion to duty is the highest form of worship of God.”

"Maybe other people will try to limit me but I don't limit myself"
From: [EMAIL PROTECTED]
Date: Sun, 2 Jun 2013 23:26:08 +0530
Subject: Re: size of input files
To: [EMAIL PROTECTED]

Counters can help. Input to mr is a directory. The counters can point to the number of bytes read from that fs directory.

Rahul
On Sun, Jun 2, 2013 at 11:22 PM, Siddharth Tiwari <[EMAIL PROTECTED]> wrote:

Hi Friends,
Is there a way to find out what was the size of the input file to each of the jobs from the logs or any other place for all jobs submitted ?
Please help
*------------------------*

Cheers !!!

Siddharth Tiwari

Have a refreshing day !!!
"Every duty is holy, and devotion to duty is the highest form of worship of God.”

"Maybe other people will try to limit me but I don't limit myself"
     

     
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB