Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> size of input files


Copy link to this message
-
Re: size of input files
You could also get the size from the JT logs. See the line  containing :

*INFO org.apache.hadoop.mapred.JobInProgress: Input size for job*

Warm Regards,
Tariq
cloudfront.blogspot.com
On Mon, Jun 3, 2013 at 12:42 AM, Mohammad Tariq <[EMAIL PROTECTED]> wrote:

> Hello Siddharth,
>
>           You can find the exact filesize from the *File Input Format
> Counters* - *Bytes Read *by visiting the page of a particular job through
> the Map/Reduce Administration page.
>
> Warm Regards,
> Tariq
> cloudfront.blogspot.com
>
>
> On Mon, Jun 3, 2013 at 12:08 AM, Siddharth Tiwari <
> [EMAIL PROTECTED]> wrote:
>
>> Do the counters provide the input file size ? I mean is bytes read equal
>> to input file size ?
>> Is there any log where I could find input file size submitted to each
>> job. I believed that bytes read from fs is different from the input file
>> size to the job.
>>
>> **------------------------**
>> *Cheers !!!*
>> *Siddharth Tiwari*
>> Have a refreshing day !!!
>> *"Every duty is holy, and devotion to duty is the highest form of
>> worship of God.” *
>> *"Maybe other people will try to limit me but I don't limit myself"*
>>
>>
>> ------------------------------
>> From: [EMAIL PROTECTED]
>> Date: Sun, 2 Jun 2013 23:26:08 +0530
>> Subject: Re: size of input files
>> To: [EMAIL PROTECTED]
>>
>>
>> Counters can help. Input to mr is a directory. The counters can point to
>> the number of bytes read from that fs directory.
>>
>> Rahul
>>
>>
>> On Sun, Jun 2, 2013 at 11:22 PM, Siddharth Tiwari <
>> [EMAIL PROTECTED]> wrote:
>>
>> Hi Friends,
>>
>> Is there a way to find out what was the size of the input file to each of
>> the jobs from the logs or any other place for all jobs submitted ?
>>
>> Please help
>>
>>
>> **------------------------**
>> *Cheers !!!*
>> *Siddharth Tiwari*
>> Have a refreshing day !!!
>> *"Every duty is holy, and devotion to duty is the highest form of
>> worship of God.” *
>> *"Maybe other people will try to limit me but I don't limit myself"*
>>
>>
>>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB