Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # user - size of input files


Copy link to this message
-
Re: size of input files
Mohammad Tariq 2013-06-02, 19:20
You could also get the size from the JT logs. See the line  containing :

*INFO org.apache.hadoop.mapred.JobInProgress: Input size for job*

Warm Regards,
Tariq
cloudfront.blogspot.com
On Mon, Jun 3, 2013 at 12:42 AM, Mohammad Tariq <[EMAIL PROTECTED]> wrote:

> Hello Siddharth,
>
>           You can find the exact filesize from the *File Input Format
> Counters* - *Bytes Read *by visiting the page of a particular job through
> the Map/Reduce Administration page.
>
> Warm Regards,
> Tariq
> cloudfront.blogspot.com
>
>
> On Mon, Jun 3, 2013 at 12:08 AM, Siddharth Tiwari <
> [EMAIL PROTECTED]> wrote:
>
>> Do the counters provide the input file size ? I mean is bytes read equal
>> to input file size ?
>> Is there any log where I could find input file size submitted to each
>> job. I believed that bytes read from fs is different from the input file
>> size to the job.
>>
>> **------------------------**
>> *Cheers !!!*
>> *Siddharth Tiwari*
>> Have a refreshing day !!!
>> *"Every duty is holy, and devotion to duty is the highest form of
>> worship of God.” *
>> *"Maybe other people will try to limit me but I don't limit myself"*
>>
>>
>> ------------------------------
>> From: [EMAIL PROTECTED]
>> Date: Sun, 2 Jun 2013 23:26:08 +0530
>> Subject: Re: size of input files
>> To: [EMAIL PROTECTED]
>>
>>
>> Counters can help. Input to mr is a directory. The counters can point to
>> the number of bytes read from that fs directory.
>>
>> Rahul
>>
>>
>> On Sun, Jun 2, 2013 at 11:22 PM, Siddharth Tiwari <
>> [EMAIL PROTECTED]> wrote:
>>
>> Hi Friends,
>>
>> Is there a way to find out what was the size of the input file to each of
>> the jobs from the logs or any other place for all jobs submitted ?
>>
>> Please help
>>
>>
>> **------------------------**
>> *Cheers !!!*
>> *Siddharth Tiwari*
>> Have a refreshing day !!!
>> *"Every duty is holy, and devotion to duty is the highest form of
>> worship of God.” *
>> *"Maybe other people will try to limit me but I don't limit myself"*
>>
>>
>>
>