-Re: issue about total input byte of MR job
Ranjini Rathinam 2013-12-03, 08:40
How to process a xml file via mapreduce and load them into hbase table.
please suggest with sample code.
Thanks in advance.
On Tue, Dec 3, 2013 at 1:58 PM, ch huang <[EMAIL PROTECTED]> wrote:
> i run the MR job,at the MR output i see
> 13/12/03 14:02:28 INFO mapreduce.JobSubmitter: number of splits:2717
> because my each data block size is 64M,so total byte is 2717*64M/1024= 170G
> but in the summary of end i see follow info ,so the HDFS read byte is
> 126792190158/1024/1024/1024 = 118G ,the two number is not very close ,why?
> File System Counters
> FILE: Number of bytes read=9642910241
> FILE: Number of bytes written=120327706125
> FILE: Number of read operations=0
> FILE: Number of large read operations=0
> FILE: Number of write operations=0
> HDFS: Number of bytes read=126792190158
> HDFS: Number of bytes written=0
> HDFS: Number of read operations=8151
> HDFS: Number of large read operations=0
> HDFS: Number of write operations=0