Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce, mail # dev - The work-process file creation and block generation and InputSplit generation


Copy link to this message
-
The work-process file creation and block generation and InputSplit generation
Yoonmin Nam 2013-10-08, 10:41
Hello.

 

 

I want to see every detail of

 

1)     Block generation on hdfs when I put the file which has size over at

lease 64MB.

 

2)     Generating inputsplits given several blocks in hdfs.

 

 

So, at first I tried to find related unit test source codes and I found
TestFileCreationClient.java.

 

However, there is any specific reference such as Block, BlockMetaInfo and I
just found a FSDataOutputStream reference.

 

 

In addition, In the TestFileCreation.java in mapred test package, it is so
complicated and really hard to figure how this all processes were occurred.

 

 

Please give me some tips and advices or relative documents for these
processes.

 

 

Thanks!