Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS >> mail # dev >> The work-process file creation and block generation.


Copy link to this message
-
The work-process file creation and block generation.
Hello.

 

I want to see every detail of

1)     Block generation on hdfs when I put the file which has size over at
lease 64MB.

2)     Generating inputsplits given several blocks in hdfs.

 

So, at first I tried to find related unit test source codes and I found
TestFileCreationClient.java.

However, there is any specific reference such as Block, BlockMetaInfo and I
just found a FSDataOutputStream reference.

 

In addition, In the TestFileCreation.java in mapred test package, it is so
complicated and really hard to figure how this all processes were occurred.

 

Please give me some tips and advices or relative documents for these
processes.

 

Thanks!

NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB