Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS, mail # dev - HDFS splitting file to 64MB blocks


Copy link to this message
-
HDFS splitting file to 64MB blocks
Karim Awara 2013-09-24, 08:33
Hi,

I want to know when I upload a file from the local disk to hdfs in a
distributed environment (local cluster),  the file gets split into blocks
of 64MB each. assuming the file resides on the namenode, who splits the
file (namenode)?  what if the file resides on a datanode, does the scenario
change?

I want to change the way the file is split to 64MB chunks; I want to
use my own 'split partitioner' .. which class handles the split file into
64MB chunks in the hdfs?
--
Best Regards,
Karim Ahmed Awara

--

------------------------------
This message and its contents, including attachments are intended solely
for the original recipient. If you are not the intended recipient or have
received this message in error, please notify me immediately and delete
this message from your computer system. Any unauthorized use or
distribution is prohibited. Please consider the environment before printing
this email.