Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> Re: Cutting a line in between while creating blocks


Copy link to this message
-
Re: Cutting a line in between while creating blocks
HDFS is agnostic about the contents of the data you store. Think about
it: Line ending character is not the universal way for files to
separate their records.

This question's been asked several times before (search on
http://search-hadoop.com for example). Read
http://wiki.apache.org/hadoop/HadoopMapReduce to understand how
despite HDFS splitting at exactly the 64th megabyte, MR (or other HDFS
file reading operations) make sure to read records whole.

On Sat, Jan 4, 2014 at 3:59 PM, VJ Shalish <[EMAIL PROTECTED]> wrote:
> Hi,
>
> While creating the blocks for a file containing n number of lines, how does
> Hadoop take care of the problem of not Cutting a line in  between while
> creating blocks?
> Is it taken care of by Hadoop?
>
> Thanks
> Shalish.

--
Harsh J
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB