Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> changing split size in Hadoop configuration


Copy link to this message
-
Re: changing split size in Hadoop configuration
For what it's worth, mapreduce.jobtracker.split.metainfo.maxsize is related
to the size of the file containing the information describing the input
splits. It is not related directly to the volume of data but to the number
of splits which might explode when using too many (small) files. It's
basically a safeguard. Alternatively, you might want to reduce the number
of splits ; raising the block size is one way to do it.

Bertrand Dechoux
On Mon, Jul 14, 2014 at 7:50 PM, Adam Kawa <[EMAIL PROTECTED]> wrote: