|
|
-
Hadoop I/O buffer sizeMark Olimpiati 2012-12-07, 02:35
Hi guys,
Need help figuring out Hadoop I/O buffer size. This link: http://developer.yahoo.com/blogs/hadoop/posts/2009/08/the_anatomy_of_hadoop_io_pipel/, implies that *io.file.buffersize* is 4K or 64K which makes me wonder. Is that the buffer which gets filled with records from disk when I have the following in code: *While( reader.next(Key,Value)) ; ? * If yes, when I change io.file.buffersize = 64M will that fetch 64MB of records into memory when I do "reader.next" ? if no, then how many records are fetched when I do one "reader.next(key,value)" ? Thank you, Mark |