Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop, mail # user - Re: DFS and the RecordReader

Copy link to this message
Re: DFS and the RecordReader
Harsh J 2012-12-06, 23:07
Ah ok, understood what you seem to be looking for.

Lets follow the simple LineReader implementation in that case.

TextInputFormat uses LineRecordReader: [1] - Line 52
LineRecordReader has the calls you look for and wraps over a
LineReader implementation, to take care of reading lines over block
boundaries: [2] - Line 88
LineReader has all the functional code to make it work for anyone
reading lines off of text files: [3]

[1] - http://svn.apache.org/viewvc/hadoop/common/tags/release-2.0.2-alpha/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/input/TextInputFormat.java?view=markup
[2] - http://svn.apache.org/viewvc/hadoop/common/tags/release-2.0.2-alpha/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/input/LineRecordReader.java?view=markup
[3] - http://svn.apache.org/viewvc/hadoop/common/tags/release-2.0.2-alpha/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/LineReader.java?view=markup

On Fri, Dec 7, 2012 at 4:17 AM, Jay Vyas <[EMAIL PROTECTED]> wrote:
> Hmm... so when a record reader calls fs.open(...) , I guess Im looking for
> an example of how the input stream is created... ?

Harsh J