|
|
-
Re: DFS and the RecordReaderHarsh J 2012-12-06, 23:07
Ah ok, understood what you seem to be looking for.
Lets follow the simple LineReader implementation in that case. TextInputFormat uses LineRecordReader: [1] - Line 52 LineRecordReader has the calls you look for and wraps over a LineReader implementation, to take care of reading lines over block boundaries: [2] - Line 88 LineReader has all the functional code to make it work for anyone reading lines off of text files: [3] [1] - http://svn.apache.org/viewvc/hadoop/common/tags/release-2.0.2-alpha/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/input/TextInputFormat.java?view=markup [2] - http://svn.apache.org/viewvc/hadoop/common/tags/release-2.0.2-alpha/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/input/LineRecordReader.java?view=markup [3] - http://svn.apache.org/viewvc/hadoop/common/tags/release-2.0.2-alpha/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/LineReader.java?view=markup On Fri, Dec 7, 2012 at 4:17 AM, Jay Vyas <[EMAIL PROTECTED]> wrote: > Hmm... so when a record reader calls fs.open(...) , I guess Im looking for > an example of how the input stream is created... ? -- Harsh J |