Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS >> mail # user >> DFS and the RecordReader


+
Jay Vyas 2012-12-06, 21:37
+
Harsh J 2012-12-06, 22:15
Copy link to this message
-
Re: DFS and the RecordReader
This is an excellent explanation  thanks harsh...... Ill diagram this with some details out and share

> LineRecordReader has the calls you look for and wraps over a
> LineReader implementation, to take care of reading lines over block
> boundaries: [2] - Line 88
> LineReader has all the functional code to make it work for anyone
> reading lines off of text files: [3]
>
> [1] - http://svn.apache.org/viewvc/hadoop/common/tags/release-2.0.2-alpha/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/input/TextInputFormat.java?view=markup
> [2] - http://svn.apache.org/viewvc/hadoop/common/tags/release-2.0.2-alpha/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/input/LineRecordReader.java?view=markup
> [3] - http://svn.apache.org/viewvc/hadoop/common/tags/release-2.0.2-alpha/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/LineReader.java?view=markup
>
> On Fri, Dec 7, 2012 at 4:17 AM, Jay Vyas <[EMAIL PROTECTED]> wrote:
>> Hmm... so when a record reader calls fs.open(...) , I guess Im looking for
>> an example of how the input stream is created... ?
>
>
>
> --
> Harsh J