Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # dev >> NullPointerExceptipn in getPos of buffered FsDataInputStream


Copy link to this message
-
NullPointerExceptipn in getPos of buffered FsDataInputStream
I'm finding funny behaviour in the multifilewordcount example .  The getPos implementation in the RecirdReader is failing for me when the input steam is closed, and throwing a cryptic null pointer exception.

What is the correct behaviour for getPos  in a record reader, and how
should it behave when the underlying stream is null?  It appears this can
happen in the rawlocalfilesystem.  Not sure if its implemented more safely
in DistributedfileSYstem just yet.
   I've found that the getPos in the RawLocalFileSystem's input stream can
throw a null pointer exception if its underlying stream is closed.

I discovered this when playing with a custom record reader.

to patch it, I simply check if a call to "stream.available()" throws an
exception, and if so, I return 0 in the getPos() function.

The existing getPos() implementation is found here:

https://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20/src/examples/org/apache/hadoop/examples/MultiFileWordCount.java

What should be the correct behaviour of getPos() in the RecordReader?
http://stackoverflow.com/questions/18708832/hadoop-rawlocalfilesystem-and-getpos

--
Jay Vyas
http://jayunit100.blogspot.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB