Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> map reduce and sync

Copy link to this message
Re: map reduce and sync
Could you please clarify, are you opening the file in your mapper code and
reading from there ?


On Friday, February 22, 2013, Lucas Bernardi wrote:

> Hello there, I'm trying to use hadoop map reduce to process an open file. The
> writing process, writes a line to the file and syncs the file to readers.
> (org.apache.hadoop.fs.FSDataOutputStream.sync()).
> If I try to read the file from another process, it works fine, at least
> using
> org.apache.hadoop.fs.FSDataInputStream.
> hadoop -fs -tail also works just fine
> But it looks like map reduce doesn't read any data. I tried using the word
> count example, same thing, it is like if the file were empty for the map
> reduce framework.
> I'm using hadoop 1.0.3. and pig 0.10.0
> I need some help around this.
> Thanks!
> Lucas