Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # user - a file can be used as a queue?


Copy link to this message
-
Re: a file can be used as a queue?
Joey Echeverria 2011-06-13, 11:23
This feature doesn't currently work. I don't remember the JIRA for it, but there's a ticket which will allow a reader to read from an HDFS file before it's closed. In that case, you implement a queue by having the producer write to the end of the file and the reader read from the beginning of the file.

I'm not sure if there will be a way to tell that a file is still being written, so you may need your own end of stream marker.

-Joey

On Jun 13, 2011, at 2:55, ltomuno <[EMAIL PROTECTED]> wrote:

> I heard a HDFS file as a producer - consumer queue, a file can be used as a queue? I am very confused