Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # user - Direct HDFS access from a streaming job


Copy link to this message
-
Re: Direct HDFS access from a streaming job
Keith Wiley 2011-03-24, 16:09
On Mar 24, 2011, at 8:31 AM, Harsh J wrote:

> Hello,
>
> On Thu, Mar 24, 2011 at 8:45 PM, Keith Wiley <[EMAIL PROTECTED]> wrote:
>> Thanks.  Actually, I think that with reference to the passage I quoted in my first post, the unstated intent was to simply do a system() call and invoke "hadoop fs -get" or "hadoop fs -copyToLocal".
>
> Some would consider that a bit ugly in C/C++, but its almost the same
> thing I guess. For shell programs passed to streaming though, its a
> good solution.
I agree.  I like the API you found.  I was just saying that after posting my question, all references I found online that seemed to address that passage in the docs described the solution I then posted, that of directly invoking "hadoop get".

________________________________________________________________________________
Keith Wiley     [EMAIL PROTECTED]     keithwiley.com    music.keithwiley.com

"I do not feel obliged to believe that the same God who has endowed us with
sense, reason, and intellect has intended us to forgo their use."
                                           --  Galileo Galilei
________________________________________________________________________________