-Re: Direct HDFS access from a streaming job
Keith Wiley 2011-03-24, 16:09
On Mar 24, 2011, at 8:31 AM, Harsh J wrote:
> On Thu, Mar 24, 2011 at 8:45 PM, Keith Wiley <[EMAIL PROTECTED]> wrote:
>> Thanks. Actually, I think that with reference to the passage I quoted in my first post, the unstated intent was to simply do a system() call and invoke "hadoop fs -get" or "hadoop fs -copyToLocal".
> Some would consider that a bit ugly in C/C++, but its almost the same
> thing I guess. For shell programs passed to streaming though, its a
> good solution.
I agree. I like the API you found. I was just saying that after posting my question, all references I found online that seemed to address that passage in the docs described the solution I then posted, that of directly invoking "hadoop get".
Keith Wiley [EMAIL PROTECTED] keithwiley.com music.keithwiley.com
"I do not feel obliged to believe that the same God who has endowed us with
sense, reason, and intellect has intended us to forgo their use."
-- Galileo Galilei