-Re: UDF discussion? Here or on the dev list? / Json Loading
Jacob Perkins 2011-01-30, 21:01
Yes, you would have to distribute ruby (though. it's typically
installed by default) as well as the wukong and json libraries to all
the nodes in the cluster. Unfortunately this isn't something wukong
gives you for free at the moment though it is planned.
As far as I know Pig doesn't do anything more complex than launch a
hadoop streaming job and use the output in the subsequent steps
btw I write 90% of my mr jobs using either wukong or Pig. Only when
it's absolutely required do I use a language with as much overhead as
Sent from my iPhone
On Jan 30, 2011, at 2:09 PM, Alex McLintock <[EMAIL PROTECTED]>
> On 29 January 2011 13:43, Jacob Perkins <[EMAIL PROTECTED]>
>> Write a map only wukong script that parses the json as you want it.
>> the example here:
> Hi Jacob,
> Thanks very much for helping me out. I haven't heard of Wukong before.
> I am a bit concerned though by adding Ruby into my tool stack as
> well as
> Pig. It seems like a step too far.
> Presumably I have to distribute Ruby and Wukong across all my job
> nodes in
> the same way as if I were writing perl or C++ streaming programs.
> With STREAMing - the script is launched once per file, right, not
> once per