Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Create rdbms like sequence in Pig on Pig Relation


Copy link to this message
-
Re: Create rdbms like sequence in Pig on Pig Relation
Right.. Russel, the reason DynamicInvokers weren't working is that
InvokeForString expects a function that returns a String. randomUUID
returns a UUID, not a String.
You could of course call this trivially using jruby udfs (less work
than the java version).

D

On Sun, May 27, 2012 at 2:39 PM, Dragan Nedeljkovic <[EMAIL PROTECTED]> wrote:
> You have to call UUID.randomUUID() to get an UUID, but you cannot use DEFINE
> to do that since DEFINE does not support methods that return arbitrary classes.
>
> Wrapping it into an UDF, works just fine,
>
> package piggybank;
>
> import java.io.IOException;
> import java.util.UUID;
>
> import org.apache.pig.EvalFunc;
> import org.apache.pig.data.Tuple;
>
> public class CreateUUID
> extends EvalFunc<String>
> {
> public String exec(Tuple input)
> throws IOException
> {
> try
> {
> return UUID.randomUUID().toString();
> }
> catch(Exception e)
> {
> // Throwing an exception will cause the task to fail.
> throw new IOException("Something bad happened!", e);
> }
> }
> }
> // eof
>
>
> register 'mypiggybank.jar';
> define CreateUUID piggybank.CreateUUID();
>
> input_lines = LOAD 'test_CreateUUID.in' AS (line:chararray);
> describe input_lines;
> dump input_lines;
>
> new_list = FOREACH input_lines GENERATE line, CreateUUID();
> describe new_list;
> dump new_list;
>
> -- eof
>
>
>>________________________________
>> From: Russell Jurney <[EMAIL PROTECTED]>
>>To: [EMAIL PROTECTED]
>>Sent: Sunday, May 27, 2012 4:56:07 PM
>>Subject: Re: Create rdbms like sequence in Pig on Pig Relation
>>
>>It helps, but I am not able to invoke java.util.UUID.toString, maybe
>>because it doesn't take an argument.  This is from the docs:
>>
>>DEFINE UrlDecode InvokeForString('java.net.URLDecoder.decode', 'String
>>String');
>>encoded_strings = LOAD 'encoded_strings.txt' as (encoded:chararray);
>>decoded_strings = FOREACH encoded_strings GENERATE UrlDecode(encoded,
>>'UTF-8');
>>
>>
>>Maybe I forgot, but is this how I do it?
>>
>>DEFINE UUID InvokeForString('java.util.UUID.toString');
>>with_uuid = FOREACH my_stuff generate UUID(), *;
>>
>>
>>Sorry, I only understand example code - not APIs. My Java is quite weak.
>>
>>http://docs.oracle.com/javase/6/docs/api/java/util/UUID.html#toString()
>>
>>On Sun, May 27, 2012 at 2:33 AM, Subir S <[EMAIL PROTECTED]> wrote:
>>
>>> I hope this helps. DynamicInvoker feature in Pig. Added in 0.8.0
>>>
>>>
>>> http://squarecog.wordpress.com/2010/08/20/upcoming-features-in-pig-0-8-dynamic-invokers/
>>>
>>> Thanks
>>>
>>> On 5/24/12, Russell Jurney <[EMAIL PROTECTED]> wrote:
>>> > Thanks, I mean how do you invoke it directly in grunt> from Pig?
>>> >
>>> > I keep messing it up for the last 30 minutes. Should I check the settings
>>> > on my pacemaker, I feel like Fabio on NyQuil messing with this.
>>> >
>>> > On Wed, May 23, 2012 at 10:19 PM, Subir S <[EMAIL PROTECTED]>
>>> > wrote:
>>> >
>>> >> Hope this helps ->
>>> >> http://www.javapractices.com/topic/TopicAction.do?Id=56
>>> >>
>>> >> and this ->
>>> >>
>>> >>
>>> http://docs.oracle.com/javase/1.5.0/docs/api/java/util/UUID.html#randomUUID%28%29
>>> >>
>>> >> Thanks
>>> >>
>>> >>
>>> >>
>>> >> On Thu, May 24, 2012 at 10:42 AM, Russell Jurney
>>> >> <[EMAIL PROTECTED]>wrote:
>>> >>
>>> >> > How do you invoke java.util.UUID.randomUUID?  There is no invoker that
>>> >> > doesn't take an arg?
>>> >> >
>>> >> > On Sun, May 20, 2012 at 6:26 PM, Rajesh Balamohan <
>>> >> > [EMAIL PROTECTED]> wrote:
>>> >> >
>>> >> > > I dont think so. However, its a single line java command. You can
>>> >> create
>>> >> > > customUDF for this and use in your code.
>>> >> > >
>>> >> > > java.util.UUID.randomUUID();
>>> >> > >
>>> >> > > ~Rajesh.B
>>> >> > >
>>> >> > > On Sun, May 20, 2012 at 8:15 AM, DIPESH KUMAR SINGH
>>> >> > > <[EMAIL PROTECTED]>wrote:
>>> >> > >
>>> >> > > > Thanks Rajesh.
>>> >> > > >
>>> >> > > > Is GUID a built in UDF?
>>> >> > > >
>>> >> > > >
>>> >> > > > --
>>> >> > > > Dipesh