Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Create rdbms like sequence in Pig on Pig Relation


Copy link to this message
-
Re: Create rdbms like sequence in Pig on Pig Relation
Thanks, I mean how do you invoke it directly in grunt> from Pig?

I keep messing it up for the last 30 minutes. Should I check the settings
on my pacemaker, I feel like Fabio on NyQuil messing with this.

On Wed, May 23, 2012 at 10:19 PM, Subir S <[EMAIL PROTECTED]> wrote:

> Hope this helps -> http://www.javapractices.com/topic/TopicAction.do?Id=56
>
> and this ->
>
> http://docs.oracle.com/javase/1.5.0/docs/api/java/util/UUID.html#randomUUID%28%29
>
> Thanks
>
>
>
> On Thu, May 24, 2012 at 10:42 AM, Russell Jurney
> <[EMAIL PROTECTED]>wrote:
>
> > How do you invoke java.util.UUID.randomUUID?  There is no invoker that
> > doesn't take an arg?
> >
> > On Sun, May 20, 2012 at 6:26 PM, Rajesh Balamohan <
> > [EMAIL PROTECTED]> wrote:
> >
> > > I dont think so. However, its a single line java command. You can
> create
> > > customUDF for this and use in your code.
> > >
> > > java.util.UUID.randomUUID();
> > >
> > > ~Rajesh.B
> > >
> > > On Sun, May 20, 2012 at 8:15 AM, DIPESH KUMAR SINGH
> > > <[EMAIL PROTECTED]>wrote:
> > >
> > > > Thanks Rajesh.
> > > >
> > > > Is GUID a built in UDF?
> > > >
> > > >
> > > > --
> > > > Dipesh
> > > >
> > > > On Sun, May 20, 2012 at 8:06 AM, Rajesh Balamohan <
> > > > [EMAIL PROTECTED]> wrote:
> > > >
> > > > > If you do not bother about sequence number and the intention is to
> > > create
> > > > > just unique key, you can just use GUID which doesn't require any
> > > > > synchronization at all (all mappers can run in parallel).
> > > > >
> > > > > The approached I suggested in earlier mail comes into picture
> mainly
> > > for
> > > > > sequence number.
> > > > >
> > > > > ~Rajesh.B
> > > > >
> > > > > On Sun, May 20, 2012 at 8:02 AM, Rajesh Balamohan <
> > > > > [EMAIL PROTECTED]> wrote:
> > > > >
> > > > > > Pig doesn't have that facility yet. Moreover, its not very
> > efficient
> > > to
> > > > > do
> > > > > > this in PIG/MR as it requires synchronization.
> > > > > >
> > > > > > However, if this is unavoidable situation for you, following
> things
> > > can
> > > > > be
> > > > > > considered
> > > > > >
> > > > > > 1. Maintaining the seq number details in zookeeper
> > > > > > 2. Having a simple structure in HBase table (seqNumber -->
> Value).
> > > You
> > > > > can
> > > > > > get a bucket of values (ex: 1000-2000) from this and use it in
> your
> > > > UDF.
> > > > > > When the range depletes, you have to query/update HBase table
> (ex:
> > > > > > 3000-4000). There are corner cases which needs to be handled.
> > > > > >
> > > > > >
> > > > > > ~Rajesh.B
> > > > > >
> > > > > >
> > > > > > On Sat, May 19, 2012 at 12:04 AM, DIPESH KUMAR SINGH <
> > > > > > [EMAIL PROTECTED]> wrote:
> > > > > >
> > > > > >> Sorry, if my point was not clear.
> > > > > >>
> > > > > >> I wish to create a sequence on a pig relation.
> > > > > >>
> > > > > >> Say For example i have a relation with data:
> > > > > >> (John, A-1)
> > > > > >> (Jack, B-2)
> > > > > >> (Jim, C-1)
> > > > > >>
> > > > > >> I want to create sequence i.e to add one more column to the
> > > relation,
> > > > > like
> > > > > >> a counter and keep on increasing the count for each record read.
> > > > > Expected
> > > > > >> output should be something like this:
> > > > > >>
> > > > > >> (If 200 is the start sequence. )
> > > > > >> (John, A-1, 201)
> > > > > >> (Jack, B-2, 202)
> > > > > >> (Jim, C-1, 203)
> > > > > >>
> > > > > >> Could you please suggest to proceed on this?
> > > > > >>
> > > > > >> Thanks,
> > > > > >> Dipesh
> > > > > >>
> > > > > >> On Fri, May 18, 2012 at 6:50 AM, Thejas Nair <
> > > [EMAIL PROTECTED]>
> > > > > >> wrote:
> > > > > >>
> > > > > >> > What do you mean by 'rdbms like sequence' ?
> > > > > >> > Thanks,
> > > > > >> > Thejas
> > > > > >> >
> > > > > >> >
> > > > > >> > On 5/16/12 10:41 AM, DIPESH KUMAR SINGH wrote:
> > > > > >> >
> > > > > >> >> I want to create a rdbms like sequence on a Pig relation.
> > > > > >> >>
> > > > > >> >> Is there any existing UDF which could do this?

Russell Jurney twitter.com/rjurney [EMAIL PROTECTED] datasyndrome.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB