Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig, mail # user - dereferencing bag of map


Copy link to this message
-
Re: dereferencing bag of map
Pradeep Gollakota 2013-06-21, 12:42
Suresh,

Look into using JsonStorage(). This seems to be what you're looking for.
http://pig.apache.org/docs/r0.10.0/func.html#jsonloadstore
On Fri, Jun 21, 2013 at 8:35 AM, Shahab Yunus <[EMAIL PROTECTED]>wrote:

> Have you tried flattening the bag first?
>
>
> On Fri, Jun 21, 2013 at 5:43 AM, Suresh Saggar <[EMAIL PROTECTED]> wrote:
>
> > Facing a similar challenge. Here X contains one column named 'metadata'
> of
> > type bytearray. But the actual content is a JSON i.e. the value of
> metadata
> > field is a JSON (keys as sId & cId) as shown below:
> >
> > grunt> describe X
> > X: {metadata: bytearray}
> >
> > grunt> dump X
> > ({"sId":"003_w","cId":"k"})
> > ({"sId":"001_rf","cId":"r"})
> > ({"sId":"001_rf","cId":"r"})
> > ({"sId":"004_rf","cId":"r"})
> >
> > Any idea how can I generate cId & sId as separate chararray columns? TIA
> >
> > Ss
> >
> > On Tue, Jun 18, 2013 at 5:52 AM, Pradeep Gollakota <[EMAIL PROTECTED]
> > >wrote:
> >
> > > What's the error you are seeing? What does you bag of maps look like?
> > What
> > > exactly is a userId? Is it a field or is it a key in the map?
> > >
> > >
> > > On Mon, Jun 17, 2013 at 8:18 PM, Jerry Lam <[EMAIL PROTECTED]>
> wrote:
> > >
> > > > Hi Pig users,
> > > >
> > > > anyone has experience in dereferencing a bag of maps? For instance
> (in
> > > the
> > > > example below), doc in the B contains maps of userId and time. I want
> > to
> > > > keep only userId in C. Pig throws an exception on C. Any help is
> > > > appreciated.
> > > >
> > > > A = LOAD 'data' AS doc:bytearray;
> > > >
> > > > B = FOREACH A GENERATE (bag{})doc;
> > > >
> > > > -- C = FOREACH B GENERATE doc.userId; // this doesn't work.
> > > >
> > > > Best Regards,
> > > >
> > > > Jerry
> > > >
> > >
> >
>