Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> jsonStorage and pig maps, not sure whats wrong with this?


Copy link to this message
-
Re: jsonStorage and pig maps, not sure whats wrong with this?
Ah, pig 11 stores something but this seems to be generating an extra
"map_0" entry not the name value pair I was expecting.

I am getting

{"timestamp":"2013-01-31","map_0":{"@tags":"web"}}
{"timestamp":"2013-02-05","map_0":{"@tags":"web"}}

what I was hoping for was:

{"timestamp":"2013-01-31","@tags":"web"}
{"timestamp":"2013-02-05","@tags":"web"}

Anybody know how I can achieve this? I can't use

GENERATE platform as @tags
or
GENERATE platform as "@tags" or GENERATE platform as '@tags'

as it complains about the " ' @. Thought the TOMAP('@tags',platform) was a
clever work around.

On 1 March 2013 15:30, Robert McCarthy <[EMAIL PROTECTED]>wrote:

> Hi Dmitriy,
>
> This looks like it was a bug in pig 10 within the JsonStorage class as the
> same code now works in pig 11.
>
> The script is is
> A = load '/user/rob/events' USING PigStorage('\t') AS
> (dt_dt:chararray,platform:chararray);
> X = FOREACH A GENERATE dt_dt as timestamp,TOMAP('@tags',platform);
> STORE X INTO '/user/rob/events.json' USING JsonStorage;
>
> the data can be anything that has two columns that conform to chararray,
> it doesn't matter.
>
>
> On 27 February 2013 09:26, Dmitriy Ryaboy <[EMAIL PROTECTED]> wrote:
>
>> Sounds odd. Can you send a complete script that reproduces the error
>> (include sample data and load statements).
>>
>>
>> On Thu, Feb 21, 2013 at 2:55 AM, Robert McCarthy <
>> [EMAIL PROTECTED]> wrote:
>>
>> > If I have some information in A, that contains dt_dt and platform, I
>> want
>> > to store it in a different json format,
>> > So I can create a simple new bag like this
>> > X = FOREACH A GENERATE dt_dt as timestamp;
>> > and it stores ok in JsonStorage
>> > STORE X INTO '/user/rob/events.json' USING JsonStorage;
>> >
>> > I want to create a json object containing a pig map, because in the json
>> > object I want "@tags": platform,
>> > I can create the map ok like this..
>> > X = FOREACH A GENERATE dt_dt as timestamp,TOMAP('@tags',platform);
>> >
>> > Describe and illustrate all show the pig data as expected.
>> >
>> > describe x
>> > x: {timestamp: chararray,map[]}
>> >
>> > illustrate x
>> > --------------------------------------------------
>> > | x     | timestamp:chararray      | :map        |
>> > --------------------------------------------------
>> > |       | 2013-02-05               | {@tags=web} |
>> > --------------------------------------------------
>> >
>> > but it will not store using JsonStorage
>> > STORE X INTO '/user/rob/events.json' USING JsonStorage;
>> >
>> > Failed to parse: <line 1, column 15>  mismatched input '[' expecting
>> > IDENTIFIER_L
>> >
>> > I have tried without the @ to see if that was causing the error but get
>> the
>> > same result.
>> > Any help much appreciated.
>> >
>> > Rob,
>> >
>>
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB