Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> JsonLoader schema field order shouldn't matter


+
Tim Sell 2013-01-07, 19:56
+
meghana narasimhan 2013-01-07, 21:32
+
Tim Sell 2013-01-08, 01:03
Copy link to this message
-
Re: JsonLoader schema field order shouldn't matter
Currently the JsonLoader does assume ordering of the fields.  It does not do any name matching against the given schema to find the right field.

Alan.

On Jan 7, 2013, at 11:56 AM, Tim Sell wrote:

> When using JsonLoader with Pig 0.10.0
>
> if I have an input.json file that looks like this:
>
> {"date": "2007-08-25", "id": 16}
> {"date": "2007-09-08", "id": 17}
> {"date": "2007-09-15", "id": 18}
>
> And I use
>
> a = LOAD 'input.json' USING JsonLoader('id:int,date:chararray');
> DUMP a;
>
> I get errors when it tries to force the date fields into an integer.
>
> Shouldn't this work independent of the ordering of the schema fields?
> Json writers generally don't make guarantees about the ordering.
>
> One alternative (though annoying) would to be use elephant bird
> instead, but I can't get that to compile against hadoop 2.0.0 and Pig
> 0.10.0.
>
> ~Tim
+
Tim Sell 2013-01-08, 01:02
+
Alan Gates 2013-01-08, 17:38
+
Dmitriy Ryaboy 2013-01-11, 03:35
+
Ruslan Al-Fakikh 2013-04-05, 00:51