Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> JsonLoader schema field order shouldn't matter


Copy link to this message
-
JsonLoader schema field order shouldn't matter
When using JsonLoader with Pig 0.10.0

if I have an input.json file that looks like this:

{"date": "2007-08-25", "id": 16}
{"date": "2007-09-08", "id": 17}
{"date": "2007-09-15", "id": 18}

And I use

a = LOAD 'input.json' USING JsonLoader('id:int,date:chararray');
DUMP a;

I get errors when it tries to force the date fields into an integer.

Shouldn't this work independent of the ordering of the schema fields?
Json writers generally don't make guarantees about the ordering.

One alternative (though annoying) would to be use elephant bird
instead, but I can't get that to compile against hadoop 2.0.0 and Pig
0.10.0.

~Tim
+
meghana narasimhan 2013-01-07, 21:32
+
Tim Sell 2013-01-08, 01:03
+
Alan Gates 2013-01-07, 20:24
+
Tim Sell 2013-01-08, 01:02
+
Alan Gates 2013-01-08, 17:38
+
Dmitriy Ryaboy 2013-01-11, 03:35
+
Ruslan Al-Fakikh 2013-04-05, 00:51
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB