Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> How to filter by pig datatype?


Copy link to this message
-
How to filter by pig datatype?
Attached is a tiny testcase illustrating my problem.

What I would like to know is how to filter by Pig datatype.
e.g. something like:
filtered = FILTER some_data BY some_variable IS_MAP_TYPE;

Can anyone advise if this can be accomplished with Pig?

We have a field that is sometimes a 'map' sometimes a chararray.

Doing something like the following statement fails, presumable because it's
trying to a key-value lookup on something that's not a 'map'.

-- json#'data' is sometimes a map, sometimes not.
trivias = FOREACH data GENERATE json#'data'#'trivia' AS trivia:charray;

This has come about from us working with JSON data with Pig via Elephant
Bird's JsonLoader.

Thanks,

Lex.
+
Ruslan Al-Fakikh 2012-11-22, 12:11
+
pablomar 2012-11-22, 17:48
+
Lex H 2012-11-22, 22:54
+
pablomar 2012-11-22, 23:19
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB