Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> How to filter by pig datatype?


Copy link to this message
-
How to filter by pig datatype?
Attached is a tiny testcase illustrating my problem.

What I would like to know is how to filter by Pig datatype.
e.g. something like:
filtered = FILTER some_data BY some_variable IS_MAP_TYPE;

Can anyone advise if this can be accomplished with Pig?

We have a field that is sometimes a 'map' sometimes a chararray.

Doing something like the following statement fails, presumable because it's
trying to a key-value lookup on something that's not a 'map'.

-- json#'data' is sometimes a map, sometimes not.
trivias = FOREACH data GENERATE json#'data'#'trivia' AS trivia:charray;

This has come about from us working with JSON data with Pig via Elephant
Bird's JsonLoader.

Thanks,

Lex.
+
Ruslan Al-Fakikh 2012-11-22, 12:11
+
pablomar 2012-11-22, 17:48
+
Lex H 2012-11-22, 22:54
+
pablomar 2012-11-22, 23:19