Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> problem filtering null values with pig


Copy link to this message
-
Re: problem filtering null values with pig
Hi,

I am not sure what's the problem because I can't reproduce it. To me, null
values are printed as an empty "( )" not "(null)", so it doesn't seem like
null.

I am wondering whether OpenJDK is the problem. Can you try Oracle HotSpot
JDK 1.6 and see that fixes it?

Thanks,
Cheolsoo

On Wed, Oct 31, 2012 at 1:06 PM, Arian Pasquali <[EMAIL PROTECTED]>wrote:

> hey people
> I'm having some troubles with a silly task, I can´t find a way to filter
> null values from my rows. This is the result when I dump the object
> geoinfo:
>
> DUMP geoinfo;
> ([longitude#70.95853,latitude#30.9773])
> ([longitude#-9.37944507,latitude#38.91780853])
> (null)
> (null)
> (null)
> ([longitude#-92.64416,latitude#16.73326])
> (null)
> (null)
> ([longitude#-9.15199849,latitude#38.71179122])
> ([longitude#-9.15210796,latitude#38.71195131])
>
> and here is the description
>
> DESCRIBE geoinfo;
> geoinfo: {geoLocation: bytearray}
>
> What I'm trying to do is to filter null values like this:
>
> geoinfo_no_nulls = FILTER geoinfo BY geoLocation is not null;
>
> but the result remains the same. nothing is filtered.
>
> I also tried something like this
>
> geoinfo_no_nulls = FILTER geoinfo BY geoLocation != 'null';
>
>  and I got an error
>
> org.apache.pig.backend.executionengine.ExecException: ERROR 1071: Cannot
> convert a map to a String
>
> What am I doing wrong here?
>
> env details,
>
> Ubuntu 12.04.1 LTS,
> hadoop-1.0.3
> pig 0.9.3 version 0.9.3-SNAPSHOT (rexported) compiled Oct 24 2012, 19:04:03
> java version "1.6.0_24" OpenJDK Runtime Environment (IcedTea6 1.11.4)
> (6b24-1.11.4-1ubuntu0.12.04.1)
> OpenJDK 64-Bit Server VM (build 20.0-b12, mixed mode)
>
>
> ArianP
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB