Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> how to stop dereferencing after join - error setting schema


+
Michael West 2013-03-02, 16:11
Copy link to this message
-
Re: how to stop dereferencing after join - error setting schema
Each field needs to be dereferenced individually:

A::name AS name, A::age AS age...

On Saturday, March 2, 2013, Michael West wrote:

>
> I would like to set the schema after joining so that I do not have to
> always dereference.  However, I receive an error when I try this.  How can
> I resolve this error?
>
> pig version 0.11
>
> Error message:
>
> [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1031: Incompatable
> field schema: declared is
> "tuple_0:tuple(name:chararray,age:int,gpa:float)", infered is "A::gpa:float"
>
>
> Trivial sample code to demonstrate issue:
>
> /*
>   studenttab10k file from
>   http://people.apache.org/~hashutosh/
> */
>
> A = LOAD '/Users/mike/Documents/code/hadoop/pig/data/studenttab10k' AS
> (name:chararray, age:int, gpa:float);
> B = LOAD '/Users/mike/Documents/code/hadoop/pig/data/studenttab10k' AS
> (name:chararray, age:int, gpa:float);
> C = JOIN A by name, B by name;
> ILLUSTRATE C;
>
> D = FOREACH C GENERATE A::name, A::age, A::gpa AS (name:chararray,
> age:int, gpa:float);
> DESCRIBE D;
>
>
> ILLUSTRATE C output:
>
> ---------------------------------------------------------
> | A     | name:chararray    | age:int    | gpa:float    |
> ---------------------------------------------------------
> |       | xavier steinbeck  | 58         | 2.99         |
> |       | xavier steinbeck  | 23         | 0.59         |
> ---------------------------------------------------------
> ---------------------------------------------------------
> | B     | name:chararray    | age:int    | gpa:float    |
> ---------------------------------------------------------
> |       | xavier steinbeck  | 58         | 2.99         |
> |       | xavier steinbeck  | 23         | 0.59         |
> ---------------------------------------------------------
>
> ---------------------------------------------------------------------------------------------------------------------------
> | C     | A::name:chararray    | A::age:int    | A::gpa:float    |
> B::name:chararray    | B::age:int    | B::gpa:float    |
>
> ---------------------------------------------------------------------------------------------------------------------------
> |       | xavier steinbeck     | 58            | 2.99            | xavier
> steinbeck     | 58            | 2.99            |
> |       | xavier steinbeck     | 58            | 2.99            | xavier
> steinbeck     | 23            | 0.59            |
> |       | xavier steinbeck     | 23            | 0.59            | xavier
> steinbeck     | 58            | 2.99            |
> |       | xavier steinbeck     | 23            | 0.59            | xavier
> steinbeck     | 23            | 0.59            |
>
> ---------------------------------------------------------------------------------------------------------------------------

--
Sent from Gmail Mobile
+
Michael West 2013-03-02, 18:10
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB