Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> how to stop dereferencing after join - error setting schema


Copy link to this message
-
Re: how to stop dereferencing after join - error setting schema
Each field needs to be dereferenced individually:

A::name AS name, A::age AS age...

On Saturday, March 2, 2013, Michael West wrote:

>
> I would like to set the schema after joining so that I do not have to
> always dereference.  However, I receive an error when I try this.  How can
> I resolve this error?
>
> pig version 0.11
>
> Error message:
>
> [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1031: Incompatable
> field schema: declared is
> "tuple_0:tuple(name:chararray,age:int,gpa:float)", infered is "A::gpa:float"
>
>
> Trivial sample code to demonstrate issue:
>
> /*
>   studenttab10k file from
>   http://people.apache.org/~hashutosh/
> */
>
> A = LOAD '/Users/mike/Documents/code/hadoop/pig/data/studenttab10k' AS
> (name:chararray, age:int, gpa:float);
> B = LOAD '/Users/mike/Documents/code/hadoop/pig/data/studenttab10k' AS
> (name:chararray, age:int, gpa:float);
> C = JOIN A by name, B by name;
> ILLUSTRATE C;
>
> D = FOREACH C GENERATE A::name, A::age, A::gpa AS (name:chararray,
> age:int, gpa:float);
> DESCRIBE D;
>
>
> ILLUSTRATE C output:
>
> ---------------------------------------------------------
> | A     | name:chararray    | age:int    | gpa:float    |
> ---------------------------------------------------------
> |       | xavier steinbeck  | 58         | 2.99         |
> |       | xavier steinbeck  | 23         | 0.59         |
> ---------------------------------------------------------
> ---------------------------------------------------------
> | B     | name:chararray    | age:int    | gpa:float    |
> ---------------------------------------------------------
> |       | xavier steinbeck  | 58         | 2.99         |
> |       | xavier steinbeck  | 23         | 0.59         |
> ---------------------------------------------------------
>
> ---------------------------------------------------------------------------------------------------------------------------
> | C     | A::name:chararray    | A::age:int    | A::gpa:float    |
> B::name:chararray    | B::age:int    | B::gpa:float    |
>
> ---------------------------------------------------------------------------------------------------------------------------
> |       | xavier steinbeck     | 58            | 2.99            | xavier
> steinbeck     | 58            | 2.99            |
> |       | xavier steinbeck     | 58            | 2.99            | xavier
> steinbeck     | 23            | 0.59            |
> |       | xavier steinbeck     | 23            | 0.59            | xavier
> steinbeck     | 58            | 2.99            |
> |       | xavier steinbeck     | 23            | 0.59            | xavier
> steinbeck     | 23            | 0.59            |
>
> ---------------------------------------------------------------------------------------------------------------------------

--
Sent from Gmail Mobile