Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # dev >> Handling of untyped execution in type branch?


Copy link to this message
-
RE: Handling of untyped execution in type branch?
Pi,

I pushed out a patch today to handle null schemas in LOUnion. I am
reviewing your patch which has the changes for merging schemas.

We should go with 2 where every operator supports null schemas.

Thanks,
Santhosh
 

-----Original Message-----
From: pi song [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, May 20, 2008 4:23 PM
To: [EMAIL PROTECTED]
Subject: Re: Handling of untyped execution in type branch?

Attention!!!: This is an important design decision. Within 24 hours if
nobody replies, I will assume option (1) to keep the work going.

Pi
On 5/20/08, pi song <[EMAIL PROTECTED]> wrote:
>
> How do we handle untyped executions?
>
> There are two ways:-
>
> 1. Stick dummy schema of ByteArrays in LOLoad.  This way is a bit ugly
but
> very simple to do.  Bad things: 1) This enforces maximum tuple arity
for
> untyped execution.  2) We also have to carry a number of schema fields
to
> downstream operators.
>
> 2. Implement all LOs to support null schema. This is more clean but
> requires a bit more work. I still see some LOs like LOUnion that will
die
> immediately in getSchema() if one of the inputs have a null schema.
>
> I am more tempted to do (2) just because I want it to be clean but we
could
> do (1) first if we want to get the whole thing done quickly.
> What about other people's opinion?
>
> Pi
>