Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # dev >> Handling of untyped execution in type branch?


Copy link to this message
-
RE: Handling of untyped execution in type branch?
Pi,

I pushed out a patch today to handle null schemas in LOUnion. I am
reviewing your patch which has the changes for merging schemas.

We should go with 2 where every operator supports null schemas.

Thanks,
Santhosh
 

-----Original Message-----
From: pi song [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, May 20, 2008 4:23 PM
To: [EMAIL PROTECTED]
Subject: Re: Handling of untyped execution in type branch?

Attention!!!: This is an important design decision. Within 24 hours if
nobody replies, I will assume option (1) to keep the work going.

Pi
On 5/20/08, pi song <[EMAIL PROTECTED]> wrote:
>
> How do we handle untyped executions?
>
> There are two ways:-
>
> 1. Stick dummy schema of ByteArrays in LOLoad.  This way is a bit ugly
but
> very simple to do.  Bad things: 1) This enforces maximum tuple arity
for
> untyped execution.  2) We also have to carry a number of schema fields
to
> downstream operators.
>
> 2. Implement all LOs to support null schema. This is more clean but
> requires a bit more work. I still see some LOs like LOUnion that will
die
> immediately in getSchema() if one of the inputs have a null schema.
>
> I am more tempted to do (2) just because I want it to be clean but we
could
> do (1) first if we want to get the whole thing done quickly.
> What about other people's opinion?
>
> Pi
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB