Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # dev >> POStatus.STATUS_NULL Semantics


Copy link to this message
-
Re: POStatus.STATUS_NULL Semantics
We shall definitely clear the ambiguity. +1 for #1.

For #2, I did some digging into POPackage. What I see is in POPackage we
produce STATUS_NULL, but not POJoinPackage. And the return value for
POPackage is not used:
if (pack instanceof POJoinPackage) {
    while (true)
    {
        ....
        if (processOnePackageOutput(context))
            break;
    }
}
else {
     ....
     processOnePackageOutput(context);
}

So it seems STATUS_NULL is not effectively in use in
POPackage/POJoinPackage. We shall always use STATUS_EOP to indicate the end
of pull (for both POPackage and pipeline). We shall not use STATUS_NULL
anywhere once we make #1 clear. Am I right?

Thanks,
Daniel
On Tue, Nov 12, 2013 at 8:35 AM, Cheolsoo Park <[EMAIL PROTECTED]> wrote:

> >> I'll post a patch that makes the second option official and changes all
> the examples of the first to return null with STATUS_OK (this is what
> happens with the constant null currently).
>
> I am +1 for this.
>
>
> On Fri, Nov 8, 2013 at 10:16 PM, Mark Wagner <[EMAIL PROTECTED]
> >wrote:
>
> > Hi Devs,
> >
> > While discussing the Pig on Tez project today, we were unable to reach
> > a conclusion on what the semantics of what POStatus.STATUS_NULL are
> > and would like to ask those who remember the history of Pig to chime
> > in. The two interpretations are:
> >
> > 1. POStatus.STATUS_NULL indicates that the pulled output IS null. This
> > is used in some places like EqualToExpr.
> >
> > 2. POStatus.STATUS_NULL indicates that the pull did not produce any
> > output. This is backed up by it's usage in POPackage for flattening an
> > empty bag, and PigGenericMapBase where pulls on the operator pipeline
> > that result in STATUS_NULL are discarded.
> >
> > If nobody has any concrete documentation or explanation for the
> > discrepancy, I'll post a patch that makes the second option official
> > and changes all the examples of the first to return null with
> > STATUS_OK (this is what happens with the constant null currently).
> >
> > Thanks,
> > Mark
> >
>

--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.