Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Project the last field of a tuple


Copy link to this message
-
Re: Project the last field of a tuple
here's a UDF to do it that took me about 10s to write, so may have errors:

import java.io.IOException;
import org.apache.pig.EvalFunc;
import org.apache.pig.data.Tuple;
import org.apache.pig.impl.logicalLayer.schema.Schema;

public class LastInTuple extends EvalFunc<Object> {
    public Object exec(Tuple input) throws IOException {
        int size = input.size();
        if (size > 0) {
            return input.get(size - 1);
        }
        return null;
    }

    public Schema outputSchema(Schema input) {
        try {
        int size = input.size();
            if (size > 0) {
                return new Schema(input.getField(size - 1));
            }
        } catch (Exception e) {}
        return null;
    }
}

2012/8/23 Ruslan Al-Fakikh <[EMAIL PROTECTED]>

> Hi Fabian,
>
> I don't know whether there is a built-in feature for this, but here is the
> idea:
> try to load the whole line as one field (ignoring the delimiter at
> this step) and then try to extract the last part using substring,
> regex, etc.
>
> Ruslan
>
> On Thu, Aug 23, 2012 at 12:53 PM, Fabian Alenius
> <[EMAIL PROTECTED]> wrote:
> > Hi,
> >
> > is there anyway to project the last field of a tuple (when you don't
> > know how many fields there are) without creating a UDF?
> >
> >
> > Thanks,
> >
> > Fabian
>
>
>
> --
> Best Regards,
> Ruslan Al-Fakikh
>