Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Project the last field of a tuple


Copy link to this message
-
Re: Project the last field of a tuple
here's a UDF to do it that took me about 10s to write, so may have errors:

import java.io.IOException;
import org.apache.pig.EvalFunc;
import org.apache.pig.data.Tuple;
import org.apache.pig.impl.logicalLayer.schema.Schema;

public class LastInTuple extends EvalFunc<Object> {
    public Object exec(Tuple input) throws IOException {
        int size = input.size();
        if (size > 0) {
            return input.get(size - 1);
        }
        return null;
    }

    public Schema outputSchema(Schema input) {
        try {
        int size = input.size();
            if (size > 0) {
                return new Schema(input.getField(size - 1));
            }
        } catch (Exception e) {}
        return null;
    }
}

2012/8/23 Ruslan Al-Fakikh <[EMAIL PROTECTED]>

> Hi Fabian,
>
> I don't know whether there is a built-in feature for this, but here is the
> idea:
> try to load the whole line as one field (ignoring the delimiter at
> this step) and then try to extract the last part using substring,
> regex, etc.
>
> Ruslan
>
> On Thu, Aug 23, 2012 at 12:53 PM, Fabian Alenius
> <[EMAIL PROTECTED]> wrote:
> > Hi,
> >
> > is there anyway to project the last field of a tuple (when you don't
> > know how many fields there are) without creating a UDF?
> >
> >
> > Thanks,
> >
> > Fabian
>
>
>
> --
> Best Regards,
> Ruslan Al-Fakikh
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB