Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig, mail # dev - Re: Project the last field of a tuple


Copy link to this message
-
Re: Project the last field of a tuple
Russell Jurney 2012-08-23, 21:59
This is neat. This should maybe be a built in: Tuple.first and
Tuple.last. Very convenient in Ruby.

Russell Jurney http://datasyndrome.com

On Aug 23, 2012, at 1:53 PM, Jonathan Coveney <[EMAIL PROTECTED]> wrote:

> here's a UDF to do it that took me about 10s to write, so may have errors:
>
> import java.io.IOException;
> import org.apache.pig.EvalFunc;
> import org.apache.pig.data.Tuple;
> import org.apache.pig.impl.logicalLayer.schema.Schema;
>
> public class LastInTuple extends EvalFunc<Object> {
>    public Object exec(Tuple input) throws IOException {
>        int size = input.size();
>        if (size > 0) {
>            return input.get(size - 1);
>        }
>        return null;
>    }
>
>    public Schema outputSchema(Schema input) {
>        try {
>        int size = input.size();
>            if (size > 0) {
>                return new Schema(input.getField(size - 1));
>            }
>        } catch (Exception e) {}
>        return null;
>    }
> }
>
> 2012/8/23 Ruslan Al-Fakikh <[EMAIL PROTECTED]>
>
>> Hi Fabian,
>>
>> I don't know whether there is a built-in feature for this, but here is the
>> idea:
>> try to load the whole line as one field (ignoring the delimiter at
>> this step) and then try to extract the last part using substring,
>> regex, etc.
>>
>> Ruslan
>>
>> On Thu, Aug 23, 2012 at 12:53 PM, Fabian Alenius
>> <[EMAIL PROTECTED]> wrote:
>>> Hi,
>>>
>>> is there anyway to project the last field of a tuple (when you don't
>>> know how many fields there are) without creating a UDF?
>>>
>>>
>>> Thanks,
>>>
>>> Fabian
>>
>>
>>
>> --
>> Best Regards,
>> Ruslan Al-Fakikh
>>