Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig, mail # user - pass configuration param to UDF


Copy link to this message
-
Re: pass configuration param to UDF
Dexin Wang 2010-11-29, 20:47
Thanks Daniel!

First option won't work for me since the property is not known to UDF
itself, I need to pass it to UDF.

Second option, I won't use pig.properties since the property passed to UDF
are per pig script specific, not a global setting. How do I pass a -D option
to pig script run (pig -f myscript.pig)?

Thanks.

On Tue, Nov 23, 2010 at 6:12 PM, Daniel Dai <[EMAIL PROTECTED]> wrote:

> The only hook in frontend for a UDF is outputSchema. You can put your
> property into UDFContext in outputSchema, and read back in exec.
>
>   public String exec(Tuple input) throws IOException {
>       UDFContext context = UDFContext.getUDFContext();
>       String a > context.getUDFProperties(this.getClass()).getProperty("Hello");
>       return a;
>   }
>
>   public Schema outputSchema(Schema input) {
>       UDFContext context = UDFContext.getUDFContext();
>       context.getUDFProperties(this.getClass()).setProperty("Hello",
> "World");
>       return null;
>   }
>
> The other option is to provide a system wide configuration in command line
> (-D), or pig.properties, which can be retrieved in UDF.exec using:
> UDFContext.getUDFContext().getJobConf().get("propertyname")
>
> Daniel
>
>
> Dexin Wang wrote:
>
>> Hi all,
>>
>> I was reading this:
>>
>> http://pig.apache.org/docs/r0.7.0/udf.html#Passing+Configurations+to+UDFs
>>
>> It sounded like I can pass some configuration or context to the UDF but I
>> can't figure out how I would do that after I searched quite a bit on
>> internet and past discussion.
>>
>> In my UDF, I can also do this:
>>
>>                UDFContext context = UDFContext.getUDFContext();
>>                Properties properties >> context.getUDFProperties(this.getClass());
>>
>> so if the context is set on the front end, supposedly, it will be in that
>> properties object. But how do I set it on the front end or whichever way
>> to
>> pass it to UDF?
>>
>> Thanks!
>> Dexin
>>
>>
>
>