Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> pass configuration param to UDF


Copy link to this message
-
Re: pass configuration param to UDF
Thanks Daniel!

First option won't work for me since the property is not known to UDF
itself, I need to pass it to UDF.

Second option, I won't use pig.properties since the property passed to UDF
are per pig script specific, not a global setting. How do I pass a -D option
to pig script run (pig -f myscript.pig)?

Thanks.

On Tue, Nov 23, 2010 at 6:12 PM, Daniel Dai <[EMAIL PROTECTED]> wrote:

> The only hook in frontend for a UDF is outputSchema. You can put your
> property into UDFContext in outputSchema, and read back in exec.
>
>   public String exec(Tuple input) throws IOException {
>       UDFContext context = UDFContext.getUDFContext();
>       String a > context.getUDFProperties(this.getClass()).getProperty("Hello");
>       return a;
>   }
>
>   public Schema outputSchema(Schema input) {
>       UDFContext context = UDFContext.getUDFContext();
>       context.getUDFProperties(this.getClass()).setProperty("Hello",
> "World");
>       return null;
>   }
>
> The other option is to provide a system wide configuration in command line
> (-D), or pig.properties, which can be retrieved in UDF.exec using:
> UDFContext.getUDFContext().getJobConf().get("propertyname")
>
> Daniel
>
>
> Dexin Wang wrote:
>
>> Hi all,
>>
>> I was reading this:
>>
>> http://pig.apache.org/docs/r0.7.0/udf.html#Passing+Configurations+to+UDFs
>>
>> It sounded like I can pass some configuration or context to the UDF but I
>> can't figure out how I would do that after I searched quite a bit on
>> internet and past discussion.
>>
>> In my UDF, I can also do this:
>>
>>                UDFContext context = UDFContext.getUDFContext();
>>                Properties properties >> context.getUDFProperties(this.getClass());
>>
>> so if the context is set on the front end, supposedly, it will be in that
>> properties object. But how do I set it on the front end or whichever way
>> to
>> pass it to UDF?
>>
>> Thanks!
>> Dexin
>>
>>
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB