Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig, mail # user - Understanding LoadFunc sequence


Copy link to this message
-
Re: Understanding LoadFunc sequence
Stan Rosenberg 2012-02-04, 04:04
Hi Bill,

I've used the following in my UDFs:

public static boolean isBackend(JobContext ctx) {
// HACK borrowed from HCatLoader: this property should only be set
on the backend
return ctx.getConfiguration().get("mapred.task.id", "").length() > 0;
    }

I recall seeing other hacks to accomplish the same task; the above
seems to work for now.

stan

On Fri, Feb 3, 2012 at 2:21 PM, Bill Graham <[EMAIL PROTECTED]> wrote:
> Hi,
>
> Is there any way to tell in the constructor of a StoreFunc whether it's
> being invoked on the client or on the cluster?
>
> Background: on the client I read configs from a config file and set them on
> the Configuration object, which I then use on the server. Works great, but
> I want to throw an exception if the config isn't found on the client only.
>
> thanks,
> Bill