Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> Question about properties for Loader


+
Jeff Yuan 2013-02-24, 23:33
Copy link to this message
-
Re: Question about properties for Loader
Hi Jeff,

How do you see your loader being used? Would users specify schema file or
would that be something your loader sets without user being aware of it?
Can you pass it in as a constructor argument instead?

UDFContext could be used, like you said to set/retrieve properties. You
might want to take a look at PigStorage that does something very similar
(look for the method applySchema(Tuple tup) )

On Sun, Feb 24, 2013 at 3:33 PM, Jeff Yuan <[EMAIL PROTECTED]> wrote:

> I'm trying to write a loader, extending LoadFunc, to read a specific
> file format.
>
> My question, how do I pass properties to it (for example the schema of
> the file type I'm loading)?  Would it be using the -p parameter from
> the cmdline when issuing the query?
>
> The second part of the question is, how would I access the passed in
> property/configuration from the code?  So far I'm theorizing it's
> something like this:
>         Properties p = udfc.getUDFProperties(this.getClass(), new
> String[]{ contextSignature });
>         Configuration conf = udfc.getJobConf();
> Then get it from p or conf?
>
> Thanks a lot for any pointers.
>
> -Jeff
>
+
Jeff Yuan 2013-02-25, 01:07
+
Dmitriy Ryaboy 2013-02-25, 03:24