Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig, mail # user - Dynamically generating load/store path


+
Miki Tebeka 2012-11-14, 00:36
+
Jonathan Coveney 2012-11-14, 00:45
Copy link to this message
-
Re: Dynamically generating load/store path
Prashant Kommireddi 2012-11-14, 00:55
You could write a custom load/storefunc  and override setLocation(String
location, Job job). The logic you want the UDF for could go in there.
On Tue, Nov 13, 2012 at 4:45 PM, Jonathan Coveney <[EMAIL PROTECTED]>wrote:

> If it's a parameter, it could just be passed in as a $var
>
>
> 2012/11/13 Miki Tebeka <[EMAIL PROTECTED]>
>
> > Greetings,
> >
> > Is there a way to dynamically generate (maybe via UDF) the path to
> > load/store data? (something like "A = LOAD InputPath() USING
> > PigStorage();")
> >
> > Currently we calculate the load/store path outside the pig script and
> pass
> > it as parameter. However we'd like to have a UDF that does that.
> >
> > Thanks,
> > --
> > Miki
> >
>