|
|
-
Dynamically generating load/store path
Miki Tebeka 2012-11-14, 00:36
Greetings,
Is there a way to dynamically generate (maybe via UDF) the path to load/store data? (something like "A = LOAD InputPath() USING PigStorage();")
Currently we calculate the load/store path outside the pig script and pass it as parameter. However we'd like to have a UDF that does that.
Thanks, -- Miki
-
Re: Dynamically generating load/store path
Jonathan Coveney 2012-11-14, 00:45
If it's a parameter, it could just be passed in as a $var 2012/11/13 Miki Tebeka <[EMAIL PROTECTED]>
> Greetings, > > Is there a way to dynamically generate (maybe via UDF) the path to > load/store data? (something like "A = LOAD InputPath() USING > PigStorage();") > > Currently we calculate the load/store path outside the pig script and pass > it as parameter. However we'd like to have a UDF that does that. > > Thanks, > -- > Miki >
-
Re: Dynamically generating load/store path
Prashant Kommireddi 2012-11-14, 00:55
You could write a custom load/storefunc and override setLocation(String location, Job job). The logic you want the UDF for could go in there. On Tue, Nov 13, 2012 at 4:45 PM, Jonathan Coveney <[EMAIL PROTECTED]>wrote:
> If it's a parameter, it could just be passed in as a $var > > > 2012/11/13 Miki Tebeka <[EMAIL PROTECTED]> > > > Greetings, > > > > Is there a way to dynamically generate (maybe via UDF) the path to > > load/store data? (something like "A = LOAD InputPath() USING > > PigStorage();") > > > > Currently we calculate the load/store path outside the pig script and > pass > > it as parameter. However we'd like to have a UDF that does that. > > > > Thanks, > > -- > > Miki > > >
|
|
All projects made searchable here are trademarks of the Apache Software Foundation.
Service operated by
Sematext