Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig, mail # dev - Injecting plans


Copy link to this message
-
Re: Injecting plans
Prashant Kommireddi 2013-02-12, 02:17
That would work in the case when it's a static path? The problem here is my
base path will vary entirely for each different Pig job, and I would not
like the client to set that. Rather my framework plugs-in the base path and
I am guessing that can happen only after parsing/buildLP?

On Mon, Feb 11, 2013 at 5:55 PM, Bill Graham <[EMAIL PROTECTED]> wrote:

> We've done this before by overriding  relativeToAbsolutePath
> and setLocation in the LoadFunc, or the correspoding methods in StoreFunc.
>
> On Mon, Feb 11, 2013 at 2:10 PM, Prashant Kommireddi <[EMAIL PROTECTED]
> >wrote:
>
> > Hey,
> >
> > I wanted to run an idea by you guys. I have a use-case where I try
> > injecting load/store paths into the script. So if a user says A = load
> > 'input'; I would like to add a base path to it and make it A = load
> > 'base_path/foo/bar/input'.
> >
> > I would like to achieve this programatically, and 1 way I can of doing it
> > is by allowing setter on LOLoad/LOStore that can modify the FileSpec. Is
> > there a cleaner or better way to approach this?
> >
> > In the future I would probably be meddling with more than just Load/Store
> > (for eg, disabling rmf or mkdir commands etc) but this is something I am
> > looking at currently.
> >
> > Thanks,
> > Prashant
> >
>
>
>
> --
> *Note that I'm no longer using my Yahoo! email address. Please email me at
> [EMAIL PROTECTED] going forward.*
>