Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig, mail # user - imports and macro expansions


Copy link to this message
-
imports and macro expansions
Lauren Blau 2012-10-10, 10:14
maybe I'm misunderstanding the scope and ordering of the imports and macro
substitutions.
I have 3 actual scripts.
macros.pig has a single macro which takes 4 params and generates a load
instruction.
header.pig has a number of DEFINE statements for the UDFs used by our later
scripts and IMPORT 'macros.pig' and then
a number of calls to the macro DEFINE'd in the macros.pig script, using
parameters passed on the command line.
work.pig has an IMPORT 'header.pig' followed by all the commands needed to
generate and output the wanted data.

If i dryrun header.pig and import header.pig.expanded in work.pig
everything works fine. header.pig.expanded has no macro definition in it,
just the load instructions resulting from each of the calls to the macro.

If I try to run work.pig with the import of header.pig, then I get the an
error 'unexpected symbol at or near '$' being reported in header.pig

are the nested imports the problem?

On Tue, Oct 9, 2012 at 11:12 PM, Aniket Mokashi <[EMAIL PROTECTED]> wrote:

> The fix is-
> grunt> run -param key=value script.pig
>
> ~Aniket
>
> On Mon, Oct 8, 2012 at 5:56 PM, Russell Jurney <[EMAIL PROTECTED]
> >wrote:
>
> > I did not know about the run command from inside grunt, but generally
> > speaking grunt does not yet support macros or parameters. I am eager
> > to get this fixed, myself.
> >
> > Russell Jurney http://datasyndrome.com
> >
> > On Oct 8, 2012, at 5:53 PM, Lauren Blau
> > <[EMAIL PROTECTED]> wrote:
> >
> > > I have a script something like
> > >
> > > DEFINE udf ..
> > > DEFINE udf2 ..
> > >
> > > IMPORT 'macros.pig'
> > >
> > > rel = calltomacro('string',$keyparam);
> > > rel2 = calltomacro('string2',$keyparam);
> > > ....
> > >
> > >
> > > if I run this with pig -p keyparam=testparam --dryrun script.pig
> > > I get a valid script.pig.expanded created.
> > >
> > > but if I run pig -p keyparam=testparam
> > > grunt> run script.pig (or IMPORT 'pig.script';)
> > >
> > > it fails with an error 'undefined param $keyparam'
> > > why does this behave differently during --dryrun and inside grunt?
> > >
> > > Thanks,
> > > lauren
> >
>
>
>
> --
> "...:::Aniket:::... Quetzalco@tl"
>