Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig, mail # user - Shared script commands


+
Eric Czech 2013-01-22, 15:17
+
Cheolsoo Park 2013-01-22, 18:40
Copy link to this message
-
Re: Shared script commands
Jonathan Coveney 2013-01-22, 22:25
At Twitter, we have a lightweight framework that handles stitching code
together....so I think with pig, stitching stuff together in some organized
way is the current best practice.
2013/1/22 Cheolsoo Park <[EMAIL PROTECTED]>

> Hi Eric,
>
> You can move REGISTER and SET to a properties file and DECLARE and DEFAULT
> to a param file. Then, you can create an alias of pig like "pig -p <param
> file> -P <property file>".
>
> This is the best that I can think of. I am wondering if anyone has a better
> suggestion.
>
> Thanks,
> Cheolsoo
>
>
> On Tue, Jan 22, 2013 at 7:17 AM, Eric Czech <[EMAIL PROTECTED]> wrote:
>
> > Hi everyone,
> >
> > I'm trying to determine the best way for all of my scripts to have
> > shared initialization statements like jar register commands, default
> > variable declarations, etc. and I'm not sure what the best way to do
> > it is.
> >
> > Is it possible to create a script that does all of these boilerplate
> > things and then use the "exec" command from all of my other scripts to
> > call that "initialization" script?
> >
> > For example, here's a somewhat abstracted version of what all of my
> > scripts start with:
> >
> > myscript.pig -->
> >
> > ------------- Boilerplate declarations necessary in ALL scripts
> > -------------
> >
> > %DECLARE USERNAME `echo \$USER`
> > REGISTER /home/$USERNAME/apps/hadoop/build/share/myudfs.jar
> > %DEFAULT SCRIPT_MODE 'development'
> > %DEFAULT BUILD_ID '0'
> > SET pig.build.id '$BUILD_ID'
> >
> >
> >
> --------------------------------------------------------------------------------------
> >
> > // pig code to do useful things
> >
> >
> > */ end myscript.pig
> >
> >
> > Or is there a better way to use shared pig code like this (macros
> > don't allow a lot of the statements I need)?
> >
> > Thank you!
> >
>