Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Shared script commands


Copy link to this message
-
Re: Shared script commands
At Twitter, we have a lightweight framework that handles stitching code
together....so I think with pig, stitching stuff together in some organized
way is the current best practice.
2013/1/22 Cheolsoo Park <[EMAIL PROTECTED]>

> Hi Eric,
>
> You can move REGISTER and SET to a properties file and DECLARE and DEFAULT
> to a param file. Then, you can create an alias of pig like "pig -p <param
> file> -P <property file>".
>
> This is the best that I can think of. I am wondering if anyone has a better
> suggestion.
>
> Thanks,
> Cheolsoo
>
>
> On Tue, Jan 22, 2013 at 7:17 AM, Eric Czech <[EMAIL PROTECTED]> wrote:
>
> > Hi everyone,
> >
> > I'm trying to determine the best way for all of my scripts to have
> > shared initialization statements like jar register commands, default
> > variable declarations, etc. and I'm not sure what the best way to do
> > it is.
> >
> > Is it possible to create a script that does all of these boilerplate
> > things and then use the "exec" command from all of my other scripts to
> > call that "initialization" script?
> >
> > For example, here's a somewhat abstracted version of what all of my
> > scripts start with:
> >
> > myscript.pig -->
> >
> > ------------- Boilerplate declarations necessary in ALL scripts
> > -------------
> >
> > %DECLARE USERNAME `echo \$USER`
> > REGISTER /home/$USERNAME/apps/hadoop/build/share/myudfs.jar
> > %DEFAULT SCRIPT_MODE 'development'
> > %DEFAULT BUILD_ID '0'
> > SET pig.build.id '$BUILD_ID'
> >
> >
> >
> --------------------------------------------------------------------------------------
> >
> > // pig code to do useful things
> >
> >
> > */ end myscript.pig
> >
> >
> > Or is there a better way to use shared pig code like this (macros
> > don't allow a lot of the statements I need)?
> >
> > Thank you!
> >
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB