Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig, mail # user - Pig-cassandra Scritps and Oozie


Copy link to this message
-
Re: Pig-cassandra Scritps and Oozie
Miguel Angel Martin junqu... 2013-11-28, 15:31
hi Jeremy,

I do not try test it  still, I only test examples pig from oozie project
 without cassadra.

* pig-cassandra* sets the cassandra pig libraries .jar in the the
PIG_CLASSPATH env var. and after call the original shell script  *pig* from
PIG_HOME/bin/pig and , up to now, I launch pig scripts with pig_cassandra
directly.

I do not know and did not  see how oozie launch pig and I supose that Oozie
launch the PIG_HOME/bin/pig.

If you are using  this config and the pig scripts that use cassandra works
fine  , I suspose that the trick is  putting  the cassandra jars
dependencies and other udf or libraries that you use in the pig scripts  in
the oozie  sharelib or in the lib folder of the job.
On the other hand, I do not know if  i have to configure some thing  like
this.

http://wiki.apache.org/cassandra/HadoopSupport#Oozie

I am using Cassandra 1.2.10, Oozie 4.0.0 adn pig 0.11.1.

I try to test these options and see if it works-

Thanks in advance

2013/11/28 Jeremy Hanna <[EMAIL PROTECTED]>

> If I remember correctly when I configured pig, cassandra, and oozie to
> work together, I just used vanilla pig but gave it the jars it needed.
>
> What is the problem you’re experiencing that you are unable to do this?
>
> Jeremy
>
> On 28 Nov 2013, at 12:56, Miguel Angel Martin junquera <
> [EMAIL PROTECTED]> wrote:
>
> > hi all;
> >
> > What is the best way to integrate cassandra pig-extension with oozie?
> >
> > can be configure  oozie to use pig-cassandra instead of pig?
> >
> > Some ideas that I thinking are:
> >
> > Launching a Shell job    that runs ./pig-cassandra script.pig
> > or   changing environment variables  vakues
> > or the original to include the pig-cassandra code .... etc
> >
> > Thanks and regards
>
>