Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Multiple STORE statements in one script


Copy link to this message
-
Re: Multiple STORE statements in one script
@Peter thanks for the tip. -no_multiquery flag works but I guess it will be
a performance hit. It is weird because it does not generate any error
otherwise. In fact success message is explicitly logged even if nothing is
saved.

@Ruslan, yeah I was thinking of that too but then the thing is that if we
indeed have to split the processing and first generate multiple HDFS files
and then use SQOOP to load RDMS, then why not write few more short PIG
scripts to load those HDFS files in RDMS?

Regards,
Shahab
On Wed, May 8, 2013 at 12:27 PM, Ruslan Al-Fakikh <[EMAIL PROTECTED]>wrote:

> Hi,
>
> It is possible to have multiple store statements, but I can't tell why you
> have nothing in the result.
> I recommend to split the task to the appropriate tools: store everything in
> HDFS and then run Sqoop to upload data to an RDBMS.
>
> Ruslan
>
>
> On Wed, May 8, 2013 at 6:11 PM, Shahab Yunus <[EMAIL PROTECTED]>
> wrote:
>
> > Hello,
> >
> > In a Pig script I want to store the results in 2 different MySql tables
> > (using DBStorage) and a file on HDFS. This means 3 different STORE
> > statements. Right now when I do that, it does give success message in the
> > logs but saves nothing. What am I missing? Is it even possible?
> >
> > I know I can use MultiStorage from PiggBank but I think it i sonly for 2
> > HDFS files. I want to store in 2 different MySQL tables or a MySQL table
> > and an HDFS file.
> >
> > Any recommendations? Thanks.
> >
> > Regards,
> > Shahab
> >
>