Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Multiple STORE statements in one script


Copy link to this message
-
Re: Multiple STORE statements in one script
I am running it as a script through the exec command with the filename.
Using . -no_multiquery works though.

Regards,
Shahab
On Thu, May 9, 2013 at 9:07 PM, Russell Jurney <[EMAIL PROTECTED]>wrote:

> Are you running the script from the command line by filename, or
> pasting it into an interactive grunt session? If you're pasting, it
> only runs the first STORE statement. Just a guess.
>
> Russell Jurney http://datasyndrome.com
>
> On May 9, 2013, at 5:56 PM, Ruslan Al-Fakikh <[EMAIL PROTECTED]> wrote:
>
> > I was just thinking that using Sqoop is a more reliable and robust
> > solution. I guess it may work in cases where other tools don't. And
> > probably it can provide some more useful functionality.
> >
> >
> > On Wed, May 8, 2013 at 9:47 PM, Shahab Yunus <[EMAIL PROTECTED]>
> wrote:
> >
> >> @Peter thanks for the tip. -no_multiquery flag works but I guess it
> will be
> >> a performance hit. It is weird because it does not generate any error
> >> otherwise. In fact success message is explicitly logged even if nothing
> is
> >> saved.
> >>
> >> @Ruslan, yeah I was thinking of that too but then the thing is that if
> we
> >> indeed have to split the processing and first generate multiple HDFS
> files
> >> and then use SQOOP to load RDMS, then why not write few more short PIG
> >> scripts to load those HDFS files in RDMS?
> >>
> >> Regards,
> >> Shahab
> >>
> >>
> >> On Wed, May 8, 2013 at 12:27 PM, Ruslan Al-Fakikh <[EMAIL PROTECTED]
> >>> wrote:
> >>
> >>> Hi,
> >>>
> >>> It is possible to have multiple store statements, but I can't tell why
> >> you
> >>> have nothing in the result.
> >>> I recommend to split the task to the appropriate tools: store
> everything
> >> in
> >>> HDFS and then run Sqoop to upload data to an RDBMS.
> >>>
> >>> Ruslan
> >>>
> >>>
> >>> On Wed, May 8, 2013 at 6:11 PM, Shahab Yunus <[EMAIL PROTECTED]>
> >>> wrote:
> >>>
> >>>> Hello,
> >>>>
> >>>> In a Pig script I want to store the results in 2 different MySql
> tables
> >>>> (using DBStorage) and a file on HDFS. This means 3 different STORE
> >>>> statements. Right now when I do that, it does give success message in
> >> the
> >>>> logs but saves nothing. What am I missing? Is it even possible?
> >>>>
> >>>> I know I can use MultiStorage from PiggBank but I think it i sonly for
> >> 2
> >>>> HDFS files. I want to store in 2 different MySQL tables or a MySQL
> >> table
> >>>> and an HDFS file.
> >>>>
> >>>> Any recommendations? Thanks.
> >>>>
> >>>> Regards,
> >>>> Shahab
> >>
>