Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> Multiple STORE statements in one script


+
Shahab Yunus 2013-05-08, 14:11
+
Ruslan Al-Fakikh 2013-05-08, 16:27
+
Shahab Yunus 2013-05-08, 17:47
Copy link to this message
-
Re: Multiple STORE statements in one script
I was just thinking that using Sqoop is a more reliable and robust
solution. I guess it may work in cases where other tools don't. And
probably it can provide some more useful functionality.
On Wed, May 8, 2013 at 9:47 PM, Shahab Yunus <[EMAIL PROTECTED]> wrote:

> @Peter thanks for the tip. -no_multiquery flag works but I guess it will be
> a performance hit. It is weird because it does not generate any error
> otherwise. In fact success message is explicitly logged even if nothing is
> saved.
>
> @Ruslan, yeah I was thinking of that too but then the thing is that if we
> indeed have to split the processing and first generate multiple HDFS files
> and then use SQOOP to load RDMS, then why not write few more short PIG
> scripts to load those HDFS files in RDMS?
>
> Regards,
> Shahab
>
>
> On Wed, May 8, 2013 at 12:27 PM, Ruslan Al-Fakikh <[EMAIL PROTECTED]
> >wrote:
>
> > Hi,
> >
> > It is possible to have multiple store statements, but I can't tell why
> you
> > have nothing in the result.
> > I recommend to split the task to the appropriate tools: store everything
> in
> > HDFS and then run Sqoop to upload data to an RDBMS.
> >
> > Ruslan
> >
> >
> > On Wed, May 8, 2013 at 6:11 PM, Shahab Yunus <[EMAIL PROTECTED]>
> > wrote:
> >
> > > Hello,
> > >
> > > In a Pig script I want to store the results in 2 different MySql tables
> > > (using DBStorage) and a file on HDFS. This means 3 different STORE
> > > statements. Right now when I do that, it does give success message in
> the
> > > logs but saves nothing. What am I missing? Is it even possible?
> > >
> > > I know I can use MultiStorage from PiggBank but I think it i sonly for
> 2
> > > HDFS files. I want to store in 2 different MySQL tables or a MySQL
> table
> > > and an HDFS file.
> > >
> > > Any recommendations? Thanks.
> > >
> > > Regards,
> > > Shahab
> > >
> >
>
+
Russell Jurney 2013-05-10, 01:07
+
Shahab Yunus 2013-05-10, 01:21