Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Sqoop, mail # user - Intermittent problems with sqoop using Oracle JDBC driver


+
Andre Araujo 2013-07-12, 05:57
+
Andre Araujo 2013-07-12, 15:05
Copy link to this message
-
Re: Intermittent problems with sqoop using Oracle JDBC driver
Jarek Jarcec Cecho 2013-07-12, 15:35
Hi Andre,
thank you for sharing the solution with us! The link do not seem to be valid, would you mind checking if it's correct?

Jarcec

On Sat, Jul 13, 2013 at 01:05:19AM +1000, Andre Araujo wrote:
> Hi, all,
>
> I've found the answer for my own question.
> The solution is explained here:
> http://www.pythian.com/blog/?p=56405&preview=true
>
> Regards,
> Andre
>
>
>
> On 12 July 2013 15:57, Andre Araujo <[EMAIL PROTECTED]> wrote:
>
> > Hi,
> >
> > While using sqoop to import data from an Oracle database I was noticing
> > intermittent problems with sqoop hanging. After some troubleshooting I
> > narrowed the problem down to the issue described in the following pages:
> >
> >
> > http://stackoverflow.com/questions/2327220/oracle-jdbc-intermittent-connect> > ion-issue
> > https://forums.oracle.com/message/3701989
> >
> > The problem seems to be directly related to the Oracle JDBC driver. As per
> > the articles above, I can tell the driver to use /dev/urandom by using the
> > option -Djava.security.egd. I tested it with a standalone Java program
> > connecting to the database using the JDBC driver and it works well.
> >
> > sqoop, though, uses JDBC in two ways. It connects directly to the database
> > thru JDBC to gather details about the table/query and generate the mapred
> > job. It also uses the driver indirectly in the mapred tasks.
> >
> > We can use the -D mapred.child.java.opts=3D"-Djava.security.egd=3D..."
> > opti> > on
> > of sqoop to tell the mapred tasks to use the option. However, I haven't
> > found any way to tell sqoop to use it while connecting directly to the
> > database. Because of this, it hangs soon after I start it and even before
> > it kicks off the mapred job.
> >
> > I also tried to pass the java.security.egd option directly to sqoop with
> > the following command line but it didn't work:
> >
> > -D java.security.egd=3D/dev/urandom -D mapred.child.java.opts=3D"-D
> > java.security.egd=3D/dev/urandom"
> >
> > I added the option to the java.security file, but for some reason it also
> > didn't sort effect for sqoop.
> >
> > Any idea on a workaround/solution for this?
> >
> > Thanks,
> > Andre
> >
> > --
> > André Araújo
> > Database Administrator / SDM
> > The Pythian Group - Australia - www.pythian.com
> >
> > Office (calls from within Australia): 1300 366 021 x1270
> > Office (international): +61 2 9191 7427  x270 *OR* +1 613 565 8696   x1270
> > Mobile: +61 410 323 559
> > Fax: +61 2 9805 0544
> > IM: pythianaraujo @ AIM/MSN/Y! or [EMAIL PROTECTED] @ GTalk
> >
> >
>
>
> --
> André Araújo
> Database Administrator / SDM
> The Pythian Group - Australia - www.pythian.com
>
> Office (calls from within Australia): 1300 366 021 x1270
> Office (international): +61 2 9191 7427  x270 *OR* +1 613 565 8696   x1270
> Mobile: +61 410 323 559
> Fax: +61 2 9805 0544
> IM: pythianaraujo @ AIM/MSN/Y! or [EMAIL PROTECTED] @ GTalk
>
> --
>
>
> --
>
>
>
+
Andre Araujo 2013-07-15, 00:29
+
Andre Araujo 2013-07-15, 03:12
+
Jarek Jarcec Cecho 2013-07-15, 16:55
+
Andre Araujo 2013-07-15, 20:47
+
Jarek Jarcec Cecho 2013-07-15, 23:41
+
David Robson 2013-07-15, 03:31
+
Andre Araujo 2013-07-15, 12:34