Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop >> mail # user >> Re: Sqoop error


Copy link to this message
-
Re: Sqoop error
Hi Srinivas,

In addition to Jarcec's excellent advice, perhaps you might find
http://archive.cloudera.com/cdh/3/sqoop/SqoopUserGuide.html#_mysql_connection_failurehelpful?

Regards,
Kathleen

On Sat, May 26, 2012 at 6:54 AM, Jarek Jarcec Cecho <[EMAIL PROTECTED]>wrote:

> Hi Srinivas,
> exception "com.mysql.jdbc.exceptions.jdbc4.CommunicationsException:
> Communications link failure" is very generic and basically means that there
> was something wrong with connecting to the MySQL. Could you share your
> entire execution log that was generated with --verbose flag?
>
> Blindly shooting - it's suspicious that your issue appears only when
> certain amount of jobs is generated. MySQL is allowing limiting maximal
> number of concurrent connections. I would recommend checking defined
> connection limit on MySQL side (by SQL query show variables like
> "max_connections";) and compare it to number of created map tasks. In case
> that configured maximal value for connections is lower, then you need to
> either reconfigure mysql or sqoop (for example limit number of used mappers
> with --num-mappers argument or do not import all tables at once).
>
> Jarcec
>
> On Sat, May 26, 2012 at 09:40:47AM -0400, Srinivas wrote:
> >
> > > I have couple of problems that I am running into:
> > >
> > >     1. I was able to initiate only 5 sqoop jobs parallel which is > number of datanode's.
> > >     2. If i try to invoke more jobs parallel then only 5 jobs are
> picked by and the rest of them throwing an error saying
> "com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications
> link failure"
> > >
> > >     Do you have any idea why this is happening.
> > >
> > > Also, I have made sure that the mysql allows parallel connections from
> datanode's.
> > >
> > >
> > >
>