Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop >> mail # user >> Re: Sqoop error


Copy link to this message
-
Re: Sqoop error
Hi Srinivas,

In addition to Jarcec's excellent advice, perhaps you might find
http://archive.cloudera.com/cdh/3/sqoop/SqoopUserGuide.html#_mysql_connection_failurehelpful?

Regards,
Kathleen

On Sat, May 26, 2012 at 6:54 AM, Jarek Jarcec Cecho <[EMAIL PROTECTED]>wrote:

> Hi Srinivas,
> exception "com.mysql.jdbc.exceptions.jdbc4.CommunicationsException:
> Communications link failure" is very generic and basically means that there
> was something wrong with connecting to the MySQL. Could you share your
> entire execution log that was generated with --verbose flag?
>
> Blindly shooting - it's suspicious that your issue appears only when
> certain amount of jobs is generated. MySQL is allowing limiting maximal
> number of concurrent connections. I would recommend checking defined
> connection limit on MySQL side (by SQL query show variables like
> "max_connections";) and compare it to number of created map tasks. In case
> that configured maximal value for connections is lower, then you need to
> either reconfigure mysql or sqoop (for example limit number of used mappers
> with --num-mappers argument or do not import all tables at once).
>
> Jarcec
>
> On Sat, May 26, 2012 at 09:40:47AM -0400, Srinivas wrote:
> >
> > > I have couple of problems that I am running into:
> > >
> > >     1. I was able to initiate only 5 sqoop jobs parallel which is > number of datanode's.
> > >     2. If i try to invoke more jobs parallel then only 5 jobs are
> picked by and the rest of them throwing an error saying
> "com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications
> link failure"
> > >
> > >     Do you have any idea why this is happening.
> > >
> > > Also, I have made sure that the mysql allows parallel connections from
> datanode's.
> > >
> > >
> > >
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB