Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop >> mail # user >> Sqoop connection to Informix Database


Copy link to this message
-
Re: Sqoop connection to Informix Database
Hi James,
please accept my apologies for this late reply. I was travelling in last few days and I had limited access to my email.

I've checked task log attached to the previous email and it seems that the failure is actually somewhere in Informix JDBC driver:

2013-01-17 11:17:15,537 ERROR org.apache.sqoop.mapreduce.db.DBRecordReader: Top level exception:
java.sql.SQLException: System or internal error java.io.IOException
  at com.informix.util.IfxErrMsg.getSQLException(IfxErrMsg.java:488)
  at com.informix.jdbc.IfxSqli.a(IfxSqli.java:9532)
  at com.informix.jdbc.IfxSqli.receiveMessage(IfxSqli.java:2632)
  at com.informix.jdbc.IfxSqli.j(IfxSqli.java:2280)
  at com.informix.jdbc.IfxSqli.getaRow(IfxSqli.java:4681)
  at com.informix.jdbc.IfxResultSet.next(IfxResultSet.java:526)
  at org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:238)
  at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:458)
  at org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:76)
  at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:85)
  at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:139)
  at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:182)
  at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:645)
  at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325)
  at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
  at java.security.AccessController.doPrivileged(Native Method)
  at javax.security.auth.Subject.doAs(Subject.java:396)
  at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332)
  at org.apache.hadoop.mapred.Child.main(Child.java:262)
Caused by: java.io.IOException
  at com.informix.asf.IfxDataInputStream.readFully(IfxDataInputStream.java:148)
  at com.informix.asf.IfxDataInputStream.readSmallInt(IfxDataInputStream.java:453)
  at com.informix.jdbc.IfxSqli.receiveMessage(IfxSqli.java:2597)
  ... 16 more

I'm not familiar with Informix, so I'm not sure what might be causing such issues. Are there any logs on database side that might help understand what went wrong?

Jarcec

On Thu, Jan 17, 2013 at 04:48:49PM +0000, James Hogarth wrote:
> On 17 January 2013 13:02, James Hogarth <[EMAIL PROTECTED]> wrote:
>
> >
> > Hi James,
> >> I'm expecting that this exception is being raised on mapper side, would
> >> you mind sharing entire map task log with us?
> >>
> >> Also, would you mind sharing your Hadoop and Sqoop version? Sqoop command
> >> line that you're using might be also helpful.
> >>
> >>
> > We're using CDH4.1 (sqoop-1.4.1+54-1.cdh4.1.2.p0.21.el6.noarch)
> >
> >  Attached is the log of the task attempt and anonymised copy/paste of the
> > console on the import attempt - you can see the command line on there.
> >
> > I'm using a single mapper to try and avoid segmentation/boundary issues I
> > was bumping into as well...
> >
> >
> >
>
> Hmm on further testing I'm not sure it is a serial8 issue actually and that
> was a red herring...
>
> When I restricted the dataset to something smaller the import went pretty
> cleanly...
>
> I'm trying to work out the boundary point where it starts getting issues at
> the moment...
>
> James
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB