Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Sqoop >> mail # user >> Sqoop exporting error


+
Matthieu Labour 2012-11-02, 23:18
Copy link to this message
-
Re: Sqoop exporting error
Hi Matthieu,
would you mind sharing with us writable class that Sqoop generates for your table? I'm expecting that this file will be named "ml_ys_log_gmt.java" and it will be available in working directory where you're executing sqoop.

You might try utilizing parameter type mapping argument --map-column-java ts=Long or ts=String to see if it helps.

Jarcec

On Fri, Nov 02, 2012 at 07:18:44PM -0400, Matthieu Labour wrote:
> Hi
>
> Sometimes sqoop craps out. I am looking for any tips or help. Thank you
> very much !
>
> The following command runs
>
> ~/$SQOOP_ROOT/bin/sqoop export --connect jdbc:postgresql://
> ec2-XX-XX-XXX-XX.compute-1.amazonaws.com:5662/xxxxxxxxx --username xxxxxxxx
> --password xxxxxxxx --table ml_ys_log_gmt --export-dir
> $HADOOP_INPUT/$LOGS_HOME/dt=$(date +%Y-%m-%d)
> --input-fields-terminated-by='\t' --lines-terminated-by='\n' --verbose
> --batch
>
> on the following line
>
> 2012-10-27 23:44:25.699 1351381465699
> remote-addr=10.119.81.125&user-agent=SAMSUNG-SGH-I637%2FUCIJ2+Profile%2FMIDP-2.1+Configuration%2FCLDC-1.1+Mozilla%2F4.0+%28compatible%3B+MSIE+6.0%3B+Windows+CE%3B+IEMobile+8.12%3B+MSIEMobile+6.0%29
>   hxCA    MLNL    1006    10014   1410            kUQN    10014
>       click                                           SGH-i637
>                    Samsung                 320
>
> I get the error below.
>
> The insert craps out on the second column 'ts' in INSERT INTO ml_ys_log_gmt
> (date, ts
>
> The value passed is '[B@6127da' when it should be 1351381465699 (second
> field in the tab delimited line above)
>
> Here is the description in the table
> ts                     | bigint                   |
>         | plain    |
>
>
> 12/10/28 06:57:20 INFO mapred.JobClient: Task Id :
> attempt_201210280647_0004_m_000000_0, Status : FAILED
> java.io.IOException: java.sql.BatchUpdateException: Batch entry 59 INSERT
> INTO ml_ys_log_gmt (date, ts, environment, resource, network, advertiser,
> campaign, creative, cost, click, flavor, ui_element_id, ui_element_type,
> event, charge_id, charge_type, charge_vertical, charge_payer,
> charge_amount, model, imageformatsupport, inputmodesupport, scriptsupport,
> vendor, stylesheetsupport, markupsupport, displaywidth, cookiesupport,
> displayheight, version, inputdevices, displaycolordepth, device_os,
> device_os_version, mobile_browser, mobile_browser_version, is_tablet,
> dual_orientation, marketing_name) VALUES ('2012-10-27 23:44:25.699000
> +00:00:00', '[B@6127da',
> 'remote-addr=10.119.81.125&user-agent=SAMSUNG-SGH-I637%2FUCIJ2+Profile%2FMIDP-2.1+Configuration%2FCLDC-1.1+Mozilla%2F4.0+%28compatible%3B+MSIE+6.0%3B+Windows+CE%3B+IEMobile+8.12%3B+MSIEMobile+6.0%29',
> 'hxCA', 'MLNL', '1006', '10014', '1410', NULL, 'kUQN', '10014', '', '',
> 'click', '', '', '', '', NULL, 'SGH-i637', '', '', '', 'Samsung', '', '',
> '320', '', '240', '', '', '', 'Windows Mobile OS', '5.0', 'Microsoft Mobile
> Explorer', '8.12', 'false', 'false', 'BlackJack') was aborted.  Call
> getNextException to see the cause.
> at
> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.write(AsyncSqlRecordWriter.java:220)
> at
> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.write(AsyncSqlRecordWriter.java:46)
> at
> org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.write(MapTask.java:646)
> at
> org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
> at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:78)
> at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:36)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> at
> org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:182)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:771)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:375)
> at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
+
Matthieu Labour 2012-11-06, 18:17
+
Jarek Jarcec Cecho 2012-11-06, 19:33
+
Matthieu Labour 2012-11-06, 20:30
+
Matthieu Labour 2012-11-07, 18:10
+
Jarek Jarcec Cecho 2012-11-08, 17:11
+
Matthieu Labour 2012-11-09, 18:26
+
Jarek Jarcec Cecho 2012-11-09, 23:53
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB