Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Sqoop >> mail # user >> Sqoop exporting error


+
Matthieu Labour 2012-11-02, 23:18
+
Jarek Jarcec Cecho 2012-11-03, 00:19
Copy link to this message
-
Re: Sqoop exporting error
Hi Jarcec

I apologize for not getting back sooner to you. I live in NYC and life has
been hectic here with the hurricane. Thank you for taking the time to look
in my issue. Please find below the information requested.
1. Here is the command being executed followed by the error produced.
 Attached is ml_ys_log_gmt_test.java

~/$SQOOP_ROOT/bin/sqoop export --connect jdbc:postgresql://
ec2-XX-XX-XXX-XX.compute-1.amazonaws.com:5662/dfg2tulf7263ut --username
ufjb0gfs1n5kut --password xxxxxxxxx  --table ml_ys_log_gmt_test
--export-dir $HADOOP_INPUT/$LOGS_HOME/dt=$(date +%Y-%m-%d)
--input-fields-terminated-by='\t' --lines-terminated-by='\n' --verbose
--batch

12/11/06 17:35:33 INFO mapred.JobClient: Task Id :
attempt_201211061658_0014_m_000000_0, Status : FAILED
java.io.IOException: java.sql.BatchUpdateException: Batch entry 71 INSERT
INTO ml_ys_log_gmt_test (date, ts, environment, resource, network,
advertiser, campaign, creative, cost, click, flavor, ui_element_id,
ui_element_type, event, charge_id, charge_type, charge_vertical,
charge_payer, charge_amount, model, imageformatsupport, inputmodesupport,
scriptsupport, vendor, stylesheetsupport, markupsupport, displaywidth,
cookiesupport, displayheight, version, inputdevices, displaycolordepth,
device_os, device_os_version, mobile_browser, mobile_browser_version,
is_tablet, dual_orientation, marketing_name) VALUES ('2012-10-31
20:42:56.767000 +00:00:00', '[B@ba6c13',
'remote-addr=10.84.101.45&user-agent=LG-CT810%2FV10x+NetFront%2Fv3.5+Profile%2FMIDP-2.0+Configuration%2FCLDC-1.1',
'hxCA', 'MLNL', '1006', '10014', '1410', NULL, 'mdkP', '10014', '', '',
'click', '', '', '', '', NULL, 'CT810', '', '', '', 'LG', '', '', '400',
'', '240', '', 'stylus', '', 'Windows Mobile OS', '6.1', 'Microsoft Mobile
Explorer', '7.11', 'false', 'false', '') was aborted.  Call
getNextException to see the cause.
        at
org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.write(AsyncSqlRecordWriter.java:220)
        at
org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.write(AsyncSqlRecordWriter.java:46)
        at
org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.write(MapTask.java:646)
        at
org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
        at
org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:78)
        at
org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:36)
        at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
        at
org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:182)
        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:771)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:375)
        at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:396)
        at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1132)
        at org.apache.hadoop.mapred.Child.main(Child.java:249)
2. Here is the command being executed followed by the error produced.
 Attached is ml_ys_log_gmt_test_with_map_column_java_ts_long.java

~/$SQOOP_ROOT/bin/sqoop export --connect jdbc:postgresql://
ec2-XX-XX-XXX-XX.compute-1.amazonaws.com:5662/dfg2tulf7263ut --username
ufjb0gfs1n5kut --password xxxxxxx --table ml_ys_log_gmt_test --export-dir
$HADOOP_INPUT/$LOGS_HOME/dt=$(date +%Y-%m-%d)
--input-fields-terminated-by='\t' --lines-terminated-by='\n'
--map-column-java ts=Long --verbose --batch
12/11/06 17:57:17 INFO mapred.JobClient: Task Id :
attempt_201211061658_0016_m_000000_0, Status : FAILED
java.io.IOException: java.sql.BatchUpdateException: Batch entry 71 INSERT
INTO ml_ys_log_gmt_test (date, ts, environment, resource, network,
advertiser, campaign, creative, cost, click, flavor, ui_element_id,
ui_element_type, event, charge_id, charge_type, charge_vertical,
charge_payer, charge_amount, model, imageformatsupport, inputmodesupport,
scriptsupport, vendor, stylesheetsupport, markupsupport, displaywidth,
cookiesupport, displayheight, version, inputdevices, displaycolordepth,
device_os, device_os_version, mobile_browser, mobile_browser_version,
is_tablet, dual_orientation, marketing_name) VALUES ('2012-10-31
20:42:56.767000 +00:00:00', '[B@1d709a5',
'remote-addr=10.84.101.45&user-agent=LG-CT810%2FV10x+NetFront%2Fv3.5+Profile%2FMIDP-2.0+Configuration%2FCLDC-1.1',
'hxCA', 'MLNL', '1006', '10014', '1410', NULL, 'mdkP', '10014', '', '',
'click', '', '', '', '', NULL, 'CT810', '', '', '', 'LG', '', '', '400',
'', '240', '', 'stylus', '', 'Windows Mobile OS', '6.1', 'Microsoft Mobile
Explorer', '7.11', 'false', 'false', '') was aborted.  Call
getNextException to see the cause.
        at
org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.write(AsyncSqlRecordWriter.java:220)
        at
org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.write(AsyncSqlRecordWriter.java:46)
        at
org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.write(MapTask.java:646)
        at
org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
        at
org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:78)
        at
org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:36)
        at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
        at
org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:182)
        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:771)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:375)
        at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:396)
        at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1132)
        at org.apache.hadoop.mapred.Child.main(Child.java:249)

3. I executed the
+
Jarek Jarcec Cecho 2012-11-06, 19:33
+
Matthieu Labour 2012-11-06, 20:30
+
Matthieu Labour 2012-11-07, 18:10
+
Jarek Jarcec Cecho 2012-11-08, 17:11
+
Matthieu Labour 2012-11-09, 18:26
+
Jarek Jarcec Cecho 2012-11-09, 23:53
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB