Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Sqoop >> mail # user >> Sqoop exporting error


+
Matthieu Labour 2012-11-02, 23:18
+
Jarek Jarcec Cecho 2012-11-03, 00:19
+
Matthieu Labour 2012-11-06, 18:17
+
Jarek Jarcec Cecho 2012-11-06, 19:33
+
Matthieu Labour 2012-11-06, 20:30
Copy link to this message
-
Re: Sqoop exporting error
Hi Jarcec

I did the following

I changed the type of 'ts' column to decimal. ALTER TABLE
ml_ys_log_gmt_test ALTER ts TYPE decimal;

                                     Table "public.ml_ys_log_gmt_test"
         Column         |           Type           |            Modifiers
         | Storage  | Description
------------------------+--------------------------+---------------------------------+----------+-------------
 ts                     | numeric                  |
          | main     |
Then I run the following command

~/$SQOOP_ROOT/bin/sqoop export --connect jdbc:postgresql://
ec2-XX-XX-XXX-XX.compute-1.amazonaws.com:5662/dfg2tulf7263ut --username
ufjb0gfs1n5kut --password xxxxxx --table ml_ys_log_gmt_test --export-dir
$HADOOP_INPUT/$LOGS_HOME/dt=$(date +%Y-%m-%d)
--input-fields-terminated-by='\t' --lines-terminated-by='\n' --verbose
--batch
12/11/07 17:25:32 INFO mapred.JobClient: Task Id :
attempt_201211071722_0002_m_000000_0, Status : FAILED
java.io.IOException: java.sql.BatchUpdateException: Batch entry 71 INSERT
INTO ml_ys_log_gmt_test (date, ts, environment, resource, network,
advertiser, campaign, creative, cost, click, flavor, ui_element_id,
ui_element_type, event, charge_id, charge_type, charge_vertical,
charge_payer, charge_amount, model, imageformatsupport, inputmodesupport,
scriptsupport, vendor, stylesheetsupport, markupsupport, displaywidth,
cookiesupport, displayheight, version, inputdevices, displaycolordepth,
device_os, device_os_version, mobile_browser, mobile_browser_version,
is_tablet, dual_orientation, marketing_name) VALUES ('2012-10-31
20:42:56.767000 +00:00:00', '1351716176767',
'remote-addr=10.84.101.45&user-agent=LG-CT810%2FV10x+NetFront%2Fv3.5+Profile%2FMIDP-2.0+Configuration%2FCLDC-1.1',
'hxCA', 'MLNL', '1006', '10014', '1410', NULL, 'mdkP', '10014', '', '',
'click', '', '', '', '', NULL, 'CT810', '', '', '', 'LG', '', '', '400',
'', '240', '', 'stylus', '', 'Windows Mobile OS', '6.1', 'Microsoft Mobile
Explorer', '7.11', 'false', 'false', '') was aborted.  Call
getNextException to see the cause.
        at
org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.write(AsyncSqlRecordWriter.java:220)
        at
org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.write(AsyncSqlRecordWriter.java:46)
        at
org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.write(MapTask.java:646)
        at
org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
        at
org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:78)
        at
org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:36)
        at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
        at
org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:182)
        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:771)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:375)
        at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:396)
        at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1132)
        at org.apache.hadoop.mapred.Child.main(Child.java:249)

I have attached ml_ys_log_gmt_test_with_map_column_java_ts_numeric.java

Please note that ts = '1351716176767' in the INSERT clause. The value seems
right. But since the definition of the column is numeric, the quotes (')
surrounding 1351716176767 should not be there.

Thank you for your help!

Matthieu
On Tue, Nov 6, 2012 at 2:33 PM, Jarek Jarcec Cecho <[EMAIL PROTECTED]>wrote:

> Hi Matthieu,
> thank you very much for your outputs. I've took a look and I'm really not
> sure what is wrong. It seems that from strange reason sqoop submits for ts
> column value '[B@ba6c13' that seems to me an object address. But code
> that you've sent is correctly serializing long, so I'm really not sure what
> might be an issue here.
Matthieu Labour, Engineering | *Action**X* |
584 Broadway, Suite 1002 – NY, NY 10012
415-994-3480 (m)
+
Jarek Jarcec Cecho 2012-11-08, 17:11
+
Matthieu Labour 2012-11-09, 18:26
+
Jarek Jarcec Cecho 2012-11-09, 23:53