Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop >> mail # user >> Sqoop exporting error

Copy link to this message
Re: Sqoop exporting error
Hi Jarcec

I did the following

I changed the type of 'ts' column to decimal. ALTER TABLE
ml_ys_log_gmt_test ALTER ts TYPE decimal;

                                     Table "public.ml_ys_log_gmt_test"
         Column         |           Type           |            Modifiers
         | Storage  | Description
 ts                     | numeric                  |
          | main     |
Then I run the following command

~/$SQOOP_ROOT/bin/sqoop export --connect jdbc:postgresql://
ec2-XX-XX-XXX-XX.compute-1.amazonaws.com:5662/dfg2tulf7263ut --username
ufjb0gfs1n5kut --password xxxxxx --table ml_ys_log_gmt_test --export-dir
$HADOOP_INPUT/$LOGS_HOME/dt=$(date +%Y-%m-%d)
--input-fields-terminated-by='\t' --lines-terminated-by='\n' --verbose
12/11/07 17:25:32 INFO mapred.JobClient: Task Id :
attempt_201211071722_0002_m_000000_0, Status : FAILED
java.io.IOException: java.sql.BatchUpdateException: Batch entry 71 INSERT
INTO ml_ys_log_gmt_test (date, ts, environment, resource, network,
advertiser, campaign, creative, cost, click, flavor, ui_element_id,
ui_element_type, event, charge_id, charge_type, charge_vertical,
charge_payer, charge_amount, model, imageformatsupport, inputmodesupport,
scriptsupport, vendor, stylesheetsupport, markupsupport, displaywidth,
cookiesupport, displayheight, version, inputdevices, displaycolordepth,
device_os, device_os_version, mobile_browser, mobile_browser_version,
is_tablet, dual_orientation, marketing_name) VALUES ('2012-10-31
20:42:56.767000 +00:00:00', '1351716176767',
'hxCA', 'MLNL', '1006', '10014', '1410', NULL, 'mdkP', '10014', '', '',
'click', '', '', '', '', NULL, 'CT810', '', '', '', 'LG', '', '', '400',
'', '240', '', 'stylus', '', 'Windows Mobile OS', '6.1', 'Microsoft Mobile
Explorer', '7.11', 'false', 'false', '') was aborted.  Call
getNextException to see the cause.
        at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:771)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:375)
        at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:396)
        at org.apache.hadoop.mapred.Child.main(Child.java:249)

I have attached ml_ys_log_gmt_test_with_map_column_java_ts_numeric.java

Please note that ts = '1351716176767' in the INSERT clause. The value seems
right. But since the definition of the column is numeric, the quotes (')
surrounding 1351716176767 should not be there.

Thank you for your help!

On Tue, Nov 6, 2012 at 2:33 PM, Jarek Jarcec Cecho <[EMAIL PROTECTED]>wrote:

> Hi Matthieu,
> thank you very much for your outputs. I've took a look and I'm really not
> sure what is wrong. It seems that from strange reason sqoop submits for ts
> column value '[B@ba6c13' that seems to me an object address. But code
> that you've sent is correctly serializing long, so I'm really not sure what
> might be an issue here.
Matthieu Labour, Engineering | *Action**X* |
584 Broadway, Suite 1002 – NY, NY 10012
415-994-3480 (m)