Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop >> mail # user >> Sqoop exporting error


Copy link to this message
-
Re: Sqoop exporting error
Hi Matthieu,
would you mind sharing with us writable class that Sqoop generates for your table? I'm expecting that this file will be named "ml_ys_log_gmt.java" and it will be available in working directory where you're executing sqoop.

You might try utilizing parameter type mapping argument --map-column-java ts=Long or ts=String to see if it helps.

Jarcec

On Fri, Nov 02, 2012 at 07:18:44PM -0400, Matthieu Labour wrote:
> Hi
>
> Sometimes sqoop craps out. I am looking for any tips or help. Thank you
> very much !
>
> The following command runs
>
> ~/$SQOOP_ROOT/bin/sqoop export --connect jdbc:postgresql://
> ec2-XX-XX-XXX-XX.compute-1.amazonaws.com:5662/xxxxxxxxx --username xxxxxxxx
> --password xxxxxxxx --table ml_ys_log_gmt --export-dir
> $HADOOP_INPUT/$LOGS_HOME/dt=$(date +%Y-%m-%d)
> --input-fields-terminated-by='\t' --lines-terminated-by='\n' --verbose
> --batch
>
> on the following line
>
> 2012-10-27 23:44:25.699 1351381465699
> remote-addr=10.119.81.125&user-agent=SAMSUNG-SGH-I637%2FUCIJ2+Profile%2FMIDP-2.1+Configuration%2FCLDC-1.1+Mozilla%2F4.0+%28compatible%3B+MSIE+6.0%3B+Windows+CE%3B+IEMobile+8.12%3B+MSIEMobile+6.0%29
>   hxCA    MLNL    1006    10014   1410            kUQN    10014
>       click                                           SGH-i637
>                    Samsung                 320
>
> I get the error below.
>
> The insert craps out on the second column 'ts' in INSERT INTO ml_ys_log_gmt
> (date, ts
>
> The value passed is '[B@6127da' when it should be 1351381465699 (second
> field in the tab delimited line above)
>
> Here is the description in the table
> ts                     | bigint                   |
>         | plain    |
>
>
> 12/10/28 06:57:20 INFO mapred.JobClient: Task Id :
> attempt_201210280647_0004_m_000000_0, Status : FAILED
> java.io.IOException: java.sql.BatchUpdateException: Batch entry 59 INSERT
> INTO ml_ys_log_gmt (date, ts, environment, resource, network, advertiser,
> campaign, creative, cost, click, flavor, ui_element_id, ui_element_type,
> event, charge_id, charge_type, charge_vertical, charge_payer,
> charge_amount, model, imageformatsupport, inputmodesupport, scriptsupport,
> vendor, stylesheetsupport, markupsupport, displaywidth, cookiesupport,
> displayheight, version, inputdevices, displaycolordepth, device_os,
> device_os_version, mobile_browser, mobile_browser_version, is_tablet,
> dual_orientation, marketing_name) VALUES ('2012-10-27 23:44:25.699000
> +00:00:00', '[B@6127da',
> 'remote-addr=10.119.81.125&user-agent=SAMSUNG-SGH-I637%2FUCIJ2+Profile%2FMIDP-2.1+Configuration%2FCLDC-1.1+Mozilla%2F4.0+%28compatible%3B+MSIE+6.0%3B+Windows+CE%3B+IEMobile+8.12%3B+MSIEMobile+6.0%29',
> 'hxCA', 'MLNL', '1006', '10014', '1410', NULL, 'kUQN', '10014', '', '',
> 'click', '', '', '', '', NULL, 'SGH-i637', '', '', '', 'Samsung', '', '',
> '320', '', '240', '', '', '', 'Windows Mobile OS', '5.0', 'Microsoft Mobile
> Explorer', '8.12', 'false', 'false', 'BlackJack') was aborted.  Call
> getNextException to see the cause.
> at
> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.write(AsyncSqlRecordWriter.java:220)
> at
> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.write(AsyncSqlRecordWriter.java:46)
> at
> org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.write(MapTask.java:646)
> at
> org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
> at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:78)
> at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:36)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> at
> org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:182)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:771)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:375)
> at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)