Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop >> mail # user >> Fwd: Sqoop export not working when using "update-key"


Copy link to this message
-
Re: Fwd: Sqoop export not working when using "update-key"
I think I found something. The java class generated when using update-key
differs from the one without update-key. The one that throws exception is
missing to write the fields that are not specified in the update-key. I
also see (with --verbose) that when using --update-key there is an extra
debug line that says
*14/07/17 22:53:27 DEBUG orm.ClassWriter: db write column order:*
*14/07/17 22:53:27 DEBUG orm.ClassWriter:   SEARCH_DATE*

Below is the method generated for the command *without* --update-key

  public int write(PreparedStatement __dbStmt, int __off) throws
SQLException {
    JdbcWritableBridge.writeTimestamp(SEARCH_DATE, 1 + __off, 93, __dbStmt);
    JdbcWritableBridge.writeString(SEARCH_TYPE, 2 + __off, 12, __dbStmt);
    JdbcWritableBridge.writeString(USER_AGENT, 3 + __off, 12, __dbStmt);
    JdbcWritableBridge.writeString(SRCH_KEYWORD, 4 + __off, 12, __dbStmt);
    JdbcWritableBridge.writeBigDecimal(SRCH_COUNT, 5 + __off, 2, __dbStmt);
    return 5;
  }

Below is the one generated for the command *with* --update-key
  public int write(PreparedStatement __dbStmt, int __off) throws
SQLException {
    JdbcWritableBridge.writeTimestamp(SEARCH_DATE, 1 + __off, 93, __dbStmt);
    return 1;
  }

I tried to force export to use the properly generated class with parameters
"jar-file" and "class-name" but that didn't work, if like those params are
not allowed in the export command. This is what I tried to force using the
properly generated source
*sqoop export \*
*--connect jdbc:oracle:thin:@ddb04.local.com:1541/test04
<http://jdbc:oracle%3Athin%[EMAIL PROTECTED]:1541/test04> \*
*--update-key "SEARCH_DATE" \*
*--columns $columns \*
*--table $table --username $user --password $passwd \*
*--fields-terminated-by "=" --export-dir $exportDir*
*--jar-file SEARCH_TABLE.jar --class-name SEARCH_TABLE*

On Thu, Jul 17, 2014 at 5:04 PM, Leonardo Brambilla <
[EMAIL PROTECTED]> wrote:
 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB