Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop, mail # dev - Review Request: SQOOP-483 Allow target dir to be set to a different name than table name for hive import


Copy link to this message
-
Review Request: SQOOP-483 Allow target dir to be set to a different name than table name for hive import
Cheolsoo Park 2012-05-25, 23:18

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/5248/
-----------------------------------------------------------

Review request for Sqoop and Jarek Cecho.
Summary
-------

Currently, the table name (set by --table) has to be the same as the target dir (set by --target-dir) for hive import. But it would be nice if the target dir could be set to a different name than the table name.

The changes include:

1) Implement the following logic in TableDefWriter:
   - If target dir is specified, use it as the source.
   - Otherwise, use the name of input table as the source.

The original logic was less flexible: if input table is specified, use it as the source; otherwise, use target dir as the source. However, this disallows target dir to be different from input table name when importing the entire table. With the new logic, the user can freely specify input table, target dir, and output table in any case.

2) Clean up output-path-related code in Hive import.

3) Add a unit test that verifies that TableDefWriter generates correct statements given input table, output table, and target-dir are all different.
This addresses bug SQOOP-483.
    https://issues.apache.org/jira/browse/SQOOP-483
Diffs
-----

  /src/java/org/apache/sqoop/hive/HiveImport.java 1342749
  /src/java/org/apache/sqoop/hive/TableDefWriter.java 1342749
  /src/test/com/cloudera/sqoop/hive/TestTableDefWriter.java 1342749

Diff: https://reviews.apache.org/r/5248/diff
Testing
-------

1) Ran ant test, ant test -Dthirdparty=true, and ant checkstyle.

2) Tested with Hive the following scenarios:
- sqoop hive-import ... --table FOO
- sqoop hive-import ... --table FOO --hive table FOO
- sqoop hive-import ... --table FOO --hive table BAR
- sqoop hive-import ... --table FOO --hive-table FOO --target-dir FOO
- sqoop hive-import ... --table FOO --hive-table FOO --target-dir BAR
- sqoop hive-import ... --table FOO --hive-table BAR --target-dir FOO
- sqoop hive-import ... --table FOO --hive-table BAR --target-dir BAR
- sqoop hive-import ... --table FOO --hive-table BAR --target-dir BAZ
Thanks,

Cheolsoo