Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop >> mail # dev >> Review Request: SQOOP-931 - Integration of Sqoop and HCatalog


Copy link to this message
-
Re: Review Request: SQOOP-931 - Integration of Sqoop and HCatalog

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/10688/#review21552
-----------------------------------------------------------

Ship it!
Hi Venkat,
please do fix the last final typo and attach the patch to JIRA, I'll go ahead and commit it!
bin/configure-sqoop
<https://reviews.apache.org/r/10688/#comment44592>

    Nit: there seems to be extra "`" that is breaking the script.
Jarcec

- Jarek Cecho
On June 6, 2013, 10:55 p.m., Venkat Ranganathan wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/10688/
> -----------------------------------------------------------
>
> (Updated June 6, 2013, 10:55 p.m.)
>
>
> Review request for Sqoop and Jarek Cecho.
>
>
> Description
> -------
>
> This patch implements the new feature of integrating HCatalog and Sqoop.   With this feature, it is possible to import and export data between Sqoop and HCatalog tables.   The document attached to SQOOP-931 JIRA issue discusses the high level appraches.  
>
> With this integration, more fidelity can be brought to the process of moving data between enterprise data stores and hadoop ecosystem.
>
>
> Diffs
> -----
>
>   bin/configure-sqoop 61ff3f2
>   build.xml 636c103
>   ivy.xml 1fa4dd1
>   ivy/ivysettings.xml c4cc561
>   src/docs/user/SqoopUserGuide.txt 01ac1cf
>   src/docs/user/hcatalog.txt PRE-CREATION
>   src/java/org/apache/sqoop/SqoopOptions.java f18d43e
>   src/java/org/apache/sqoop/config/ConfigurationConstants.java 5354063
>   src/java/org/apache/sqoop/hive/HiveImport.java 838f083
>   src/java/org/apache/sqoop/manager/ConnManager.java a1ac38e
>   src/java/org/apache/sqoop/mapreduce/DataDrivenImportJob.java ef1d363
>   src/java/org/apache/sqoop/mapreduce/ExportJobBase.java 1065d0b
>   src/java/org/apache/sqoop/mapreduce/ImportJobBase.java 2465f3f
>   src/java/org/apache/sqoop/mapreduce/JdbcExportJob.java 20636a0
>   src/java/org/apache/sqoop/mapreduce/JobBase.java 0df1156
>   src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatExportFormat.java PRE-CREATION
>   src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatExportMapper.java PRE-CREATION
>   src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatImportMapper.java PRE-CREATION
>   src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatInputSplit.java PRE-CREATION
>   src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatRecordReader.java PRE-CREATION
>   src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatUtilities.java PRE-CREATION
>   src/java/org/apache/sqoop/tool/BaseSqoopTool.java 42f521f
>   src/java/org/apache/sqoop/tool/CodeGenTool.java dd34a97
>   src/java/org/apache/sqoop/tool/ExportTool.java 215addd
>   src/java/org/apache/sqoop/tool/ImportTool.java 2627726
>   src/perftest/ExportStressTest.java 0a41408
>   src/test/com/cloudera/sqoop/ThirdPartyTests.java 06f7122
>   src/test/com/cloudera/sqoop/hive/TestHiveImport.java 462ccf1
>   src/test/com/cloudera/sqoop/testutil/BaseSqoopTestCase.java cf41b96
>   src/test/com/cloudera/sqoop/testutil/ExportJobTestCase.java e13f3df
>   src/test/org/apache/sqoop/hcat/HCatalogExportTest.java PRE-CREATION
>   src/test/org/apache/sqoop/hcat/HCatalogImportTest.java PRE-CREATION
>   src/test/org/apache/sqoop/hcat/HCatalogTestUtils.java PRE-CREATION
>   src/test/org/apache/sqoop/hcat/TestHCatalogBasic.java PRE-CREATION
>   testdata/hcatalog/conf/hive-log4j.properties PRE-CREATION
>   testdata/hcatalog/conf/hive-site.xml PRE-CREATION
>   testdata/hcatalog/conf/log4j.properties PRE-CREATION
>
> Diff: https://reviews.apache.org/r/10688/diff/
>
>
> Testing
> -------
>
> Two new integration test suites with more than 20 tests in total have been added to test various aspects of the integration.  A unit test to test the option management is also added.   All tests pass
>
>
> Thanks,
>
> Venkat Ranganathan