Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop, mail # dev - Review Request 15589: SQOOP-1232 Sqoop2: Provide tooling infrastructure for Sqoop2


Copy link to this message
-
Re: Review Request 15589: SQOOP-1232 Sqoop2: Provide tooling infrastructure for Sqoop2
Abraham Elmahrek 2013-11-25, 22:46

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/15589/#review29394
-----------------------------------------------------------
This looks really good.

Do you think that developers will want to add their own tool commands in the future?

Just a couple of nits below.
tools/src/main/java/org/apache/sqoop/tools/tool/BuildinTools.java
<https://reviews.apache.org/r/15589/#comment56624>

    built-in??

tools/src/main/java/org/apache/sqoop/tools/tool/VerifyTool.java
<https://reviews.apache.org/r/15589/#comment56596>

    tier = tear
- Abraham Elmahrek
On Nov. 15, 2013, 8:02 p.m., Jarek Cecho wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/15589/
> -----------------------------------------------------------
>
> (Updated Nov. 15, 2013, 8:02 p.m.)
>
>
> Review request for Sqoop.
>
>
> Bugs: SQOOP-1232
>     https://issues.apache.org/jira/browse/SQOOP-1232
>
>
> Repository: sqoop-sqoop2
>
>
> Description
> -------
>
> I've provided basic infrastructure for the tooling by adding new maven module sqoop-tools that is depending on the core module. I've also provided example tool called "verify" that will try to bootstrap all sqoop components to see if they are configured correctly.
>
> I was having difficulties to make run the code with the same environment as the Sqoop 2 server itself, but without the need to open the Tomcat for outside connections. I finally ended up using Tool mechanism that is exposed by Tomcat. Sadly this mechanism won't set up the environment exactly as we need it - common.loader and webapp class loaders are not loaded by this mechanism. so I've provided a class that will finish configuring the environment and call the usual ToolRunner. To keep the footprint of this class/jar file to a bare minimum I've placed it into standalone maven module that do not depend on any Sqoop libraries.
>
>
> Diffs
> -----
>
>   dist/pom.xml 9186a38b3b9c57a62731f01839a99dc7cad9fc99
>   dist/src/main/bin/sqoop.sh e3ed5ef4bbaba0c3048cb005d4e183e66768117e
>   pom.xml a4bc085ce808db912b6ca516b51c70529bd081e7
>   server/pom.xml a07ecf41c2aa687aa4a2a60367e1d1265610b7e9
>   tomcat/pom.xml PRE-CREATION
>   tomcat/src/main/java/org/apache/sqoop/tomcat/TomcatToolRunner.java PRE-CREATION
>   tools/pom.xml PRE-CREATION
>   tools/src/main/java/org/apache/sqoop/tools/Tool.java PRE-CREATION
>   tools/src/main/java/org/apache/sqoop/tools/ToolRunner.java PRE-CREATION
>   tools/src/main/java/org/apache/sqoop/tools/tool/BuildinTools.java PRE-CREATION
>   tools/src/main/java/org/apache/sqoop/tools/tool/VerifyTool.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/15589/diff/
>
>
> Testing
> -------
>
> I'm currently not attaching any unit tests as this task should be better covered some sort of integration testing. I did however tested the provided verification tool on a real cluster both with correct configuration and miss configuration (missing Hadoop dependencies).
>
>
> Thanks,
>
> Jarek Cecho
>
>