Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop >> mail # user >> Sqoop2 setup questions


Copy link to this message
-
Re: Sqoop2 setup questions
Hi Devin,
Please see inline.

On Thu, Apr 3, 2014 at 10:52 AM, Devin Suiter RDX <[EMAIL PROTECTED]> wrote:
client and server can be run on the same box. You may be using the wrong
(old) tar ball release for sqoop2 server.

use this -
You are using old svn code build and setup.

Check this link to build:
http://sqoop.apache.org/docs/1.99.3/BuildingSqoop2.html
Installation part given in :
http://sqoop.apache.org/docs/1.99.3/Installation.html
mvn compile has a bug, use mvn install instead

Check tomcat's connector port in server.xml
it should look something like this -

<Connector port="${sqoop.http.port}" protocol="HTTP/1.1"

If you find that its hardcoded to a default port number instead, then that
is your problem, the port is not configurable. Change that line to the
above so it recognizes the -D option given to start the sqoop2 server
process.

You can run sqoop 1 and sqoop2 server on different nodes in the cluster
simultaneously, but not on the same box.

So that will help you a lot in migration, as sqoop2 server still has many
limitations as compared below in cdh documentation -
Feature Differences - Sqoop and Sqoop 2
   *Note*:

*Moving from Apache Sqoop to Sqoop 2:* Sqoop 2 is essentially the future of
the Apache Sqoop project. However, since Sqoop 2 currently lacks some of
the features of Sqoop, Cloudera recommends you use Sqoop 2 only if it
contains all the features required for your use case, otherwise, continue
to use Sqoop.
 Feature Sqoop Sqoop 2Connectors for all major RDBMS Supported.

Not supported.

*Workaround*: Use the generic JDBC Connector which has been tested on the
following databases: Microsoft SQL Server, PostgreSQL, MySQL and Oracle.

This connector should work on any other JDBC compliant database. However,
performance might not be comparable to that of specialized connectors in
Sqoop.
Kerberos Security IntegrationSupported.

Not supported.
Encryption of Stored PasswordsNot supported. No workaround. Supported using
Derby's on-disk
encryption<http://db.apache.org/derby/docs/10.2/devguide/cdevcsecure24366.html>
.

*Disclaimer:* Although expected to work in the current version of Sqoop 2,
this configuration has not been verified.
Data transfer from RDBMS to Hive or HBaseSupported.

Not supported.
*Workaround:* Follow this two-step approach.

   1. Import data from RDBMS into HDFS (either as a text or sequence file)
   2. Export to Hive or HBase using Sqoop 2

Data transfer from Hive or HBase to RDBMSNot supported.
*Workaround:* Follow this two-step approach.

   1. Import data from Hive or HBase into HDFS (either as a text or
   sequence file)
   2. Export to RDBMS using Sqoop 2

Not supported.

Follow the same workaround as for Sqoop.

Thanks,
Suhas.