Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop, mail # user - Sqoop 1.99.2 + Cloud Era Hadoop 200 error


Copy link to this message
-
Re: Sqoop 1.99.2 + Cloud Era Hadoop 200 error
Mengwei Ding 2013-07-03, 14:00
Hi Madhanmohan,

Thank you for providing all these detailed information. Help a lot to
diagnose the problem.

First, the addtowar.sh is not good enough for every situation, we apologize
for that. We have already figured out a new way to add dependency library,
which will coming out along with next version of Sqoop2.

Currently, it seems like the hadoop-core.jar has not been added. I could
show you all the libraries existing in the webapps/sqoop/WEB-INF/lib
folder, please check below:
avro-1.7.4.jar
commons-cli-1.2.jar
commons-configuration-1.6.jar
commons-dbcp-1.4.jar
commons-lang-2.5.jar
commons-logging-1.1.1.jar
commons-pool-1.5.4.jar
derby-10.8.2.2.jar
guava-11.0.2.jar
hadoop-auth-2.0.0-cdh4.3.0.jar
hadoop-common-2.0.0-cdh4.3.0.jar
*hadoop-core-2.0.0-mr1-cdh4.3.0.jar*
hadoop-hdfs-2.0.0-cdh4.3.0.jar
hadoop-mapreduce-client-app-2.0.0-cdh4.3.0.jar
hadoop-mapreduce-client-common-2.0.0-cdh4.3.0.jar
hadoop-mapreduce-client-core-2.0.0-cdh4.3.0.jar
hadoop-mapreduce-client-jobclient-2.0.0-cdh4.3.0.jar
hadoop-yarn-api-2.0.0-cdh4.3.0.jar
hadoop-yarn-common-2.0.0-cdh4.3.0.jar
jackson-core-asl-1.8.8.jar
jackson-mapper-asl-1.8.8.jar
json-simple-1.1.jar
log4j-1.2.16.jar
mysql-connector-java-5.1.25-bin.jar
protobuf-java-2.4.0a.jar
slf4j-api-1.6.1.jar
slf4j-log4j12-1.6.1.jar
sqoop-common-1.99.2.jar
sqoop-connector-generic-jdbc-1.99.2.jar
sqoop-core-1.99.2.jar
sqoop-execution-mapreduce-1.99.2-hadoop200.jar
sqoop-repository-derby-1.99.2.jar
sqoop-spi-1.99.2.jar
sqoop-submission-mapreduce-1.99.2-hadoop200.jar

I have the same Hadoop and Sqoop2 installation directories with you. And I
am running a pseudo cluster in a single Ubuntu virtual machine.

So, now, you could try to add the hadoop-core.jar manually, and then go
ahead to see whether the sqoop2 server could run. Please follow the
following steps:

*./bin/addtowar.sh
-jars /usr/lib/hadoop-0.20-mapreduce/hadoop-core-2.0.0-mr1-cdh4.3.0.jar*

Please find the hadoop-core.jar in your own machine. It should be in a
similar place. But still, if you have problem, please let me know.
The reason why it's better to remove the "sqoop" folder is to clear the
cached old servlet. Because Tomcat cannot alway extract the sqoop.war file
immediately after you add dependency library to sqoop.war file. By removing
the sqoop folder, the Tomcat is forced to extract the sqoop.war to keep the
sqoop folder up-to-date. So in this way, you could know whether you have
correctly setup the dependency library. Does this explanation help?

Best,
Mengwei
On Tue, Jul 2, 2013 at 9:19 PM, Madhanmohan Savadamuthu
<[EMAIL PROTECTED]>wrote:

> Hi  Mengwei,
>
> Following are details
>
> Hadoop Version: Hadoop 2.0.0-cdh4.2.1
> Linux Version: Linux version 2.6.32-358.2.1.el6.x86_64 (
> [EMAIL PROTECTED]) (gcc version 4.4.7 20120313
> (Red Hat 4.4.7-3) (GCC) ) #1 SMP Wed Feb 20 12:17:37 EST 2013
> Hadoop Installation Location: /usr/lib/hadoop
> Sqoop2 Installation Location: /usr/lib/sqoop2
> Sqoop2 Dependency Configuration  Command Used: ./bin/addtowar.sh
> -hadoop-auto
> Files in :
>
> avro-1.7.3.jar
> commons-cli-1.2.jar
> commons-configuration-1.6.jar
> commons-dbcp-1.4.jar
> commons-lang-2.5.jar
> commons-logging-1.1.1.jar
> commons-pool-1.5.4.jar
> derby-10.8.2.2.jar
> guava-11.0.2.jar
> hadoop-auth-2.0.0-cdh4.2.1.jar
> hadoop-common-2.0.0-cdh4.2.1-tests.jar
> hadoop-hdfs-2.0.0-cdh4.2.1.jar
> hadoop-mapreduce-client-app-2.0.0-cdh4.2.1.jar
> hadoop-mapreduce-client-common-2.0.0-cdh4.2.1.jar
> hadoop-mapreduce-client-core-2.0.0-cdh4.2.1.jar
> hadoop-mapreduce-client-jobclient-2.0.0-cdh4.2.1.jar
> hadoop-yarn-api-2.0.0-cdh4.2.1.jar
> hadoop-yarn-common-2.0.0-cdh4.2.1.jar
> jackson-core-asl-1.8.8.jar
> jackson-mapper-asl-1.8.8.jar
> json-simple-1.1.jar
> log4j-1.2.16.jar
> mysql-connector-java-5.1.25-bin.jar
> protobuf-java-2.4.0a.jar
> slf4j-api-1.6.1.jar
> slf4j-log4j12-1.6.1.jar
> sqoop-common-1.99.2.jar
> sqoop-connector-generic-jdbc-1.99.2.jar
> sqoop-core-1.99.2.jar
> sqoop-execution-mapreduce-1.99.2-hadoop200.jar