-
Re: need help on merge tool in sqoopArvind Prabhakar 2012-02-22, 05:35
Hi Narasimharao,
When you invoke the merge command, try specifying the complete path of the
jar file to use. From the errors you have described, it appears that Sqoop
is unable to load the jar file correctly.
Thanks,
Arvind
On Tue, Feb 21, 2012 at 8:28 PM, chelikani narasimharao <
[EMAIL PROTECTED]> wrote:
> hi guys,
> Can any one please help me for the below issue.
>
> Advanced Thanks,
> Narasimharao
>
>
> On Thu, Feb 16, 2012 at 8:24 PM, chelikani narasimharao <
> [EMAIL PROTECTED]> wrote:
>
>> Hi All,
>>
>> I am trying to merge the two datasets using the merge tool
>> functionality in sqoop.But i got struck up with the following error.
>>
>> *java.lang.RuntimeException: java.lang.ClassNotFoundException:
>> CNTCT_D.class*
>>
>> *---->below is my command executed to get the first data set specifying
>> the location to store the first dataset to dataset1 under narasimharao
>> directory*
>>
>> sqoop import --connect jdbc:hpt4jdbc://
>> g4n0401a.houston.hp.com:18650/:schema=EDW_DEV --driver
>> com.hp.t4jdbc.HPT4Driver --username [EMAIL PROTECTED] -P --verbose
>> --split-by SRC_SYS_KY --table CNTCT_D --where "SRC_SYS_KY=363 and CNTCT_ID
>> in('100838','101324','101474','102212','100654') and ACT_FG='Y'"
>> --target-dir /home/narasimharao/dataset1
>>
>> successfully able to import the dataset1 and also the .class and .jar
>> files were written to the following location
>>
>> */tmp/sqoop-narasimharao/compile/08c0cc665bd3e89124987a7c297827a2*
>>
>>
>> *--->below is my command executed to get the second data set after
>> updating the few records which exists in the dataset1 with the same
>> merge-key*
>>
>>
>> sqoop import --connect jdbc:hpt4jdbc://
>> g4n0401a.houston.hp.com:18650/:schema=EDW_DEV --driver
>> com.hp.t4jdbc.HPT4Driver --username [EMAIL PROTECTED] -P
>> --verbose --split-by SRC_SYS_KY --table CNTCT_D --where "SRC_SYS_KY=363 and
>> CNTCT_ID in('100838','101324','101474','102212','100654') and ACT_FG='Y'"
>> --target-dir /home/narasimharao/dataset2
>>
>> successfully able to import the dataset1 and also the .class and .jar
>> files were written to the following location
>>
>> *
>> /tmp/sqoop-narasimharao/compile/e26282f14a672e7722352fb424781e5a/CNTCT_D.jar
>> *
>>
>>
>> *--->Now the actually Sqoop merge command was given like below:*
>>
>> *sqoop merge --new-data /home/narasimharao/dataset1 --onto
>> /home/narasimharao/dataset2 --target-dir /home/narasimharao/merged
>> --jar-file CNTCT_D.jar --class-name CNTCT_D.class --merge-key CNTCT_ID*
>>
>>
>> Can you please help me which class name i need to mention over here under
>> the class-name attribute and what's wrong in this statement.
>>
>> I tried the below options
>>
>> 1.copied the dataset1 corresponding class and jar files into my home
>> direcotry and tried like below...but getting the same error as below
>>
>> sqoop merge --new-data /home/narasimharao/dataset2 --onto
>> /home/narasimharao/dataset1 --target-dir /home/narasimharao/merged
>> --jar-file ./CNTCT_D.jar --class-name CNTCT_D.class --merge-key CNTCT_ID
>>
>> Warning: /usr/lib/hbase does not exist! HBase imports will fail.
>> Please set $HBASE_HOME to the root of your HBase installation.
>>
>> 12/02/16 13:56:08 INFO input.FileInputFormat: Total input paths to
>> process : 2
>> 12/02/16 13:56:08 WARN snappy.LoadSnappy: Snappy native library is
>> available
>> 12/02/16 13:56:08 INFO util.NativeCodeLoader: Loaded the native-hadoop
>> library
>> 12/02/16 13:56:08 INFO snappy.LoadSnappy: Snappy native library loaded
>> 12/02/16 13:56:10 INFO mapred.JobClient: Running job:
>> job_201202161202_0005
>> 12/02/16 13:56:11 INFO mapred.JobClient: map 0% reduce 0%
>> 12/02/16 13:56:55 INFO mapred.JobClient: Task Id :
>> attempt_201202161202_0005_m_000000_0, Status : FAILED
>> *java.lang.RuntimeException: java.lang.ClassNotFoundException:
>> CNTCT_D.class
>> * at
>> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:996)
>> at
>> com.cloudera.sqoop.mapreduce.MergeTextMapper.setup(MergeTextMapper.java:45)
When you invoke the merge command, try specifying the complete path of the
jar file to use. From the errors you have described, it appears that Sqoop
is unable to load the jar file correctly.
Thanks,
Arvind
On Tue, Feb 21, 2012 at 8:28 PM, chelikani narasimharao <
[EMAIL PROTECTED]> wrote:
> hi guys,
> Can any one please help me for the below issue.
>
> Advanced Thanks,
> Narasimharao
>
>
> On Thu, Feb 16, 2012 at 8:24 PM, chelikani narasimharao <
> [EMAIL PROTECTED]> wrote:
>
>> Hi All,
>>
>> I am trying to merge the two datasets using the merge tool
>> functionality in sqoop.But i got struck up with the following error.
>>
>> *java.lang.RuntimeException: java.lang.ClassNotFoundException:
>> CNTCT_D.class*
>>
>> *---->below is my command executed to get the first data set specifying
>> the location to store the first dataset to dataset1 under narasimharao
>> directory*
>>
>> sqoop import --connect jdbc:hpt4jdbc://
>> g4n0401a.houston.hp.com:18650/:schema=EDW_DEV --driver
>> com.hp.t4jdbc.HPT4Driver --username [EMAIL PROTECTED] -P --verbose
>> --split-by SRC_SYS_KY --table CNTCT_D --where "SRC_SYS_KY=363 and CNTCT_ID
>> in('100838','101324','101474','102212','100654') and ACT_FG='Y'"
>> --target-dir /home/narasimharao/dataset1
>>
>> successfully able to import the dataset1 and also the .class and .jar
>> files were written to the following location
>>
>> */tmp/sqoop-narasimharao/compile/08c0cc665bd3e89124987a7c297827a2*
>>
>>
>> *--->below is my command executed to get the second data set after
>> updating the few records which exists in the dataset1 with the same
>> merge-key*
>>
>>
>> sqoop import --connect jdbc:hpt4jdbc://
>> g4n0401a.houston.hp.com:18650/:schema=EDW_DEV --driver
>> com.hp.t4jdbc.HPT4Driver --username [EMAIL PROTECTED] -P
>> --verbose --split-by SRC_SYS_KY --table CNTCT_D --where "SRC_SYS_KY=363 and
>> CNTCT_ID in('100838','101324','101474','102212','100654') and ACT_FG='Y'"
>> --target-dir /home/narasimharao/dataset2
>>
>> successfully able to import the dataset1 and also the .class and .jar
>> files were written to the following location
>>
>> *
>> /tmp/sqoop-narasimharao/compile/e26282f14a672e7722352fb424781e5a/CNTCT_D.jar
>> *
>>
>>
>> *--->Now the actually Sqoop merge command was given like below:*
>>
>> *sqoop merge --new-data /home/narasimharao/dataset1 --onto
>> /home/narasimharao/dataset2 --target-dir /home/narasimharao/merged
>> --jar-file CNTCT_D.jar --class-name CNTCT_D.class --merge-key CNTCT_ID*
>>
>>
>> Can you please help me which class name i need to mention over here under
>> the class-name attribute and what's wrong in this statement.
>>
>> I tried the below options
>>
>> 1.copied the dataset1 corresponding class and jar files into my home
>> direcotry and tried like below...but getting the same error as below
>>
>> sqoop merge --new-data /home/narasimharao/dataset2 --onto
>> /home/narasimharao/dataset1 --target-dir /home/narasimharao/merged
>> --jar-file ./CNTCT_D.jar --class-name CNTCT_D.class --merge-key CNTCT_ID
>>
>> Warning: /usr/lib/hbase does not exist! HBase imports will fail.
>> Please set $HBASE_HOME to the root of your HBase installation.
>>
>> 12/02/16 13:56:08 INFO input.FileInputFormat: Total input paths to
>> process : 2
>> 12/02/16 13:56:08 WARN snappy.LoadSnappy: Snappy native library is
>> available
>> 12/02/16 13:56:08 INFO util.NativeCodeLoader: Loaded the native-hadoop
>> library
>> 12/02/16 13:56:08 INFO snappy.LoadSnappy: Snappy native library loaded
>> 12/02/16 13:56:10 INFO mapred.JobClient: Running job:
>> job_201202161202_0005
>> 12/02/16 13:56:11 INFO mapred.JobClient: map 0% reduce 0%
>> 12/02/16 13:56:55 INFO mapred.JobClient: Task Id :
>> attempt_201202161202_0005_m_000000_0, Status : FAILED
>> *java.lang.RuntimeException: java.lang.ClassNotFoundException:
>> CNTCT_D.class
>> * at
>> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:996)
>> at
>> com.cloudera.sqoop.mapreduce.MergeTextMapper.setup(MergeTextMapper.java:45)
