-
Re: need help on import toolArvind Prabhakar 2012-01-23, 17:27
Hi Narasimharao,
>From the trace in your mail, it seems that the query generated by
DBRecordReader is invalid. There are no obvious reasons why this may happen
since the other parts of the system seem to be working well. One thing to
check would be to run the import using a single mapper (-m 1) and see if
that works. Also, look at the task logs to see if they have any more
information as to why this has failed.
Alternatively, you could build the code locally and instrument it to
produce more debugging output in DBRecordReader to see exactly why it is
behaving the way it is.
Thanks,
Arvind
On Mon, Jan 23, 2012 at 5:31 AM, chelikani narasimharao <
[EMAIL PROTECTED]> wrote:
> hi guys,
> Can any one help me on this..I got struck up with this
> error.
>
> Appreciate Your Help
>
> Narasimharao
>
>
> On Fri, Jan 20, 2012 at 8:58 PM, chelikani narasimharao <
> [EMAIL PROTECTED]> wrote:
>
>> Hi ,
>> I am using sqoop to get extract the data from HPNeoView database
>> and then want to write onto Hadoop file system.When i am using the
>> following command i am getting the below error. Here i am providing full
>> error details what i was received when i am running the below command
>>
>> sqoop import --connect jdbc:hpt4jdbc://
>> g4n0601a.houston.hp.com:18650/chema=EDW_INT --driver
>> com.hp.t4jdbc.HPT4Driver --username boyapatr_write -P --verbose --table
>> EDW_INT.OPPTY_ALNC_PTNR_F --split-by SRC_SYS_KY --target-dir
>> /home/narasimharao/opptyfact
>>
>> DEBUG manager.SqlManager: Using fetchSize for next query: 1000
>> 12/01/20 19:32:31 INFO manager.SqlManager: Executing SQL statement:
>> SELECT t.* FROM EDW_INT.OPPTY_ALNC_PTNR_F AS t WHERE 1=0
>> 12/01/20 19:32:40 DEBUG manager.SqlManager: Using fetchSize for next
>> query: 1000
>> 12/01/20 19:32:40 INFO manager.SqlManager: Executing SQL statement:
>> SELECT t.* FROM EDW_INT.OPPTY_ALNC_PTNR_F AS t WHERE 1=0
>> 12/01/20 19:32:47 DEBUG orm.ClassWriter: selected columns:
>> 12/01/20 19:32:47 DEBUG orm.ClassWriter: OPPTY_ALNC_PTNR_ID
>> 12/01/20 19:32:47 DEBUG orm.ClassWriter: SRC_SYS_KY
>> 12/01/20 19:32:47 DEBUG orm.ClassWriter: PTNR_SRC_SYS_KY
>> 12/01/20 19:32:47 DEBUG orm.ClassWriter: OPPTY_ID
>> 12/01/20 19:32:47 DEBUG orm.ClassWriter: PTNR_ID
>> 12/01/20 19:32:47 DEBUG orm.ClassWriter: PTNR_ROLE_CD
>> 12/01/20 19:32:47 DEBUG orm.ClassWriter: PRIM_PTNR_FG
>> 12/01/20 19:32:47 DEBUG orm.ClassWriter: INS_GMT_TS
>> 12/01/20 19:32:47 DEBUG orm.ClassWriter: UPD_GMT_TS
>> 12/01/20 19:32:47 DEBUG orm.ClassWriter: LOAD_JOB_NR
>> 12/01/20 19:32:47 DEBUG orm.ClassWriter: REC_ST_NR
>> 12/01/20 19:32:47 DEBUG orm.ClassWriter: HPQ_RATING_CD
>> 12/01/20 19:32:47 DEBUG orm.ClassWriter: SRC_SYS_UPD_TS
>> 12/01/20 19:32:47 DEBUG orm.ClassWriter: Writing source file:
>> /tmp/sqoop-narasimharao/compile/eb164bb8e2c0031416457bda6b86dd19/EDW_INT_OPPTY_ALNC_PTNR_F.java
>> 12/01/20 19:32:47 DEBUG orm.ClassWriter: Table name:
>> EDW_INT.OPPTY_ALNC_PTNR_F
>> 12/01/20 19:32:47 DEBUG orm.ClassWriter: Columns: OPPTY_ALNC_PTNR_ID:1,
>> SRC_SYS_KY:-5, PTNR_SRC_SYS_KY:-5, OPPTY_ID:1, PTNR_ID:1, PTNR_ROLE_CD:1,
>> PRIM_PTNR_FG:1, INS_GMT_TS:93, UPD_GMT_TS:93, LOAD_JOB_NR:2, REC_ST_NR:5,
>> HPQ_RATING_CD:1, SRC_SYS_UPD_TS:93,
>> 12/01/20 19:32:47 DEBUG orm.ClassWriter: sourceFilename is
>> EDW_INT_OPPTY_ALNC_PTNR_F.java
>> 12/01/20 19:32:47 DEBUG orm.CompilationManager: Found existing
>> /tmp/sqoop-narasimharao/compile/eb164bb8e2c0031416457bda6b86dd19/
>> 12/01/20 19:32:47 INFO orm.CompilationManager: HADOOP_HOME is
>> /usr/lib/hadoop
>> 12/01/20 19:32:47 INFO orm.CompilationManager: Found hadoop core jar at:
>> /usr/lib/hadoop/hadoop-core.jar
>> 12/01/20 19:32:48 DEBUG orm.CompilationManager: Adding source file:
>> /tmp/sqoop-narasimharao/compile/eb164bb8e2c0031416457bda6b86dd19/EDW_INT_OPPTY_ALNC_PTNR_F.java
>> 12/01/20 19:32:48 DEBUG orm.CompilationManager: Invoking javac with args:
>> 12/01/20 19:32:48 DEBUG orm.CompilationManager: -sourcepath
>> 12/01/20 19:32:48 DEBUG orm.CompilationManager:
>From the trace in your mail, it seems that the query generated by
DBRecordReader is invalid. There are no obvious reasons why this may happen
since the other parts of the system seem to be working well. One thing to
check would be to run the import using a single mapper (-m 1) and see if
that works. Also, look at the task logs to see if they have any more
information as to why this has failed.
Alternatively, you could build the code locally and instrument it to
produce more debugging output in DBRecordReader to see exactly why it is
behaving the way it is.
Thanks,
Arvind
On Mon, Jan 23, 2012 at 5:31 AM, chelikani narasimharao <
[EMAIL PROTECTED]> wrote:
> hi guys,
> Can any one help me on this..I got struck up with this
> error.
>
> Appreciate Your Help
>
> Narasimharao
>
>
> On Fri, Jan 20, 2012 at 8:58 PM, chelikani narasimharao <
> [EMAIL PROTECTED]> wrote:
>
>> Hi ,
>> I am using sqoop to get extract the data from HPNeoView database
>> and then want to write onto Hadoop file system.When i am using the
>> following command i am getting the below error. Here i am providing full
>> error details what i was received when i am running the below command
>>
>> sqoop import --connect jdbc:hpt4jdbc://
>> g4n0601a.houston.hp.com:18650/chema=EDW_INT --driver
>> com.hp.t4jdbc.HPT4Driver --username boyapatr_write -P --verbose --table
>> EDW_INT.OPPTY_ALNC_PTNR_F --split-by SRC_SYS_KY --target-dir
>> /home/narasimharao/opptyfact
>>
>> DEBUG manager.SqlManager: Using fetchSize for next query: 1000
>> 12/01/20 19:32:31 INFO manager.SqlManager: Executing SQL statement:
>> SELECT t.* FROM EDW_INT.OPPTY_ALNC_PTNR_F AS t WHERE 1=0
>> 12/01/20 19:32:40 DEBUG manager.SqlManager: Using fetchSize for next
>> query: 1000
>> 12/01/20 19:32:40 INFO manager.SqlManager: Executing SQL statement:
>> SELECT t.* FROM EDW_INT.OPPTY_ALNC_PTNR_F AS t WHERE 1=0
>> 12/01/20 19:32:47 DEBUG orm.ClassWriter: selected columns:
>> 12/01/20 19:32:47 DEBUG orm.ClassWriter: OPPTY_ALNC_PTNR_ID
>> 12/01/20 19:32:47 DEBUG orm.ClassWriter: SRC_SYS_KY
>> 12/01/20 19:32:47 DEBUG orm.ClassWriter: PTNR_SRC_SYS_KY
>> 12/01/20 19:32:47 DEBUG orm.ClassWriter: OPPTY_ID
>> 12/01/20 19:32:47 DEBUG orm.ClassWriter: PTNR_ID
>> 12/01/20 19:32:47 DEBUG orm.ClassWriter: PTNR_ROLE_CD
>> 12/01/20 19:32:47 DEBUG orm.ClassWriter: PRIM_PTNR_FG
>> 12/01/20 19:32:47 DEBUG orm.ClassWriter: INS_GMT_TS
>> 12/01/20 19:32:47 DEBUG orm.ClassWriter: UPD_GMT_TS
>> 12/01/20 19:32:47 DEBUG orm.ClassWriter: LOAD_JOB_NR
>> 12/01/20 19:32:47 DEBUG orm.ClassWriter: REC_ST_NR
>> 12/01/20 19:32:47 DEBUG orm.ClassWriter: HPQ_RATING_CD
>> 12/01/20 19:32:47 DEBUG orm.ClassWriter: SRC_SYS_UPD_TS
>> 12/01/20 19:32:47 DEBUG orm.ClassWriter: Writing source file:
>> /tmp/sqoop-narasimharao/compile/eb164bb8e2c0031416457bda6b86dd19/EDW_INT_OPPTY_ALNC_PTNR_F.java
>> 12/01/20 19:32:47 DEBUG orm.ClassWriter: Table name:
>> EDW_INT.OPPTY_ALNC_PTNR_F
>> 12/01/20 19:32:47 DEBUG orm.ClassWriter: Columns: OPPTY_ALNC_PTNR_ID:1,
>> SRC_SYS_KY:-5, PTNR_SRC_SYS_KY:-5, OPPTY_ID:1, PTNR_ID:1, PTNR_ROLE_CD:1,
>> PRIM_PTNR_FG:1, INS_GMT_TS:93, UPD_GMT_TS:93, LOAD_JOB_NR:2, REC_ST_NR:5,
>> HPQ_RATING_CD:1, SRC_SYS_UPD_TS:93,
>> 12/01/20 19:32:47 DEBUG orm.ClassWriter: sourceFilename is
>> EDW_INT_OPPTY_ALNC_PTNR_F.java
>> 12/01/20 19:32:47 DEBUG orm.CompilationManager: Found existing
>> /tmp/sqoop-narasimharao/compile/eb164bb8e2c0031416457bda6b86dd19/
>> 12/01/20 19:32:47 INFO orm.CompilationManager: HADOOP_HOME is
>> /usr/lib/hadoop
>> 12/01/20 19:32:47 INFO orm.CompilationManager: Found hadoop core jar at:
>> /usr/lib/hadoop/hadoop-core.jar
>> 12/01/20 19:32:48 DEBUG orm.CompilationManager: Adding source file:
>> /tmp/sqoop-narasimharao/compile/eb164bb8e2c0031416457bda6b86dd19/EDW_INT_OPPTY_ALNC_PTNR_F.java
>> 12/01/20 19:32:48 DEBUG orm.CompilationManager: Invoking javac with args:
>> 12/01/20 19:32:48 DEBUG orm.CompilationManager: -sourcepath
>> 12/01/20 19:32:48 DEBUG orm.CompilationManager:
