Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop >> mail # user >> Import data to HDFS using Sqoop2


Copy link to this message
-
Re: Import data to HDFS using Sqoop2
Hi Mengwei

After seeing my table schema, do you have any advanced idea about this
problem?
On Fri, Sep 6, 2013 at 9:02 AM, Yanting Chen <[EMAIL PROTECTED]> wrote:

> TABLE DS_MSG_LOG
>     (
>         MESSAGE_ID VARCHAR2(23) NOT NULL,
>         LOGIN_ID VARCHAR2(30),
>         ACPT_DTS TIMESTAMP(6),
>         DLVR_DTS TIMESTAMP(6),
>         SENDER_ID VARCHAR2(30),
>         SENDER_VAC_ID VARCHAR2(39),
>         RECEIVER_ID VARCHAR2(30),
>         RECEIVER_VAC_ID VARCHAR2(39),
>         STATUS VARCHAR2(1),
>         MESSAGE_TYPE VARCHAR2(8),
>         FLOW_TYPE VARCHAR2(5),
>         SERVICE_TYPE VARCHAR2(1),
>         SOURCE_FILE_NAME VARCHAR2(150),
>         ARCHIVE_FILE_NAME VARCHAR2(250),
>         ARCHIVE_CHAR_COUNT NUMBER,
>         DECRYPT_FILE_NAME VARCHAR2(250),
>         DECRYPT_CHAR_COUNT NUMBER,
>         RESP_FILE_NAME VARCHAR2(250),
>         RESP_CHAR_COUNT NUMBER,
>         RESP_FLAG VARCHAR2(1),
>         RTG_SEQ VARCHAR2(8),
>         RESENT_FLAG VARCHAR2(1) DEFAULT 'N',
>         TOTAL_INV_COUNT NUMBER,
>         CONSTRAINT PK_DS_MSG_LOG PRIMARY KEY (MESSAGE_ID)
>     )
>
>
> On Fri, Sep 6, 2013 at 1:55 AM, Abraham Elmahrek <[EMAIL PROTECTED]> wrote:
>
>> Could you provide your schema from PostGreSQL? Mengwei is likely right.
>>
>>
>> On Wed, Sep 4, 2013 at 7:36 PM, Yanting Chen <[EMAIL PROTECTED]>wrote:
>>
>>> Actually the schema comes from Oracle. However, I try to modify it to
>>> let it fit into PostgreSQL database. So, now the database I use is
>>> PostgreSQL.
>>>
>>>
>>> On Thu, Sep 5, 2013 at 10:33 AM, Abraham Elmahrek <[EMAIL PROTECTED]>wrote:
>>>
>>>> Yanting,
>>>>
>>>> Also, it seems like the schema you've provided is for an Oracle
>>>> database. i.e. VARCHAR2 and NUMBER are datatypes specific to Oracle. Could
>>>> you please use an oracle connection string and driver? i.e. oracle.jdbc.driver.OracleDriver
>>>> and jdbc:oracle:thin:@host:port:SID.
>>>>
>>>> -abe
>>>>
>>>>
>>>> On Wed, Sep 4, 2013 at 7:32 PM, Mengwei Ding <[EMAIL PROTECTED]>wrote:
>>>>
>>>>> Hmm... would you mind showing us your most updated job configuration
>>>>> by typing "show job --jid 3"? I just want to make sure that you provide the
>>>>> partition column correctly.
>>>>>
>>>>> Also, I notice that the primary key for this table is "VARCHAR(23)"
>>>>> type, this might be the problem.
>>>>>
>>>>> Mengwei
>>>>>
>>>>>
>>>>> On Wed, Sep 4, 2013 at 10:23 PM, Yanting Chen <[EMAIL PROTECTED]>wrote:
>>>>>
>>>>>> Hi Mengwei
>>>>>>
>>>>>> I try to set primary key into partition column. But still the same
>>>>>> error!
>>>>>>
>>>>>>
>>>>>> On Thu, Sep 5, 2013 at 10:17 AM, Mengwei Ding <[EMAIL PROTECTED]
>>>>>> > wrote:
>>>>>>
>>>>>>> Hi Yanting,
>>>>>>>
>>>>>>> It seems like you did not specify the 'partition column' for the
>>>>>>> job. Generally, the primary key of  the table is a good choice for
>>>>>>> 'parition column'.
>>>>>>>
>>>>>>> You could use 'update job -jid 3' to update the job configuration.
>>>>>>>
>>>>>>> Mengwei
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Sep 4, 2013 at 9:37 PM, Yanting Chen <[EMAIL PROTECTED]
>>>>>>> > wrote:
>>>>>>>
>>>>>>>> Abraham,
>>>>>>>>
>>>>>>>> Thanks for you answer. I review my database.
>>>>>>>> I think the database name is invoice and the schema name is public,
>>>>>>>> just like the picture below.
>>>>>>>> http://imgur.com/ns0iNLi
>>>>>>>>
>>>>>>>> So, I change the schema name with a new value, "public"
>>>>>>>> Then, I run this job and get a different error.
>>>>>>>>
>>>>>>>> Status: FAILURE_ON_SUBMIT
>>>>>>>> Creation date: 2013-09-05 09:30:44 CST
>>>>>>>> Last update date: 2013-09-05 09:30:44 CST
>>>>>>>> Exception: org.apache.sqoop.common.SqoopException:
>>>>>>>> GENERIC_JDBC_CONNECTOR_0011:The type is not supported - 12
>>>>>>>> Stack trace: org.apache.sqoop.common.SqoopException:
>>>>>>>> GENERIC_JDBC_CONNECTOR_0011:The type is not supported - 12
>>>>>>>> at
>>>>>>>> org.apache.sqoop.connector.jdbc.GenericJdbcImportPartitioner.getPartitions(GenericJdbcImportPartitioner.java:87)