Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop >> mail # user >> Import data to HDFS using Sqoop2


Copy link to this message
-
Re: Import data to HDFS using Sqoop2
Yanting,

I'm sorry, I'm a bit confused. The database you are using here is called
"invoice". Do you also have a schema called "invoice"? Try removing the
"Schema name" from your job and see if that works for you.

http://www.postgresql.org/docs/9.1/static/ddl-schemas.html

-Abe
On Tue, Sep 3, 2013 at 6:34 PM, Yanting Chen <[EMAIL PROTECTED]> wrote:

> However, with Sqoop 1.4.4, all schema and table names are lower-case, and
> it is successful to make import.
>
> As a result, I am not sure if case sensitivity will be a problem in Sqoop
> 2.
>
> *This is the connection I created:*
> 1 connection(s) to show:
> Connection with id 1 and name ds_msg_log (Created 9/2/13 10:54 AM, Updated
> 9/3/13 6:35 PM)
> Using Connector id 1
>   Connection configuration
>     JDBC Driver Class: org.postgresql.Driver
>     JDBC Connection String: jdbc:postgresql://MY_IP:MY_PORT/invoice
>     Username: hadoop
>     Password:
>     JDBC Connection Properties:
>   Security related configuration options
>     Max connections: 123
>
> *This is the job I created:*
> Job with id 3 and name ds_msg_log (Created 9/2/13 11:37 AM, Updated 9/3/13
> 5:57 PM)
> Using Connection id 1 and Connector id 1
>   Database configuration
>     Schema name: invoice
>     Table name: ds_msg_log
>     Table SQL statement:
>     Table column names:
>     Partition column name:
>     Boundary query:
>    Output configuration
>     Storage type: HDFS
>     Output format: TEXT_FILE
>     Output directory: /user/root/ds_msg_log_sqoop2
>   Throttling resources
>     Extractors:
>     Loaders:
>
> *The command I use to start this job is:*
> submission start --jid 3
>
>
> On Wed, Sep 4, 2013 at 9:24 AM, Abraham Elmahrek <[EMAIL PROTECTED]> wrote:
>
>> Sorry, I missed that you are using PostGreSQL. I think case sensitivity
>> will be a problem with that database.
>>
>>
>> On Tue, Sep 3, 2013 at 6:23 PM, Abraham Elmahrek <[EMAIL PROTECTED]>wrote:
>>
>>> What database are you using? Are table names or schema names case
>>> sensitive in your database? Sqoop quotes all names, so case sensitivity
>>> makes a difference here.
>>>
>>> Also, could you provide the corresponding Sqoop 1.4.4 command that
>>> worked for you?
>>>
>>> -Abe
>>>
>>>
>>> On Tue, Sep 3, 2013 at 6:06 PM, 陳彥廷(Yanting Chen) <
>>> [EMAIL PROTECTED]> wrote:
>>>
>>>> I am pretty sure that the schema "invoice" exists because I
>>>> successfully imported data to this schema using Sqoop 1.4.4
>>>>
>>>> Also, I already remove "*" when creating job.
>>>>
>>>> The following is the table in invoice schema:
>>>>     TABLE DS_MSG_LOG
>>>>     (
>>>>         MESSAGE_ID VARCHAR2(23) NOT NULL,
>>>>         LOGIN_ID VARCHAR2(30),
>>>>         ACPT_DTS TIMESTAMP(6),
>>>>         DLVR_DTS TIMESTAMP(6),
>>>>         SENDER_ID VARCHAR2(30),
>>>>         SENDER_VAC_ID VARCHAR2(39),
>>>>         RECEIVER_ID VARCHAR2(30),
>>>>         RECEIVER_VAC_ID VARCHAR2(39),
>>>>         STATUS VARCHAR2(1),
>>>>         MESSAGE_TYPE VARCHAR2(8),
>>>>         FLOW_TYPE VARCHAR2(5),
>>>>         SERVICE_TYPE VARCHAR2(1),
>>>>         SOURCE_FILE_NAME VARCHAR2(150),
>>>>         ARCHIVE_FILE_NAME VARCHAR2(250),
>>>>         ARCHIVE_CHAR_COUNT NUMBER,
>>>>         DECRYPT_FILE_NAME VARCHAR2(250),
>>>>         DECRYPT_CHAR_COUNT NUMBER,
>>>>         RESP_FILE_NAME VARCHAR2(250),
>>>>         RESP_CHAR_COUNT NUMBER,
>>>>         RESP_FLAG VARCHAR2(1),
>>>>         RTG_SEQ VARCHAR2(8),
>>>>         RESENT_FLAG VARCHAR2(1) DEFAULT 'N',
>>>>         TOTAL_INV_COUNT NUMBER,
>>>>         CONSTRAINT PK_DS_MSG_LOG PRIMARY KEY (MESSAGE_ID)
>>>>     )
>>>>
>>>>
>>>> On Wed, Sep 4, 2013 at 1:46 AM, Abraham Elmahrek <[EMAIL PROTECTED]>wrote:
>>>>
>>>>> Hey User,
>>>>>
>>>>> It looks like the schema "invoice" does not exist in your database.
>>>>> Could you please provide your database schema? Also, "*" is unnecessary
>>>>> when specifying "Table column names". If you leave it blank it will import
>>>>> all columns by default.
>>>>>
>>>>> -Abe
>>>>>
>>>>>
>>>>> On Tue, Sep 3, 2013 at 3:03 AM, 陳彥廷(Yanting Chen) <
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB