Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Sqoop, mail # user - Sqoop not picking up immediate changes


+
burberry blues 2013-11-13, 06:36
Copy link to this message
-
Re: Sqoop not picking up immediate changes
Jarek Jarcec Cecho 2013-11-13, 16:32
Hi Blues,
would you mind sharing details about your use case? Table schemas, exact commands (both on database and in command line) and associated logs?

Wild guess - when you are changing the rows in the database, are you committing the ongoing transaction? Sqoop will create a new connection with new transaction, so due to ACID it won't pick up any uncommitted changes.

Jarcec

On Tue, Nov 12, 2013 at 10:36:10PM -0800, burberry blues wrote:
> Hi Team,
>
> I am having a problem with following scenario.
>
> In Db i update a column1 of a row and the column 2 got modified with
> current timestamp.
> But when i try to import those changes through sqoop using --incremental
> lastmodified --check-column column2 --last-value <less than current
> date>,it shows 0 records imported which are changed.
>
> There are changes in the DB but sqoop qorks as if it couldnt find the
> updated once and still pointing to the old records.
>
> i.e Before updating i have 3 records with date as 10th Nov,i asked sqoop to
> import records after 9th Nov. It imports all 3 records.
> Now i change 1 row and date is updated to 12 Nov. Immediate I ask sqoop to
> import records after 11th Nov .But it imports 0 records now.If i run the
> same import with date as 9th nov again it works fine and also* give me
> duplicate records*.
>
> Please help me in this issue at the earliest.
>
> Thanks,
> Blues
+
burberry blues 2013-11-13, 17:02
+
Anas Mosaad 2013-11-14, 09:13
+
Jarek Jarcec Cecho 2013-11-14, 16:37
+
burberry blues 2013-11-15, 05:37
+
Jarek Jarcec Cecho 2013-11-15, 18:37