Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Sqoop >> mail # user >> Sqoop Writes Inaccurate Records in DB


+
Adarsh Sharma 2012-09-13, 14:53
Copy link to this message
-
Re: Sqoop Writes Inaccurate Records in DB
Hi Adarsh, can you re-run with the --verbose option enabled? Also,
please paste in the entire Sqoop command used.

Thanks, Kathleen

On Thu, Sep 13, 2012 at 7:53 AM, Adarsh Sharma <[EMAIL PROTECTED]> wrote:
> Hi all,
>
> I am using sqoop-1.4.2 with cloudera hadoop and doing some tesing. We need
> to export some tables from CSV's in HDFS.
> As sqoop provides a mechanism of staging tables to write data in main tables
> only if all maps are succeeded.
>
> While executing a sqoop job on hadoop , suppose a map fails & hadoop
> reattempt the map to re-run and finish after 3 attempts, it results in
> duplicate records in staging table and the job finished but data inserted is
> higher than in CSV's. Below is the output :
>
> 12/09/13 14:46:55 INFO mapreduce.ExportJobBase: Exported 4071315 records.
> 12/09/13 14:46:55 INFO mapreduce.ExportJobBase: Starting to migrate data
> from staging table to destination.
> 12/09/13 14:47:29 INFO manager.SqlManager: Migrated 5391315 records from
> table1_tmp to table
>
> Is this is a bug in Sqoop and is there any fix or patch for it. Please let
> me know.
>
>
> Thanks
+
Adarsh Sharma 2012-09-14, 04:27
+
Jarek Jarcec Cecho 2012-09-14, 07:42
+
Adarsh Sharma 2012-09-16, 16:34