Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop >> mail # user >> Sqoop Writes Inaccurate Records in DB

Copy link to this message
Re: Sqoop Writes Inaccurate Records in DB
Hi Adarsh, can you re-run with the --verbose option enabled? Also,
please paste in the entire Sqoop command used.

Thanks, Kathleen

On Thu, Sep 13, 2012 at 7:53 AM, Adarsh Sharma <[EMAIL PROTECTED]> wrote:
> Hi all,
> I am using sqoop-1.4.2 with cloudera hadoop and doing some tesing. We need
> to export some tables from CSV's in HDFS.
> As sqoop provides a mechanism of staging tables to write data in main tables
> only if all maps are succeeded.
> While executing a sqoop job on hadoop , suppose a map fails & hadoop
> reattempt the map to re-run and finish after 3 attempts, it results in
> duplicate records in staging table and the job finished but data inserted is
> higher than in CSV's. Below is the output :
> 12/09/13 14:46:55 INFO mapreduce.ExportJobBase: Exported 4071315 records.
> 12/09/13 14:46:55 INFO mapreduce.ExportJobBase: Starting to migrate data
> from staging table to destination.
> 12/09/13 14:47:29 INFO manager.SqlManager: Migrated 5391315 records from
> table1_tmp to table
> Is this is a bug in Sqoop and is there any fix or patch for it. Please let
> me know.
> Thanks