Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Sqoop, mail # user - Sqoop import - large volume dataset


+
Erravelli, Venkat 2012-12-11, 14:38
Copy link to this message
-
Re: Sqoop import - large volume dataset
Jarek Jarcec Cecho 2012-12-11, 16:55
Hi Venkat,
you seems to be using CDH version of Sqoop and Cloudera proprietary connector, so I would recommend to send this question to cdh-user [1] mailing list instead. You will likely to get more support there.

I would also recommend attaching Sqoop log file generated with parameter --verbose as it might provide more insight. Feel free to change sensitive values like password.

Jarcec

List:
1: https://groups.google.com/a/cloudera.org/forum/?fromgroups#!forum/cdh-user

On Tue, Dec 11, 2012 at 09:38:18AM -0500, Erravelli, Venkat wrote:
> Hello,
>
> I'm trying to import 40 million rows(wide table. Around 400 columns) from Teradata to HDFS, but the tasks get killed after 10 minutes. Increased the timeout to 30 minutes, still see the same issue.  Sqoop Version :: Sqoop 1.4.1-cdh4.1.0
>
> Last successful import I had on this table was for 20 million rows.
>
> Are there any other areas that needs to be looked into it apart from increasing the timeout value.  Sqoop Import failure doesn't provide enough details.
>
> Thanks!
>
> ----------------------------------------------------------------------
> This message, and any attachments, is for the intended recipient(s) only, may contain information that is privileged, confidential and/or proprietary and subject to important terms and conditions available at http://www.bankofamerica.com/emaildisclaimer.   If you are not the intended recipient, please delete this message.