Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop, mail # user - Sqoop 2 Import


Copy link to this message
-
Re: Sqoop 2 Import
Jarek Jarcec Cecho 2013-05-03, 21:41
Hi Kyle,
what Hadoop version are you using?

Based on the exception it seems that you're using YARN which is suggesting Hadoop 2. I hit similar NullPointerException during release candidate testing on Hadoop 2.0.0-alpha. There is a known HDFS bug HADOOP-8726 [1] that is causing this. It was fixed in 2.0.2-alpha, so I'm wondering if you're using 2.0.0 or 2.0.1 by any chance?

Jarcec

Links:
1: https://issues.apache.org/jira/browse/HADOOP-8726

On Fri, May 03, 2013 at 12:11:05PM -0700, Kyle B wrote:
> Hello,
>
> I am migrating from Sqoop 1.4.3 to Sqoop 2, and am kind of stumbling my way
> through the intro stuff. I'd like to import data from a MySQL database into
> HDFS, and a simple import seems to be failing in 2, which works fine in 1.
>
> - From (1.4.3) -
> sqoop import --connect jdbc:mysql://127.0.0.1:3306/db --username kyle -P
> --table kyle_table --target-dir /user/kyle/table
>
> - To (1.99.2) -
> sqoop:000> create job --xid 1 --type import
> Creating job for connection with id 1
> Please fill following values to create new job object
> Name: test
>
> Database configuration
>
> Schema name:
> Table name: kyle_table
> Table SQL statement:
> Table column names: *
> Partition column name:
> Boundary query:
>
> Output configuration
>
> Storage type:
>   0 : HDFS
> Choose: 0
> Output format:
>   0 : TEXT_FILE
>   1 : SEQUENCE_FILE
> Choose: 0
> Output directory: /user/kyle/table2
>
> Throttling resources
>
> Extractors:
> Loaders:
> New job was successfully created with validation status FINE  and
> persistent id 12
> sqoop:000> submission start --jid 12
> Submission details
> Job id: 12
> Status: BOOTING
> Creation date: 2013-05-03 11:38:01 MST
> Last update date: 2013-05-03 11:38:01 MST
> External Id: job_1367275490217_0075
>         http://server:8088/proxy/application_1367275490217_0075/
> Progress: Progress is not available
> sqoop:000> submission status --jid 12
> Exception has occurred during processing command
> Server has returned exception: Exception: java.lang.NullPointerException
> Message:
>
>
> --
> On both cases, I see the jobs made their way to the job history, and on the
> first, the output was saved to HDFS. On the second, I just have a blank
> /user/kyle/table2 folder, and the task failed in Hadoop.
>
>
>  - Hadoop Logs -
>  2013-05-03 11:38:10,172 WARN [main] org.apache.hadoop.mapred.YarnChild:
> Exception running child : java.lang.NullPointerException
>  at java.lang.String.<init>(Unknown Source)
>  at
> org.apache.sqoop.job.mr.ConfigurationUtils.loadConfiguration(ConfigurationUtils.java:77)
>  at
> org.apache.sqoop.job.mr.ConfigurationUtils.getConnectorConnection(ConfigurationUtils.java:38)
>  at org.apache.sqoop.job.mr.SqoopMapper.run(SqoopMapper.java:69)
>  at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
>  at org.apache.hadoop.mapred.MapTask.run(MapTask.java:333)
>  at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:157)
>  at java.security.AccessController.doPrivileged(Native Method)
>  at javax.security.auth.Subject.doAs(Unknown Source)
>  at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1367)
>  at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:152)
>
> 2013-05-03 11:38:10,172 ERROR [OutputFormatLoader-consumer]
> org.apache.sqoop.job.mr.SqoopOutputFormatLoadExecutor: Error while loading
> data out of MR job.
> java.lang.NullPointerException
>  at java.lang.String.<init>(Unknown Source)
>  at
> org.apache.sqoop.job.mr.ConfigurationUtils.loadConfiguration(ConfigurationUtils.java:77)
>  at
> org.apache.sqoop.job.mr.ConfigurationUtils.getFrameworkConnection(ConfigurationUtils.java:50)
>  at
> org.apache.sqoop.job.mr.SqoopOutputFormatLoadExecutor$ConsumerThread.run(SqoopOutputFormatLoadExecutor.java:204)
>  at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
>  at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
>  at java.util.concurrent.FutureTask.run(Unknown Source)
>  at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)