Sqoop, mail # user - Issue with sqoop merge command - 2014-08-04, 16:52
 Search Hadoop and all its subprojects:

Switch to Threaded View
Copy link to this message
-
Issue with sqoop merge command
Hi,

I am getting a weird error this morning! When running sqoop merge command
Deleted s3://xxx-xx-xxx/data/xxxx/xxxx/xxxx/merged

merge folder exists so deleted 0

Next,

Merge command fails with the error

14/08/04 11:39:24 ERROR security.UserGroupInformation: PriviledgedActionException as:hadoop (auth:SIMPLE) cause:org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory s3://xx-xx-xx/xx/xxxx/xxx/xxx/merged already exists

14/08/04 11:39:24 ERROR tool.MergeTool: Encountered IOException running import job: org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory s3://xx-xx-xx/xx/xxxx/xxxx/xxxx/merged already exists

               at org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.checkOutputSpecs(FileOutputFormat.java:146)

               at org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:456)

               at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:342)

               at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1268)

               at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1265)

               at java.security.AccessController.doPrivileged(Native Method)

               at javax.security.auth.Subject.doAs(Subject.java:415)

               at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)

               at org.apache.hadoop.mapreduce.Job.submit(Job.java:1265)

               at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1286)

               at org.apache.sqoop.mapreduce.JobBase.runJob(JobBase.java:312)

               at org.apache.sqoop.mapreduce.MergeJob.runMergeJob(MergeJob.java:138)

               at org.apache.sqoop.tool.MergeTool.run(MergeTool.java:57)

               at org.apache.sqoop.Sqoop.run(Sqoop.java:145)

               at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)

               at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)

               at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)

               at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)

               at org.apache.sqoop.Sqoop.main(Sqoop.java:238)

11:39:24: sqoopMergeStatus 1

11:39:24: 0

11:39:24: sqoop Merge Failed  for XXXX

Any idea, how to fix it?

________________________________

This message is for the designated recipient only and may contain privileged, proprietary, or otherwise confidential information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the e-mail by you is prohibited. Where allowed by local law, electronic communications with Accenture and its affiliates, including e-mail and instant messaging (including content), may be scanned by our systems for the purposes of information security and assessment of internal compliance with Accenture policy.
______________________________________________________________________________________

www.accenture.com

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB