Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # user >> Re: Gridmix version 1.0.4 Error


Copy link to this message
-
Re: Gridmix version 1.0.4 Error
@Harsh J

I was following an explanation for Gridmix and it runs it using the java command but here I have ran it as hadoop jar with the jar file that is located in the contrib/gridmix directory that is in 1.0.4

as you can see before I run Gridmix the bench folder in hdfs is non existent but I keep getting the FileAlreadyExistsException when I run it

SeanBarry-Macbook-Pro:gridmix seanbarry$ hadoop fs -ls /user/seanbarry
Found 5 items
drwxrwxrwx   - seanbarry staff          0 2013-01-03 15:16 /user/seanbarry/Rumen
drwxrwxrwx   - seanbarry staff          0 2013-01-03 15:37 /user/seanbarry/gutenberg
drwxrwxrwx   - seanbarry staff          0 2013-01-03 12:05 /user/seanbarry/gutenberg-output
drwxrwxrwx   - seanbarry staff          0 2013-01-03 12:06 /user/seanbarry/gutenberg-output-1
drwxrwxrwx   - seanbarry staff          0 2013-01-03 12:07 /user/seanbarry/gutenberg-output-2

SeanBarry-Macbook-Pro:gridmix seanbarry$ hadoop jar hadoop-gridmix-1.0.4.jar org.apache.hadoop.mapred.gridmix.Gridmix -generate 10m hdfs://hostname:54310/user/seanbarry/bench file:///Users/seanbarry/sfbarry/job-trace.json
13/01/07 10:12:11 INFO gridmix.SubmitterUserResolver:  Current user resolver is SubmitterUserResolver
13/01/07 10:12:11 WARN gridmix.Gridmix: Resource null ignored
13/01/07 10:12:11 INFO gridmix.Gridmix:  Submission policy is STRESS
13/01/07 10:12:11 INFO gridmix.Gridmix: Generating 10.0m of test data...
13/01/07 10:12:11 INFO mapred.JobClient: Cleaning up the staging area hdfs://hostname:54310/app/hadoop/tmp/mapred/staging/seanbarry/.staging/job_201301070957_0003
13/01/07 10:12:11 ERROR security.UserGroupInformation: PriviledgedActionException as:seanbarry cause:org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory hdfs://hostname:54310/user/seanbarry/bench already exists
13/01/07 10:12:11 ERROR security.UserGroupInformation: PriviledgedActionException as:seanbarry cause:org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory hdfs://hostname:54310/user/seanbarry/bench already exists
13/01/07 10:12:11 WARN gridmix.JobSubmitter: Failed to submit GRIDMIX_GENDATA as seanbarry
org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory hdfs://hostname:54310/user/seanbarry/bench already exists
at org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.checkOutputSpecs(FileOutputFormat.java:137)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:887)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:850)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:850)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:500)
at org.apache.hadoop.mapred.gridmix.GenerateData$1.run(GenerateData.java:116)
at org.apache.hadoop.mapred.gridmix.GenerateData$1.run(GenerateData.java:101)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
at org.apache.hadoop.mapred.gridmix.GenerateData.call(GenerateData.java:101)
at org.apache.hadoop.mapred.gridmix.GenerateData.call(GenerateData.java:57)
at org.apache.hadoop.mapred.gridmix.JobSubmitter$SubmitTask.run(JobSubmitter.java:106)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:680)
13/01/07 10:12:11 INFO gridmix.JobMonitor:  Job submission failed notify if anyone is waiting org.apache.hadoop.mapreduce.Job@1b980630
13/01/07 10:12:21 INFO mapred.JobClient: Cleaning up the staging area hdfs://hostname:54310/app/hadoop/tmp/mapred/staging/seanbarry/.staging/job_201301070957_0004
13/01/07 10:12:21 ERROR gridmix.Gridmix: Startup failed
java.lang.IllegalArgumentException: Wrong FS: hdfs://hostname:54310/app/hadoop/tmp/mapred/staging/seanbarry/.staging/job_201301070957_0003/job.jar, expected: file:///
at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:381)
at org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:55)
at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:393)
at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:251)
at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:213)
at org.apache.hadoop.fs.FileSystem.copyFromLocalFile(FileSystem.java:1195)
at org.apache.hadoop.fs.FileSystem.copyFromLocalFile(FileSystem.java:1171)
at org.apache.hadoop.fs.FileSystem.copyFromLocalFile(FileSystem.java:1143)
at org.apache.hadoop.mapred.JobClient.copyAndConfigureFiles(JobClient.java:779)
at org.apache.hadoop.mapred.JobClient.copyAndConfigureFiles(JobClient.java:655)
at org.apache.hadoop.mapred.JobClient.access$300(JobClient.java:174)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:865)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:850)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:850)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:500)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:530)
at org.apache.hadoop.mapred.gridmix.Gridmix.writeInputData(Gridmix.java:118)
at org.apache.hadoop.mapred.gridmix.Gridmix.start(Gridmix.java:283)
at org.apache.hadoop.mapred.gridmix.Gridmix.runJob(Gridmix.java:263)
at org.apache.hadoop.mapred.gridmix.Gridmix.access$000(Gridmix.java:55)
at org.apache.hadoop.mapred.gridmix.Gridmix$1.run(Gridmix.java:217)
at org.apache.hadoop.mapred.gridmix.Gridmix$1.run(Gridmix.java:215)
at java.security.A