Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig, mail # user - Problems with STORE


Copy link to this message
-
Problems with STORE
Kris Coward 2011-01-31, 18:29

So I have a relation apa which when DUMPed, ends up getting output just
fine, but when I run
STORE apa INTO '/rawfiles/f3453efd460348bbaeee2e9496e25871/1294311600/apa' USING PigStorage(',');
I get the following error:

java.io.IOException: Mkdirs failed to create
file:/rawfiles/f3453efd460348bbaeee2e9496e25871/1294311600/apa/_temporary/_attempt_local_0007_m_000000_0
    at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:367)
    at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:526)
    at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:507)
    at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:414)
    at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:406)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.<init>(PigOutputFormat.java:177)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.getRecordWriter(PigOutputFormat.java:96)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.getRecordWriter(PigOutputFormat.java:80)
    at org.apache.hadoop.mapred.MapTask$DirectMapOutputCollector.<init>(MapTask.java:624)
    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:352)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
    at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:176)
2011-01-31 16:42:31,999 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Submitting job: job_local_0007 to execution engine.
2011-01-31 16:42:32,501 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 0% complete
2011-01-31 16:42:37,021 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 100% complete
2011-01-31 16:42:37,021 [main] ERROR org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 1 map reduce job(s) failed!
2011-01-31 16:42:37,021 [main] ERROR org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Failed to produce result in: "file:/rawfiles/f3453efd460348bbaeee2e9496e25871/1294311600/apa"
2011-01-31 16:42:37,022 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Failed!

I've manually created the directory /rawfiles/f3453efd460348bbaeee2e9496e25871/1294311600/apa
from within grunt to verify that it wasn't a permissions problem (and
then removed apa so that STORE wouldn't fail on account of the directory
already existing), and the error persists.

Any advice on what might be causing this problem?

Thanks,
Kris

--
Kris Coward http://unripe.melon.org/
GPG Fingerprint: 2BF3 957D 310A FEEC 4733  830E 21A4 05C7 1FEB 12B3