Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> Reading a directory in standalone Hadoop


Copy link to this message
-
Re: Reading a directory in standalone Hadoop
Hi,

The stack trace mentions that it is getting an access denied. Could
you check the permissions of the directory /folder/timezone ? Also,
are you using the local job tracker, and not a cluster ?

In general, please ensure your configuration is pointing to the right
cluster where the job needs to run when launching the mapred job - be
it local, or a cluster set up in pseudo distributed mode or a real
cluster.

Thanks
Hemanth

On Fri, Aug 31, 2012 at 1:41 PM, JOAQUIN GUANTER GONZALBEZ <[EMAIL PROTECTED]> wrote:
> Hello Hadoopers,
>
>
>
> I am trying to write a test that runs some MR jobs. One of these jobs needs
> to read a file produced by another job with a SequenceFile.Reader class. The
> job that produces this file has its output path set to “/folder/timezone”.
> After running in my standalone Hadoop environment, it produces 4 files in
> this folder:
>
>
>
> -          .SUCCESS.crc
>
> -          .part-r-00000.crc
>
> -          _SUCCESS
>
> -          part-r-00000
>
>
>
> So far, so good. The problem is that when the next job tries to read
> “/folder/timezone” with a SequenceFile.Reader class, it gets a
> FileNotFoundException, presumably because it is trying to open the directory
> as a “file” in my local filesystem. Here’s the stacktrace I get:
>
>
>
> java.io.FileNotFoundException: /folder/timezone (Acess denied)
>
>                 at java.io.FileInputStream.open(Native Method)
>
>                 at
> java.io.FileInputStream.<init>(FileInputStream.java:138)
>
>                 at
> org.apache.hadoop.fs.RawLocalFileSystem$TrackingFileInputStream.<init>(RawLocalFileSystem.java:72)
>
>                 at
> org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileInputStream.<init>(RawLocalFileSystem.java:108)
>
>                 at
> org.apache.hadoop.fs.RawLocalFileSystem.open(RawLocalFileSystem.java:178)
>
>                 at
> org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.<init>(ChecksumFileSystem.java:127)
>
>                 at
> org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:284)
>
>                 at
> org.apache.hadoop.io.SequenceFile$Reader.openFile(SequenceFile.java:1508)
>
>                 at
> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1487)
>
>                 at
> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1480)
>
>                 at
> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1475)
>
>                 at
> es.tid.smartsteps.footfalls.lookups.CartesianConverterMapperBase.setup(CartesianConverterMapperBase.java:41)
>
>                 at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:142)
>
>                 at
> org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:647)
>
>                 at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323)
>
>                 at
> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:210)
>
>
>
> Any idea on how to solve this issue? Is this a supported scenario by
> SequenceFile.Reader or am I out of luck?
>
>
>
> Many thanks!
>
> Ximo.
>
>
> ________________________________
> Este mensaje se dirige exclusivamente a su destinatario. Puede consultar
> nuestra política de envío y recepción de correo electrónico en el enlace
> situado más abajo.
> This message is intended exclusively for its addressee. We only send and
> receive email on the basis of the terms set out at:
> http://www.tid.es/ES/PAGINAS/disclaimer.aspx