Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS >> mail # user >> Reading a directory in standalone Hadoop


+
JOAQUIN GUANTER GONZALBEZ... 2012-08-31, 08:11
Copy link to this message
-
Re: Reading a directory in standalone Hadoop
Hi,

The stack trace mentions that it is getting an access denied. Could
you check the permissions of the directory /folder/timezone ? Also,
are you using the local job tracker, and not a cluster ?

In general, please ensure your configuration is pointing to the right
cluster where the job needs to run when launching the mapred job - be
it local, or a cluster set up in pseudo distributed mode or a real
cluster.

Thanks
Hemanth

On Fri, Aug 31, 2012 at 1:41 PM, JOAQUIN GUANTER GONZALBEZ <[EMAIL PROTECTED]> wrote:
> Hello Hadoopers,
>
>
>
> I am trying to write a test that runs some MR jobs. One of these jobs needs
> to read a file produced by another job with a SequenceFile.Reader class. The
> job that produces this file has its output path set to “/folder/timezone”.
> After running in my standalone Hadoop environment, it produces 4 files in
> this folder:
>
>
>
> -          .SUCCESS.crc
>
> -          .part-r-00000.crc
>
> -          _SUCCESS
>
> -          part-r-00000
>
>
>
> So far, so good. The problem is that when the next job tries to read
> “/folder/timezone” with a SequenceFile.Reader class, it gets a
> FileNotFoundException, presumably because it is trying to open the directory
> as a “file” in my local filesystem. Here’s the stacktrace I get:
>
>
>
> java.io.FileNotFoundException: /folder/timezone (Acess denied)
>
>                 at java.io.FileInputStream.open(Native Method)
>
>                 at
> java.io.FileInputStream.<init>(FileInputStream.java:138)
>
>                 at
> org.apache.hadoop.fs.RawLocalFileSystem$TrackingFileInputStream.<init>(RawLocalFileSystem.java:72)
>
>                 at
> org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileInputStream.<init>(RawLocalFileSystem.java:108)
>
>                 at
> org.apache.hadoop.fs.RawLocalFileSystem.open(RawLocalFileSystem.java:178)
>
>                 at
> org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.<init>(ChecksumFileSystem.java:127)
>
>                 at
> org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:284)
>
>                 at
> org.apache.hadoop.io.SequenceFile$Reader.openFile(SequenceFile.java:1508)
>
>                 at
> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1487)
>
>                 at
> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1480)
>
>                 at
> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1475)
>
>                 at
> es.tid.smartsteps.footfalls.lookups.CartesianConverterMapperBase.setup(CartesianConverterMapperBase.java:41)
>
>                 at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:142)
>
>                 at
> org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:647)
>
>                 at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323)
>
>                 at
> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:210)
>
>
>
> Any idea on how to solve this issue? Is this a supported scenario by
> SequenceFile.Reader or am I out of luck?
>
>
>
> Many thanks!
>
> Ximo.
>
>
> ________________________________
> Este mensaje se dirige exclusivamente a su destinatario. Puede consultar
> nuestra política de envío y recepción de correo electrónico en el enlace
> situado más abajo.
> This message is intended exclusively for its addressee. We only send and
> receive email on the basis of the terms set out at:
> http://www.tid.es/ES/PAGINAS/disclaimer.aspx
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB