Hey guys,

I'm running a S3->EMR job that needs to save some temp files in a
local dir. Unfortunately, I'm getting this message:

java.lang.IllegalArgumentException: This file system object
(hdfs://domU-12-31-38-00-08-A1.compute-1.internal:9000) does not
support access to the request path
's3n://dts-attensity-data/hdfs/attensity-whirlpool-gz/indexes/facts/2010-07-14-08-01-09-627/part-00004'
You possibly called FileSystem.get(conf) when you should of called
FileSystem.get(uri, conf) to obtain a file system supporting your
path.

This line creates the offending path:   final Path temp = new
Path(job.getWorkingDirectory().toString() +"index/_" +
Integer.toString(random.nextInt()));
What should I have instead? I don't want my tmp files on S3 :)

-B

--
Bradford Stephens,
Founder, Drawn to Scale
drawntoscalehq.com
727.697.7528

http://www.drawntoscalehq.com --  The intuitive, cloud-scale data
solution. Process, store, query, search, and serve all your data.

http://www.roadtofailure.com -- The Fringes of Scalability, Social
Media, and Computer Science

As the error message suggested, call LocalFileSystem.get()

Regards

On Wed, Jul 14, 2010 at 8:21 AM, Bradford Stephens <
[EMAIL PROTECTED]> wrote:

> Hey guys,
>
> I'm running a S3->EMR job that needs to save some temp files in a
> local dir. Unfortunately, I'm getting this message:
>
> java.lang.IllegalArgumentException: This file system object
> (hdfs://domU-12-31-38-00-08-A1.compute-1.internal:9000) does not
> support access to the request path
>
> 's3n://dts-attensity-data/hdfs/attensity-whirlpool-gz/indexes/facts/2010-07-14-08-01-09-627/part-00004'
> You possibly called FileSystem.get(conf) when you should of called
> FileSystem.get(uri, conf) to obtain a file system supporting your
> path.
>
> This line creates the offending path:   final Path temp = new
> Path(job.getWorkingDirectory().toString() +"index/_" +
> Integer.toString(random.nextInt()));
>
>
> What should I have instead? I don't want my tmp files on S3 :)
>
> -B
>
>
>
> --
> Bradford Stephens,
> Founder, Drawn to Scale
> drawntoscalehq.com
> 727.697.7528
>
> http://www.drawntoscalehq.com --  The intuitive, cloud-scale data
> solution. Process, store, query, search, and serve all your data.
>
> http://www.roadtofailure.com -- The Fringes of Scalability, Social
> Media, and Computer Science
>