Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> example usage of s3 file system


Copy link to this message
-
Re: example usage of s3 file system
Hi,

Here is an example, might help you.

http://muhammadkhojaye.blogspot.in/2012/04/how-to-run-amazon-elastic-mapreduce-job.html
Cheers!
Manoj.

On Tue, Aug 28, 2012 at 12:55 PM, Chris Collins
<[EMAIL PROTECTED]>wrote:

>
>
>
> Hi I am trying to use the Hadoop filesystem abstraction with S3 but in my
> tinkering I am not having a great deal of success.  I am particularly
> interested in the ability to mimic a directory structure (since s3 native
> doesnt do it).
>
> Can anyone point me to some good example usage of Hadoop FileSystem with
> s3?
>
> I created a few directories using transit and AWS S3 console for test.
>  Doing a liststatus of the bucket returns a FileStatus object of the
> directory created but if I try to do a liststatus of that path I am getting
> a 404:
>
> org.apache.hadoop.fs.s3.S3Exception:
> org.jets3t.service.S3ServiceException: Request Error. HEAD '/aaaa' on Host
> ....
>
> Probably not the best list to look for help, any clues appreciated.
>
> C
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB