Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Using S3 instead of HDFS


Copy link to this message
-
Re: Using S3 instead of HDFS
Well, here is my error message

Starting Hadoop namenode daemon: starting namenode, logging to
/usr/lib/hadoop-0.20/logs/hadoop-hadoop-namenode-ip-10-126-11-26.out
ERROR. Could not start Hadoop namenode daemon
Starting Hadoop secondarynamenode daemon: starting secondarynamenode,
logging to
/usr/lib/hadoop-0.20/logs/hadoop-hadoop-secondarynamenode-ip-10-126-11-26.out
Exception in thread "main" java.lang.IllegalArgumentException: Invalid URI
for NameNode address (check fs.default.name): s3n://myname.testdata is not
of scheme 'hdfs'.
        at
org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(NameNode.java:224)
        at
org.apache.hadoop.hdfs.server.namenode.NameNode.getServiceAddress(NameNode.java:209)
        at
org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.initialize(SecondaryNameNode.java:182)
        at
org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.<init>(SecondaryNameNode.java:150)
        at
org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.main(SecondaryNameNode.java:624)
ERROR. Could not start Hadoop secondarynamenode daemon

And, if I don't need to start the NameNode, then where do I give the S3
credentials?

Thank you,
Mark
On Wed, Jan 18, 2012 at 12:36 AM, Harsh J <[EMAIL PROTECTED]> wrote:

> Hey Mark,
>
> What is the exact trouble you run into? What do the error messages
> indicate?
>
> This should be definitive enough I think:
> http://wiki.apache.org/hadoop/AmazonS3
>
> On Wed, Jan 18, 2012 at 11:55 AM, Mark Kerzner <[EMAIL PROTECTED]>
> wrote:
> > Hi,
> >
> > whatever I do, I can't make it work, that is, I cannot use
> >
> > s3://host
> >
> > or s3n://host
> >
> > as a replacement for HDFS while runnings EC2 cluster. I change the
> settings
> > in the core-file.xml, in hdfs-site.xml, and start hadoop services, and it
> > fails with error messages.
> >
> > Is there a place where this is clearly described?
> >
> > Thank you so much.
> >
> > Mark
>
>
>
> --
> Harsh J
> Customer Ops. Engineer, Cloudera
>