Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig, mail # user - Unable to upload to S3


Copy link to this message
-
Re: Unable to upload to S3
Mohit Anchlia 2013-03-05, 17:45
Keys that are given here were changed so it's not the real key. I did find
updating the keys in pig.properties file helpful.

On Mon, Mar 4, 2013 at 4:16 PM, Eli Finkelshteyn <[EMAIL PROTECTED]>wrote:

> That looks like an awsSecretAccessKey. If that's what it is, you probably
> want to change it now, because you definitely don't want everyone on this
> list knowing your secret key. You also definitely do not want to specify
> your secret key as part of your bucket name. That stuff should be in your
> pig.properties file specified as "fs.s3.awsSecretAccessKey=some_key."
> There's more info in this thread.
>
> Eli
>
> On Mar 4, 2013, at 4:07 PM, Aniket Mokashi wrote:
>
> > What's BBBBBCCKIAJV5KGMZVA:KKKKxmw5F7I4AWd6rDRA@?
> >
> > To work with S3-
> > 1. Your path should be - s3n://bucket-name/key
> > 2. Have your aws keys in core-site.xml
> >
> >
> >
> > On Mon, Mar 4, 2013 at 3:32 PM, Mohit Anchlia <[EMAIL PROTECTED]
> >wrote:
> >
> >> I am trying to upload to S3 using pig but I get:
> >>
> >> grunt> store A into 's3://BBBBBCCKIAJV5KGMZVA:KKKKxmw5F7I4AWd6rDRA@
> >> /bucket/1/2/a';
> >> 2013-03-04 18:24:39,475 [main] INFO
> >> org.apache.pig.tools.pigstats.ScriptState - Pig features used in the
> >> script: UNKNOWN
> >> 2013-03-04 18:24:39,528 [main] ERROR org.apache.pig.tools.grunt.Grunt -
> >> ERROR 1002: Unable to store alias A
> >> Details at logfile: /data-ebs/misc/pig/pig_1362439271484.log
> >> (java.lang.IllegalArgumentException: Invalid hostname in URI
> >> s3://BBBBBCCKIAJV5KGMZVA:KKKKxmw5F7I4AWd6rDRA@/bucket/1/2/a)
> >> (,at
> >> org.apache.hadoop.fs.s3.S3Credentials.initialize(S3Credentials.java:41))
> >> (,at
> >>
> >>
> org.apache.hadoop.fs.s3native.NativeS3FileSystem.initialize(NativeS3FileSystem.java:478))
> >> (,at
> >> org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1453))
> >> (,at org.apache.hadoop.fs.FileSystem.access$100(FileSystem.java:69))
> >> (,at
> >> org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:1487))
> >> (,at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1469))
> >> (,at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:235))
> >> (,at org.apache.hadoop.fs.Path.getFileSystem(Path.java:191))
> >> (,at
> >>
> >>
> org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.checkOutputSpecs(FileOutputFormat.java:131))
> >> (,at
> >>
> >>
> org.apache.pig.newplan.logical.rules.InputOutputFileValidator$InputOutputFileVisitor.visit(InputOutputFileValidator.java:80))
> >> (,at
> >>
> org.apache.pig.newplan.logical.relational.LOStore.accept(LOStore.java:77))
> >> (,at
> >>
> >>
> org.apache.pig.newplan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:64))
> >>
> >
> >
> >
> > --
> > "...:::Aniket:::... Quetzalco@tl"
>
>