Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> Unable to upload to S3


+
Mohit Anchlia 2013-03-04, 23:32
+
Aniket Mokashi 2013-03-05, 00:07
Copy link to this message
-
Re: Unable to upload to S3
That looks like an awsSecretAccessKey. If that's what it is, you probably want to change it now, because you definitely don't want everyone on this list knowing your secret key. You also definitely do not want to specify your secret key as part of your bucket name. That stuff should be in your pig.properties file specified as "fs.s3.awsSecretAccessKey=some_key." There's more info in this thread.

Eli

On Mar 4, 2013, at 4:07 PM, Aniket Mokashi wrote:

> What's BBBBBCCKIAJV5KGMZVA:KKKKxmw5F7I4AWd6rDRA@?
>
> To work with S3-
> 1. Your path should be - s3n://bucket-name/key
> 2. Have your aws keys in core-site.xml
>
>
>
> On Mon, Mar 4, 2013 at 3:32 PM, Mohit Anchlia <[EMAIL PROTECTED]>wrote:
>
>> I am trying to upload to S3 using pig but I get:
>>
>> grunt> store A into 's3://BBBBBCCKIAJV5KGMZVA:KKKKxmw5F7I4AWd6rDRA@
>> /bucket/1/2/a';
>> 2013-03-04 18:24:39,475 [main] INFO
>> org.apache.pig.tools.pigstats.ScriptState - Pig features used in the
>> script: UNKNOWN
>> 2013-03-04 18:24:39,528 [main] ERROR org.apache.pig.tools.grunt.Grunt -
>> ERROR 1002: Unable to store alias A
>> Details at logfile: /data-ebs/misc/pig/pig_1362439271484.log
>> (java.lang.IllegalArgumentException: Invalid hostname in URI
>> s3://BBBBBCCKIAJV5KGMZVA:KKKKxmw5F7I4AWd6rDRA@/bucket/1/2/a)
>> (,at
>> org.apache.hadoop.fs.s3.S3Credentials.initialize(S3Credentials.java:41))
>> (,at
>>
>> org.apache.hadoop.fs.s3native.NativeS3FileSystem.initialize(NativeS3FileSystem.java:478))
>> (,at
>> org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1453))
>> (,at org.apache.hadoop.fs.FileSystem.access$100(FileSystem.java:69))
>> (,at
>> org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:1487))
>> (,at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1469))
>> (,at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:235))
>> (,at org.apache.hadoop.fs.Path.getFileSystem(Path.java:191))
>> (,at
>>
>> org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.checkOutputSpecs(FileOutputFormat.java:131))
>> (,at
>>
>> org.apache.pig.newplan.logical.rules.InputOutputFileValidator$InputOutputFileVisitor.visit(InputOutputFileValidator.java:80))
>> (,at
>> org.apache.pig.newplan.logical.relational.LOStore.accept(LOStore.java:77))
>> (,at
>>
>> org.apache.pig.newplan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:64))
>>
>
>
>
> --
> "...:::Aniket:::... Quetzalco@tl"

+
Mohit Anchlia 2013-03-05, 17:45
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB