-Re: cannot use distcp in some s3 buckets
Tom White 2011-10-13, 21:12
On Thu, Oct 13, 2011 at 2:06 PM, Raimon Bosch <[EMAIL PROTECTED]> wrote:
> By the way,
> The url I'm trying has a '_' in the bucket name. Could be this the problem?
Yes, underscores are not permitted in hostnames.
> 2011/10/13 Raimon Bosch <[EMAIL PROTECTED]>
>> I've been having some problems with one of our s3 buckets. I have asked on
>> amazon support with no luck yet
>> I'm getting this exception only with our oldest s3 bucket with this
>> command: "hadoop distcp s3://<MY_BUCKET_NAME>/logfile-20110815.gz
>> java.lang.IllegalArgumentException: Invalid hostname in URI
>> s3://<MY_BUCKET_NAME>/logfile-20110815.gz /tmp/logfile-20110815.gz
>> at org.apache.hadoop.fs.s3.S3Credentials.initialize(S3Credentials.java:41)
>> As you can see, hadoop is rejecting my url before starting to do the
>> authorization steps. Someone has been in a similar issue? I have already
>> tested the same operation in newer s3 buckets and the command is working
>> Thanks in advance,
>> Raimon Bosch.