Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # user >> distcp error.


+
Tao 2012-08-28, 15:36
Copy link to this message
-
Re: distcp error.
Hi, Tao. This problem is only with 2.0.1 or with the two versions?
Have you tried to use distcp from 1.0.3 to 1.0.3?

El 28/08/2012 11:36, Tao escribi锟斤拷:
>
> Hi, all
>
> I use distcp copying data from hadoop1.0.3 to hadoop 2.0.1.
>
> When the file path(or file name) contain Chinese character, an
> exception will throw. Like below. I need some help about this.
>
> Thanks.
>
> [hdfs@host ~]$ hadoop distcp -i -prbugp -m 14 -overwrite -log
> /tmp/distcp.log hftp://10.xx.xx.aa:50070/tmp/锟斤拷锟斤拷路锟斤拷锟斤拷锟斤拷hdfs:
> //10.xx.xx.bb:54310/tmp/distcp_test14
>
> 12/08/28 23:32:31 INFO tools.DistCp: Input Options:
> DistCpOptions{atomicCommit=false, syncFolder=false,
> deleteMissing=false, ignoreFailures=true, maxMaps=14,
> sslConfigurationFile='null', copyStrategy='uniformsize',
> sourceFileListing=null, sourcePaths=[hftp://10.xx.xx.aa:50070/tmp/锟斤拷锟斤拷
> 路锟斤拷锟斤拷锟斤拷], targetPath=hdfs://10.xx.xx.bb:54310/tmp/distcp_test14}
>
> 12/08/28 23:32:33 INFO tools.DistCp: DistCp job log path: /tmp/distcp.log
>
> 12/08/28 23:32:34 WARN conf.Configuration: io.sort.mb is deprecated.
> Instead, use mapreduce.task.io.sort.mb
>
> 12/08/28 23:32:34 WARN conf.Configuration: io.sort.factor is
> deprecated. Instead, use mapreduce.task.io.sort.factor
>
> 12/08/28 23:32:34 WARN util.NativeCodeLoader: Unable to load
> native-hadoop library for your platform... using builtin-java classes
> where applicable
>
> 12/08/28 23:32:36 INFO mapreduce.JobSubmitter: number of splits:1
>
> 12/08/28 23:32:36 WARN conf.Configuration: mapred.jar is deprecated.
> Instead, use mapreduce.job.jar
>
> 12/08/28 23:32:36 WARN conf.Configuration:
> mapred.map.tasks.speculative.execution is deprecated. Instead, use
> mapreduce.map.speculative
>
> 12/08/28 23:32:36 WARN conf.Configuration: mapred.reduce.tasks is
> deprecated. Instead, use mapreduce.job.reduces
>
> 12/08/28 23:32:36 WARN conf.Configuration:
> mapred.mapoutput.value.class is deprecated. Instead, use
> mapreduce.map.output.value.class
>
> 12/08/28 23:32:36 WARN conf.Configuration: mapreduce.map.class is
> deprecated. Instead, use mapreduce.job.map.class
>
> 12/08/28 23:32:36 WARN conf.Configuration: mapred.job.name is
> deprecated. Instead, use mapreduce.job.name
>
> 12/08/28 23:32:36 WARN conf.Configuration: mapreduce.inputformat.class
> is deprecated. Instead, use mapreduce.job.inputformat.class
>
> 12/08/28 23:32:36 WARN conf.Configuration: mapred.output.dir is
> deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
>
> 12/08/28 23:32:36 WARN conf.Configuration:
> mapreduce.outputformat.class is deprecated. Instead, use
> mapreduce.job.outputformat.class
>
> 12/08/28 23:32:36 WARN conf.Configuration: mapred.map.tasks is
> deprecated. Instead, use mapreduce.job.maps
>
> 12/08/28 23:32:36 WARN conf.Configuration: mapred.mapoutput.key.class
> is deprecated. Instead, use mapreduce.map.output.key.class
>
> 12/08/28 23:32:36 WARN conf.Configuration: mapred.working.dir is
> deprecated. Instead, use mapreduce.job.working.dir
>
> 12/08/28 23:32:37 INFO mapred.ResourceMgrDelegate: Submitted
> application application_1345831938927_0039 to ResourceManager at
> baby20/10.1.1.40:8040
>
> 12/08/28 23:32:37 INFO mapreduce.Job: The url to track the job:
> http://baby20:8088/proxy/application_1345831938927_0039/
>
> 12/08/28 23:32:37 INFO tools.DistCp: DistCp job-id: job_1345831938927_0039
>
> 12/08/28 23:32:37 INFO mapreduce.Job: Running job: job_1345831938927_0039
>
> 12/08/28 23:32:50 INFO mapreduce.Job: Job job_1345831938927_0039
> running in uber mode : false
>
> 12/08/28 23:32:50 INFO mapreduce.Job: map 0% reduce 0%
>
> 12/08/28 23:33:00 INFO mapreduce.Job: map 100% reduce 0%
>
> 12/08/28 23:33:00 INFO mapreduce.Job: Task Id :
> attempt_1345831938927_0039_m_000000_0, Status : FAILED
>
> Error: java.io.IOException: File copy failed: hftp://10.1.1.26:50070
> /tmp/锟斤拷锟斤拷路锟斤拷锟斤拷锟斤拷/part-r-00017 -->
> hdfs://10.1.1.40:54310/tmp/distcp_test14/part-r-00017
>
> at
> org.apache.hadoop.tools.mapred.CopyMapper.copyFileWithRetry(CopyMapper.java:262)
10mo. ANIVERSARIO DE LA CREACION DE LA UNIVERSIDAD DE LAS CIENCIAS INFORMATICAS...
CONECTADOS AL FUTURO, CONECTADOS A LA REVOLUCION

http://www.uci.cu
http://www.facebook.com/universidad.uci
http://www.flickr.com/photos/universidad_uci
+
Tao 2012-08-28, 16:30
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB