-Re: copy files from ftp to hdfs in parallel, distcp failed
Ram 2013-07-12, 12:01
Please configure the following in core-ste.xml and try.
Use hadoop fs -ls file:/// -- to display local file system files
Use hadoop fs -ls ftp://<your ftp location> -- to display ftp files if
it is listing files go for distcp.
fs.ftp.host0.0.0.0FTP filesystem connects to this serverfs.ftp.host.port21FTP
filesystem connects to fs.ftp.host on this port
and try to set the property also
reference from hadoop definitive guide hadoop file system.
Filesystem URI scheme Java implementation
(all under org.apache.hadoop)
FTP ftp fs.ftp.FTPFileSystem
A filesystem backed by an FTP server.
On Fri, Jul 12, 2013 at 1:04 PM, Hao Ren <[EMAIL PROTECTED]> wrote:
> Le 11/07/2013 20:47, Balaji Narayanan (பாலாஜி நாராயணன்) a écrit :
>> multiple copy jobs to hdfs
> Thank you for your reply and the link.
> I read the link before, but I didn't find any examples about copying file
> from ftp to hdfs.
> There are about 20-40 file in my directory. I just want to move or copy
> that directory to hdfs on Amazon EC2.
> Actually, I am new to hadoop. I would like to know how to do multiple copy
> jobs to hdfs without distcp.
> Thank you again.
> Hao Ren