Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce, mail # user - copy files from hdfs to local fs


Copy link to this message
-
RE: copy files from hdfs to local fs
java8964 java8964 2013-08-29, 23:48
What's wrong by using old Unix pipe?
hadoop fs -cat /user/input/foo.txt | head -100 > local_file

Date: Thu, 29 Aug 2013 13:50:37 -0700
Subject: Re: copy files from hdfs to local fs
From: [EMAIL PROTECTED]
To: [EMAIL PROTECTED]

tail will work as well.. ??? but i want to extract just (say) n lines out of this file?

On Thu, Aug 29, 2013 at 1:43 PM, Kim Chew <[EMAIL PROTECTED]> wrote:

hadoop fs -copyToLocal
or
hadoop fs -get

It copies the whole file and won't be able just to copy part of the file, what is interesting is there is a "tail" command but no "head".

Kim
On Thu, Aug 29, 2013 at 1:35 PM, Chengi Liu <[EMAIL PROTECTED]> wrote:
Ok,
  A very stupid question...
I have a large file in
    /user/input/foo.txt
I want to copy first 100 lines from this location to local filesystem...

And the data is very sensitive so i am bit hesistant to experiment.
What is the right way to copy sample data from hdfs to local fs.