Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> copy files from hdfs to local fs


Copy link to this message
-
RE: copy files from hdfs to local fs
What's wrong by using old Unix pipe?
hadoop fs -cat /user/input/foo.txt | head -100 > local_file

Date: Thu, 29 Aug 2013 13:50:37 -0700
Subject: Re: copy files from hdfs to local fs
From: [EMAIL PROTECTED]
To: [EMAIL PROTECTED]

tail will work as well.. ??? but i want to extract just (say) n lines out of this file?

On Thu, Aug 29, 2013 at 1:43 PM, Kim Chew <[EMAIL PROTECTED]> wrote:

hadoop fs -copyToLocal
or
hadoop fs -get

It copies the whole file and won't be able just to copy part of the file, what is interesting is there is a "tail" command but no "head".

Kim
On Thu, Aug 29, 2013 at 1:35 PM, Chengi Liu <[EMAIL PROTECTED]> wrote:
Ok,
  A very stupid question...
I have a large file in
    /user/input/foo.txt
I want to copy first 100 lines from this location to local filesystem...

And the data is very sensitive so i am bit hesistant to experiment.
What is the right way to copy sample data from hdfs to local fs.
     
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB