Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Importing log files from various machines


Copy link to this message
-
Re: Importing log files from various machines
Just use the hadoop client tools.  That is the hadoop package and configure
it to point to your running cluster.  You don't need to start any hadoop
processes on the node with your logs.  Just use the command line (hadoop dfs
-put) or (hadoop distcp) to move the files from each application server
directly into your HDFS cluster.

Marc

On Mon, Jun 28, 2010 at 5:10 PM, Blargy <[EMAIL PROTECTED]> wrote:

>
> I am currently looking into importing all of our application log files
> (~100+
> host machines) into HDFS. Can someone point me in the right direction or
> walk me through the process of how I can accomplish this? Any good reading
> material on this subject? Videos?
>
> I hope I don't need to physically copy all of the log files to one target
> machine before importing.
>
> Thanks
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Importing-log-files-from-various-machines-tp929423p929423.html
> Sent from the Hadoop lucene-users mailing list archive at Nabble.com.
>