| clear query|facets|time |
Search criteria: .
Results from 171 to 180 from
321 (0.086s).
|
|
|
Loading phrases to help you
refine your search...
|
|
Re: What is the fastest way to get a large amount of data into the Hadoop HDFS file system (or Hbase)? - Hadoop - [mail # general]
|
|
...The problem there is that HDFS isn't a first class file system. That means that the nice and easy ways of mounting will lead to problems (notably NFS which maintains no state will requ...
|
|
|
Author: Ted Dunning,
2010-12-29, 21:19
|
|
|
Re: Hadoop RPC call response post processing - Hadoop - [mail # user]
|
|
...Nah... this is hadoop RPC On Tue, Dec 28, 2010 at 6:42 PM, Lance Norskog wrote: ...
|
|
|
Author: Ted Dunning,
2010-12-29, 06:48
|
|
|
Re: help for using mapreduce to run different code? - Hadoop - [mail # user]
|
|
...Also take a look at the optimizations as done in this paper: http://www.deepdyve.com/lp/association-for-computing-machinery/flumejava-easy-efficient-data-parallel-pipelines-wwPgFt2hWB ...
|
|
|
Author: Ted Dunning,
2010-12-29, 06:47
|
|
|
Re: how to run jobs every 30 minutes? - Hadoop - [mail # user]
|
|
...Good quote. On Tue, Dec 28, 2010 at 3:46 PM, Chris K Wensel wrote: ...
|
|
|
Author: Ted Dunning,
2010-12-28, 23:50
|
|
|
Re: help for using mapreduce to run different code? - Hadoop - [mail # user]
|
|
...if you mean running different code in different mappers, I recommend using an if statement. On Tue, Dec 28, 2010 at 2:53 PM, Jander g wrote: ...
|
|
|
Author: Ted Dunning,
2010-12-28, 23:08
|
|
|
Re: What is the fastest way to get a large amount of data into the Hadoop HDFS file system (or Hbase)? - Hadoop - [mail # general]
|
|
...if the data is coming off of a single machine then simply running multiple threads on that machine spraying the data into the cluster is likely to be faster than a map-reduce program. ...
|
|
|
Author: Ted Dunning,
2010-12-28, 23:07
|
|
|
Re: Hadoop RPC call response post processing - Hadoop - [mail # user]
|
|
...Knowing the tenuring distribution will tell a lot about that exact issue. Ephemeral collections take on average less than one instruction per allocation and the allocation itself is ge...
|
|
|
Author: Ted Dunning,
2010-12-28, 23:01
|
|
|
Re: Hadoop RPC call response post processing - Hadoop - [mail # user]
|
|
...I would be very surprised if allocation itself is the problem as opposed to good old fashioned excess copying. It is very hard to write an allocator faster than the java generational g...
|
|
|
Author: Ted Dunning,
2010-12-28, 07:21
|
|
|
Re: How to simulate network delay on 1 node - Hadoop - [mail # user]
|
|
...See also https://github.com/toddlipcon/gremlins On Sun, Dec 26, 2010 at 11:26 AM, Konstantin Boudnik wrote: ...
|
|
|
Author: Ted Dunning,
2010-12-27, 05:04
|
|
|
Re: Hadoop/Elastic MR on AWS - Hadoop - [mail # user]
|
|
...EMR instances are started near each other. This increases the bandwidth between nodes. There may also be some enhancements in terms of access to the SAN that supports EBS. ...
|
|
|
Author: Ted Dunning,
2010-12-24, 16:48
|
|
|
|
