| clear query|facets|time |
Search criteria: .
Results from 81 to 90 from
275 (0.165s).
|
|
|
Loading phrases to help you
refine your search...
|
|
Re: Which FileInputFormat to use for fixed length records? - Hadoop - [mail # user]
|
|
...I think these would be good to add to mapreduce in the {{org.apache.hadoop.mapreduce.lib.input}} package. Please file a JIRA and apply a patch! - Aaron On Wed, Oct 28, 2009 at 11:15 AM...
|
|
|
Author: Aaron Kimball,
2009-10-28, 19:58
|
|
|
Re: How to give consecutive numbers to output records? - Hadoop - [mail # user]
|
|
...There is no in-MapReduce mechanism for cross-task synchronization. You'll need to use something like Zookeeper for this, or another external database. Note that this will greatly complicate ...
|
|
|
Author: Aaron Kimball,
2009-10-28, 04:27
|
|
|
Re: Can I have multiple reducers? - Hadoop - [mail # user]
|
|
...If you need another shuffle after your first reduce pass, then you need a second MapReduce job to run after the first one. Just use an IdentityMapper. This is a reasonably common situa...
|
|
|
Author: Aaron Kimball,
2009-10-23, 02:54
|
|
|
Re: openssh - can't achieve passphraseless ssh - Hadoop - [mail # user]
|
|
...Another sneaky permissions requirement is that ~/.ssh/ itself must be mode 0750 or more strict. - Aaron On Wed, Oct 21, 2009 at 2:47 PM, Edward Capriolo wrote: ...
|
|
|
Author: Aaron Kimball,
2009-10-22, 04:27
|
|
|
Re: streaming data from HDFS outside of hadoop - Hadoop - [mail # user]
|
|
...You shouldn't directly instantiate and intialize FileSystem implementations; there's a factory method you should use. Do instead: private void initHadoop(String ip, int port) thr...
|
|
|
Author: Aaron Kimball,
2009-10-21, 05:02
|
|
|
Re: Which FileInputFormat to use for fixed length records? - Hadoop - [mail # user]
|
|
...You'll need to write your own, I'm afraid. You should subclass FileInputFormat and go from there. You may want to look at TextInputFormat / LineRecordReader for an example of how an IF/RR ge...
|
|
|
Author: Aaron Kimball,
2009-10-21, 05:00
|
|
|
Re: How can I run such a mapreduce program? - Hadoop - [mail # user]
|
|
...If you're working with the Cloudera distribution, you can install CDH1 (0.18.3) and CDH2 (0.20.1) side-by-side on your development machine. They'll install to /usr/lib/hadoop-0.18 and ...
|
|
|
Author: Aaron Kimball,
2009-10-17, 06:55
|
|
|
Re: Error in FileSystem.get() - Hadoop - [mail # user]
|
|
...Bhupesh: If you use FileSystem.newInstance(), does that return the correct object type? This sidesteps CACHE. - A On Thu, Oct 15, 2009 at 3:07 PM, Bhupesh Bansal wrote: ...
|
|
|
Author: Aaron Kimball,
2009-10-15, 22:51
|
|
|
Re: Locality when placing Map tasks - Hadoop - [mail # user]
|
|
...Map tasks are generated based on InputSplits. An InputSplit is a logical description of the work that a task should use. The array of InputSplit objects is created on the client by the Input...
|
|
|
Author: Aaron Kimball,
2009-10-06, 22:20
|
|
|
Re: FileSystem Caching in Hadoop - Hadoop - [mail # user]
|
|
...Edward, Interesting concept. I imagine that implementing "CachedInputFormat" over something like memcached would make for the most straightforward implementation. You could store 64MB ...
|
|
|
Author: Aaron Kimball,
2009-10-06, 22:12
|
|
|
|
