| clear query|facets|time |
Search criteria: .
Results from 51 to 60 from
112 (0.165s).
|
|
|
Loading phrases to help you
refine your search...
|
|
Re: LineReader, Buffering for FileInputFormat - Hadoop - [mail # user]
|
|
...Thank you. Is 64KB a good choice? From experience, there is a payoff between large chunks and time taken to read the chunk. I wonder if a larger value would be better. On Sun, Aug 9, 2...
|
|
|
Author: Saptarshi Guha,
2009-08-09, 23:43
|
|
|
LineReader, Buffering for FileInputFormat - Hadoop - [mail # user]
|
|
...Hello, I am using the TextInputFormat and its associated LineReader. In the RecordReader for this class, it reads key and value, using LineReader. My question is does LineReader hit the disk...
|
|
|
Author: Saptarshi Guha,
2009-08-09, 22:38
|
|
|
Re: Running 145K maps, zero reduces- does Hadoop scale? - Hadoop - [mail # user]
|
|
...Simulation trials. Let N be the trials and T be the number of Map Tasks (==splits) Also assume there is much variation in the running time per trial. If there are ~ K=N/T (assume K is an int...
|
|
|
Author: Saptarshi Guha,
2009-07-31, 14:17
|
|
|
Re: Running 145K maps, zero reduces- does Hadoop scale? - Hadoop - [mail # user]
|
|
...In this particular example, the record reader emits a single number per split as both key and value. Regards S On Fri, Jul 31, 2009 at 1:55 AM, Saptarshi Guha wrote: ...
|
|
|
Author: Saptarshi Guha,
2009-07-31, 06:37
|
|
|
Re: PCAP file format support - Hadoop - [mail # user]
|
|
...Quite true. In fact there is no such record as number of packets in a PCAP file. One has to get the filesize and divide by cumulative (plus some other things) bytes to find out what % one is...
|
|
|
Author: Saptarshi Guha,
2009-07-31, 06:04
|
|
|
Running 145K maps, zero reduces- does Hadoop scale? - Hadoop - [mail # user]
|
|
...Hello, Does Hadoop scale well for 100K+ input splits? I have not tried with sequence files. My custom inputformat, generates 145K splits. The record reader emits about 15 bytes as key and 8 ...
|
|
|
Author: Saptarshi Guha,
2009-07-31, 05:55
|
|
|
Re: Hadoop in a Heterogeneous Environment - taking advantage of different processor specs - Hadoop - [mail # user]
|
|
...Tsk tsk, silly of me. Of course I could do that. Thanks for the confirmation Regards Saptarshi On Tue, Jul 28, 2009 at 11:25 AM, Harish Mallipeddi wrote: ...
|
|
|
Author: Saptarshi Guha,
2009-07-28, 15:46
|
|
|
Hadoop in a Heterogeneous Environment - taking advantage of different processor specs - Hadoop - [mail # user]
|
|
...Hello, Not sure if this has been asked or answered. Suppose I have tasktrackers A1,A2,A3 each with 4 cores and 16GB ram. mapred.tasktracker.map.tasks.maximum = 6 mapred.tasktracker.red...
|
|
|
Author: Saptarshi Guha,
2009-07-28, 14:40
|
|
|
Mapfileoutput format: reading in the results? - Hadoop - [mail # user]
|
|
...Hello, Not sure if I sent to this to the right email address, so here it goes again. I am using Hadoop 0.19.2 and am experimenting with the MapFileOutputFormat. The job is complete, th...
|
|
|
Author: Saptarshi Guha,
2009-07-02, 22:46
|
|
|
Mapfileoutput format: reading in the results? - Hadoop - [mail # user]
|
|
...Hello, I am using Hadoop 0.19.2 and am experimenting with the MapFileOutputFormat. The job is complete, the output folder has several part-* files though none of them directories (as I thoug...
|
|
|
Author: Saptarshi Guha,
2009-07-02, 20:51
|
|
|
|
