| clear query|facets|time |
Search criteria: .
Results from 1 to 10 from
22 (0.123s).
|
|
|
Loading phrases to help you
refine your search...
|
|
Re: partition as block? - HDFS - [mail # user]
|
|
...Yes it is a problem at the first stage. What I'm wondering, though, is wether the intermediate results - which happen after the mapper phase - can be optimized. On Tue, Apr 30, ...
|
|
|
Author: Jay Vyas,
2013-04-30, 19:59
|
|
|
Re: partition as block? - HDFS - [mail # user]
|
|
...Well, to be more clear, I'm wondering how hadoop-mapreduce can be optimized in a block-less filesystem... And am thinking about application tier ways to simulate blocks - i.e. by making the ...
|
|
|
Author: Jay Vyas,
2013-04-30, 19:04
|
|
|
transfer -> CreateSocketForPipeline : hardcoded length of 2? - HDFS - [mail # dev]
|
|
...Hi guys: I noticed that in the call to createSocketForPipeline, there is a hardcoded length of "2". //from sock = createSocketForPipeline(src, 2, dfsClient); This cascades down t...
|
|
|
Author: Jay Vyas,
2013-04-23, 13:25
|
|
|
Re: Append MR output file to an exitsted HDFS file - HDFS - [mail # user]
|
|
...I might be misunderstanding, but if you want each Reducer to append its outputs to outputs to corresponding files that already exist in HDFS? Remember that the reducers usually are out...
|
|
|
Author: Jay Vyas,
2013-04-22, 04:04
|
|
|
Re: Writing intermediate key,value pairs to file and read it again - HDFS - [mail # user]
|
|
...How many intermediate keys? If small enough, you can keep them in memory. If large, you can just wait for the job to finish and siphon them into your job as input with the MultipleInpu...
|
|
|
Author: Jay Vyas,
2013-04-20, 16:06
|
|
|
Re: Convention question for using DFSConfigKey constants : are zeros magic? - HDFS - [mail # dev]
|
|
...Thanks ! okay that helps to clarify things. Okay, so the value in referencing the static comment is that it is commented in the DFS_KEYS file, and declared as 0. Having the ==0 i...
|
|
|
Author: Jay Vyas,
2013-04-19, 22:40
|
|
|
Convention question for using DFSConfigKey constants : are zeros magic? - HDFS - [mail # dev]
|
|
...I recently looked into the HDFS source tree to determine idioms with respect to a hairy debate about the threshold between what is and is not a magic number, and found that : And it ap...
|
|
|
Author: Jay Vyas,
2013-04-19, 16:10
|
|
|
MapReduce Integration tests for test FileSystems - HDFS - [mail # dev]
|
|
...MapReduce has some very demanding file operations at at job submission time, and is an important integration test for any hadoop compliant FileSystem. How in the hadoop HDFS source tre...
|
|
|
Author: Jay Vyas,
2013-04-13, 20:46
|
|
|
Re: testHDFSConf.xml - HDFS - [mail # dev]
|
|
...Thats great. Yes this is certainly something we could benefit from. But the "trunk" github appears to only have "./src/test/hdfs/org/apache/hadoop/cli/testHDFSConf.xml" in the source t...
|
|
|
Author: Jay Vyas,
2013-04-11, 13:14
|
|
|
testHDFSConf.xml - HDFS - [mail # dev]
|
|
...Hello HDFS brethren ! I've noticed that the testHDFSConf.xml has alot of references to supergroup. https://svn.apache.org/repos/asf/hadoop/common/trunk/hadoop-hdfs-project/hadoop...
|
|
|
Author: Jay Vyas,
2013-04-10, 17:16
|
|
|
|
