| clear query|facets|time |
Search criteria: .
Results from 1 to 10 from
29 (0.103s).
|
|
|
Loading phrases to help you
refine your search...
|
|
Re: fundamental doubt - Hadoop - [mail # user]
|
|
...Hi Jamal It is performed at a frame work level map emits key value pairs and the framework collects and groups all the values corresponding to a key from all the map tasks. Now the red...
|
|
|
Author: Bejoy KS,
2012-11-21, 20:03
|
|
|
Re: guessing number of reducers. - Hadoop - [mail # user]
|
|
...Hi Manoj If you intend to calculate the number of reducers based on the input size, then in your driver class you should get the size of the input dir in hdfs and say you intende...
|
|
|
Author: Bejoy KS,
2012-11-21, 18:34
|
|
|
Re: guessing number of reducers. - Hadoop - [mail # user]
|
|
...Hi Andy It is usually so because if you have more reduce tasks than the reduce slots in your cluster then a few of the reduce tasks will be in queue waiting for its turn. So it is bett...
|
|
|
Author: Bejoy KS,
2012-11-21, 18:21
|
|
|
Re: guessing number of reducers. - Hadoop - [mail # user]
|
|
...Hi Sasha In general the number of reduce tasks is chosen mainly based on the data volume to reduce phase. In tools like hive and pig by default for every 1GB of map output there will b...
|
|
|
Author: Bejoy KS,
2012-11-21, 16:50
|
|
|
Re: Hadoop installation on mac - Hadoop - [mail # user]
|
|
...Hi Suneel You can get the latest stable versions of hadoop from the following url http://hadoop.apache.org/releases.html#Download to download choose a mirror and slec...
|
|
|
Author: Bejoy KS,
2012-10-16, 12:06
|
|
|
Re: How to lower the total number of map tasks - Hadoop - [mail # user]
|
|
...Hi Shing Is your input a single file or set of small files? If latter you need to use CombineFileInputFormat. Regards Bejoy KS Sent from handheld, please excuse typos. &nb...
|
|
|
Author: Bejoy KS,
2012-10-02, 17:46
|
|
|
Re: How to lower the total number of map tasks - Hadoop - [mail # user]
|
|
...Shing This doesn't change the block size of existing files in hdfs, only new files written to hdfs will be affected. To get this in effect for old files you need to re copy them atleas...
|
|
|
Author: Bejoy KS,
2012-10-02, 17:37
|
|
|
Re: Disable retries - Hadoop - [mail # user]
|
|
...Hi Marco You can disable retries by setting mapred.map.max.attempts and mapred.reduce.max.attempts to 1. Also if you need to disable speculative execution. You can disable ...
|
|
|
Author: Bejoy KS,
2012-08-03, 00:02
|
|
|
Re: Merge Reducers Output - Hadoop - [mail # user]
|
|
...Hi Why not use 'hadoop fs -getMerge ' while copying files out of hdfs for the end users to consume. This will merge all the files in 'outputFolderInHdfs' into one file and...
|
|
|
Author: Bejoy KS,
2012-07-31, 04:30
|
|
|
Re: Retrying connect to server: localhost/127.0.0.1:9000. - Hadoop - [mail # user]
|
|
...Hi Keith Your NameNode is not up still. What does the NN logs say? Regards Bejoy KS Sent from handheld, please excuse typos. From: anil gupta Date: Fri, 27 Jul ...
|
|
|
Author: Bejoy KS,
2012-07-27, 18:34
|
|
|
|
