Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
clear query|facets|time Search criteria: .   Results from 1 to 10 from 21 (0.211s).
Loading phrases to help you
refine your search...
Re: Will hadoop always spread the work evenly between nodes? - HDFS - [mail # user]
...I think in your case it will have to be even, because all the slots will get filled. A more interesting case is if you have 40 nodes, will you get exactly 5 slots used for each of the nodes?...
   Author: Jeffrey Buell, 2013-03-13, 20:47
Re: Difference between HDFS_BYTES_READ and the actual size of input files - Hadoop - [mail # user]
...Jeff,   Probably because records are split across blocks, so some of the data has to be read twice. Assuming you have a 64 MB block size and 128 GB of data, I'd estimate the overhead at...
   Author: Jeffrey Buell, 2013-03-06, 19:12
[expand - 1 more] - Re: Format the harddrive - Hadoop - [mail # user]
...What make and model are these machines? What is the storage controller? You may need to go into the storage configuration tool during hardware boot and look at how the controller has configu...
   Author: Jeffrey Buell, 2013-02-25, 22:40
Re: Hadoop efficient resource isolation - MapReduce - [mail # user]
...This is one reason to consider virtualizing Hadoop clusters. The idea is to create multiple virtual clusters on a single physical cluster and apply various kinds of resource controls (CPU, m...
   Author: Jeffrey Buell, 2013-02-25, 21:37
Re: Estimating disk space requirements - Hadoop - [mail # user]
...I disagree. There are some significant advantages to using "many small nodes" instead of "few big nodes". As Ted points out, there are some disadvantages as well, so you have to look at the ...
   Author: Jeffrey Buell, 2013-01-19, 01:01
Re: config for high memory jobs does not work, please help. - HDFS - [mail # user]
...Try:  -Dmapred.tasktracker.map.tasks.maximum=1  Although I usually put this parameter in mapred-site.xml.  Jeff   Dear all,  I know it is best to use small amount of...
   Author: Jeffrey Buell, 2013-01-18, 20:23
Re: question of how to take full advantage of cluster resources - MapReduce - [mail # user]
...Number of CPU cores is just one of several hardware constraints on the number of tasks that can be run efficiently at the same time. Other constraints:   - Usually 1 to 2 map tasks per ...
   Author: Jeffrey Buell, 2012-12-14, 23:29
RE: HDFS using SAN - MapReduce - [mail # user]
...It will be difficult to make a SAN work well for Hadoop, but not impossible .  I have done direct comparisons (but not published them yet).  Direct loc al storage is likely to have...
   Author: Jeffrey Buell, 2012-10-16, 21:24
RE: Spindle per Cores - MapReduce - [mail # user]
...I've done some experiments along these lines.  I'm using high-performance 1 5K RPM SAS drives instead of the more usual SATA drives, which should reduc e the number of drives I need. &n...
   Author: Jeffrey Buell, 2012-10-12, 21:19
RE: Cannot browse job.xml while running the job - MapReduce - [mail # user]
...Does /var/log/hadoop have write permission for the hadoop user?  From: Gaurav Dasgupta [mailto:[EMAIL PROTECTED]] Sent: Thursday, September 06, 2012 9:45 AM To: [EMAIL PROTECTED] Subjec...
   Author: Jeffrey Buell, 2012-09-06, 16:49
Sort:
project
MapReduce (10)
HDFS (8)
Hadoop (3)
type
mail # user (21)
date
last 7 days (0)
last 30 days (0)
last 90 days (0)
last 6 months (0)
last 9 months (21)
author
Ted Yu (1775)
Harsh J (1297)
Jun Rao (999)
Todd Lipcon (993)
Stack (980)
Andrew Purtell (852)
Jonathan Ellis (846)
Jean-Daniel Cryans (750)
stack (740)
Yusaku Sako (735)
Jarek Jarcec Cecho (724)
Eric Newton (695)
Jonathan Hsieh (674)
Roman Shaposhnik (673)
Namit Jain (649)
Hitesh Shah (645)
Steve Loughran (634)
Josh Elser (628)
Siddharth Seth (626)
Owen O'Malley (624)
Brock Noland (600)
Neha Narkhede (555)
Arun C Murthy (546)
Eli Collins (545)
Hyunsik Choi (543)
Jeffrey Buell
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB