| clear query|facets|time |
Search criteria: .
Results from 71 to 80 from
275 (2.841s).
|
|
|
Loading phrases to help you
refine your search...
|
|
Re: Processing 10MB files in Hadoop - Hadoop - [mail # user]
|
|
...By default you get at least one task per file; if any file is bigger than a block, then that file is broken up into N tasks where each is one block long. Not sure what you mean by "properly ...
|
|
|
Author: Aaron Kimball,
2009-11-28, 02:21
|
|
|
Re: conf.Configuration: java.io.IOException: config(config) - Hadoop - [mail # dev]
|
|
...When running Hadoop with DEBUG logging on, this IOException was actually responsible for well-over 90% of the lines of text in my logs, making them unreadable. We actually removed this...
|
|
|
Author: Aaron Kimball,
2009-11-27, 20:09
|
|
|
Re: Processing 10MB files in Hadoop - Hadoop - [mail # user]
|
|
...More importantly: have you told Hadoop to use all your cores? What is mapred.tasktracker.map.tasks.maximum set to? This defaults to 2. If you've got 16 cores/node, you should set this ...
|
|
|
Author: Aaron Kimball,
2009-11-27, 19:02
|
|
|
Re: Good idea to run NameNode and JobTracker on same machine? - Hadoop - [mail # user]
|
|
...The real kicker is going to be memory consumption of one or both of these services. The NN in particular uses a large amount of RAM to store the filesystem image. I think that those who are ...
|
|
|
Author: Aaron Kimball,
2009-11-27, 18:50
|
|
|
Re: part-00000.deflate as output - Hadoop - [mail # user]
|
|
...You are always free to run with compression disabled. But in many production situations, space or performance concerns dictate that all data sets are stored compressed, so I think Tim was as...
|
|
|
Author: Aaron Kimball,
2009-11-27, 18:44
|
|
|
Re: Re: Doubt in Hadoop - Hadoop - [mail # user]
|
|
...When you set up the Job object, do you call job.setJarByClass(Map.class)? That will tell Hadoop which jar file to ship with the job and to use for classloading in your code. - Aaron &n...
|
|
|
Author: Aaron Kimball,
2009-11-27, 18:42
|
|
|
Re: RE: please help in setting hadoop - Hadoop - [mail # user]
|
|
...You've set hadoop.tmp.dir to /home/hadoop/hadoop-${user.name}. This means that on every node, you're going to need a directory named (e.g.) /home/hadoop/hadoop-root/, since it seems as...
|
|
|
Author: Aaron Kimball,
2009-11-27, 18:40
|
|
|
Re: error setting up hdfs? - Hadoop - [mail # user]
|
|
...You don't "need" to specify a path. If you don't specify a path argument for ls, then it uses your home directory in HDFS ("/user/"). When you first started HDFS, /user/hadoop didn't exist, ...
|
|
|
Author: Aaron Kimball,
2009-11-10, 21:47
|
|
|
Re: How to build and deploy Hadoop 0.21 ? - Hadoop - [mail # dev]
|
|
...On Thu, Nov 5, 2009 at 2:34 AM, Andrei Dragomir wrote: Those are build-time dependencies. Ideally you'll ignore them post-build. Yup. I have created a ...
|
|
|
Author: Aaron Kimball,
2009-11-09, 04:13
|
|
|
Re: Can I install two different version of hadoop in the same cluster ? - Hadoop - [mail # user]
|
|
...Also hadoop.tmp.dir and mapred.local.dir in your xml configuration, and the environment variables HADOOP_LOG_DIR and HADOOP_PID_DIR in hadoop-env.sh. - Aaron On Thu, Oct 29, 2009...
|
|
|
Author: Aaron Kimball,
2009-10-30, 06:00
|
|
|
|
