| clear query|facets|time |
Search criteria: .
Results from 51 to 60 from
155 (0.712s).
|
|
|
Loading phrases to help you
refine your search...
|
|
Re: problem using s3 instead of hdfs - MapReduce - [mail # user]
|
|
...Parth, I notice in the below stack trace that the LocalJobRunner, instead of the JobTracker is being used. Are you sure this is a distributed cluster ? Could you please check the value...
|
|
|
Author: Hemanth Yamijala,
2012-10-16, 15:10
|
|
|
Re: problem using s3 instead of hdfs - MapReduce - [mail # user]
|
|
...Hi, I've not tried this on S3. However, the directory mentioned in the exception is based on the value of this particular configuration key: mapreduce.jobtracker.staging.root.dir. This...
|
|
|
Author: Hemanth Yamijala,
2012-10-16, 07:11
|
|
|
Re: Question about how to find which file takes the longest time to process and how to assign more mappers to process that particular file - MapReduce - [mail # user]
|
|
...Hi, Roughly, this information will be available under the 'Hadoop map task list' page in the Mapreduce web ui (in Hadoop-1.0, which I am assuming is what you are using). You can reach ...
|
|
|
Author: Hemanth Yamijala,
2012-10-05, 04:21
|
|
|
Re: Submitting a job to a remote cluster - MapReduce - [mail # user]
|
|
...Hi, Could you please share your setup details - i.e. how many slaves, how many datanodes and tasktrackers. Also, the configuration - in particular hdfs-site.xml ? To answer your ...
|
|
|
Author: Hemanth Yamijala,
2012-10-05, 04:08
|
|
|
Re: hadoop issue on distributed cluster - MapReduce - [mail # user]
|
|
...Hi, Didn't check everything. But found this in the mapred-site.xml: mapred.job.tracker hdfs://10.99.42.9:8021/ true The value shouldn't be a HDFS URL. C...
|
|
|
Author: Hemanth Yamijala,
2012-10-05, 04:02
|
|
|
Re: A small portion of map tasks slows down the job - MapReduce - [mail # user]
|
|
...Hi, Would reducing the output from the map tasks solve the problem ? i.e. are reducers slowing down because a lot of data is being shuffled ? If that's the case, you could see if...
|
|
|
Author: Hemanth Yamijala,
2012-10-03, 13:37
|
|
|
Re: How not to clean MapReduce temp data? - MapReduce - [mail # dev]
|
|
...Adding to Bejoy's response, if you want to just keep the temp output of failed tasks, you can also use keep.failed.task.files. Also, these config variable names are as per Hadoop 1.0. ...
|
|
|
Author: Hemanth Yamijala,
2012-09-28, 04:38
|
|
|
Re: How to output according to the key in reducer? - MapReduce - [mail # user]
|
|
...Hi, Can you see if the MultipleOutputs class can work for you ? ( http://hadoop.apache.org/docs/r1.0.3/api/org/apache/hadoop/mapreduce/lib/output/MultipleOutputs.html ) On ...
|
|
|
Author: Hemanth Yamijala,
2012-09-17, 08:08
|
|
|
Re: Ignore keys while scheduling reduce jobs - MapReduce - [mail # user]
|
|
...Hi, Does the mapper know what is the 1st point in the data set and the cluster id corresponding to it ? I don't know much about the kmeans algorithm, hence may be wrong .. If the...
|
|
|
Author: Hemanth Yamijala,
2012-09-14, 12:08
|
|
|
Re: Ignore keys while scheduling reduce jobs - MapReduce - [mail # user]
|
|
...Hi, When do you know the keys to ignore ? You mentioned "after the map stage" .. is this at the end of each map task, or at the end of all map tasks ? Thanks hemanth On Fri...
|
|
|
Author: Hemanth Yamijala,
2012-09-14, 11:13
|
|
|
|
