| clear query|facets|time |
Search criteria: .
Results from 1 to 10 from
38 (0.269s).
|
|
|
Loading phrases to help you
refine your search...
|
|
RE: JobClient: Error reading task output - after instituting a DNS server - MapReduce - [mail # user]
|
|
...So simple I was hoping to avoid admitting to it. ;-) I had set the tasks java options at -Xmx1.5g, that needed to be -Xmx1500m, the telltale output of a mistake like that is ra...
|
|
|
Author: David Parks,
2013-05-15, 07:28
|
|
|
RE: JobClient: Error reading task output - after instituting a DNS server - MapReduce - [mail # user]
|
|
...I just hate it when I figure out a problem right after asking for help. Finding the task logs via the task tracker website identified the problem which didn't show up elsewhere...
|
|
|
Author: David Parks,
2013-05-14, 06:29
|
|
|
JobClient: Error reading task output - after instituting a DNS server - MapReduce - [mail # user]
|
|
...So we just configured a local DNS server for hostname resolution and stopped using a hosts file and now jobs fail on us. But I can't figure out why. You can see the error below...
|
|
|
Author: David Parks,
2013-05-14, 06:19
|
|
|
RE: Access HDFS from OpenCL - MapReduce - [mail # user]
|
|
...Hadoop just runs as a standard java process, you should find something that bridges between OpenCL and java, a quick google search yields: http://www.jocl.org/ I expect that yo...
|
|
|
Author: David Parks,
2013-05-13, 14:11
|
|
|
600s timeout during copy phase of job - MapReduce - [mail # user]
|
|
...I have a job that's getting 600s task timeouts during the copy phase of the reduce step. I see a lot of copy tasks all moving at about 2.5MB/sec, and it's taking longer than 10 min to do tha...
|
|
|
Author: David Parks,
2013-05-13, 06:05
|
|
|
What's the best disk configuration for hadoop? SSD's Raid levels, etc? - MapReduce - [mail # user]
|
|
...We've got a cluster of 10x 8core/24gb nodes, currently with 1 4TB disk (3 disk slots max), they chug away ok currently, only slightly IO bound on average. I'm going to upgrade ...
|
|
|
Author: David Parks,
2013-05-11, 06:30
|
|
|
RE: Uploading file to HDFS - MapReduce - [mail # user]
|
|
...I just realized another trick you might trying. The Hadoop dfs client can read input from STDIN, you could use netcat to pipe the stuff across to HDFS without hitting the hard drive, I haven...
|
|
|
Author: David Parks,
2013-04-19, 08:42
|
|
|
RE: Uploading file to HDFS - MapReduce - [mail # user]
|
|
...I think the problem here is that he doesn't have Hadoop installed on this other location so there's no Hadoop DFS client to do the put directly into HDFS on, he would normally copy the file ...
|
|
|
Author: David Parks,
2013-04-19, 07:02
|
|
|
Mapreduce jobs to download job input from across the internet - MapReduce - [mail # user]
|
|
...For a set of jobs to run I need to download about 100GB of data from the internet (~1000 files of varying sizes from ~10 different domains). Currently I do this in a simple lin...
|
|
|
Author: David Parks,
2013-04-17, 04:26
|
|
|
RE: Hadoop distcp from CDH4 to Amazon S3 - Improve Throughput - MapReduce - [mail # user]
|
|
...4-20MB/sec are common transfer rates from S3 to *1* local AWS box, this was, of course, a cluster, and s3distcp is specifically designed to take advantage of the cluster, so it was a 45 minu...
|
|
|
Author: David Parks,
2013-03-31, 01:26
|
|
|
|
