Home | About | Sematext search-lucene.com search-hadoop.com
clear query|facets|time Search criteria: .   Results from 1 to 10 from 36 (0.177s).
Loading phrases to help you
refine your search...
[expand - 4 more] - Re: Child Error - Hadoop - [mail # user]
...It sometimes works and sometimes fails. Usually, if my tasks take more than an hour, at least one node fails and gets blacklisted. I have seen at most five nodes do this.  I cleared my ...
   Author: Jim Twensky, 2013-05-28, 17:02
[expand - 1 more] - Re: Question on HDFS_BYTES_READ and HDFS_BYTES_WRITTEN - MapReduce - [mail # user]
...Hello Harsh,  Thanks for the useful feedback. You were right. My map tasks open additional files from hdfs. The catch was that I had thousands of map tasks being created and each of the...
   Author: Jim Twensky, 2013-05-17, 21:50
[expand - 1 more] - Re: Wrapping around BitSet with the Writable interface - MapReduce - [mail # user]
...Thanks for the suggestions. I ended up switching to jdk 1.7+ just to make the code more readable. I will take a look at the EWAH implementation as well.  Jim   On Sun, May 12, 2013...
   Author: Jim Twensky, 2013-05-13, 15:51
[expand - 1 more] - Redundant synchronization in ReduceTask.java - MapReduce - [mail # dev]
...Hi,  I have recently been reading Hadoop 1.1.0 source code to better understand the internals and learned a lot from it, so far. When I was looking at ReduceTask.java, I saw some synchr...
   Author: Jim Twensky, 2012-12-08, 20:58
[expand - 4 more] - Re: Chaning Multiple Reducers: Reduce -> Reduce -> Reduce - HDFS - [mail # user]
...Hi Mike,  I'm already doing that but the output of the reduce goes straight back to HDFS to be consumed by the next Identity Mapper. Combiners just reduce the amount of data between map...
   Author: Jim Twensky, 2012-10-08, 20:53
FileSystem API - Moving files in HDFS - Hadoop - [mail # user]
...Hi,  I'd like to move and copy files from one directory in HDFS to another one. I know there are methods in the Filesystem API that enable copying files between the local disk and HDFS,...
   Author: Jim Twensky, 2011-05-13, 20:21
Tasktracker failing and getting black listed - Hadoop - [mail # user]
...Hi,  I have a 16+1 node hadoop cluster where all tasktrackers (and datanodes) are connected to the same switch and share the exact same hardware and software configuration. When I run a...
   Author: Jim Twensky, 2010-12-23, 22:37
Creating an RMI client inside a mapper - Hadoop - [mail # user]
...I'm trying to create an instance of an RMI client that queries a remote RMI server inside my Mapper class. My application runs smoothly without the RMI client. When I add:  if (System.g...
   Author: Jim Twensky, 2010-04-29, 19:45
Re: Algorithm used "Shuffle and Sort" step - Hadoop - [mail # user]
...I'm not quite sure what you mean by the Shuffle algorithm but briefly for each (key,value) pair, a hash value is computed and then the record is sent to a node by taking the modulo of that h...
   Author: Jim Twensky, 2010-04-28, 18:47
[expand - 1 more] - Re: Questions on MultithreadedMapper - Hadoop - [mail # user]
...Thanks Ted. Is it correct to assume that all class members defined inside my Mapper are visible to all of the threads, so I should pay careful attention and take synchronization into account...
   Author: Jim Twensky, 2010-04-28, 15:52
Hadoop (32)
MapReduce (3)
HDFS (1)
mail # user (35)
mail # dev (1)
last 7 days (0)
last 30 days (0)
last 90 days (0)
last 6 months (0)
last 9 months (36)
Ted Yu (1700)
Harsh J (1295)
Jun Rao (1060)
Todd Lipcon (1001)
Stack (977)
Jonathan Ellis (844)
Andrew Purtell (822)
Jean-Daniel Cryans (753)
Yusaku Sako (732)
stack (714)
Jarek Jarcec Cecho (702)
Eric Newton (698)
Jonathan Hsieh (673)
Brock Noland (668)
Roman Shaposhnik (664)
Neha Narkhede (661)
Namit Jain (649)
Hitesh Shah (627)
Owen O'Malley (625)
Steve Loughran (623)
Siddharth Seth (614)
Josh Elser (588)
Eli Collins (545)
Arun C Murthy (543)
Doug Cutting (533)
Jim Twensky