Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # user >> Map performance with custom binary format


+
william kinney 2009-07-28, 15:18
+
Scott Carey 2009-07-28, 16:58
+
Ted Dunning 2009-07-28, 18:13
+
william kinney 2009-07-28, 19:15
+
Ted Dunning 2009-07-28, 20:25
Copy link to this message
-
Re: Map performance with custom binary format
On Tue, Jul 28, 2009 at 01:25:49PM -0700, Ted Dunning wrote:
> On Tue, Jul 28, 2009 at 12:15 PM, william kinney
> <[EMAIL PROTECTED]>wrote:
>
> >
> > Also, from the job page (different job, same Map method, just more
> > data...~40GB. 781 files):
> > Map input records       629,738,080
> > Map input bytes         41,538,992,880
> >
> > Anything else I can look into?
>
>
> Yes.  The number of data local maps and how many maps total.
>

Do "data local maps" short-circuit to the local filesystem at all, or do
they read data over HTTP from the data node's jetty instance over the
loopback device?

-Erik
+
Ted Dunning 2009-07-28, 22:15
+
william kinney 2009-07-28, 21:40
+
Ted Dunning 2009-07-28, 20:27
+
Scott Carey 2009-07-28, 22:35
+
Jason Venner 2009-07-29, 02:01
+
william kinney 2009-07-30, 04:10
+
william kinney 2009-07-30, 04:07
+
Scott Carey 2009-07-30, 05:31
+
william kinney 2009-07-30, 14:37
+
william kinney 2009-07-30, 17:42
+
Scott Carey 2009-07-30, 18:39
+
Todd Lipcon 2009-07-30, 18:51
+
Scott Carey 2009-07-30, 20:32
+
Scott Carey 2009-07-30, 18:19
+
william kinney 2009-07-30, 21:32
+
Scott Carey 2009-08-01, 01:07
+
Scott Carey 2009-07-31, 21:31
+
Steve Loughran 2009-07-29, 09:17
+
Todd Lipcon 2009-07-29, 18:47
+
Scott Carey 2009-07-29, 19:07
+
Todd Lipcon 2009-07-29, 19:10