Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # user >> Re: Map Shuffle Bytes


Copy link to this message
-
Re: Map Shuffle Bytes
Hi,

What do you mean by "shuffled bytes [to] the mappers"? If you mean
"from", it is "Reduce shuffle bytes" you look for; otherwise, you may
be looking for the per-map counter of "Map output bytes".

Per-partition counters can be constructed on the user side if needed,
by pre-computing the partition before emit (using the same
partitioner) and counting up the bytes of your objects for its
counter.

On Tue, Dec 25, 2012 at 6:03 PM, Eduard Skaley <[EMAIL PROTECTED]> wrote:
> Hello guys,
>
> I need a counter for shuffled bytes to the mappers.
> Is there existing one or should I define one myself ?
> How can I implement such a counter?
>
> Thank you and happy Christmas time,
> Eduard

--
Harsh J
+
Eduard Skaley 2012-12-26, 11:46
+
Harsh J 2012-12-26, 11:50
+
Eduard Skaley 2012-12-26, 12:56