Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop, mail # dev - Reading partition for reducer


+
Vikas Jadhav 2013-04-01, 12:06
+
Harsh J 2013-04-01, 13:51
Copy link to this message
-
Re: Reading partition for reducer
Vikas Jadhav 2013-04-01, 16:38
Hi

I want process/store  all data pertaining to one reducer.

i want store it in some data structure depending on key for example

(0,ABC)
(0,TER)
(1,DEF)
(1,XYZ)

for key 0 and 1 data will be in different data structure.

then perform cross product of above datasets

Currently i am looking in ReduceTask.java

Thank You.
On Mon, Apr 1, 2013 at 7:21 PM, Harsh J <[EMAIL PROTECTED]> wrote:

> The question should be more specific here: Do you want to process a
> map's sorted total output or do you want to pre-process a whole
> partition (i.e. all data pertaining to one reducer)? Former would be
> more ideal inside MapTask.java, latter in ReduceTask.java.
>
> On Mon, Apr 1, 2013 at 5:36 PM, Vikas Jadhav <[EMAIL PROTECTED]>
> wrote:
> > Hello
> >
> > I want to process output of mapper to processed before it is sent to
> > reducer.
> >
> > @ what point i should hook in my code processing
> >
> >
> > i guess it is ReduceTask.java file
> >
> > if anyone knows reagarding this please help me in this.
> >
> >
> > Thank You.
> >
> >
> > --
> > *
> > *
> > *
> >
> > Thanx and Regards*
> > * Vikas Jadhav*
>
>
>
> --
> Harsh J
>

--
*
*
*

Thanx and Regards*
* Vikas Jadhav*
+
Steve Loughran 2013-04-01, 20:59
+
Vikas Jadhav 2013-04-02, 04:19