Hi All,
     How to compare to input file In M/R Job.
     let A Log file around 30GB
    and B Log file size is around 60 GB

  I wanted to know how  i will  define <K,V> inside the mapper.

 Thanks
  samir.

Samir,

You can use MultipleInputs for multiple forms of inputs per mapper
(with their own input K/V types, but common output K/V types) with a
common reduce-side join/compare.

See http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/mapreduce/lib/input/MultipleInputs.html.

On Thu, May 24, 2012 at 1:17 AM, samir das mohapatra
<[EMAIL PROTECTED]> wrote:
> Hi All,
>     How to compare to input file In M/R Job.
>     let A Log file around 30GB
>    and B Log file size is around 60 GB
>
>  I wanted to know how  i will  define <K,V> inside the mapper.
>
>  Thanks
>  samir.

--
Harsh J

You might want to start with http://hadoop.apache.org/common/docs/stable/mapred_tutorial.html.

Arun

On May 23, 2012, at 12:47 PM, samir das mohapatra wrote:

> Hi All,
>     How to compare to input file In M/R Job.
>     let A Log file around 30GB
>    and B Log file size is around 60 GB
>
>  I wanted to know how  i will  define <K,V> inside the mapper.
>
> Thanks
>  samir.

--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/

Samir,
  
Depends upon your data format and what you want to achieve. More information could have helped....

-Shailesh.

-----Original Message-----
From: samir das mohapatra [mailto:[EMAIL PROTECTED]]
Sent: Thursday, May 24, 2012 1:17 AM
To: [EMAIL PROTECTED]
Subject: Right way to implement MR ?

Hi All,
     How to compare to input file In M/R Job.
     let A Log file around 30GB
    and B Log file size is around 60 GB

  I wanted to know how  i will  define <K,V> inside the mapper.

 Thanks
  samir.

Thanks
  Harsh J for your help.

On Thu, May 24, 2012 at 1:24 AM, Harsh J <[EMAIL PROTECTED]> wrote:

> Samir,
>
> You can use MultipleInputs for multiple forms of inputs per mapper
> (with their own input K/V types, but common output K/V types) with a
> common reduce-side join/compare.
>
> See
> http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/mapreduce/lib/input/MultipleInputs.html
> .
>
> On Thu, May 24, 2012 at 1:17 AM, samir das mohapatra
> <[EMAIL PROTECTED]> wrote:
> > Hi All,
> >     How to compare to input file In M/R Job.
> >     let A Log file around 30GB
> >    and B Log file size is around 60 GB
> >
> >  I wanted to know how  i will  define <K,V> inside the mapper.
> >
> >  Thanks
> >  samir.
>
>
>
> --
> Harsh J
>