Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Right way to implement MR ?


Copy link to this message
-
Re: Right way to implement MR ?
Thanks
  Harsh J for your help.

On Thu, May 24, 2012 at 1:24 AM, Harsh J <[EMAIL PROTECTED]> wrote:

> Samir,
>
> You can use MultipleInputs for multiple forms of inputs per mapper
> (with their own input K/V types, but common output K/V types) with a
> common reduce-side join/compare.
>
> See
> http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/mapreduce/lib/input/MultipleInputs.html
> .
>
> On Thu, May 24, 2012 at 1:17 AM, samir das mohapatra
> <[EMAIL PROTECTED]> wrote:
> > Hi All,
> >     How to compare to input file In M/R Job.
> >     let A Log file around 30GB
> >    and B Log file size is around 60 GB
> >
> >  I wanted to know how  i will  define <K,V> inside the mapper.
> >
> >  Thanks
> >  samir.
>
>
>
> --
> Harsh J
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB