Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # user - passing arguments to hadoop job


Copy link to this message
-
Re: passing arguments to hadoop job
Satbeer Lamba 2013-01-22, 01:40
Do you mean to say you want to count the words from 200000 th line onwards?

OR

You want to start counting from 2000000?
For example if HELLO appears once it's count is 2000001.

Please clarify
On Jan 22, 2013 5:22 AM, "jamal sasha" <[EMAIL PROTECTED]> wrote:

> Hi,
>   Lets say I have the standard helloworld program
>
> http://hadoop.apache.org/docs/r0.17.0/mapred_tutorial.html#Example%3A+WordCount+v2.0
>
> Now, lets say, I want to start the counting not from zero but from 200000.
> So my reference line is 200000.
>
> I modified the Reduce code as following:
>  public static class Reduce extends MapReduceBase implements Reducer<Text,
> IntWritable, Text, IntWritable> {
>      *private static int baseSum ;*
> *      public void configure(JobConf job){*
> *      baseSum = Integer.parseInt(job.get("basecount"));*
> *      *
> *      }*
>        public void reduce(Text key, Iterator<IntWritable> values,
> OutputCollector<Text, IntWritable> output, Reporter reporter) throws
> IOException {
>          int sum =* baseSum*;
>         while (values.hasNext()) {
>           sum += values.next().get();
>          }
>         output.collect(key, new IntWritable(sum));
>       }
>      }
>
>
> And in main added:
>    conf.setInt("basecount",200000);
>
>
>
> So my hope was this should have done the trick..
> But its not working. the code is running normally :(
> How do i resolve this?
> Thanks
>