Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop, mail # user - passing arguments to hadoop job


+
jamal sasha 2013-01-21, 23:52
+
Satbeer Lamba 2013-01-22, 01:40
Copy link to this message
-
Re: passing arguments to hadoop job
jamal sasha 2013-01-22, 01:47
The second one.
If the word hello appears once, its count is  2000001.
:)
On Mon, Jan 21, 2013 at 5:40 PM, Satbeer Lamba <[EMAIL PROTECTED]>wrote:

> Do you mean to say you want to count the words from 200000 th line onwards?
>
> OR
>
> You want to start counting from 2000000?
> For example if HELLO appears once it's count is 2000001.
>
> Please clarify
> On Jan 22, 2013 5:22 AM, "jamal sasha" <[EMAIL PROTECTED]> wrote:
>
>> Hi,
>>   Lets say I have the standard helloworld program
>>
>> http://hadoop.apache.org/docs/r0.17.0/mapred_tutorial.html#Example%3A+WordCount+v2.0
>>
>> Now, lets say, I want to start the counting not from zero but from 200000.
>> So my reference line is 200000.
>>
>> I modified the Reduce code as following:
>>  public static class Reduce extends MapReduceBase implements
>> Reducer<Text, IntWritable, Text, IntWritable> {
>>      *private static int baseSum ;*
>> *      public void configure(JobConf job){*
>> *      baseSum = Integer.parseInt(job.get("basecount"));*
>> *      *
>> *      }*
>>        public void reduce(Text key, Iterator<IntWritable> values,
>> OutputCollector<Text, IntWritable> output, Reporter reporter) throws
>> IOException {
>>          int sum =* baseSum*;
>>         while (values.hasNext()) {
>>           sum += values.next().get();
>>          }
>>         output.collect(key, new IntWritable(sum));
>>       }
>>      }
>>
>>
>> And in main added:
>>    conf.setInt("basecount",200000);
>>
>>
>>
>> So my hope was this should have done the trick..
>> But its not working. the code is running normally :(
>> How do i resolve this?
>> Thanks
>>
>
+
Hemanth Yamijala 2013-01-22, 07:58
+
Mohammad Tariq 2013-01-22, 06:23