Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # user >> passing arguments to hadoop job


+
jamal sasha 2013-01-21, 23:52
+
Satbeer Lamba 2013-01-22, 01:40
Copy link to this message
-
Re: passing arguments to hadoop job
The second one.
If the word hello appears once, its count is  2000001.
:)
On Mon, Jan 21, 2013 at 5:40 PM, Satbeer Lamba <[EMAIL PROTECTED]>wrote:

> Do you mean to say you want to count the words from 200000 th line onwards?
>
> OR
>
> You want to start counting from 2000000?
> For example if HELLO appears once it's count is 2000001.
>
> Please clarify
> On Jan 22, 2013 5:22 AM, "jamal sasha" <[EMAIL PROTECTED]> wrote:
>
>> Hi,
>>   Lets say I have the standard helloworld program
>>
>> http://hadoop.apache.org/docs/r0.17.0/mapred_tutorial.html#Example%3A+WordCount+v2.0
>>
>> Now, lets say, I want to start the counting not from zero but from 200000.
>> So my reference line is 200000.
>>
>> I modified the Reduce code as following:
>>  public static class Reduce extends MapReduceBase implements
>> Reducer<Text, IntWritable, Text, IntWritable> {
>>      *private static int baseSum ;*
>> *      public void configure(JobConf job){*
>> *      baseSum = Integer.parseInt(job.get("basecount"));*
>> *      *
>> *      }*
>>        public void reduce(Text key, Iterator<IntWritable> values,
>> OutputCollector<Text, IntWritable> output, Reporter reporter) throws
>> IOException {
>>          int sum =* baseSum*;
>>         while (values.hasNext()) {
>>           sum += values.next().get();
>>          }
>>         output.collect(key, new IntWritable(sum));
>>       }
>>      }
>>
>>
>> And in main added:
>>    conf.setInt("basecount",200000);
>>
>>
>>
>> So my hope was this should have done the trick..
>> But its not working. the code is running normally :(
>> How do i resolve this?
>> Thanks
>>
>
+
Hemanth Yamijala 2013-01-22, 07:58
+
Mohammad Tariq 2013-01-22, 06:23
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB