Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> newbie question


Copy link to this message
-
RE: newbie question
I get an Invalid ID when I try to access pastebin, so I haven't seen the actual code.  But in the snippet below method that is not getting called should be named configure, not config.

public class BuildGraph extends MapReduceBase implements Mapper<> {

@Override
public void configure(JobConf conf) {
                                ...
}

@Override
public void map() {
                ...
}
}

Dave Shine
Sr. Software Engineer
321.939.5093 direct |  407.314.0122 mobile
CI Boost(tm) Clients  Outperform Online(tm)  www.ciboost.com<http://www.ciboost.com/>

From: jamal sasha [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, January 15, 2013 9:04 PM
To: [EMAIL PROTECTED]
Subject: Re: newbie question

Hi,
 The relevant code snippet posted here
http://pastebin.com/DRPXUm62

On Tue, Jan 15, 2013 at 5:31 PM, jamal sasha <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:
My bad. Sorry I fixed it. It is BuildGraph.class

On Tue, Jan 15, 2013 at 5:30 PM, Serge Blazhiyevskyy <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:
Don't you need to set this

     conf.setMapperClass(Mapper.class);

to BuildGraph?
Serge
SERGE BLAZHIYEVSKY
Architect
(T) +1 (650) 226-0511<tel:%2B1%20%28650%29%20226-0511>
(M) +1 (408) 772-2615<tel:%2B1%20%28408%29%20772-2615>
[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]><mailto:[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
www.nice.com<http://www.nice.com><http://www.nice.com>
On Jan 15, 2013, at 5:24 PM, jamal sasha <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]><mailto:[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>> wrote:

I have a mapper
    public class BuildGraph{
       public void config(JobConf job){ <==this block doesnt seems to be exexcuting at all :(
        super.configure(job);
        this.currentId = job.getInt("currentId",0);
        if (this.currentId!=0){
        // I call a method from differnt class to build a distributed cache
        }
       }
      public void map(....){
    ....
    }
}

now the main code where this is called..

    public void run( String params,curId){
     JobConf conf = new JobConf(classname.class);
     conf.setInt("currentId",299); <--note this i am setting the value here
     conf.setMapperClass(Mapper.class);
     //....
      JobClient.runJob(conf);
    }

But the problem is config method in the code is not executing as though "currentId" returns 299 in main loop but it is not set at all in the mapper class.
what am i doing wrong

________________________________
The information contained in this email message is considered confidential and proprietary to the sender and is intended solely for review and use by the named recipient. Any unauthorized review, use or distribution is strictly prohibited. If you have received this message in error, please advise the sender by reply email and delete the message.
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB