Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # user >> Re: Calling the JobTracker from Reducer throws InvalidCredentials GSSException


+
Harsh J 2013-09-28, 18:06
+
Manish Verma 2013-09-29, 05:21
Copy link to this message
-
Re: Calling the JobTracker from Reducer throws InvalidCredentials GSSException
Hm, I think I forgot the bit where Oozie also adds a DT for itself to
use: https://github.com/apache/oozie/blob/release-4.0.0/core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java#L374.
Doing that same additional thing to the job's driver, works just fine
for me.

On Sun, Sep 29, 2013 at 10:51 AM, Manish Verma
<[EMAIL PROTECTED]> wrote:
> Hi Harsh,
>
> Thanks for the pointer but it didn't fix my problem. Basically I am trying
> to read the counters set by the mapper in my reducer. And the reducer has
> trouble reading the mapper's counter using the JobClient. I tested this
> logic in a single node cloudera VM which did not have kerberos installed.
>
> Thanks
> Manish
>
>
> On Sat, Sep 28, 2013 at 11:06 AM, Harsh J <[EMAIL PROTECTED]> wrote:
>>
>> You'll need to reuse the security tokens of the current job to
>> communicate without requiring a re-auth from the task. See the
>> specific code snippet (of what Oozie does for similar goals) at
>>
>> https://github.com/apache/oozie/blob/release-4.0.0/sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java#L90
>> and also see MAPREDUCE-4324.
>>
>> On Sat, Sep 28, 2013 at 2:43 AM, Manish Verma
>> <[EMAIL PROTECTED]> wrote:
>> > I am trying to get the job tracker counters in my reducer. It works on
>> > single node demo hadoop but fails on a real cluster where kerberos is
>> > used
>> > for authentication.
>> >
>> > ********************************************
>> > RunningJob parentJob >> >                     client.getJob(JobID.forName(
>> > context.getConfiguration().get("mapred.job.id") ));
>> >
>> > Counters counters = parentJob.getCounters();
>> > ********************************************
>> >
>> > The call to getCounter() API throws GSSException (No valid credentials
>> > provided - Failed to find any kerberos tgt).
>> >
>> > I launched this job using hadoop jar command.
>> >
>> > Any help would be much appreciated.
>> >
>> > Thanks
>> > Manish
>> >
>>
>>
>>
>> --
>> Harsh J
>
>

--
Harsh J
+
Manish Verma 2013-09-29, 16:16
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB