Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce, mail # user - Re: Calling the JobTracker from Reducer throws InvalidCredentials GSSException


+
Harsh J 2013-09-28, 18:06
+
Manish Verma 2013-09-29, 05:21
Copy link to this message
-
Re: Calling the JobTracker from Reducer throws InvalidCredentials GSSException
Harsh J 2013-09-29, 10:51
Hm, I think I forgot the bit where Oozie also adds a DT for itself to
use: https://github.com/apache/oozie/blob/release-4.0.0/core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java#L374.
Doing that same additional thing to the job's driver, works just fine
for me.

On Sun, Sep 29, 2013 at 10:51 AM, Manish Verma
<[EMAIL PROTECTED]> wrote:
> Hi Harsh,
>
> Thanks for the pointer but it didn't fix my problem. Basically I am trying
> to read the counters set by the mapper in my reducer. And the reducer has
> trouble reading the mapper's counter using the JobClient. I tested this
> logic in a single node cloudera VM which did not have kerberos installed.
>
> Thanks
> Manish
>
>
> On Sat, Sep 28, 2013 at 11:06 AM, Harsh J <[EMAIL PROTECTED]> wrote:
>>
>> You'll need to reuse the security tokens of the current job to
>> communicate without requiring a re-auth from the task. See the
>> specific code snippet (of what Oozie does for similar goals) at
>>
>> https://github.com/apache/oozie/blob/release-4.0.0/sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java#L90
>> and also see MAPREDUCE-4324.
>>
>> On Sat, Sep 28, 2013 at 2:43 AM, Manish Verma
>> <[EMAIL PROTECTED]> wrote:
>> > I am trying to get the job tracker counters in my reducer. It works on
>> > single node demo hadoop but fails on a real cluster where kerberos is
>> > used
>> > for authentication.
>> >
>> > ********************************************
>> > RunningJob parentJob >> >                     client.getJob(JobID.forName(
>> > context.getConfiguration().get("mapred.job.id") ));
>> >
>> > Counters counters = parentJob.getCounters();
>> > ********************************************
>> >
>> > The call to getCounter() API throws GSSException (No valid credentials
>> > provided - Failed to find any kerberos tgt).
>> >
>> > I launched this job using hadoop jar command.
>> >
>> > Any help would be much appreciated.
>> >
>> > Thanks
>> > Manish
>> >
>>
>>
>>
>> --
>> Harsh J
>
>

--
Harsh J
+
Manish Verma 2013-09-29, 16:16