Hi,

I figured out what the issue is.

I'd configured my yarn.nodemanager.remote-app-log-dir to be
${yarn.log.dir}/logs. This doesn't per se make sense because this is a
remote location (on HDFS) and not related to yarn.log.dir which is more
like a local directory. However, the issue was that the script which starts
the history server ultimately uses the 'mapred' script that doesn't export
yarn.log.dir as a system property. Hence, while the logs were getting
aggregated correctly, the history server wasn't able to resolve to the
right path to show the logs from.

Fixing the value of the yarn.nodemanager.remote-app-log-dir variable fixed
the issue for me.

Thanks
Hemanth

On Mon, Sep 10, 2012 at 4:52 PM, Joshi, Rekha <[EMAIL PROTECTED]>wrote:

>  Hi Hemanth,
>
>  I am still getting my hands dirty on yarn, so this is preliminary –
> maybe as the hdfs path in AggregatedLogsBlock points to /tmp/logs and you
> say service is unable to read it, possibly check perm or change the
> configuration in yarn-site.xml and try?
>
>  Thanks
>  Rekha
>
>   From: Hemanth Yamijala <[EMAIL PROTECTED]>
> Reply-To: <[EMAIL PROTECTED]>
> Date: Mon, 10 Sep 2012 16:19:04 +0530
> To: <[EMAIL PROTECTED]>
> Subject: Log aggregation in Yarn
>
>  Hi,
>
>  I enabled log aggregation in Yarn and can see that files are getting
> created under the configured directory on HDFS. I can access the files via
> FS shell. However, when I try to retrieve the logs via the history server,
> they fail with a message:
>
>   Logs not available for attempt_1347261424213_0001_r_000000_0.
> Aggregation may not be complete, Check back later or try the nodemanager.
>
>  This is with trunk.  From code, it looks like this happens because the
> service is unable to access the file on HDFS. However, this is a single
> node setup, all processes running as the same user. Security is off in
> HDFS. Anything else wrong ? Do I need to configure something more ?
>
>  Thanks
> Hemanth
>