Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Chukwa, mail # user - the check point offset is bigger than the log file size


Copy link to this message
-
Re: the check point offset is bigger than the log file size
Eric Yang 2012-05-19, 23:51
I am unable to reproduce this if the log file has not rotated.

I know there is a separated bug that when file is rotated, the check
point offset does not change until the new file exceeds the last check
point offset.

I only use Log4j SocketAppender and SocketAdaptor for streaming data.
I don't use CharFileTailingAdaptorUTF8 adaptor.

regards,
Eric

On Mon, May 14, 2012 at 4:01 AM, IvyTang <[EMAIL PROTECTED]> wrote:
> The gamelog size is 158023223, but the check point file is
>
> ADD adaptor_2963225a90653a309cf779d4a1d815a3 > org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8
> Gamelog 0 /var/log/dataproxy/gamelog 229406124
>
> The gamelog didn't rotate , i'm sure.
>
> But the check point file size is bigger than the file size , the chukwa
> WARN Thread-2 FileTailingAdaptor -
> Adaptor|adaptor_2963225a90653a309cf779d4a1d815a3| file:
> /var/log/dataproxy/gamelog, has rotated and no detection - reset counters to
> 0L
> And the agent began to transfer the whole log file.
>
> I just feel confused why agent generate a offset size is bigger than the log
> size when the gamelog did not rotate.
>
> The chukwa version is 0.4.0
>
> --
> Best regards,
>
> Ivy Tang
>
>
>