Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Chukwa, mail # user - How to check whether chukwa collecting logs


Copy link to this message
-
Re: How to check whether chukwa collecting logs
Shouvanik Haldar 2013-01-10, 07:11
Can you please help me find out why the following log is generated by the
collector

2013-01-10 07:09:23,864 INFO Timer-1 root -
stats:ServletCollector,numberHTTPConnection:18,numberchunks:166
2013-01-10 07:09:23,884 WARN LocalToRemoteHdfsMover CopySequenceFile -
Error during .chukwa file recovery
java.io.EOFException
        at java.io.DataInputStream.readFully(DataInputStream.java:180)
        at
org.apache.hadoop.io.DataOutputBuffer$Buffer.write(DataOutputBuffer.java:63)
        at
org.apache.hadoop.io.DataOutputBuffer.write(DataOutputBuffer.java:101)
        at
org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1988)
        at
org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1888)
        at
org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1934)
        at
org.apache.hadoop.chukwa.util.CopySequenceFile.createValidSequenceFile(CopySequenceFile.java:78)
        at
org.apache.hadoop.chukwa.datacollection.writer.localfs.LocalToRemoteHdfsMover.cleanup(LocalToRemoteHdfsMover.java:186)
        at
org.apache.hadoop.chukwa.datacollection.writer.localfs.LocalToRemoteHdfsMover.run(LocalToRemoteHdfsMover.java:216)
2013-01-10 07:09:23,891 INFO LocalToRemoteHdfsMover LocalToRemoteHdfsMover
- Deleted .recover file,
/tmp/chukwa/dataSink/201308112441296_domU12313907193B_216836a313c19d565ee7ff8.recover
2013-01-10 07:09:23,891 INFO LocalToRemoteHdfsMover LocalToRemoteHdfsMover
- Copying .chukwa file to valid sink file before moving,
/tmp/chukwa/dataSink/201310063103378_domU1231390B80BA_1082291f13c22c778057f42.chukwa
2013-01-10 07:09:23,899 WARN LocalToRemoteHdfsMover FSInputChecker -
Problem opening checksum file:
/tmp/chukwa/dataSink/201310063103378_domU1231390B80BA_1082291f13c22c778057f42.chukwa.
Ignoring exception: java.io.EOFException
        at java.io.DataInputStream.readFully(DataInputStream.java:180)
        at java.io.DataInputStream.readFully(DataInputStream.java:152)
        at
org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.<init>(ChecksumFileSystem.java:134)
        at
org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:283)
        at
org.apache.hadoop.io.SequenceFile$Reader.openFile(SequenceFile.java:1495)
        at
org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1482)
        at
org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1475)
        at
org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1470)
        at
org.apache.hadoop.chukwa.util.CopySequenceFile.createValidSequenceFile(CopySequenceFile.java:73)
        at
org.apache.hadoop.chukwa.datacollection.writer.localfs.LocalToRemoteHdfsMover.cleanup(LocalToRemoteHdfsMover.java:186)
        at
org.apache.hadoop.chukwa.datacollection.writer.localfs.LocalToRemoteHdfsMover.run(LocalToRemoteHdfsMover.java:216)

2013-01-10 07:09:23,931 WARN LocalToRemoteHdfsMover CopySequenceFile -
Error during .chukwa file recovery
java.io.EOFException
        at java.io.DataInputStream.readFully(DataInputStream.java:180)
        at
org.apache.hadoop.io.DataOutputBuffer$Buffer.write(DataOutputBuffer.java:63)
        at
org.apache.hadoop.io.DataOutputBuffer.write(DataOutputBuffer.java:101)
        at
org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1988)
        at
org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1888)
        at
org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1934)
        at
org.apache.hadoop.chukwa.util.CopySequenceFile.createValidSequenceFile(CopySequenceFile.java:78)
        at
org.apache.hadoop.chukwa.datacollection.writer.localfs.LocalToRemoteHdfsMover.cleanup(LocalToRemoteHdfsMover.java:186)
        at
org.apache.hadoop.chukwa.datacollection.writer.localfs.LocalToRemoteHdfsMover.run(LocalToRemoteHdfsMover.java:216)
2013-01-10 07:09:24,348 INFO Timer-3 LocalWriter -
stat:datacollection.writer.local.LocalWriter dataSize=29534 dataRate=984
2013-01-10 07:09:54,349 INFO Timer-3 LocalWriter -
stat:datacollection.writer.local.LocalWriter dataSize=34440 dataRate=1147
Regards,
Shouvanik
On Wed, Jan 9, 2013 at 4:24 PM, Shouvanik Haldar <[EMAIL PROTECTED]
Thanks,
*Shouvanik*