Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Chukwa >> mail # user >> How to check whether chukwa collecting logs


Copy link to this message
-
Re: How to check whether chukwa collecting logs
Can you please help me find out why the following log is generated by the
collector

2013-01-10 07:09:23,864 INFO Timer-1 root -
stats:ServletCollector,numberHTTPConnection:18,numberchunks:166
2013-01-10 07:09:23,884 WARN LocalToRemoteHdfsMover CopySequenceFile -
Error during .chukwa file recovery
java.io.EOFException
        at java.io.DataInputStream.readFully(DataInputStream.java:180)
        at
org.apache.hadoop.io.DataOutputBuffer$Buffer.write(DataOutputBuffer.java:63)
        at
org.apache.hadoop.io.DataOutputBuffer.write(DataOutputBuffer.java:101)
        at
org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1988)
        at
org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1888)
        at
org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1934)
        at
org.apache.hadoop.chukwa.util.CopySequenceFile.createValidSequenceFile(CopySequenceFile.java:78)
        at
org.apache.hadoop.chukwa.datacollection.writer.localfs.LocalToRemoteHdfsMover.cleanup(LocalToRemoteHdfsMover.java:186)
        at
org.apache.hadoop.chukwa.datacollection.writer.localfs.LocalToRemoteHdfsMover.run(LocalToRemoteHdfsMover.java:216)
2013-01-10 07:09:23,891 INFO LocalToRemoteHdfsMover LocalToRemoteHdfsMover
- Deleted .recover file,
/tmp/chukwa/dataSink/201308112441296_domU12313907193B_216836a313c19d565ee7ff8.recover
2013-01-10 07:09:23,891 INFO LocalToRemoteHdfsMover LocalToRemoteHdfsMover
- Copying .chukwa file to valid sink file before moving,
/tmp/chukwa/dataSink/201310063103378_domU1231390B80BA_1082291f13c22c778057f42.chukwa
2013-01-10 07:09:23,899 WARN LocalToRemoteHdfsMover FSInputChecker -
Problem opening checksum file:
/tmp/chukwa/dataSink/201310063103378_domU1231390B80BA_1082291f13c22c778057f42.chukwa.
Ignoring exception: java.io.EOFException
        at java.io.DataInputStream.readFully(DataInputStream.java:180)
        at java.io.DataInputStream.readFully(DataInputStream.java:152)
        at
org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.<init>(ChecksumFileSystem.java:134)
        at
org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:283)
        at
org.apache.hadoop.io.SequenceFile$Reader.openFile(SequenceFile.java:1495)
        at
org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1482)
        at
org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1475)
        at
org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1470)
        at
org.apache.hadoop.chukwa.util.CopySequenceFile.createValidSequenceFile(CopySequenceFile.java:73)
        at
org.apache.hadoop.chukwa.datacollection.writer.localfs.LocalToRemoteHdfsMover.cleanup(LocalToRemoteHdfsMover.java:186)
        at
org.apache.hadoop.chukwa.datacollection.writer.localfs.LocalToRemoteHdfsMover.run(LocalToRemoteHdfsMover.java:216)

2013-01-10 07:09:23,931 WARN LocalToRemoteHdfsMover CopySequenceFile -
Error during .chukwa file recovery
java.io.EOFException
        at java.io.DataInputStream.readFully(DataInputStream.java:180)
        at
org.apache.hadoop.io.DataOutputBuffer$Buffer.write(DataOutputBuffer.java:63)
        at
org.apache.hadoop.io.DataOutputBuffer.write(DataOutputBuffer.java:101)
        at
org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1988)
        at
org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1888)
        at
org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1934)
        at
org.apache.hadoop.chukwa.util.CopySequenceFile.createValidSequenceFile(CopySequenceFile.java:78)
        at
org.apache.hadoop.chukwa.datacollection.writer.localfs.LocalToRemoteHdfsMover.cleanup(LocalToRemoteHdfsMover.java:186)
        at
org.apache.hadoop.chukwa.datacollection.writer.localfs.LocalToRemoteHdfsMover.run(LocalToRemoteHdfsMover.java:216)
2013-01-10 07:09:24,348 INFO Timer-3 LocalWriter -
stat:datacollection.writer.local.LocalWriter dataSize=29534 dataRate=984
2013-01-10 07:09:54,349 INFO Timer-3 LocalWriter -
stat:datacollection.writer.local.LocalWriter dataSize=34440 dataRate=1147
Regards,
Shouvanik
On Wed, Jan 9, 2013 at 4:24 PM, Shouvanik Haldar <[EMAIL PROTECTED]
Thanks,
*Shouvanik*
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB