Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Chukwa >> mail # user >> More about the removing of duplicate chunks


Copy link to this message
-
More about the removing of duplicate chunks
Thanks to the simple archiver , we do remove almost all the duplicate
chunks.

But we found that there are still few ,very few duplicate chunks left .

And strangely , these chunks's key are't the same. The DataType,StreamName
and SeqId are the same , but the TimePartition are different. The log in
these chunks are the same.

Could we just distinguish the duplicate chunks using
the DataType,StreamName and SeqId ? What's the TimePartition meaning for?

Thanks!
--
Best regards,

Ivy Tang
+
Ariel Rabkin 2012-03-28, 13:03