Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka, mail # user - filter before flush to disk


Copy link to this message
-
filter before flush to disk
S Ahmed 2012-05-15, 13:38
Would it be possible to filter the collection before it gets flush to disk?

Say I am tracking page views per user, and I could perform a rollup before
it gets flushed to disk (using a hashmap with the key being the sessionId,
and increment a counter for the duplicate entries).

And could this be done w/o modifying the original source, maybe through
some sort of event/listener?
+
Jay Kreps 2012-05-15, 15:24
+
S Ahmed 2012-05-15, 15:42
+
S Ahmed 2012-05-15, 15:43
+
S Ahmed 2012-05-17, 13:40
+
Jay Kreps 2012-05-17, 15:02
+
S Ahmed 2012-05-17, 21:32
+
Jay Kreps 2012-05-17, 22:34
+
S Ahmed 2012-05-29, 13:30