Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> aggregation by time window


Copy link to this message
-
aggregation by time window
Hi ,
    I have such row data structure:

event_id  |   time
=============event1     |  10:07
event2     |  10:10
event3     |  10:12

event4     |   10:20
event5     |   10:23
event6     |   10:25

Numbers of records is  50-100 million.

Question:
   I need to get events that was during time T.

For example: if T=7 munutes.
     event1 , event2 , event3 were detected durint 7 minutes.
     event4 , event5 , event6 were detected during 7 minutes.

How can I implement such aggregation using map/reduce.

Thanks
Oleg.
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB