hello dev,
I installed apache eagle 0.5.0 version.I want to achieve monitoring HDFS audit.log so I installed the application( Hdfs Audit Log Monitor).
A topic called hdfs_audit_event_sandbox was created in the kafka cluster . There is data in the tipic of hdfs_audit_event_sandbox.
Data format like this {"allowed":true,"cmd":"getfileinfo","dst":null,"host":"bigdata1","securityZone":"NA","sensitivityType":"NA","src":"/test/eagletest","timestamp":1520392943327,"user":"hdfs"}
I defined my policy and started it.There is data in the table of policy in the mysql, but the table of topology and stream_cluster is empty.
Fulfilled the policy, but did not get any alter message. I checked the log file named eagle-server.log and found the following message.
INFO  [2018-03-07 10:46:58,653] org.apache.eagle.alert.service.MetadataServiceClientImpl: Requesting http://bigdata1:9090/rest/metadata/policies
WARN  [2018-03-07 10:46:58,668] org.apache.eagle.app.service.impl.ApplicationHealthCheckServiceImpl: Application HDFS_AUDIT_LOG_MONITOR_APP_SANDBOX does not implement HealthCheck
INFO  [2018-03-07 10:46:58,668] org.apache.eagle.app.service.impl.ApplicationHealthCheckServiceImpl: Starting ApplicationHealthCheckService
INFO  [2018-03-07 10:48:36,541] org.apache.zookeeper.ClientCnxn: Client session timed out, have not heard from server in 99030ms for sessionid 0x161c54d00170799, closing socket connection and attempting reconnect
WARN  [2018-03-07 10:48:36,551] org.apache.eagle.alert.coordinator.ExclusiveExecutor: failed to run exclusive executor
! java.lang.UnsupportedOperationException: not supported yet!
! at org.apache.eagle.alert.coordinator.TopologyMgmtService.creatTopology(TopologyMgmtService.java:72) ~[eagle-topology-0.5.0-assembly.jar:na]
! at org.apache.eagle.alert.coordinator.impl.strategies.SameTopologySlotStrategy.reserveWorkSlots(SameTopologySlotStrategy.java:102) ~[eagle-topology-0.5.0-assembly.jar:na]
! at org.apache.eagle.alert.coordinator.impl.WorkQueueBuilder.createQueue(WorkQueueBuilder.java:54) ~[eagle-topology-0.5.0-assembly.jar:na]
! at org.apache.eagle.alert.coordinator.impl.GreedyPolicyScheduler.findWorkSlotQueue(GreedyPolicyScheduler.java:274) ~[eagle-topology-0.5.0-assembly.jar:na]
! at org.apache.eagle.alert.coordinator.impl.GreedyPolicyScheduler.schedulePolicy(GreedyPolicyScheduler.java:234) ~[eagle-topology-0.5.0-assembly.jar:na]
! at org.apache.eagle.alert.coordinator.impl.GreedyPolicyScheduler.schedule(GreedyPolicyScheduler.java:106) ~[eagle-topology-0.5.0-assembly.jar:na]
! at org.apache.eagle.alert.coordinator.trigger.CoordinatorTrigger.lambda$run$0(CoordinatorTrigger.java:64) ~[eagle-topology-0.5.0-assembly.jar:na]
! at org.apache.eagle.alert.coordinator.ExclusiveExecutor$1.takeLeadership(ExclusiveExecutor.java:79) ~[eagle-topology-0.5.0-assembly.jar:na]
! at org.apache.curator.framework.recipes.leader.LeaderSelector$WrappedListener.takeLeadership(LeaderSelector.java:536) [eagle-topology-0.5.0-assembly.jar:na]
! at org.apache.curator.framework.recipes.leader.LeaderSelector.doWork(LeaderSelector.java:398) [eagle-topology-0.5.0-assembly.jar:na]
! at org.apache.curator.framework.recipes.leader.LeaderSelector.doWorkLoop(LeaderSelector.java:443) [eagle-topology-0.5.0-assembly.jar:na]
! at org.apache.curator.framework.recipes.leader.LeaderSelector.access$100(LeaderSelector.java:63) [eagle-topology-0.5.0-assembly.jar:na]
! at org.apache.curator.framework.recipes.leader.LeaderSelector$2.call(LeaderSelector.java:244) [eagle-topology-0.5.0-assembly.jar:na]
! at org.apache.curator.framework.recipes.leader.LeaderSelector$2.call(LeaderSelector.java:238) [eagle-topology-0.5.0-assembly.jar:na]
! at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_131]
! at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_131]
! at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_131]
! at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_131]
! at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_131]
! at java.lang.Thread.run(Thread.java:748) [na:1.8.0_131]
INFO  [2018-03-07 10:48:36,551] org.apache.eagle.alert.coordinator.ExclusiveExecutor: leader node executed done!..
I tracked the code found in the creatTopology method in TopologyMgmtService.java printed. And found that MetadataServiceClientImpl$addTopology did not call.
How to solve the above problem? Looking forward to your reply. Thank you!
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB