Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> Details needed about RADIUS LOGS


Copy link to this message
-
RE: Details needed about RADIUS LOGS
I am not sure if there is a Flume Hive Sink for you to put your records into Hive directly.
https://issues.apache.org/jira/browse/FLUME-1008

Also, another approach can be to push the events to HBase with Flume-HBase Sink. [Just in case if none of your other options work for you].

From: Manu Moncy K [mailto:[EMAIL PROTECTED]]
Sent: Thursday, August 30, 2012 3:49 PM
To: [EMAIL PROTECTED]
Subject: Details needed about RADIUS LOGS

Tue Aug  7 00:00:00 2012
        User-Name = "xxxxxxxx"
        NAS-Port = xxxxxxxx
        NAS-IP-Address = xxxxxxxx
        Framed-IP-Address = xxxxxxxx
        Filter-Id = " xxxxxxxx "
        Class = " xxxxxxxx "
        NAS-Identifier = " xxxxxxxx "
        Acct-Status-Type = xxxxxxxx
        Acct-Delay-Time = 0
        Acct-Session-Id = " xxxxxxxx "
        Acct-Authentic = RADIUS
        Event-Timestamp = 1344286800
        NAS-Port-Type = Ethernet
        Calling-Station-Id = " xxxxxxxx "
        NAS-Port-Id = " xxxxxxxx "
        Service-Type = Framed-User
        Framed-Protocol = PPP
        Acct-Link-Count = 0
        RB-Agent-Circuit-Id = " xxxxxxxx "
        DSLForum-Agent-Circuit-Id = " xxxxxxxx "
        DSLForum-Access-Loop-Encapsulation = ""
        Timestamp = 1344286800
        OSC-Service-Identifier = "DSLUsers"
        Proxy-State = OSC-Extended-Id=40682
        Timestamp = 1344286800

Tue Aug  7 00:00:00 2012
        User-Name = " xxxxxxxx "
        NAS-Port = xxxxxxxx
        NAS-IP-Address = xxxxxxxx
        Framed-IP-Address = xxxxxxxx
        Class = "44620232:04:"
        NAS-Identifier = " xxxxxxxx "
        Acct-Status-Type = Stop
        Acct-Delay-Time = 0
        Acct-Input-Octets = 6021
        Acct-Output-Octets = 323749
        Acct-Session-Id = " xxxxxxxx "
        Acct-Authentic = RADIUS
        Acct-Session-Time = 1348
        Acct-Input-Packets = 53
        Acct-Output-Packets = 3187
        Acct-Terminate-Cause = User-Request
        Acct-Input-Gigawords = 0
        Acct-Output-Gigawords = 0
        Event-Timestamp = 1344286800
        NAS-Port-Type = Ethernet
        Calling-Station-Id = " xxxxxxxx "
        NAS-Port-Id = " xxxxxxxx "
        Service-Type = Framed-User
        Framed-Protocol = PPP
        Acct-Link-Count = 0
        Timestamp = 1344286800
        OSC-Service-Identifier = "DSLUsers"
        Proxy-State = OSC-Extended-Id=24386
        Timestamp = 1344286800
Above given log format (2 events given) is the RADIUS LOG I am working on, I wanted to know if there is a way i can use flume and put this log into hive in JSON format and take the required fields for each event.
--
Manu K Moncy
Data Scientist
Flutura Business Solutions Pvt. Ltd
Electronics and Communication Engineering(2008-2012)
Govt. Model Engineering College,
Cochin - 21
☎: +91-9740245341
☎: +91-9895163190
✉: [EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>
✉: [EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>
______________________________________________________________________
Disclaimer:This email and any attachments are sent in strictest confidence for the sole use of the addressee and may contain legally privileged, confidential, and proprietary data.  If you are not the intended recipient, please advise the sender by replying promptly to this email and then delete and destroy this email and any attachments without any further use, copying or forwarding
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB