The solution came in the form of Prolance, a protocol surveillance tool that Knoldus built on the Rust programming language. Let’s understand what this project enabled:
1. Monitoring the running activities of the protocols
Prolance was build to monitor the activities of the following network protocols -
Prolance automated the extraction and analysis of the DHCP audit logs which monitor the network devices and prove to be an invaluable diagnostic tool when security is compromised over the network. Similarly, AD logs provide an insight into possible abuse of access rights & privileges.
2. Filtering out the logs as per monitoring requirements
Once we started processing these logs, we had two scenarios - either the user requires raw logs of the protocol or logs can be filtered as per the monitoring requirements. Take, for instance, the scenario where the user is interested in obtaining only the IP addresses and the device name from thousands of DHCP logs. Filtration will be required so that only the relevant pieces of information (like Discovery, Offer Request & Acknowledge phases) are extracted as per user requirements.
3. Continuous streaming of the logs onto the Kafka Topic after compression
Whether the logs are filtered or used raw, they go through the process of compression before being produced onto the Kafka topic. Apache Kafka has been used here for message queuing which lets you scale your processing. This happens as Kafka enables you to divide the processing over multiple consumer instances. We used the gzip compression technique and enabled the user to schedule this process as per requirement.