Your customer is willing to consolidate their log streams (access logs application logs security
logs etc.) in one single system. Once consolidated, the customer wants to analyze these logs
in real time based on heuristics. From time to time, the customer needs to validate heuristics,
which requires going back to data samples extracted from the last 12 hours?
What is the best approach to meet your customer’s requirements?
Send all the log events to Amazon SQS. Setup an Auto Scaling group of EC2 servers to consume
the logs and apply the heuristics.
Send all the log events to Amazon Kinesis develop a client process to apply heuristics on the logs
Configure Amazon Cloud Trail to receive custom logs, use EMR to apply heuristics the logs
Setup an Auto Scaling group of EC2 syslogd servers, store the logs on S3 use EMR to apply
heuristics on the logs
Amazon Kinesis Streams allows for real-time data processing. With Amazon Kinesis Streams,
you can continuously collect data as it is generated and promptly react to critical information
about your business and operations.