Basic Log Management service – what and why?
We at CYBERS see many small and medium sized organizations struggling to find a way of establishing and building logging infrastructure. There is a wide range of offerings in the field of SIEM’s. Most of them do not come cheap and intrest is filled with horror stories of SIEM projects started off on the wrong foot. The topic itself can be overwhelming. Which solution to choose and what to log? The answers may vary in great lengths and there is no single truth. One-size does not fit all, but all organizations need to start from similar ground. Not every organization need a full blown SIEM to start off. Most organizations need just a little bit guidance and a simple strategy to start with management of application and security logs to start creating immediate value. Since logging and log needs change with the organization a solid foundation is needed. The basic ones, twos, and threes for almost all logging needs are similar and can be narrowed down into easy bite size chunks.
Basic questions to answer:
- Where logs come from? What sources do we need to include into logging strategy?
– What applications? Servers? Devices?
- Where logs go to and find their peace?
– Central place where to forward logs. On-prem? Cloud?
- How long do we need to store the logs? Do we have compliance needs that help dictate that?
– The retention period and (initial) capacity planning.
- What are the first (business) use cases and who are the first log consumers in organization?
– What reports do they need? What alerts they need?
Bear in mind that all those answers are subject to change as time goes on. More questions arise as appetite changes and maturity level grows. That is normal. Stay calm and log on.
Log sources that are first included in the initial logging project need to be identified and mapped out. They can be either applications (cloud or on prem), servers, services, devices (firewalls, …) and/or workstations. They need to be grouped into same types to get an idea of how many unique types of sources there are and what is the volume. This highly affects the capacity and complexity of the initial project. At this stage prioritization and exclusion of onboarding log sources occurs. If we are unable to identify ANY consumer of the information, then why bother? In the list of log sources there are most likely some standard pieces of software that produce logs as well as customer applications. During mapping process there is a need of both technical and analytical input. Questions that typically arise in this stage are:
- How do we fetch/get the logs? What are the possibilities for log transport?
- Do we get the logs that we need? Logging policy? Does the source produce what we need or is some change needed?
- Do we need to create a customer parser?
And behold! Logging baseline policy and configuration just happened.
Where logs go?
Whether it is on-prem or cloud the logs usually need to go into a central place where magic happens. Since the topic we are discussing here is Basic Log Management then the most basic central point is a syslog server that pushes logs into files. This is okay when you need logs for pure archiving use case with a possibility of rare searches.
Big step up from that is to use Elastic stack. The basic entry to Elastic is free to use. The ELK stack holds all the tools for getting logs from various sources whether they would be Windows or Linux servers or workstations, syslog sources or something else. Elastic indexes and stores the logs in a way that they can be accessed and searched fast in an easy-to-use web interface. Report and query generation is breeze. This makes logs accessible to a wide range of users that can dig through log to make use of the data otherwise left untouched.
We at CYBERS aim to help organizations of different sizes in establishing log management and with our services we guide you through the process of log source identification and analytical tasks, parser writings and ELK implementation. After initial log management and use cases have been established and foundation put to place it is easy to build a roadmap into further development and gap filling.