Basic Log Management service – what and why?

We at CYBERS see many small and medium-sized organizations struggling to find a way of establishing and building logging infrastructure.

There is a wide range of offerings in the field of SIEM’s. Most of them do not come cheap and interest is filled with horror stories of SIEM projects that started off on the wrong foot. The topic itself can be overwhelming.

Which solution to choose and what to log? The answers may vary in great lengths and there is no single truth. One size does not fit all, but all organizations need to start from a similar ground. Not every organization needs a full-blown SIEM to start off. Most organizations need just a little bit of guidance and a simple strategy to start with the management of application and security logs to start creating immediate value. Since logging and log needs change with the organization a solid foundation is needed. The basic ones, twos, and threes for almost all logging needs are similar and can be narrowed down into easy bite-size chunks. 

Basic questions to answer:

• Where do logs come from? What sources do we need to include in the logging strategy?
  – What applications? Servers? Devices?
• Where do logs go to and find their peace?
  – Central place where to forward logs. On-prem? Cloud?
• How long do we need to store the logs? Do we have compliance needs that help dictate that?
  – The retention period and (initial) capacity planning.
• What are the first (business) use cases and who are the first log consumers in the organization?
  – What reports do they need? What alerts do they need?

Bear in mind that all those answers are subject to change as time goes on. More questions arise as appetite changes and maturity level grows. That is normal. Stay calm and log on.

The Source(s)

Log sources that are first included in the initial logging project need to be identified and mapped out. They can be either applications (cloud or on-prem), servers, services, devices (firewalls, …), and/or workstations. They need to be grouped into the same types to get an idea of how many unique types of sources there are and what is the volume. This highly affects the capacity and complexity of the initial project. At this stage prioritization and exclusion of onboarding, log sources occur. If we are unable to identify ANY consumer of the information, then why bother? In the list of log sources, there are most likely some standard pieces of software that produce logs as well as customer applications. During the mapping process, there is a need for both technical and analytical input. Questions that typically arise in this stage are:

• How do we fetch/get the logs? What are the possibilities for log transport?
• Do we get the logs that we need? Logging policy? Does the source produce what we need or is some change needed?
• Do we need to create a customer parser?

And behold! Logging baseline policy and configuration just happened.

Where do logs go?

Whether it is on-prem or cloud the logs usually need to go into a central place where the magic happens. Since the topic we are discussing here is Basic Log Management then the most basic central point is a Syslog server that pushes logs into files. This is okay when you need logs for a pure archiving use case with a possibility of rare searches.

A big step up from that is to use Elastic stack. The basic entry to Elastic is free to use. The ELK stack holds all the tools for getting logs from various sources whether they would be Windows or Linux servers or workstations, Syslog sources, or something else. Elastic indexes and stores the logs in a way that they can be accessed and searched fast in an easy-to-use web interface. Report and query generation is a breeze. This makes logs accessible to a wide range of users that can dig through logs to make use of the data otherwise left untouched.

We at CYBERS  aim to help organizations of different sizes in establishing log management and with our services we guide you through the process of log source identification and analytical tasks, parser writings, and ELK implementation. After initial log management and use, cases have been established and the foundation put in place it is easy to build a roadmap into further development and gap filling.