Practical guide ·
We have a SIEM — but are we detecting real attacks in time?
Assess the path from a relevant attack scenario to a usable signal and an authorised response. A SIEM collects and analyses security events, but its presence does not establish coverage. Test what it can see, who investigates and what happens when the normal responder is unavailable.
What does “working” mean for this organisation?
Start with a scenario that threatens an important service: misuse of an administrator account, unexpected access to sensitive data or a compromised endpoint reaching another system. Decide what should be visible and which action the signal should enable.
Do not begin with the total number of connected devices. A large inventory can conceal a missing identity log or an application that never sends the event needed to recognise the scenario. Coverage needs a reason, not just a percentage.
Can we follow a signal from source to decision?
Check that the source produces the event, that it arrives with usable timestamps and context, and that the relevant logic evaluates it. Then check the analyst’s view. An alert without the affected service or account owner can consume time without enabling a decision.
Use authorised simulations or controlled test events. Record the expected observation and actual outcome. A missed signal may come from a missing source, a parsing problem, unsuitable logic or an operational handover failure; each needs a different correction.
| Stage | Question | Useful evidence |
|---|---|---|
| Source | Was the expected event recorded? | Source event and timestamp |
| Detection | Did the agreed logic recognise it? | Test result and configuration |
| Investigation | Could the analyst establish context? | Traceable findings |
| Response | Could someone take the required action? | Escalation record and authority |
How should we evaluate an AI feature?
Ask the provider to demonstrate a defined task on representative, appropriately protected data. Compare the result with a baseline: useful investigation time, missed relevant events and unsupported conclusions. Include the effort needed to verify the AI output.
The SIEMvolution research listed below concerns AI enhancements in managed security. For a purchasing decision, require evidence from your environment. A fluent summary is useful only if its claims can be traced to events and it helps an authorised person act.
What should the service agreement make clear?
Name the monitoring hours, supported sources, investigation boundary and escalation contacts. Define who may disable an account or isolate a device. Clarify what happens outside coverage and which failures are visible to the customer.
Report the tested scenarios and remaining blind spots alongside performance measures. If a provider reports only alert volume, ask what business decision that number supports. The next investment may be improved telemetry or clearer response authority rather than another platform feature.
Explore in detail: AI in security monitoring: what it improves and what it does not · We have MFA. Why that is not enough.
How Dyasol can help
Dyasol can review monitoring architecture and operational responsibilities within an agreed scope. This is an assessment and improvement engagement; it does not imply a bundled 24/7 SOC service.
Sources and context
Examples are illustrative. Practical recommendations should be adapted to the organisation.