Define the service you expect
Providers use managed detection and response and security operations centre labels for different combinations of technology, analysis and response. Compare the written scope. Identify which endpoints, identities, cloud services and network sources are included, who maintains the integrations and what happens when a source stops reporting.
A proposal can cover continuous monitoring while leaving containment entirely to your team. That can be a valid arrangement if the responsibility and contact route work in practice. It becomes a problem when the buyer assumes response is included and the provider assumes the client will act.
Choose scenarios from your business exposure
Start with a few events whose missed detection would affect a real service. For example, an unexpected mailbox forwarding change, a privileged role assignment, suspicious endpoint activity or interruption of an important log source. Agree the exact event, permitted test method and expected evidence with the provider before running anything.
Use approved test accounts and safe exercises. Do not introduce live malware or surprise a production response team to obtain a dramatic demonstration. The point is to test the contracted chain under controlled conditions and record what the exercise does and does not establish.
Follow the evidence through the chain
| Stage | Demonstration to request |
|---|---|
| Collection | The event reaches the intended data source with a usable timestamp |
| Detection | A relevant rule or analytic creates an identifiable result |
| Analysis | The analyst explains the context, uncertainty and priority |
| Escalation | The agreed contact receives an actionable notification |
| Response | The authorised party decides and performs the agreed action |
| Closure | The record links evidence, outcome and any follow-up work |
Measure these stages separately. A rule may work while the analyst lacks context; an analyst may recognise the issue while the escalation contact is unavailable. Our SIEM effectiveness guide explains why data collection and operational detection are different checks.
Define the clocks in the contract
Ask when each promised response time begins and ends. Event occurrence, ingestion, alert creation, analyst acknowledgement and customer notification are different timestamps. Clarify time zones, severity categories, exceptions and the treatment of missing telemetry. An average across easy cases can conceal a delay in an important one.
Do not substitute a sales demonstration for an acceptance test in the agreed environment. A fictional example illustrates the distinction: a test alert reaches an analyst quickly, but the notification is sent to an unmonitored shared mailbox. Detection succeeded; the response chain did not. There is no need to invent a universal minute target to recognise that gap.
Keep authority and access explicit
Record who may isolate a device, block an account or change a rule, and under which conditions. Include an alternate approver and a route for a decision that would interrupt a critical service. The incident authority guide provides a starting point for those boundaries.
Ask who can access raw records, how long agreed evidence remains available and what can be exported when the contract ends. Identify dependencies on licences, storage and provider-specific tooling. A report that cannot be traced to supporting records limits both investigation and later review. CISA's guidance for managed-service customers also emphasises access levels and contingency planning with suppliers.
Use a scorecard without false precision
Download the MDR and SOC acceptance scorecard. For each agreed scenario it records expected behaviour, observation, evidence, result, owner and retest date. Use “passed”, “failed” or “not tested” with an explanation. An untested scenario must not silently count as successful.
Keep results separate by scenario. A single percentage obscures which critical path failed and whether the sample represents your environment. Before purchase, define which unresolved findings prevent acceptance and which can enter a dated improvement plan. Repeat the relevant test after a material source, rule or responsibility change.
What Dyasol's role can be
Dyasol can help define evaluation scenarios, review proposals and assess the evidence under an agreed security improvement engagement. This article does not offer a Dyasol 24/7 SOC. The chosen provider's monitoring and response obligations must be in its contract.
Where AI is part of the service, also ask how analyst suggestions are checked and what actions the model can trigger. Our AI in security monitoring article addresses that additional layer. A useful purchasing decision rests on demonstrated behaviour and clear responsibilities, supported by the tools rather than inferred from their names.