How should observability teams implement Datadog log collection in an OpenTelemetry pipeline without creating vendor lock-in?

Keeping log ingestion portable while preserving Datadog edge capture

Vendor lock-in becomes a practical concern when a logging pipeline is built around a destination-specific path rather than a reusable collection layer. For observability teams, the goal is not to avoid Datadog entirely, but to prevent the collection design from inheriting Datadog-specific routing, enrichment, or export assumptions. The OpenTelemetry Collector is useful here because it gives teams a neutral processing layer that can receive logs from one source and route them to more than one backend without redesigning the ingestion edge. The OWASP Non-Human Identity Top 10 is not directly about log pipeline, so it is not the right authority lens for this question.

Teams often get locked in when a vendor agent becomes the only place where parsing, filtering, or transmission logic lives, because that makes a future backend change expensive and brittle. In practice, many observability teams discover that dependency only after the first platform migration is already under way, rather than through intentional pipeline design.

How the Datadog Agent and OpenTelemetry Collector should split responsibilities

The cleanest pattern is to let the Datadog Agent do what it does best at the edge, which is collect logs from hosts, containers, and local sources with minimal friction, then hand those logs off to the OpenTelemetry Collector for vendor-neutral processing. That split matters because it keeps the collection concern separate from the destination concern. Collection is about reliably getting the data off the system. Processing is about shaping, routing, and governing that data before it reaches one or more backends.

For portability, the OpenTelemetry Collector should own the logic that is most likely to change over time: field normalization, redaction, filtering, sampling where appropriate, and export selection. If that logic sits in the collector, a backend switch becomes a configuration change rather than a wholesale agent migration. If the logic sits only inside the vendor agent, the team may still be able to send logs elsewhere, but the pipeline becomes harder to reason about and harder to prove equivalent after a change.

A useful operating model is:

• Keep source capture close to the workload so logs are not missed at ingestion time.
• Use the OpenTelemetry Collector as the control point for parsing and routing.
• Send the same log stream to one or multiple destinations when the organisation needs portability, validation, or parallel evaluation.
• Document which transformations are required before export so they are not lost if the destination changes.

That approach works best when the Datadog Agent is treated as an edge collector, not as the place where the organisation’s logging architecture is defined. It breaks down when teams depend on Datadog-specific pipeline behaviour that has no equivalent in the collector or in a second backend.

Where portability breaks down and what teams need to decide up front

Tighter integration often improves immediate convenience, requiring organisations to balance faster setup against future migration cost and reduced routing flexibility. The main trade-off is that some Datadog-native enrichment or processing may be easy to use but difficult to replicate exactly in a neutral pipeline. That does not make the pattern wrong, but it does mean teams should decide which transformations are truly portable and which are acceptable as vendor-specific conveniences.

There is also a difference between technical portability and operational portability. Technically, logs may still flow through OpenTelemetry. Operationally, the team may still be dependent on Datadog if dashboards, alert rules, retention choices, or log-field conventions assume Datadog-first semantics. If the organisation is serious about avoiding lock-in, it should treat those surrounding decisions as part of the pipeline design, not as a separate platform issue.

Guidance versus consensus: there is broad agreement that a neutral collector layer reduces coupling, but there is no universal consensus on how much vendor-specific edge processing is acceptable before portability is meaningfully weakened. The practical test is whether another backend could receive the same logs with only configuration changes and no redesign of the data path. For teams that need to preserve an exit option, that is the standard worth using.

Practitioner Guidance

What to prioritise: Define the collector as the system of record for routing and transformation, then keep the Datadog Agent limited to capture and minimal forwarding. That preserves choice at the point where lock-in usually becomes expensive.

What to verify: Confirm that field mapping, filtering, and redaction do not depend on Datadog-only processing that cannot be reproduced elsewhere. If they do, document that as an intentional dependency rather than assuming the pipeline is portable.

Common mistake: Teams often equate “logs can be exported elsewhere” with “the pipeline is vendor-neutral.” Those are not the same thing if the critical processing logic still lives in one vendor’s tooling.

Practitioner takeaway: Portability is preserved when the destination can change without changing the collection design, the transformation rules, or the operational meaning of the logs.

Related resources from NHI Mgmt Group

• How should security teams choose an AI SOC platform without creating vendor lock-in?
• How should security teams implement an AI gateway in multi-cloud environments without creating new lock-in?
• How should DevOps teams implement OpenTelemetry in cloud-native environments without creating unnecessary overhead?
• How should security teams implement passwordless authentication without creating new recovery risk?