Datadog has launched Datadog Observability Pipelines, a new product that gives organizations more control over their data so they can reliably scale their observability practices.
Datadog Observability Pipelines is powered by Vector, an open-source, high-performance framework for building telemetry pipelines.
Organizations today are tasked with managing numerous applications, which are powered by hundreds of services and result in the creation of billions of logs, metrics, and traces per month. The large amount of data involved and the resulting application and organizational complexity can lead to reliability issues, technology lock-in, security risks and exorbitant costs.
Datadog Observability Pipelines gives customers a unified view to control and monitor the flow of all their infrastructure and application metrics, logs, and traces. Users can now seamlessly collect, enrich, transform, sanitize and route observability data from any source to any destination, before the data leaves their environment.
This unified view gives businesses better visibility into how much and where they spend, what tools they use, and who has access to what data. This allows them to more accurately manage costs, reduce technology lock-in, improve data quality compliance and standardization, and ultimately adapt their observability practices.
“We want to empower our teams to use all of our systems and tools while staying within budget. Observability Pipelines gives us flexibility in how our data is processed and where it ends up, so our teams act on the right insights as we continue to scale,” said Andreas Kasprzok, Observability Tech Lead at Blockfi.
“As the amount of telemetry continues to grow in an organization, teams are often completely overwhelmed, if not blindsided, by the exorbitant costs, reliability, and compliance risks that come from a lack of visibility into infrastructure and application data,” said Zach Sherman, senior product manager at Datadog. “We created Datadog Observability Pipelines to give organizations a powerful way to regain control of their data, without compromising visibility for engineering, security, and SRE teams.”
Datadog Observability Pipelines helps IT and security teams in their goals of affordable observability management and scaling with complete flexibility and control over how their logs, metrics, and traces are collected, transformed, and routed . It helps organizations:
- Master the costs: Aggregate, filter, and route all observability data based on use case without compromising visibility
- Simplify migrations and reduce lock-in: Orchestrate and monitor data processing from any source to any destination in a unified view
- Protect sensitive data: Filter, sanitize and monitor sensitive data before it leaves your network to better meet compliance requirements
- Apply data quality: standardize the format of logs, metrics and traces to improve observability between teams
- Grow with confidence: Scale seamlessly with a product powered by Vector, a vendor-neutral open source project, and an engaged community with millions of monthly downloads deployed in production by companies processing petabytes of data each month
- Easily collect and route data: Observability Pipelines comes with over 80 out-of-the-box integrations so organizations can quickly and easily collect and route data to any of the tools their teams already use, without disrupting existing workflows
Datadog Observability Pipelines is generally available now.