Data observability - the rise of the data guardians

In MMC’s latest research report, we have outlined what you need to know about data observability, why you should care and some of the market dynamics at play, sharing our learnings from over 40 interviews with vendors, data practitioners and industry experts.

For many years, MMC has been focused on AI and its potential to shape a wide range of sectors. However, a bottleneck to realising the benefits of these data-driven applications is bad data quality; garbage in, garbage out. Bad data is becoming progressively harder to guard against as data volumes skyrocket and data pipelines grow ever more complex. Gartner now estimates that bad data quality causes a business to lose, on average, $12.9m a year. As a result of this growing pain point, a new product category has emerged to protect companies against bad data: data observability.

The north star for data observability solutions is to reduce the time to detect and resolve data quality issues.

Data observability solutions work by collecting data from the data warehouse and other sources in the data stack, utilising machine learning to monitor data quality. The data observability category is still young and relatively small, but we expect substantial growth. We also expect rapid innovation to continue, where an exciting area here are developments in technologies that prevent data quality issues from occurring in the first place. However, there is also potential disruption from larger tech players and consolidation appears to be on the horizon.

We at MMC have identified 26 start-ups and open-source projects in this space, all with a unique approach. These solutions have been met with much enthusiasm from data teams, and have also had a warm reception from investors raising over $370m in their latest financing rounds.

Want to learn more about data observability? Read the full report

TLDR: Five things you need to know about the data observability space

Large greenfield opportunity
The data observability category is still at its nascent stages, but we expect massive growth going forward as every sector will increase its adoption of analytics and AI where data quality is paramount.
Still a ton of innovation to come
Leading vendors are coming up with new functionality every quarter, as well as building out their suite of integrations. A particularly exciting area here is ‘shift left’ developments, which involve technologies that catch and prevent bad data earlier in the data lifecycle, before it can cause harm downstream.
Consolidation on the horizon
We believe we will see substantial consolidation of data solutions over the next five years. Specifically, we see a natural category merge between data observability and data catalogues due to overlapping capabilities and potential synergies. This potentially means some significant exits in the coming years, but that vendors also need to build out their products with half an eye on where they might fit in the landscape as the market heads towards a consolidation phase.
Disruption from larger tech players
There are worries that players within the modern data stack, such as dbt and Snowflake, could infringe on the data observability space as they are providing some similar capabilities. In the medium term, we think they may continue to expand these capabilities, but will remain confined to their area of the stack. In contrast, data observability will have an advantage of covering the stack more broadly. However, the wildcards are the established software observability companies, such as Datadog, who many believe their next logical step is to enter the data observability space.
The operational technology market opportunity
We see a significant opportunity for data observability in the operational technology (OT) market (e.g. energy, manufacturing sectors). This is due to the massive OT market becoming increasingly data-driven. Most of the IT data observability players we discuss in the report are not at all focused on the OT opportunity today, and we do not expect them to do so over the next few years. We therefore believe this will remain a separate category likely to see its own winners emerge.