Demystifying the Data Mesh: Key Aspects and Strategic Considerations.

Introduction

The concept of Data Mesh and its implementation has really taken off in recent years and emerges as a response to traditional challenges in managing data within complex organizations. It proposes decentralizing data management to democratize access and expedite its utilization by business units. This is achieved by distributing responsibility for data across multiple domains, rather than centralizing it within a single team.

This decentralization aims to empower business units and self-sufficient business teams, enabling them to make quicker and more informed decisions by having direct access to relevant data. As a result, waiting times and associated bottlenecks are significantly reduced, facilitating greater organizational agility and responsiveness to market changes and opportunities.

Strategic Requirements

Successfully implementing a Mesh project requires critical support and active participation from senior management. Initiatives that impact an entire organization typically require sponsorship from those in leadership positions, or they risk failure. This sponsorship also helps promote a data-centric organizational culture.

It is common to see Data Governance projects struggle despite efforts in training, acquiring tools, and emphasizing the importance of data. If there is not genuine support from senior levels and a belief in the initiative, the project may not achieve its objectives.

Lastly, in line with the above, managing expectations and professionalism when proposing a corporate Mesh is essential. To fulfill the enticing idea behind the Mesh, the company needs a certain degree of maturity. These projects involve significant investments, restructuring, and changes in workflow dynamics.

The Outcome of a Mesh

Given that these blogs are intended for a diverse audience, I want to illustrate with a simple example one of the consequences of implementing a Mesh in your company. This objective fact may sound promising to some organizations and entirely daunting to others, but that is precisely the essence of Mesh: it is not a data architecture that applies in practice to all institutions.

Decentralizing data from the data team to business units means that some tasks formerly handled by the data team will now be led by the business units themselves. They will need to create their own data models, prepare their reports, expose their services, manage data access, and oversee data lifecycle, among other tasks. This clearly involves having specialized data personnel within the teams, in short, as you can imagine, institutional strategy is fundamental in these projects. It's about shifting from a "I give you what you asked for" approach to a "I provide you with the tools and guidelines to do it yourself." If the departmental team is not prepared for this, it will not be able to carry out its operations and will not be able to have the autonomy that is intended, which will translate into returning to the previous state.

Disadvantages of a Data Mesh

If we ignore the strategic variable and its derivatives (which we have already discussed), Mesh has mainly three additional disadvantages.

• The first is the initial complexity of such a project. The market still lacks enough professionals with real experience in the field, especially from a strategic and execution perspective, leading to few lessons learned and theory being more relevant than practice.

• The second is that maintaining a single truth is very complicated due to the distribution and fragmentation of data. Most companies still struggle with governing their information and sometimes fail to meet regulatory demands. The fragmented nature of data management in a Mesh environment makes it challenging to ensure data quality and completeness, leading to potential gaps and errors in the data.

• The third is the significant loss of economies of scale. In a Data Mesh architecture, the data stack is distributed across various domains, which can lead to inefficiencies and increased costs. Managing and integrating data from multiple sources requires more resources and coordination, which can be costly and time-consuming. 

Is Mesh for Everyone?

As a general rule, this type of architecture is intended only for large institutions with multiple business units that have the capacity and resources to manage the technical and organizational complexity of Data Mesh. Therefore, small and medium-sized enterprises are often excluded from this equation and have other more suitable options.

Highly regulated industries should proceed with caution, as the additional complexity in data governance could outweigh its potential benefits. Introducing a Mesh of data involves adding an additional level of difficulty to governance from strategic and tactical perspectives, which can end up becoming a headache for institutions that are not mature enough, especially considering the regulatory factor.

Despite the above, some regulated sectors, such as banking, have partially advanced towards the implementation of a Data Mesh approach. An example is JP Morgan, an institution that has been a pioneer in this field. It is important to note that JP Morgan has spent years making strategic investments in data management, achieving a maturity level significantly above average, which enables it to effectively mitigate associated risks and endorse such innovative initiatives (which they were actually doing before the term became popular).

At Sunny Data, we have a questionnaire that we use to assess the suitability of a Data Mesh for a client. You can contact us at the following link to request it for free.

The Data Mesh Buble and Our Conclusions 

The last trendy topics in the data world before the GPT-3.5 revolution were Data Governance and Data Mesh. In my opinion, it would have been beneficial if interest in both topics had lasted longer. Data Governance is absolutely essential in every organization, and there is still much work to be done in this area. On the other hand, Data Mesh, conceptually speaking, represents a coherent and logical approach. Empowering business units to model, explore, manage, and protect their data is the practical realization of a utopian vision.

In a final comment, our readers may have perceived throughout the blog a certain omission of the positive aspects of Data Mesh. However, at Sunny Data, we decided to focus on raising awareness among all companies exploring or in the process of implementing this approach. The reality is that there are numerous articles praising the benefits and virtues of these approaches, so we prefer to address the less discussed aspects and avoid wasting the reader's time by providing genuine, practical, and quality information.

We will be back next week with more content.