By Lee Wearne, Data Strategist at Insight Consulting
The world is more polarised than ever before, with global conflict and geopolitical tensions drawing strong lines between regions and even within countries. Conflict, social unrest, inflation, climate change, and much more, are leading the world towards a trend of de-globalisation on a scale not many anticipated as recently as 10 or 20 years ago. The effect of this is that power and data are becoming fragmented.
The result is that businesses may well be aware of data fabrics, data meshes and modern data stacks and may feel inclined to rush towards these technology solutions, but unless they address the cultural obstacles within their organisations and embrace a socio-technical approach, their investments and efforts are likely to be in vain.
How did we get here? Our new ever-fragmented reality leads to uncertainty in technology, which is naturally reacting to what’s happening in the world. As a result of the uncertainty and rapid changes in technology – which includes artificial intelligence (AI) and the decentralised blockchain and web 3.0 – the global skills shortage bites harder than ever before. In addition to this, regulations have become more complicated with even more red tape around rules about where data may and may not be stored in our fragmented world.
The impact here, and most organisations would attest to the fact that they are likely flooded by large and complex datasets from various sources. They have difficulty integrating and managing data from different systems precisely because the data may be stored in different formats, structures, and locations. In addition to this, locating data from different systems in a large organisation can be challenging, while rapidly changing compliance regulations make it difficult to comply.
The only way to succeed, in 2024 and beyond, is to have a clear and comprehensive data management strategy.
Data management refers to the process of collecting, storing, organising and maintaining data to support analysis and decision-making. Integrating a decentralised data world means there has to be interoperability between platforms and applications.
But what does this mean for organisations that need a clear and comprehensive data management strategy? It means they need a fabric or mesh to help them govern and control data. The more decentralised and fragmented the world gets, the more technology is trying to weave it back together.
Data fabrics, data meshes exist to ease the challenges of managing data and to make sense of a multipolar and decentralised world. Fabric and meshes are two different approaches with the intention to ease the challenges of data management in a multi polar decentralised world. Modern data stacks are a collection of tools that enable organisations to collect, process, store and analyse data. These form part of a mesh or fabric data management strategy.
Data fabric is a tech-centric architecture for data management that unifies and integrates data across multiple systems. Data fabric uses a variety of approaches to create a unified data management system that allows organisations to access, process, and share data more efficiently.
Data mesh, on the other hand, is a decentralised data architecture where data is treated as a product and managed by dedicated data product owners. This approach transfers the responsibility from the central data team to the business units that create and consume data.
To improve the odds of successfully building an effective data management strategy, working with a trusted and experienced data partner to help shift the organisation’s data culture is a crucial – and often missing – step. The Data and Analytics Leadership Annual Executive Survey 2023 found that cultural factors are the biggest obstacle to delivering value from data investments.
Data fabrics, meshes and modern data stacks will continue to consolidate an increasingly decentralised world by making the management of data easier. However, to ensure control over security and governance, and to extract value from data that is trustworthy requires a tactical shift to what we call a socio-technical approach. In other words, any strategy must be made up of an investment in people, process and technology to be successful.
This is because data management involves more than the technical aspects of data storage, processing and analysis. It also includes the social aspects of data governance, change management, data quality management, user upskilling and collaboration between different teams. Organisations that know how to use technology the best will have an edge over their competitors.
Organisations would do well to engage with data partners who embrace a socio-technical approach to data management if they’d like to improve their odds of deriving value from data and extracting insights that can help them make better business decisions.
