Challenges with Current Enterprise Data Architecture
Current Data Architecture has become complex and prone to have multiple siloed copies of data across multitudes of tools being used for each of the point requirements. Lets look at how the current architecture has become complex over time in the diagram below:
To the disadvantage of the enterprises using the above architecture, It has led to multitude of newer issues. Some of them are listed below:
-
High Fragility and Low Tolerance to Change.
-
Low Trust Data Tables No Trust Gaurantees By Detault
-
Lack of Product Thinking (Lifecycle Management of Data Tables, Models, Customer Feedback, Revisions, Version Management etc)
-
Complexity and High Running Costs - Consumption Models hinder experimentation and innovation, when ROI is not clear
-
Non-trivial Integration Effort (Fragemented Ecosystem)
-
Friction and More munging needed before actual consumption
-
Lack of Product Level Ownership.
-
No Clear Handoff, Leading to mistrust and shifting blames when output goes wrong
-
Regulatory Risks not factored while moving or processing data.
-
Set of requirements around Data driven Automation, Advanced Analytics and Machine Learning use cases and users for the same rising.
-
Current Architecture built Primarily around Adhoc Analytics and Reporting
-
Vendor lock both storage and compute engines hampers use of other open-source and commercial engines from the platform itself.
-
Broken Data Governance, with potential regulatory risks in a rising data regulatory environment.
-
Higher Operating cost
