IT PARK
    Most Popular

    Wireless charging principle

    May 08, 2025

    Is the enterprise ready to protect its cloud computing?

    Apr 18, 2025

    How to prove you're human in the AI jungle?

    May 14, 2025

    IT PARK IT PARK

    • Home
    • Encyclopedia

      What is the hosts file? Where is the hosts file?

      May 19, 2025

      Apple phone into the water how to do? Four first aid measures to help you

      May 18, 2025

      A one-minute walk through the difference between a switch and a router

      May 17, 2025

      What are the Wi-Fi password security levels?

      May 16, 2025

      What is Qualcomm three carrier aggregation

      May 15, 2025
    • AI

      Can AI work this round when you ask a doctor online to break a disease?

      May 19, 2025

      NASA is developing an artificial intelligence interface where astronauts can talk directly to AI

      May 18, 2025

      76-year-old father of deep learning Hinton left Google! Publishes AI threat theory, pessimistic prediction of catastrophic risk

      May 17, 2025

      What is the neural network of artificial intelligence?

      May 16, 2025

      What is the core issue of AI technology?

      May 15, 2025
    • Big Data

      Has the development of big data come to an end?

      May 19, 2025

      How Research Institutes Should Use Data Analytics Tools to Improve Research Efficiency

      May 18, 2025

      How to Program Big Data Effectively

      May 17, 2025

      Five database concepts, read the database layout of Amazon Cloud Technologies

      May 16, 2025

      What is streaming data?

      May 15, 2025
    • CLO

      Remote work and cloud computing create a variety of endpoint security issues

      May 19, 2025

      Three common misconceptions about sustainability and cloud computing

      May 18, 2025

      Ten Ways Cloud-Native Development is Changing Cybersecurity

      May 17, 2025

      What is a multi-cloud network?

      May 16, 2025

      Cloud computing kicks off sports revolution, market could reach $5.2 billion

      May 15, 2025
    • IoT

      Internet of Things and the Elderly

      May 19, 2025

      The Future of the Internet of Things and Self-Storage

      May 18, 2025

      Skills shortage remains the biggest barrier to IoT adoption in the oil and gas industry

      May 17, 2025

      Why the Metaverse Matters for the Future of Manufacturing

      May 16, 2025

      6 Ways the Internet of Things is Transforming Agriculture

      May 15, 2025
    • Blockchain

      Blockchain Wallet

      May 19, 2025

      Scientists propose quantum proof-of-work consensus for blockchain

      May 18, 2025

      How blockchain technology can be applied to environmental protection to drive a green economy

      May 17, 2025

      After the collision between quantum computing and blockchain - quantum blockchain

      May 16, 2025

      How to Use Blockchain Technology to Enhance Data Security

      May 15, 2025
    IT PARK
    Home » Big Data » Has the development of big data come to an end?
    Big Data

    Has the development of big data come to an end?

    The data grid can overcome many challenges inherent in big data by driving higher levels of autonomy and data engineering alliances among a wider range of stakeholders. However, big data is not a panacea, it brings a series of risks for enterprises to manage.
    Updated: May 19, 2025
    Has the development of big data come to an end?

    The data grid can overcome many challenges inherent in big data by driving higher levels of autonomy and data engineering alliances among a wider range of stakeholders. However, big data is not a panacea, it brings a series of risks for enterprises to manage.

    For many enterprises, data is a huge resource that has hardly been developed. Many institutions and organizations have realized that data is a key asset, and it is more important than ever to use the insight gained from enterprise data. In practice, innovators, disruptors and start-ups are much more flexible in using data to change, compete and win the market. Moreover, if they make good use of these data, they will gain more customers. They may not have as much data as large enterprises, but they are more able to use it.

    Big data is a popular term, which defines a series of methods that many enterprises use to develop solutions to generate the required insight. However, according to the statistics of Gartner and other institutions, most companies fail to achieve their goals through big data methods.

    The core of the big data approach is to focus on ingestion, transformation, governance and insight in most of the enterprise's data. This will lead to bottlenecks that significantly inhibit the delivery of business value within a meaningful time frame. Instead of facilitating the flow of data, it has been stifled.

    So is there any other choice?

    What if there was a new approach based on federalism rather than centralization to help enterprises gain the insight they need to remain competitive?

    For many enterprises, the data grid approach solves the challenges they face. The core of data grid is a data federation method based on the proven and tested principles of software engineering. Many enterprises have applied it to customer journey development.

     

    Three principles of data grid

    The data grid method utilizes three core principles of modern software engineering:

    Domain ownership

    product development

    Self service software platform

    These principles enable the development of data solutions to be united, which can unlock important and greater insights faster, so that enterprises can realize business value.

     

    (1) Domain ownership

    This follows the current domain modeling principles and adds data coverage to the model. The domain model is a visual representation of key concepts/objects in the problem domain, which is the highest level of the enterprise. It decomposes an enterprise and establishes clear ownership and boundaries for business functions and technical solutions. This enables microservice based software engineering methods to drive autonomy and reuse.

    The same domain model can be used within the enterprise to establish ownership of domain datasets. Each dataset should belong to the domain where the data was created/generated. The goal is to make each dataset owned by a single domain, and adjust the domain model where necessary to achieve this result.

    (2) Product development

    This shows that enterprises treat data as products, just as they treat customer journeys as products.

    The focus is on what customers want to do and the best solution to help them do it. A product team is a group of individuals with multiple skills who bring together business and technical personnel to create the best possible customer results.

    Applying this to data means understanding different data roles in the enterprise, including customers, internal business users, other engineering teams, B2B partners and regulators. This will help define the work to be done for different user groups, allowing the product team to focus on solving the challenges of each user group. Product development consistent with domain ownership will create clear accountability results for data sets and user results, enabling the team to stay in step.

    (3) Self service software platform

    Self service software platform for data is the core of autonomy and agility required to achieve the delivery results of data grid methods. At its core, enterprises need to think like cloud computing service providers and create an API driven self-service data platform. The platform needs to provide three groups of functions, first of all, storage, database, access control and other infrastructure. The engineering tool of workflow abstracts the complexity of infrastructure through infrastructure such as code and DevOps. Finally, the platform needs to provide central management functions for discovery, compliance, and monitoring.

    Implementing the three principles of data grid helps eliminate the inherent bottleneck in big data methods. With the data platform, each product team can define the data work they want to do, and determine the priority of different user results according to the value release to the enterprise. Each product team can work independently and quickly within the investment budget allocated to them.

     

    How to prepare for success?

    The implementation of data grid method requires domain modeling and product development as part of the normal software engineering life cycle. Successful implementation of domain ownership and product development affects people, skills, and enterprise design, which requires buying from senior stakeholders to achieve success.

    Many enterprises are already transforming their enterprises in this way. Taking the data grid method as a part of a wider transformation should reduce the overall work and cost involved in implementing the data grid. If enterprises have changed their operational models, the data grid approach is the logical next step, which can gain more value from the model and alleviate many challenges inherent in the big data approach.

    Another key factor in the successful implementation of the data grid is to ensure that the self-service data platform is set up correctly. This requires some pre thinking to define the functions, architecture, team skills and structure required by the platform to achieve the autonomy of the product team. It also requires a team that understands the vision of the self-service platform and has the skills needed to achieve it.

    Finally, it is recommended to start small. Create an MVP self-service platform to enable a small group of non critical data sets to prove the technical and operational model in an enterprise environment.

    Is big data a panacea?

    The data grid can overcome many challenges inherent in big data by driving higher levels of autonomy and data engineering alliances among a wider range of stakeholders. However, big data is not a panacea, it brings a series of risks for enterprises to manage.

    By understanding the risks brought by the data grid method, formulating plans to reduce these risks, and selecting the correct architecture to support the self-service platform and stakeholders to support the vision, enterprises' ability to use data can change significantly.

    Neil Mulholland has been the chief architect of Enterprise Blueprint since 2010, and has been engaged in digital and integrated architecture since the 1990s. Recently, he has played an important role in helping enterprises adopt agile methods on a large scale, focusing on how architecture and architecture governance work in this environment.

    Neil believes that trust is gained by demonstrating the ability to focus on the most important things for customers. Understanding customers' needs and achieving the results they want are critical to determining successful solutions and building long-term, mutually beneficial partnerships.

    Neil contributes to the customer's enterprise through informal and formal architecture team guidance to help them improve the skills of the entire architecture team through better solutions and operational methods. He likes to try new technologies and learn how to best apply them to solve customer problems, but he doesn't like a steep learning curve when introducing new technologies, because it will delay real adoption. ​

    big data data grid enterprise
    Previous Article Who owns the copyright of the paintings created by AI for you?
    Next Article What does cloud platform mean?

    Related Articles

    Big Data

    Design and implementation of visualization big screen in the era of big data

    Apr 05, 2025
    Big Data

    10 Misunderstandings of Big Data Application

    Apr 30, 2025
    Big Data

    What is the biggest gap in the big data trend sweeping the world?

    Apr 28, 2025
    Most Popular

    Wireless charging principle

    May 08, 2025

    Is the enterprise ready to protect its cloud computing?

    Apr 18, 2025

    How to prove you're human in the AI jungle?

    May 14, 2025
    Copyright © 2025 itheroe.com. All rights reserved. User Agreement | Privacy Policy

    Type above and press Enter to search. Press Esc to cancel.