What is Data Architecture? A Data Management Blueprint (2023)

Data architecture is a discipline that documents an organization's data assets, maps how data flows through its systems and provides a blueprint for managing data. The goal is to ensure that data is managed properly and meets business needs for information.

While data architecture can support operational applications, it most prominently defines the underlying data environment for business intelligence (BI) and advanced analytics initiatives. Its output includes a multilayer framework for data platforms and data management tools, as well as specifications and standards for collecting, integrating, transforming and storing data.

Ideally, data architecture design is the first step in the data management process. But that usually isn't the case, which creates inconsistent environments that need to be harmonized as part of a data architecture. Also, despite their foundational nature, data architectures aren't set in stone and must be updated as data and business needs change. That makes them an ongoing concern for data management teams.

Data architecture goes hand in hand with data modeling, which creates diagrams of data structures, business rules and relationships between data elements. They're separate data management disciplines, though. In an article on how data modeling and data architecture differ, David Loshin, president of consultancy Knowledge Integrity Inc., distinguished between modeling's micro focus on data assets and data architecture's broader macro perspective.

This guide to data architecture further explains what it is, why it's important and the business benefits it provides. You'll also find information on data architecture frameworks, best practices and more. Throughout the guide, there are hyperlinks to related articles that cover the topics in more depth.

How have data architectures evolved?

In the past, most data architectures were less complicated than they are now. They mostly involved structured data from transaction processing systems that was stored in relational databases. Analytics environments consisted of a data warehouse, sometimes with smaller data marts built for individual business units and an operational data store as a staging area. The transaction data was processed for analysis in batch jobs, using traditional extract, transform and load (ETL) processes for data integration.

Starting in the mid-2000s, the adoption of big data technologies in businesses added unstructured and semistructured forms of data to many architectures. That led to the deployment of data lakes, which often store raw data in its native format instead of filtering and transforming it for analysis upfront -- a big change from the data warehousing process. The new approach is driving wider use of ELT data integration, an alternative to ETL that inverts the load and transform steps.

The increased use of stream processing systems has also brought real-time data into more data architectures. Many architectures now support artificial intelligence and machine learning applications, too, in addition to the basic BI and reporting driven by data warehouses. The shift to cloud-based systems further adds to the complexity of data architectures.

(Video) Data Architecture: How it Glues Data Strategy and Data Management Together

Another emerging architecture concept is the data fabric, which aims to streamline data integration and management processes. It has a variety of potential use cases in data environments.

What is Data Architecture? A Data Management Blueprint (1)

Why are data architectures important?

A well-designed data architecture is a crucial part of the data management process. It supports data integration and data quality improvement efforts, as well as data engineering and data preparation. It also enables effective data governance and the development of internal data standards. Those two things, in turn, help organizations ensure that their data is accurate and consistent.

A data architecture is also the foundation of a data strategy that supports business goals and priorities. In an article on key data strategy components, Donald Farmer, principal of consultancy TreeHive Strategy, wrote that "a modern business strategy depends on data." That makes data management and analytics too important to leave to individuals, Farmer said. To manage and use data well, an organization needs to create a comprehensive data strategy, underpinned by a strong data architecture.

What is Data Architecture? A Data Management Blueprint (2)

What are the characteristics and components of a data architecture?

In an article on the principles of modern data architectures, Farmer stressed the importance of including both data governance and regulatory compliance processes and the growing need to support multi-cloud environments. He concluded by noting that data's potential business value will be wasted if a data architecture doesn't make it available for analytics uses.

"It's a cliché of modern data management that data is a business asset," Farmer wrote. "But data that just sits there is only a cost center, requiring maintenance without providing any business benefits."

Other common characteristics of well-designed data architectures include the following:

  • a business-driven focus that's aligned with organizational strategies and data requirements;
  • flexibility and scalability to enable various applications and meet new business needs for data; and
  • strong security protections to prevent unauthorized data access and improper use of data.

From a purist's point of view, data architecture components don't include platforms, tools and other technologies. Instead, a data architecture is a conceptual infrastructure that's described by a set of diagrams and documents. Data management teams then use them to guide technology deployments and how data is managed.

Some examples of those components, or artifacts, are as follows:

(Video) Data Architecture Strategies –Data Architecture Solution Architecture Platform Architecture

  • data models, data definitions and common vocabularies for data elements;
  • data flow diagrams that illustrate how data flows through systems and applications;
  • documents that map data usage to business processes, such as a CRUD matrix -- short for create, read, update and delete;
  • other documents that describe business goals, concepts and functions to help align data management initiatives with them;
  • policies and standards that govern how data is collected, integrated, transformed and stored; and
  • a high-level architectural blueprint, with different layers for processes like data ingestion, data integration and data storage.
What is Data Architecture? A Data Management Blueprint (3)

What are the benefits of a data architecture?

Ideally, a well-designed data architecture helps an organization develop effective data analytics platforms that deliver useful information and insights. In companies, those insights improve strategic planning and operational decision-making, potentially leading to better business performance and competitive advantages. They also aid in various other applications, such as the diagnosis of medical conditions and scientific research.

Data architecture also helps improve data quality, streamline data integration and reduce data storage costs, among other benefits. It does so by taking an enterprise view compared to domain-specific data modeling or focusing on architecture at the database level, according to Peter Aiken, a data management consultant and associate professor of information systems at Virginia Commonwealth University.

"When we look at it from a data architecture perspective, we have the greater value potential, and that's because we're looking at broad use [of data] across all of the databases," Aiken said during a Dataversity webinar in May 2021.

What are the risks of bad data architecture design?

One data architecture pitfall is too much complexity. The dreaded "spaghetti architecture" is evidence of that, with a tangle of lines representing different data flows and point-to-point connections. The result is a ramshackle data environment with incompatible data silos that are hard to integrate for analytics uses. Ironically, data architecture projects often aim to bring order to existing messy environments that developed organically. But if not managed carefully, they can create similar problems.

Another challenge is getting universal agreement on standardized data definitions, formats and requirements. Without that, it's hard to create an effective data architecture. The same goes for putting data in a business context. Done well, data architecture "captures the business meaning of the data required to run the organization," Aiken said in the Dataversity webinar. But failing to do so may create a disconnect between the architecture and the strategic data requirements it's supposed to meet.

Data architecture vs. data modeling

Data modeling focuses on the details of specific data assets. It creates a visual representation of data entities, their attributes and how different entities relate to each other. That helps in scoping the data requirements for applications and systems and then designing database structures for the data, a process that's done through a progression of conceptual, logical and physical data models.

Data architecture takes a more global view of an organization's data to create a framework for data management and usage. But, as consultant Loshin wrote in his article comparing the two, data modeling and data architecture complement each other. Data models are a crucial element in data architectures, and an established data architecture simplifies data modeling, said Loshin, who also is director of the Master of Information Management program at the University of Maryland's College of Information Studies.

Rick Sherman, managing partner at consulting firm Athena IT Solutions, separately explained seven techniques for modeling data, including the entity-relationship, dimensional and graph modeling approaches that are most used now. He also outlined a set of data modeling best practices, including these recommendations:

(Video) Data Modeling vs. Data Architecture

  • Gather both business and data requirements upfront, before building models.
  • Develop data models iteratively and incrementally to make the process manageable.
  • Use data models as a tool for communicating with business users about their needs.
  • Manage data models just like any other type of application code.
What is Data Architecture? A Data Management Blueprint (4)

Data architecture vs. information architecture and enterprise architecture

In a second article, Sherman described the difference between data architecture and information architecture in enterprise applications. "Information is data in context," he wrote. "An information architecture defines the context that an enterprise uses for its business operations and management." A data architecture that delivers high-quality, reliable data is the foundation for the information architecture, he added.

Meanwhile, data architecture is commonly viewed as a subset of enterprise architecture (EA), which aims to create an organizational blueprint for an organization in four domains. EA also encompasses the following:

  • business architecture, which involves business strategy and key business processes;
  • application architecture, which focuses on individual applications and their relationships to business processes; and
  • technology architecture, which includes IT systems, networks and other technologies that support the other three domains.

What data architecture frameworks are available?

Organizations can use standardized frameworks to design and implement data architectures instead of starting completely from scratch. These are three well-known framework options:

DAMA-DMBOK2. The DAMA Guide to the Data Management Body of Knowledge is a data management framework and reference guide created by DAMA International, a professional association for data managers. Now in its second edition and commonly known as DAMA-DMBOK2, the framework addresses data architecture along with other data management disciplines. The first edition was published in 2009, and the second one became available in 2017.

TOGAF. Created in 1995 and updated several times since then, TOGAF is an enterprise architecture framework and methodology that includes a section on data architecture design and roadmap development. It was developed by The Open Group, and TOGAF initially stood for The Open Group Architecture Framework. But it's now referred to simply as the TOGAF standard.

The Zachman Framework. This is an ontology framework that uses a 6-x-6 matrix of rows and columns to describe an enterprise architecture, including data elements. It doesn't include an implementation methodology; instead, it's meant to serve as the basis for an architecture. The framework was originally developed in 1987 by John Zachman, an IBM executive who retired from the company in 1990 and founded a consulting firm called Zachman International.

Key steps for creating a data architecture

Data management teams must work closely with business executives and other end users to develop a data architecture. If they don't, it may not be in tune with business strategies and data requirements. Engaging with senior execs to get their support and meeting with users to understand their data needs are two of the nine data architecture planning steps that consultant Loshin listed in an article.

Among other steps, he also recommended that organizations do the following:

(Video) Data Architecture Explained in under 4 minutes.

  • evaluate data risks based on data governance directives;
  • track data flows, as well as data lifecycle and data lineage info;
  • document and appraise the existing data management technology infrastructure; and
  • scope out a roadmap for the data architecture deployment projects.

Another article, by technology writer George Lawton, provides tips on building a cloud-based architecture for data management and analytics. It also outlines potential challenges that data management teams can face in the cloud, including data security requirements, regulatory compliance mandates and data gravity issues that can complicate migrations of data sets.

Read more about data architecture:

Top data architect and data engineer certifications

Data architect skills required, responsibilities and salaries

What are the different roles in data architecture design and development?

The lead role in data architecture initiatives typically goes to data architects. They need a variety of technical skills, as well as the ability to interact and communicate with business users. A data architect spends a lot of time working with end users to document business processes and existing data usage, as well as new data requirements.

On the technical side, data architects create data models themselves and supervise modeling work by others. They also build data architecture blueprints, data flow diagrams and other artifacts. Other duties may involve outlining data integration processes and overseeing the development of data definitions, business glossaries and data catalogs. In some organizations, data architects also are responsible for designing data platforms and evaluating and selecting technologies.

(Video) Data Architecture for Data Managers

Other data management professionals who often are involved in the data architecture process include the following:

  • Data modelers. They also work with business users to assess data needs and review business processes. Then, they use the information they've gathered to create data models.
  • Data integration developers. Once the architecture is implemented, they're tasked with creating ETL and ELT jobs to integrate data sets.
  • Data engineers. They build pipelines to funnel data to data scientists and other analysts. They also help data science teams with the data preparationprocess.

This was last updated in August 2021

Continue Reading About What is data architecture? A data management blueprint

  • Align enterprise data architecture, governance for 'quick wins'
  • Data warehouse vs. data lake: Key differences
  • Data fabrics help data lakes seek the truth
  • Data model design tips to help standardize business data
  • Why and how to adopt a data-centric architecture

FAQs

What is data architecture blueprint? ›

A solid data architecture is a blueprint that helps align your company's data with its business strategies. The data architecture guides how the data is collected, integrated, enhanced, stored, and delivered to business people who use it to do their jobs.

What is data architecture simple definition? ›

Data architecture is a framework for how IT infrastructure supports your data strategy. The goal of any data architecture is to show the company's infrastructure how data is acquired, transported, stored, queried, and secured. A data architecture is the foundation of any data strategy.

What is a data architecture diagram? ›

Data architecture diagrams demonstrate how and where data flows, gets processed, and is utilized. A data architecture diagram can help decide how to update and streamline data storage resources.

What is data architecture and design? ›

Data architecture design is a set of principles that are made out of specific strategies, rules, models, and guidelines that manage, what kind of information is gathered, from where it is gathered, the course of action of gathered information, storing that information, using and getting the information into the systems ...

How do you create a data architecture diagram? ›

How to draw an architectural diagram
  1. Document your shapes. ...
  2. Label the edges. ...
  3. Keep your arrows consistent. ...
  4. Use colors sparingly. ...
  5. Use multiple diagrams, if necessary. ...
  6. Merge incomplete diagrams. ...
  7. Include legends/keys/glossaries. ...
  8. Use diagramming software.
Oct 9, 2019

What is a good data architecture? ›

Good data architecture eliminates silos by combining data from all parts of the organization, along with external sources as needed, into one place to eliminate competing versions of the same data. In this environment, data is not bartered among business units or hoarded, but is seen as a shared, companywide asset.

What are the key data architecture principles? ›

By definition, data architecture is a blanket term that covers both the policies, rules, standards, and models that govern data collection and how data is stored, managed, processed, and used within the organization.

What is a data architect responsibilities? ›

A data architect is an IT professional responsible for defining the policies, procedures, models and technologies to be used in collecting, organizing, storing and accessing company information. The position is often confused with a database architect and data engineer.

What is new in data architecture? ›

From the long list of Data Architecture trends that shaped 2021, the ones worth mentioning here are democratization of data access, AI-ready architecture, and the rise of the analytics engineer, data fabric, data catalog, DevOps, and of course, the cloud.

How many types of data architecture are there? ›

The data architect breaks the subject down by going through three traditional architectural stages: Conceptual - represents all business entities. Logical - represents the logic of how entities are related. Physical - the realization of the data mechanisms for a specific type of functionality.

Why is data management important? ›

Data management helps minimize potential errors by establishing processes and policies for usage and building trust in the data being used to make decisions across your organization. With reliable, up-to-date data, companies can respond more efficiently to market changes and customer needs.

How do you read data architecture? ›

Data architecture is a big umbrella term that encapsulates data storage, computational resources, and everything in between. All the technology that supports the collection, processing, and dashboarding of data is included in the architecture. experience, etc) to design and maintain an ideal data architecture.

What is a data architecture document? ›

Data architecture is a discipline that documents an organization's data assets, maps how data flows through its systems and provides a blueprint for managing data. The goal is to ensure that data is managed properly and meets business needs for information.

What are data architecture artifacts? ›

Data Architecture Outcomes: Models, definitions, and data flows on depicted at various levels, usually referred to as Data Architecture artifacts. Data Architecture Activities: Forms, deploys, and fulfills Data Architecture intentions.

What is the difference between data governance and data architecture? ›

Data governance helps define policies and procedures for maintaining data security and compliance. Data architecture will be the formal structure for managing data flow.

What are the key data architecture principles? ›

By definition, data architecture is a blanket term that covers both the policies, rules, standards, and models that govern data collection and how data is stored, managed, processed, and used within the organization.

Videos

1. The Six Pillars of a Data Strategy | Data and Analytics Guide
(Cynozure)
2. Enterprise Architecture vs Data Architecture
(DATAVERSITY)
3. Data-Ed Webinar: Data Architecture vs. Data Modeling
(DATAVERSITY)
4. What is Enterprise Architecture (EA) and why is it important? EA concepts explained in a simple way.
(Dr. Raj Ramesh)
5. Webinar: Data Architecture – The Foundation for Enterprise Architecture and Governance
(DATAVERSITY)
6. A105 Blueprints for Building a Real-Time Data Architecture
(Database Trends and Applications)
Top Articles
Latest Posts
Article information

Author: Lakeisha Bayer VM

Last Updated: 27/09/2023

Views: 5574

Rating: 4.9 / 5 (69 voted)

Reviews: 84% of readers found this page helpful

Author information

Name: Lakeisha Bayer VM

Birthday: 1997-10-17

Address: Suite 835 34136 Adrian Mountains, Floydton, UT 81036

Phone: +3571527672278

Job: Manufacturing Agent

Hobby: Skimboarding, Photography, Roller skating, Knife making, Paintball, Embroidery, Gunsmithing

Introduction: My name is Lakeisha Bayer VM, I am a brainy, kind, enchanting, healthy, lovely, clean, witty person who loves writing and wants to share my knowledge and understanding with you.