Diving deeper into what makes up each element of OLAP and how to put them to work for you.

Table of contents

· OLAP dimensions
· OLAP indicators
· OLAP representation
· The importance of OLAP dimensions, indicators, and representation
· Conclusion

As we’ve discussed, dimensions can be expressed in the form of hierarchies or groups in Online Analytical Process (OLAP). For example, data can be grouped by country, state, and city, each falling under one another. Data can also be grouped by year, month, and day in the same way.

But let’s dive deeper into the ideas that data dimensions can share and introduce the concepts of measures, indicators, and representations in OLAP.

OLAP dimensions

A dimension is a structure that categorizes facts and measures to enable users to answer business questions. Commonly used dimensions are people, products, place, and time. Data dimensions give context to analysis and can help define a hierarchy in a data set. Think of dimensions as the “labels” or “perspectives” you use to look at data in a database.

For example, imagine you are analyzing sales data. You might want to look at sales based on:

1. Time: Analyze sales by year, month, or day
2. Location: Check sales by country, state, or city
3. Product: See which products are selling the most
4. Customer: Group sales by customer demographics or categories

These categories — Time, Location, Product, and Customer — are the dimensions.

OLAP indicators

In OLAP, indicators refer to the measurable values or metrics that you analyze. These are also called measures or facts. They represent the numbers or data points you want to study, track, or evaluate.

Examples of indicators include:

• Sales: Total sales amount in dollars
• Profit: Net profit for a product or region
• Quantity sold: Number of items sold
• Revenue growth: Percentage increase in revenue over a period
• Customer count: Number of customers in a given category

Indicators are typically numerical values that you aggregate (e.g., sum, average, count). They are analyzed in the context of dimensions. For example:

• “What is the total sales (indicator) by year (time dimension)?”
• “What is the profit (indicator) for each product category (product dimension)?”

Indicators provide insights that help make decisions based on trends, performance, or comparisons.

OLAP representation

Representations in OLAP refer to how data is visually or structurally presented to help users analyze and understand it. These formats or views make it easier to explore the data.

Examples of OLAP representations include:

1. Tables or grids: Data displayed in a structured, spreadsheet-like format with rows and columns (like pivot tables).
2. Charts and graphs: Data shown visually, such as in bar charts, line graphs, or pie charts, to highlight trends or comparisons.
3. Cubes: A 3D representation of data (conceptually) that allows users to explore multiple dimensions (e.g., time, location, product).
4. Dashboards: Interactive visual summaries that combine tables, charts, and other views to give an overview of data.
5. Drill-down views: A hierarchical representation where users can click to see more detailed levels of data (e.g., year → quarter → month).

The importance of OLAP dimensions, indicators, and representation

Understanding the differences between dimensions, indicators, and representations in OLAP is essential for a data professional because these concepts are foundational to designing, analyzing, and optimizing multidimensional data systems. Here’s why they matter:

Dimensions: The contextual framework

• Data organization: They organize data hierarchically, making it easier to drill down or roll up in analysis.
• Analysis accuracy: Misdefining dimensions can lead to skewed insights (e.g., aggregating data incorrectly across time or geography).
• Scalability: Properly defined dimensions ensure that new data or hierarchies (e.g., adding subregions) can be seamlessly integrated.

Indicators: The data being analyzed

• Insightful metrics: Selecting the right indicators ensures that the analysis aligns with business goals.
• Performance impact: Optimizing measures for aggregation and calculation impacts system performance and usability.
• Actionable insights: Indicators provide the values that drive decisions, so their definition and calculation (e.g., summing vs. averaging) must align with the use case.

Representation: How data is displayed

• User experience: Clear and intuitive representations help users make sense of complex data quickly.
• Decision support: Poor representation (e.g., cluttered dashboards) can obscure critical insights, reducing decision-making efficiency.
• Customization: Data professionals often need to adapt representations to different stakeholders (e.g., executives vs. analysts).

Conclusion

In conclusion, the interplay between dimensions, indicators, and representation in OLAP is pivotal for effective data analysis and decision-making.

For data professionals, mastering these components is not just about understanding data — It’s about leveraging it to its fullest potential, crafting systems and visuals that empower stakeholders to uncover trends, optimize performance, and make informed decisions with confidence.

By appreciating the unique roles and synergy of these elements, professionals can design data solutions that truly meet the demands of today’s dynamic analytical landscape.

Sebastien Thomas is the CEO and co-founder of DataGalaxy, a complete data governance platform and creator of the Data Knowledge Catalog. With over 20 years of experience in the field of data governance, Sebastien is deeply passionate about bridging the gap between IT and business teams through comprehensive knowledge of their data.

Follow me on LinkedIn for more insights like these!

Data by design: Exploring dimensions, indicators, and representation in OLAP was originally published in datagalaxy-engineering on Medium, where people are continuing the conversation by highlighting and responding to this story.

Understanding the use, benefits, and critical points of OLAP.

Table of contents
· What is OLAP?
· Why is OLAP important?
· OLAP vs. OLATP
· OLAP data dimensions & data measures
· Conclusion

Welcome to Data by Design, a new series in which we’ll explore the fascinating world of analytical thinking.

Today, let’s start with an introduction to OLAP (online analytical processing) — Join us as we explore the importance of data modeling in supporting decision-making processes. We’ll break down the concept of OLAP, understanding the role of dimensions and measures in data analysis.

Discover how dimensions provide context for our analysis, allowing us to zoom in or out based on specific criteria. We’ll explore examples of dimensions, including a Geography Hierarchy, to help you grasp the concept more effectively.

What is OLAP?

Online analytical processing (OLAP) is software that analyzes business data from different perspectives. Since organizations collect and store data from multiple sources, such as websites, applications, smart meters, and internal systems, OLAP combines and groups this data into categories to provide actionable insights for strategic planning.

For example, a retailer stores data about all its products, such as color, size, cost, and location. The retailer also collects customer purchase data in a different system, such as the name of the items ordered and total sales value. OLAP combines the datasets to answer questions about which color products are more popular or how product placement impacts sales.

Why is OLAP important?

OLAP helps organizations process and benefit from a growing amount of digital information. Some benefits of OLAP include the following.

• Faster decision-making: Businesses use OLAP to make quick and accurate decisions to remain competitive in a fast-paced economy. Performing analytical queries on multiple relational databases is time-consuming because the computer system searches through multiple data tables. On the other hand, OLAP systems precalculate and integrate data so business analysts can generate reports faster when needed.
• Non-technical user support: OLAP systems make complex data analysis easier for non-technical business users. Business users can create complex analytical calculations and generate reports instead of learning how to operate databases.
• Integrated data view: OLAP provides a unified platform for marketing, finance, production, and other business units. Managers and decision-makers can see the bigger picture and effectively solve problems. They can perform what-if analysis, showing the impact of decisions one department takes on other business areas.

OLAP vs. OLATP

OLAP and online transactional processing (OLTP) are the two primary data processing systems used in data science. The key difference between OLAP and OLTP is that OLAP is used for complex data analysis, while OLTP is used for real-time processing of online transactions at scale.

Although each one’s purpose and method of processing data differ, OLAP and OLTP systems are valuable for solving complex business problems. Let’s dive deeper into the differences between OLAP and OLTP and look at various applications of each that illustrate their use cases.

The main distinction between OLAP and OLTP is the core purpose of each system. An OLAP system can quickly process large amounts of data, enabling in-depth data analysis across multiple dimensions for decision-making and problem-solving. Teams can use this data for decision-making and problem-solving.

In contrast, OLTP systems are designed to handle large volumes of transactional data involving multiple users. Relational databases rapidly update, insert, or delete small amounts of data in real time. Most OLTP systems are used for executing transactions such as online hotel bookings, mobile banking transactions, e-commerce purchases, and in-store checkout. Many OLAP systems pull their data from OLTP databases via an ETL pipeline and can provide insights, such as analyzing ATM activity and performance over time.

Simply put, organizations use OLTP systems to run their business, while OLAP systems help them understand their business.

OLAP data dimensions & data measures

Before we can begin to discuss OLAP’s effect on data dimensions and data measures, we must first define these key topics.

• Data dimensions: A dimension is a structure that categorizes facts and measures to enable users to answer business questions. Commonly used dimensions are people, products, place, and time. Data dimensions give context to analysis and can help define a hierarchy in a data set.

Before trying to train your data model with OLAP, I suggest looking for patterns in data elements and dimensions to train your analytical thinking.

• Data measures: A data measure is a numerical attribute of your data and can be broken out by one or more dimensions. These fields in your database contain numerical values that can be calculated or aggregated. Measures are quantitative; you can perform calculations on them like sums, averages, or standard deviations.

OLAP helps to further quantify dataset elements and give value for analyses. OLAP data measures must be expressed with a unit, can be aggregated, and must be given context.

Conclusion

In conclusion, OLAP plays a crucial role in modern data analysis by enabling organizations to process and analyze large amounts of data from multiple perspectives. Using dimensions and measures, OLAP helps contextualize and quantify data, providing actionable insights for decision-making.

Its ability to support fast, complex queries and generate integrated views across business units makes it a valuable tool for strategic planning. While OLTP systems focus on real-time transaction processing, OLAP complements them by offering deep analytical capabilities, helping businesses run efficiently and understand and optimize their operations.

Sebastien Thomas is the CEO and co-founder of DataGalaxy, the Data Knowledge Catalog. With over 20 years of experience in the field of data governance, Sebastien is deeply passionate about bridging the gap between IT and business teams through comprehensive knowledge of their data.

Data by design: An introduction to OLAP was originally published in datagalaxy-engineering on Medium, where people are continuing the conversation by highlighting and responding to this story.

Employing active metadata for an informed tomorrow.

Table of contents

· Introduction
· The evolution of metadata management
· Elevating the role of metadata
· The role of a Data Knowledge Catalog in active metadata management
· DataGalaxy for active metadata management
· Conclusion

Introduction

As modern businesses look to drive greater value from their data, many are exploring new strategies and approaches to deliver clear and trustworthy information in support of business decisions.

For many organizations, this means embracing a culture of data governance as an enabler of the data-driven organization and shifting their approach to metadata management through the use of a data knowledge catalog.

Together, these approaches bring data to the end users in a user-friendly way while still employing the policies and processes needed to ensure the data is integrated, protected, and secure.

The evolution of metadata management

In its most basic form, metadata is data about data. It provides context that helps users find the data they need easily and understand key attributes such as ownership and lineage so they can use it more effectively. While metadata management or the data management processes that manage the context of the data isn’t new, many organizations are realizing that their traditional approaches are no longer sufficient in today’s world where data volume and complexity are growing at a staggering rate.

For years, passive metadata management successfully delivered basic technical information about the data, including the data’s profile and the operational features such as who accessed which data and how often they did so. While useful, relying on passive metadata alone is no longer sufficient for organizations that wish to be truly data-driven.

Active metadata, on the other hand, introduces richer contextual elements across all levels of the data stack. It is generally more complex than passive metadata, spanning operational, business, and social metadata as well as technical metadata. It adds value to the data and makes it more meaningful, which, in turn, helps end-users make more meaningful decisions with it.

Active metadata enables a dynamic metadata management model that enables organizations to reveal a more holistic data story that goes well beyond the static data profile. It shows how and where data flows in the data infrastructure, including all modifications, data transformations, and calculations made up to that point.

By employing artificial intelligence and machine learning algorithms to analyze their data, organizations can automatically list, tag, classify, and inform the origins of the data to understand data lineage. Using this information, organizations can uncover new patterns and identify potential blind spots in their data stack, allowing them to fix them before they become an organizational-wide problem.

In fact, according to Gartner, “Through 2024, organizations that adopt aggressive metadata analysis across their complete data management environment will decrease time to delivery of new data assets to users by as much as 70%.”

Elevating the role of metadata

For years, organizations have been using metadata to manage their data. But as data becomes increasingly more strategic to an organization, the role of metadata, specifically active metadata, becomes increasingly important as well.

Gone are the days when metadata simply provided documentation about the data profile or usage of the data. Instead, active metadata now plays a critical role in the management of data access, data classification, and data quality.

Active metadata forms the foundation for modern governance and data management, benefiting organizations in six key ways:

1. Purging outdated or unused data
   Using active metadata, organizations can systematically determine the last time someone within the organization used a data set or batch of data. They can reveal the number of people who have used the data within a given period. And, using systematic rules, the organization can archive or purge the data when it hasn’t been used for a defined duration.
2. Dynamically allocating data processing resources
   Active metadata management enables organizations to scale up or down their IT resources based on peak periods of usage of BI tools. For example, if an organization knows that more users access their BI tools during the end of a fiscal quarter, they can scale resources accordingly to accommodate this peak load.
3. Enriching the user experience in business intelligence tools
   Active metadata management allows organizations to bring richer context to dashboards within their BI tools which drives increased efficiency and a better user experience. Instead of switching between the BI tool and a data catalog, users can view a table within their BI tool and see relevant metadata including business terms, descriptions, data ownership, and history.
4. Automatically classifying sensitive data for compliance and governance
   Using active metadata management, organizations can classify sensitive and protected data and control who can — and can’t — see it, making it easier to comply with regulations as well as their own governance policies.
5. Identifying the most frequently used assets across the organization
   Active metadata management enables organizations to create custom popularity scores for each data resource based on usage information from query logs, provenance, and BI dashboards, enabling them to identify the data sets accessed most often in reporting and analytics. The organization can then check these resources regularly for data quality issues and ensure they appear more frequently in search results.
6. Alerting downstream users to resolve issues quickly
   Using active metadata management, the data team can receive a notification when a user modifies a database or creates a potential anomaly, allowing them to review the data and determine if the change caused an error. If it did, they can quickly notify the user of the error or correct it themselves to maintain the integrity of the data.

The role of a Data Knowledge Catalog in active metadata management

According to Gartner, “Existing metadata management tools are increasingly incapable of fulfilling comprehensive metadata needs in the enterprise.”

That’s why organizations that employ a data knowledge catalog accelerate their efforts to manage metadata more strategically and aggressively in both traditional data management as well as in a data fabric environment.

A data knowledge catalog goes beyond the capabilities of a traditional data catalog by including metadata mapping, active metadata management, and metadata knowledge sharing capabilities. Built for business users, a data knowledge catalog provides a seamless experience that delivers data in a user-friendly way, eliminating the need for extensive searching. Through in-app and in-browser features, users can easily share data knowledge and collaborate to deliver better data and analytics across the business.

In a data fabric environment, a data knowledge catalog can help improve data governance by providing a centralized platform for data management, ensuring that data is consistent, accurate, and secure. A key benefit of a data fabric architecture is its ability to provide a unified view of data across an organization by providing a single, standardized platform for data management. By incorporating a data knowledge catalog as part of the data fabric environment, organizations can break down data silos, share data knowledge, and create collaborative governance that enables teams to work more efficiently.

As active metadata management technology matures, organizations will recognize the need to move beyond the capabilities offered in traditional data catalogs to embrace the machine learning-driven capabilities of a data knowledge catalog that enable them to orchestrate more advanced active metadata management use cases across the organization.

DataGalaxy for active metadata management

DataGalaxy provides active metadata management capabilities that extend beyond those delivered in a traditional data catalog. Dedicated to providing metadata mapping, active metadata management, and metadata knowledge sharing for the largest number of people, DataGalaxy’s Data Knowledge Catalog delivers key capabilities including the ability to:

• Enhance user experiences within business intelligence tools by integrating relevant metadata into an organization’s BI tools
• Classify sensitive data to enable automatic compliance with regulations by customizing access policies according to the organization’s defined governance strategy
• Identify frequently used assets and create a popularity score based on usage information from sources such as query logs, data provenance, and BI dashboards
• Integrate with collaboration tools such as Slack and Microsoft Teams to provide a seamless, collaborative experience for business users

Conclusion

When organizations integrate a data knowledge catalog into their data management processes, they are better positioned to accelerate their use of active metadata management to increase collaboration and productivity, champion data governance, and enable data literacy.

Sebastien Thomas is the CEO and co-founder of DataGalaxy, the Data Knowledge Catalog. With over 20 years of experience in the field of data governance, Sebastien is deeply passionate about bridging the gap between IT and business teams through comprehensive knowledge of their data.

Managing active metadata with a Data Knowledge Catalog was originally published in The DataGalaxy Digest on Medium, where people are continuing the conversation by highlighting and responding to this story.

Data governance and AI: Both here to stay!

Table of contents

∘ The rapid acceleration of AI initiatives
∘ AI in financial services
∘ AI in healthcare
∘ AI in retail
∘ The critical role of data in AI projects
∘ Risks of scaling AI without data governance
∘ How DataGalaxy supports AI integration
∘ Conclusion

The pace of change in modern business is relentless. From customer demands for better experiences and increased globalization to a dynamic regulatory environment and the proliferation of data, the pressure is on, prompting businesses large and small to explore new ways to remain competitive. And with new, innovative companies disrupting traditional markets, remaining a leader in their respective industries is a challenge, requiring organizations to embrace new strategies and new tools to support their goals.

Increasingly, one technology is emerging as a transformative force: artificial intelligence (AI). Once the domain of science fiction, today, AI is rewriting the rules for modern business. According to PWC, 73% of businesses currently use AI in some aspect of their business, proof that AI is quickly becoming a bedrock of modern business business strategy.

But as AI becomes increasingly important, so, too does the data that supports it. After all, AI models are only as good as the data that feeds them. The insights AI delivers are a direct reflection of the data that trains them. When that data is high-quality, the insights follow suit, giving users confidence in the integrity and validity of the results. When the data is inaccurate, outdated, or incomplete, the outputs AI delivers are flawed, causing faulty decisions that can lead to poor customer experiences, missed opportunities, and tarnished reputations.

The rapid acceleration of AI initiatives

The general availability of tools such as ChatGPT, Bard, and GitHub Copilot sparked companies to increase their investments in AI-driven technologies. According to Gartner, in October 2023, 55% of organizations reported that they are piloting or in production with generative AI, up 35% from a Gartner poll conducted earlier in 2023 which reported that less than 20% of respondents were either piloting or in production with generative AI solutions.

As AI takes a firmer hold on modern business, organizations across industries are benefiting from increased efficiency, improved customer service, and better, more informed decision-making.

AI in financial services

In financial services, AI plays a vital role in risk mitigation. Using AI, firms can analyze large volumes of transactional data to detect anomalies and flag suspicious behavior that could lead to fraud. Further, AI helps financial services streamline regulatory processes to ensure they remain compliant, even as regulations evolve. Further, more than 30% of financial institutions currently use AI in their product development.

AI in healthcare

Healthcare organizations employ AI to transform patient care through the use of automated imaging, providing critical insights to healthcare providers. AI also improves efficiency by automating routine, administrative tasks such as labeling images and generating reports.

AI in retail

AI enables retailers to improve customer experiences by surfacing personalized recommendations via targeted campaigns, websites, and mobile apps. Retailers also use AI to manage and optimize inventory and provide dynamic pricing to customers based on stock levels and seasonal demand. And that’s only the beginning. Retail use of AI is expected to grow at a compound annual growth rate of 30% from 2023 to 2030.

These examples showcase just a few ways AI is revolutionizing the business landscape today — and it’s only the start. As companies across industries continue to realize the transformative power that AI can have on customer service, marketing, operations, and decision-making, their adoption of AI will only accelerate.

The critical role of data in AI projects

While AI may be taking center stage, data is the star of the show. Data fuels AI algorithms, enabling them to learn, adapt, and make decisions. Here’s how it works.

AI tools start by collecting data — the more, the better. Ensuring that the data is clean and accurate is paramount, as the quality of the data ultimately drives the quality of the insights and predictions. If the data has questionable quality, it’s a good idea to fix the errors, remove duplicates, or label the data before the model consumes it. Once the model you select consumes the data, the model will process the data, look for patterns, and make predictions. Some will be right and some will not. That’s all part of the learning process.

Next, it’s time for you to test the model by providing new data and evaluating the accuracy of its predictions. If the model does well, the training was successful. If it returns flawed results, then you’ll need to go back and adjust the model parameters or add more training data until the accuracy improves.

By now, it should be clear that the quality of the data — its precision, reliability, and relevance — directly impacts the performance and outcomes of AI applications. Training models with high-quality data generally yield good, trustworthy results.

But when garbage data goes into AI models, garbage results come out. That’s when problems begin to arise. Bad insights lead to bad decisions which, in turn, cause problems for the business. Customer experiences are disjointed, revenue declines, operations become inefficient, and perceptions of the company suffer.

Risks of scaling AI without data governance

The importance of data quality cannot be understated. Poor data quality leads to inaccurate AI predictions which, in turn, causes business leaders to make poor decisions that lead to skewed outcomes, biased predictions, and error-filled conclusions. And when stakeholders lose faith in the integrity of the data, they become reluctant to use data to make decisions.

Even worse, when organizations scale their AI using poor-quality data, they can face financial and reputational harm, legal issues, or hefty fines stemming from data breaches, unauthorized access, and non-compliance with regulations such as GDPR and CCPA. New regulations focused on the ethical use of AI are emerging as well, increasing the need for greater oversight and governance when using AI-driven insights.

Overcoming poor data quality requires organizations to embrace data governance. By implementing robust frameworks for managing data throughout its lifecycle, organizations can improve the accuracy and reliability of their data, ensuring it is fit for purpose.

Data governance also establishes organization-wide policies that govern the collection, storage, processing, and sharing of data within the organization, as well as the processes needed to ensure regulatory compliance. Furthermore, data governance promotes transparency, accountability, and traceability of data, enabling stakeholders to more easily spot errors and mitigate risks associated with flawed or misrepresented AI insights before they negatively impact business operations.

How DataGalaxy supports AI integration

An established data governance strategy helps organizations minimize errors, inconsistencies, and biases in their data. This, in turn, fosters greater trust in the data and boosts users’ confidence as they apply AI-driven insights in their decision-making processes.

DataGalaxy supports AI by empowering organizations to embrace the data governance tools and frameworks needed to ensure their data remains secure, compliant, and of the highest quality. Using their Data Knowledge Catalog, DataGalaxy provides in-depth clarity about data definitions, data lineage, and other essential business attributes so all users can understand the data and use it as a strategic asset. User-friendly features such as an all-in-one Business Glossary, intuitive data visualization tools, detailed data lineage tracking, and natural language search enable users from all departments to find — and trust — the data they need to power AI.

As AI continues to gain momentum, businesses will, inevitably, want to scale their AI initiatives. Effective data governance holds the key to unlocking AI’s potential and expanding its use effectively and responsibly. Using DataGalaxy’s Data Knowledge Catalog, organizations can make it easier for everyone to find the data they need to run the business — and they can do so with the proper security and controls in place to protect sensitive data. By eliminating data silos, DataGalaxy also democratizes data collaboration, providing stakeholders with access to reliable, interconnected sources so they can make better decisions.

Conclusion

Clearly, AI is here to stay. Its use is poised to grow at an unprecedented rate, prompting businesses worldwide to rethink their strategies as they relate to the use and adoption of AI applications. With data fueling the learning and predictive capabilities of AI models, it is imperative that organizations adopt data governance as a way to ensure the integrity of their data. With DataGalaxy as a partner, organizations can harness the power and potential of AI to drive innovation, increase efficiency, and remain a competitive force in their industry.

To learn how DataGalaxy can help your business scale its use of AI, visit www.datagalaxy.com.

Sébastien (Seb) Thomas has over 16 years of experience in data-centric projects. He has served as a team leader, modeling expert, database manager, and now as a data entrepreneur by co-founding DataGalaxy, the industry’s first Data Knowledge Catalog delivering data culture and literacy across organizations globally.

Today, more than 170 international companies of all sizes trust DataGalaxy to enable data literacy, connect business & IT teams, and champion data governance for their entire organization.

Learn more about how DataGalaxy can help your teams bring data to the people: https://www.datagalaxy.com/en/

Why scaling your AI without data governance is a mistake was originally published in The DataGalaxy Digest on Medium, where people are continuing the conversation by highlighting and responding to this story.