METADATA MANAGEMENT
Metadata management — is about the organisational process and policies to govern the use of metadata. Firstly, what is metadata? This gives basic information of your data, which includes the type of file, size, author, timestamp and so on. There are several classifications of metadata including administrative, descriptive, reference, statistical, structural and technical metadata.
How do we create metadata? There are manual approach and automated way of creating metadata in your organisation. The manual approach allows more flexibility with more details while automatic creation generally contains only very basic information. Generally, the more important the information asset, the more insightful your metadata. Nowadays, artificial intelligence (AI) and machine learning (ML) can also be used to automatically profile, tag and classify lineage information to metadata and this approach is a new evolving part of the data.
So, the need to have a structured approach, tools to effectively maintain metadata in an organisation is of utmost important. Implementing a robust metadata management framework is a multi-faceted process that involves technology, people, and processes, all in one. Business benefits of implementing metadata management are numerous, just to mention a few:
i. Less data retrieval issues because of the consistency in metadata definition.
ii. Minimal dependence on IT for information maintenance
iii. Aligning with compliance requirements
iv. Quality of data and improved accessibility of the data.
Get Christopher Ozurumba’s stories in your inbox
Join Medium for free to get updates from this writer.
v. Better data driven insight and decisions.
Strategic approach and detailed planning are a good recipe to a robust implementation of metadata management. Following the steps are pretty important for the implementation.
Identifying the key stakeholders — data analyst, BI analyst, product team, backend engineers, marketing team will provide clarity on the objective required for management of the metadata. This sets the direction to destination.
Understanding the data sources, the data catalog and inherent use of the data is a good step in the right direction. Afterwards, strategy for metadata management based on the use case and requirement(s) which would have been captured during requirement gathering — and will align with the objectives.
Choosing the right tool for another important step to achieving the end goal. There are several tools, both open source and enterprise edition. Decision record with factors such as scalability, features, price, deployment options, integration/storage tools etc will provide a better decision on the best tool to use.
Implement the metadata repository comes next, after developing the metadata standards. There are some repos you can start with, to give guidance and visibility on the approach to take. Start the development process and carry out all required testing and validation phases prior to production.
https://github.com/doowb/github-metadata
https://github.com/topics/metadata-management
Implementing metadata management is not the end -goal as putting in a framework for continuous improvement, adding data governance as well as training of end users for effective monitoring and compliance will make all efforts not to be in vain.
