Data governance during migration sets companies up for long-term success

Often during the data migration or conversion process, from on-prem to cloud or from version to version of a technology platform, the idea of taking a minimal amount of time and effort to document and design data governance processes is overlooked. Even for companies doing full-scale chart of accounts updates or mass dimension enrichment, the governance process and oversight of these new data domains are often left unnoticed until after go live, if at all.

However, layering a data governance target operating model (TOM) over the data migration process allows companies to beta test their operational and technical processes against the new and old systems, providing a sneak peek at a long-term future-state solution. Companies believing that their current process can adapt need only look at the hours used to cleanse the legacy data to realize this is not a best practice, and implementing data governance can provide an enterprise-level return on investment.

See how the four major phases of migration can influence the data governance of people, process and technology:

Phase 1: Data preparation

• Key stakeholders are engaged to provide insight into the reliability and scope of a data migration, which naturally reveals a data governance organizational structure.
• Data governance council members can easily be placed into three areas—council members, stewards and consumers—to provide oversight and accountability for the migration and serve as long-term governance leaders.
• Potential data stewards will be fully engaged and spearhead data preparation, while data owners’ engagement is key to providing the insight needed to scope the migration effort. These resources will be evaluated for data stewardship/owner long-term roles within the organization.

Phase 2: Data mapping

• The next key principle is the data dictionary and catalog. During the cleansing process, table structures and associated attributes should be documented, outlining where data will live in future-state repositories and giving insight into how data is used by the business, creating natural definitions. The combination of the metadata needed for the cleansing activity and the structures created in the models become the basis of the data catalog and dictionary that should be implemented in a master data management (MDM) technology solution.
• The dimension mapping in the transactional data during a migration or transformation effort is a good time to create a data dictionary to define the model attributes clearly and create one common language.
• Each master data and reference data element that will be part of the new system must be reviewed. Including an effort to define the elements, structures, relationships and other metadata is a best practice.
• The technical work for the mapping and cleansing processes is often tightly coupled. However, the review session, the documentation of mappings and the data review are the basis for creating data lineage diagrams, and they are often done prior to cleansing.
• The data dictionary should influence the catalog. The detailed information will give a road map for the data organization and how the definitions can be propagated across the landscape.

Phase 3: Data cleansing

• During the data cleansing and normalization phase of a migration effort, two key data governance principles can be found: data quality and data normalization. Data quality is the most notable governance practice to create.
• The scripts written to normalize data from the legacy system can provide insight into the future state scripts that will become the foundation of the data quality rules implemented in a new MDM system. Although legacy systems will have different structures, the output of scripts should be replicated by the new systems.
• Even if some data cleansing is done before the mapping, the normalized data should adhere to the definitions during the technical mapping process, and scripts should be written to ensure this happens.
• Data quality rules are the foundation, and no data migration can be successful without clear quality rules. Documenting these rules will provide the key controls to ensure future data issues do not arise, proper error handling is implemented, and logging requirements are visible in data quality and normalization conflicts.
• During the data migration validation, the data quality and cleansing rules and activities can be tested to ensure viability, reliability and accuracy, giving a dry run of that future-state business-as-usual process.

Phase 4: Integrate

• Pushing data from a legacy system or a staging area where it has been mapped and normalized provides insight into the overall data flow, synchronization and trusted source hierarchies.
• Creating a single source of truth for all domains will guide how survivorship and data accountability by the system can be documented and put in place.
• As data is migrated to new systems, documenting the systems that house the original records creation can be noted, as well as those that capture that data (i.e. Salesforce system of record for the customer, enterprise resource planning (ERP) received customer). Understanding where data originated will guide governance processes and show where most integration controls are needed.
• For organizations moving to data warehouses or implementing new extract, transform and load tools, moving to a hub and spoke model to centralize rules, quality issues, error logging, quarantining and other principles can stop bad data from propagating the systems.

Are you ready for the opportunity?

One of the most difficult things when approaching data governance or MDM is properly quantifying or showing the proposed value of good data governance within an organization. For organizations doing a finance transformation, data lake project or beginning their AI journey, data migration or preparation activities will occur, which can provide insight into why data governance is needed to be successful, how to begin the data governance journey, and what the overall ROI is for investing time and resources during this activity as opposed to going through it alone. Your company should leverage this opportunity and ensure you set the foundation for data governance and data quality going forward.