Miktysh Blog.
Your go-to source for everything records management, eDRMS, Office 365, TRIM/Content Manager, RecordPoint, AvePoint, and EncompaaS related.
Categories
Topics
Data Migration Validation Best Practices
Feb 8, 2019 | Data Migration

When performing any type of data migration, validation and testing is imperative to mitigate the risk of unplanned downtime, loss or corruption of data. It is also necessary to ensure functionality of the current system is preserved during the migration process. Cutting corners by failing to perform data migration validation best practices can incur tremendous costs for enterprises.
When Data Migration Goes Wrong
Recently, UK TSB were involved in a disastrous data migration in an effort to upgrade their systems. The technical failures that occurred during and after the migration saw millions of clients locked out of their online and mobile banking accounts. To make matters worse, former CEO of TSB Bank, Paul Pester, who stepped down due to the migration failure, admitted during a hearing in front of members of parliament that the migration failure resulted in 1,300 customers being defrauded. Overall, the migration botch cost the bank over $300 million AUD, and led to over 12,000 customers leaving the bank. The TSB incident shines a spotlight on just how necessary data migration validation best practices are.
Data Migration Validation Using Sampling
The most common process for validating data migration is based on sampling. This means that a random subset of the migrated data is chosen and subsequently inspected. It should be noted that although sampling is an effective method of migration validation, it is by definition based on an inherent level of accepted risk. This means that you cannot ensure that 100% of the data will be migrated without error. When using the sampling method, it is important for migration specialists to meet the statistical standards established for sampling plans and procedures that have been identified by ASQ/ANSI Quality Control Standards Z1.4 and Z1.9.
Automated Testing Tools
Alternatively, for data migrations that pose a significantly high level of business or compliance risk, automated testing tools can be used. These tools can be used on a subset of data or on 100% of the data, depending on what is required based on the risk level of the migration project. The tools themselves however, also need to be configured and tested. There are a few automated testing tools available on the market, and although they are less susceptible to human error, for most organisations’ requirements sample testing is all that is needed.
Crucial Stages in Migration Testing
Â
Pre-Data Migration Testing
During the migration testing process, pre-data migration testing is vital for complex data structures, although it may not always be necessary for simple data structures. Firstly, the source data will need to be tested against the requirements of the destination system, using source-to-destination mappings. The destination system schema will then need to be inspected i.e. mandatory fields, min/max values, field length, field level validation etc.
Migration Testing
The first step in the migration testing stage will be to test the migration script(s) and make sure firewalls and ports are open, and hardware and software configurations are correct. Performance requirements have to be determined to meet migration timing (downtime) requirements. Supplementary systems are identified and verified to work with the destination system, and source system test cases will need to be converted for use with the destination system post migration.
Post Migration Testing
During the post migration testing phase, the number of records in the source and destination systems will first need to be compared. The source system data will then need to be checked to ensure it hasn’t changed during the migration. If any schema changes were required in the destination system this will also need to be validated. Reconciliation and NULL validation checks between the source and destination fields will also need to be undertaken for all fields. Finally, integration testing with the destination system and supplementary systems will be performed.
Final Thoughts
Ensuring that data migration validation best practices are followed is imperative for organisations undergoing data and content migrations. The damage that can be caused by a migration failure is potentially incalculable, as was highlighted in the incident involving UK Bank, TSB.
To ensure your data migration project is successful and doesn’t put your business’s information at risk, ensure you employ a data migration specialist with a proven track record in successfully migrating large amounts of data in complicated environments. Test their expertise and credentials to ensure they are following data validation best practices and have a deep technical understanding of the systems you are migrating to and from.
At Miktysh we have a solid, proven history in successfully migrating enterprise data since inception in 2008. Our vast experience spans numerous enterprise content management systems such as TRIM/Content Manager, SharePoint, OneDrive, RecordPoint, Dropbox, OpenText, Alfresco and many more. Our team of specialists always ensure data migration validation best practices are followed, using Xillio, which allows organisational content to be safely and efficiently migrated to the cloud, or between repositories. If you are considering a move to the cloud or to a new enterprise content management system, please get in touch with our team for a confidential discussion about your unique requirements.

Thien Nguyen
Technical Consultant
Thien is a technical consultant with an instrumental role in leading content and data migration projects, utilising our migration tool Xillio. Thien has expertise with SQL Servers, database and system administration.Â
Follow us on social
Popular content.
Analytics Driven Information Governance
14.07.2021
The New Normal Requires Zero Trust
27.06.2021
Microsoft 365 Records Management Whitepaper
06.11.2020
Micro Focus Launches Content Manager Select
28.05.2020
Content Manager SharePoint Integration Demo
20.05.2020
Top 7 Hidden Features in Microsoft Teams
11.02.2020
Integrate Content Manager with SharePoint
15.01.2020
Information Management Strategy Example
06.12.2019
How to Approach a Digital Transformation
13.11.2019
Top 5 Benefits of Migrating to Office 365
10.10.2019
Content Manager 9.4 Web Client
19.09.2019
Content Manager 9.4 Sneak Peek – Video Demo
02.09.2019
What Version of SharePoint is on Office 365?
27.06.2019
New SharePoint 2019 Features
08.03.2019
Cloud Governance – RIMPA Queensland Event
27.09.2018
RecordPoint Records365 Demo
09.05.2018
How RecordPoint Enhances Office 365
09.05.2018
6 GDPR Tools to Assist with GDPR Compliance
17.04.2018
Analytics Driven Information Governance
14.07.2021
The New Normal Requires Zero Trust
27.06.2021
Microsoft 365 Records Management Whitepaper
06.11.2020
Micro Focus Launches Content Manager Select
28.05.2020
Content Manager SharePoint Integration Demo
20.05.2020
Top 7 Hidden Features in Microsoft Teams
11.02.2020
Integrate Content Manager with SharePoint
15.01.2020
Information Management Strategy Example
06.12.2019
How to Approach a Digital Transformation
13.11.2019
Top 5 Benefits of Migrating to Office 365
10.10.2019
Content Manager 9.4 Web Client
19.09.2019
Content Manager 9.4 Sneak Peek – Video Demo
02.09.2019
What Version of SharePoint is on Office 365?
27.06.2019
New SharePoint 2019 Features
08.03.2019
Cloud Governance – RIMPA Queensland Event
27.09.2018
RecordPoint Records365 Demo
09.05.2018
How RecordPoint Enhances Office 365
09.05.2018
6 GDPR Tools to Assist with GDPR Compliance
17.04.2018