With more than 80% of data migration projects running either over time or over budget, the importance of developing a thorough migration plan that clearly outlines the various data migration phases is undeniable. Following a thorough, well-defined plan becomes particularly important when dealing with complex, large amounts of data, however the same steps can and should be applied to all data migration projects.

At Miktysh, we deal with a variety of migration projects and apply the following step-by-step approach to help ensure a successful and painless migration. These steps can be summarised as discovery, data enhancement and clean-up, proof of concept, test migration and the production migration. Each of these phases incorporate the five ETL processes of analysis, extraction, transformation, loading and validation. By following these steps, we are able to improve the efficiency of the migration process, improve search and usability, and reduce the risk of unplanned downtime, lost or corrupted data or compatibility problems stemming from the migration.

Five Crucial Data Migration Phases

Data Migration Phase 1 – Discovery

The first step of the data migration phases is the discovery. This phase will help to develop a high-level understanding of the current repository, which can include information about documents, containers, fields, locations and schedules. The current approach to security and access, identification of ROT (redundant, trivial and obsolete) information, as well as current vs. target information architecture should be determined and included in a comprehensive report that details any dependencies and constraints that have been identified, to understand how it may impact the migration.

Data Migration Phase 2 – Data Clean-up and Enhancement

When moving to a new house, people typically won’t just pack up everything they own, rubbish and broken items included, and haphazardly shove them into a box to be moved into the new home. Unless you’re a hoarder, the majority of us will take some time to sort through our belongings, purging items that are no longer used or needed, improving the organisation of things and labelling the respective boxes in a logical manner. A similar approach should be taken when moving organisational data and information. A lift-and-shift approach benefits no one unless the information architecture is current and adequately meets business needs, and it’s rare that an organisation couldn’t benefit from removal of ROT data.

This phase is critical to improving the repository’s usability and driving search optimisation. It focuses on filtering out and removing redundant, trivial and obsolete information that was identified in phase one, as well as enhancing the remaining information’s metadata. Typically, when migrating from one system to another, metadata fields don’t match up one-for-one, the way metadata was captured differs, and sometimes no file metadata has been captured at all, making it more challenging to classify and move. We utilise varying techniques including optical character recognition (OCR) technology to enhance and enrich file metadata, which enables document contents to be read and thereby classified, to deliver the most efficient enterprise content management systems for our clients.

Data Migration Phase 3 – Proof of Concept

The next phase in the data migration plan is the proof of concept. Like its name denotes, this phase is used to perform an initial test migration on a set of sample data within test environments to prove the migration would be successful on the entire repository or data set. Information acquired in the discovery session is used to help perform field mapping between the current repository and the target repository. ETL scripts are then configured and tested against a trial data set in test systems. Specifically, these are extract scripts, transform scripts, load scripts and validation scripts. This phase can also help in developing a performance benchmark that can be used to help determine the time required to execute the production migration.

Data Migration Phase 4 – Test Migrations

After proof of concept and the initial test migration has been completed, the next data migration phase is to execute subsequent test migrations that iterate through the ETL processes – analysis, extraction, transformation, loading and validation. These additional test migrations are performed to refine the migration until all errors and exceptions have either been eradicated or accepted. Additional performance benchmarks are recorded and any calibrations are implemented or recommended.

Data Migration Phase 5 – Production Migration

Once all the test migrations have been successfully completed with key performance benchmarks recorded, the fifth and final step of data migration phases is initialised – the production migration. This phase takes into account what was learned in the discovery, data clean-up and enrichment, proof of concept and test migration phases, in order to put forward a final migration approach. This will include the timeframe required for the migration, which is dependent on a number of factors including the migration approach and technical infrastructure limitations such as network bandwidth, server processing power, disk read/write performance etc. Once all of this has been determined and approved by relevant stakeholders, the data migration will be executed. Data validation best practices should be utilised afterwards to ensure migration success, along with agreed upon reports. We always issue a content integrity report to validate success.

In Closing

Regardless of what repository you are migrating from or to, having a comprehensive migration plan that clearly outlines the requirements of each of the data migration phases is imperative for businesses wishing to avoid the potential catastrophe of a data migration failure.

At Miktysh we deal with all manner of data migrations to the cloud and between various repositories such as TRIM/Content Manager, SharePoint, OneDrive, Google Drive, OpenText, Documentum, Box and more. In all data migrations we follow these five data migration phases in order to execute a successful migration for our clients. If you are looking to migrate data to the cloud or between repositories, or simply want to know more information, contact our team to speak to a data migration expert.