Data Migration Tips: Keep your migration projects in the green

Financial services’ data migrations are complex, risky, and expensive exercises. While the concept of moving data from point A to point B is conceptually simple enough, the reality of execution is far more complicated. Although sources of such complexity will differ between migrations, there are pervasive issues that firms can expect to encounter and should be proactively addressing.

Here are five common migration themes that firms should consider in order to keep migration project plans looking green throughout.

1. Test Using Production Data
2. Cater for Sufficient Test Cycles and Time to Fix Testing Issues
3. Tackle Data Extraction Early
4. Reduce Manual Intervention Through Automation
5. Prioritise Smoke Testing Outside of Migration Events

1. Test Using Production Data

Traditional IT governance has relied on conducting testing of a solution with predefined test data. Whilst this approach is sufficient when testing functionality, data migration testing scrutinises and validates the production data itself. For that reason, the reliability of testing is dependent on the veracity of the data used.

Due to the sheer volume of financial services’ data, along with its various nuances, manufactured test data cannot compete with production or production replica data. However, due to concerns around data security, such testing should be raised early on, and adequate data governance should be put in place to mitigate risk.

2. Cater for Sufficient Test Cycles and Time to Fix Testing Issues

Migrations, especially in the beginning, should not be expected to perform flawlessly. In fact, bugs and defects are inevitable in the beginning. Comprehensive testing cycles are critical to unearthing these issues and enabling the various teams to fix and revise their respective solutions accordingly.

For this reason, it is important that sufficient time – around several weeks – is provided between migration events. This ensures that issues are adequately and thoroughly fixed, which drives quicker finalisation of the migration solution

3. Tackle Data Extraction Early

Extracting data from systems and tools has become increasingly difficult as firms embed stricter data governance and internal data controls. This is a direct result of the elevated risk of cybersecurity attacks that financial services face. However, seamless and secure extraction is still a critical component of a data migration.

Timeous and complete raw extracts enable migration teams to begin assessing data quality, understanding migration complexity, completing data mapping and enabling migration development. Migration leads must be conscious of procedural delays regarding data access and extraction and should manage them upfront and with urgency.

4. Reduce Manual Intervention Through Automation

Automation is essential to removing human error from a migration and can be leveraged to effectively control and structure data processes, thereby mitigating the risk of data violation during extraction and cutover periods. Furthermore, automation allows for exception handling to built-in to migration processes and allows process failures to be identified and addressed quickly. Data processes, including data transformations and scheduled extract-transform- load (ETL) jobs, are ideal tasks to be automated.

Data migrations are often opportunities to also tackle inefficiencies in firms’ processes. Post migration, robotic process automation (RPA) is a key technology to remove manual processes and ensure the target state solution is supported by controlled and efficient processes – such as data transformations and scheduled load jobs.

5. Prioritise Smoke Testing Outside of Migration Events

With the frequency of data migration set to continue growing as firms move to SaaS solutions, migrate off legacy systems and transform their digital capabilities, migration project teams need to refine their capabilities consistently.

Where possible, micro components – specifically those that are either specifically complex or identified as an error in previous migration events, particularly regarding conversion, verification, functional/load – should be tested between teams, outside of formal events in order to build proficiency and reduce in-event errors and issues.

Due to a lack of proactive consideration for issues, such as data quality, data migration projects can overrun their budgets by 25% to 100%.[2] With our specialised migration expertise and our deep understanding of data management and governance, Monocle is perfectly positioned to meet our clients’ data migration needs and proactively address the complex issues that continually trouble these initiatives.