Welcome to the third blog post in the overview series dedicated to our 12-step Migration and Modernization Methodology that was adopted by AWS. In our first and second blog posts, we explored the critical role of defining workloads in migration projects and laid out the process of capturing the current architecture and designing the future state architecture as part of the first step of the methodology — Future State Architecture Design.
This provided a solid foundation for understanding the scope and dependencies of the migration effort. In this third post, we’ll dive deeper into the Migration Analysis phase, which focuses on estimating project efforts and establishing a robust project plan.
If you’d like to see the same information covered in this blog post presented visually, check out the third video from our four-part video series.
Migration Analysis – Project scope and effort estimation
Once we defined the workload and current and future state architecture, we move to the next phase — collecting and analyzing all necessary artifacts associated with the workload components, including database schemas, application codes, scripts, jobs, and ETL processes. The analysis phase is essential for steps 2 through 5, covering various aspects such as database schema conversion, application conversion and remediation, scripts and reports transition, and integration with third-party applications. Each artifact contributes to understanding the complexity and specific challenges we might face during migration.
Step 2: Database Schema Conversion
Our approach to migration analysis involves leveraging tools like AWS Schema Conversion Tool (AWS SCT) and AWS Database Migration Service (AWS DMS) when migrating to AWS. AWS SCT provides detailed migration assessment reports that help identify areas where automatic conversion is possible and where manual conversion will be needed. Similarly, for Azure or SQL Server migrations, we employ Azure Database Migration Service or SQL Server Migration Assistant (SSMA) to perform the same function, while for migrations to Google Cloud, we leverage Database Migration Service (DMS). These tools generate comprehensive reports that outline:
- Types of database objects (e.g., tables, constraints, indexes) and their conversion statistics.
- Automatic and manual conversion rates for each object type.
- Complexity and frequency of conversion issues (referred to as Action Items).
Each instance that necessitates manual conversion is referred to as an “Action Item,” which further specifies the frequency and complexity of each conversion issue. The Migration Engineering Team reviews each action item, assessing their impact on the overall project and revising effort estimates accordingly. This detailed analysis informs us about the required manual work for database schema conversion, allowing us to plan effectively and avoid unexpected delays during migration.
Step 3: Application Conversion / Remediation
The next step involves analyzing the application code that interacts with the converted database schema. During this phase, our engineers focus on evaluating the required changes in SQL queries, connection mechanisms, transaction management, and error-handling processes to ensure seamless integration with the future state architecture.
For instance, converting a C++ application to integrate with a PostgreSQL database might involve translating thousands of SQL queries and validating their correctness. The report generated during this stage categorizes each query as either automatically convertible or requiring manual intervention. The estimates then account for the time needed to validate auto-converted queries and resolve action items for manual conversions.
Step 4: Scripts, ETL, and Reports Conversion
The fourth step focuses on converting scripts, ETL jobs, and reports, which are integral parts of the workload. We leverage the same set of analysis tools as in the previous steps, depending on the target cloud provider, to understand how existing scripts and ETL workflows must be modified to accommodate the new data structures and connections.
Step 5: Integration with Third-Party Applications
In the integration step, we focus on identifying how components within the workload interact with third-party systems. Each integration point is thoroughly analyzed to determine which modifications are necessary to re-integrate these systems with the migrated or modified components. Based on this analysis, we define the required actions, such as adjusting APIs, reconfiguring data connectors, or updating configuration settings. After outlining these tasks, we provide a detailed effort estimation to assess the scope of work needed to ensure successful re-integration.
Step 6: Data Migration Mechanism
The data migration mechanism outlines a step-by-step approach to transferring data between the source and target environments. This involves mapping data types, scripting keys and constraints for efficient data transfer, and setting up Change Data Capture (CDC) to handle incremental updates during the migration.
The complexity of data migration is often tied to differences in data types and structures between source and target databases. For example, resolving Character Large Object (CLOB) data type differences between Oracle and PostgreSQL can be a time-intensive task, depending on the volume of data and the number of affected tables.
The steps 7 through 9 (Testing, Performance Tuning, and DevOps/Security) are usually driven by the analysis of previous steps.
Step 7: Testing and Bug Fixing
Testing plays a crucial role in validating the converted components and ensuring that they function as expected in the future state architecture. This step includes unit testing, system integration testing, and performance testing. Each test is designed to uncover potential issues early in the process, reducing the likelihood of disruptions during production cutover.
The testing phase often involves extensive coordination between development and QA teams. It may also include customer involvement, particularly for user acceptance testing (UAT) and performance validation.
Step 8: Performance Tuning
The Performance Tuning step focuses on optimizing database and application performance to ensure that the migrated solution meets or exceeds the performance levels of the legacy system. This involves refining indexes, rewriting queries to leverage new database capabilities, and implementing partitioning strategies. Additionally, memory and resource allocations are adjusted to enhance overall system efficiency and stability.
Step 9: Setup, DevOps, Integration, Deployment, and Security
This step establishes a secure and robust operational environment. This includes configuring the new system, integrating CI/CD pipelines for automated deployment, implementing version control for code and configurations, and setting up security measures to protect data and application integrity. Finally, it involves testing all integration points to ensure seamless operation across the entire architecture.
Step 10: Documentation and Knowledge Transfer
This step ensures that the entire migration process is well-documented for future reference. It includes creating runbooks, migration reports, and detailed descriptions of the new environment configurations. Knowledge transfer sessions are conducted with the client’s team, providing them with the necessary understanding to maintain and support the new system.
Step 11: Project Management and Version Control
The Project Management and Version Control step spans all 12 phases of the migration project. It involves planning, resource allocation, risk management, and continuous monitoring to ensure the project stays on schedule. Version control systems are used to maintain consistent updates across the project’s lifecycle, and any changes are carefully managed and tracked.
Step 12: Post-Production Support
The Post-Production Support step guarantees a smooth transition from project completion to full operational status. It involves resolving any initial issues that may arise, providing continuous monitoring and incident response, and making minor enhancements based on client feedback. This support phase ensures that the migrated environment functions reliably and meets the client’s performance expectations.
Detailed breakdown of effort estimation
The table below presents a comprehensive breakdown of effort estimates for each of the 12 steps in our migration methodology. This structured approach enables us to provide clients with an accurate understanding of the effort required for each phase of the migration, ultimately forming the basis for the project plan and the final migration proposal.
Creating the Project Plan
Once the analysis is complete, the focus shifts to crafting a detailed project plan. This plan outlines the migration strategy for each workload component and maps out the timelines and dependencies between tasks. The project plan is designed to maximize parallel execution of tasks, minimizing the overall project timeline.
Conclusion and what’s next
In this third blog post, we’ve delved into the Migration Analysis phase. We’ve described how we leverage various tools and methodologies to perform a detailed analysis, estimate project efforts, and create a comprehensive project plan. This phase is essential for identifying potential challenges, aligning expectations, and ensuring that the project is on track to meet business objectives.
In our next blog post, we’ll cover the final step before execution: the Migration Solution Document. This document consolidates all our findings, strategies, and plans into a formal proposal that outlines the entire migration process, ensuring transparency and providing a roadmap for the client’s approval.
