Module 8 Final
Module 8 Final
AWS Migration and Hybrid Architecture, AWS Database Migration Service (DMS), AWS Server
Migration Service (SMS), AWS Snowball, AWS Snowball Edge, AWS Snowmobile
AWS Migration:
AWS offers a wide range of migration tools, guidance, services, and programs to help customers
assess, migrate and modernize applications and data from building the business case to
leveraging AWS services to deliver new experiences.
Services:
• Overview: A centralized service that provides visibility into the migration status of
applications across AWS and partner solutions. Cost saving and simplified experience.
• Use Case: Ideal for tracking multiple migration projects, offering a comprehensive view
of applications in various stages of migration.
• Challenges:
• May require integration with other AWS services for complete tracking.
• Users must set up and maintain their own workflows for effective monitoring.
• Dependency mapping can be complex and may not capture all interactions
accurately.
• Data collection can require substantial time and resources, especially in large
environments.
• Requires careful planning and testing to ensure data integrity during migration.
• Potential performance impacts on the source database during the migration
process.
Migration Phases:
• Assess your current situation, application architecture, challenges, and business goals.
• Build a business case for migration by defining objectives (e.g., reducing costs,
improving scalability, enhancing reliability).
• Identify which applications to migrate based on these goals.
• Examine your IT portfolio and determine migration strategies for each application.
• Explore AWS migration tools such as Server Migration Service (SMS), Database
Migration Service (DMS), and Amazon Direct Connect relevant to your use case.
• Create a detailed migration plan for applications, starting with a Proof of Concept.
• Validate migration strategies and tools in your environment, then secure stakeholder buy-
in for full migration.
Phase 5: Operate
• Operate applications within AWS, decommissioning on-premises versions unless opting
for a hybrid model.
• Use lessons learned from initial migrations to streamline subsequent application
migrations.
Strategies:
• Definition: Move applications as-is to the cloud using tools like Server
Migration Service (SMS).
• Suitable for: Large legacy migrations and organizations with limited cloud
skills.
• Pros: Simplifies the migration process; faster time to market.
• Cons: Less flexibility and inefficient use of cloud resources.
4. Refactor / Re-architect
5. Retire
• Definition: Identify and turn off applications that are no longer useful.
• Benefit: Frees up resources for more valuable applications.
6. Retain
• Challenge: Ensure high availability and resilience for applications and maintain network
connectivity.
• Solution: Use reserved instances for longevity and services like Elastic Beanstalk for
deployment. Implement active/standby IPSec tunnels and AWS Direct Connect for
resilient networking.
Hybrid Architecture
Use Cases:
• Large-scale migrations (e.g., Johnson & Johnson) use hybrid setups for
continuity during gradual AWS migration.
• VMware Cloud on AWS allows running VMware workloads on AWS,
enabling organizations like Stagecoach Group to benefit from both
environments.
• The AWS Marketplace and Outposts Service Ready program support third-
party software in hybrid or distributed models, facilitating seamless transition
and deployment of enterprise software.
• Compute
• Containers
• Storage
• Networking
• Management
Challenges:
1. Complex Management
• Managing and monitoring resources across both cloud and on-premises environments
can be complex, often requiring additional tools and expertise.
• Applications may experience latency issues due to the physical distance between cloud
and on-premises resources, which can impact performance, especially for data-intensive
workloads.
4. Cost Management
• Hybrid architectures can increase costs due to the need to maintain both on-prem and
cloud resources, as well as potential unexpected data transfer fees.
5. Integration Complexity
• Integrating legacy systems with cloud-native applications requires careful planning and
may involve compatibility issues, middleware, or custom development.
7. Operational Consistency
• AWS Database Migration Service (DMS) facilitates the migration of databases to AWS
quickly and securely. DMS supports both homogeneous (e.g., Oracle to Oracle) and
heterogeneous (e.g., SQL Server to Amazon Aurora) database migrations. It allows
continuous data replication during migration, ensuring minimal downtime for applications.
• Trusted by customers globally to securely migrate 1M+ databases with minimal downtime
• AWS Database Migration Service (AWS DMS) is a cloud service that makes it possible to
migrate relational databases, data warehouses, NoSQL databases, and other types of data
stores. You can use AWS DMS to migrate your data into the AWS Cloud or between
combinations of cloud and on-premises setups.
Benefits
• Discover, assess, convert, and migrate your database and analytics workloads to AWS with
automated migration
• Maintain high availability and minimal downtime during the migration process with Multi-
AZ and ongoing data replication and monitoring
• Supports homogeneous and heterogeneous database migrations from Oracle, SQL Server,
PostgreSQL, MySQL, MongoDB, MariaDB, and other databases
• Migrate a terabyte-sized database at a low cost, paying only for the compute resources and
additional log storage used during the migration process
Migration Process:
1. Source Database: The on-premises or cloud-based database that you want to migrate.
2. Replication Instance: An Amazon EC2 instance that manages the migration process,
including extracting data from the source, transforming it, and loading it into the target
database.
3. Target Database: The AWS database (e.g., Amazon RDS, Amazon Aurora, Amazon
Redshift) that receives the migrated data.
With AWS DMS, you can discover your source data stores, convert your source schemas, and migrate
your data.
• To discover your source data infrastructure, you can use DMS Fleet Advisor. This service collects data
from your on-premises database and analytic servers, and builds an inventory of servers, databases,
and schemas that you can migrate to the AWS Cloud.
• To migrate to a different database engine, you can use DMS Schema Conversion. This service
automatically assesses and converts your source schemas to a new target engine. Alternatively, you
can download the AWS Schema Conversion Tool (AWS SCT) to your local PC to convert your source
schemas.
• After you convert your source schemas and apply the converted code to your target database, you can
use AWS DMS to migrate your data. You can perform one-time migrations or replicate ongoing
changes to keep sources and targets in sync. Because AWS DMS is a part of the AWS Cloud, you get
the cost efficiency, speed to market, security, and flexibility that AWS services offer.
At a basic level, AWS DMS is a server in the AWS Cloud that runs replication software. You create a
source and target connection to tell AWS DMS where to extract data from and where to load it. Next,
you schedule a task that runs on this server to move your data. AWS DMS creates the tables and
associated primary keys if they don't exist on the target. You can create the target tables yourself if you
prefer. Or you can use AWS Schema Conversion Tool (AWS SCT) to create some or all of the target
tables, indexes, views, triggers, and so on.
• Minimal Downtime: Continuous data replication ensures that the source database remains
operational during the migration.
• Broad Database Support: Supports a wide range of source and target databases, including
Amazon RDS, Amazon Aurora, Amazon Redshift, and on-premises databases.
• Schema Conversion: When migrating between different database engines, the AWS
Schema Conversion Tool (SCT) helps convert the database schema and code.
• Monitoring and Logging: Provides detailed monitoring of the migration process through
Amazon CloudWatch.
• High Availability: Supports multi-AZ deployment for high availability.
1. Set Up a Replication Instance: Create and configure a replication instance in AWS DMS.
2. Create Source and Target Endpoints: Define the source and target databases, including
their connection information.
3. Create and Run a Migration Task:
o Choose the migration type: full load, full load with ongoing replication, or ongoing
replication only.
o Configure settings like data transformation and mapping rules.
4. Monitor the Migration: Use AWS DMS dashboard and Amazon CloudWatch for real-
time monitoring of the migration process.
5. Cutover: Once the data is fully migrated, cutover to the new database.
Types of Migration:
• Homogeneous Migration: Moving between the same database engines (e.g., Oracle to
Oracle).
• Heterogeneous Migration: Moving between different database engines (e.g., SQL
Server to PostgreSQL). Requires schema transformation using AWS SCT.
AWS DMS takes over many of the difficult or tedious tasks involved in a migration project:
1. In a traditional solution, you need to perform capacity analysis, procure hardware and software, install
and administer systems, and test and debug the installation. AWS DMS automatically manages the
deployment, management, and monitoring of all hardware and software needed for your migration.
Your migration can be up and running within minutes of starting the AWS DMS configuration process.
2. With AWS DMS, you can scale up (or scale down) your migration resources as needed to match your
actual workload. For example, if you determine that you need additional storage, you can easily
increase your allocated storage and restart your migration, usually within minutes.
3. AWS DMS uses a pay-as-you-go model. You only pay for AWS DMS resources while you use them, as
opposed to traditional licensing models with up-front purchase costs and ongoing maintenance
charges.
4. AWS DMS automatically manages all of the infrastructure that supports your migration server,
including hardware and software, software patching, and error reporting.
5. AWS DMS provides automatic failover. If your primary replication server fails for any reason, a backup
replication server can take over with little or no interruption of service.
6. AWS DMS Fleet Advisor automatically inventories your data infrastructure. It creates reports that help
you identify migration candidates and plan your migration.
7. AWS DMS Schema Conversion automatically assesses the complexity of your migration for your source
data provider. It also converts database schemas and code objects to a format compatible with the
target database and then applies the converted code.
8. AWS DMS can help you switch to a modern, perhaps more cost-effective, database engine than the
one you are running now. For example, AWS DMS can help you take advantage of the managed
database services provided by Amazon Relational Database Service (Amazon RDS) or Amazon Aurora.
Or it can help you move to the managed data warehouse service provided by Amazon Redshift, NoSQL
platforms like Amazon DynamoDB, or low-cost storage platforms like Amazon Simple Storage Service
(Amazon S3). Conversely, if you want to migrate away from old infrastructure but continue to use the
same database engine, AWS DMS also supports that process.
9. AWS DMS supports nearly all of today's most popular DBMS engines as source endpoints. AWS DMS
provides a broad coverage of available target engines.
10. You can migrate from any of the supported data sources to any of the supported data targets. AWS
DMS supports fully heterogeneous data migrations between the supported engines.
11. AWS DMS ensures that your data migration is secure. Data at rest is encrypted with AWS Key
Management Service (AWS KMS) encryption. During migration, you can use Secure Socket Layers (SSL)
to encrypt your in-flight data as it travels from source to target.
Use Cases
• Move to managed databases
• Migrate from legacy or on-premises databases to managed cloud services through a
simplified migration process, removing undifferentiated database management tasks.
• Remove licensing costs and accelerate business growth
• Modernize to purpose-built databases to innovate and build faster for any use case at
scale for one-tenth the cost.
• Replicate ongoing changes
• Create redundancies of business-critical databases and data stores to minimize downtime
and protect against any data loss.
• Improve integration with data lakes
• Build data lakes and perform real-time processing on change data from your data stores.
•
Import/Export Disk
o It accelerates moving large amounts of data into and out of the AWS cloud using portable
storage devices for transport.
o For example, if you have 500 TB data and you got a slow internet connection, i.e., 1mbps.
Instead of sending the data over the internet, you can send it to Amazon through an external
hard disk, and they would transfer your data directly onto and off of storage devices using
Amazon's high-speed internal network and bypassing an internet.
o Lots of people started using it, and they were all sending different types of disks, connections
which became difficult to manage.
Re: invent 2015: Amazon released Standard Snowball.
Snowball
o Snowball is a petabyte-scale data transport solution that uses secure appliances to transfer
large amounts of data into and out of aws.
o It is a streamline bringing the data into aws and bypassing an internet. Instead of managing
all the external disks, Amazon provided you an appliance, and you loaded an appliance with
the data. Finally, the data is export from the appliance to Amazon S3.
o The common challenges with large scale data transfers such as high network costs, long
transfer time, and a security issue have been resolved by using Snowball addresses.
o Transferring data with Snowball is simple, fast, secure and one-fifth of the cost of the high-
speed internet.
o Finally, there are 80TB Snowball in all the regions.
o Snowball provides tamper-resistant enclosures, 256-bit encryption, and an industry-standard
Trusted Platform Module (TPM) to ensure security.
o Once the data transfer job has been processed and verified, the AWS performs software
erasure of the software appliance.
Benefits
Simple migration
Easily migrate terabytes of data to the cloud without limits in storage capacity or compute power.
Accelerate performance
Accelerate application performance in disconnected, austere edge environments and run compute
workloads with little or no connectivity.
Protect data
Protect your data in transit with Snowball’s ruggedized chassis, integrated logistics, and tamper-
evident box, and get data to the right place quickly.
AWS Snowball Edge
AWS Snowball Edge is a type of Snowball device with on-board storage and compute power for
select AWS capabilities. Snowball Edge can process data locally, run edge-computing workloads,
and transfer data to or from the AWS Cloud.
Each Snowball Edge device can transport data at speeds faster than the internet. This transport is
done by shipping the data in the devices through a regional carrier. The appliances are rugged,
complete with E Ink shipping labels.
o Snowball Edge is a 100 TB data transfer device with on-board storage and
compute capabilities.
o Snowball Edge is like an AWS data center that you can bring on-premises.
o Snowball edge can be used to move large amounts of data into and out of AWS.
o We can also run Lambda functions from Snowball edge, it brings compute capacity
where we are not able to do it. For example, an Aircraft engine manufacturer can
place the Snowball edge on to the Aircraft to gather the information of how aircraft
engine is running. When the Aeroplane lands, take out the Snowball edge from the
Aircraft and ship it to the AWS Data Center. Therefore, we observe that the
Snowball edge has both storage and compute capacity.
If AWS detects an irreparable issue, and there is a need to replace physical equipment,
AWS will notify you. You can then place a replacement job that we will ship to your site.
There is no additional charge for this, as Snow device monitoring is included as part of
the Snow device service fee.
AWS Snowmobile
o It was announced in re: invent 2016.
o A Snowmobile is an exabyte-scale data transfer service.
o It can transfer large amounts of data in and out of AWS.
o You can transfer 100 PB per Snowmobile, a 45-foot long ruggedized shipping
container, pulled by a semi-trailer truck.
o Snowmobile allows you to move massive volumes of data to the cloud, including
video libraries, image repositories or even a complete migration of data center.
o Transferring data with Snowmobile is secure, fast and cost-effective.
AWS Snowmobile:
AWS Snowmobile is a petabyte to exabyte-scale data transfer service that physically transports
massive amounts of data to AWS. It is designed for organizations needing to migrate vast data
volumes, such as entire data centers, quickly and securely when network-based data transfer
would be impractical due to time, cost, or bandwidth limitations.
1. Capacity:
o AWS Snowmobile can store up to 100 petabytes (PB) of data per unit.
o For larger migrations, multiple Snowmobiles can be deployed, capable of transferring
exabyte-scale data.
2. Physical Specifications:
o A Snowmobile is a 45-foot long shipping container mounted on a truck, specially
designed to transport large amounts of data securely.
o It is equipped with high-speed data transfer capabilities to load data from the
customer’s location onto the device.
3. Data Security:
o End-to-end encryption: Data is encrypted using 256-bit encryption, and keys are
managed through AWS Key Management Service (KMS).
o Access Control: The Snowmobile comes with dedicated security personnel, GPS
tracking, and video surveillance during transit.
o Physical Security: Tamper-evident seals and sophisticated security measures protect the
data during transport.
4. Process:
o Assessment and Planning: AWS collaborates with the customer to understand their
data transfer needs and plan the Snowmobile deployment.
o On-site Setup: The Snowmobile is delivered to the customer’s data center or facility,
and AWS personnel help set up the data transfer.
o Data Transfer: The data is loaded onto the Snowmobile via high-speed connections. This
process can take days to weeks, depending on the volume of data.
o Transport to AWS: Once the data is loaded, the Snowmobile is securely transported
back to an AWS Region where the data is ingested into AWS services such as Amazon S3
or Amazon Glacier.
5. Use Cases:
o Data Center Migration: Ideal for businesses relocating large-scale data from on-
premises data centers to AWS.
o Disaster Recovery: Helps in quickly migrating backup data to AWS for disaster recovery
planning.
o Media Archives: Suitable for transferring extensive video, film, and content libraries.
o Scientific Data: Used by research institutions for moving large datasets like genome
data, satellite images, or simulations.
6. Advantages:
o Speed: Can transfer petabytes of data much faster than over traditional internet or even
dedicated high-speed connections.
o Cost Efficiency: Reduces the costs associated with high-bandwidth network data
transfers over long periods.
o Security: Provides robust physical and digital security, ensuring data integrity during
transit.
7. Alternatives:
o For smaller data transfers, AWS offers Snowcone and Snowball devices. These are
designed for terabyte-scale transfers but lack the capacity of Snowmobile.