SAP Data Modernization

Migration Design Patterns

1
SAP Modernization on GCP

2
SAP Data Modernization
Why enterprises need a modern data platform?

Complexity Flexibility Expertise Integrations Performance and Cost

Scalability

• SAP data • SAP ecosystem • Lack of strong • Lack of robust • Query • High Operations
analytics has a rigid technical tools and performance cost and
ecosystem is architecture expertise in developer issues with data maintenance
complex with which limits the building and ecosystem for growth • Increased
multiple ability to quickly managing integrations • Limitations with licensing costs
components and adapt to evolving SAP • Time consuming respect to with data growth
layers changing landscape and costly scalability
business • Lack of pace in development • Hardware limits
requirements development of cycles often reached
new data models

3
SAP Data Modernization to GCP
Data Integration Tools

Providing customer choice to leverage data

integration and harmonization solutions that
complement existing skill sets, investments, and
strategies

Google Tooling 3rd Party Tooling

• Leverage existing investment in SAP System Landscape • Leverage existing partner solution investments -
Replication Server (SAP SLT) Decoupled from SAP SLT
• 3rd party data integration solutions offer robust
• Cortex Data Foundation compatible with raw table data integration across source and targets beyond SAP ERP
replication in near real time and BigQuery
• Use BigQuery Connector for SAP to connect SAP ERP and • They offer additional data management capabilities such
BigQuery directly via Google Cloud supported SLT addin as data quality and harmonization features
• SAP Integration tooling - Data Services, Data Intelligence
• Use Cloud Data Fusion SAP SLT Replication Plugin to • 3rd party tools - Informatica, Fivetran, Qlik, Palantir etc.
connect SAP ERP with Google Cloud Storage and
BigQuery, and take advantage of hundreds of other
plugins

4
SAP Migrations
Google/Partner Tooling Options On Premises GCP Cloud

SAP BO GCP Project

Dashboards

SAP ERP (OLTP)

2
SAP BW File Server
App Layer
SAP Business Suite 7,
Infocubes, Infoproviders, Data store 2 Extracted data in CSV, 2
Objects Parquet
SAP ECC (Netweaver) Data extraction API

DB Layer 1 SAP LT Replication

Server
DB BQ
4 Connector 1
Cloud Interconnect / VPN
1
4
S4/HANA SAP HANA (OLAP)
App Layer
3 3

SAP DS / DI*
3 4
App Data and DB data

1 SAP SLT - Extract and move data from from SAP apps (SAP ECC) to BigQuery 4 SLT and SAP Data Services - Extract and move data from from SAP apps (SAP
ECC) to BigQuery
2 SAP BW Data Extraction API - Extract and move data from from SAP BW to BigQuery

3 SAP Data Services / Data Intelligence* - Extract and move data from from SAP apps
(S/4Hana) to BigQuery
* Part of SAP BTP 5
SAP Migrations
Google Tooling Options - Cloud Data Fusion plugins

SAP Ariba GCP Project

• Cloud Data Fusion is a fully managed,
cloud-native, enterprise data integration
SAP BW Open
Hub service for quickly building and managing
data pipelines.
SAP OData • The Cloud Data Fusion web UI lets you to
build scalable data integration solutions to
SAP ODP Cloud Data clean, prepare, blend, transfer, and
Fusion Plug-in
transform data, without having to manage
SAP SLT
the infrastructure.
• Cloud Data Fusion is powered by the open
source project CDAP.
SAP Success
Factors

SAP Table
Batch

6
SAP Migrations
BigQuery Connector
Pre-requisites:
On Premises GCP Cloud

Supported SAP Apps

• SAP Business Suite 7
SAP LT Replication GCP Project
• S/4Hana
SAP Apps Server • SAP Apps running on SAP Netweaver
SAP Apps BigQuery
SAP Business Suite 7,
S/4Hana, SAP ECC BQ Cloud Interconnect / VPN SAP Data Licensing:
(Netweaver)
Connector • SAP licensing (SLT)
• BigQuery Connector - Free

SLT S/W Requirements:

https://cloud.google.com/solutions/sap/docs/bq-connector/late
st/planning#s4hana

Limitations:
SAP recommends against replicating data from SAP Business
Warehouse by using SAP LT Replication Server

7
SAP Migrations
SAP Tooling Options - Pros and Cons
Tool Description Pros Cons
SAP LT • SLT is the an ETL ( Extract , Transform , • Real-time/Schedule based data replication and • Limited data transformation capabilities
Replication Load ) tool which allows us to load and transformation tool compared to SAP Data Services
Server replicate data in real-time or schedule • Simple configuration and monitoring through GUI • Can only replicate data from a limited number of
data from SAP source system or Non-SAP • Provides filtering and transformation options for source systems
System into SAP HANA Database. data transformations • Works at table level and not business layers
• SAP SLT server uses a trigger-based • Offers HA and DR functionalities
replication approach to pass data from • Provides low latency for real-time data replication
source system to target system. • Less licensing cost compared to DS

SAP Data • SAP Data Services improves the quality of • Provides advanced and more comprehensible data • More complex configuration and setup than SLT
Services data across the enterprise. transformation capabilities • Longer implementation time compared to SLT
• As part of the information management • Supports batch processing of large volumes of data • Higher Licensing costs compared to SLT
layer of SAP’s Business Technology • Provides data quality and metadata management
Platform, it delivers trusted, relevant, and features
timely information to drive better • Can integrate with multiple data sources and targets,
business outcomes. including SAP HANA
• Provides a flexible and customizable approach to
data integration and migration
SAP Data • SAP Data Intelligence Cloud is a • Cloud-based platform provided as a service • Highly complex data integration tool that needs
Intelligence comprehensive data management • End-to-end data pipeline from data integration and skills and expertise
Cloud solution transformation to machine learning and data • Some integration scenarios will need heavy
• As the data orchestration layer of SAP visualization customization
Business Technology Platform, it • Open platform that supports a wide range of data
transforms distributed data sprawls into sources and data formats, as well as integration with
vital data insights, supporting innovation third-party tools and platforms.
and business growth. • Machine learning and AI capabilities including pre-
built machine learning models, model training and
deployment, and support for TensorFlow and other
popular machine learning frameworks
• Workflow management capabilities, to streamline
data processing and analytics workflows, and
automate data processing tasks

8
SAP Modernization on Azure

9
SAP Data Modernization to Azure
Data Integration Tools

SAP Data Integration scenarios:

 One time migration of historical data from SAP
 Batch Data ingestion from SAP
10
SAP Data Modernization to Azure
Data Integration Tools

Providing customer choice to leverage data

integration and harmonization solutions that
complement existing skill sets, investments, and
strategies

Azure Tooling 3rd Party Tooling

• Leverage Microsoft provided Azure Data Factory • Leverage existing partner solution investments -
Connectors for SAP source systems Decoupled from SAP SLT
• 3rd party data integration solutions offer robust
• SAP ECC and SAP Table Connectors use Netweaver stack integration across source and targets beyond SAP ERP
on SAP for different requirements and Azure Synapse
• SAP HANA Connector allows ODBC/JDBC connections • They offer additional data management capabilities such
as data quality and harmonization features
• SAP BW systems data can be extracted with BW Open • SAP Integration tooling - Data Services, Data Intelligence
Hub or MDX Connectors • 3rd party tools - Informatica, Fivetran, Qlik, Theobald etc.
• SAP CDC Connector (In Public Preview) uses SAP ODP
stack to enable Delta / incremental loads

11
SAP Migrations
Azure Tooling Options - I On Premises Azure Cloud

SAP Gateway 1

ODATA Azure Subscription

1
SAP ERP (OLTP) SAP .Net
App Layer Library
SAP Business Suite 7,
SAP ECC (Netweaver) 2
Data Lake Gen2
DB Layer SAP HANA Azure SHIR
Connector Server
DB Self Hosted
ODBC
4 Integration Runtime ExpressRoute/VPN
Data Factory
5
S4/HANA SAP HANA (OLAP)
App Layer Azure Synapse
3 3

SAP DS / DI*
3
App Data and DB data

1 SAP ECC Connector - SAP ECC Connector uses SAP ODATA services to extract data. SAP 4 SAP HANA Connector- Extract and move data from HANA database to Azure
ECC on SAP NetWeaver version 7.0 (EHP 7 has >= 7.4).
2 SAP Table Connector - Extract and move data from from SAP BW to BigQuery
5 SHIR - Self Hosted Runtime is the common connection point between SAP
3 SAP Data Services / Data Intelligence* - Extract and move data from from SAP apps Sources and Azure Cloud. SHIR is hosted near Sources for improved
(S/4Hana) to Azure Data Stores performance and security
12
SAP Migrations
Azure Tooling Options - II On Premises Azure Cloud

SAP BW
Infocubes, Infoproviders, Data store
Objects
Open Hub
Data extraction API Azure Subscription
Destinatio
n Table
2
RFC
SAP ERP (OLTP) SAP
Netweaver 2
App Layer Library
SAP Business Suite 7, 4
SAP ECC (Netweaver)
Data Lake Gen2
DB Layer 1
Azure SHIR
1 SAP LT Replication Server
DB Server Self Hosted
ExpressRoute/VPN
Integration Runtime Data Factory
5
S4/HANA
App Layer Azure Synapse
SAP HANA (OLAP)
3
3

SAP DS / DI*
3
App Data and DB data

1 SAP SLT - Extract and move data from from SAP apps (SAP ECC) to SAP BW 4 SAP MDX BW Connector- Extract and move data from SAP BW to Azure

2 SAP Open Hub BW Connector- Extract and move data from from SAP BW to Azure
5 SHIR - Self Hosted Runtime is the common connection point between SAP
3 SAP Data Services / Data Intelligence* - Extract and move data from from SAP apps Sources and Azure Cloud. SHIR is hosted near Sources for improved
(S/4Hana) to BigQuery performance and security
* Part of SAP BTP 13
SAP Migrations
Azure Tooling Options - III On Premises Azure Cloud

SAP ECC
Extractors
Azure Subscription

ABAP CDS
Views
ODP API

SAP ECC
Data Lake Gen2
Extractors
SAP ODP Azure SHIR
Framework Server
1 Self Hosted
ExpressRoute/VPN
SAP BW Integration Runtime Data Factory
SAP BW4HANA
2

Azure Synapse
SAP SLT Server

SAP HANA
Views

1 SAP CDC Connector** - SAP CDC Connector uses the ODP Framework for SAP Sources. 2 SHIR - Self Hosted Runtime is the common connection point between SAP
This supports both SAP ECC, BW and S4/HANA systems. There is support of CDC / Delta Sources and Azure Cloud. SHIR is hosted near Sources for improved
Change load as provided by SAP ODP Framework. performance and security

** In Public Preview 14
Azure Tooling Choices
Connectors for SAP ERP sources

ADF connector
SAP Table SAP ECC SAP CDC**
options
OData entities exposed via ODP framework based
Table (Transparent, Pooled, Cluster
Objects to extract SAP Gateway (BAPI, extraction. Good support of
Table) and View
ODP) Delta loads
SAP side configuration None SAP Gateway SAP ODP Framework

Queue based, can be used for

Fast w/ built-in parallel loading based
Performance Slower throttling extractions. Slower
on configurable partitioning
compared to Table Connector.

Medium Data volumes. Delta

Suitable workload Large volume Small volume
Loads.

** In Public Preview
15
Azure Tooling Choices
Connectors for SAP BW Sources

ADF connector SAP BW SAP BW

SAP Table
options Open Hub via MDX
DSO, InfoCube ,
Table (Transparent, Pooled, Cluster
Objects to extract MultiProvider ,DataSource
Table) and View
, etc InfoCubes, QueryCubes
SAP Open Hub
SAP side configuration None
Destination None

Fast w/ built in parallel

Fast w/ built-in parallel loading based
Performance loading based on OHD
on configurable partitioning
specific schema
Slower
Well thought through Exploratory workload Small
Suitable workload Large volume
workload. Large volume volume

16
Prerequisites - SAP ERP

SAP Table Connector SAP ECC Connector SAP CDC Connector

• ECC NetWeaver 7.01 or higher • SAP NetWeaver version 7.0 or higher • SAP ODP Framework
• SAP BW NetWeaver 7.01 or higher • SAP Gateway installed (NetWeaver 7.4 • Self-Hosted IR
has Gateway by default)
• SAP S4/HANA • For CDS view based attraction
• Self Hosted IR NetWeaver 7.4 or higher
• Self Hosted IR
Capabilities • For CDC Delta extractions NetWeaver 7.5
Capabilities or higher is needed.
• Extract SAP ECC entities via OData
• High volume, parallel table loads • SLT server is needed if we want to use
services
SLT as a proxy for CDC (not mandatory)
Limitations • OData query options for filter
• OOTB CDC is not available, use RFC Capabilities
Limitations
filters based on timestamp columns for • Extract SAP ECC entities via OData
Delta loads • OOTB CDC is not available, use RFC filters services
based on timestamp columns for Delta loads
• Supports ECC, S4/HANA, BW,
• CDS views are not supported BW/4HANA with ODP framework
• Standard 7500+ extractors, 2500+ CDS
views with SAP content are available
Limitations
• Only Mapping Data flow is supported for
CDC/Delta extractors

17
Prerequisites - BW Connectors

SAP BW Open Hub

SAP BW MDX Connector SAP CDC Connector
Connector

• SAP BW NetWeaver 7.01 or higher • SAP NetWeaver version 7.0 or higher • SAP ODP Framework
• Self Hosted IR • SAP Gateway installed (NetWeaver 7.4 • Self-Hosted IR
has Gateway by default)
Capabilities • For CDS view based attraction
• Self Hosted IR NetWeaver 7.4 or higher
• Supports DSO, InfoCube, MultiProvider,
DataSource Capabilities • For CDC Delta extractions NetWeaver 7.5
or higher is needed.
• Delta Load using Open Hub Data is • Extract SAP ECC entities via OData
supported, need to manage “last services • SLT server is needed if we want to use
requestId” loaded locally in Azure SLT as a proxy for CDC (not mandatory)
• OData query options for filter
Limitations Capabilities
Limitations
• Extract SAP ECC entities via OData
• BW4/HANA is not supported • OOTB CDC is not available, use RFC services
filters based on timestamp columns for
Delta loads • Supports ECC, S4/HANA, BW,
BW/4HANA with ODP framework
• Standard 7500+ extractors, 2500+ CDS
views with SAP content are available
Limitations
• Only Mapping Data flow is supported for
CDC/Delta extractors

18
SAP Modernization on AWS

19
SAP Data Modernization to AWS
Data Integration Tools

A high-level architecture for end-to-end enterprise analytics is

shown above.
20
Database level Extraction- Tool – Option 1

Database-level extraction, as the name suggests, taps into SAP data at database level. There are various APN Partner solutions— Attunity
Replicate, HVR for AWS, and others—that capture raw data as it is written to the SAP database transaction logs. They transform it with
required mappings and store the data in S3. These solutions are also able to decode SAP cluster and pool tables

Key
Third Party: considerations AWS Glue / AWS
• Third-party adapters pull data from transaction logs, there • Lambda:
Only can pull data from HANA database.
is minimal performance impact to the SAP database • No additional software required; can use JDBC drivers
application or Node.js/python packages to pull data from the
• Supports data change capture database.
• Potential license restrictions • No support for capturing data changes
• Doesn’t retain application context; need to rebuild • Additional custom development effort.
transformations and re-establish relationships between
tables.

21
Application level Extraction- Tool – Option 2

In SAP ERP applications, business logic largely resides in the ABAP layer. Even with the code push-down capabilities of SAP HANA
database, the ABAP stack still provides an entry point for API access to business context.
Key
Third Party: considerations AWS Glue / AWS
• SAP has also published “PyRFC”(Python RFC) library that can be
Lambda:
• Application-level extractors like SAP Data Services used in AWS Glue or Lambda to natively integrate using RFC SDK.
extract data from SAP applications using integration SAP IDOCs can be integrated with S3 using an HTTP push pattern.
frameworks in ABAP stack and store it in S3 through • The SAP NetWeaver RFC SDK offers a C/C++ interface for connecting
default connectors. Using Remote Function Call (RFC to SAP systems from release R/3 4.6C up to today's SAP S/4HANA
SDK) libraries, these extractors are able to natively systems.
connect with SAP applications to pull data from remote • Using AWS native services like AWS Glue or Lambda removes the
function modules, tables, views, and queries. requirement for a third-party application, hence reducing the total cost
• Less Transformation work outside of SAP. of ownership. However, customers might see an increase in custom
development effort to wire the HTTP or RFC integrations with SAP
applications.

22
Operational data provisioning–based extraction tool – Option 3(SAP BW)

SAP Data Intelligence

The Operational data provisioning (ODP) supports both full data extraction as well as change data capture using operational delta queues.
The business logic for extraction is implemented using SAP Data Sources (transaction code RSO2), SAP Core Data Services (CDS) Views,
SAP HANA Information Views, or SAP Landscape Replication Server (SAP SLT). ODP, in turn, can act as a data source for OData services,
enabling REST-based integrations with external applications.
• SAP_BASIS < 730 - SAP Note - 1521883 - ODP Replication API 1.0
• SAP_BASIS >= 730 - SAP Note 1931427 - ODP Replication API 2.0
For a functional comparison between ODP 1.0 and ODP 2.0, see SAP Note 2481315 – Operational Data
Provisioning (ODP): Extracting from SAP Systems to SAP BW or SAP BW/4HANA – Availability and
Limitations
Key
• considerations
Because business logic for extractions is supported at application • Data Services and Data Hub might have better performance in
layer, the business context for the extracted data is fully retained. pulling the data from SAP because they have access to ODP
• All table relationships, customizations, and package configurations in integration using RFC layer. SAP hasn’t opened the native RFC
the SAP application are also retained, resulting in less transformation integration capability to ODP for non-SAP applications, so AWS
effort. Glue and Lambda must rely on HTTP-based access to OData.
• Change data capture is supported using operation delta queue
mechanisms. Full data load with micro batches is also supported using
OData query parameters.
23
 SAP Landscape Transformation (SLT)-based extraction – option 3

SAP Data Intelligence

Basic requirements for the SAP LT Replication Server:

• SAP NetWeaver 7.02 or greater
• SAP Kernel 7.20 patch level 110 or higher
• Unicode
• Same or higher SAP Basis release than all connected system
SAP Landscape Transformation Replication Server (SLT) supports near real-time and batch data replication from SAP applications.

Key
• considerations
Trigger based replication; supports change data capture.
• An SAP SLT enterprise license might be required for replicating to non-SAP-supported targets.
• Additional custom development in ABAP is required to integrate with targets not supported by SAP
• Additional license cost will be incurred if opted for SAP Data Intelligence.
• Additional custom development effort in AWS Glue when replicating from an SAP-supported database to S3.

24
Architecture Options for Extracting SAP Data with
AWS Services

25
Architecture Design Pattern Characteristics

Middleware
Number Architecture Pattern Extraction Method Delta Handling Pros and Cons
Services
S/4HANA or ECC 6.0 EHP7/8, OData, with SAP Gateway

A1 S/4HANA or ECC 6.0 EHP7/8 with pre-built OData Pre-Built Standard OData Consider timestamp Amazon Amazon Appflow is a serverless and no-code managed AWS service
Services Services field AppFlow which can extract and write back to SAP.
AWS Glue/Lambda require you to deploy code, maintain and upgrade
when necessary.
SAP Data Intelligence subscription is part of SAP BTP (Business
AWS
Technology Platform) with a pay-per-use model. SAP Data Services
Glue/Lambda
requires a perpetual license.

SAP Data
Intelligence
A2 S/4HANA or ECC 6.0 EHP7/8 with Data Extractors Standard BW Extractors Delta is handled Low upgrade impact because the BW Extractors are standard.
(BW extractor) through OData (ODP Based) within ODP
SAP Data
Services
A3 S/4HANA or ECC 6.0 EHP7/8 with Custom OData Custom OData (ABAP CDS Consider timestamp Custom ABAP CDS views and custom OData Services maintenance
Services View) field fixes will be required especially during upgrade.

ECC 6.0 EHP8 or prior, RFC, no SAP Gateway

A4 ECC 6.0 EHP7/8 or earlier with Data Extractors Standard BW Extractors (ODP Delta is handled within AWS AWS Glue/Lambda require you to deploy code, maintain and upgrade
(BW Extractors) thru RFC Based) ODP Glue/Lambda when necessary.
To be built within BW
Custom BW Extractors SAP Data Services requires a perpetual license.
Extractors
A5 ECC 6.0 EHP7/8 or earlier with BAPI thru RFC Consider timestamp SAP Data Any Custom BW Extractors and BAPI will require you to develop the
Standard BAPI
field Services code, maintain and modify/upgrade when necessary.
Consider timestamp
Custom BAPI
field

26
Continuation
Architecture Extraction Middleware
Number Delta Handling Pros and Cons
Pattern Method Services
ECC 6.0 EHP8 or prior, HTTP-XML, no SAP Gateway
Maintenance of IDOC requires SAP knowledge. If
your system is S/4HANA, we recommend use of
OData which provides better options and limits
upgrade impact for further enhancements.
Any Versions Standard IDOC is able to process near real-time push for delta
Delta is handled within IDOCs API changes as well as batches.
of ECC or IDOCs
A6 Gateway/AWS
S/4HANA with AWS Lambda functions requires you to develop
Lambda
IDOCs code, maintain and upgrade when necessary.
Custom IDOCs require you to develop code,
maintain and upgrade when necessary.
Custom
Delta is handled within IDOCs
IDOCs
ECC 6.0 EHP8 or prior, JDBC, no SAP Gateway
Data structures at the Database level have limited or
no application context. SAP application knowledge is
required to perform transformation on extracted data
at the target.
Any Versions The SAP ECC or S/4HANA DB license is a runtime
of ECC or AWS license. This limits direct access to the database.
A7 Database Consider timestamp field
S/4HANA with Glue/Lambda This mechanism may require additional database
Database enterprise licenses.
Major changes to the database schema should be
expected when upgrades occur for ECC to S/4HANA
systems.

ECC 6.0 EHP8 or prior, Files, no SAP Gateway

This method can be used in may versions like SAP
ERP 6.0, S/4HANA but it requires custom
ECC 6.0 development and maintenance effort. It may also be
EHP7/8 or costly to upgrade.
AWS
A8 earlier with Flat Files Consider timestamp field
Glue/Lambda If your system is S/4HANA or has been upgraded to
BAPI thru
S/4HANA OData based extraction is recommended
Files
instead.

27