Data Provisioning
Topic
___________________________________________________________________________
SAP HANA Administration: Topic -- Data Provisioning in SAP HANA
DATA PROVISIONING IN SAP HANA
Data can be loaded from any source system to target system through
many ways. This process of data loading is generally called Data
Provisioning.
In SAP HANA we have many data provisioning options.
Importing data from multiple SAP and Non-SAP systems into SAP
HANA.
Data Provisioning options
SAP INBUILT TOOL EXTERNAL TOOLS
Source Systems
Flat file upload SLT (SAP Landscape
Transmission)
Smart Data Streaming SRS (SAP Replication
server)
Smart Data Access DXC (Direct Extractor
connection)
BODS (SAP Data
services)
SAP HANA Administration: Topic – Data Provisioning in SAP HANA
SAP ECC SAP BW HADOOP
SOCIAL SENSORS SAP(ORACLE
NETWORK , DB2
NO-
1)Flat file upload
We have an option to load data from either .CSV or Excel files to
HANA database directly without using any of the ETL tools.
Loading Data from Flat files (Five steps)
1. Create a table.
2. Get the file containing data in local file.
3. Import using HANA studio.
4. Manage the mapping.
5. load the data.
___________________________________________________
2) Smart Data Streaming:
SAP HANA Administration: Topic – Data Provisioning in SAP HANA
SAP Smart data streaming is HANA optional component available from SP09
on words.
High speed, highly scalable real- time event streaming process.
That provides the ability to process incoming information as fast as it
arrives.
capture useful information in the HANA database and monitor the
incoming data to enable you to respond immediately, as things
happen.
Streaming Data Sources-------- Few examples
Smart Devices – Sensor Reading.
Scanner, RFID Readers.
Click Stream from web server.
Prices – trades and quotes from the financial markets
Social Media : tweets, posts, check-ins
IT System logging events.
3) Smart Data Access
SAP HANA smart data access enables remote data to be accessed via SQL
queries as if they are local tables in HANA, without copying the data into SAP
HANA.
SAP HANA Administration: Topic – Data Provisioning in SAP HANA
Not only does this capability provides operational and cost benefits, but
most importantly it supports the development and the development of the
next generation of analytical applications which are required the ability to
access and integrate data from multiple systems in real time regardless of
where the data is located or what systems are generating it.
COLUMN Remote Object
VIRTUAL
TABLE
TABLE (Table/Views)
ROW TABLE
DB Catalog Remote Object catalog
Remote Data source:Eg..
Netezza,DB2, MS Sql, Oracle, SAP
HANA, Teradata, ASE/IQ.
Hadoop/spark, ESP.
It is based on local virtual tables that map to an existing object at the
remote data source site. Data required from other sources will remain
in virtual tables. Virtual tables under will point to remote tables in a
SAP HANA Administration: Topic – Data Provisioning in SAP HANA
different data source. It will enable real time access to data regardless
of its location and at the same time, it will not affect a SAP HANA
database.
Customers can then write SQL queries in
SAP HANA, which could operate on virtual tables. The HANA query
process optimizes these queries and executes the relevant part of the
query in the target database returns the result of the quarry to HANA
and complicates the operation.
Restrictions on usage of Smart data access
Data in virtual tables cannot be modified ie insert, update and
delete operations are not possible on virtual tables, How ever
we can do select on virtual table and put data in HANA table.
Virtual Tables does not support Data types like
SAP HANA Administration: Topic – Data Provisioning in SAP HANA
BLOB (Binary large objects) : Are you to store a large amount of
binary data/
CLOB (Character large objects)
Maximum size of an LOB use 2GB. Used to store large amounts
of ASCII character data.
Virtual tables cannot be used in multi node HANA cluster
however it is possible to access virtual table from one HANA
server to other.
Some analytical view does not support virtual tables
Supported Remote Sources by SAP HANA smart data Access
Oracle 12c, MS SQL Server, TERADATA, IBM DB2 LUW
SAP Sybase Adaptive severer enterprise SAP Sybase ESP
APACHE Hive SAP HANA(BW ON HANA, Suite on HANA)
SAP MII Netezza MaxDB Spark SQL
SAP HANA Administration: Topic – Data Provisioning in SAP HANA
SLT – SAP LANSCAPE TRANSFORMATION
One of the main features of HANA is that it can provide real time data to
the customer at any point of time. This is made possible with the help of
SLT .Where real time data is loaded to HANA from SAP or Non-SAP source
systems.
Uses trigger based technology to transfer the data from any source to SAP
HANA in real time.
Benefit of SLT
Allows real time or schedule time data replication.
During Replicating data in real time, we can migrate data in SAP HANA
format.
SLT handles cluster and pool tables.
This is fully integrated with SAP HANA studio.
SLT has table setting and transformation capabilities.
SLT have monitoring capabilities with SAP HANA solution manager.
SAP HANA Administration: Topic – Data Provisioning in SAP HANA
Architecture of SLT
The architecture of a SLT differs slightly based on the type of source system
1) SAP source system to HANA
If the source system is SAP then you can install SLT as a separate
system or in source system itself.
Real time replication works based on a DB trigger.
The connection type between source and SLT is RFC connection.
The connection between SLT and Hana is DB connection.
The below figure shows SLT as a separate system where source
system is SAP.
APPLICATION
TABLE
SAP HANA Administration: Topic – Data Provisioning in SAP HANA
READ MODULE STRUCTURE
MAPPING &
TRANSFORMATION
APPLICATION LOGGING
TABLE TABLE
WRITE
MODULE DB CONNECTION
DB Trigger RFC
SAP LT REPLICATION
SAP SOURCE SYSTEM SERVER SAP HANA SYSTEM
The below figure shows SLT inside SAP Source System
SAP SOURCE SYSTEM
STRUCTURE MAPPING &
SAP HANA SYSTEM
READ MODULE
TRANSFORMATION
APPLICATION TABLE
APP LOGGIN WRITE MODULE
TABLES G DATA
If you install
DB CONNECTION
DB TRIGGER SLT in
Source
system itself, then we no more need to have an RFC Connection.
SAP HANA Administration: Topic – Data Provisioning in SAP HANA
2) NON-SAP Source System to SAP HANA
APPLICATION LOGGING READ MODULE
TABLE TABLE
APPLICATION
STRUCTURE MAPPING & TABLE
Transformation
DB TRIGGER
WRITE MODULE DB
CONNECTION
DB CONNECTION
NON-SAP SOURCE SYS SAP LT REPLICATION SRV SAP HANA SYS
The above figure shows real time replication of data from NON- SAP
source to HANA system. When source is NON- SAP we have to install
SLT as a separate system.
The main changes when compared to first scenario where source is
SAP system are:
connection between source and SLT is going to be a DB
connection.
Read module will be in SLT instead of source.
SAP HANA Administration: Topic – Data Provisioning in SAP HANA
COMPONENTS OF SLT
1) Logging Tables
2) Read Modules
3) Control Modules
4) Write Modules
5)
Logging Tables:
Logging tables are used to capture the changed/new records from
application tables since last successful replication to Hana.
Read modules:
Read modules are used to read the data from application tables for
initial loading and convert the cluster type tables into transparent.
Control Module:
The control module is used to perform small transformations on the
source data. Data from here will be moved to write tables.
Write Tables:
The functionality of write table is to write the data to HANA system
In order to replicate data from source system to Hanna two major
steps are involved
1. create schema in SLT using T code LTR.
2. Replicate the tables of source system to HANA studio.
SAP HANA Administration: Topic – Data Provisioning in SAP HANA
PREPARATION:
Preparation steps for source system
SAP LT replication server is shipped in an add on called DMIS_*. You
must install the add on in your ABAP source system and in the SAP LT
replication server system.
SAP VERSION ABAP SOURCE SYSTEM SAP LT
REPLICATION
SERVER
NW/WAS- 620,640,700,701 DMIS _ 2011 N/A
702 to 740 DMIS _2011 BMIS _ 2011
Preparation steps for SAP LT Replication Server system
The system running the SAP LT replication server must be an ABAP system with
at least NetWeaver 7.02 (basis support pack 8) with an ABAP stack using SAP
kernel 7.20 (64-bit Unicode) patch level 110 or higher.
1) Download DIMS From SMP.
2) Apply DMIS use the Tcode : SAINT
SAP HANA Administration: Topic – Data Provisioning in SAP HANA
Post Installation Activities
Activation of web delpro and relevant services
After the installation of the SAP LT replication server all required web Dynpro
SAP LT replication server services are initially disabled. You must enable these
services in order to run the SAP LT replication server user interface.
Activate web din pro services as described
Run transaction SICF
In the hierarchy type field enter service and choose run
Expand the node default_post and navigate to SAP --- bc--
webdynprosap
Activate following services
1) IUUC_*
2) /SAP/PUBLIC/BC*
3) /SAP/BC/*
User creation and connection for a web source system
in order to replicate data using the SAP-LT Replication server
you must create an RFC connection to the source system.
The user is required in the SAP HANA in memory database so that replication
data can be stored.
SAP HANA Administration: Topic – Data Provisioning in SAP HANA
following Role is required.
SAP_IUUC_REPL_REMOTE
If you use a new client after the DMIS add-on is applied.You must Transport the
necessary roles from client 000 into your target client.
Creating RFC connections
Either dialog or system user in the source system
SAP_IUUC_REPL_REMOTE
Note:
do not use user DDIC. The role SAP_IUUC_REPL_REMOTE is not Generated by
default. You must generate and assign this role to the user that you want to use
for the RFC connection.
Create an RFC connection type 3 ABAP from the SAP-LT replication server to the
source system(if both systems are Unicode, Specify the RFG collection as
Unicode)
user creation for SAP-LT replication server system
SAP-LT replication server is delivered with the role SAP_IUUC_REPL_ADMIN and
SAP_IUUC_REPL_DISPLAY. To activate the roles use tcode PFCG.
SAP HANA Administration: Topic – Data Provisioning in SAP HANA
DXC ( Direct Extractor connection)
Is used to redirect data from embedded SAP system(SAP
ECC extractors) to HANA tables using HTTP connection.
DXC is a batch driven process and data Extraction using
DXC at certain interval is enough in many cases you can
set an interval when batch jobs executes example: every
20 minutes and in most cases it is sufficient to extract
data using batch jobs at a certain time intervals.
NOTE: BW is automatically included in SAP ERP system
since SAP NW 7.0 SAP BW technology that exists inside
such an ERP system is called “Embedded BW.”
DXC utilizes the embedded BW to enable extraction and
monitoring. The date flow is redirected to HANA from
embedded BW.
when a data source is activated inside ambidded BW
(data warehousing workbench) a corresponding in
memory data source object(DSO) is generated in the
SAP HANA Administration: Topic – Data Provisioning in SAP HANA
HANA. The memory DSO is utilized as a foundation table
for further HANA modeling, example analytic view,
calculation view etc.
Advantages of DXC
This method requires no additional server or
applicatation in SAP HANA system landscape.
DXC method reduces complexity of data modeling in SAP
HANA as data sends to HANA after applying all business
extractor logics in source system.
It speeds up the time lines for SAP HANA
implementation project
It reuses existing proprietary extraction, transformation
and loading mechanism built in SAP business suite
system Over a simple HTTP(s) connection to SAP HANA.
SAP HANA Administration: Topic – Data Provisioning in SAP HANA
limitations
Data source must have a predefined mechanism for
extraction, transformation and load and if not we need
to define them.
It requires business suite based on NW 7.0 or higher.
SAP DATA SERVICES:
SAP dataservice product has gone many pages and
sometimes referred with some different names like
Cassandra business object data services oh just some
other name.
SAP Data service (SDS) was not directly by SAP. It went
through multiple evaluations and acquisitions.
Acta Technology:
Data Management (DM)
Data quality tool (DQ)
SAP HANA Administration: Topic – Data Provisioning in SAP HANA
French company Acquired ACTA Technology in 2003
Renamed above products as
Business object Data Integration (BODI)
Business Object Data Quality tool (BODQ)
SAP Acquired in the year 2007.Renamed it as
BODI and BODQ
In 2008 SAP merged two softwares and called
BODS(Business Object Data Services).Which provides
Data Integration and Data Management Solutions.
From 4.2 version onwards They removed BODS Name
and set new name SDS(SAP Data Services)
SAP Data services is a certified ETL (Extraction,
Transformation and loading) tool from SAP to perform
batch loading into SAP HANA.
SAP data Services is a data
integration and transmission software application.It
SAP HANA Administration: Topic – Data Provisioning in SAP HANA
allows users to develop and execute workflows that take
data from multiple sources and then allows the user to
combine, transform and refine that data, and then send
to to destination systems.
BUSINESS SCENARIO
Suppose you are working in an organization where data is stored in
various disparate databases like oracle,DB2, and other legacy systems.
you are asked to recommend the best application for consolidating and
replicating data into SAP HANA from SAP and NON-SAP sources using the
ETL method.
SAP ERP
SAP BW SAP HANA
NON-SAP
SOURCE
SAP HANA Administration: Topic – Data Provisioning in SAP HANA
Extract data from homogeneous or heterogeneous data sources.
Transform the data for storing it in proper format or structure for
querying analysis purpose.
Loads it into the final target (database or data warehouse)
SAP HANA Administration: Topic – Data Provisioning in SAP HANA
