How To Implement Snaplogic
How To Implement Snaplogic
How to Implement
SnapLogic
Contents
Get Started With SnapLogic 4
Setting Up SnapLogic 5
Team size 5
User Controls 5
Plex Setup 5
Nodes Setup 6
Nodes Sizing 7
Password Management 8
Metadata 8
Sensitive Data 8
Prioritizing Integrations 11
How to Proceed 11
Considerations 12
Human Resources 14
Customer Support 15
Pipeline Management 16
Tasks 16
Administration 17
Additional Resources 17
REST API
SaaS Data
Host
SOAP
Metadata
Cloud Apps and Data
CONTROL PLANE
Centrally Monitor
and Manage
Metadata
Firewall
Private Data
Host
y A Cloudplex is a SnapLogic-managed plex where we run You can start by setting up 1 Snaplex (Cloudplex or
computing resources in the cloud to process your data. Groundplex) in your environment.
Cloudplex is the easiest and the most ideal solution if
you want SnapLogic to manage everything
Nodes Setup
y A Groundplex is a plex that runs in your own managed
A Node is a dedicated processing engine that is governed
domain, on your hardware resources, whether it’s
by a Groundplex or Cloudplex. The Node can be installed
on-premises or in the cloud. Even though it includes
on a physical or virtual machine running a Windows or
‘ground’ in the name Groundplex, a Groundplex can
Linux based operating system. We recommend setting up
be in the cloud. The distinction is that it runs behind
a minimum of two nodes in the beginning so that if there
your firewall on hardware that you manage. In some
are hardware or software failures on a node, you will have a
cases, organizations prefer running their pipelines
backup node to keep the job running. Additionally, we also
only in Groundplexes, regardless of whether they are
recommend customers to calculate the number of nodes
on-premises or cloud applications, to adhere to their
with ‘ N+1’ where the N is the minimum of nodes needed
organization’s security and compliance requirements.
to run workloads, and the extra node can be used for load
For example, you can configure enhanced account
balancing during rolling restarts.
encryption on Groundplexes and have full ownership of
the key for encryption Be aware that you will need to distinguish production and
You will need to be a SnapLogic Admin to set up Snaplexes. non-production nodes. Depending on how DevOps teams
operate, Some DevOps teams run nodes 24/7, while other
Go to the SnapLogic Manager to set Go to the SnapLogic DevOps teams shut down non-production Groundplex
Manager to set up a new Snaplex. Click the Snaplex tab in nodes in the evenings and weekends to reduce their own
the page that displays the assets for that Project. Then infrastructure costs.
the ‘Create Snaplex’ dialog that appears contains the
following tabs: Identify the type of integration jobs you will do in order to
set up the appropriate number of nodes. We recommend
y Settings that you run batch or streaming jobs or handle sensitive
y Logging and non-sensitive data in
separate nodes.
y Node Properties
y Node Proxies (Enabled only for Groundplex nodes) Node setup considerations:
Depending on your company’s architecture infrastructure y Identify batch, event, and streaming data
strategy (Cloud only, on-premises only or hybrid), you
y Sensitive vs. non-sensitive data
should take in consideration which Snaplex type to use. We
recommend using Cloudplexes for ease of management. y Long-running jobs vs. short-running jobs
We have also seen customers who deploy Cloudplexes y Environments: Development, Staging, Production
who want a more SaaS-like experience, may have a small
y Teams or departments that need separate
DevOps team or plan to have a NoOps IT environment.
execution nodes
y Ultra pipelines should have their dedicated recommend starting with medium sized nodes (2 vCPU 8GB
feedmaster and execution nodes as they are always memory, 40GB minimum storage, 100GB recommended).
running. Ultra pipelines provide the speed and The node limits will be calculated with your SnapLogic
scalability to run the most important integrations Sales Engineer and the node sizing will be indicated on the
that require high availability, high throughput, and contract. You can, however, increase the size of your nodes
persistent execution as needed. Work with your Sales Engineer to calculate the
node size.
Node Sizing
The chart below shows examples of SnapLogic implementation depending on the size and scope of your integration needs.
Nodes: 2 (1 dev/test, 1 prod) Nodes: 8 (4 dev/test, 4 prod) Nodes: 24 (16 dev/test, 8 prod)
Some customers deploy additional Snaplex sandboxes to test SnapLogic’s quarterly releases before they make the
updates in their production environment. This is to ensure mission-critical jobs are not disrupted and the customer is
ready to update their production environment.
The SnapLogic Metadata (definitions of pipelines, tasks, implement enhanced security, users may use the Enhanced
execution runtimes, etc., and not customer data) is stored Account Encryption feature. With this feature, the user
in the SnapLogic Control plane (currently running on the designates his or her own key-pair for use within their
Amazon EC2 infrastructure). The metadata is secured Groundplex deployment, and does not share the private
inside the protected SnapLogic environment, and only key with SnapLogic. The data is encrypted with a public key
accessed by the SnapLogic Control Plane Services – no before it leaves the SnapLogic domain, then is decrypted
access is permitted by any outside service. with a private key on the Groundplex. In this scenario,
account information is only usable in the Groundplexes
where the private key is available. These Groundplexes can
Sensitive Data
run in customer domain - either in the cloud or in their own
SnapLogic Control Plane encrypts sensitive data such as data centers. Enhanced Account Encryption is not available
account passwords, secret keys, and other sensitive fields for Cloudplex deployments.
depends on the account type. Such fields are encrypted
by default with keys generated during the execution node Below are useful links to help you set up your SnapLogic
deployment and SnapLogic supports replacing environment:
these keys with customer-specified encryption keys
Requirements for Groundplex
if desired.
Configuration Options
Networking Setup
HR_Analytics
Sales_Corporate
DataSync<Application name> b. Child Pipeline: Child pipeline name can represent the
DataSync_CustomerAccounts specific business functionality. You can also prepend child
pipelines with the same characters, such as sub_ or z_ as
Pipeline Naming Convention shown below.
a. Parent Pipeline: Every parent pipeline can represent the the Snap notes.
SnapLogic accounts are usually the login credentials Triggered task for getCustomerOrder pipeline -
to the specific data provider or an application. These getCustomerOrder_Triggered_Task
accounts are specific to the SnapLogic environment,
and required to be created in each environment (Dev, Real-time (Ultra) task for getCustomerOrder pipeline -
Stage, QA and Prod). We recommend creating an getCustomerOrder_RealTime_Task
account per SnapLogic Project.
For larger, distributed companies, we recommend
Note: Use the “Accounts” tab within your project limiting task creation to only SnapLogic Admins.
directory. An account can be created in the “Accounts”
tab within the specific project.
Expence
Database Application Database CRM VOC
Management
Quality Customer
3PL Manufacturing ERP Audit POS
Systems Shipments
Work with your business stakeholders to build out your desired architecture that will
help them automate business processes and data reporting. In some cases, you may
completely run in the cloud or run a hybrid environment, connecting cloud applications
with on-premises systems.
Considerations
y Consider migrating batch jobs instead of streaming, real-time data. Batch jobs are
predictable and scheduled so you know when they are being processed. Then move on
to processing streaming, real-time data as these are more complex, unpredictable,
and they may touch many business processes that cannot be interrupted
y Consider building or migrating integrations that do not touch external data, such as
customer data
Dependencies
Assumptions
Description
Constraints
Sub Type
UC #
BR #
SR #
(BR = Features of (SR = Functions of
the solution, The the solution, The
solution shall .....) solution shall …..)
The solution
shall provide a
documented
understanding
of how the
coexistence tool
will work relative
to our existing
_____ applications
after moving to
We have pre-built Enterprise Automation pipelines that help you speed up integrations. Below are the pre-built pipelines
based on business process and department. You will see more information about each business process and how to
configure the pipeline on the links provided below. You can also find more pre-built pipelines on the SnapLogic Community.
Human Resources
Create Opportunities in Microsoft Dynamics CSM and Create an Invoice in Workday if Sale Occurs Pipeline
Customer Support
Scaling Integrations and y Gain initial set of integration successes achieved by the
central team
Beyond y Outline the process of how users can get access to
SnapLogic
We have seen many companies begin their SnapLogic
journey with a centralized team building and managing y Create a Wiki for new users to get Started with
SnapLogic integrations. Over time, they decentralize SnapLogic. This may include materials from SnapLogic,
integrations and offer SnapLogic as a self-service model such as access to the SnapLogic University (online
to enable their business counterparts to build their own learning), How-to videos and exercises, SnapLogic
integrations and scale. documentation, SnapLogic Community, company-
specific guidelines on how to use SnapLogic, Do’s and
Larger companies have created a self-service program with Don’ts, and more
different levels of self-service and support depending on
y Identify Admins or SnapLogic experts users can contact
how technical the integrator is. In many cases, there are
for additional support
departments who will prefer integrations built for them,
while other departments prefer more autonomy to build y Identify type of integrations that users can build and
their own integrations. manage on their own and mission-critical integrations
that SnapLogic Admins need to be involved in
As you start building out a self-service program, consider
the following checklist:
If the pipeline exposes an unconnected output view, store the Account in that project.
SnapLogic provides the #1 intelligent integration platform. The company’s AI-powered workflows and self-service
integration capabilities make it fast and easy for organizations to manage all their application integration, data
integration, and data engineering projects on a single, scalable platform. Hundreds of Global 2000 customers —
including Adobe, AstraZeneca, Box, Emirates, GameStop, and Wendy’s — rely on SnapLogic to automate business
processes, accelerate analytics, and drive digital transformation. Learn more at snaplogic.com.