IBP Replicator 5.0: User Manual
IBP Replicator 5.0: User Manual
0
© 2016 IBPhoenix Editors
User Manual
Build 5.0.04
28/05/2016
Build 5.004 28/05/2016 © 2016 IBPhoenix Editors
Replicator is software that integrates with your databases, allowing users to replicate
data seamlessly and transparently between databases on both local and remote sites.
This manual provides an overview of IBP Replicator and how IBP Replicator and its tools
can be used to meet your replication needs. It provides a step by step breakdown on how
to install, configure and use the Replicator software. Included are descriptions of the
various replication models you can implement and how to use the Replication Manager
tool to set up your replication schemas and to operate and schedule replications.
Chapter 1 describes the platform, database engine, system and licensing requirements.
Chapter 3 describes the architecture and terminology of IBP Replicator and Chapter 4
goes on to explain how the replication actually works
In Chapter 5, you will find a discussion of the planning you will need to consider in
designing your custom Replicator system.
Chapter 6 contains instructions for installing the pieces of your replication system on
source and target servers.
In Chapter 7, you will find the details for defining a replication schema--the rules.
In Chapter 9 you will find a miscellany of topics concerning issues that will arise in more
complex setups and how to deal with them.
The Version Notes at the end of this volume contain details of enhancements in Version
5.0.
Installation and configuration of the Avalerion server are not discussed in this manual. Please
consult the documentation accompanying the Avalerion server software.
Contents 3
Table of Contents
Chapter 1 - IBP Replicator Requirements 6
Chapter 2 - Overview 9
Chapter 3 - IBP Replicator Architecture 12
Terminology ................................................................................................................................... 13
Appendices 142
...................................................................................................................................
Appendix I: Supported Data Types and Compatibilites 142
...................................................................................................................................
Appendix II: Command-Line Editor: cfged 146
...................................................................................................................................
Appendix III: Version Notes 172
Index 174
· IBPhoenix Replicator Server runs on Windows and Linux platforms, with both 32-bit
and 64-bit versions available.
· The current version of the Replication Manager application needs to run on a Windows
host, which may be a Wine environment on a Linux server. It can access a remote
Replicator server on Linux.
Database Engine
· Firebird (any released version, including the special Avalerion build with plug-in
capability)
Replication from any of these sources can be targeted to an ODBC system DSN
connecting to other database management systems, if the ODBC driver support is
reasonably well implemented.
The client libraries must be named according to their "native" conventions. On Windows,
this means fbclient.dll for Firebird, gds32.dll for InterBase and oci.dll for Oracle; on
Linux it means libfbclient.so for Firebird, libgds.so for InterBase and libclntsh.so for
Oracle versions that can run on Linux.
Disk Space
To install the IBPhoenix Replicator software you need at least 5.25MB of available disk
space.
Database Installation
Both the Replication Server and the Manager require at least the appropriate client library
for the database to be installed. Installation and configuration will involve creating a
configuration database, so these activities must take place on a running instance of the
database server.
Licensing
There are two types of licence: Server and Replicant. Licences must be installed into the
configuration database that you want used for your active replication.
Existing Licences
Existing IBPhoenix Replicator V.3 licences are valid for V.4 and will be preserved.
Licences from versions prior to V.3 are not valid for Replicator 4 and will be removed
from the configuration altogether.
There are more notes about licences in Chapter 6, About Licences. Instructions for
installing them can be found in the topic about the License Manager tool.
Chapter 5 describes some possible configuration models, which may help to clarify the
licensing requirements.
Chapter 2 - Overview
Chapter 2 - Overview 9
Chapter 2 - Overview
IBPhoenix Replicator is a management system that is is added to servers hosting
Firebird, Interbase or Oracle databases. It keeps track of data changes in
databases designated as sources and enables users to synchronise two or more
databases at one or more sites, locally or remotely.
Replicator can also replicate to or from a file, to cater for conditions where two-tier
networks connections are too slow or unreliable to sustain "live" replication.
Replicator connects to database servers directly through the client APIs. No middleware
or third party driver is used, overhead is low and the system is small, robust and fast.
· any data type supported by Firebird or InterBase, including BLOBs and arrays
Replication to or from databases other than Firebird or InterBase, notably Oracle 9 and
higher and to ODBC targets, is supported.
Replicator Security
Security provisions use the same mechanisms as those used to secure your normal
databases. User names and passwords that Replicator uses to connect to source and
target databases are stored in the regular server security structures of the respective
server installations.
Benefits of Replicator
Replication can enhance the usability of your system in a number of ways. For example,
Data availability - Work groups can have their own copy of a replicated database,
enabling them to function independently within a larger network. A local copy of
a database means that the work group does not have to compete for larger
network resources or suffer response delays by accessing databases on remote
sites. Decision support applications can be separated from high-volume
transactional applications if necessary, which can reduce network traffic and
improve data availability and system performance.
Replicator Server
The IBPhoenix Replicator Server (ReplServer) is the program that performs replication
operations. To determine what schemas need to be replicated where, it uses a
configuration database. A single server can replicate multiple source databases
cyclically. On Windows server platforms it can be run as a service (recommended) or as an
application.
Replication Manager
The Replication Manager is a Windows graphical tool for the user to define, manage and
monitor replication schemas.
Configuration Database
IBP Replicator's configuration database is where the program saves and maintains the
details of what is to be replicated and where it should be replicated from (the source) and
to (the target). The specifications for a replication operation are referred to as a
replication schema.
You can have Replicator create multiple configuration databases to handle multiple
schemas, or you can store multiple schemas in one single configuration database.
IMPORTANT: For IBPhoenix Replicator to work properly, each base table that is to be
involved in replication must be uniquely identifiable. Ideally, each table involved should
have its own unique or primary key; However, unique identification in the absence of
keys is sufficient.
Refer to the Appendix Supported Data Types and their Compatibilities for
information about matching columns with non-identical data types.
The Create System Objects function generates potentially three kinds of triggers--insert,
update and/or delete triggers--to track changes that are made to the database tables.
Changes are recorded in the Log Table.
The Log Table stores the key information on all the changes that have been made to the
tables in the source database.
The Manual Conflict Resolution Log contains a list of operations from the Log Table that
were not successfully replicated because of a data conflict problem and now require
manual user intervention to correct the problem.
3.1 Terminology
Source (database)
When we talk about the source database, we refer to the database containing the "master
version" of data that is intended to be copied to one or more other databases.
Target (database)
A target database is one that that receives changes to the contents of its tables (inserts,
updates and deletions) from a source database, generally on a regular basis.
Replication
Replication is the process of passing newer versions of table data from a source to a
target database, thus "replaying" changes that have occurred in the source tables.
Replication of subsets (parts of records, filtered sets) is also possible. This process is
sometimes referred to as "log shipping".
Synchronization
If the target tables are "empty" before synchronization, the process is referred to as
symmetric synchronization.
Replicant
Registering a database
Asynchronous Replication
Synchronous Replication
database are committed. IBP Replicator does not support synchronous replication.
Data Subset(ting)
When only certain rows in a table are replicated to a target, it is called data subsetting. It
is entirely possible that replication from a single source table could replicate different
subsets to multiple targets.
· the objects in the source database that are to provide data for replication
· the objects in the target database that are to receive data from each particular
source object
System Objects
System Objects can be regarded as the metadata of a replication schema. They comprise
triggers and tables that ensure that, for each changed row in the source database, the
table identifier, primary key and the action (insert, update, delete) are logged.
The logic of the triggers handles new, updated or deleted data created on the local
(source) database and, if applicable, incoming data from other databases that are
replicating back to the local database. The log table forms a queue of committed work
waiting to be replicated.
As an insert, update or delete takes place on a specified table or set, a trigger fires to
write information to a log table when the transaction is committed. It is this underlying
mechanism that prevents information from being replicated before it is committed in the
source database: no commit, no log entry, no replication.
Executing Replication
The replication server can receive the instruction to replicate from these sources:
· in response to an event
Sequence of events
Whenever the server receives the instruction to replicate, it follows this sequence:
1. It looks at the data that has been read from the configuration database to determine
the source and target pairs of databases.
2. Next, it looks at the replication log table in the source database to find out which
changes in the source database need to be replicated to the target.
3. The server processes the rows by querying their data from the source database tables
and duplicating the logged action (insert, update, delete) to the target database,
according to the specifications defined by the replication schema.
Rows are processed in FIFO order, i.e., in exactly the same order that they were put
into the log by the triggers. The operation specified by the log record is applied to
rows and columns of the tables in the target database :
Ø An update is replicated by finding the row with the same primary key and
updating the replicated columns to their new values.
Ø A delete is replicated by finding the row with the same primary key and
deleting it.
Clearly, it is essential that both source and target table must have primary keys or, at
least, unique column sets that can be treated as keys. The corresponding keys or
key-like sets must be compatible.
Conflict Resolution
Central-to-Standby
This simple model fulfils a requirement to replicate one way from a source database to a
target. It is the scheme typically used to fulfil a "failover" requirement, whereby a
standby database must be available to resume activity if a source database should fail.
Peer to Peer
Cascade
Cascade replication takes data from a source database and moves it to a target that, in
turn, becomes a source for another target, and so on, down a hierarchy. An example of
cascading replication might be where Head Office replicates to a regional office which
then replicate to a branch. The cascade model might be combined with a
Central-to-Branch schema (see below) so that each level is replicating separate sets of
data to a number of levels below it.
Central-to-Branch
Hub-and-Spoke
Unique Keys
All tables must have unique column sets as keys. As a rule, the unique key used to map
a source row to a target row should be the primary key, although it can be any column
set that resolves as unique.
IMPORTANT: Firebird and Oracle allow null elements in uniquely constrained keys. Such
keys cannot be used for mapping source and target tables for replication.
Maintaining keys that are unique across database boundaries are always going to be the
major issue when you are setting out to merge records from different databases. In
scenarios where rows are being inserted in multiple databases using generated primary
keys, it will be necessary to implement some changes to avoid unresolvable key violation
conflicts.
· It is possible to use a uniquely constrained key that is not the primary key. This
might be necessary in some heterogeneous schemas.
· Another alternative is to replicate changes, from all of the targets that are going
to receive an update replication, back to the central database first, before
anything gets replicated back out to the targets. All inserts from the targets go
into the central database via an insert trigger which generates a new primary
key for them from the central database. Replication out to the targets just has
to be able to ascertain which of the "new inserts" originated from the target in
focus and treat them as updates.
The main need would be to tag records on both sides, so that both the central
and the target databases know that they need special treatment. How (or
whether) this could be done would depend on considerations such as the
impact if records that had been locally inserted by the target changed their
primary keys after replication, whether a target's original key needed to persist
for local usage, and so on.
DATE vs TIMESTAMP
Timefield conflict resolution does not fail if either or both of the compared fields are of
DATE type rather than TIMESTAMP. However, use of the date-only type is not
recommended because replication might not occur where the source and target rows
The choice is not an issue if the databases are all Dialect 1, since a date-only type is not
supported there.
Clock Synchronisation
Don't overlook the need to implement mechanisms to synchronise the system clocks of
the source and target computers and, where necessary, to adjust for time zone
differences.
Distribution Conditions
One-way Replication
In one-way replication, one database is always the source. In physical terms, its targets
receive data that they do not overwrite, append to or delete. Replication refreshes the
content of the targets without further modifying the source database.
Heterogenous Replication
Source and target databases do not always have the same structure. One may contain
tables that don't exist in the other. Tables may have different numbers of columns and
different names for columns. A source dataset may replicate different sets of data to
different targets. Any of these heterogeneous schemas is possible.
Issues like this generate conflicts during replication. Not all conflicts will be of
the same type and different types of conflicts can be handled in various ways.
Each source and target pair of databases can have its own conflict resolution
settings.
Handling Conflicts
Priority-based Resolution
Databases can be given priorities. The database with the higher priority number takes
precedence. For example, if a source database has precedence, conflicts are resolved as
follows:
Master-Slave
The source database always takes precedence: resolution will be as described in the
example above.
Timestamping
This method determines the newer version of data by comparing server timestamps
written into the data. It needs a suitable timestamp column defined in both source and
target tables. Older rows in the source database will not overwrite newer rows in the
target.
The configuration database structure in IBP Replicator 5 is not compatible with those of
releases before v.4.
The newer Replication Manager will automatically upgrade old configuration databases on
opening. Licences from Replicator versions older than v.4 are not valid for Replicator 5.
Existing version 3 licences will be preserved and marked invalid. Any licences older than
version 3 will be removed from the configuration altogether.
The installer program may be compressed in Winzip format and distributed as an archive
named IBPReplicatorSetup-n.n.n-WinXX.zip. Decompress the archive into a temporary
location of your choice, using your favourite decompression tool--WinZip, 7Zip, WinRAR,
etc.
Follow the dialogs as the installer runs, providing a custom root location for the software
if you don't want the defaults.
Once the installer has finished, close it. If instructed to do so, close down any
applications you have running and reboot the machine.
As a quick summary, here is what each of the shortcuts takes you to:
Starts the Replication Manager console program. See the next topic, First
Configuration Database.
ODBC
Replicator provides support for databases accessed through ODBC system DSN
connections. The software for creating and managing a system DSN is provided in your
operating system, either as a default application or (on non-Windows platforms) as an
installable option. An ODBC driver for the database management system that Replicator
will connect to is required. These components do not come with Replicator. They will
need to be installed and working before you begin configuring the replication schema.
MySQL: if it is likely the same record will be modified more than once in one
replication session, you must enable the DSN setting "Details ->
Cursors/Results -> Return matched rows instead of affected rows".
PostgreSQL: To replicate BLOBs to the PostgreSQL bytea type, the DSN setting
ByteaAsLongVarBinary must be enabled.
You do not need ODBC to work with or between Firebird, InterBase or Oracle databases.
The configuration database doesn't have to be on the same host server as the replication
service. The dialog starts by prompting you to select the server type (actual list depends
on DBMS clients you have installed in your system) that the database will run on. This
enables the programs to take advantage of features in Firebird 1.5 and higher that are
not available on InterBase, such as database aliasing:
The rest of the configuration dialog prompts you to define connection string for the
configuration database and a valid username and password. The Comment field is
optional: it may be used in the title bar of some displays.
For example, suppose you decide to store your configuration databases in a subdirectory
"databases" in some folder structure on your server. Your setup entries might be similar
to this:
The server connection string in the screenshot is an example only, using a "serverless"
protocol that is available locally with some models of Firebird and InterBase. Use the
connection string that is appropriate for the client server environment in which you are
working. If you are uncertain about it, check with your network administrator or advisor.
For Firebird, consult the Quick Start Guide applicable to your server version.
The User Name is that of a user on the server that is to be the owner of the new
database. The user ADMIN in the example is an ordinary user name on the host server.
Some sites might prefer that the SYSDBA user be the owner of replication configuration
databases.
When you are ready to create your configuration database, just click the "Create" button
and you're done.
You will get a dialog almost exactly like the one for creating the regular configuration
database. Fill in the fields to access the database, remembering to include a valid
username and password, and submit it.
You might like to include a useful nickname for the configuration in the Comment field,
e.g. "Synch Checkouts" might be appropriate for a replication that is to be used to keep
point-of-sale terminals in synch with a Sales database.
(For Oracle, the message may differ slightly but the same question is implicit.)
Click Yes to have the program create the replication metadata in the source database. If
you don't want to proceed, click No to abandon the idea.
NOTE Normally, you won't want the overhead of performing all replication instantly; it is
perfectly OK to break your replications out so that most of the replication runs
asynchronously and the source database configures just the essential
"close-to-real-time" replication.
The database will be added to the selection list beneath the File menu.
Once the configuration database has been created, you can register the databases that
are going to be involved in replication.
To start registering a database, stay on the Databases tab. Above the panes is a toolbar.
The Toolbar
The toolbar gives access to the commands available during database registration. They
provide shortcuts to the same commands that are available from the Databases menu.
The active buttons at any point appear in an "enabled" state, while unavailable buttons
are greyed out. Hover hints are available on all of the buttons.
New Schema launches the creation of the replication schema that you are
about to define
Registering a Database
Click the Add button in the toolbar:
A new database record is initiated in the fields editor, ready for adding the necessary
information for the replication server to find the database:
Because of variations between the different database drivers, the set of fields displayed
may not be exactly as shown in the screenshots.
Versions of IBPReplicator before v.4.1 did not behave correctly with the
Firebird driver if the source database was Dialect 1. The Firebird driver now
selects the correct dialect automatically. If you are upgrading Replicator
from v.4.0 or an earlier version on a Dialect 1 database, any workaround
that involved using the InterBase driver should be corrected to use the
Firebird driver, regardless of dialect.
Descriptive name (required): A "user-friendly" name (not the database file name)
that will be meaningful to anyone managing the replications. For
example, "Head Office Source" is more meaningful than "New database"! It
is not a formal identifier, so you may include spaces and punctuation as
you wish.
Priority: Used when priority-based conflict resolution is being set for the
replication. In this case, replication conflicts are resolved by ensuring that
data from the database with the higher priority number is preserved when
conflicts arise.
Key Size: Size of the fields in the operation log table that contain the old and new
key values of every replicated record. This setting is used, with other
system objects, during creation of the table and is ignored later. The value
must be big enough to allow the field to contain a string representation of
your longest primary key. However, setting it to an unnecessarily large
value will degrade database performance.
Server (required): This is the network path (where applicable) and the full file path
(or database alias, for Firebird) of the database. It follows the syntax used
by the Firebird, InterBase or Oracle client, respectively, to connect to
databases. It should accord correctly with the format required by the
network transport protocol and the file system of the host server that is
running the database engine.
For Oracle, this will be the URL, TNS or equivalent that will find the
database according to your server's configuration.
For Firebird and InterBase, suppose the database file name is "source.fdb"
and it is located on a Windows server named "dev" in a directory called
G:\IBPhoenix\IBP Replicator\data.
dev:G:\IBPhoenix\IBP Replicator\data\source.fdb
For Named Pipes (sometimes referred to as NetBEUI) protocol, the server entry
should be
\\dev\G:\IBPhoenix\IBP Replicator\data\source.fdb
For an older InterBase database, named "source.gdb" on Novell, with the G-drive
mapped as "vol5" it should be
dev@vol5:\IBPhoenix\IBP Replicator\data\source.gdb
For a Linux or UNIX host, our example path would be of the form
dev:/IBPhoenix/IBP Replicator/data/source.fdb
When focus is in the Server field, you can click a button to launch a Find File
dialog:
DSN Appears only if the selected server type is ODBC. A selector button will
appear when you move focus into the field. Use it to select the system
DSN you have prepared for this database.
Administrative user name (required for InterBase, optional for Firebird and Oracle
if using trusted authentication): The database server authentication
username that the Replication Manager should use when connecting to
and administering the database being registered. It can be a different user
name from the Replication username.
For a database used as a source for any schema the user must be one of
following:
a) SYSDBA
c) A user that has been granted the RDB$ADMIN role. In this case, the
Administrative Role must be registered as RDB$ADMIN.
IMPORTANT
Failure to comply with this rule will prevent the Replication Manager and
cfged from creating the logging triggers on replicated tables.
If this is an ODBC database, use this field if the connection requires it;
otherwise leave blank.
Administrative password (required for InterBase, optional for Firebird and Oracle
if using trusted authentication): The password for the administrative user
name. The username and password belong together, as the two parts of a
user login defined in the security database of Firebird or InterBase. See
also Administrative Role (below).
If this is an ODBC database, use this field if the connection requires it;
otherwise leave blank.
When you start to make an entry in one of these fields, a dialog will pop up for
completion of both entries.
Character set (required if the default character set of the database is other than
NONE or, for Oracle, is not the same as the one in the environment
variable NLS_LANG.) It should be a character set that is compatible with
the DEFAULT CHARACTER SET for the database.
Time field: Used when time-stamped conflict resolution is being set for the
replication. If you require this form of conflict resolution, your tables
should all include a column with exactly the same name for each, that
contains the date and time when each row was inserted or last updated. If
this functionality has been implemented, IBP Replicator will ensure that the
more recent row is preserved when conflicts arise. Refer to the design
notes about Timefields for discussion about choosing the right date/time
type for this style of resolution.
Administrative role: Similarly to the administrative user name and password, this
is the SQL role that is used by the Replication Manager tool when
administering the database being registered. If the Administrative User
parameter is a regular user that has been granted the RDB$ADMIN role,
then only RDB$ADMIN is valid here.
The replication server uses another "replication role" that is provided when
databases are specified in a replication schema.
IBP Replicator can use the Windows Remote Access Services (RAS) to replicate across a
dial-up connection. The following database parameters control how it works:
RAS name This is the unique name of the dial-up connection as defined in
the Windows "Dial-up Networking" dialog
RAS The dial-up username that the replication should use when
username dialling out
RAS The password that the replication should use when dialling out
password
If you select Files.. as the Database type, the fields that appear are fewer:
These fields refer only to off-line replication. For more information, see the
advanced topic Offline Replication and Synchronization in Chapter 9.
Inbound folder: the full path to the folder where replicated data in off-line files
will be received. If you are using Avalerion, make this folder the same one
that is configured in ibpr_cdc.conf as OutputDirectory.
Outbound folder: the full path to the folder where replicated data in files will be
written for subsequent transfer by external means.
After apply: in this and higher versions of Replicator, you can opt either to delete
the offline files after they have been applied or to have Replicator preserve
and rename them automatically with an extension of ".done".
Use metadata from: You first need to have registered a database whose table
definitions will be used to determine the format of data to be replicated
off-line. When you focus in this field, the descriptive names of registered
databases will be available to select from.
If the connection attempt fails, check and correct the parameters until the connection
test succeeds.
Finally, to register the database, just click the OK button. The newly registered
database's descriptive name is added to the list.
Once you save the last of your databases, you are ready to move on and define a
replication schema.
This is the Toolbar button that launches the creation of the replication
schema. You can also right click on the legend in the right-hand pane
and select New from its context menu or use Replication>Schema>New in
the menu at the top of the screen.
Existing IBPhoenix Replicator V.3 licences are valid for V.4 and will be preserved.
Licences from versions prior to V.3 are not valid for Replicator 4 and will be removed
from the configuration altogether.
Licences Required
Two types of licence are needed for replication: Server and Replicant. Licences must be
installed into the configuration database that is to be used for your active replication.
Licenses are entered using the License Manager dialog, which you access from the Tools
Menu of Replication Manager.
As a rule of thumb, you require one Replicator licence to run a replication server and one
Replicant licence for each database that is a source or a target. Replicator server and
Replicant licences are bundled in denominations to suit your particular needs. One
Replicant licence is needed for each database that is involved, including the Source
database.
For example, the minimum IBP Replicator configuration of one source database
being replicated one-way to one target database requires a single Server licence
and two Replicant licences.
A Replication Server licence is bundled with one Replicant licence automatically. For this
version of Replicator, the included Replicant licence is issued separately to facilitate its
use with off-line replication.
Replicant licences are counted when the Server process checks that there are enough
Replicant licences available for all of the databases defined in the current replication
schema. Once a database is licensed for replication, it can be involved in multiple
replication schemas without the need to add more Replicant licences for it.
Licences are "server agnostic": you do not need different licences for Firebird, InterBase
or Oracle.
Licence Counting
Many configurations are possible. Checking is concerned only with ensuring that the
count of Replicant licences is at least equal to the number of databases configured in the
schema. Additional Replicant licences are available.
Evaluation Licences
You do not need to buy licences for testing IBPhoenix Replicator. Just start by using the
EVAL license for testing a simple, one-way replication schema. It will supply four
Replicant licences for your tests for 14 days.
Ø For more details about IBP Replicator licensing, see the topic in the IBP
Replicator Requirements section.
Summary of Steps
The steps involved are.-
5. Table settings
6. One-time synchronization
Important :: This task will write to the Windows Registry, which requires Replication
Manager to run with escalated privileges on Windows 7, Windows 2008 server and higher
Windows versions. To do this, right-click on the shortcut for Replication Manager and
select Run as administrator:
In Replication Manager you can pick up the configuration you want to work with by
clicking File on the menu bar and selecting one from the drop-down list:
Select the configuration database you want to be the default and click to open it.
Now, select Configuration from the menu and simply click on "Set as Default":
Next
Your next step will be to create the Schema for your replication.
However, you may wish to look over the default settings first and decide whether you
want to customise the options.
However, you can define multiple schemata for multiple replications that are to be cycled
through in order. Typically you would already have registered databases, defaults, etc.
already defined, and would wish to pick up definitions from some level of the "tree" to
The interface provides several different ways you can approach definition and
maintenance tasks at various levels:
You can mix the tricks in whatever way suits you best. Following are some examples of
the various approaches.
Here, the root node of the replication tree is selected and the user right-clicks on it to
display the context menu. The New... option is selected and a left click will provide the
input dialog for defining a new schema:
After you click OK, the new schema appears in the tree with nodes for the source and
target databases:
As another approach, you could have double-clicked on the "New schema" icon in
the task bar or you could have used the Replication > Schema > New in the Menu
bar.
TIP Use schema names that will be meaningful not just to you but to anyone else
who is going to manage your replications!
Note that you could have stayed within the schema definition dialog to develop the
schema definition further, as described below, clicking OK once you were ready to save
the work.
Notice how the icons in the Tasks pane change, according to where the focus in the tree
pane is. For example, if you single-click the newly created Schema node you will see a
new icon in the Tasks pane enabling you to add another target database to the schema.
As with other nodes, you also have similar contextual options by selecting and
right-clicking the schema's node directly in the the tree. In each approach, you will be
able to edit the name of the schema or delete it entirely:
Be aware that deleting a schema will remove all replication information for that
schema, including anything stored in lower nodes. Once the schema is gone, the
replication cannot occur!
Expanding the schema node reveals two new nodes labelled source database and target
database. These nodes can be approached in the succeeding steps.
Once you have more than one schema, the Shift Up and Shift Down buttons are enabled
in the toolbar when a schema node is selected. They provide the means to alter the order
in which your replications will take place in the cycle:
Click on the Shift Up button to move the selected schema up the order or Shift Down to
move it down:
You can change other attributes of registered databases, too. If you change the path of a
database, don't forget to test the connection before clicking OK to save the changes.
Source Database
If you want to edit some characteristics of the source database with respect to the
schema, or just check those characteristics, expand the schema node, right-click the
named node beneath the Source database node and select Edit...:
The Edit dialog that appears contains the characteristics, spread across three tabs:
Connection Tab
The first tab of the dialog shows the database that will be the source of your replication.
Registered database
Because there can be only one source database for a replication, the "friendly name" of
your source database is greyed out and not editable.
You can change the username, password and role (if applicable) to be used by the
replication server when it connects to the source database.
The name of the "replication user" does not have to be the SYSDBA or that of an
Administrator of the operating system.
For example, the REPL (replicator) user can be used for any replication. It must be used if
you plan to implement bi-directional or n-way replication.
The user named REPL is recognised by IBP Replicator as a special user. Any changes
made to the source database by the user REPL will NOT be replicated. This is by design,
to prevent replications from a target database back to a source from going into a loop.
Whatever user you assign, make sure that it is added to the relevant security databases
on each machine involved in the replication. Once you have done that, click the Test
Connection button. If login is unsuccessful, review your entries and make any necessary
corrections before trying again.
It is pointless proceeding now unless you can make a successful connection test.
Settings
The second tab enables you to customise settings for this source-target pairs in this
schema. They are initialised to the global default settings, but can be modified without
affecting either the defaults you have already set or any settings you might have
customised for another replication.
You can add any free-form comments you like in the Comments box, to augment or
override any comments inherited from the defaults.
Replication enabled
This check box should be checked ON unless you need to prevent replication from
occurring. For example, you would check it off if you were taking the network down or
powering one of the servers down for maintenance.
The replication is initalised to use the default conflict resolution strategy that you might
have customised previously¾refer back to Customising the Default Settings. The current
default setting appears above, greyed out. In the illustration it is Master-slave.
You can override the default conflict resolution strategy by checking off Use default
conflict resolution strategy and specifying one.
Don't select Time-stamped unless you have the necessary Timefield columns in all
of your source and target tables.
Separator character
IBP Replicator stores the primary keys in text format. The separator character is used to
separate the values in primary keys with multiple columns. Thus it is essential that this
be a character that cannot appear in a key value.
Comments
The comment field is entirely for your own convenience. Use it for any form of
documentation that is useful to you.
Event Logging
Use the default logging strategy, or specify a specific logging strategy for this source
target pair. A newly configured source database is initialised to use the server defaults:
Default logging causes the run-time replication log to be named replication.log and to be
written to default locations:
If you are defining multiple replication schemas, you might wish to have the server log
each one to its own log file. In that case, check off Use Default Logging and enter a path
that exists on the filesystem of the IBP Replicator host machine and, if needed, a different
log file name that can be identified easily by the system administrator:
The commands for creating a removing system objects are discussed in Creating Source
System Objects.
Each schema can only have one source database, so you need to define a schema for each
of your replications. A source database in one schema can be a target database in another
schema, and a single source can replicate to multiple targets.
To define a target database, right-click the Target databases node and select the Add
Target task.
Alternatively, you could drag the target database from the list of registered databases in
the left pane over to the Target databases node:
Connection Tab
Ø If necessary, drop down the list of registered databases and choose the one that
is to be the target database.
Ø Supply the username, password and, if used, the role to be used by the
replication server when it connects to the target database. The name of the
"replication user" does not have to be the SYSDBA or that of an Administrator of
the operating system. The default user provided in the dialog is REPL, which can
be used with a suitable role for any replication and must be used if you plan to
implement bi-directional or n-way replication.
Settings Tab
Periodic commit
Ordinarily IBP Replicator commits changes once per replication, after all the logged
changes have been successfully replicated to the target database. If a network error
occurs during replication, all of the changes will be rolled back and IBP Replicator will try
again.
In the field labelled Periodic commit after records set a positive value representing the
number of successfully replicated rows that IBP Replicator is to group and commit
together.
CAVEAT Don't implement this option unless it is actually necessary. It can impact the
general performance of the database over time and breaking a replication into numerous
separate transactions breaks the overall atomicity of a replication.
This is an optional setting that allows rows to be selected for replication on the basis of
some value in the data.
For example, if you have defined a multi-site database with conditional replication in
view, you might have a key on some tables that identifies the site when the data in a
record pertains to that site and not to other sites in the schema. This setting can define
this condition.
To implement this, any tables that you want to apply this global condition to should
include a column with an identifier that is the same for all such tables. Express the
condition like a predicate inside an IF() clause.
In the example illustrated the condition :SITE_ID = 100 will tell IBP Replicator to look out
for columns named "SITE_ID" in the replications it performs to this target and ignore any
rows that have any value other than 100 in this column.
Row-level conditioning can also be done at individual table level. For details and more
information about the row-level conditioning expressions to use, look here.
You can optionally override the default log file and define a specific log file and logging
levels for this particular target database:
Check off "Use default logging" and adjust the event logging settings to suit your needs.
When you are done, click OK to save the details of the target database node to the tree.
The descriptive name of your target database appears, along with two new nodes—
Replicated Tables and Procedures and Replicated Sequences:
Ø select and map the tables and procedures for the replication
Ø tell the Replication Manager to create the system objects in the source database
Cloning Targets
A schema can support one or multiple target databases for a source database, and
different data can be replicated to different targets¾see Choosing replicated tables
(next). A separate definition of the tables, primary keys, columns and rows to be
replicated is required for each target database involved. That makes it possible to have
very fine-grained control for any schema.
Cloning a target can be done several ways, much as you do with other tasks already
discussed. You can
Ø drag the specific target database node that you want to clone and drop it onto
the general target database node or another schema node
Ø select the specific target database, right click and choose Clone from the
context menu
Once the new target is in the tree, supply the name by selecting from the list of
databases that has been registered. Click OK and a copy of the schema defined for the
original target database will be created for the cloned target database. Any differences
required can be tidied up by editing the cloned definitions.
TIP Perform any cloning once you have defined most or all of the mappings for your
"base" target.
It is probably stating the obvious to remind you that you do need to start out with both a
source database and a target database that already have at least the relation objects you
want to replicate defined in their database schemata!
Don't forget the two essential metadata attributes needed in tables you are going to be
replicating from and to: they must have compatible sets of unique columns and, if you
intend to use timestamping for conflict resolution, both the source and target tables must
have compatible TIMEFIELD columns that can be referred to for an age reference.
Expand the node for a specific target database and select the Replicated tables and
procedures node.
You can use this selection to access the mappings pane at any time, once you
have table objects defined in your schema.
Filtering
If the lists of objects are too big to manage conveniently, you can set a filter on the lists
in the two panes. Click the Filter button to display the filter bar below the lists:
Type in letters to filter the list down to just names containing those letters. You can aim
at single letters or substrings—the filtering takes place as you type:
The excluded items have not been unmapped: they are simply invisible. If you use Ctrl-A
or the conventional Shift-click/Ctrl-click to select multiple items, only the visible ones
are selected.
The Tables order setting is persistent. Later, you may need to revert to User
defined if it is necessary to set a custom order for synchronizing table
objects. For more information about custom ordering for synchronization,
refer to the topic Customising the Order of Tables later in this chapter.
Stored procedures and views are listed along with tables in the Target pane.
When the target object is dropped onto the source object, the mapping is added and the
object disappears from the list of target objects:
Ø You can multi-select more than one table, view or procedure (using shift-click
or ctrl-click, as required) and drag them as whole
Ø Dragging from the source database to the target allows you to break an already
established mapping
Example:
REPL_PROC
(K1 INTEGER,
K2 VARCHAR2,
D1 DATE,
D2 CLOB,
OLD_K1 INTEGER,
OLD_K2 VARCHAR2,
OP CHAR,
RESULT OUT INTEGER)
Mapping in Bulk
The mapping of sequences (generators), tables, primary keys and fields can be done
automatically by
Ø selecting Tables | Autogenerate from the Replication menu or any context menu
that offers it
This choice causes all tables, keys and fields in the source database to be automatically
selected for mapping to corresponding objects of the same name in the target database.
Previous mappings will be respected.
You can multi-select a selective group of tables and autogenerate the mappings and
objects for them.
The Autogenerate dialog appears with all of your unmapped objects—tables, primary
keys, procedures, views and sequences (generators)—selected with best-guess source-
target mappings based on object names:
Use asterisk
Check Use asterisk if you would like to have Replication Manager also map all of the
source and target objects automatically at the same time, by mapping column names to
column names. Although this is a time-saver if your source and target databases have
the same structure, its main purpose is to enable mappings to be easily updated if the
table structure should change. In that event, editing of the schema would not be
required: it would be enough just to reload the configuration.
...and it is done! Now, the mappings appear in new nodes in the tree:
Check off any objects you do not want to autogenerate objects for, e.g. because you do
not want to replicate them.
Notice that icons now appear for adding, editing and removing objects when you select a
lower-level node in the tree:
This button takes you to a dialog where you can map or remap the columns
involved in primary keys or unique constraints. In the tree, a similar icon
marks the node where you can do that task.
This one takes you to the dialog for mapping the non-key fields manually.
Similar icons in the tree mark the nodes for each non-key column or for
the "asterisk" mapping (as in the screenshot) which does all of the column
mappings by matching column names in source and target.
The "best guess" mappings for the primary keys have occurred where primary key names
and the key fields involved are the same in both source and target or key mappings were
deduced from foreign key metadata. The heuristics of this "best-guessing" are not
infallible: you should verify the mappings.
Tables that do not yet have their primary keys mapped are marked with a red
exclamation point ( ! ).
In tables that have no primary keys, but a matching UNIQUE constraint exists in both
tables, that mapping will show instead. You can use the edit tools to tidy up individual
key and field mappings, as required.
Heterogeneous Mappings
It is entirely feasible to create heterogenous table mappings where source and target
tables have different table names and even if they have the same table names but
different column names. However both tables must have unique sets of same-named
columns that identify ("key") the rows to be replicated and the columns being mapped
must have compatible data types.
Refer to the Appendix Supported Data Types and their Compatibilities for information
about matching columns with non-identical data types.
The replication can replicate the values of sequences (generators) from source to target.
Be careful with this: replicating sequences in a multi-master scenario will corrupt targets
in a heartbeat. If your schema is replicating in such a scenario, you MUST check all
sequences (generators) OFF.
Because the ODBC interface is, by design, "blind" to the technical capabilities of the
datasource, replication of sequences is not available for ODBC targets.
Removing Mappings
Primary key, column and generator mappings can be similarly removed by selecting the
appropriate node and working with the tools activated at that level.
To access this dialog, select the table's node in the tree and either
This is a condition that would be inserted into the IF() predicate of a branching statement
in a trigger to limit the rows selected.
IF (<some_condition>) THEN
BEGIN
...
END
You would pick up the <some_condition> part of the predicate and enter that into the
row level replication condition as
Although this example is simple, there is no limit to the complexity of the condition: any
condition that would be valid in a normal IF(..) statement in PSQL can be used as a
replication condition.
Synchronization enabled
Keep statistics
Check this box to use the Monitor tool to review IBP Replicator's performance. By default,
it is checked OFF for all tables, to reduce performance overhead by allowing replserver to
work with read-only access to the tables in the configuration database.
NOTE :: Monitoring of the table is disabled while Keep Statistics is checked off.
By default, conflicts are resolved using the default method for the schema. If a
source and target pair of tables needs a different strategy for resolving conflicts,
it can be specified individually, here, by checking the required strategy instead of
the one selected from the existing default.
Default :: if you later want the strategy for this object to revert to that specified
for the schema, select this option.
If you are using the time-stamped conflict resolution strategy, both the source and target
tables need a timestamp column from which IBP Replicator is to read for comparing the
ages of the rows to resolve conflicts. It should be a TIMESTAMP type (DATE type in IB 5.x)
into which a regular Before Insert or Update trigger (separate triggers for InterBase)
writes the system timestamp [ CAST ('NOW' AS TIMESTAMP) ] whenever an insert or
update occurs.
IMPORTANT
The time field must have the same name (identifier) in both the source and target tables.
The field name is case-sensitive: it must be entered, without quotes, exactly as it is seen
by the server.
Thus, for example, if the identifier was defined with double quotes as
"TimeFieldForReplication" then type TimeFieldForReplication here. If it was defined
without the quotes, e.g., TimeFieldForReplication, then type TIMEFIELDFORREPLICATION
here.
The Time field name is where you name and map this column for the two tables.
Separator character
Use this if the separator character for the elements of composite primary keys in an
individual table need to be different from the global separator character.
If you want a foreign key violation to be treated differently for this relation than the
method specified in Global Settings, you can specify it here. Initially, the global setting is
selected by default. Choose an alternative behavior as follows:
Conflict :: treats the violation as a conflict. The conflict will be written to the
manual log table and replication continues.
Replicated operations
By default, all operations (insert, update and delete) will be replicated. You can choose to
exclude an operation by checking it off.
Comments
Use this box for any free-form documentation about the object that could be useful to
you and others in future.
Ø asymmetric, where the target table is synchronized from the source table.
The target table becomes an exact copy of the source table with IBP
Replicator performing all the necessary inserts and deletes. The source
"database" for an asymmetric synchronization can be a suite of files
previously off-loaded to a Files.. target database. This off-line
asymmetric synchronization is described in more detail in a later chapter.
Ø symmetric, which does not perform any deletes but, instead, inserts any
records missing on the target from the source and vice-versa. It also
performs updates according to the current conflict resolution strategy.
Synchronization is not replication. It is a one-time operation that can be run for a new or
modified schema.
You won't be able to perform this customisation if you have the interface
explicitly set up to order the objects alphabetically. Before you start, you will
need to use the View>Tables order option in the toolbar of the main display to
switch to "User defined" ordering.
Select the Replicated Tables and Procedures node of the target database and select a
table that you want to move. Then click either the Up or the Down button in the toolbar,
to shift it up or down one position:
In this example, we want to move MEMBER down, because it has dependencies on nearly
all of the other tables. F_TRANSACTION has to go to the very last position because it
depends on all of the other tables:
Simply continue to shift tables and procedures down and up until you are happy that you
have taken care of the dependency issues.
Select the database node of the target you want to synchronize with the source,
right-click and select the type of synchronization you want:
The dialog window shows all of the synchronizations checked for the mapped tables (and
Oracle procedures). If you want to synchronize sequences (generators) as well, check
them on manually. You can also check off any objects you don't want synchronized.
Options
Write detailed log to file :: Check this on if you want a log of the synchronization.
The field beneath shows the default location for logs on your system,
according to your global or schema-level setting. When logging is
selected, this field becomes writable and you can specify a custom
location for this log if you wish.
Add timestamps :: Select this if you want the screen and log messages to include
timestamps.
Show message every ... :: the process will return a progress message to the
window every 100,000 records. Change this if you want messages to be
more or less frequent. Numbers are in thousands.
A monitor window will pop up and display the synchronization as it happens. The
window will fill up very fast but you can scroll back to check what has been done:
Single-table Synchronization
If you just want to synchronize an individual table, select the table in the Replicated
Tables and Procedures node of the tree, right-click and select a synchronization option
form the context menu. The same dialog appears but, this time, with just that table
selected.
It is recommended that you perform Create system objects when you are the exclusive
user on the source database, particularly if the source database is on a Firebird Classic
server. The triggers created by IBP Replicator will be seen by other database users when
the database metadata cache is refreshed. On Classic, each client has its own cache and
won't see the changes until next time that user logs in. It is less of an issue with
Superserver, since the cache is shared by all clients and none will see the changes until
after all users have logged out.
As usual, you have several ways to make this request. This time, you will be operating on
the Source database, so select it in the Replication tree and do one of the following:
Ø right click on the Source database node and select Create System Objects from
the context menu
TIP You can create all of the system objects globally by using the main menu option.
If there are any problems--for example, missing primary key assignments--you will see
an error message and no system objects will be created in this operation. Since each
successful operation is committed, previously created system objects will persist. Correct
the source of the current exception and try again.
You can also choose to remove just one or more system objects selectively. Select the
object in the Target Database node and then right-click>Remove System Objects.
Click OK to confirm and a message will inform you that the object has been excluded.
Changing the global settings does not change the settings of schemata that have already
been defined; only newly defined schemata will have their settings initialized to the new
values.
For schemata you are going to define in future, custom default behaviour can be set up
to
Ø control the way foreign key violations are handled
Global Settings
If you are upgrading from an old version of IBP Replicator, you may notice that the
default replication scheduling attributes no longer appear in this display. It disappeared
in v.2.5, when replication scheduling was moved into a new engine within Replication
Manager. Please refer to the revised notes about the Scheduler.
When replicating new data from a source database to a target database where the
primary key for the replicated row already exists, IBP Replicator can be defaulted to use a
specific strategy to resolve conflicts between source and target records:
Priority-based: When you register a database, you can give it a priority so that
when conflicts occur, the database with the higher priority number takes
precedence. If the source database has the higher precedence, the
conflicted row is over-written on the target, but if the target has the
higher priority, the conflicted row is preserved in the target and the
replicated row is queued for manual conflict resolution.
Master-slave: The source database's row always replaces the conflicting row in
the target.
Conflicts that cannot be resolved directly by the conflict resolution strategy are logged
for manual intervention using the Conflicts tool.
The default behaviour when the Replicator encounters a Foreign Key Violation is to log an
error in the replication.log and abort the replication. You can configure different
behaviour:
Conflict: Treat it as a conflict and write the replication record to the manual log
table
You can override the foreign key violation behaviour for specific tables in
specific schemas. For more information, see the topic Foreign key violation
behavior in Table Settings.
Event Logging
Each configuration database can have its own error/information logging settings and log
file. Allowing IBP Replicator to record its activity in either a window and/or on a log file
on disk. The user defines what is written (level) and where logs are written.
0: No Logging
1: Errors Only
2: Warnings and Detailed Error Description
3: Information
4: Detailed Information
5: Debug
Logs can be written to a disk file in either the default path (which depends on platform
and run mode) or in a different one, which you you specify.
If you are setting up more than one replication you will probably want to specify
non-default paths and/or file names for each one's log.
To enable email notification, make sure the Mail Sysdba checkbox is checked. An email
notification will be sent to the designated TO address each time Replicator encounters an
error.
Once you have set up the notification details, click the Test Mail button to verify that it
works.
To prevent your mailbox from overflow, you can set a minimum interval (in minutes)
between email notifications. If it is set,
Replication Server will swallow mail messages for the specified number of minutes after
successfully sending one. The default setting of 0 minutes disables this protection.
On Windows, you will (ideally) run IBP Replicator as a service. On Windows, the
installer will set everything up to run the service according to your preferences.
Ø your server and replicant licences are installed and that you have enough
replicants to perform the configured replication[s]
Ø the source and target databases have the necessary permissions granted for the
REPL user or, if you are using a role (e.g. REPL_ADMIN as in our examples) that
the role has been created and the permissions granted. (And remember to
grant the role to REPL!)
Ø for Firebird or InterBase sources and targets, the respective servers are running
As a Service
The Windows installer should have installed the service for you. You can check this by
inspecting the Services display by way of the administration tools on your server:
Ø If you elected to have the service start automatically, it will have the Startup
Type "Automatic" and should be running ("Started") unless someone has
stopped it.
Ø If you elected to have it start manually, or it has been stopped (shut down), you
can start it the same way as you would start any other service, i.e. by right-
clicking on its entry in the list and selecting "Start" or "Restart", or by selecting
its property sheet and manipulating it from there.
As an Application
To start ReplServer.exe as an application from the graphical interface, click its icon in the
Start Menu:
The graphical interface appears, populated with the parameters for the default replication
that it has started:
If you watch long enough, the log window will fill up with an account of the replication
that is going on. If it is doing what you want, just minimise the display and let 'er rip.
You can maximise it and look in on your replication at your pleasure:
You can use the toolbar buttons to load a different set of parameters and run a different
replication:
In the example, the Timer button is depressed, indicating that the configuration is doing
(or will do) a timed replication. If you are going to do a different replication as a one-off,
you would click this button off.
Click the Clear Parameters button. Note that the application will start to nag at you to
enter something if you wait around too long! Proceed to make your alternative entries:
Ø In the Config Database field, browse for (or type in) the full connection path to
an alternative configuration database
Ø Fill in the User and Password fields and, if needed, the Role
Ø Change the default log to a different valid path and filename if you wish
Click the Replicate button to start replicating (or to do a one-off replication, if the Timer
button is off).
Either:
Ø Method 2: Stop the service from the Services applet in the Administration Tools
Note also that you can stop and restart the service from the Services applet by choosing
Restart.
Ø by selecting it in the Windows Task Manager list and clicking End Task
-m
Start replication
Convenient for running replication
automatically and quit after
--go from another application/script
replication is complete
--log-file file-path Provides an initial log file Enables troubleshooting for cases
or -l file-path for use between server start where ReplServer.exe has problems
and configuration load loading the configuration but the
default location for the log file is
unavailable for some reason
Installation Switches
In case you need to know the installation switches for making your own service installer
script, they are as follows:
/uninstall Uninstall the replicator service. (This uninstalls the service, not
the software!)
/dep=<service Used for passing the <service name> of a service that the
name> replication service depends on
Ø the command-line switches are in line with those used with the Windows
version, although not all of the Windows switches apply to Linux
Ø Refer to the list of command-line switches for details of the options available.
You can also just run ./replserver --help for a succinct list of these options
All management tasks for the Linux server can be done either through the daemon
interface or via direct commands with switches. Process instances not under daemon
control are stopped using the Linux kill/killall commands with signals.
NOTE IF you do not have a Windows machine available for defining your schemas and
installing licences, it is possible to run the Windows ReplMgr.exe program in Wine, the
Linux Windows emulation (compatibility) system.
Topics
Installing and Configuring on Linux
(The -n.n character in the tarball name will vary according to sub-release version.)
Important
#-------------------------------------------------------------------------------
# Parameters settings to start replicator
# Uncomment and modify as appropriate
# Configuration DB user
#CONFIG_USER=SYSDBA [see note below]
# Configuration DB password
#CONFIG_PASSWORD=masterkey [see note below]
# Configuration DB role
#CONFIG_ROLE=repl_admin [see note below]
The settings CONFIG_USER and CONFIG_PASSWORD refer to the user name and password for
the adminstrator of the configuration database, as you defined them (or have yet to
define) when setting up the configuration using the Replication Manager utility, either
from a Windows client on the network or installed locally under wine.
If you needed to create an SQL role with permissions to enable a non-SYSDBA and
non-Owner user to operate on the configuration database then configure this parameter;
otherwise, ignore it.
This setting determines where the replserver will write the pid file that behaves as a lock
to prevent two instances running concurrently. It can be reconfigured here or,
alternatively, with a command-line switch.
chkconfig IBPReplicator on
TIP :: You can run multiple replserver daemons, as long as each uses its own PID file.
Log Files
You can use this method to run multiple instances of replserver as applications. You
should not try to run it as a daemon if there is a service already running under chkconfig
control.
Switch Format
Note that the syntax and semantics of the switches are now the same as those used for
command-line operation on Windows, although not all of the switches available on
Windows are available on Linux.
-x=parameter
Required Switches
Should be the name of the Make sure this user exists in the
user that is connecting to security database on the server
-u=username
the configuration where the configuration database
database. resides. Not case-sensitive.
Optional Switches
-a, --application Run as application Use this switch when you want to
run replserver as an application.
--log-file file-path Provides an initial log file Enables troubleshooting for cases
or -l file-path for use between server where replserver has problems
start and configuration loading the configuration but the
load default location for the log file is
unavailable for some reason
--pid-file=filespec <filespec> Location and Default when only one instance of
name of the file that replserver is running is
Replicator uses to prevent /var/run/IBPReplicator.pid. Use
multiple instances from different names for multiple
running concurrently out instances.
of the same config file
© 2016 IBPhoenix Editors
Chapter 8 - Managing Replication 99
A process can be controlled using signals sent via the kill and killall commands.
Use kill to target a specific process by its process ID ( pid ) or killall to target all
processes named replserver. The available parameters are:
TERM
TERM (terminate, 15) is the default signal. It stops the current replication and shuts
down the replication server:
HUP
HUP (hang-up, 01) allows you to pause the process and reload the configuration:
USR1
Multiple replication processes can run concurrently on the same host system, provided
each instance of the replication server uses a different configuration database.
Trying to run multiple instances of replserver against the same configuration database
will have unpredictable results.
In the event that you have created or restored your configuration database on a Linux
server and you don't have the Windows Replication Manager connected, you can install
your licences manually using the command-line configuration editor, cfged.
In a graphics terminal, go to the directory where the Replicator executables are installed
and start cfged. Open the text email attachment file that contains your licences in a text
editor. Note the licence ID, e.g., 'REPL_99' in our example, and start the ADD LICENSE
command as indicated in the screenshot. Type a backslash followed by a Return to get a
continuation prompt.
Copy the license key to your clipboard application and paste it beside the =>
continuation prompt:
Continue to add any additional licences you have in the same manner. Use EXIT to end
your cfged session.
For more information about cfged, refer to the Configuration Editor topic in Chapter 9,
Advanced Topics.
Recently visited configurations are stored in the Registry of the machine from which they
are accessed. Replication manager retrieves the paths to these configurations, enabling
"quickstart" access to the config database via a picklist in the File.. menu:
For existing configuration databases that have not been accessed before from this
machine, you can use File..Open or click the Open button on the toolbar. An input form
will appear, similar to that used for creating a new configuration database:
If you need to remind yourself what to enter into these input fields, refer back to the
notes about filling these fields when creating your first configuration database.
The Tools
Replication Monitor
Replication Scheduler
Schema View
Notify Server
View Conflicts
License Manager
If monitoring is enabled, open the monitor by right-clicking on the Target database you
want to monitor and selecting the Monitor option from the context menu:
Click the Start button to begin monitoring the selected types. While monitoring is going
on, the button caption changes to Stop. Click this button again to stop the graphing
process.
You can graph just one type of replication statistic or any combination of types. While
the monitoring and graphing is going on, you can
Setting up a Schedule
Adding a Schedule
Initially, the add dialog shows the default "schedule", which is to replicate once, at
midnight on the current day—which won't happen today, of course! In this example, we
are choosing an hourly replication, which we can choose to be on the hour or at a
selected number of minutes after the hour. Of course, you can drop down the options
and pick the ones that you need. Click OK to add the schedule.
You can add more schedules. Any schedules you create are enabled by default, meaning
that they will all be scheduled. For example, you might want to include an hourly and a
daily schedule and keep both enabled. Any that you want to keep for alternative or
occasional use can be disabled by deselecting Enabled on the final window. Disabled
schedules appear as greyed-out in the list:
In future, you can select a schedule item in the list and use Options>Edit to change the
parameters, including enabling or disabling the item:
Ø If you want to hear an audible signal when a replication starts, also check on
Beep when Replicate
Ø To update the Scheduler and replace any schedule that is currently in effect,
select Reload Schedule from the Options menu, press F5, or click the Reload
Schedule button on the toolbar:
TIP If you need only occasional replications, you need not schedule replications at all:
simply use the menu option Tools | Notify Server | Replicate Now to replicate whenever it
suits you.
On Firebird and InterBase, this utility uses events to alert the server. It is essential to
open ports in your firewall on both the server and the target database hosts for the event
traffic.
Instructs the server to reload all its settings from the current configuration database,
usually because the configuration has been altered and there is no need to shut the
server down and restart it.
Instructs the server to replicate immediately, without waiting for its next scheduled
replication. If you need only occasional replications, you need not schedule replications
at all: simply use this command to replicate whenever you like.
If you are checking the log, you will be aware of any conflicts encountered,
because they are summarised there after each replication. There are other places
in Replication Manager where conflict information can be viewed in more detail.
Viewing Conflicts
With the replication schema you are interested in selected, either
NOTE to focus on all conflicts that may have occurred when replicating from a particular
source database or to a particular target database, have that database node selected.
If there are no conflicts, the conflict viewer simply says so. If there are unresolved
conflicts, the viewer displays them:
Here we see that the target database had 133 conflicts. We want to open the Detail tab
to see what those conflicts were:
Every conflict for this table is the same—so we select a record and click the Show Data
button at the right to find out the offending field. A new panel opens beneath the Detail
display and shows the conflict for this record. Ah—yes! we previously used a
synchronization to get all the records from the source table over to this table, but the
server clock at the target was 20 minutes fast!
Resolution
If any conflicts are listed, you can click on one to highlight it and select a button from the
Options menu to resolve it:
Force All IBP Replicator will try to replicate all the problem rows during its next
Replications scheduled replication
Force If you want to force replication on some but not all of the conflicting
Selected records, use the checkboxes at the left of the detail display to
Replications select/deselect. Then, the button will become enabled and you can use it
to tell IBP Replicator to try to replicate those rows during its next
scheduled replication
Force IBP Replicator will try to replicate the problem row during its next
Replication scheduled replication
Delete Delete the selected conflict. Data in source and target tables will stay as
record they are
Delete If you want to delete groups of conflict records, select those you want
selected deleted. This will enable the button, ready for you to click it to have the
records conflict records abandoned.
Delete all Delete all the conflicts. Data in source and target tables will stay as they
records are
After an option is executed, the record in the manual log will be deleted and the conflict
will disappear from the display, considered to have been resolved.
Finally, the view reports the "all clear" and you can close it.
Off-line Replication
Evaluation Licence
No specific restrictions are placed on the distribution of the Evaluation version, except
for the following:
Ø The evaluation licence is limited to one server and four replicant licences
Normal Licensing
On each host machine that acts as a replication server, a server licence must be installed.
Each database¾whether source or target—needs a replicant license. The minimum
server licence needs two replicant licences. One is bundled with the server licence
automatically and issued separately. You should order your initial Server licence with
enough Replicant licences attached to cover all of the databases that will be involved in a
So, for example, a machine that acts as the replication server and has a single Source
database, and which is replicating to one remote Target database, will need only a Server
licence with the bundled Replicant licence plus one more replicant licence purchased
separately.
No database needs more than one Replicant licence. If the same Source or Target
database is being controlled by more than one instance of the Replication server then you
need to account for one database only once as a replicant when calculating the replicant
denominations for your server licences.
Installing Licences
Once you have created your first configuration database, the Tools menu of the
Replication Manager becomes visible. It contains the License Manager applet for
installing your licences:
Click the License Manager item to start the applet, then click the Add button to add your
first licence.
Evaluation License
Enter EVAL as the License ID and leave the License Key field to be filled automatically.
Normal Licensing
Type in or copy/paste the first of the License ID and License Key value pairs from your
licensing document and press OK. The first licence registration appears in the display
with a "success" message:
If you get an error message, try again. The codes are case-sensitive.
Continue to add each of your licences in the same fashion, until you have registered all of
them. They should appear on the display as a list.
Don't worry about the order in which the licences are registered. When the time comes
for action, each server is only concerned that there are enough available licences of each
type to perform the tasks it is asked to do.
When you are finished, click the Close button to terminate the applet. You can access the
License Manager any time by returning to the Tools menu or running it from the Start
Menu.
Although this tool is aware of how the sources and targets in a configuration relate to
one another, at this point it is not a visual doorway to your replication schemas.
However, it can be useful as a quick way to identify information about the databases in
your replication system and to store it diagrammatically.
You can pick up a database icon and drag it to somewhere meaningful on your
background image and then drop it where you want it. Of course, this would be so much
more meaningful to most of us if the map were on a smaller scale! So the tool allows you
to use any image you like as your background: perhaps a map of your region, country,
plant, server room¾whatever works for you!
Create or copy the image (.bmp, .jpg, etc.) into a convenient location and scale it to suit.
This will open a file browser dialog, enabling you to locate and select your image.
To open a dialog where you can customise your display, click the Options button:
If you wish to save your layout, simply click on the Save button.
To restore the original (default) layout simply click the Reset button. You can also
restore the original layout from the Options dialog, by checking Use Defaults there.
The message can then be pasted into a text file or directly into an email using Ctrl-V.
The screenshot below shows the content of the message above, pasted from the
clipboard into Notepad:
[email protected]?subject=subscribe
Alternatively, visit the web interface to fill in and submit a subscription form.
Ø Metadata Changes describes what happens when metadata changes occur in the
source database
Ø Adding Timefields provides a way to generate a script that you can run to add
Timefields to all of the tables in a database
In this shot, we are looking at the configuration database for the replication used in this
manual.
So far, we have only one schema in the schemata table. Its replication number is 1.
Here we see two databases and our offline target, in the order that they were defined.
Again, no surprises. We don't see our Source database here because it is the "published"
database for this configuration, i.e., the one that is replicating its data to the targets.
In the next shot we look at some system objects created in our source database:
We find some triggers have been created for our replicated tables. The prefix is REPL$1
and the number following it is the replication number of the table¾it was the fourth table
we defined in the schema. Each configuration database numbers its objects
independently. As long as the internal Schema Numbers are unique across all
configuration databases, no conflicts can occur.
For example, ConfigOne.fdb could start at 1, whilst ConfigTwo.fdb could start at 500.
Each new Schema created within any configuration database will cause the internal
Schema number to be incremented by one¾the default Current Schema No. that you see
when this dialog pops up is generated internally by a generator called REPL$GEN, so there
is no risk of conflict with multiple schemas inside the same configuration database. Most
configuration databases have fewer than 10 replication schemas defined, so an easy rule
would be to start each configuration database with values separated by 100.
IMPORTANT When you do have multiple configuration databases, use the command-line
switches with ReplServer.exe/replserver to set the one to use for a specified instance.
Ø Plan a down-time "window" for the IBP Replicator service. Enter the "window" by
disabling the IBP Replicator service before you run the DDL script on the source.
Allow enough time in this window to apply the script to all affected databases,
test their effects and modify or replace all affected replication schemata.
Ø It is strongly recommended that you use a tidy shutdown to remove all users,
including other administrative users, from the affected systems while databases
and replication configurations are being altered.
Ø You might wish to archive the replication log file[s] before resuming services.
Ø Put the database online and restart IBP Replicator once you are happy that the
revision is complete.
For large databases, this could be a very tedious and error-prone task involving an
extended period of downtime for the users at all of the replication sites. However, you
can prepare a script ahead of time to speed up the task. You can reduce the effort
involved by defining and executing a stored procedure to generate the DDL and export
the required statements into an external table.
Generating a Script
Start by defining an external table in the database that the timefield value is going to be
applied to:
Our sample procedure takes as input the timefield identifier that we have set up for the
configuration.
The generated script depends on all table names having lengths of < 28 characters. If
you have any table names that are longer, you will need to edit the script later to replace
the trigger names in the generated script.
data = crlf ||' generated by procedure INJECT_TIMESTAMPS ' || date_today || ' */';
insert into ext_data (data) values (:data);
data = crlf;
insert into ext_data (data) values (:data);
data = crlf ||'SET TERM ^;';
insert into ext_data (data) values (:data);
for
select RDB$RELATION_NAME from RDB$RELATIONS
where RDB$RELATION_NAME not starting with 'RDB$'
and RDB$VIEW_SOURCE IS NULL
and RDB$RELATION_NAME <> 'EXT_DATA'
into :tablename do
begin
data = crlf ||'ALTER TABLE ' || tablename || ' ADD ' || timefield_id || ' TIMESTAMP ^';
insert into ext_data (data) values (:data);
data = crlf ||'COMMIT ^';
insert into ext_data (data) values (:data);
data = crlf ||'/* */';
insert into ext_data (data) values (:data);
data = crlf ||'CREATE TRIGGER BI5_' || tablename || ' FOR ' || tablename ;
insert into ext_data (data) values (:data);
data = crlf ||'ACTIVE BEFORE INSERT OR UPDATE POSITION 5';
insert into ext_data (data) values (:data);
data = crlf ||'AS BEGIN';
insert into ext_data (data) values (:data);
data = crlf ||'NEW.' || timefield_id || ' = CAST (''NOW'' AS TIMESTAMP);';
insert into ext_data (data) values (:data);
data = crlf ||'END ^';
insert into ext_data (data) values (:data);
data = crlf;
insert into ext_data (data) values (:data);
data = crlf ||'COMMIT ^';
Requirements
In designing a Stored Procedure that IBP Replicator can use, you need to make provision
for everything that will be wanted for the operations.
Input Arguments
Output Argument
Provide an integer RETURNS argument that indicates the result of the action. Possible
return values are:
A Simplistic Example
The example here is simplistic and should not be used as a boilerplate that would cover
your requirements. Refer to the comments at the end of this topic if the reasons for that
are not obvious.
CREATE TABLE T (
K1 INTEGER NOT NULL,
K2 VARCHAR(10) NOT NULL,
A stored procedure can then be defined on a target database with the same structure as
the source table as follows:
/* Inserts: If the row already exists, then exit with result=1. This
causes the Replication Server to apply conflict rules, which will
probably cause the procedure to be called again, but now with ACTION_TYPE of “U”.
An alternative approach would be to simply change the ACTION_TYPE to “U” and proceed
to update the row instead. */
IF (ACTION_TYPE = 'I'
AND ROW_EXISTS = 1 ))
THEN
BEGIN
RESULT = 1;
EXIT;
END
/* Updates: If the row does not exist, then exit with result=1. This
causes the Replication Server to apply conflict rules, which will
probably cause the procedure to be called again, but with ACTION_TYPE of “I”.
An alternative approach would be to simply change ACTION_TYPE to “I” and proceed
to insert the row instead. */
/* Deletes: If the row does not exist then exit with result=1. The
Replication Server will log the error, but otherwise ignore it. */
UPDATE T SET
K1 = :K1_NEW, K2 = K2_NEW,
K3 = K3_NEW,
F1 = :F1, F2 = :F2,
F3 = :F3, F4 = :F4,
F5 = :F5, F6 = :F6, F7 = :F7
WHERE K1 = :K1
AND K2 = :K2
AND K3 = :K3;
Comments
In real life, a less simplistic rendering of example above would need to test the
parameters K1, K2, K3, K1_NEW, K2_NEW and K3_NEW for null and ensure that the local
variables were initialized and reset in a timely way, to avoid the constraint exceptions and
unpredictable effects that null-matching and inserts into non-nullable table columns
could cause.
A change made to a source database gets replicated to a target. If the target also acts as
a source database to the original source (now a target), this change will get picked up on
the target and be replicated back, and so on.
The solution to the looping problem is to have the user REPL defined on the servers.
REPL is a “special” user within the IBP Replicator application, as IBP Replicator recognises
REPL as "myself". Any change made by a REPL user will not get replicated to the target
database.
NOTE To use the REPL user you do need to set the user up in the server's authentication
system with the appropriate privileges it needs to do its job.
Ø The other method¾usually used for very complex situations¾is to replicate all
data to a Stored Procedure on the target database and let the logic of the rules
implemented in the procedure target database decide whether it wants to
receive a row. Refer to the previous topic, Advanced Mapping Techniques, for
some guidelines for writing such procedures.
To illustrate the 'row level replication condition', we will use the following scenario as an
example.
The head office is in London, and there are two branch offices in Leeds and Manchester.
Sales data needs to be replicated from the branch offices to the head office, and certain
other specific data needs to be replicated from the head office to the branch offices.
Manchester (source) -> All Data -> London (target) <- All Data <- Leeds (source)
Manchester (target) <- DeptNo= 20 <- London (source) -> DeptNo=10 -> Leeds
(target)
On the Leeds target database, we need to define a row level replication condition for each
table, based on the DeptNo. This is done via in the table settings in the Table mapping
properties dialog.
:DeptNo=10
:DeptNo=20 would be applied in the same way to the Manchester target database.
In this scenario a central database controls the replication, fetching and sending data
from spoke databases to and from the hub.
This means that London acts both as a source and as a target for Leeds, Manchester and
Birmingham.
In this illustration, London (the central database) fetches changes made by Manchester
and forwards those changes onto Leeds and Birmingham.
In this case we cannot use the username REPL, since data would (for example) replicate
from Leeds to London, but would not get forwarded to the other branches. On the other
hand, we do not want data to be replicated from Leeds to London and then from London
back to Leeds.
Our solution is to have more than one dedicated "replication user" in our systems, e.g.
REPL_BIR, REPL_LEE and REPL_MAN for our three spokes, and use the applicable
alternative replication user as our username.
CAVEAT :: These examples are for guidance toward a custom solution to a complex
business problem. They are in no way intended to be templates or a recommendation to
meddle with the generated System Objects.
Use row-level replication conditions to achieve the effects indicated in the examples.
Trigger sets created in the source database for London (for each target) would be
modified by row-level conditions, to take the localised username into account when
determining what to replicate:
This ensures that data is replicated from one branch into the hub in London. The hub
then replicates this data to the other branches, but not back to the originating branch.
The defined replication schema for such a scenario would look something like this:
Manchester To London
Birmingham To London
Leeds To London
Target Databases
This feature will be useful where it is not possible to maintain a connection between the
source server and the replicant database that is sufficiently fast or reliable. At the source
server you replicate "outbound" to a target file, transport the file, or a batch of files, to
the target server by your preferred means (SFTP, HTTP or some transportable medium
such as DVD or flash card). At the destination, you have a file defined as the "inbound"
source and the replication server will pull the replications into the target database.
On each server. you will need replicant licences for both the source or target database
and the outbound or inbound file, respectively.
Setup Steps
On the servers, create the appropriate directories to which your offline replications are to
be written (e.g. outbound) and/or from which files are to be read (e.g., inbound). If your
replications are going to be one-way, you will not need both.
The Outbound and Inbound directories can be any file system location to which the
replserver process is allowed to write directly, including a mountable network directory
that is on-line. You can write to a removable storage device such as a flash drive or a
media card. When writing to mountable or removable storage devices, it is your
responsibility to ensure that the device is on-line whenever it needs to be accessed—
which includes any time when you are configuring the "pseudo" database—and that the
device has enough available capacity for the file[s] that will be written.
Archiving
From Replicator 4.2 onward, it is possible to preserve the files after replication
completes, rather than have them deleted, for archiving or other purposes. If you plan to
use this feature, it would be a good idea to set up an archive location as well. Preserved
files are renamed with the extension ".done".
In the examples that follow, we will set up an Outbound offline replication (Source
database to file). For setting up and Inbound replication (file to Target database) the
procedure is similar.
Decide on a friendly descriptive name for this pseudo database, e.g. "Offline", "Outbound"
or whatever you like. Fill in the fields, defining the full paths to the "inbound" and/or
"outbound" folders and selecting the database from whose metadata you want to
"borrow" metadata definitions:
Now you are ready to begin defining your offline replication schema.
For simplicity, we'll assume here that your purpose in using offline replication is to
substitute the 2-tier database-to-database schema entirely. It is possible to include an
offline replication in an existing schema to meet a specific need. You can ask in the
users' support list about any aspects of your specific case that might not be obvious.
Create a new schema and assign the database that you want to be the Source.
Now, you are ready to set up the file that you prepared as a pseudo database previously
to be the target of this outbound replication. Select the Target node and choose Add.
You may click the Test connection button to confirm that the database you named as the
one from which to borrow definitions is online.
Proceed to set up keys and fields just as you would for the target of a database-to-
database replication. When you are done, you are ready to create the system objects.
Again, this step behaves just as you would expect if you were setting up a database-to-
database schema. Creating the system objects in the Source database defines the
procedures and permissions needed to write records to the REPL$LOG table and identifies
the mappings that will be rendered to the destination file.
Now, the replication is ready to go, just like a normal 2-tier replication. You can verify it
by checking the Outbound directory and the replication log.
In the Outbound directory, a fresh file is created for each replication cycle. The file
names are of the form Rxxxxxxxx.dat, where the x's are an alphanumeric representation
of a number. The initial value of the number is calculated from the CURRENT_TIMESTAMP
value at the launch of the Replication server and is incremented by 1 for each successive
file.
During replication, the file name has the suffix '.tmp'. If the transaction is rolled back for
some reason, this temporary version of the file is deleted. Should something untoward
happen that leaves the file in place with this .tmp suffix, do not use it.
Once the file has been renamed with the '.dat' suffix, it is ready for its journey to the
target server. That file must be used to replicate to a database in a schema that is within
a different configuration database. IBP Replicator does not provide any tools for porting
the offline files between the servers: it is your responsibility to design and set up a
transport mechanism to suit your conditions.
The next section contains brief instructions for configuring the target server to receive
the inbound file.
It is strongly recommended that you read the whole of the Offline Replication topic. The
outbound side of offline replication is described in detail. Since the procedure at each
side is almost identical, this section is merely a brief overview of the inbound side.
Databases
At this side, the on-line database is configured as the Target, while the 'pseudo-
database' inbound directory will be configured with Files... as the Source.
Schema Configuration
Configuring the schema for the inbound replication will be the reverse of what you did
when configuring the outbound one. No system objects are created in the on-line Target
for this replication.
The IBP Replicator package does not provide any tools for porting the offline files
between the servers. It is up to you to set up a mechanism for picking up a file from the
source server and delivering it to the configure Inbound directory.
Do not overlook the fact that the inserts, updates and deletes replicated from the file will
be timestamped with the source server's timestamps, which may have a bearing on their
age relative to the timestamps on any records affected by the inbound replication and/or
on any REPL$LOG records awaiting a downstream replication.
Offline synchronization generates a set of files containing all of the data from tables
specified for an offline replication. After such a synchronization, the specified outbound
"database" directory will contain one file for each table specified. The off-loaded files are
named as "STableName.synch", where TableName is the name of a table. The binary
format of each .synch file is the same as that used for the replication ".dat" file. If the
".synch" extension on these files is renamed to ".dat", the Replicator can process them as
an off-line replication.
Gotchas
An original "Rnnnnn.dat" file is the output from processing all of the specified tables, in
the order specified when the replication was defined. However, offline replication does
not respect the order in which tables were defined: the replicator service simply starts at
the beginning of the directory and works its way through it in alphabetical order, deleting
each file as it finishes with it.
inbound directory. There will be one such file for each conflict, which will be listed by
the Conflict Resolution tool if it is invoked on the "database" that is defined as the offline
replication source.
Conflict files are written during the up-replication, as each conflict occurs. Since the
data in the conflict files is not under transaction control, a rollback of the replication
does not delete any conflict files that were created before the rollback. Thus, careful
management of these files is needed, to avoid accidentally processing the same conflicts
more than once.
These "*.synch" files can be saved for use—or re-use—as backups for point-in-time
recovery for a whole database or just selected tables. Each time a file is re-used it will
reset the target table to the state of the source table when the file was off-loaded.
Synchronization vs Replication
The synchronization files have the same structure as files created for off-line replication.
The off-loaded files are named as "STableName.synch", where TableName is the name of
a table. By renaming the ".synch" extension to ".dat" and passing them to the Replicator
process as an off-line replication, you can preserve the pre-existing records in the target
table.
For more information about replicating from .synch files, refer to the topic Replicating
from an Offline Synchronization Set in the previous section.
You can synchronize all of the tables configured for the target database ("whole
database") or just one table. If you have more than one table to synchronize, although
not the entire database, the synchronizations need to be performed one by one. There is
no way to "multi-select" a sub-set of tables.
"Missing" Tables
If the whole database is selected for synchronization and there is a mismatch between
the source table names available and those configured for the target database, a stop
error will occur and the synchronization will be rolled back.
Suppose you have a table created with CREATE TABLE "Holy Mackerel"... The Replication
Manager, running on Windows, will off-load a synchronization file named "SHoly
Mackerel.synch". Windows does not care about case-sensitivity or the presence of spaces
in a name. You will run the up-load part of the operation from Replication Manager as
well; so the application will not be bothered by either issue.
Where there might be an issue with such file names is in a server environment that
requires them to be single-quoted in entirety, e.g. 'SHoly Mackerel.synch' or to have
spaces and other special characters escaped, e.g., SHoly\ Mackerel.synch. You might
bump into such an issue if you are on-loading and moving files around at the command
line on some POSIX platforms.
If non-ANSI characters were used in defining tables in either database, there is the
possibility of problems occurring with the names of the OLS files at filesystem level if the
target platform is not configured to interpret those characters correctly. This is not
Replicator's problem, per se, but it will need to be checked to ensure that your OLS can
proceed smoothly.
Character Sets
The off-loaded files are in a binary format so there should not be any character set issues
at application level. If you are using OLS to populate a new, empty database, make sure
that the character set and collation attributes of the target database and all relevant
objects in it are the same as those in the source database.
Running an Off-load
As with off-line replication, you need to define a target "database" that is a folder
somewhere in your filesystem. If you have already set one up for off-line replication, it is
possible to use that. Otherwise, go to the start of this section for instructions on how to
do this setup.
Once this target structure exists, the approach for an off-line synchronization is no
different from that for an on-line one. In the Replication tab, select the offline "database"
and right-click on it. Select Synchronize.
Notice that the Symmetric synchronize option is disabled, for obvious reasons.
As with the online synchronization, A monitor window will pop up and display the
synchronization as it happens. The window will fill up very fast but you can scroll back
to check what has been done:
When it is done, you can check the contents of your "outbound" directory. There should
be one ".synch" file for each table defined in the replication.
To use the off-loaded files to synchronize a database, set up a File.. type of database as
the source, pointing to the folder where the Replication Manager can find your off-loaded
files. If you already have a schema set up for an offline up-replication, simply use the
same schema. It won't interfere with any waiting replication, since the file extensions are
different.
When you are ready, just right-click on it in the tree panel and choose Synchronize. The
monitor list will pop up and show you what it is doing.
The up-synchronization will stop and roll back if dependency problems are encountered.
Make sure that the table order defined in the target is correct for the referential integrity
dependencies in your database: parent tables must precede their children. Use the Up
and Down buttons in the toolbar to rearrange the table order.
Appendices
142 IBP Replicator 5.0
Appendices
10.1 Appendix I: Supported Data Types and Compatibilites
Table 1: Interbase/Firebird
1 No datatype or dimension transformation. Total size of data must fit available RAM.
2 In Interbase only
Table 2: Oracle
1 Fields with character set OCTETS are copied as-is, others must be in hexadecimal form.
2 For DATE and TIMESTAMP types string is expected in ANSI format, not current NLS.
3 For INTERVAL types string must fit current client NLS format.
4 If a destination field has character set OCTETS, its content is copied as-is; otherwise it
is converted into hexadecimal representation.
Starting cfged
Using a graphical window or a shell on Linux, or a command window on Windows,
change to the directory where the Replicator executables are located and enter the
command cfged:
The OPEN command opens a configuration for editing. The first parameter, which is the
only one not accompanied by a keyword, is the absolute path or (for Firebird, the alias)
for the location of the configuration database. In this screenshot, a Firebird configuration
database with the alias REPLCFG has been opened.
Any parameter that is omitted, other than ROLE, will be taken from the default
configuration, if it is set.
USER - either the database owner or the user name of a user with SYSDBA privileges. If
you need this, you will also need to include PASSWORD along with the password as its
argument.
ROLE role_name if the user credentials are tied to a role for access to configuration
structures.
SILENTLY - can be included to stop cfged from showing warnings about a missing
configuration or a wrong version.
If a configuration is already open, it will be closed before the requested one is opened.
Ø SHOW SCHEMAS lists the schemas that have been defined in this configuration
database
Common Rules
Although there is variation in the keywords, syntax and parameters for different
commands (discussed later), a small set of rules is common to all commands.
The continuation sign => that appears is ready to receive the pasted licence key
from the clipboard:
OPENREPLCFG - rejected
4. Parameters that contain spaces, such as file paths, must be delimited by pairs
of quotes (single or double) or by square brackets. For example:
or
Command-line Switches
When you start cfged, in either interactive or direct mode, or within a batch script, some
switches are available to control the way the program runs.
Commands
The cfged application has a command vocabulary consisting of keywords for a number of
verbs and operands and parameters. All commands start with a verb. Some verbs are
self-contained while others require objects, parameters, or both.
The [ ] and < > symbols that appear in the Syntax sections are not syntactic
elements.
On-line Help
HELP
Syntax
HELP [<command>|ALL]
Used alone, HELP displays some usage notes and a list of the available commands.
HELP ALL gives detailed help for all of the commands, in alphabetical order.
The equivalent switches when passed from the command shell are -h and ? .
Where Am I?
SHOW CURRENT
Shows the numbers of the current database, schema, target database and table. It takes
no parameters.
Syntax
SHOW CURRENT
SET CURRENT
Set the specified object (DATABASE | SCHEMA | TARGET | TABLE) to be the current one of
its type.
Syntax
Take care with this command if specifying the object number as the argument, as
no existence check is performed.
Neither is there any check to verify whether a specified table or target database
belongs to the current schema.
Program Settings
SET BAIL
Controls whether cfged should exit immediately after an error occurs. Useful during
script debugging, it is alternatively available as a command switch.
Syntax
Equivalent switch -b
The equivalent switch when passed from the command shell is -b.
SET ECHO
Controls whether cfged displays executed commands. ECHO ON is useful in batch mode
when standard input has been redirected.
Syntax
Equivalent switch -e
The equivalent switch when passed from the command shell is -e.
SET PROMPT
Controls whether cfged should display the input prompt. PROMPT OFF Is useful in batch
mode.
Syntax
CREATE CONFIGURATION
For the Oracle driver, the command CREATE CONFIGURATION is equivalent to OPEN+INIT.
Syntax
The first parameter is the absolute path or (for Firebird, the alias) for the location of the
configuration database. The keyword CONFIGURATION is optional.
The remaining parameters can be in any order. Any that is omitted will be taken from the
default configuration. The default configuration, if it is set, is taken from the Windows
registry or the Linux config file (usually /etc/IBPReplicator.conf).
Other parameters
USER - either the database owner or the user name of a user with SYSDBA privileges. If
you need this, you will also need to include PASSWORD along with the password as its
argument.
ROLE role_name if the user credentials are to be tied to a role for access to configuration
structures.
DROP
Syntax
DROP
EXTRACT
Export the current configuration into a file or a console. The resulting output is a script
that can be used for cfged.
Syntax
INIT
Syntax
INIT
CLEAR
Use to clear a replication configuration from a user database without dropping the
database. It takes no parameters or arguments.
Syntax
CLEAR
SET AS DEFAULT
Syntax
UPGRADE
Upgrades the configuration structures in the currently opened database to the current
version. Use this command after installing a new version of IBPReplicator over an older
version (Replicator 3 or higher). It takes no parameters.
Syntax
UPGRADE
ADD LICENSE
Syntax
Separate the license id from the license key with a backslash followed by Enter
and paste the license key into the new line.
REMOVE LICENSE
Syntax
You can use SHOW LICENSES to identify the licence you want to remove. You
cannot remove an EVAL licence.
SHOW LICENSES
Database Registration
ADD DATABASE
Syntax
Common parameters
If you want to look up the numbers and descriptive names of existing registered
databases, use SHOW DATABASES.
If this parameter is not used, or is empty, the default character set from the
environment variable NLS_LANG is used.
KEY_SIZE - the size of the field in the log tables to store the values of the key field
(s).
USER - administrative user name. For a database used as a source for any schema
the user must be one of following, to enable cfged to create the logging
triggers on replicated tables:
a) SYSDBA
c) A user that has been granted the RDB$ADMIN role. In this case, the
Role [below] must be registered as RDB$ADMIN.
SERVER - the entire connection string, including the host name and port number
if applicable to the connection protocol. Required.
or
DSN - the name of the system DSN that has been configured for an ODBC target
ROLE - administrative role. If the USER parameter is a regular user that has been
granted the RDB$ADMIN role, then RDB$ADMIN is required here.
InterBase only:
DIALECT - SQL dialect for the connection. Must be AUTO (the default), 3 or 1.
Don't use this parameter unless there is a real reason to do so.
Don't use this parameter for Firebird databases: Firebird selects the correct
client dialect automatically.
INBOUND - the name of the inbound directory. The directory must exist. For an
Alaverion server with embedded replication enabled, this should be the
same directory that is configured as OutputDirectory in ibpr_cdc.conf.
OUTBOUND - the name of the outbound directory. The directory must exist.
AFTER APPLY Delete | Rename - Used for optionally preserving files after
successful replication. The default is Delete. If Rename is supplied, files
will be renamed with the extension ".done".
If you want to look up the numbers and descriptive names of existing databases,
use SHOW DATABASES or SHOW SCHEMAS.
MODIFY DATABASE
Modify database registration information. Rules and parameters follow those for ADD
DATABASE (above). If the database number or name is omitted, the current database is
modified.
Syntax
REMOVE DATABASE
Remove the database registration info from a configuration. All replication schemas that
use this database will also be removed.
Syntax
SHOW DATABASE
Syntax
SHOW DATABASE <db number or name> shows information about the specified
database.
SHOW DATABASES
Lists the names, numbers and types of all registered databases. It takes no parameters.
Syntax
SHOW DATABASES
ADD SCHEMA
Add a new schema with a given set of parameters. Some parameters may be omitted. In
that case default values will be used.
Syntax
Schema parameters
[NAME] <schema name> - your own user-friendly name for a new schema. It is
required. Make sure it is different from other schema names in the
configuration. Use of the keyword NAME is optional if the schema name is
the first parameter.
SOURCE <source database number or name> - provide either the number or the
friendly name of a registered database. If this parameter is omitted, the
current database will be used.
Although cfged will allow you to set an ODBC DSN as the source database
for a schema, do not do it. The underlying DBMS engine is inaccessible to
the application, making it impossible to create system objects.
ENABLED | DISABLED - By default, the new schema will become the active one.
Use this parameter to control this option.
COMMENT <schema comments> - Option comments you can include with the
schema.
USER <user name>, PASSWORD <user password>, ROLE <user role> - User
name, password and (if required) the name of the role that is authorised to
read the source database and write to the target databases to be included
in this schema.
If you provide the strategy number explicitly, then possible strategies are:
Ø 0 - priority-based
Ø 1 - timestamp-based
Ø 2 - master-slave (default)
LOG <log levels> | DEFAULT - one of two possible LOG parameters available.
This one sets the logging levels and takes two integer arguments, for
logging to the window (first argument) and logging to the log file (second
argument). For example,
ADD SCHEMA 'My Initial Schema' SOURCE 1 USER REPL PASSWORD xxxxxxx \
ROLE REPL_ADMIN CRS 1 LOG 3 4 LOG INISCHEMA.LOG ORDER 0
specifies level 3 logging to the window and level 4 to the log file.
Log levels are from 0 (no log) to 5 for ordinary builds of the Replicator server. If
you are running a debug build, the highest logging level is 7.
LOG <log file name> - allows the name of the log file to be customised. By
default, it will be REPLICATE.LOG and you do not have to specify that.
You can use both LOG parameters in a single command, if need be.
SEPARATOR <separator character code> - Use this if you need to change the
character that will be used in the replication log table to separate the
segments of multi-segment keys in the text representation of records
changed. By default, the value is 9 (indicating ASCII decimal 9, TAB).
Don't change this unless there is a real reason to do so.
ORDER <number> - This parameter enables you to set the order of multiple
schemas in your configuration, corresponding to moving them up and
down in the Replication Manager program (view topic).
MODIFY SCHEMA
Use to modify a replication schema. Parameters and arguments follow the same rules as
ADD SCHEMA (above).
Syntax
REMOVE SCHEMA
Syntax
If SILENTLY is used, any warnings about possible failures during the removal of the
schema are suppressed.
SHOW SCHEMA
Shows information about the schema. If the schema number or name is omitted, the
information about the current schema is displayed.
Syntax
SHOW SCHEMAS
ADD TARGET
Add a new target database to the schema. If a schema or a database name is omitted
current values are used.
Syntax
[TO] [SCHEMA] <schema name or number> - if you have multiple schemas, use
this to designate the schema to which the new target is to be added. The
keywords TO and SCHEMA are optional.
The three parameters USER <user name>, PASSWORD <user password> and (if
required) ROLE <user role> are for the credentials of the replication user
(normally REPL).
COMMIT <commit count> sets the number of records that constitute a batch that
is to be committed, if and only if you need to use the "periodic commit"
provision. Don't use this parameter unless it is actually necessary. It can
impact the general performance of the database over time and breaking a
replication into numerous separate transactions breaks the overall
atomicity of a replication. For more information, see the topic Periodic
Commit.
LOG <log levels> | DEFAULT - one of two possible LOG parameters available if
you need to override the schema setting for this target. One sets the
logging levels and takes two integer arguments, for logging to the
window (first argument) and logging to the log file (second argument).
Log levels are from 0 (no log) to 5 for ordinary builds of the Replicator server. If
you are running a debug build, the highest logging level is 7.
If DEFAULT is provided (or implied by being omitted), the level set for the
schema will be used for logging replication to this target.
LOG <log file name> - allows the name of the log file for replications to this
target to be customised.
You can use both LOG parameters in a single command, if need be.
ORDER <number> - This parameter enables you to set the order of replication
when this target not the only one. Use SHOW TARGETS to view the current
order of replication per schema.
MODIFY TARGET
Syntax
Target DB parameters
If the OF SCHEMA parameter is omitted, the current schema will be used. The
keyword OF is optional.
REMOVE TARGET
Syntax
If SILENTLY is used, there will be no warnings about any failures during the
removal of the target.
SHOW TARGET
Syntax
Description:
If a database number or name is omitted, then the current target database is used.
If the schema number or name is omitted, then the current schema is used.
When specifying the schema explicitly, either of the keywords OF and SCHEMA
may be omitted , but it is recommended to use one or both.
SHOW TARGETS
Syntax
ADD DATA
Syntax
Field parameters
FIELD source field name - The name of the column to add. '*' means 'All fields'.
The keyword FIELD is optional.
NUMBER field number - A number for the field that is unique amongst the fields
in the replicated record. Use a number that is the same as the field
number used by the database engine in the metadata of the target
database.
It is not mandatory to have these field numbers matching but it does reduce the
risk of errors when later editing the field definitions in the Replication Manager
program.
TO target field name - The name of the field in the target record. If the target
field name is not set, it is assumed to be the same as the source field
name. Omit this parameter if the FIELD argument is '*'.
SCHEMA schema number or name - required if the configuration has more than
one schema
TARGET target database number or name - required if the configuration has more
than one target database
TABLE table number or name - required if more than one table is replicated in
the schema.
Syntax
The parameters for SET in this command follow the same rules as for ADD DATA (above).
Use to remove a non-key data field from a list of replicated fields in a record. The
parameters and arguments follow the same rules as for ADD DATA and MODIFY DATA
FIELD (above).
Syntax
Key fields cannot be removed with this command. The entire table must be
removed to undefine a key field.
SHOW FIELDS
Lists the replicated fields for the specified target database and table in the specified
schema.
Syntax
The keywords SCHEMA, TARGET and TABLE are optional but, if any keyword is
omitted, the parameters must be in strict order.
ADD KEY
Syntax
NUMBER field number - A number for the field that is unique amongst the fields
in the replicated record. Use a number that is the same as the field
number used by the database engine in the metadata of the target
database.
It is not mandatory to have these field numbers matching but it does reduce the
risk of errors and confusion when later editing the field definitions in the
Replication Manager program.
FIELD source field name - The name of the key column segment to add.
TO target field name - The name of the corresponding key field in the target
record. If the target field name is not set, it is assumed to be the same as
the source field name.
SCHEMA schema number or name - required if the configuration has more than
one schema
TARGET target database number or name - required if the configuration has more
than one target database
TABLE table number or name - required if more than one table is replicated in
the schema.
MODIFY KEY
Use to modify a key field definition. Field parameters follow the same rules as ADD KEY
(above).
Syntax
ADD SEQUENCE
Add a replicated sequence to a replication schema. By default the same sequence name is
added to the current schema and the current target database.
Syntax
Sequence parameters
[FROM] <source sequence name> - the name of the generator in the source
database that you want added to the schema. It is mandatory, although
the FROM keyword is optional.
For Oracle databases, the sequence name can include the database
schema in the format SCHEMA.SEQUENCE. Both parts of the sequence
name are case-sensitive.
TARGET <target db number or name> - Use this parameter if the target database
is not the current target database (or if you are not sure!)
MODIFY SEQUENCE
Use to modify the properties of a replicated sequence. Parameters follow the same rules
as for ADD SEQUENCE (above).
Syntax
REMOVE SEQUENCE
Syntax
SHOW SEQUENCES
List the replicated sequences for a given or current schema and target database.
Syntax
In contrast with the behaviour of other commands, if the target database number
or name is omitted, sequences for all target databases will be shown, not just
the current one.
ADD TABLE
Add a replicated table to a replication schema. By default a table is added to the current
schema and the current target database.
Most of the settings are effective only during the creation of system objects. If you alter
them afterwards, it will be necessary to remove and recreate the system objects.
Syntax
Table parameters
For Oracle databases, table or procedure names can include the database schema
in the format SCHEMA.TABLE. Both parts of the table name are case-sensitive.
The FROM parameter, with the name of the source table as its argument, comes
first and is obligatory. The keyword FROM is optional.
Use SHOW TABLES n (where n is the schema number) to list the currently defined
tables with their ordinal numbers.
COMMENT <table comment> - Optional text comment: use it for any notes you
TIME_FIELD <TS field name> - the name of the time field column that is used for
timestamp-based conflict resolution. It must be the same in both the
source and target tables, or it won't work. Only needed if this table pair
uses a different time field name than the one set for the schema.
TARGET <target db number or name> - Use this parameter if the target database
is not the current target database (or if you are not sure!)
SEPARATOR <separator character code> - Use this if you need to change the
character that will be used in the replication log table to separate the
segments of multi-segment keys in the text representation of records
changed. Needed only if a different character has to be used for the key
of this table than the one defined for the schema, for some reason.
KEEP_STATS <boolean value> - Set True to enable the GUI performance monitor
tool. By default, it is disabled for all tables, to reduce performance
overhead by allowing replserver to work with read-only access to the
tables in the configuration database. Omit this parameter unless you want
to set it True immediately. It can be enabled later, using MODIFY TABLE.
ORDER <number> - Use this parameter if you want the table to occupy a specific
position within the replication order, e.g., to precede other tables that
have foreign key dependencies on it. You can use SHOW TABLES SCHEMA
n to view the order of tables already defined.
You may wish to omit this parameter and defer changing the order of the tables
until you have finished defining them all. A unique number will be auto-
generated if you don't set it here explicitly.
CRS <strategy number >| DEFAULT - Use this parameter if you need to override
the conflict resolution strategy that is set for the schema, for this table
only. Available strategies are:
Ø 0 - priority-based
Ø 1 - timestamp-based
Ø 2 - master-slave
Error :: causes an error to be written to the log and the record will not be
replicated.
MODIFY TABLE
Use to modify properties for replicated table. Parameters follow the same rules as for
ADD TABLE (above).
Syntax
REMOVE TABLE
Syntax
If SILENTLY is used, there will be no warnings about any failures during the
removal of the table.
SHOW TABLE
Syntax
SHOW TABLES
Lists the replicated tables for a given or current schema and target database.
Syntax
Create system objects such as log tables and triggers for a given configuration. If a
schema number or name is omitted, the current schema is used. No defaults for the
target database and table.
Syntax
Parameters
FOR SCHEMA <schema number or name> - Use this parameter if the schema is
not the current one (or if you are not sure!) The keyword FOR is optional.
TARGET <target db number or name> - Use this parameter if the target database
is not the only target database. The keyword TARGET is optional.
TABLE <target table number or name> - optionally create system objects for just
a specified table. If this parameter is omitted, system objects will be
created for all tables. If TABLE is specified with no argument, system
objects will not be created.
Use to remove a system object or all system objects for a given configuration. On
completion, all pending replications and unresolved conflicts for the specified part of the
schema are dropped.
Syntax
Parameters
FOR SCHEMA <schema number or name> - Use this parameter if the schema is
not the current one (or if you are not sure!) The keyword FOR is optional.
TARGET <target db number or name> - Use this parameter if the target database
is not the only target database. The keyword TARGET is optional.
TABLE <target table number or name> - optionally remove system objects for
just a specified table. If this parameter is omitted, all system objects for
the schema will be removed. If TABLE is specified with no argument, no
system objects will be removed.
ADD SCHEDULE
Syntax
A new schedule is enabled by default. Include DISABLED if you just want to add
the schedule item without enabling it.
Argument Formats
Ø Time - HH:MM:SS
MODIFY SCHEDULE
Use to modify an item in the replication schedule. Parameters and arguments follow the
same rules as ADD SCHEDULE (above).
Syntax
REMOVE SCHEDULE
Syntax
SHOW SCHEDULE
Syntax
SHOW SCHEDULE
MODIFY GLOBAL
Syntax
MODIFY GLOBAL
SET <global parameters>
Global parameters
If you provide the strategy number explicitly, then possible strategies are:
Ø 0 - priority-based
Ø 1 - timestamp-based
Ø 2 - master-slave (default)
LOG <log levels> | DEFAULT - one of two possible LOG parameters available.
This one sets the logging levels and takes two integer arguments, for
logging to the window (first argument) and logging to the log file (second
argument).
Log levels are from 0 (no log) to 5 for ordinary builds of the Replicator server. If
you are running a debug build, the highest logging level is 7.
LOG <log file name> - allows the name of the log file to be customised. By
default, it will be REPLICATE.LOG and you do not have to specify that.
You can use both LOG parameters in a single command, if need be.
[MAIL] TO <e-mail address> - An email address for the administrator who wants
the notifications. The keyword MAIL is optional if this is not the first MAIL
parameter contained in the command.
[MAIL] HOST <SMTP server host name> - Host name of the sending email server.
[MAIL] FROM <e-mail address> - A Sender email address that has been set up for
IBPReplicator's use.
[MAIL] NAME <visible name> - The Sender name that is to appear in the headers
of emails sent by IBPReplicator.
[MAIL] PERIOD <minimum time between mails> - To prevent your mailbox from
overflow, you can set a minimum interval (in minutes) between email
notifications. If it is set, IBPReplicator will swallow mail messages for the
specified number of minutes after successfully sending one.
BEEP <boolean value> - Set to True if you want the server to give an audible
signal when it begins replicating.
Error :: causes an error to be written to the log and the record will not be
replicated.
SHOW GLOBAL
Syntax
SHOW GLOBAL
SIGNAL SERVER
Sends a notification to the replication server to perform the specified action. The action
parameter (REPLICATION|RELOAD|EXIT) is mandatory.
Syntax
The configuration database structure in version 4 and higher releases of this software is
not compatible with those of previous releases.
The newer Replication Manager will automatically upgrade old configuration databases on
opening. If the configuration editor cfged is used, executing the command UPGRADE will
achieve the same thing.
Existing IBPhoenix Replicator V.3 and V.4 licences are valid for V.5 and will be preserved.
Licences from versions prior to V.3 are not valid for Replicator 5 and will be removed
from the configuration altogether.
If you are upgrading from 2.1 or lower versions, you will find that the Replication
scheduler is now built into the Replication Server and no longer runs as a separate
service. Configuring and applying a schedule are now done using the Scheduler tool or
the configuration editor. Upgrading from a pre-2.5 configuration will convert a "regular
interval" Replication into a Schedule.
If you are currently using Replicator 2.1 or older, you must uninstall the old Replicator
Scheduler service before upgrading. It was superseded at v.2.5 and is no longer a
component of the product.
Replicator 5.0 can be "plugged in" to the IBPhoenix "Avalerion" builds of Firebird 3.0 to
run internally, i.e., the replicator engine runs as plug-in engine code, not as a client
application.
-U-
Unique keys, 23
UPGRADE (cfged), 152
Username,for source database, 51
Username,for target database, 57
Using .synch files for Replication, 135
Utilities, 100
-V-
View Conflicts tool, 108
-W-
Working on definitions, 48