SQL Redo Log Topic Sink - WIP

Reads SQL_REDO field from the redo-log-topic messages by the Oracle CDC Source Kafka connector and executes the statement in the Oracle destination.

Parse statement

• Remove ; at the end
• Add TO_TIMESTAMP format.
• Remove and ROWID='xxx' (UPDATE/DELETE)

Statements

• INSERT
• UPDATE
• DELETE
• ALTER TABLE
  • ADD COLUMN
  • MODIFY COLUMN

• Connector configuration:

{
"name": "sqlredo-sink-connector",
"config": {
  "connector.class": "mcolomer.connectors.sqlredo.SqlRedoSinkConnector",
  "connection.url": "jdbc:oracle:thin:@oracleTarget:1521:ORCLCDB",
  "connection.user": "MYUSER",
  "connection.password": "mypassword",
  "topics": "redo-log-topic",
  "value.converter": "io.confluent.connect.avro.AvroConverter",
  "value.converter.schema.registry.url": "http://schemaregistry:8081",
  "key.converter": "io.confluent.connect.avro.AvroConverter",
  "key.converter.schema.registry.url": "http://schemaregistry:8081"
  }
}

Create the table on the destination database:

CREATE TABLE ReplAudit (operation VARCHAR(100), tableName VARCHAR(100), sourceTs TIMESTAMP, destTs TIMESTAMP, latency NUMBER(10));

The sink connector will insert one Audit record per operation.

Maven build:

mvn clean install

Connector JAR: ./target/oracle-sql-redo-sink-1.0-SNAPSHOT-jar-with-dependencies.jar

CONNECT_PLUGIN_PATH: "/usr/share/java,/usr/share/confluent-hub-components"