In-memory caching and persistence v7

The data changes are fetched and stored in memory buffers to optimize the data replication process. This technique avoids the overhead associated with repeatedly fetching the same set of changes from the database server with multiple target databases.

This approach is sufficient as long as all of the target databases are accessible during a replication event and the data fits in the available cache.

However, if one or more of the target databases is unavailable due to network connectivity problems, server down time, etc., the in-memory data changes must be persisted for later retrieval when the target databases becomes available for synchronization with the source database.

The Replication Server architecture uses Java object serialization to persist the in-memory state of the data. Object serialization converts object data and other relevant information to a sequence of bytes that can then be stored in a file.

The following are examples that can result in evicting in-memory data to persistent storage:

• Before the next replication event occurs, the in-memory cache is filled with the data changes and needs to be evicted to accommodate a new set of changes.
• The replication system has multiple target databases. During a synchronization event, all of the changes available in the cache are applied successfully to some of the target databases. However, one or more of the other target databases can't be accessed. All of the applied changes held in memory must be persisted and retained so that these changes can be reloaded and applied when the inaccessible databases becomes available.

The cache size corresponds to the heap size configured for the publication server by the -Xmxnnnm setting of the JAVA_HEAP_SIZE parameter in the Replication Server configuration file. See Replication configuration file for information on the Replication Server configuration file.

You can minimize the persistence I/O overhead by increasing the heap size value and defining a more frequent synchronization interval, such as for every few seconds. See Creating a schedule for information on setting a replication schedule.

The data changes are persisted in a local file on the host running the publication server. The file is stored in the directory XDB_HOME/xdata.

Each time persistence occurs, a new file is created. After the files are processed, they are periodically removed from disk.