Attunity Full Load and CDC Processes

The full load process creates files or tables at the target endpoint, automatically defines

the metadata that is required at the target, and populates the tables with data from the

source. Unlike the CDC process, the full load process loads the data one entire table or file

at a time, for maximum efficiency.

The source tables may be subject to update activity during the Load process. However,

there is no need to stop processing in the source. Replicate automatically starts the CDC

process as soon as the load process starts. It does not apply the changes to the target until

after the load of a table completes because the data on the target might not be consistent

while the load process is active. At the conclusion of the load process, however, Replicate

guarantees consistency and integrity of the target data.

If the load process is interrupted, it continues from wherever it stopped when restarted.

You can add new tables to an existing target without reloading the existing tables.

Similarly, you can add or drop columns in previously populated target tables without

reloading.

This is the beauty of Attunity Replicate which takes care of full load followed by CDC without losing data while maintaining data consistency and integrity. Other replicate tool like Golden Gate isn't easy as easy replicate because you have to perform initial load aka full load, followed by CDC with handle collision options and then later  remove that feature. This is manual process and hard to follow if you are new to Golden Gate Replicate and GG users know what I am taking about.

Attunity full load does not play well when the target tables have Triggers and the bulk load feature is enabled. The  following steps will help you to overcome Full Load otherwise it will throw and error and the job terminates with error. 

Yes, you have an option to disable bulk load option on target end point. This just works fine but it takes 5 times more to complete the Full Load. Do you really want to wait that long? You may run out of maintenance window. The below steps is a manual process but it takes less time to complete full reload and you don't have to baby sit the task too long.

1. Stop replicate task and note down current timestamp of Replicate Server
2. Disable Change Data Capture (CDC)
3. Truncate table (If  the truncate is disabled)
4. Disable Triggers
5. Run Full load
6. The task stops when the Full Load is complete
7. Enable Triggers.
8. Enable CDC
9. Save the Settings
10. Select Advanced Run Options and start the job from the timestamp from step one.
11. Monitor till the job become current.
12. Validate the count on source/target tables.

The CDC process captures changes in the source data or metadata as they occur and applies them to the target endpoint as soon as possible in near real time. It captures and applies the changes as units of single committed transactions and can update several different target tables as the result of a single source commit. This guarantees transnational integrity in the target endpoint. The CDC process for any file or table starts as soon as the data load process for the file or table begins. 

CDC operates by reading the recovery log file of the source endpoint management system and grouping together the entries for each transaction. The process employs techniques that ensure efficiency without seriously impacting the latency of the target data. If the CDC process cannot apply the changes to the target within a reasonable amount of time (for example when the target is not accessible), it buffers the changes on the Replication server for as long as necessary. There is no need to re-read the source DBMS logs, which may take a long time.

Attunity Replicate engineers did a fantastic job automating full load followed by CDC without losing any data and maintain consistency and data integrity. This is why I recommend Attunity Replicate of Golden Gate. Only Golden Gate users knows the pain and Qlik (Attunity Replicate) solved it.