Skip to main content
Question

How to get updates into V from a transactional SQL server system

  • May 19, 2022
  • 4 replies
  • 9 views

DieterC
Forum|alt.badge.img+2

Hi forum,

a partner of us is architecting a DWH solution. Major data source is a transactional MS SQL server system. Target is Vertica.
To get the updates from SQL Server into V the customer is looking at Qlik Replicate, formerly Attunity Replicate.
https://www.qlik.com/us/-/media/files/resource-library/global-us/direct/datasheets/ds-qlik-accelerate-your-data-pipeline-en.pdf

Is this a good idea?
Would you have alternative suggestions?
What is the best way to capture updates from a SQL Server system into V please?

Thank you

Dieter

4 replies

mosheg
Forum|alt.badge.img+2
  • Participating Frequently
  • May 19, 2022

Attunity Replicate should work but it might be costly.
If the incremental data source is relatively small use for free Maurizio's DBLINK()
To download the files and get further information how to configure the ODBC layer see: https://github.com/vertica/dblink


viganog
Forum|alt.badge.img+1
  • Participating Frequently
  • May 19, 2022

This is the architecture OTE is using to replicate data from Postgres .
Postgres -> Attunity -> Vertica
(Attunity is expensive)


s_crossman
Forum|alt.badge.img
  • Participating Frequently
  • May 19, 2022

If this is a long term solution they are looking for Qlik Replicate may not be a good choice. In February 2022 Qlik sent a letter to customers citing retirement of several source and target endpoints. Vertica is amongst the targets being retired. Retirement date is July 2023. Below is an extract. We tried to reverse their decision but they used their tracking info for licenses and usage as evidence of minimal revenue generated.

The following endpoints will be retired on July 31, 2023 (the Retirement Date).
Source Endpoints:
• HP Non-Stop Source
• Open VMS RMS Source
• Hadoop Source
Target Endpoints:
• MapR Target
• MapR Streams Target
• MS APS PDW Target
• SAP Sybase IQ Target
• HP Vertica Target
• Actian Vector Target
• Netezza Target
• Pivotal Greenplum Target

Regards,


Shahzeb_Shafi
  • New Participant
  • May 19, 2022

Hi there,

Any ETL tool will also satisfy the request to get the transactional data into Vertica. Moreover, ETL tools will also provide the transformation phase where transactional data can also be aggregated and loaded into Vertica along with other kinds of transformations. Qlik replicate does not provide many transformations and as Steve said, Vertica is amongst the targets being retired.

Please check the below document to get one of the ETL tools understanding.

https://www.vertica.com/kb/Vertica-Integration-with-Informatica-Connection-Guide/Content/Partner/Vertica-Integration-with-Informatica-Connection-Guide.htm

https://docs.informatica.com/data-integration/powercenter/10-4-0/advanced-workflow-guide/incremental-aggregation/incremental-aggregation-overview.html

Regards
Shahzeb