Skip to main content
Question

Data extraction with ODBC benchmarks

  • July 13, 2021
  • 4 replies
  • 9 views

chaima
Forum|alt.badge.img+2
  • Participating Frequently

Hi Vertica team,

Our customer requested some benchmark of Vertica performance extracting data with odbc/jdbc. They use SAS and SPSS to retrieve large amounts of data through select statements and they're having performance degradations and trying to identify the root cause.
Network might be the culprit but they need some reference from Vertica on what should be the expected performance. For example for some test they run, they noticed that only 100mbps of the network were used. Does any one have any benchmark or performance tests that could help?
Many thanks,

Chaima

4 replies

siddyash
Forum|alt.badge.img
  • Participating Frequently
  • July 13, 2021

Hello Chaima,
We have not done any performance testing with the tools that you have listed. However, have a session will the customer will help our SME to understand the issue and trying to help them.
Thanks,
Jawahar.


chaima
Forum|alt.badge.img+2
  • Author
  • Participating Frequently
  • July 15, 2021

Hi @Hibiki , from SAS logs, for example:
Before:
Total row fetch seconds were:                     104.270299
Total SQL execution seconds were:                   2.833655
After:
Total row fetch seconds were:                     574.147720
Total SQL execution seconds were:                   2.746438


chaima
Forum|alt.badge.img+2
  • Author
  • Participating Frequently
  • July 15, 2021

The fetch time for this particular query went from 104seconds in February to 574 seconds in may, this applies to many of their queries that extract large amounts of data (for this example 9 columns, 30 million rows)


Thakur
Forum|alt.badge.img
  • Participating Frequently
  • July 29, 2021

hi @chaima the data you posted has the answers to the Question you are looking for.
Total SQL execution time has not changed, the only change is the change in buffers transfer and write I/O's this is outside of vertica.
ask the customer to push the output to /dev/null and record the timings.
did you check what is the buffer size set?

Before:
Total row fetch seconds were: 104.270299
Total SQL execution seconds were: 2.833655
After:
Total row fetch seconds were: 574.147720
Total SQL execution seconds were: 2.746438