How to configure Vertica EON mode for NetApp ONTAP

Hi,
Netapp ONTAP has not been validated with Vertica yet, so there is no specific info on whether it is compatible with Vertica,, nor is there anyt performance data to indicate if it performs well or not. And finally, there's no guide to setting up and configuring the ONTAP side for Vertica EON mode communal storage. We have had recent talks with Netapp about ONTAP and how it compares to StorageGrid, but to date they have not run the Vertica benchmark tests to validate is is api and performance compatible.
Regards,

Hi @s_crossman ,

Thank you for the response.

We did find this article on NetApp's web site regarding connecting to the ONTAP 9 s3 provider.
https://docs.netapp.com/us-en/ontap/s3-config/enable-client-access-from-s3-app-task.html

Maybe it will help?

Kind regards,

J.

Hi @s_crossman ,

You mentioned the Vertica benchmark test have not been run yet.
Is that a stand-alone tool that we can provide to our customer to test with to at least locally confirm that it will work?

Thank you,

J.

Hi,

Regarding the link to the ONTAP S3 article, it is helpful in the context of ONTAP but not for Vertica. Netapp had tried to run our api test on NTAP back in 2021 and it failed. They stated they have made improvements to the api but they haven't rerun the api test to confirm the original issues were resolved. Once that's confirmed then the benchmark testing has to be run to ensure that the apis work under a simulated customer environment with various data sizes and concurrent loads. The benchmark tests are run on a very specific Vertica hardware setup and the results are compared with those we captured in AWS using S3. The object storage numbers have ot meet or exceed the AWS numbers for a validation pass.

Regarding the test suite, Vertica works with object storage partners to validate their S3 object stores. We also build a technical relationship with them so once validation is complete we can put collateral together about specific nuances of the pairing, tuning info, best practices, observations, etc. We also establish a support relationship that allows us to properly support customers who implement the two products in their environment.

To date we have not sent the suite to any customers as the validation process requires experts on the Vertica and object storage sides to run, diagnose, tune, rerun, etc. The object storage partners know their s3 storage better than we do, and we know Vertica better than they do. So it takes a joint effort to get the validation environment setup, tests run, and validation completed. Not something that would work as well in a customer environment, especially if the two tech partners hadn't seen the combination together yet..

I would recommend contacting your Netapp account rep and let them know you have interest in running Vertica with Netapp ONTAP. The technical folks at Netapp have the latest test suite and docs and such. I believe they were holding off validation until they got some indication there was potential customer business to drive allocating resources to the validation.

Regards,

Thank you so much @s_crossman ! I appreciate the time and details. We will follow up with our customer to see if they can move this forward via their NetApp account rep.

No problem. I passed some info related to this to my contacts at Netapp so they are aware they may get a request from a customer. It might at least plant a seed to start planning out the validation testing.

So, quick update.

The customer is on NetApp ONTAP 9.11.1P7.
We were able to configure the S3 storage provider with certs.

We successfully tested using a couple of utilities on the same server we installed Vertica (12.1) on (s3cmd and mc).

For example, the MinIO mc command can fully connect and authenticate and add and remove files from the target bucket.

But, when we are running create_db, we can see Vertica was able to connect to the ONTAP S3 endpoint and actually create the test file in the target bucket, but is unable to then read it (gets zero bytes, vs the 44 bytes of data in the file) and the run ends.

And, we did ask them if they can try to get their NetApp rep to reach out to all y'all for help too. :-)
But, from their perspective, it is now something on the Vertica side.
(And yes, we know, it's not an officially supported platform.)

Still, any clues or thoughts on where we might look would be appreciated.

Hi,
My guess is that the failure is during the bootstrap routine of create_db where it writes a test file and then immediately reads it back for size info to confirm it wrote correctly. The bootstrap log file may have more details although it may not show a failure, just that it successfully read 0 bytes.

We saw this in one other storage partner's object storage. They had to fix their consistency level. Vertica expects immediate vs eventual consistency. If the s3 object store takes too long to complete the object write an immediate GET after PUT acts like the file doesn’t exist or it's 0 bytes.

Here's a link that covers AWS S3 consistency model, which Vertica requires.
https://aws.amazon.com/s3/consistency/

Because we haven't qualified ONTAP I'm not sure if they follow the S3 consistency model or not.

Just checking back to see if that script I sent was used and if so whether it helped confirm this might bean eventual vs strong consistency issue. My Netapp contact had mentioned that the S3 api support in Ontap has evolved quite a bit over the last year or so, and that the Ontap version and platform might be important to narrow down which level of compatibility they have.

Hi @s_crossman ,

Thank you for providing the script and checking back in.
We provided the link to this thread, and they said they shared it with their NetApp rep.
They confirmed that they could not move off of ONTAP (to StorageGrid) as they are using features that are not available in the latter.
So, hopefully, NetApp will be reaching out and working with you and Vertica to get ONTAP's S3 stack certified to use with V12.
Did your NeApp contact mention which specific version of ONTAP they felt should be compatible?

• J.

Hello @fatimachoudary1, as per the information shared by @s_crossman, NetApp ONTAP has not been validated with Vertica yet. Therefore, we don't have any information regarding the performance or Functional compatibility of it with Vertica.

Hello @s_crossman @VikasGarg
We also have 1 if not multiple customers who are requesting ONTAP S3 validation. While the StorageGrid validation is great, it isn't always the best fit for every configuration. I will be asking our account team to add us to the list of customers who are asking for ONTAP certification. Thank you. S.

Hello @s_cipresse , Please have account team contact me through email (Jawahar A). Thanks.