Skip to main content
Question

Communal Storage Locations

  • April 1, 2020
  • 6 replies
  • 7 views

J_Kelley

Why support of more communal storage locations is not on your roadmap?
Migrating your Vertica Cluster to the Cloud
@SumeetAgrawal @Chris_Daly_HPE
Question from @Jan_Soubusta @JanSoubusta

6 replies

dsprogis
Forum|alt.badge.img
  • Participating Frequently
  • April 1, 2020

Supporting additional communal storage locations is definitely on our roadmap!
Eon currently runs on AWS, Pure Storage FlashBlade, and Minio.
With Version 10.0, Eon will run on HDFS and Google Cloud Platform (GCP).
You can expect to see Eon on Azure next.
We are working with Alibaba to get Eon running on OSS.
We are looking at CEPH which, like Minio, can be deployed on a large variety of hardware.
And we are looking at a number of hardware specific solutions as well.
Please post the platforms that you would be interested in seeing Eon run on.


Jan_Soubusta
Forum|alt.badge.img
  • Participating Frequently
  • April 1, 2020

Hi David ;-)
I thought the support for having two different communal storage location in the same database cluster.
Esp. I am interested in the scenario when recent data are in the different S3-like (high-end) storage then archive data (low-end cheap storage) and you can archive from the recent data.


jheffner
Forum|alt.badge.img
  • Participating Frequently
  • April 2, 2020

We may support a single database having multiple communal storage locations in the future, for reasons related to tiering and also due to capacity limitations of some shared storage systems. However, there is benefit in the simplicity of having all durable database storage in one place.

One method available today for archiving infrequently-accessed data is to export to Parquet or ORC format, which can be queried as an external table. While this requires some additional automation, and queries of archived data may not perform as well, it has the added benefit that metadata is also in cool storage and does not consume catalog memory in Vertica nodes.


Jan_Soubusta
Forum|alt.badge.img
  • Participating Frequently
  • April 3, 2020

Thanks for explanation, now I understand, why it is not on the roadmap - there is quite suitable workaround solution! ;-)
Just, please, consider this:
Swapping partition between two communal storages would be much less resource demanding than exporting to parquet.


jheffner
Forum|alt.badge.img
  • Participating Frequently
  • April 3, 2020

Definitely! Multiple, tiered storage locations is something we've supported for some time in Enterprise Edition, and is something on the list of possible future features for communal storage. (Another motivation is capacity limitations for some shared storage systems.)


SruthiA
Forum|alt.badge.img+1
  • Participating Frequently
  • August 23, 2024

This feature has been released and is available for use in 24.3

https://docs.vertica.com/24.3.x/en/new-features/24.3/eon-mode/