Vertica catalog - memory issue
+2Hi , I am checking vertica as an database solution to a product we develop that will run on SAAS , when we design SAAS solution one of the basic test we run is to check how database behave to big catalogs (many tables) . Our test show that vertica probably load the entire catalog into memory as we see dramatic memory growth when we run out test 10K tables+ super projection. Btw – the memory growth was not show on the resource pools data dictionary tables , so checking them don’t present any issue . We look behave was present on the level of OS monitor tools that we run on top of vertica process . I would like to know if this us expect behave .
+2Hi Eli, This is correct -- Vertica currently keeps a full copy of its catalog in memory. As a result, it may not work well for cases where the catalog is extremely large. We do have customers running with many more than 10k tables, though. It depends heavily on the schemas of the tables, the number and type of projections, etc. (And, of course, on the amount of RAM available on each node.) If you're seeing memory usage that would make even 10k tables untenable on solid modern hardware, feel free to post a more-specific question or to check with Vertica Support. Adam
+2Hi thanks for the replay ,i understand that it related to the amount of projections ,but how it related to the amounts of schemas ? Can you elaborate on this ?
+2Hey Eli, No problem. Sorry, though, I was unclear: I didn't mean number of schemas created by CREATE SCHEMA; I meant the exact definition of each table, the table's columns, etc. (That definition is also sometimes referred to as the table's schema.) If you have, for example, lots and lots of columns, that's more metadata for the catalog to track. Adam
+2Hi Adam , Any idea why resource_pool_status table do not show any special memory allocation ? The general pool and all others pools show very minimal allocation comparing to what is going on the level of the DB server . Eli .
+2Hey Eli, Yes -- resource pools govern resources granted to queries (and to the WOS, as a special case), but the catalog isn't a query, so it's not managed by resource pools. Unfortunately, I don't know offhand where the best place to look would be, for catalog data structures. The system-table documentation might or might not have some suggestions for you. Adam
Sign up
Already have an account? Login
Welcome to the Rocket Forum!
Please log in or register:
Employee Login | Registration Member Login | RegistrationEnter your E-mail address. We'll send you an e-mail with instructions to reset your password.
77 4th Avenue
Waltham, MA 02451 USA
