This article explains why a Vertica query spilled a large amount of data to temp space—up to 495 GB on the busiest node. The behavior is expected when hash tables used by the GroupByHash operator exceed available memory, especially in the presence of data skew.
Environment
- Vertica Database
- Distributed query execution
- Temp space usage
- Resource pool monitoring
Situation
A query executed in Vertica spilled up to 495 GB of data to temp space on the busiest node. The query ran unevenly across nodes, with some finishing faster than others. The resource_pool_status showed different numbers of running queries per node.
