Skip to main content
Question

Vertica: Cannot allocate sufficient memory for copy statement

  • January 7, 2021
  • 8 replies
  • 10 views

PavanVejju
Forum|alt.badge.img+1

While using copy command in vertica, getting below error.
ERROR : Exception while merge- [Vertica]VJDBC ERROR: Cannot allocate sufficient memory for COPY statement (536870912 requested, 268435456 permitted)

8 replies

SruthiA
Forum|alt.badge.img+1
  • Participating Frequently
  • January 7, 2021

it looks like your resource pool where query is getting executed is not having enough resources. please allocate more resources. please review the below link which can help you in fine tuning the resource pools.

https://www.vertica.com/kb/BestPracticesforManagingResourcePools/Content/BestPractices/BestPracticesforManagingResourcePools.htm


PavanVejju
Forum|alt.badge.img+1
  • Author
  • Participating Frequently
  • January 7, 2021

Thank you for replying
These are the details of my pool
dbadmin=> SELECT name, memorysize, maxmemorysize, priority, maxconcurrency FROM V_CATALOG.RESOURCE_POOLS;
name | memorysize | maxmemorysize | priority | maxconcurrency
----------+------------+---------------+----------+----------------
general | | Special: 95% | 0 |
sysquery | 1G | | 110 |
sysdata | 0M | 0M | |
wosdata | 0% | 2G | |
tm | 4G | | 105 | 7
refresh | 0% | | -10 |
recovery | 0% | | 107 | 7
dbd | 0% | | 0 |
jvm | 0% | 2G | 0 |
blobdata | 0% | 10% | |
metadata | 0% | | |
(11 rows)

Can you please let me know where exactly need to update my configurations


SruthiA
Forum|alt.badge.img+1
  • Participating Frequently
  • January 7, 2021

can you please share me the output of the following

select * from host_resources;


PavanVejju
Forum|alt.badge.img+1
  • Author
  • Participating Frequently
  • January 8, 2021

select * from host_resources;
host_name | open_files_limit | threads_limit | core_file_limit_max_size_bytes | processor_count | processor_core_count | processor_description | opened_file_count | opened_socket_count | opened_nonfile_nonsocket_count | total_memory_bytes | total_memory_free_bytes | total_buffer_memory_bytes | total_memory_cache_bytes | total_swap_memory_bytes | total_swap_memory_free_bytes | disk_space_free_mb | disk_space_used_mb | disk_space_total_mb | system_open_files | system_max_files
----------------+------------------+---------------+--------------------------------+-----------------+----------------------+----------------------------------------+-------------------+---------------------+--------------------------------+--------------------+-------------------------+---------------------------+--------------------------+-------------------------+------------------------------+--------------------+--------------------+---------------------+-------------------+------------------
98.218.112.345 | 65536 | 256876 | 0 | 1 | 12 | Intel(R) Xeon(R) E-2176G CPU @ 3.70GHz | 7 | 6 | 6 | 67401281536 | 1682550784 | 145334272 | 60176822272 | 34325131264 | 34218962944 | 588732 | 278673 | 867405 | 1120 | 6560404
(1 row)


Jim_Knicely
Forum|alt.badge.img+2
  • Participating Frequently
  • January 8, 2021

@PavanVejju: Can you post the results of this query?

SELECT memory_size_kb, memory_size_actual_kb, memory_inuse_kb, queueing_threshold_kb, max_memory_size_kb, planned_concurrency, max_concurrency, query_budget_kb FROM resource_pool_status WHERE pool_name = 'general';


PavanVejju
Forum|alt.badge.img+1
  • Author
  • Participating Frequently
  • January 10, 2021

dbadmin=> SELECT memory_size_kb, memory_size_actual_kb, memory_inuse_kb, queueing_threshold_kb, max_memory_size_kb, planned_concurrency, max_concurrency, query_budget_kb FROM resource_pool_status WHERE pool_name = 'general';
memory_size_kb | memory_size_actual_kb | memory_inuse_kb | queueing_threshold_kb | max_memory_size_kb | planned_concurrency | max_concurrency | query_budget_kb
----------------+-----------------------+-----------------+-----------------------+--------------------+---------------------+-----------------+-----------------
55219873 | 55219873 | 0 | 52458876 | 55219873 | 12 | | 4371573
(1 row)


PavanVejju
Forum|alt.badge.img+1
  • Author
  • Participating Frequently
  • January 11, 2021

Can anyone help me the same, I am facing this issue randomly


SruthiA
Forum|alt.badge.img+1
  • Participating Frequently
  • January 11, 2021

@PavanVejju : Please open a support case and provide a scrutinize for review. so that we can analyze logs and provide recommendations accordingly.