Skip to main content
Question

Is there a way to adjust receiving buffer of COPY between nodes?

  • July 6, 2019
  • 1 reply
  • 5 views

Dingqiang
Forum|alt.badge.img+1

Hi team,

I have a prospect who is evaluating Vertica. But he met a performance issue when COPY data. There are low CPU/IO/network usages when 3 COPYs are loading data from local files, and no any bottleneck. No abnormal result from vioperf/vnetperf checking. All network bandwidth can be used when COPYs going to phase II. His statement looks like:

    COPY ... from
    'file1' on node1,
    'file2' on node2,
    'file3' on node3,
    -- 'file4' on node4,
    -- 'file5' on node5,
    -- 'file6' on node6,
    -- 'file7' on node7,
    'file8' on node8,
    'file9' on node9,
    'file10' on node10
    ... ;

The weird thing is, when he comments out about any 4~6 files/nodes in his COPY statements, IO/network usages can achive good higher numbers! It looks like there is some bottleneck when receiving data from other nodes.

Do you know whether there is a way to adjust receiving buffer of COPY between nodes?

Regards,
DQ

1 reply

Dingqiang
Forum|alt.badge.img+1
  • Author
  • Participating Frequently
  • July 8, 2019

After tried tuning TCP read/write memory, and DataBufferDepth/NetworkBufferSize of Vertica, but them help nothing.

Finally this prospect find out the truth, servers of Vertica connected to 2 switches, but the bandwidth between the 2 switches is limited.

But our vnetperf can not show evidence about this. Looks like vnetperf only testing pairs of nodes.

So this thread is closed.