Skip to main content

Vertica Backup fails with rsync code 23 error.

  • September 6, 2019
  • 4 replies
  • 10 views
chandeeswarg
Hi, vertica version is 9.2.1-4. Our vbr.py process fails intermittently with the below error. so we are not able to run any backup/listbackup etc commands. but if we re-try multiple times, its working. can someone please help me on this ? error: /opt/vertica/bin/vbr.py --task listbackup --config-file test.ini Error: Error accessing remote storage: failed accessing remote storage on nrusca-clp44004: rsync: rename "/vertica_backup/bmdbix/..index.LUxCP4" (in vbr) -> ".index": No such file or directory (2) rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1042) [sender=3.0.7] returncode=23
Listbackup FAILED.

Ini file:
[Misc]
snapshotName = test_backup
restorePointLimit = 7
retryCount = 2
retryDelay = 180

[Database]
dbName = test
dbUser = vsd
[Transmission]
[Mapping]
v_test_node0001 = nrusca-clp44001:/vertica_backup/test
v_test_node0002 = nrusca-clp44002:/vertica_backup/test
v_test_node0003 = nrusca-clp44003:/vertica_backup/test
v_test_node0004 = nrusca-clp44004:/vertica_backup/test
v_test_node0005 = nrusca-clp44005:/vertica_backup/test
v_test_node0006 = nrusca-clp44006:/vertica_backup/test

    4 replies

    Jim_Knicely
    Forum|alt.badge.img+2
    • Jim_Knicely
    • Participating Frequently
    • September 9, 2019

    Hi,
    In your post it says "Error accessing remote storage: failed accessing remote storage on nrusca-clp44004". Is the issue always with that same remote server (nrusca-clp44004)?

    chandeeswarg
    • chandeeswarg
    • Author
    • New Participant
    • September 9, 2019

    Hi,
    Thank You for responding.
    The error keeps rotating with all the nodes and not just one node. Once its node4, then its node2, sometimes its node3. So, if i keep issuing the command "/opt/vertica/bin/vbr.py --task backup/listbackup --config-file test.ini", sometimes it fails with the error "rsync code 23" sometimes it works fine with no issues.
    Thank You.

    Bryan_H
    Forum|alt.badge.img+2
    • Bryan_H
    • Participating Frequently
    • September 9, 2019

    Hi, rsync error 23 is "Partial transfer due to error" This could mean some issue with network or the underlying storage hadrware.
    In vbr.ini, you can try increasing retries and timeout settings. Also check whether you have optimized the sysctl and ulimit settings as recommended.

    chandeeswarg
    • chandeeswarg
    • Author
    • New Participant
    • September 9, 2019

    @Bryan_H said:
    Hi, rsync error 23 is "Partial transfer due to error" This could mean some issue with network or the underlying storage hardware.
    In vbr.ini, you can try increasing retries and timeout settings. Also check whether you have optimized the sysctl and ulimit settings as recommended.

    Hi,
    The settings we have in our environment are as below, which is think is within the permissible values as per vertica documentation.
    vm.min_free_kbytes = 65032
    fs.file-max = 129024
    vm.max_map_count = 8257536

    id uid=580(vsd) gid=4021(verticadba) groups=4021(verticadba) ulimit -n 128956 ulimit -u 132112 and I've increase the retry parameter to 4 and and the retrydelay to 5 mins but it fails. /opt/vertica/bin/vbr.py --task listbackup --config-file test.ini --debug 3 Error: Error accessing remote storage: failed accessing remote storage on nrusca-clp44002: rsync: rename "/vertica_backup/bmdbix/..index.lDBrne" (in vbr) -> ".index": No such file or directory (2) rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1042) [sender=3.0.7] returncode=23
    Listbackup FAILED.

    Let alone the backup, the listbackup command fails.
    Thank You.

    Still scrolling?
    Ask a question
    Terms & ConditionsAccessibility statement