Skip to main content

Backup failed unexpectedly! ERROR

  • December 19, 2014
  • 9 replies
  • 27 views

Fredou
Hi,
My Vertica backups were working great untill i updated to the last version.
I use vbr.py but i get the message "backup failed unexpectedly!" every time i try to use it.
I even tried to do the --setupconfig but it doesn't help...

Does someone has an idea ?

Thanks 

9 replies

Raul_1
Forum|alt.badge.img
  • Participating Frequently
  • December 20, 2014

Can you post the contents of the created config file?

Fredou
  • Author
  • New Participant
  • December 20, 2014
Yes, here it is:

[Misc]snapshotName = fullBackup
restorePointLimit = 1

[Database]
dbName = xxxxxx
dbUser = yyyyyy

[Transmission]

[Mapping]
v_xxxxxx_node0001 = 127.0.0.1:/tmp/vbr

If i add the --debug 3 option i can see this error:

Traceback (most recent call last):  File "/opt/vertica/bin/vbr.py", line 2526, in backup
    prepareAll()
  File "/opt/vertica/bin/vbr.py", line 1888, in prepareAll
    configCheck()
  File "/opt/vertica/bin/vbr.py", line 506, in configCheck
    concurrency_upperboud = int(subprocess.Popen(['ulimit -u'], shell=True, stdout= subprocess.PIPE).communicate()[0].strip())
ValueError: invalid literal for int() with base 10: 'unlimited'
backup failed unexpectedly!

Thansk for your help



Raul_1
Forum|alt.badge.img
  • Participating Frequently
  • December 20, 2014
Is the OS Ubuntu?
Can you provide the output of ls -l /bin/sh

Fredou
  • Author
  • New Participant
  • December 20, 2014
It's Debian,

lrwxrwxrwx 1 root root 4 Dec 17 10:53 /bin/sh -> bash

Raul_1
Forum|alt.badge.img
  • Participating Frequently
  • December 20, 2014
Can you try running -
ulimit -u

Fredou
  • Author
  • New Participant
  • December 20, 2014
I get "unlimited"

id10t
Forum|alt.badge.img
  • Participating Frequently
  • December 21, 2014
Hi Fredou!


1. Can you post output for:
python -c "import json; print json.dumps('/opt/vertica/bin/validators.sh', indent=2)"
2. Modify limits.conf file, set max number of processes (lets say db super user is dbadmin):
dbadmin  -   nproc  65536
3. Validate settings:
$ ulimit -u
65536
do reboot and try backup.


PS
Try to validate it without reboot, set limit manually and if it do not hangs an error run backup.

*********

How to set limit manually?
ulimit -u 65536
if it do not throws an error so try to do a backup.

*********

Example of error:
daniel@synapse:/etc/security$ ulimit -u 131072
bash: ulimit: max user processes: cannot modify limit: Operation not permitted

Regards, idiot :)

Fredou
  • Author
  • New Participant
  • December 22, 2014
Thanks, the "ulimit -u 65536" did the trick !


id10t
Forum|alt.badge.img
  • Participating Frequently
  • December 23, 2014
Hi!

I will ask to run a first command from my post above one more time. You have to set a proper value(65536 - is an arbitrary value) for nproc (the key is max_user_proc)
daniel@synapse:~$ python -c "import json; print json.dumps('/opt/vertica/bin/validators.sh', indent=2)"
{
"total_memory": 7844,
"vertica": {
"release": "2",
"brand": "vertica",
"version": "7.1.1",
"arch": "iamd64"
},
"nics": [
{
"broadcast": "",
"ipaddr": "",
"netmask": "",
"speed": "unknown",
"name": "eth0"
},
{
"broadcast": "255.255.255.255",
"ipaddr": "127.0.0.1",
"netmask": "255.0.0.0",
"speed": "locallink",
"name": "lo"
},
{
"broadcast": "",
"ipaddr": "",
"netmask": "",
"speed": "unknown",
"name": "ppp0"
}
],
"hostname": "synapse",
"max_user_proc": "62582",
"cpu_info": {
"number_of_cpus": 8,
"cpu_type": " Intel(R) Core(TM) i7-3612QM CPU @ 2.10GHz"
}
}