Skip to main content

Timer thread backtrace

  • March 18, 2014
  • 3 replies
  • 21 views

Dave_Jarvis
Hello! 

I'm installing the Vertica Community Edition in an LXC container and I'll handily confess to suppressing a few errors in the install process to get to this point (notably /opt/vertica/share/binlib/read/get-disks has problems, and there's no 2GB swap partition). 

In spite of that, however, I seem to be able to run admintools -t createdb without too much difficulty, and it even stays up for a few minutes before failing. I dug into the error logs and this seems to be the root of my problems:
Backtrace Generated by Error  Signal: [0x0000000000000008] PID: [0x00000000000031bf] PC: [0x0000000002a23b98] FP: [0x00007f485a67fc20]SI_ADDR : [0x0000000002a23b98]  /opt/vertica/bin/vertica(_ZN6Basics9Backtrace11DoBacktraceEiiPvS1_+0x8cc)[0x33a011e]  /opt/vertica/bin/vertica(_ZN6Basics20GlobalSignalHandlers14logFatalSignalEiPvS1_+0xc7)[0x341f305]  /opt/vertica/bin/vertica[0x341f8f3]  /lib/x86_64-linux-gnu/libc.so.6(+0x364a0)[0x7f48f5ba34a0]  /opt/vertica/bin/vertica(_ZN3Mon29CheckForLowDiskSpaceTimerTask15runTaskInternalEb+0x478)[0x2a23b98]  /opt/vertica/bin/vertica(_ZN4Util16TimerServiceTask7runTaskEb+0x4b)[0x3581d33]  /opt/vertica/bin/vertica(_ZN4Util20TimerServiceTaskList18TaskSchedulingInfo10threadShimEPv+0x215)[0x3587aeb]  /opt/vertica/bin/vertica(_ZNK5boost9function0IvEclEv+0x1bb)[0x135f28b]  /opt/vertica/bin/vertica(_ZN7Session13ThreadManager12launchThreadERKN5boost9function0IvEE+0x57)[0x327229f]  /opt/vertica/bin/vertica(thread_proxy+0x80)[0x3a179c0]  /lib/x86_64-linux-gnu/libpthread.so.0(+0x7e9a)[0x7f48f5753e9a]  /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7f48f5c613fd]  END BACKTRACE  THREAD CONTEXT  Thread type: Timer Thread  Request: Unknown request  Transaction: [0x00a000000000008c]  END THREAD CONTEXT
I'm afraid I'm not clear on what exactly is going on here; can someone explain to me what's happening (and what I can do to resolve it?)

3 replies

Forum|alt.badge.img+2
  • Participating Frequently
  • March 18, 2014
Hi Dave,

Hm...  I'm honestly not sure what's going on here.

The stack trace in question is in code that periodically checks the free disk space in all storage locations, to make sure they have not overflowed.  The fact that "get-disks" had trouble is a warning sign to me.  That said, I'm not immediately sure how this would result in a crash.

Vertica is not often run (and not officially tested) inside LXC containers -- we're typically sliced by "fraction of a data center", not "fraction of a machine".  Different mentality.  That said, I know that we have been run successfully in containers elsewhere; there are cases where it's quite useful.  I would encourage you to tweak your container configuration; maybe see if you can get more of the installation checks to pass.

Adam


Forum|alt.badge.img+2
  • Participating Frequently
  • March 18, 2014
As a means of testing -- within the container, can you as a user see available disk space?, available # of inodes?, etc.

Dave_Jarvis
  • Author
  • New Participant
  • March 19, 2014
So, I ran into this thread yesterday, which seems to have a similar issue. Like them, if I df -i, I get a drive mounted with 0 inodes, etc...:
ubuntu@box82:~$ df -i Filesystem       Inodes IUsed    IFree IUse% Mounted on /dev/xvdh             0     0        0     - / cgroup         31438790    10 31438780    1% /sys/fs/cgroup none           31438790    68 31438722    1% /run none           31438790     1 31438789    1% /run/lock none           31438790     1 31438789    1% /run/shm
I've had someone suggest I could mount a filesystem within a file within the container and try to point Vertica at that; I'm aware that it's a pretty hacky solution and I'm not sure how happy Vertica would be about it.