Skip to main content

Why does server memory footprint never reduce even if memory is released?

  • February 16, 2013
  • 0 replies
  • 0 views

Problem:

  • Product Name: VisiBroker C
  • Product Version: All
  • Product Component: Memory Allocation
  • Platform/OS Version: All

When running a test activating 1000's of servants and then deactivating and releasing them the Task Manager tool shows that the memory footprint for the server application increases but never shrinks even though a trace shows that the memory was released.

Resolution:

The best answer that can be given is that this is not related to any VisiBroker behavior but rather is a side effect of the way memory is allocated and deallocated by the C libraries.

Most operating systems and libraries allocate memory in blocks/pages which the library code requests from the OS. Then, when the memory is freed it is usually returned to a free list for the process (for reuse) since this is more efficient and the task of returning the memory to the OS is relatively expensive, especially if it would be requested again later. Sometimes the page will be returned if the allocation/deallocation is well behaved. However in general the heap for the process will become fragmented and since only complete pages at the end of the address space can be returned to the OS, the memory is often held by the process in its free list.
In addition growing and shrinking a process is an extremely expensive processes, whereas adding and removing memory from a free list is very cheap. Therefore, in cases where memory use swings wildly, it is better to spend the time and do the shrinking, but in cases where the memory use holds steadily, it is better to use the freelist and skip shrinking the process. The problem is that the memory allocator cannot know which situation it is facing, and therefore cannot perform this task optimally. Most malloc implementations do not shrink the process.

The use of the STL map would make this scenario likely since the order the servants is created and stored in map is different from the order it is released leading to fragmentation of the heap (the std::map uses a hashing technique for lookup which reorders the elements in the map).

On the question of this behavior indicating a memory leak:
The fact that running the test case client repeatedly asking for the same number of objects without the overall footprint increasing beyond the initial max value can be taken to indicate that there is no memory leak in the test case and that the server process is probably holding onto that memory as described above.

POA based servant managers (activators, locators and defaults servants) were intended to address the issues of large numbers of servants and the associated memory requirements. The VisiBroker documentation has information on these approaches and examples can be found in the "poa" directory of each approach.


#Security
#Performance
#c
#C
#VisiBroker
#memorymanagement
#Memoryleak