UniVerse 11.2.5 / RHEL7 : How to change the execution priority of UniRPC uvcs & uvserver connections

We run UV 11.2.5 on RHEL7 servers, and use UniRPC connections a lot.

I have been looking for a way to automatically apply reduced priority to some UniRPC connections based on the connecting user id.

I have tried creating a new 'conf' file in /etc/security/limits.d.
These files get applied by the PAM limits module for new login connections, but when the UniRPC daemon validates the credentials of the in-coming connection it appears to not be running through the PAM modules so when the uvcs / defcs / uvserver session is launched using the supplied user credentials the resulting PID is running with the default priority.

Does anyone know of a way to automatically have the launched uvcs / defcs / uvserver process use a non-default priority?

I'm not sure if it's available in 11.2.5, but you could use a globally cataloged UOLOGIN to perform a CHAP DOWN or something like that based on the value of @LOGNAME.​

We run UV 11.2.5 on RHEL7 servers, and use UniRPC connections a lot.

I have been looking for a way to automatically apply reduced priority to some UniRPC connections based on the connecting user id.

I have tried creating a new 'conf' file in /etc/security/limits.d.
These files get applied by the PAM limits module for new login connections, but when the UniRPC daemon validates the credentials of the in-coming connection it appears to not be running through the PAM modules so when the uvcs / defcs / uvserver session is launched using the supplied user credentials the resulting PID is running with the default priority.

Does anyone know of a way to automatically have the launched uvcs / defcs / uvserver process use a non-default priority?

For more granularity, you could try using UOLOGIN as suggested by Tyrel and get the PID of the current process. Once known then execute a Linux shell command to renice - which is the equivalent of CHAP but more flexible- e.g.

>AE BP TEST
Top of "TEST" in "BP", 3 lines, 72 characters.
*--: P
0001 declare GCI getpid
0002 MYPID=getpid()
0003 EXECUTE 'SH -c "renice +20 ':MYPID:'"'
*--: FIBR
Filed "TEST" in file "BP".
Compiling: Source = 'BP/TEST', Object = 'BP.O/TEST'


Compilation Complete.
2023: old priority 0, new priority 19
>

We run UV 11.2.5 on RHEL7 servers, and use UniRPC connections a lot.

I have been looking for a way to automatically apply reduced priority to some UniRPC connections based on the connecting user id.

I have tried creating a new 'conf' file in /etc/security/limits.d.
These files get applied by the PAM limits module for new login connections, but when the UniRPC daemon validates the credentials of the in-coming connection it appears to not be running through the PAM modules so when the uvcs / defcs / uvserver session is launched using the supplied user credentials the resulting PID is running with the default priority.

Does anyone know of a way to automatically have the launched uvcs / defcs / uvserver process use a non-default priority?

Thank John and Tyrel for the suggestion on using UOLOGIN.
I have been looking for an os-level option that would cover the ODBC connections (as the ODBCLOGIN subroutine support is not present in 11.2.5) as well as the UO connections.

That is why I dabbled with the unix limits functionality. It seems that when the unirpcd  daemon validates the connection and the requested service it does something slightly different to launch the resulting child PID as the authenticated user. This is the point at which I am keen to apply the changed priority. I just cannot find the appropriate OS level setup to make it happen.

We run UV 11.2.5 on RHEL7 servers, and use UniRPC connections a lot.

I have been looking for a way to automatically apply reduced priority to some UniRPC connections based on the connecting user id.

I have tried creating a new 'conf' file in /etc/security/limits.d.
These files get applied by the PAM limits module for new login connections, but when the UniRPC daemon validates the credentials of the in-coming connection it appears to not be running through the PAM modules so when the uvcs / defcs / uvserver session is launched using the supplied user credentials the resulting PID is running with the default priority.

Does anyone know of a way to automatically have the launched uvcs / defcs / uvserver process use a non-default priority?

Gregor,

The way the daemons launch the child processes still uses standard O/S user authentication methods - and hence PAM if configured - but is somewhat different to a user login. It uses an O/S API that impersonates the user credentials provided when the child process is launched.

I've done a little experimentation - please try using 'renice' on the UniRPC daemon service once launched. That should affect the relative priority of any slave processes that it launches as well. If this works for you, the you could add a post-startup set of script commands at the bottom of the uv.rc script (just below the lines that allow for customisation), Alternatively you could create a new start up script that runs as a lower priority after UniVerse has started and looks for the deamon process and uses renice as a separate task.

As a caution, if you are using any RPC-based services that might rely upon performance for throughout such as U2 Replication under significant load,  then these too are likely to be affected by any priority change to the launching process.

We run UV 11.2.5 on RHEL7 servers, and use UniRPC connections a lot.

I have been looking for a way to automatically apply reduced priority to some UniRPC connections based on the connecting user id.

I have tried creating a new 'conf' file in /etc/security/limits.d.
These files get applied by the PAM limits module for new login connections, but when the UniRPC daemon validates the credentials of the in-coming connection it appears to not be running through the PAM modules so when the uvcs / defcs / uvserver session is launched using the supplied user credentials the resulting PID is running with the default priority.

Does anyone know of a way to automatically have the launched uvcs / defcs / uvserver process use a non-default priority?

Thanks John.

We are already using the 'renice' approach, and are limiting the impact to non-system user connections to our application.
As this does involve having to open files and read records to determine if the 'renice' is needed, and this does only work for the 'uvcs/defcs' connections to the UniRPC daemon, I have been keen to explore the potential for possibly a systemd-related setup that reduces the impact on the uvcs/defcs connections.

I guess I will have to keep looking...

We run UV 11.2.5 on RHEL7 servers, and use UniRPC connections a lot.

I have been looking for a way to automatically apply reduced priority to some UniRPC connections based on the connecting user id.

I have tried creating a new 'conf' file in /etc/security/limits.d.
These files get applied by the PAM limits module for new login connections, but when the UniRPC daemon validates the credentials of the in-coming connection it appears to not be running through the PAM modules so when the uvcs / defcs / uvserver session is launched using the supplied user credentials the resulting PID is running with the default priority.

Does anyone know of a way to automatically have the launched uvcs / defcs / uvserver process use a non-default priority?

Gregor,

There was a change at 12.1.1 that might help:

UNV-24767UNIX only. Starting in UniVerse 12.1.1, the UniRPC daemon can be started from the command line with a port number argument. By default, unirpcd will listen on port 31438. The new capability allows the port number to be dynamically set when starting unirpcd. For example, if you enter unirpcd -p1234, the unirpcd process that was just started will listen on port 1234.

My thought was that by modifying /etc/services to add additional ports and firing off multiple unirpcd named services on different ports with different 'nice' values this could be achievable.

e.g.
Duplicate the RPC port entry in /etc/services, making the new port number 31439
Add a new line to unirpcservices for the new service with a new name - e.g. uvcslow
Start up an additional RPC service deamon for the new low-priority service - nohup nice -n15 "$unishared"/unirpc/unirpcd -p31439> /dev/null 2>&1 &

The idea is that you then give different users different named services to which to connect (uvcslow in this case), and the service name used determines the process priority of the daemon used, and hence its child processes.

While I have not had an opportunity to try this myself (at least - yet) it seems feasible. For test purposes you could fire up the new PRC daemon(s) manually and prove - or disprove the concept. For production use there would need to be some significant changes to the uv.rc script in the area of RPC deamon management as at the moment it only caters for the pssibility of one daemon that would need to be stopped or being started.

Caution - while this seems feasible I have not found an opportunity to test whether this would give what you are looking for. It relies upon the  'nice' relationship of parent and child daemon processes and some finagling is likely required..

We run UV 11.2.5 on RHEL7 servers, and use UniRPC connections a lot.

I have been looking for a way to automatically apply reduced priority to some UniRPC connections based on the connecting user id.

I have tried creating a new 'conf' file in /etc/security/limits.d.
These files get applied by the PAM limits module for new login connections, but when the UniRPC daemon validates the credentials of the in-coming connection it appears to not be running through the PAM modules so when the uvcs / defcs / uvserver session is launched using the supplied user credentials the resulting PID is running with the default priority.

Does anyone know of a way to automatically have the launched uvcs / defcs / uvserver process use a non-default priority?

@Tyrel Marak - 11.2.5 does have the *UOLOGIN ​support, and we do use it to adjust the nice level of the pid.

When I explored the CHAP DOWN command I found on our RHEL 7+ 11.2.5 setup that is applied a "nice" value of 19!

A much too sever reduction in the PID's priority than we wanted, so we are persisting with the unix renice command for now.

We run UV 11.2.5 on RHEL7 servers, and use UniRPC connections a lot.

I have been looking for a way to automatically apply reduced priority to some UniRPC connections based on the connecting user id.

I have tried creating a new 'conf' file in /etc/security/limits.d.
These files get applied by the PAM limits module for new login connections, but when the UniRPC daemon validates the credentials of the in-coming connection it appears to not be running through the PAM modules so when the uvcs / defcs / uvserver session is launched using the supplied user credentials the resulting PID is running with the default priority.

Does anyone know of a way to automatically have the launched uvcs / defcs / uvserver process use a non-default priority?

Interesting thought @John Jenkins

Whilst it might achieve the priority separation I think the additional admin load needed for startup & shutdown makes this difficult to pursue.
I can imagine a facility that allowed us to configure a "conf" file and run a command/script to establish the environment and manage the startup and shutdown would make this option really attractive right now, but alas such a beautiful thing does not yet exist.

Perhaps my tinkering in this space might head down that path.​

We run UV 11.2.5 on RHEL7 servers, and use UniRPC connections a lot.

I have been looking for a way to automatically apply reduced priority to some UniRPC connections based on the connecting user id.

I have tried creating a new 'conf' file in /etc/security/limits.d.
These files get applied by the PAM limits module for new login connections, but when the UniRPC daemon validates the credentials of the in-coming connection it appears to not be running through the PAM modules so when the uvcs / defcs / uvserver session is launched using the supplied user credentials the resulting PID is running with the default priority.

Does anyone know of a way to automatically have the launched uvcs / defcs / uvserver process use a non-default priority?

Have you taken a look at RHEL7 Control Groups (CGROUPS) it may be the answer to your question?

KR
Mark

We run UV 11.2.5 on RHEL7 servers, and use UniRPC connections a lot.

I have been looking for a way to automatically apply reduced priority to some UniRPC connections based on the connecting user id.

I have tried creating a new 'conf' file in /etc/security/limits.d.
These files get applied by the PAM limits module for new login connections, but when the UniRPC daemon validates the credentials of the in-coming connection it appears to not be running through the PAM modules so when the uvcs / defcs / uvserver session is launched using the supplied user credentials the resulting PID is running with the default priority.

Does anyone know of a way to automatically have the launched uvcs / defcs / uvserver process use a non-default priority?

@Mark Copp​, 

I have had a look around at the control groups.
So far I have not found a way to dynamically set the lower priority using it, and I think that I am not alone in this.

https://utcc.utoronto.ca/~cks/space/blog/linux/SystemdUserAndSystemConf
https://utcc.utoronto.ca/~cks/space/blog/linux/SystemdDynamicUserLimits
https://utcc.utoronto.ca/~cks/space/blog/linux/SystemdFairshareScheduling

And even if is would work, the sudden change in scheduling cascading up the unit chain introduces other concerns that will require extensive testing.

The other complexity is that the unirpc daemon does typically get started with the universe service so it is part of the universe.service control group.
If the unirpc daemon is stopped and started manually it will no longer form part of the universe.service and the control grouping is lost.
Now if the unirpc daemon was it's own systemd service there might well be better options to pursue.

We run UV 11.2.5 on RHEL7 servers, and use UniRPC connections a lot.

I have been looking for a way to automatically apply reduced priority to some UniRPC connections based on the connecting user id.

I have tried creating a new 'conf' file in /etc/security/limits.d.
These files get applied by the PAM limits module for new login connections, but when the UniRPC daemon validates the credentials of the in-coming connection it appears to not be running through the PAM modules so when the uvcs / defcs / uvserver session is launched using the supplied user credentials the resulting PID is running with the default priority.

Does anyone know of a way to automatically have the launched uvcs / defcs / uvserver process use a non-default priority?

Hi Gregor,

I wasn't able to find a way to do this via OS-configuration alone.  But here's an idea similar to the UOLOGIN one, only it's done outside of BASIC. 

You could put a shim in for uvapi_slave.  The shim would read the user's priority level in /etc/security/limits.conf, and then exec the true uvapi_slave with the priority it found.

The steps and code would be something like the following

1. Move $UVHOME/uvapi_slave to $UVHOME/uvapi_slave_real
2. Create $UVHOME/uvapi_slave_shim.sh as below; use the same permissions and ownership as $UVHOME/uvapi_slave
3. Create a new symlink from $UVHOME/uvapi_slave to $UVHOME/uvapi_slave_shim.sh

#!/bin/bash

#
# uvapi_slave_shim.sh
#

# find the priority for this user in limits.conf
# (whole-word matches only, and only at the beginning of a line)
priority=$(grep -w "^$(whoami)" /etc/security/limits.conf | grep priority | awk {'print $4'})

# see if we have a valid priority (between -20 and 19 inclusive)
if [[ $priority =~ ^[0-9]+$ ]] && ((-20 <= priority <= 19)); then
  # valid priority level, so exec with nice
  exec nice -n$priority ./uvapi_slave_real $@
else
  # invalid priority level, fallback to plain execution
  exec ./uvapi_slave_real $@
fi

We run UV 11.2.5 on RHEL7 servers, and use UniRPC connections a lot.

I have been looking for a way to automatically apply reduced priority to some UniRPC connections based on the connecting user id.

I have tried creating a new 'conf' file in /etc/security/limits.d.
These files get applied by the PAM limits module for new login connections, but when the UniRPC daemon validates the credentials of the in-coming connection it appears to not be running through the PAM modules so when the uvcs / defcs / uvserver session is launched using the supplied user credentials the resulting PID is running with the default priority.

Does anyone know of a way to automatically have the launched uvcs / defcs / uvserver process use a non-default priority?

Thanks @Steven Wingfield
This is a great idea.

I had not thought about intercepting the uvapi_slave binary as I did not realise it was executed.
Now I know I think I can become very adventurous!!​