Rocket U2 | UniVerse & UniData

 View Only
Expand all | Collapse all

How to determine when connection pools are overly busy

  • 1.  How to determine when connection pools are overly busy

    Posted 05-12-2022 08:35
    We utilize 4 connection pools used by our Web Service calls.  Occasionally we have had web services timing out and not even reaching our U2 programs.  I was hoping for some ideas on how we might  check the connection pools to determine when they are overly busy, or calls are backing up/timing out.  We have not found a way to determine this.  Our web team contacts us when their calls are failing and we must dig around looking for a problem. 

    Hoping there might be some "tools" that would specifically target the health of the connection pools.

    ------------------------------
    Nelson Schroth
    president
    C3CompleteShop LLC
    Harrison OH US
    ------------------------------


  • 2.  RE: How to determine when connection pools are overly busy

    Posted 05-18-2022 20:06
    Hi Nelson,

    Did you get any responses to this post? This is also something that we would be interest in.

    Regards,
    Barry

    ------------------------------
    Barry Green
    Director
    CAPITOL BUSINESS SERVICES
    Mawson Lakes SA AU
    ------------------------------



  • 3.  RE: How to determine when connection pools are overly busy

    Posted 05-18-2022 20:43
    Sorry Barry - nothing but crickets!

    ------------------------------
    Nelson Schroth
    president
    C3CompleteShop LLC
    Harrison OH US
    ------------------------------



  • 4.  RE: How to determine when connection pools are overly busy

    Posted 05-19-2022 04:51
    Hi Nelson,

    That's disappointing.

    @Rocket Software have you no comments or suggestions for this?





    ------------------------------
    Barry Green
    Director
    CAPITOL BUSINESS SERVICES
    Mawson Lakes SA AU
    ------------------------------



  • 5.  RE: How to determine when connection pools are overly busy

    ROCKETEER
    Posted 05-19-2022 21:59
    Edited by David Andrews 05-19-2022 22:01
    Sorry for dropping the ball on this one guys, but I thank @Neil Tiver for jumping in. We have a great customer story about how MVIS  really allowed them to scale effectively: https://www.rocketsoftware.com/customer-case-studies/mainstream-new-zealand

    Let me know if I can provide further info.

    ​All the best,
    Dave

    ------------------------------
    David Andrews
    Head of Customer Advocacy
    Rocket Software
    South Salem NY US
    ------------------------------



  • 6.  RE: How to determine when connection pools are overly busy

    Posted 05-21-2022 16:22
    So, the bottom line is that there is nothing in U2 to provide any insight into CPs.  Upgrading to MVIS seems like a lot of changes to get to something that I had hoped would be available with a lot less effort.
    Disappointing.

    ------------------------------
    Nelson Schroth
    president
    C3CompleteShop LLC
    Harrison OH US
    ------------------------------



  • 7.  RE: How to determine when connection pools are overly busy

    Posted 05-23-2022 19:34
    Hi Dave and Rocket,

    My MVIS $0.02 worth.

    I think MVIS is a fantastic tool but in our environment, we need end to end logging of online customer activity.

    Maybe I've missed something here but I have yet to find a suitable MVIS logging solution for our needs. To get anything close to useful we need to log everything (trace level) set but then there is so much information it becomes nigh impossible to find the information required. While this is sort of OK in a development environment, it is far from acceptable in a production environment where much lower "info" level of logging is desirable to both reduce the sheer volume of log data and also optimise server performance.

    This lower-level logging does not provide sufficient resource information such as the account/end point names and token information that can be used to identify activity from customer browser through to the UniVerse account being accessed. This may not be as big an issue in the white paper FMS example but I believe it is critical in financial transactions, particularly when investigating historical issues, enquiries, and complaints that can extend back in time for months and even years in some cases.

    It would be really good I think to be able to nominate certain key fields per end point, and whether these fields can be clear text or masked. Perhaps in a "log group" and be able to allocate the "group" to a particular level of MVIS logging such as "info" but able to choose error, warn, info, debug, trace as required.

    Any thoughts or recommendations?

    Cheers,
    Peter
    ------------------------------------------------------------------------------- Note: This email (inc all attachments) is for the use of the intended recipient(s) only. Privileged or confidential information may be contained in this communication. If you have received this email in error, please notify the sender immediately and then delete all copies of this message from your computer network. If you are not the intended recipient, you must not keep, use, disclose, copy or distribute this email without the author's prior permission. If you are the intended recipient and you do not wish to receive similar electronic messages from us in future, then please respond to the sender to this effect. We have taken precautions to minimise the risk of transmitting software viruses, but advise you to carry out your own virus checks on this email and its attachments. We do not accept liability for any loss or damage caused by software viruses and do not represent that this transmission is free from viruses or other defects. Firstmac Limited (ABN 59 094 145 963) (AFSL 290600)




  • 8.  RE: How to determine when connection pools are overly busy

    Posted 27 days ago
      |   view attached
    Peter, I agree with you that the logging in MVIS is very lacking at the moment. The good news is that the next maintenance release 1.3.x, expected at the end of June, will include updated logs with the account name. See attached.

    There's also development request MVIS-3348 to include a requestId/transactionId in each log so you'll be able to link mvis logs with the rest of the logs generated by your stack.

    ------------------------------
    Kenneth Thompson
    Application Developer
    Storis Inc
    Mount Arlington NJ US
    ------------------------------



  • 9.  RE: How to determine when connection pools are overly busy

    ROCKETEER
    Posted 26 days ago
    Just to clarify MVIS-3348 is not part of the MVIS 1.3.1 release

    ------------------------------
    Stuart MacKenzie
    Rocket Software Inc
    CO US
    ------------------------------



  • 10.  RE: How to determine when connection pools are overly busy

    Posted 05-24-2022 10:19
    If you're not using it already you may want to look into F5 load balancing.   It's something you'll need if heavy traffic is anticipated.

    Karl

    ------------------------------
    Karl Pozmann
    Sr. Analyst, Developer, LTC Systems
    Omnicare Inc
    Dublin OH US
    ------------------------------



  • 11.  RE: How to determine when connection pools are overly busy

    Posted 05-19-2022 19:00

    Hi Nelson,

    I like your "tools" suggestion and think it'd be a fantastic addition.

    In a *nix environment there are tools such as topas, nmon and vmstat utilities that indicate how much resources a process is using and could be an assist in determining "load". So in short, your system admin team may be able to assist here?

    We also accasionally have similar issues and find that restarting unirpcd and killing off hung child processes usually fixes it. We're running AIX 7.2 with an old unsupported uv10.3.4 therefore no chance of us getting some additional UV "toolkit". So we resort to operating system monitoring tools and utilities to get us through until we can (one day) upgrade to a supported UV version.

    Regards,
    Peter



    ------------------------------
    Peter Cheney
    Developer
    Rocket Forum Shared Account
    BRISBANE QLD AU
    ------------------------------



  • 12.  RE: How to determine when connection pools are overly busy

    PARTNER
    Posted 05-19-2022 21:24
    Hi Nelson.

    Something which could really benefit you in this scenario, would be the MultiValue Integration Server (MVIS) from Rocket:

    https://www.rocketsoftware.com/products/rocket-multivalue-integration-server

    This is specifically designed to give you greater transparency on what your Connection Pools are doing, and help diagnose issues associated with incoming connections; slow response, unavailable connections, poor UX, license issues etc.  It also improves flexibility in using the connections, better APIs, Swagger definitions, and a whole lot more.

    Definitely worth investigating :-)

    @Barry - we will reach out to you to discuss this separately from the forum :-)

    Neil​​

    ------------------------------
    Neil Tiver
    BDM
    Meier Business Systems PTY LTD
    Carnegie VIC AU
    ------------------------------



  • 13.  RE: How to determine when connection pools are overly busy

    ROCKETEER
    Posted 6 days ago
    A couple of things on this thread:
    1. MVIS-3348 to include a requestId/transactionId in each log. - Has been accepted for inclusion in MVIS-1.3.2 towards the end of the year
    2. The engineering team are currently starting an investigation on how we can best monitor connection pools and what information would be most valuable to customers. The results of this investigation will be included in a future quarterly release of Rocket's MV Experiences product.
    MV Experiences is a free product for customers on maintenance and is currently available for UniVerse 11.3.4.
    It should be available for UniVerse 12 later this year followed by UniData.

    ------------------------------
    Stuart MacKenzie
    Rocket Software Inc
    CO US
    ------------------------------



  • 14.  RE: How to determine when connection pools are overly busy

    Posted 6 days ago
    Hi Nelson, I've only just started playing with UniVerse (via uopy) and found the following strategy helpful.  Not using connecting pooling but the principles would be the same:

    When I construct a session, there are a couple of attributes on it that are helpful:  there is a "health_check()" function on the session which tries to determine whether it's useful or not, an "is_active" flag to indicate whether it's capable of doing something.  I would assume these are also in the java/.NET versions of the product.

    I've also been using a function on the session "start_idle()".  I found in my web service that the connection would drop silently overnight, so most of the time I look at the idle time on the session and if it's been idle more than an hour, I reconstruct it.  The session construction is done via a kind of lazy initialisation so getting a working session does all the health checks and whatnot before passing a working session back to the bit of the program that needs it.

    From my very limited experience with this so far, the strategy of assuming the session is dead unless proven otherwise makes for a more reliable web service in my case.

    cheers,
    dave.

    ------------------------------
    David Rubie
    Integration Specialist
    Rocket Forum Shared Account
    ------------------------------