Rocket OpenQM

We are running OpenQM 4.0.8 on W2019 in Azure.

We are running an Apache web server on a private separate server outside of Azure.

we are having what we consider slow and variable response times. we have added a considerable amount of tracking but are unable to find the bottleneck.

we are use PHP page to call a function that gets data from the QM server. we are using a http call

from the PHP function we call:    (we have 20+ of these calls for different information), for instance:

http://web.xyz.com:8280/api/getpurp?fnme="obj.maint"

we know that the pageload (and function call) is taking 1.723 seconds. We have tracked the QM side at 8ms.

we have looked at the flush time in QM   and tried WFLUSH to make sure that buffering the response is not part of our problem.

we are trying to make sure that we don' t have virus checking software causing delay on the W2019 machine

we are looking for best practice here.

do people use a private tunnel between webserver and DB server ?

do people put QM on the webserver and try to get the data locally?

I am not sure how to log the time any deeper than we are doing, and looking for some suggestions.

Jessie, I would test each hop to find out exactly where the loss is at.

1. Run the program directly in QM (no web server)
2. Use the browser on the server and hit the web service via localhost
3. From another computer in the same network hit the end point via the ip address.
4. From the Apache server (which I believe is remote) hit the end point via the IP address.

This should tell you where the time his coming from.  If this is all fast, then change all the remote IP entries to DNS and see if you are having a DNS lookup issue.

I have chased issues with Windows Servers/Azure and Security software not liking the remote web access and whatever inspection they are doing is slowing stuff down.  You will see the big difference from doing localhost:/endpoint vs other server/endpoint. 

If you do find startup/shutdown speeds are an issue QM does have an option to have pre-spawned web lines.  I believe QM calls them Connection Pools (not to be confused with U2 Connection Pools).  I know Martin implemented this specifically for Windows since Process creation is more expensive on Windows than Linux.

We are running OpenQM 4.0.8 on W2019 in Azure.

We are running an Apache web server on a private separate server outside of Azure.

we are having what we consider slow and variable response times. we have added a considerable amount of tracking but are unable to find the bottleneck.

we are use PHP page to call a function that gets data from the QM server. we are using a http call

from the PHP function we call:    (we have 20+ of these calls for different information), for instance:

http://web.xyz.com:8280/api/getpurp?fnme="obj.maint"

we know that the pageload (and function call) is taking 1.723 seconds. We have tracked the QM side at 8ms.

we have looked at the flush time in QM   and tried WFLUSH to make sure that buffering the response is not part of our problem.

we are trying to make sure that we don' t have virus checking software causing delay on the W2019 machine

we are looking for best practice here.

do people use a private tunnel between webserver and DB server ?

do people put QM on the webserver and try to get the data locally?

I am not sure how to log the time any deeper than we are doing, and looking for some suggestions.

Yep we understand most of this.

I believe the pooling only helps with the child process starting on QM.

If I use the localhost, I see some part of a second delay in displaying the results. If I send another request, a second later it is instantaneous. So, it could help with this issue.

I tested with our D3 box which is in same Azure environment.  The response time from a browser there was also fast.

From the external Apache server the response jumps up to 2-3 seconds.

I can't see much difference between IP and QM machine name.

what have you seen as best practice

1) build private tunnel from Apache to QM server

2) put QM on linux on the webserver machine

3) put a linux machine in Azure and run QM there

4) or none of these

Just looking for a direction to run in.

or....... and don't say this, go back to Flashconnect and use our D3 box

Jessie, I would test each hop to find out exactly where the loss is at.

1. Run the program directly in QM (no web server)
2. Use the browser on the server and hit the web service via localhost
3. From another computer in the same network hit the end point via the ip address.
4. From the Apache server (which I believe is remote) hit the end point via the IP address.

This should tell you where the time his coming from.  If this is all fast, then change all the remote IP entries to DNS and see if you are having a DNS lookup issue.

I have chased issues with Windows Servers/Azure and Security software not liking the remote web access and whatever inspection they are doing is slowing stuff down.  You will see the big difference from doing localhost:/endpoint vs other server/endpoint. 

If you do find startup/shutdown speeds are an issue QM does have an option to have pre-spawned web lines.  I believe QM calls them Connection Pools (not to be confused with U2 Connection Pools).  I know Martin implemented this specifically for Windows since Process creation is more expensive on Windows than Linux.

We are running OpenQM 4.0.8 on W2019 in Azure.

We are running an Apache web server on a private separate server outside of Azure.

we are having what we consider slow and variable response times. we have added a considerable amount of tracking but are unable to find the bottleneck.

we are use PHP page to call a function that gets data from the QM server. we are using a http call

from the PHP function we call:    (we have 20+ of these calls for different information), for instance:

http://web.xyz.com:8280/api/getpurp?fnme="obj.maint"

we know that the pageload (and function call) is taking 1.723 seconds. We have tracked the QM side at 8ms.

we have looked at the flush time in QM   and tried WFLUSH to make sure that buffering the response is not part of our problem.

we are trying to make sure that we don' t have virus checking software causing delay on the W2019 machine

we are looking for best practice here.

do people use a private tunnel between webserver and DB server ?

do people put QM on the webserver and try to get the data locally?

I am not sure how to log the time any deeper than we are doing, and looking for some suggestions.

Yep we understand most of this.

I believe the pooling only helps with the child process starting on QM.

If I use the localhost, I see some part of a second delay in displaying the results. If I send another request, a second later it is instantaneous. So, it could help with this issue.

I tested with our D3 box which is in same Azure environment.  The response time from a browser there was also fast.

From the external Apache server the response jumps up to 2-3 seconds.

I can't see much difference between IP and QM machine name.

what have you seen as best practice

1) build private tunnel from Apache to QM server

2) put QM on linux on the webserver machine

3) put a linux machine in Azure and run QM there

4) or none of these

Just looking for a direction to run in.

or....... and don't say this, go back to Flashconnect and use our D3 box

Jessie, I would test each hop to find out exactly where the loss is at.

1. Run the program directly in QM (no web server)
2. Use the browser on the server and hit the web service via localhost
3. From another computer in the same network hit the end point via the ip address.
4. From the Apache server (which I believe is remote) hit the end point via the IP address.

This should tell you where the time his coming from.  If this is all fast, then change all the remote IP entries to DNS and see if you are having a DNS lookup issue.

I have chased issues with Windows Servers/Azure and Security software not liking the remote web access and whatever inspection they are doing is slowing stuff down.  You will see the big difference from doing localhost:/endpoint vs other server/endpoint. 

If you do find startup/shutdown speeds are an issue QM does have an option to have pre-spawned web lines.  I believe QM calls them Connection Pools (not to be confused with U2 Connection Pools).  I know Martin implemented this specifically for Windows since Process creation is more expensive on Windows than Linux.

We are running OpenQM 4.0.8 on W2019 in Azure.

We are running an Apache web server on a private separate server outside of Azure.

we are having what we consider slow and variable response times. we have added a considerable amount of tracking but are unable to find the bottleneck.

we are use PHP page to call a function that gets data from the QM server. we are using a http call

from the PHP function we call:    (we have 20+ of these calls for different information), for instance:

http://web.xyz.com:8280/api/getpurp?fnme="obj.maint"

we know that the pageload (and function call) is taking 1.723 seconds. We have tracked the QM side at 8ms.

we have looked at the flush time in QM   and tried WFLUSH to make sure that buffering the response is not part of our problem.

we are trying to make sure that we don' t have virus checking software causing delay on the W2019 machine

we are looking for best practice here.

do people use a private tunnel between webserver and DB server ?

do people put QM on the webserver and try to get the data locally?

I am not sure how to log the time any deeper than we are doing, and looking for some suggestions.

Windows likes theads vs processes and has a higher overhead.  If your second hit was faster then I do believe session pooling could help you.  The first hit with no spawned threads will always be slow and you can tune how many exist and how long they last.  Connection Pooling (rocketsoftware.com)

You can also use a script/etc to warm them up or keep the pool up if needed.  This would match watch flashconnect was doing (it had pre-spawned phantoms).  The QM system is more robust and configurable than D3 was.

As for best practices I would say eliminating hops is always best.  If you do have hops then make sure you are minimizing how many of those hops are encrypted.  A vpn is typically encrypted, therefore layering a ssh tunnel or even using additional SSL/Cert work is slowing it down.

Linux will typically be faster than windows, especially for most MV systems.  The connection pool system was created specifically to speed up windows.  

With that said I have ran sites in this hybrid approach and if you are seeing a 1-2 second increase in time added purely going over a VPN/Route from one center to another (where apache is vs QM) I would look into that more.  I would remove QM out of the equation and focus on simple ping and speeds between the two and look at the mss response times.  Compare that to your local stuff.  I don't have direct numbers off the top of my head but if you are looking at 1000+ ms ping response times that is a pretty slow connection.  If you cannot speed it up then moving the two closer together on a local lan segment is your best solution.

Yep we understand most of this.

I believe the pooling only helps with the child process starting on QM.

If I use the localhost, I see some part of a second delay in displaying the results. If I send another request, a second later it is instantaneous. So, it could help with this issue.

I tested with our D3 box which is in same Azure environment.  The response time from a browser there was also fast.

From the external Apache server the response jumps up to 2-3 seconds.

I can't see much difference between IP and QM machine name.

what have you seen as best practice

1) build private tunnel from Apache to QM server

2) put QM on linux on the webserver machine

3) put a linux machine in Azure and run QM there

4) or none of these

Just looking for a direction to run in.

or....... and don't say this, go back to Flashconnect and use our D3 box

Jessie, I would test each hop to find out exactly where the loss is at.

1. Run the program directly in QM (no web server)
2. Use the browser on the server and hit the web service via localhost
3. From another computer in the same network hit the end point via the ip address.
4. From the Apache server (which I believe is remote) hit the end point via the IP address.

This should tell you where the time his coming from.  If this is all fast, then change all the remote IP entries to DNS and see if you are having a DNS lookup issue.

I have chased issues with Windows Servers/Azure and Security software not liking the remote web access and whatever inspection they are doing is slowing stuff down.  You will see the big difference from doing localhost:/endpoint vs other server/endpoint. 

If you do find startup/shutdown speeds are an issue QM does have an option to have pre-spawned web lines.  I believe QM calls them Connection Pools (not to be confused with U2 Connection Pools).  I know Martin implemented this specifically for Windows since Process creation is more expensive on Windows than Linux.

We are running OpenQM 4.0.8 on W2019 in Azure.

We are running an Apache web server on a private separate server outside of Azure.

we are having what we consider slow and variable response times. we have added a considerable amount of tracking but are unable to find the bottleneck.

we are use PHP page to call a function that gets data from the QM server. we are using a http call

from the PHP function we call:    (we have 20+ of these calls for different information), for instance:

http://web.xyz.com:8280/api/getpurp?fnme="obj.maint"

we know that the pageload (and function call) is taking 1.723 seconds. We have tracked the QM side at 8ms.

we have looked at the flush time in QM   and tried WFLUSH to make sure that buffering the response is not part of our problem.

we are trying to make sure that we don' t have virus checking software causing delay on the W2019 machine

we are looking for best practice here.

do people use a private tunnel between webserver and DB server ?

do people put QM on the webserver and try to get the data locally?

I am not sure how to log the time any deeper than we are doing, and looking for some suggestions.

Thanks for the insight. we are moving to put QM on linux on our webserver to test that configuration. 

we have config file for our PHP pages that allows us to change where the request goes to, so we can switch back and forth to see if we are moving closer.

If we can make some significant gains, then we will go back and look at the connection pooling.

We will also have to look at replication on QM to get from our current Testing/documentation QM box to the linux box. That is step 2 or 3 at this point.

Will post our results

Windows likes theads vs processes and has a higher overhead.  If your second hit was faster then I do believe session pooling could help you.  The first hit with no spawned threads will always be slow and you can tune how many exist and how long they last.  Connection Pooling (rocketsoftware.com)

You can also use a script/etc to warm them up or keep the pool up if needed.  This would match watch flashconnect was doing (it had pre-spawned phantoms).  The QM system is more robust and configurable than D3 was.

As for best practices I would say eliminating hops is always best.  If you do have hops then make sure you are minimizing how many of those hops are encrypted.  A vpn is typically encrypted, therefore layering a ssh tunnel or even using additional SSL/Cert work is slowing it down.

Linux will typically be faster than windows, especially for most MV systems.  The connection pool system was created specifically to speed up windows.  

With that said I have ran sites in this hybrid approach and if you are seeing a 1-2 second increase in time added purely going over a VPN/Route from one center to another (where apache is vs QM) I would look into that more.  I would remove QM out of the equation and focus on simple ping and speeds between the two and look at the mss response times.  Compare that to your local stuff.  I don't have direct numbers off the top of my head but if you are looking at 1000+ ms ping response times that is a pretty slow connection.  If you cannot speed it up then moving the two closer together on a local lan segment is your best solution.

Yep we understand most of this.

I believe the pooling only helps with the child process starting on QM.

If I use the localhost, I see some part of a second delay in displaying the results. If I send another request, a second later it is instantaneous. So, it could help with this issue.

I tested with our D3 box which is in same Azure environment.  The response time from a browser there was also fast.

From the external Apache server the response jumps up to 2-3 seconds.

I can't see much difference between IP and QM machine name.

what have you seen as best practice

1) build private tunnel from Apache to QM server

2) put QM on linux on the webserver machine

3) put a linux machine in Azure and run QM there

4) or none of these

Just looking for a direction to run in.

or....... and don't say this, go back to Flashconnect and use our D3 box

Jessie, I would test each hop to find out exactly where the loss is at.

1. Run the program directly in QM (no web server)
2. Use the browser on the server and hit the web service via localhost
3. From another computer in the same network hit the end point via the ip address.
4. From the Apache server (which I believe is remote) hit the end point via the IP address.

This should tell you where the time his coming from.  If this is all fast, then change all the remote IP entries to DNS and see if you are having a DNS lookup issue.

I have chased issues with Windows Servers/Azure and Security software not liking the remote web access and whatever inspection they are doing is slowing stuff down.  You will see the big difference from doing localhost:/endpoint vs other server/endpoint. 

If you do find startup/shutdown speeds are an issue QM does have an option to have pre-spawned web lines.  I believe QM calls them Connection Pools (not to be confused with U2 Connection Pools).  I know Martin implemented this specifically for Windows since Process creation is more expensive on Windows than Linux.

We are running OpenQM 4.0.8 on W2019 in Azure.

We are running an Apache web server on a private separate server outside of Azure.

we are having what we consider slow and variable response times. we have added a considerable amount of tracking but are unable to find the bottleneck.

we are use PHP page to call a function that gets data from the QM server. we are using a http call

from the PHP function we call:    (we have 20+ of these calls for different information), for instance:

http://web.xyz.com:8280/api/getpurp?fnme="obj.maint"

we know that the pageload (and function call) is taking 1.723 seconds. We have tracked the QM side at 8ms.

we have looked at the flush time in QM   and tried WFLUSH to make sure that buffering the response is not part of our problem.

we are trying to make sure that we don' t have virus checking software causing delay on the W2019 machine

we are looking for best practice here.

do people use a private tunnel between webserver and DB server ?

do people put QM on the webserver and try to get the data locally?

I am not sure how to log the time any deeper than we are doing, and looking for some suggestions.