I’m at my wit’s end. For the last month I’ve had intermittent system-wide freezes in Kinetic where no one could work until I did an IISRESET on the app server. Like once every week or two.
Well, yesterday it happened 4 times and I just did a reset again today. It’s been rough all week. RAM and CPU always look fine on the app server.
I have tried to at least isolate the issue. Here is how - I have 5 app pools (and 5 matching “application servers” in EAC):
Our Task Agent is on its own app pool on a different server. This never crashes.
Our integrations have their own app pool. No issues there.
EKW has its own app pool. Again, no issues there.
Starting yesterday afternoon, I put myself and my sysadmin coworker on our own app pool. No problems for us either.
But all other human users that log in through a browser are the ones that keep seeing system freeze-ups.
App pool 1 (task agent) is on the SQL server and 2-4 are on the app server.
When the user app pool (#5) freezes, it does not affect the others. I can keep working in one of the other app pools if I want. I’m often oblivious to the users’ struggles with slowness and freezing.
So, I’m not sure what else to try. Support has not helped. I guess to be honest, I have not rebooted the server…
Another question I had - at what layer do BPMs live? On the app pool? In the database? In the task agent?
I wish I could be more helpful. I always use event viewer, all the different server log options (depending on the severity of the issue and being careful of performance hits).
And then you’ll see something like the following picture.
And it showed this guy, with exceptionally long times on it. (The top one is 2,290 seconds, or 38 minutes.) This is a snapshot during a freeze-up today (before I did IISRESET to it).
You may have deduced as I did that this method is the Kinetic landing page grid that you see when you open the Receive Transfer Order app.
We don’t do a ton of transfers, so it does jibe with the fact that the problem is very intermittent.
Now, why does an OOB function get so hung up? No idea.
But, you can still receive a transfer order the old-school ways – by searching, or typing the number by hand.
So, I threw a pre-processing BPM on the evil method, and it stops the grid. I did this an hour ago, but I really do think that will put the nail in the coffin.
I’m jumping the gun - it hasn’t even been a full day yet - but I’m declaring this solved.
I’m looking today and there are no requests that last more than a second. They all come and go near-instantly.
So I’ll be using this tool often now.
My first stop 2 months ago was the query store in SSMS, but I think it’s useless if the query does not actually finish. Also, this way is better because I can see the quasi-human-readable text of the API call, rather than hoping to reverse-engineer some SQL query text in SSMS.
I haven’t had to use it yet.
