Cloud Customers Experiencing Periodic Slowness after migrating to 2022.1.3 over weekend

Now SSRS reports are down. This has been quite the upgrade :partying_face:

3 Likes

I notice at 4:00 pm yesterday performance improved.
I used one of my Dashboards which reads all Resource Groups then reports who’s clocked.
During our 18 minute black out periods it would return no data.
Yesterday at 4:06 pm it was now returning data in 8 seconds. (Normally it would in 2 seconds.)
This morning so far I’m seeing no issues in performance drop.
I can confirm it appears to be fixed.
I will monitor this today to confirm.

1 Like

we are facing terrible situation after upgrade in Live since Pilot works normaly. Task agent does not work, tasks hang constantly while agent shuts down. That is interesting to here this guys from you, becouse support told that only WE are complaining about slownes or task agent failures after upgrade and other companies has no issues

Also MRP in three days from 6 tries we was lucky to run and it finished, normaly it runs 1 hour now it took 3:30

This has been a great thread. There are Epicor employees who are monitoring it. It’s another channel to Ops people. I’m really surprised that that there was no mention of this on Epicor Cloud Status - Kinetic (formerly Epicor ERP Cloud) posts.

Transparency is extremely important for cloud providers. AWS, Azure, etc. ALL have performance or security incidents and outages. It’s going to happen. When companies are putting their trust in a service, the service provider has to be transparent with customers. The recent Okta security incident shows how a minor security incident blew up in their face because or poor communication.

Services providers, including Epicor, need to know that customers are far more forgiving when kept in the loop.

12 Likes

According to @kaybarrett, it may not be related directly to the upgrade since their system wasn’t upgraded and they too are experiencing the same issues as you and the others.

I don’t have any inside information, but I assume that Epicor re-created their former datacenter in Azure. This means it is IaaS and they have to manage all the VMs, networks, etc. So a blip on the production servers due to a configuration mishap could certainly be the cause. :person_shrugging:

2 Likes

I still see a performance drop for the same time frame every hour.
This time 8:06 to 08:26 am cst.
Only I do get data back now so its not a total BAQ timeout.
My Dashboard refresh test should get 2 seconds, now takes 8 seconds.
So for example a data entry function where you use publish and subscribe to a dashboard would be enter data hit save then next record. (Wait 8 seconds instead of 2, then continue.)

I would also like to note on transparency. I did get a response from my initial cloud team ticket entry yesterday exactly 24 hours after it was entered. “Cloud Team working on getting this resolved”. This is not acceptable.

at your side at least something is working. Our task agent is at status “Stopped” for about 3 hours and from last evening no one responds anymore in my ticket

Transparency is extremely important.

Could a conflict of interest arise when someone is trying to sell cloud and also be transparent?

2 Likes

Anyone else in public cloud experiencing an SSRS outage? First complaint came in at 6:30 AM CST. No acknowledgement yet on the status page. Receiving a Cannot create a connection to data source 'nameofdatasource' error in System Monitor.

Oh, this is true for ALL companies no matter what services they are providing!

:money_mouth_face:

2 Likes

Coming from an On Prem installation to a cloud installation has been a bit frustrating. Sometimes, it just takes a deep breath and finding an alternative to accomplish the same task.

There are definitely things that could be done to put more control in System Administrators hands to help eliminate tickets for Epicor - the classic Win-Win.

Like access to the server logs or the ability to restart the task agent.

8 Likes

3 Likes

Agreed.

This has been the worse upgrade since we went to the cloud 5 years ago. Performance is horrible. Just found out that the client in EpicWeb does not work use only the client from (https://download.epicorsaas.com/) however even that doesn’t resolve any of the performance issues

2 Likes

We also deferred our upgrade till August and are experiencing the same. We have an open ticket also.

1 Like

Updating status. So we have been suffering on printing and constant task agent failures. So support team did deep investigation today and pointed in to custom ssrs report style which seems causing task agent overload. We disabled this mtltag report and until now situation seems good, we will see after the night if mrp runs smooth again.

Nice, I am glad you were able to find somethign that was causing your issues. Thanks for updating us.

Latest update on our ticket is the following from the assigned tech:

“I am working on this on the server end and I will get back to you once there is an update.”

Also noticed the ticket status shows: “Waiting on External Assigned Task”

Also got an email alert yesterday evening from Status Page (Status Page for Epicor Cloud Status powered by StatusCast): Scheduled downtime for this Saturday “for Windows patching”. The plot thickens


The scheduled downtime for Windoz patching has been scheduled and did not just appear after the performance issue. Be maybe it is going to be part of there fix who knows. I have had now updates on my support requests however my performance is back to normal.