And those that havenāt are now fearing it. 
4 Likes
Justified feeling. Weāre SaaS and up and running and I donāt know how many customers were affected. Sure weāre one of the lucky ones who werenāt but itās still a scary situation to be in.
1 Like
When Epicor tell us they can run Epicor better than we can on-prem but then do this. Makes it hard to believe them.
8 Likes
Is everyone just locked out of Epicor until the maintenance period is complete?
We have access to Epicor, but have lost some data during the maintenance window.
I havenāt seen anyone else complaining about missing data, so Iām was just curious on if we were the only ones.
Thanks.
My quick test is via the web UX ā¦ keep getting Bad Gateway on load.
This scares me. A lot.
The idea of lost data is unfathomable. I detest using buzzwords but ābest practicesā would usually include a full verified backup prior to upgrading a serverā¦hopefully insuring thereās no data lost in the transition.
3 Likes
Just like on-prem, if our app server is running I believe users can log in.
Weāre not at the 11:45am second extension without an all clear so in limbo.
2 Likes
To be clear weāve only lost records that were changed or added during the maintenance window. But itās tough when Epicor posts that the maintenance window has been extended.
1 Like
I donāt get itā¦if Epicorās declared a maintenance window, arenāt your environments off-limits until itās back up?
Am I missing something? I know Iām a little dense butā¦
EDIT - just read @cmulford 's next messageā¦if they had to roll back to a prior backup instead of the most recentā¦that will be a nightmare trying to recreate what was lost.
We are back up. Able to login around 6:30am est. Printing started around 11:50. We had some quotes that were entered yesterday that were not in the system this morning. Still trying to get get details on the time they were entered. It appears there was a restore done at some point that rolled back. Hopefully we can more information on that from the cloud team.
1 Like
100% Our first shift starts at 6AM which was supposed to be an hour after the maintenance was scheduled to be completed. We came in with users in system already.
Thatās the exact situation I was talking about - you shouldnāt be able to get in your system while theyāre doing the maintenanceā¦
3 Likes
There is no easy ādisallow user loginā with Epicor that we can get to as SaaS --yet?ā. Which would be great even for on-prem.
edited for clarity
1 Like
Itāll be something like a backup fumble. Something not unlike, an automated backup routine specific to this event didnāt produce a recoverable .bak and a standard prior backup was used as a fallback. Itās not great but would limit the scope of the fumble and suggests that the daily automated backups are recoverable.
Migrating any production database across SQL Server updates is a massive pain. At this scale some fallout is absolutely guaranteed with a drop dead switchover. Ideally it would be done incrementally, but I suspect there are far too many client databases sharing each SQL Server for that.
We got off easy, weāre up. Looks like we were down from 2300ish Saturday and back up 0600ish Sunday. @@version says weāre on 16.0.4185.3
1 Like
Howzabout stopping the app server?
5 Likes
Thatās fine and dandy until you need Conversion Workbench and have to start the App Server. Granted since this was SQL server no reason it should of been running at all.
1 Like
For all of my fellow SaaS comrades - youāre entitled to a credit under the SLA because of the downtime.
5 Likes