Beat the hell out of your Pilot system…from the critical processes like Q2C, ME, YE to the most mundane of them…BAQs, dashboards, even base stuff that you didn’t even modify.
This. All of our issues following our most recent upgrade were because people failed to test full quote-to-cash. We took their word they tested. They didn’t and it showed.
We got burned by that once. Some tried to blame it on IT so the old CFO changed the testing so it’s now done as a meeting. We have a day long in-person meeting with all the department heads and do a Q2C in Pilot. We get lunch on the company at least too.
We are going to change our strategy for testing next time around. The testing team members claimed they tested certain things but when we tested in pilot after go-live the issue was the same. It was never tested and reported and therefore not fixed. Big old swing and miss.
You can use activity tracking to check what testing, if any has been done. Pull the records for the menu type activity and you will see what applications have even been accessed since testing was meant to have started. Its not precise but it at least tells you if they have even gone into, for example, sales order entry, shipment entry, etc at all.
as others have mentioned, Dashboards, and customizations in regular kinetic as well as Data Collection (AKA MES). But dont forget your Product Configurators. Also as someone said, make sure that you have everyone (not just you… EVERYONE) log into the system using a BROWSER, and make sure that every screen runs in the browser with the expected customizations, UD FIelds, etc. Here is the complete list so far:
Custom Dashboards
Customizations to Kinetic Screens
Customizatinos to Data Collection Screens
Product Configurator
Enhanced Quality (formerly Configurator managed) screens
If you are going browser and you are cloud then do it yourself a favour and invest some time on test automation. This will put you in good stared for coping with the upgrade cadence. Has the potential for saving money rather than paying for flexing. Of course the process is only as good as your tests, but you have the same problem with manual testing.