You’re all going to laugh at this… but I figured out what the issue was. It turns out, I had setup a BPM to run when the Schedule updated the Next Run On field. It was a trick I read somewhere in here to automate a BPM. Effectively, it’s a utility to complete tasks and close old quotes. It was working like a charm for the longest time…
Once I remembered about this BPM, I suspected it might be the culprit. So I moved all of the reports to a new schedule but left the old one still enabled. Sure enough, the reports ran just fine. Interestingly, though, I noticed that the original schedule appeared not to have run… as in the Next Run On date was in the past stuck still at 5:00am earlier in the morning. So I reviewed the Event Viewer and sure enough there was an error that started at 5:00am and had been occurring every 2 minutes or so since.
I guess something in one of the quotes decided that it needed a valid Assigned To value and it was causing the BPM to error. I’m not sure why, but the error wasn’t showing up in Event viewer when the reports were tied in with the schedule (or I somehow overlooked it). But this was causing the Next Run On not to update because every time it tried, it would error in the BPM. Something must tell it to keep trying and consequently, every 2 minutes or so it was trying to re-execute the list of tasks on that schedule - all my scheduled reports. I guess it just kept sending them to the task agent and task agent just kept on doing it’s job.
So that’s the long and short of it. I’ll try to find that original thread that I used to learn how to automate the utility and put a little nugget of caution in there. I guess I need to find a way to handle the errors so that it still allows the transaction to update in the SysAgentSched table.