Scheduled reports send twice

All reports I’ve scheduled are emailed twice, every time. Sometimes only one makes it to the inbox, but I can see in our email archiving that two were sent (no idea what to make of that discrepancy). The schedules are Monthly and Daily. I confirmed in System Agent and in System Monitor that the reports are only listed once each. We have three different reports on schedules, different filters, different recipients, different submitting users–same behavior for all of them.

It’s an annoyance at worst, but I’d be remiss if I didn’t ask. Any ideas?

My first question would be do you have a Test or Dev Epicor environment running that would have the same scheduled reports set to Email on the same Time Schedule? You would probably notice a difference in the data or dates on the reports if that were the case.

I disable all schedules in Test and Dev. I was actually prompted to look into this again because I showed a user how to schedule a report two days ago, and she’s gotten duplicates already (haven’t updated Test or Dev dbs).

And if you run the Email Process for one of these reports manually do you get only one Email or two?

Are you on 10.2.400, by chance?
The link below happens when you physically click to preview/print, so I’m not sure if it’s fully applicable to the scheduled tasks, but it’s worth mentioning.

1 Like

Good idea. Just once!

I also took a closer look at the email archive. I had not initially filtered outgoing mail, so they DON’T duplicate every single time. But when they do, it duplicates for everyone on that schedule. For instance, all the reports on the 8am schedule got sent twice on April 4, but the reports on the 1pm schedule the same day only went once. Yesterday, the reverse happened. I can also see two tasks for those reports in the history tab of system monitor.

1 Like

This is what you need to key in on.

Does the Task scheduler show just a single scheduled task? Is there a time difference between the two entries in the History? (hint: export that to Excel, and you can see the seconds too)

1 Like

Interesting…can you create a brand new Schedule that is an “off” time like 8:11AM and set one of the reports up to run off of that schedule and see if it duplicates? The “off” time is just so you can easily identify which schedule it ran under.

And yes, I agree with @ckrusen, see if there are any duplicate Tasks under that various Schedules that you have. Even if they aren’t enabled.

The scheduler shows a single scheduled task for each report, as does the Scheduled Tasks tab of the monitor. There is a time difference between the entries. It runs each report, then immediately runs each one again in the same order. Report A, Report B, Report C, Report A, Report B, Report C.

The fact that one scheduled task can create two entries in the history is pretty weird.

Does this happen with anything using that schedule? Like if you schedule a report to print, using that schedule, does it print twice? Or is this just when the output is email?

Is Advanced Printing or Break/Routing in use?

Any BPM’s that might be tied to the task monitor?

Is the time the report runs near 0:00 UTC? I recall someone saying that setting the runtime at (or near) the change of date, caused some weird things. Something to do with updating the NextRunTime, but not the date. So it runs right away again, and after the second run, it then sets the date to the next day (or month)

Here’s the duplication from this morning in the Ice.SysTask table. You can see the same agent schedule for tasks 1-16 and then 1-16 again.
image

Advanced Printing and Break/Routing are not even licensed. Just double-checked that there are no BPMs on anything SysTask-related.

I’ll try putting something besides a report email on the schedules. It might take a long time to see results, though. Looks like they only duplicated twice this month, on yesterday’s 1pm and today’s 8am. Ice.SysTask doesn’t go back further, but I did an informal tally with the email archive and it happened roughly 1 time in April, 5 in March, and 6 in February.

This is just speculation -

But maybe the 16 Tasks assigned to Schedule 80509 take so long to launch - and that the scheduler doesn’t update the next run time, until after all tasks assigned to the schedule have started.

Using the pict below…

  1. At 13:00:13.190, the Task Agent looks at the schedule and sees that schedule 80509 should be run after 13:00:00, so it creates the 16 tasks tied to that schedule.
  2. It looks like it takes anywhere from 15 to 30 seconds for each task to start.
  3. Before the last task is started, the task agent sees that the current time is past the scheduled start time of schedule 80509. So the 16 tasks are started again
  4. After starting the last task of of the first time schedule 80509’s task were started, it updates the next run time of that schedule.

edit:

Just a wild theory …
I’d try making another schedule similar to 80509, and assign half of the tasks to that one.
(don’t forget to delete those moved tasks, from schedule 80509 :wink: )

That…sounds plausible, actually. Most of these reports are the same task report, just filtered for tasks assigned to the recipient, so by design the report fails when someone doesn’t have any tasks. If enough reports fail, they all finish quickly and don’t trigger a second run. I can see in the last month that the initial run finished in under two minutes, except for the times a second run was initiated. That also fits with my impression from the email archive, that there were a lot of successful reports on duplicated events and only a few the other times. Wow!

For ease of management, I’d like to keep them together if I can. I wonder if submitting the reports to a process set, allowing simultaneous processing of tasks, and scheduling that instead would fix it. If not, your suggestion will probably take care of it. I shouldn’t jump the gun but I’m amazed to get a possible explanation so quickly! I’ll report back!

What do you have as a value for the Sys Agent Processing Delay?

I recall reading on her that it should never be zero.

Here’s what the Help says:

Processing Delay
Use this field to regulate how long it takes the system agent to process tasks. The value you enter defines the length of time the task agent remains inactive at the end of processing loops. When a processing loop begins, the task agent runs all available tasks. When these tasks complete, the task agent stays inactive (or sleeps) using the Processing Delay defined on the system agent. When the system clock reaches the end of this time duration, the current processing loop ends and the next processing loop begins.

If you have a fast server that can accommodate the network traffic, enter a lower number in this field. A value of 10 indicates ten seconds must pass at the end of each processing loop before the system agent starts the next loop; a value of 60 indicates that a minute must pass before the next loop begins.

Example: You set this field to 60 seconds. A user submits a report using the Now schedule. It will take a minimum of 60 seconds for the task agent to run this task after it finishes the current processing loop.

Checkout this semi-related topic. From the 4th paragraph on, it talks about the Processing Delay

1 Like

Calling @aidacra

Can you explain why the tasks assigned to a schedule might get started twice?

And from the order they were run in, the second starting of AgentTaskNum 1 (SysTask 171698, @ 13:02:10.093) didn’t happen until after AgentTaskNum 16 (SysTask 171698, @ 13:02:09.937) was started .

The processing delay is the default 3. And unfortunately I don’t see how I can save emailing a report to a process set, so I’ll try splitting the schedule.