"This row has been removed from a" report monitor problem

Epicor ERP 10
,
1

We are still fighting with intermittent reports not previewing or printing for users. We have not found a consistent reason and we can not reproduce it manually. I’ve put a Standard Directive BPM on the SysRptLst table to detect if users have a problem and it emails us when it happens so we are at least aware of it. What we are seeing is that the reports LastAction will be updated to “ERROR: Run in preview mode from System Monitor for details. This row has been removed from a table a”. There is error is truncated but after a google search it seems to originate from a System.Data.RowNotInTableException.

System.Data.RowNotInTableException : ‘This row has been removed from a table and does not have any data. BeginEdit() will allow creation of new data in this row.’

We can see this “Error” LastAction status in the User trace log file but then the LastAction is updated to blank or empty immediately after. Again, we can see it in the trace log file. The report never makes it to the printer or previewer.

I’m struggling how to present this to Epicor Support and also where to look for other underlying issues.

Any thoughts would be much appreciated.

Edit: We are on 10.2.400.5

2

Does this ever happen when an archive period is set?

Just thinking that some of the temp tables (like OrderHed_GUID) are getting removed prior to being used.

3

As far as we have seen, no archive period is set. In most cases, users are just using the defaults and either previewing or client printing. Some of the reports they run are Call Ticket, Customer Statement, AR Cash Receipts Group Edit Listing, AR Invoice Form, GL Journal Entry Edit List, Job Traveler, and AR Aged Receivables so it isn’t just a particular report or user. Some users will try more than once and it won’t work until they restart Epicor, which so far is our only solution(and you know how much users love hearing that!).

We can always have the user go back to the System Monitor and preview/print it from there, before it’s removed, but having the users get into the System Monitor is another can of worms we’d rather not open if we don’t have to.

4

Just to get all the facts…

Users have the client installed on their workstation, or use a remote access system like RDS?

Happens on Print and Preview?
I see that you mentioned both in the original post

When it happens when printing, it is to a client printer, server printer or either?

The report actually processes, but doesn’t print or preview? Meaning that the report does appear in Sys Monitor, on the History tab as COMPLETED (and not ERROR), and on the Reports tab. If so, using Print or Preview from the Reports tab processes as expected.

When it happens, is the Session Manager running? (Press and Hold ALT+TAB on the workstation to see if the following shows)

image

edit

I asked about it happening when archive is selected, not as potential cause, but as a diagnostic. If it doesn’t ever happen when an archive is selected, then it has to do with the temp tables disappearing too soon.

5

The client is installed on their workstation, no remote access.

Correct. That is the frustrating part. We pick up in the BPM that the reports LastAction is updated to the “Error:…” but subsequently cleared.

I have not verified this but would Epicor not launch any forms when it isn’t running? They users are active with other forms when this happens. I will see what I can find out.

Here is client log from one of our users

<tracePacket>
  <businessObject>Ice.Proxy.BO.ReportMonitorImpl</businessObject>
  <methodName>Update</methodName>
  <appServerUri>net.tcp://oursever/Prod400/</appServerUri>
  <returnType>System.Void</returnType>
  <localTime>4/22/2020 14:53:39:8761938 PM</localTime>
  <threadID>1</threadID>
  <executionTime total="44" roundTrip="42" channel="0" bpm="0" other="2" />
  <retries>0</retries>
  <parameters>
    <parameter name="ds" type="Ice.BO.ReportMonitorDataSet">
      <ReportMonitorDataSet xmlns="http://www.epicor.com/Ice/300/BO/ReportMonitor/ReportMonitor" />
    </parameter>
  </parameters>
  <paramDataSetChanges>
    <paramDataSet name="ds" useDataSetNbr="0">
      <changedValue tableName="SysRptLst" rowState="Modified" rowNum="0" colName="LastAction"><![CDATA[Process]]></changedValue>
      <changedValue tableName="SysRptLst" rowState="Modified" rowNum="0" colName="AutoAction"><![CDATA[]]></changedValue>
    </paramDataSet>
  </paramDataSetChanges>
  <serverTrace>
    <Op Utc="2020-04-22T19:53:39.6129426Z" act="Ice:BO:ReportMonitor/ReportMonitorSvcContract/Update" dur="32.5801" cli="192.168.2.39:64205" usr="saiduser" machine="oursever" pid="21812" tid="257" xmlns="">
      <BpmCustomization Source="DB" BpMethodCode="Ice.SysRptLst" Type="In Transaction Trigger" Duration="0" />
      <BpmCustomization Source="DB" BpMethodCode="Ice.SysRptLst" Type="Standard Trigger" Duration="0">
        <BpmDirective Type="1" ID="81283603-ae5e-4c7b-bf71-14da38127dfe" Name="EmailProcessingErrors" VisibilityScope="0" Duration="0" />
      </BpmCustomization>
    </Op>
  </serverTrace>
</tracePacket>

<tracePacket>
  <businessObject>Ice.Proxy.BO.ReportMonitorImpl</businessObject>
  <methodName>Update</methodName>
  <appServerUri>net.tcp://oursever/Prod400/</appServerUri>
  <returnType>System.Void</returnType>
  <localTime>4/22/2020 14:53:39:9220705 PM</localTime>
  <threadID>1</threadID>
  <executionTime total="26" roundTrip="25" channel="0" bpm="0" other="1" />
  <retries>0</retries>
  <parameters>
    <parameter name="ds" type="Ice.BO.ReportMonitorDataSet">
      <ReportMonitorDataSet xmlns="http://www.epicor.com/Ice/300/BO/ReportMonitor/ReportMonitor" />
    </parameter>
  </parameters>
  <paramDataSetChanges>
    <paramDataSet name="ds" useDataSetNbr="0" />
  </paramDataSetChanges>
  <serverTrace>
    <Op Utc="2020-04-22T19:53:39.6597991Z" act="Ice:BO:ReportMonitor/ReportMonitorSvcContract/Update" dur="16.0565" cli="192.168.2.39:64205" usr="saiduser" machine="oursever" pid="21812" tid="40" xmlns="" />
  </serverTrace>
</tracePacket>

<tracePacket>
  <businessObject>Ice.Proxy.BO.ReportMonitorImpl</businessObject>
  <methodName>Update</methodName>
  <appServerUri>net.tcp://oursever/Prod400/</appServerUri>
  <returnType>System.Void</returnType>
  <localTime>4/22/2020 14:53:39:9489984 PM</localTime>
  <threadID>1</threadID>
  <executionTime total="37" roundTrip="36" channel="0" bpm="0" other="1" />
  <retries>0</retries>
  <parameters>
    <parameter name="ds" type="Ice.BO.ReportMonitorDataSet">
      <ReportMonitorDataSet xmlns="http://www.epicor.com/Ice/300/BO/ReportMonitor/ReportMonitor" />
    </parameter>
  </parameters>
  <paramDataSetChanges>
    <paramDataSet name="ds" useDataSetNbr="0">
      <changedValue tableName="SysRptLst" rowState="Modified" rowNum="0" colName="LastActionOn"><![CDATA[4/22/2020 2:53:39 PM]]></changedValue>
      <changedValue tableName="SysRptLst" rowState="Modified" rowNum="0" colName="LastAction"><![CDATA[ERROR: Run in preview mode from System Monitor for details. This row has been removed from a table a]]></changedValue>
    </paramDataSet>
  </paramDataSetChanges>
  <serverTrace>
    <Op Utc="2020-04-22T19:53:39.6910433Z" act="Ice:BO:ReportMonitor/ReportMonitorSvcContract/Update" dur="26.8871" cli="192.168.2.39:64205" usr="saiduser" machine="oursever" pid="21812" tid="257" xmlns="">
      <BpmCustomization Source="DB" BpMethodCode="Ice.SysRptLst" Type="In Transaction Trigger" Duration="0" />
      <BpmCustomization Source="DB" BpMethodCode="Ice.SysRptLst" Type="Standard Trigger" Duration="0">
        <BpmDirective Type="1" ID="81283603-ae5e-4c7b-bf71-14da38127dfe" Name="EmailProcessingErrors" VisibilityScope="0" Duration="0" />
      </BpmCustomization>
    </Op>
  </serverTrace>
</tracePacket>

<tracePacket>
  <businessObject>Ice.Proxy.BO.ReportMonitorImpl</businessObject>
  <methodName>Update</methodName>
  <appServerUri>net.tcp://oursever/Prod400/</appServerUri>
  <returnType>System.Void</returnType>
  <localTime>4/22/2020 14:53:39:9868974 PM</localTime>
  <threadID>1</threadID>
  <executionTime total="94" roundTrip="67" channel="0" bpm="25" other="2" />
  <retries>0</retries>
  <parameters>
    <parameter name="ds" type="Ice.BO.ReportMonitorDataSet">
      <ReportMonitorDataSet xmlns="http://www.epicor.com/Ice/300/BO/ReportMonitor/ReportMonitor" />
    </parameter>
  </parameters>
  <paramDataSetChanges>
    <paramDataSet name="ds" useDataSetNbr="0">
      <changedValue tableName="SysRptLst" rowState="Modified" rowNum="0" colName="LastActionOn"><![CDATA[4/22/2020 2:53:39 PM]]></changedValue>
      <changedValue tableName="SysRptLst" rowState="Modified" rowNum="0" colName="LastAction"><![CDATA[]]></changedValue>
    </paramDataSet>
  </paramDataSetChanges>
  <serverTrace>
    <Op Utc="2020-04-22T19:53:39.7535495Z" act="Ice:BO:ReportMonitor/ReportMonitorSvcContract/Update" dur="57.4465" cli="192.168.2.39:64205" usr="saiduser" machine="oursever" pid="21812" tid="40" xmlns="">
      <BpmCustomization Source="DB" BpMethodCode="Ice.SysRptLst" Type="In Transaction Trigger" Duration="0" />
      <BpmCustomization Source="DB" BpMethodCode="Ice.SysRptLst" Type="Standard Trigger" Duration="30">
        <BpmDirective Type="1" ID="81283603-ae5e-4c7b-bf71-14da38127dfe" Name="EmailProcessingErrors" VisibilityScope="0" Duration="30" />
      </BpmCustomization>
    </Op>
  </serverTrace>
</tracePacket>

Thank you.

6

I only asked about the session manager, as it can be problematic for remote users (which you said is not your case).

If RDS is used User A is the first session of the day, the session manger is launched on the RDS server. Subsequent users connecting on the RDS sever use the same Session manager instance. If user A then exits the client, the Session manager is closed, leaving all the other users in some sort of limbo. But the Epicor KB article only mentions that this affects printing. Doesn’t mention that it would affect opening forms, running processes, etc…

7

Are these SSRS or CR reports? Or a mix?

8

So far just SSRS. We don’t use Crystal. Occasionally we will see it happen to the Financial reports and the viewer doesn’t launch. The LastAction doesn’t get cleared like the SSRS in this case, it ends up with the “Error…” so it’s clear in the system monitor that something went wrong. When the LastAction is blank for the other reports, we now make an assumption that something went wrong.

1 Like
9

Are the client and tasks using the same appserver or are they separated?
I just added a second appserver to process tasks because @aidacra had mentioned that if you called support with printing issues and had not made a background appserver that would be the first step to take.

10

We have two task agents but the problem existed before adding the second one. The task agent has done its job. The error seems to happen on the System Monitor client side. The Task Agent will mark the LastAction as “SSRSReady” when it is done and after that point the System Monitor takes over.

11

Possibly an issue with where the rendered reports are stored on the client machine? Access rights, anti-virus conflict, file locking (like by Google Drive - and possibly OneDrive), etc…

12

In this situation, as a test, have the user close just the Sys Monitor, and then have them re-launch it.

You might need to add a menu item where they can access it - as I think it only resides in System Management.

1 Like
13

That is the can of worms we don’t want to open yet. I’m 99.9% sure that restarting the System Monitor would temporarily solve the issue but it’s easier to tell them to restart Epicor. If we can’t find the underlying cause soon, your suggestion will be our workaround along with sending the user the email that a report potentially did not work.

14

Again, it was suggested as a diagnostic tool. To raise the confidence in the source of the problem (hopefully minimizing the number of things Support will make you do first)

15

I am on 10.2.400.15 and did have a couple of users that I had to train on how to print from system monitor and kill and restart it, but nothing like you are experiencing.

I took a look at my release notes and found a resolved issue in 10.2.400.9 with a description of Task Agent runs for a while and then stop processing new requests and references a stack overflow issue found. I have no idea if that could have also possibly solved your issue.

I did have a new task agent install in the 10.2.400.15 update folder dated 11/1/2019.

16

We had a break through late Friday. After digging into a couple of users trace files, we found some consistencies. I notice there was a method call to Ice.Proxy.Lib.SessionModImpl.InMobileConnect method, which I have never seen before in all the users trace files that had the problem. We also noticed they were idle prior to that call and there was also a network exception. These items in the trace file were well before the time when they tried to print/preview. But I was finally able to reproduce it. Here are the steps to reproduce.

  1. Start Epicor
  2. Navigate to Job Traveler
  3. Pick a Job, Submit Preview
  4. Close Job Traveler
  5. Disconnect from the network(e.g., unplug ethernet, disable ethernet, disconnect wifi)
  6. Wait 30-40 Seconds while the Epicor Offline form is showing.(I usually watch the the progress bar go across 5 times)[this is where Epicor makes the InMobileConnect call]
  7. Reconnect to network
  8. Let Epicor resume.
  9. Navigate to Job Traveler
  10. Pick a Job, Submit Preview
  11. No Preview will show and the Last Action will be blank.
  12. Trace file will show LastAction is was updated to “ERROR: Run in preview mode from System Monitor for details. This row has been removed from a table a” then subsequently updated to blank.
3 Likes
17

@ckrusen Thanks for talking through the issue. It definitely helped get me out of my tunnel vision.

I finally opened a case with Epicor and we’ve determined it is semi-fixed in 10.2.600. The reports will now work after a network disconnection but you will see in the system monitor that it may not show the correct Last Action. In some cases after Epicor reconnects, you will get a Last Action of “process” instead of “PREVIEW” or “PRINT” when you run a report but they still work.

18

can you share with me the case that you had for this?

19

CS0002036599 - System Monitor Fails to Process Reports after Network Disconnect.

PRB0223828 - System Monitor fails to update the correct Last Action after the client disconnects.

Like I mentioned before in our current 10.2.600 version. It is semi-fixed. I have not tested it in any of the new versions yet