One report change, now an error... works on Pilot, not on Live

Adam · November 1, 2018, 4:45pm

I made one field change to our custom Quote report in Report Builder, hit save, now I get the following error. The change worked fine on Pilot, but Live gives this error. I built a solution from Pilot (included the RDD and report) and installed on Live, but still the same error.

Other than the RDD and Report, what else may be causing this? Some server cache of some sort?
Program Ice.Services.Lib.RunTask raised an unexpected exception with the following message: RunTask: An error has occurred during report processing. —> Microsoft.ReportingServices.ReportProcessing.ProcessingAbortedException: An error has occurred during report processing. —> Microsoft.ReportingServices.ReportProcessing.ReportProcessingException: Query execution failed for dataset ‘DataSet1’. —> System.Exception: For more information about this error navigate to the report server on the local server machine, or enable remote errors
Stack Trace:
at System.Web.Services.Protocols.SoapHttpClientProtocol.ReadResponse(SoapClientMessage message, WebResponse response, Stream responseStream, Boolean asyncCall)
at System.Web.Services.Protocols.SoapHttpClientProtocol.Invoke(String methodName, Object[] parameters)
at Ice.Core.SsrsReportService.ReportExecutionService.LoadReport(String Report, String HistoryID) in C:_Releases\ICE\ICE3.2.100.15\Source\Server\Internal\Lib\TaskLib\Web References\SsrsReportService\Reference.cs:line 287
at Ice.Core.SsrsReporting.SsrsRendererBase.ConfigureReportServerForReport(String reportPath, ParameterValue[] parameters) in C:_Releases\ICE\ICE3.2.100.15\Source\Server\Internal\Lib\TaskLib\SsrsReporting\SsrsRendererBase.cs:line 163
at Ice.Core.SsrsReporting.SsrsSingleDocumentRenderer.<>c__DisplayClass1_0.b__0() in C:_Releases\ICE\ICE3.2.100.15\Source\Server\Internal\Lib\TaskLib\SsrsReporting\SsrsSingleDocumentRenderer.cs:line 49
at Ice.Core.SsrsReporting.SsrsRendererBase.<>c__DisplayClass12_0.b__0() in C:_Releases\ICE\ICE3.2.100.15\Source\Server\Internal\Lib\TaskLib\SsrsReporting\SsrsRendererBase.cs:line 87
at Epicor.Hosting.Trace.TraceHandle.TimeAction(Action actionToTime) in C:_Releases\ICE\ICE3.2.100.15\Source\Framework\Epicor.System\Hosting\Trace\TraceHandle.cs:line 92
at Epicor.Hosting.Trace.TraceHandle.AddTimedIfEnabled(String traceFlag, Action actionToTime, Func2 messageBuilder) in C:\_Releases\ICE\ICE3.2.100.15\Source\Framework\Epicor.System\Hosting\Trace\TraceHandle.cs:line 39 at Ice.Core.SsrsReporting.SsrsRendererBase.TraceReportRendered(String printProgram, String ssrsRenderFormat, Func1 timedAction) in C:_Releases\ICE\ICE3.2.100.15\Source\Server\Internal\Lib\TaskLib\SsrsReporting\SsrsRendererBase.cs:line 95
at Ice.Core.SsrsReporting.ReportProcessorBase.RenderReportForPreviewOrGenerate(RenderedSsrsReport renderedReport) in C:_Releases\ICE\ICE3.2.100.15\Source\Server\Internal\Lib\TaskLib\SsrsReporting\ReportProcessorBase.cs:line 256
at Ice.Core.SsrsReporting.ReportProcessorBase.ProcessReportPart(String reportLocation) in C:_Releases\ICE\ICE3.2.100.15\Source\Server\Internal\Lib\TaskLib\SsrsReporting\ReportProcessorBase.cs:line 150
at Ice.Core.SsrsReporting.StandardReportProcessor.ProcessReportParts() in C:_Releases\ICE\ICE3.2.100.15\Source\Server\Internal\Lib\TaskLib\SsrsReporting\StandardReportProcessor.cs:line 31
at Ice.Core.RoutingAndBreaking.ReportPersister.Persist(ReportInstanceInformation reportInstance, Func2 reportsRenderer, Action1 fillSysRptLstRow, Action2 processReport) in C:\_Releases\ICE\ICE3.2.100.15\Source\Server\Internal\Lib\TaskLib\RoutingAndBreaking\ReportPersister.cs:line 50 at Ice.Core.RptBase.ReportSsrsDatabaseBuilder.RenderUnroutedSsrsReport() in C:\_Releases\ICE\ICE3.2.100.15\Source\Server\Internal\Lib\TaskLib\RptBase\ReportSsrsDatabaseBuilder.cs:line 301 at Ice.Core.RptBase.ReportSsrsDatabaseBuilder.RunSsrsReportIfEnabled() in C:\_Releases\ICE\ICE3.2.100.15\Source\Server\Internal\Lib\TaskLib\RptBase\ReportSsrsDatabaseBuilder.cs:line 226 at Ice.Core.RptBase.ReportSsrsDatabaseBuilder.ProcessReportWithDataInPlace(Func2 executeCommand, Func2 executeReader, SqlObjectsCreated sqlObjectsCreated) in C:\_Releases\ICE\ICE3.2.100.15\Source\Server\Internal\Lib\TaskLib\RptBase\ReportSsrsDatabaseBuilder.cs:line 109 at Ice.Core.RptBase.ReportDatabaseBuilder.XMLClose() in C:\_Releases\ICE\ICE3.2.100.15\Source\Server\Internal\Lib\TaskLib\RptBase\ReportDatabaseBuilder.cs:line 76 at Ice.Core.RptTaskBase1.XMLClose() in C:_Releases\ICE\ICE3.2.100.15\Source\Server\Internal\Lib\TaskLib\RptBase\RptTaskBase.cs:line 159
at Ice.Hosting.TaskCaller.InnerExecuteTask(IceDataContext newContext) in C:_Releases\ICE\ICE3.2.100.15\Source\Framework\Epicor.Ice\Hosting\TaskCaller\TaskCaller.cs:line 104
at Ice.Hosting.TaskCaller.ExecuteTask() in C:_Releases\ICE\ICE3.2.100.15\Source\Framework\Epicor.Ice\Hosting\TaskCaller\TaskCaller.cs:line 63
at Ice.Lib.RunTask.BpmFriendlyTaskLauncher.Run(String sessionIdPrefix, IceContext db, Action taskRunner) in C:_Releases\ICE\ICE3.2.100.15\Source\Server\Services\Lib\RunTask\BpmFriendlyTaskLauncher.cs:line 63
at Ice.Services.Lib.RunTaskSvc.InnerRunTask(Int64 ipTaskNum, Boolean suppressTransaction) in C:_Releases\ICE\ICE3.2.100.15\Source\Server\Services\Lib\RunTask\RunTask.cs:line 542
at Ice.Services.Lib.RunTaskSvc.InnerRunTask(Int64 ipTaskNum, Boolean suppressTransaction) in C:_Releases\ICE\ICE3.2.100.15\Source\Server\Services\Lib\RunTask\RunTask.cs:line 542
at Ice.Services.Lib.RunTaskSvcFacade.RunTask(Int64 ipTaskNum) in C:_Releases\ICE\ICE3.2.100.15\Source\Server\Services\Lib\RunTask\RunTaskSvcFacade.cs:line 138
at SyncInvokeRunTask(Object , Object[] , Object[] )
at System.ServiceModel.Dispatcher.SyncMethodInvoker.Invoke(Object instance, Object[] inputs, Object[]& outputs)
at Epicor.Hosting.OperationBoundInvoker.InnerInvoke(Object instance, Func2 func) in C:\_Releases\ICE\ICE3.2.100.15\Source\Framework\Epicor.System\Hosting\OperationBoundInvoker.cs:line 59 at Epicor.Hosting.OperationBoundInvoker.Invoke(Object instance, Func2 func) in C:_Releases\ICE\ICE3.2.100.15\Source\Framework\Epicor.System\Hosting\OperationBoundInvoker.cs:line 28
at Epicor.Hosting.Wcf.EpiOperationInvoker.Invoke(Object instance, Object[] inputs, Object[]& outputs) in C:_Releases\ICE\ICE3.2.100.15\Source\Framework\Epicor.System\Hosting\Wcf\EpiOperationInvoker.cs:line 23
at System.ServiceModel.Dispatcher.DispatchOperationRuntime.InvokeBegin(MessageRpc& rpc)
at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage5(MessageRpc& rpc)
at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage11(MessageRpc& rpc)
at System.ServiceModel.Dispatcher.MessageRpc.Process(Boolean isOperationContextSet)
at System.ServiceModel.Dispatcher.ChannelHandler.DispatchAndReleasePump(RequestContext request, Boolean cleanThread, OperationContext currentOperationContext)
at System.ServiceModel.Dispatcher.ChannelHandler.HandleRequest(RequestContext request, OperationContext currentOperationContext)
at System.ServiceModel.Dispatcher.ChannelHandler.AsyncMessagePump(IAsyncResult result)
at System.ServiceModel.Dispatcher.ChannelHandler.OnAsyncReceiveComplete(IAsyncResult result)
at System.Runtime.Fx.AsyncThunk.UnhandledExceptionFrame(IAsyncResult result)
at System.Runtime.AsyncResult.Complete(Boolean completedSynchronously)
at System.ServiceModel.Channels.SecurityChannelListener1.ReceiveItemAndVerifySecurityAsyncResult2.InnerTryReceiveCompletedCallback(IAsyncResult result)
at System.Runtime.Fx.AsyncThunk.UnhandledExceptionFrame(IAsyncResult result)
at System.Runtime.AsyncResult.Complete(Boolean completedSynchronously)
at System.ServiceModel.Channels.TransportDuplexSessionChannel.TryReceiveAsyncResult.OnReceive(IAsyncResult result)
at System.Runtime.Fx.AsyncThunk.UnhandledExceptionFrame(IAsyncResult result)
at System.Runtime.AsyncResult.Complete(Boolean completedSynchronously)
at System.ServiceModel.Channels.SynchronizedMessageSource.ReceiveAsyncResult.OnReceiveComplete(Object state)
at System.ServiceModel.Channels.SessionConnectionReader.OnAsyncReadComplete(Object state)
at System.Runtime.Fx.AsyncThunk.UnhandledExceptionFrame(IAsyncResult result)
at System.Net.LazyAsyncResult.Complete(IntPtr userToken)
at System.Net.LazyAsyncResult.ProtectedInvokeCallback(Object result, IntPtr userToken)
at System.Net.Security.NegotiateStream.ProcessFrameBody(Int32 readBytes, Byte[] buffer, Int32 offset, Int32 count, AsyncProtocolRequest asyncRequest)
at System.Net.Security.NegotiateStream.ReadCallback(AsyncProtocolRequest asyncRequest)
at System.Net.AsyncProtocolRequest.CompleteRequest(Int32 result)
at System.Net.FixedSizeReader.CheckCompletionBeforeNextRead(Int32 bytes)
at System.Net.FixedSizeReader.ReadCallback(IAsyncResult transportResult)
at System.Runtime.AsyncResult.Complete(Boolean completedSynchronously)
at System.ServiceModel.Channels.ConnectionStream.IOAsyncResult.OnAsyncIOComplete(Object state)
at System.Net.Sockets.SocketAsyncEventArgs.OnCompleted(SocketAsyncEventArgs e)
at System.Net.Sockets.SocketAsyncEventArgs.FinishOperationSuccess(SocketError socketError, Int32 bytesTransferred, SocketFlags flags)
at System.Net.Sockets.SocketAsyncEventArgs.CompletionPortCallback(UInt32 errorCode, UInt32 numBytes, NativeOverlapped* nativeOverlapped)
at System.Threading._IOCompletionCallback.PerformIOCompletionCallback(UInt32 errorCode, UInt32 numBytes, NativeOverlapped* pOVERLAP)

Aaron_Moreng · November 1, 2018, 5:22pm

I’d look at the actual query embedded within the report first. Did you move both the report data definition and the report style?

Adam · November 1, 2018, 5:28pm

Yes, the solution had the style, RDD, and report from Pilot to Live.

I did not do anything to the embedded query and from what I can see it is exactly the same between Live and Pilot.

All I changed was Fields!Contact to Fields!Name.

I’ve deleted the RDD, Style, and report folder from custom folder, then ran the working solution from Pilot… still get the error. WHY??? So frustrating. Works fine on Pilot, dead in the water on Live.

Aaron_Moreng · November 1, 2018, 5:32pm

In the report, did you re-point the datasource to the live environment?

Adam · November 1, 2018, 5:33pm

I don’t typically have to. Where in the report do I verify the server?

Also, is there a way (or need to) to purge the temp tables on the report server?

Aaron_Moreng · November 1, 2018, 5:35pm

I’ve always had to do this for each section of the report.

Adam · November 1, 2018, 5:43pm

The data source is set properly

I even cycled the app server and rebuilt the data models with hopes that something had gone astray.

Hmmm.

Adam · November 1, 2018, 6:34pm

STUCK.

I backed up our Live DB, restored to Pilot and Pilot works fine.
I then made a solution in Live (adding the RDD, Style, and rpt file), then installed in Pilot… Pilot works fine.

What else is there to look at?!? This is so frustrating.

Any ideas anyone?

Adam · November 1, 2018, 6:36pm

It’s almost like the RDD on live is not properly creating the temp tables on the Live Reporting db.

Aaron_Moreng · November 1, 2018, 6:43pm

Might try giving support a call…

EpiUser · November 1, 2018, 6:49pm

I would enable remote errors on your report server, that could give you some more clues as to what the issue is.

ckrusen · November 1, 2018, 6:50pm

I’ll assume your test environment is made from a snapshot of your live DB. and that you tested it with real data in the Test company (say Quote 54321). Does that same dataset (from Quote 54321) in the live company also generate an error?

We had an issue that only shows up when an Order had many (over 200) lines. Since this order didn’t exist in the test company we never really tested it there. When we tried it in the live company it generated errors - which we attributed to a change in the Report or RDD. Which it wasn’t.

Adam · November 1, 2018, 6:51pm

Would you please let me know where that is / how to view the log?

Adam · November 1, 2018, 6:52pm

Yes, test snapshot of live and same quote tested in live and test… works in test, doesn’t work in live.

EpiUser · November 1, 2018, 6:53pm

may be different depending on version:

Start Management Studio and connect to a report server instance. …
Right-click the report server node, and select Properties.
Click Advanced to open the properties page. …
In EnableRemoteErrors, select True.
Click OK.

Adam · November 1, 2018, 6:56pm

Hmm. Must be a bit different than ours. We are on 2016, but I don’t see this option.

EpiUser · November 1, 2018, 6:57pm

here is a link to micorsofts website which talks about it:

ckrusen · November 1, 2018, 6:57pm

It’s a printed report, and not an emailed or break/Route, correct?
(making sure it’s not something weird about the company settings, like SMTP)

What happens if yopu just select “Generate Only” instead of printing or Previewing? Does it still error out?

Adam · November 1, 2018, 7:02pm

Just a printed report. I ran the “Generate Only” and got the error. Does that mean that the RDD is the problem?

ckrusen · November 1, 2018, 7:22pm

Most likely. Try making a report style that uses the new RDD and the base RDL. Unless you really went to town on the RDD changes, most likely all you did was to make it return more data, and the base RDL should still be able to render it.

One last thing … You didn’t happen to click the “Sync Dataset” button on the Report Style Maintenence? Doing that gives me problems 99% of the time. I avoid it like the plague.