Unterminated String Error on String Split Result

Kinetic ERP
, ,
1

Good morning,
I have a BAQ with a BPM that calls a function to pull a list of employee IDs from my timeclock API. The string EmpIDs is populated with a comma delimited list of ID numbers without spaces, or any other characters. This is the custom code block that generates the error:

InfoMessage.Publish(EmpIDs.ToString()); 
string[] emp = EmpIDs.Split(',');
var i = 0;
    while (i < emp.Length)
    {
        InfoMessage.Publish(emp[i]);
        i++;
    }

In this case. If I change [i] to a number that falls in my employee count range, then I do get the info message for that employee id. However, if I run it as shown with [i] in place, then I get this error:

Unterminated string. Expected delimiter: ". Path ‘Context.InfoMessage[54].SysRowID’, line 1, position 18724.

Application Error

Exception caught in: Newtonsoft.Json

Error Detail 
============
Message: Unterminated string. Expected delimiter: ". Path 'Context.InfoMessage[54].SysRowID', line 1, position 18724.
Program: Newtonsoft.Json.dll
Method: ReadStringIntoBuffer

Client Stack Trace 
==================
   at Newtonsoft.Json.JsonTextReader.ReadStringIntoBuffer(Char quote)
   at Newtonsoft.Json.JsonTextReader.ParseValue()
   at Newtonsoft.Json.JsonTextReader.Read()
   at Ice.Api.Serialization.JsonReaderExtensions.ReadAndAssert(JsonReader reader)
   at Ice.Api.Serialization.IceTableConverter.CreateRow(JsonReader reader, IIceTable table, Lazy`1 lazyUDColumns, JsonSerializer serializer)
   at Ice.Api.Serialization.IceTableConverter.ReadJson(JsonReader reader, Type objectType, Object existingValue, JsonSerializer serializer)
   at Ice.Api.Serialization.IceTablesetConverter.ReadJson(JsonReader reader, Type objectType, Object existingValue, JsonSerializer serializer)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.DeserializeConvertable(JsonConverter converter, JsonReader reader, Type objectType, Object existingValue)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.SetPropertyValue(JsonProperty property, JsonConverter propertyConverter, JsonContainerContract containerContract, JsonProperty containerProperty, JsonReader reader, Object target)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateObject(Object newObject, JsonReader reader, JsonObjectContract contract, JsonProperty member, String id)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize(JsonReader reader, Type objectType, Boolean checkAdditionalContent)
   at Newtonsoft.Json.JsonSerializer.DeserializeInternal(JsonReader reader, Type objectType)
   at Ice.Cloud.ProxyBase`1.ProcessJsonReturnHeader[TOut](KeyValuePair`2 messageHeader)
   at Ice.Cloud.ProxyBase`1.SetResponseHeaders(HttpResponseHeaders httpResponseHeaders)
   at Ice.Cloud.ProxyBase`1.HandleContractAfterCall(HttpResponseHeaders httpResponseHeaders)
   at Ice.Cloud.ProxyBase`1.CallWithMultistepBpmHandling(String methodName, ProxyValuesIn valuesIn, ProxyValuesOut valuesOut, Boolean useSparseCopy)
   at Ice.Cloud.ProxyBase`1.Call(String methodName, ProxyValuesIn valuesIn, ProxyValuesOut valuesOut, Boolean useSparseCopy)
   at Ice.Proxy.BO.DynamicQueryImpl.GetList(DynamicQueryDataSet queryDS, QueryExecutionDataSet executionParams, Int32 pageSize, Int32 absolutePage, Boolean& hasMorePage)
   at Ice.Adapters.DynamicQueryAdapter.<>c__DisplayClass45_0.<GetList>b__0(DataSet datasetToSend)
   at Ice.Adapters.DynamicQueryAdapter.ProcessUbaqMethod(String methodName, DataSet updatedDS, Func`2 methodExecutor, Boolean refreshQueryResultsDataset)
   at Ice.Adapters.DynamicQueryAdapter.GetList(DynamicQueryDataSet queryDS, QueryExecutionDataSet execParams, Int32 pageSize, Int32 absolutePage, Boolean& hasMorePage)
   at Ice.UI.App.BAQDesignerEntry.BAQTransaction.TestCallListBckg()
   at Ice.UI.App.BAQDesignerEntry.BAQTransaction.<>c__DisplayClass220_0.<BeginExecute>b__0()
   at System.Threading.Tasks.Task.InnerInvoke()
   at System.Threading.Tasks.Task.Execute()

The error seems to point to JSON parsing, but the string EmpIDs is returned correctly. I can prove this by changing [i] to a number. The first info message showing the entire comma delimited list is shown, and the single empid is show in the inside info message.

I have also tried a for loop with the same results. It seems I can see the EmpIDs, and I can split it, but I can’t call the result of the split properly. What did I miss?
Thanks for your time!

2

If I was looking at this, I would test for the last ID getting clipped at the source. When you print the string, is the last id complete?

3

Yes. I have a total of 275 IDs that are mushed together in the string. I can pass [0], and [274] and get the proper IDs back. This one is really throwing me for a loop. It all looks like it should work. I thought maybe the single quotes on the split command might be an issue. Testing with double quotes around the comma does not seem to change the resulting error message.

4

I would try a write diag of just x to see if it was not dying at the end, but on some bad data. If that is the case and probably a good idea anyway when you do not control what is being sent is a try catch, so you don’t die, but fail and keep moving.

5

I have used try catch block in this, but they didn’t catch or return any extra details. What is this process you are describing?

6

Maybe the list is too long and header is truncated? what if you send half of the list?

7

assuming the assignment is where the error occurs you can wrap that in a try and empty catch, so the loop finishes for the good results.

I use these rather than dialog boxes, so I can see in the serverlog or event viewer on the server all of the logging.

Ice.Diagnostics.Log.WriteEntry($" Current x is  {x}");
8

I am not sure what you mean. The only thing I do with my list of EMP IDs in (string format) is to split them, and then use one ID at a time.

I am cloud DT.

9

Olga’s asking if you’re passing information through BPMData. BPMData is sent through the REST headers and there is a size limit there, which might trucate your Json and give that message.

10

I don’t think so. I am executing the custom code in a widget inside my UBAQ BPM post processing get list.

11

that is weird because it fails inside ProcessJsonReturnHeader
so somehow it cannot process what is returned

12

I thought so too! I can break down my BPM into a simple 2-step. Step 1 is a function call to my third party API. This populates EmpIDs. If I run that by itself, it works perfectly, showing a messagebox of all the IDs in a comma delimited list. If I add on a custom code element after that function call to try to use the results of the call by splitting the string and then working on one id at a time, that is when I get the error. :thinking:

13

InfoMessage.Publish will put it in the call context header, AFAIR, so probabbly it is still where you hit the limit

2 Likes
14

Oooohhhhhh!!! I see! :smiling_face_with_three_hearts: Thanks Olga! I will drop my infomessages in case they are causing issues.

15

@Olga I think you were right on there. After I removed the InfoMessages I was able to nudge my code along. Instead of pushing debug info to an Info Message, or Diagnostic Log, I just added them to a string and then displayed the string in a dialog box at the end of my code.

Once I reconnect all my widgets, the looping process takes a long time, and often is forcibly closed by the server. It is really hard to debug these issues without step tracing. This is the timeout error I am getting after about 10 minutes of waiting.

Application Error

Exception caught in: mscorlib

Error Detail 
============
Message: An error occurred while sending the request.
Inner Exception Message: The underlying connection was closed: A connection that was expected to be kept alive was closed by the server.
Program: CommonLanguageRuntimeLibrary
Method: ThrowForNonSuccess

Client Stack Trace 
==================
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Extensions.Http.Logging.LoggingHttpMessageHandler.<SendAsync>d__5.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Extensions.Http.Logging.LoggingScopeHttpMessageHandler.<SendAsync>d__5.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Ice.Cloud.ProxyBase`1.<ExecuteAsync>d__62.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Epicor.Utilities.AsyncHelper.RunSync[TResult](Func`1 method)
   at Ice.Cloud.ProxyBase`1.Execute(String methodName, RestValueSerializerBase serializer, ProxyValuesIn valuesIn, ProxyValuesOut valuesOut)
   at Ice.Cloud.ProxyBase`1.<>c__DisplayClass60_0.<CallWithCommunicationFailureRetry>b__0(Context _)
   at Polly.Policy`1.<>c__DisplayClass32_0.<Execute>b__0(Context ctx, CancellationToken ct)
   at Polly.Retry.RetryEngine.Implementation[TResult](Func`3 action, Context context, CancellationToken cancellationToken, IEnumerable`1 shouldRetryExceptionPredicates, IEnumerable`1 shouldRetryResultPredicates, Func`1 policyStateFactory)
   at Polly.RetryTResultSyntax.<>c__DisplayClass12_0`1.<WaitAndRetry>b__0(Func`3 action, Context context, CancellationToken cancellationToken)
   at Polly.Policy`1.ExecuteInternal(Func`3 action, Context context, CancellationToken cancellationToken)
   at Polly.Policy`1.Execute(Func`3 action, Context context, CancellationToken cancellationToken)
   at Polly.Policy`1.Execute(Func`2 action, Context context)
   at Ice.Cloud.ProxyBase`1.CallWithCommunicationFailureRetry(String methodName, ProxyValuesIn valuesIn, ProxyValuesOut valuesOut, RestRpcValueSerializer serializer)
   at Ice.Cloud.ProxyBase`1.CallWithMultistepBpmHandling(String methodName, ProxyValuesIn valuesIn, ProxyValuesOut valuesOut, Boolean useSparseCopy)
   at Ice.Cloud.ProxyBase`1.Call(String methodName, ProxyValuesIn valuesIn, ProxyValuesOut valuesOut, Boolean useSparseCopy)
   at Ice.Proxy.BO.DynamicQueryImpl.GetList(DynamicQueryDataSet queryDS, QueryExecutionDataSet executionParams, Int32 pageSize, Int32 absolutePage, Boolean& hasMorePage)
   at Ice.Adapters.DynamicQueryAdapter.<>c__DisplayClass45_0.<GetList>b__0(DataSet datasetToSend)
   at Ice.Adapters.DynamicQueryAdapter.ProcessUbaqMethod(String methodName, DataSet updatedDS, Func`2 methodExecutor, Boolean refreshQueryResultsDataset)
   at Ice.Adapters.DynamicQueryAdapter.GetList(DynamicQueryDataSet queryDS, QueryExecutionDataSet execParams, Int32 pageSize, Int32 absolutePage, Boolean& hasMorePage)
   at Ice.UI.App.BAQDesignerEntry.BAQTransaction.TestCallListBckg()
   at Ice.UI.App.BAQDesignerEntry.BAQTransaction.<>c__DisplayClass220_0.<BeginExecute>b__0()
   at System.Threading.Tasks.Task.InnerInvoke()
   at System.Threading.Tasks.Task.Execute()

Inner Exception 
===============
The underlying connection was closed: A connection that was expected to be kept alive was closed by the server.



   at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)
   at System.Net.Http.HttpClientHandler.GetResponseCallback(IAsyncResult ar)


Inner Exception 
===============
Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host.



   at System.Net.Security._SslStream.EndRead(IAsyncResult asyncResult)
   at System.Net.TlsStream.EndRead(IAsyncResult asyncResult)
   at System.Net.PooledStream.EndRead(IAsyncResult asyncResult)
   at System.Net.Connection.ReadCallback(IAsyncResult asyncResult)


Inner Exception 
===============
An existing connection was forcibly closed by the remote host



   at System.Net.Sockets.Socket.EndReceive(IAsyncResult asyncResult)
   at System.Net.Sockets.NetworkStream.EndRead(IAsyncResult asyncResult)

This says to me that my API calls are not well controlled. One of them must be running on an unterminated loop, or something. Here is the final code that seems like it should work.

string[] emp = EmpIDs.Split(",");
var i = 0;
thisDeptHrs = 0;
while (i < emp.Length)
{
    // Reset myDate for each employee iteration
    var myDate = mySDate;
    try
    {
        // Call the GetTimeCard function to get EmpHrs and EmpDept    
        for (; myDate <= myEDate; myDate = myDate?.AddDays(1))
        {
            try
            {
              var response = InvokeFunction("AsureJson", "GetTimeCard", emp[i].ToString(), myDate);
              //myDebug += (emp[i].ToString() + " ");
              if (response != null && Convert.ToDecimal(response[0]) != 0)
              {
                // Ensure EmpHrs and EmpDept are converted to the appropriate types
                var EmpHrs = Convert.ToDecimal(response[0]); 
                var EmpDept = response[1].ToString(); 

                if (EmpDept == myKey) //this emp is in the department we are summing
                {
                    // EmpDeptID already exists in dictionary, add EmpHrs to its sum
                    thisDeptHrs += EmpHrs;
                }

              }
              else
              {
                  // Handle the case where response is null
                  Console.WriteLine("GetTimeCard returned null response.");
              }
            }
            catch (Exception ex)
            {
                // Handle specific exception or log the error
                Console.WriteLine($"Error while processing GetTimeCard for EmpID {emp[i]} and Date {myDate}: {ex.Message}");
            }
        }//end for loop
    }
    catch (Exception ex)
    {
        // Handle specific exception or log the error
        Console.WriteLine($"Error while processing EmpID {emp[i]}: {ex.Message}");
    }
    i++;
}//end while loop

try
{
    var ttResults_xRow = (from ttResults_Row in result.Results where ttResults_Row.Calculated_pulled == false select ttResults_Row).FirstOrDefault();
    ttResults_xRow.Calculated_TCHrs = thisDeptHrs;
    ttResults_xRow.Calculated_pulled = true;
    currentRecord = currentRecord + 1;
}
catch (Exception ex)
{
    // Handle specific exception or log the error
    Console.WriteLine($"Error while updating ttResults_xRow: {ex.Message}");
}

My BAQ returns a list of Department IDs that are mapped to the department key in my timeclock. So my BAQ has 10 rows, one for each department. The goal is to loop through all employees and add up any time that exists into the correct department bucket. Since I process one department at a time, this means that I am looping through each of the 275 employee ids and making 275 get timecard API calls, for each of my 10 departments. Clearly this is not ideal. I should probably find a way to loop though all 275 employees once, then add their hours to the correct department. But without hardcoding in my departments, I am not sure how to do this.

Do you see anything that stands out as an obvious error in my code? Based on my goal, do you have a suggestion for a better way to summarize the data and make my API calls more efficient?

For API calls I have:
Get Employee IDs: returns all the employee IDs from the timeclock software. I run this once at the beginning of my BPM.
Get Timecard Data: returns the timecard data for a single employee on a single date, along with department key*. I run this 275*10 times. This is probably the problem.
*Technically this call returns the timecard entries for a single employee in a pay period that includes the requested date. So, I have to look through all the timecard dates returned and make sure to pull only the time for the date I am working on.

Thanks again for taking the time to consider my question!
Happy Friday!

16

maybe you have myDate incorrectly?
I would add diagnosting message (that writes to the log) at the beginning of for loop and check how many times it is called

17

@NateS Can you do the call once and get all relevant data you need? If so stash it in a ud table. I have one I use just for temp data and then delete once processing is done.

Then you can use c# to do the summing by department. I would probably bring all of the UD data up into a list and then process it all from there.

I am not this fancy, but you could use this idea.
https://www.epiusers.help/t/tutorial-using-a-ud-table-as-a-temp-table-for-external-or-generated-data-to-link-with-sql/98902

18

If you’re able to get a list of all departments instead of passing one at a time, you could do something like this:

Dictionary<string, decimal> deptKeyToTotalHours = listOfDepartments.ToDictionary(dept => dept.YourDeptKey, dept => 0M); // Sets up a hash map of department to a decimal total

and then in your for loop, in the place of:

grab the item it belongs to and add your hours:

KeyValuePair<string, decimal> kvpThisDeptTotal = deptKeyToTotalHours.FirstOrDefault(kvp => kvp.Key == EmpDept);
kvpThisDeptTotal.Value += EmpHrs;

Just an idea.

You’re looping 275 Employees * 10 Departments = 2750 times for every day in your period … assuming a 2-week pay period: 38,500 iterations

If all this data is already in your DB, it seems like it would be better to gather it all with a single BAQ that you provide the date range and groups the results by date, department with a sum function to get the totals per date and department.

3 Likes
19

Thank you all for all your help! I dug further into my API and found that I can filter my employee IDs by department. Strangely, filters applied in the API URL line don’t seem to apply when testing in Postman, so I had to implement it in Epicor to see it work properly.

First, I pull the list of departments from Epicor that I want. This is returned in the BAQ as 1 row per department.

Next, I have a function to lookup the department timeclock key using the epicor department ID. Here I had to hardcode a dictionary in my function to do the lookups. Not ideal, but it works.

Now that I have the correct department key, I make an API call to get the list of employees that are in the department.

Finally, I pull the timecard data for each employee in the department, then add their time to the total. I loops this process over each day that needs to be pulled, and sum the results.

What was going to be a 30+ minute operation is now down to under 3 minutes! Now that is what I am talking about! I couldn’t have done it without you all. Thanks again!
Have an excellent weekend!

2 Likes
20

I have an interesting delay. It locks up epicor for 10+ minutes! I have deployed my dashboard as assembly and added to the menu. There are no customizations on my dashboard. I am working in modern/classic client. I am cloud DT. My dashboard is not set to auto refresh, or refresh on load.

The delay happens when I open the dashboard from the menu, then click refresh to prompt for parameters. Instead of entering BAQ parameters on the popup, I click cancel. This causes Epicor to freeze up. I assume it is trying to make the API calls from my GetList custom action.

To try to avoid this (so far without success) I have added a condition to my BAQs GetList BPM. The condition just checks for null in the start date parameter variable. If it is null it should show the message box with a warning instead of running the rest of the code to hit up the API. I imagine the API call is going crazy because it wants start and end dates but didn’t get them.

What do you think? Is my Getlist somehow ignoring my check for null in my start date parameter? What does a parameter return if you click cancel?
Thanks again for your time!