API - line manipulation speed relative to order size

I guess from my lowly point of view, if we are the make, move, sell economy I feel like the trend is consolidation into a few major conglomerates with thousands of locations (these are not examples of our customer, but of those conglomerates) like Lowes, Walmart, Home Depot, Meijer, Grocers in general, Convinence Stores like 7-11, etc.

I don’t know for how much longer we can ignore this use case for ERP… And why would you want to if you are Epicor? Figuring out a solution to help us satisfy customers like these would only bolster the “make, move, sell” mission they have in my mind.

Idk. Still waiting for that user group to form and enough of us to come together and request this together. Lot of money in selling to those types of businesses.

Fair Jonathan. I always love reading your stuff.

I see it like this, “we aren’t that large of a company and we really need to review our processes in order to fit them to Epicors limitations.”

However, I see enough of us out here working around the same “limitation” and I’d propose Epicor helps us out so that we don’t have these limitations anymore- especially given they are trying to power the make, move, sell economy. These players are giants in that industry, and this would position Epicor well in that space if they had hundreds of customers fulfilling orders for those types of entities. I’ll do however many customer references Epicor would like me to do if they ever make something to help with this.

I can’t find it, but I remember a thread with Stephen Edginton about increasing order throughput. The breakdown was to post the entire order in some efficient format, validate certain fields (customer, Part, etc.), and if all was good, make a durable copy at the server, and return a 202. If errors, mark it and requeue. The UI is done. If the validation fields are fairly static, move the validation to another service before using Kinetic resources.

With the copy at the server, do the updates server-side using the techniques mentioned above (UpdateExt, etc). If there’s an error, mark it as such and re-queue, otherwise mark it successful. I think it was also suggested to have X number of workers in parallel working on the queue. :thinking:

@Jgrissom is right. Skip the Kinetic UI and go headless.

Mark, isn’t the need to input the required data and hit the required BOs a MUST regardless of whether you’re headless?

I’m using headless in the way Epicor is. Yes, of course you need input and hit the BOs.

If you’re talking about the validation step, there are some items that you could cache externally: customer numbers, part number, terms, etc. depending on your customer base. Any kind of EDI/batch input would be a good candidate there.

There are some items you can check without BOs: date ranges, address validation, etc.

The BOs will still do the validation, but exceptions take time. Why spend the cycles for a bad transaction in Kinetic if I can avoid it altogether? Good transactions will still be checked, and they may find other errors, but it’s just a game of chance like a cache. Caches take extra effort, but statistically they can speed things up if the data is conducive.

Thanks Mark

Exactly @Mark_Wonsil

I get what you and Mark are saying. I think we’re just looking at different bottlenecks. I don’t spend much time dealing with the Kinetic UI and all the little on-blur/update calls that can make the interactive experience slow; I’m mostly calling the larger BO methods directly. So from my side, those BO methods themselves are the unavoidable bottleneck.

This is more or less what we’re doing now with our API that sits in front of the kinetic API. A lot of code dedicated to producing clean, valid payloads before Epicor ever sees them.

I don’t know. I can’t tell whether implementation has created more folds or smoothed over my brain like glass most days.

Did you do it? what was the veredict

Not yet but I will sneak it in this week and report back.

Remotely (through REST) or at the server (UBAQ or Function)?

Remotely through REST. We’ve considered building functions that will actually accept batch data for things like setting the dyn attributes (colors), and then foreaching that within the function to hit the BO method there. Testing that will probably be pushed back post go-live for us since it would probably not do much beyond shifting the code burden to the function rather than our API, with less visibility for errors.

Oh, I suspect it would speed up the processing quite a bit.

This is why I like using a queue. Each pending order is a record in the queue. If there’s an error, it gets marked. It’s quite easy to then query the queue to see the associated errors.

I understand that you’re too close to launch. This suggestion is for when you really want to squeeze out as much speed as possible. The event-driven model is what most high-volume retailers like Amazon use.

AI Provided Overview:

Amazon’s Use of an Event Model in Order Processing

Amazon’s order processing systems are built on an event-driven architecture, using AWS services like Amazon EventBridge and Amazon Kinesis to decouple and scale its workflows.

How it works

In modern Amazon order management, when an order is placed (e.g., via an e‑commerce site or mobile app), the system publishes an order event to an event bus such as Amazon EventBridge.

This event is then routed to multiple downstream services without requiring them to call each other directly.

Key components in the event model

  • Event producers: Order entry systems, eCommerce platforms, or APIs that publish events (e.g., “OrderPlaced,” “PaymentProcessed,” “InventoryReserved”).

  • Event router: Amazon EventBridge acts as the central hub, filtering and routing events to the right consumers.

  • Event consumers: Services like inventory management, payment processing, fulfillment, and notifications that react to events in real time.

Example flow

  1. Order received → published to Amazon EventBridge.

  2. Inventory service checks stock and reserves items.

  3. Payment service processes the transaction.

  4. Fulfillment service picks, packs, and ships the order.

  5. Notification service sends order confirmation and tracking updates.

  6. Analytics service logs and analyzes the event for performance insights.

Benefits of this model

  • Decoupling: Services communicate via events, not direct calls, allowing independent scaling and deployment.

  • Scalability: Can handle traffic spikes (e.g., Black Friday) without system-wide bottlenecks.

  • Resilience: If one service fails, others can continue processing.

  • Real-time responsiveness: Customers can be notified immediately when an order is received, while background processing continues asynchronously.

Is this pretty much Microservices?

Microservices Architecture Style - Azure Architecture Center | Microsoft Learn

More like event-driven architecture with services. Microservices get more granular than what’s listed here and are less recommended except in certain scenarios where you have multiple development teams working on the same system for example.

Using functions overall is a downgrade I have to say.

Most importantly, writing and debugging the code is a nightmare. There error logging and handling is a mess and barely useful.

Function speed itself does not seem to be better either, and anything running (like complex tasks) will have an effect on your server.

You also lose the ability to usefully structure code, and passing data around between functions is a pain. You either have to re-query the same data over and over, or you have to have a ton of variables that you pass between functions. Code re-usability is very low.

The amazon event driven thing is overblown, its just another buzzword for roughly the same thing. Events are just direct calls but abstracted.

Managed to get UpdateExt to create a new line and explode the kit BOM into the sales order, also auto generated a release. It took 9.1 seconds to get a 200 back though so that’s not great, and I was testing through the swagger.

I just fed it a single OrderDtl record though, if I can feed an entire orders worth of lines in and it doesnt multiply the processing time it will definitely be worth it. Big fan of the minimal payload required at least, that’s pretty nice:

Summary
{
  "ds": {
    "OrderHed": [
      {
        "OrderNum": 280174,
        "Company": ""
      }
    ],
    "OrderDtl": [
      {
        "OrderNum": 280174,
        "Company": "",
        "OrderLine": 0,
        "PartNum": "270-0121",
        "KitFlag": "P",
        "KitAllowUpdate": true,
        "OrderQty": 1,
        "SellingQuantity": 1,
        "SalesUM": "EA",
        "RowMod": "A"
      }
    ],
    "OrderRel": []
  },
  "continueProcessingOnError": false,
  "rollbackParentOnChildError": true
}

5 of the same sales kit in one call worked, took 33.8 seconds. 40 kits 3.3 minutes. The math is making my brain hurt but I’m happy to take better performance per kit the higher the kit count without questioning it any day!

Thanks Jose I’ll see if I can get it to set all the other fields needed and condense some order steps.

Edit: ran into more scaling problems when I tried a test with 80 kits, but I’m not sure if it’s the size of the order now or the number of kits. Will do more testing but it’s promising.

The fun thing you will find is that every business object has a specific concurrency limitation, and that they fire child BOs that also have concurrency limitations.. deciphering what all of these are is quite fun, and they differ depending on the makeup of the records (a boolean flag being set on a particular row will trigger a different logic chain which then causes concurrency to behave differently)

dataset is your friend.