Any chance you have a backup you could use to compare the previous DLL (prior to disablin/re-enabling the BPM) to the newly created one.
Once we imported the Live into Test is was working fine. So we would need a backup of the snapshot of the VM. Not just the DB.
DLL’s are stored in the Db in a SQL bloob I suppose I could… But I believe it had to be some kind of working memory issue or cache IDK cause like @Banderson said when we restored the DB to a different instance it worked fine.
1 Like
Wouldn’t stopping and restarting the App Server cause the DLL to be reloaded from the DB blob?
edit
And any memory used by the DLL and its functions to be released, and then re-reserved?
That’s what is supposed to happen. Unless it writes the DLL to disk when compiled and re-uses it. (investigating)
This is what I have in my notes, hope it helps Jose. Let us know what you find!
Q: Where are the BPMs compiled to and stored?
A: As of 10.1.400, the default storage location is the SQL BLOB within the Epicor database itself. This means there are no files that need to be transferred around when copy/restoring databases as the BPMs are already compiled and active within the database itself.
Q: What other custom solution options is stored in the database?
A: All items that were traditionally compiled to the file system are now stored within the SQL BLOB in the Epicor database. This includes: Method Directives, Data Directives, Updatable Business Activity Queries (UBAQ), Generic Imports, Electronic Data Interface (EDI), Posting Engine (PE) Rules and Product Configurator (PC).
Q: Do I need to recompile these after a Live to Test restore?
A: No, the BPMs, Compiled Dashboards and other Custom Solutions will move along with the database and are already compiled.
Q: How can I see the uncompiled code?
A: The intermediate files are stored on the file system. By default they are located under the inetpub\wwwroot{AppPoolName}\Server\BPM folder. These are strictly for viewing only and any changes to these files will not impact the Epicor system. This can be used to help you troubleshoot any of the wizard generated code from the modules listed above.
Q: Where is the SQL BLOB data actually stored?
A: It is stored in a binary (not human readable) format in the Ice.Customization and Ice.CustomizationStore tables.
1 Like
The only time I had issues was when using async BPM Code. An exception usually told the TA to try again, and never give up 
Unfortunately, that all says “It’s stored in the database”, however, when we moved the DB, it fixed itself…
edit: maybe in the inetpub\wwwroot{AppPoolName}\Server\BPM folder
@Banderson you can grab the file from PRD ( inetpub\wwwroot{AppPoolName}\Server\BPM ) and then do a Diff Compare with the Files in that BO in TST see if the Intermediate files have any differences. I doubt, but good to double check.
Make sure you grab the right version, everytime you do enable/disable it will recompile itself into a new folder.
Taking a step back here…
Server side code executes on the servers CPU. So at some point the App server running on the server needs to load that code into memory.
Does the app server fetch directly from the SQL DB? This seems pretty dangerous to bypass the O/S. Essentially becoming “self modifying code”. I’d have bet dollars to donuts that the app server fetches the code then writes it to the servers file system, so that it can be called like any other traditional DLL.
Doing anything out of the ordinary inside of that specific BPM?
There are no BPM dlls on the server (give it a search you won’t find them unless explicitly enabled > 10.1.400 
). They are likely loaded directly into appserver memory. The only thing my production outputs anymore are sources, so I can run searches of code in VS Code.
1 Like
Does the blob contain executable image? Or is it source code that is then compiled (or interpreted?) on demand?
Epicor does this in the Epicor.Customization.dll on the Server
Assembly.Load(blob.Data, blob.Symbols);
It is my understanding that the Blob already contains a compiled Assembly, which is generated during Enable/Disable, that if you wrote it to disk you probably end-up with a valid .dll.
A Blob is an actual file. It’s already compiled just loaded from the blob into memory. The “source” is stored as metadata and a thumbnail preview in Ice.BpDirective
I did not take a memory dump since it was kind of an emergency situation since everything was dragging and timing out.
All the BPM does is Post Processing Update in memorry TT records
This is all it did Post Processing on CashRecGetInvoices.
foreach(var x in ttARInvcAlloc)
{
var data = (from i in Db.InvcDtl where i.Company == x.Company && i.InvoiceNum == x.InvoiceNum group i by 1 into g
select new { LineTotal = g.Sum(z=>z.DocExtPrice), MiscChargeSum = g.Sum(z=>z.DocTotalMiscChrg)}).FirstOrDefault();
var tax =(from i in Db.InvcTax where i.Company == x.Company && i.InvoiceNum == x.InvoiceNum select i.DocTaxAmt).DefaultIfEmpty(0M).Sum();
x["TotLineCharges_c"] = data.LineTotal;
x["TotMiscCharges_c"] = data.MiscChargeSum;
x["TotTaxes_c"] = tax;
}
Did you review serverlog at the time when the directive slowed down the process?
Probably it had some exceptions that didn’t go to the client.
PS. Regarding transferring DB from pilot to production without regeneration of the ECF-based stuff…
It scares me 
No ideas. Looks like voodoo magic.
I have only one idea… For some reason BPM customization including this directive was not regenerated and runtime tried to use old version of it. But I have no idea how it can occur.
3 appserver load balancer so looking at logs is tricky but I did poke around nothing much of interest in any of the servers
Voodoo magic indeed
When this happened to @jgiese.wci a few years ago I told him he was a crazy idiot and that it couldn’t happen…. turns out 
sorry bud 
lol
Oh well I’ll keep an eye out if it happens a again I’ll do a memory dump and more things
Gremlins!
Thanks all
1 Like