This is really a two part question and it’s coming from a place of ignorance, so forgive me if this is common knowledge.
We’ve been using Epicor since 2013 (E9 at the time), tacking on various new processes and procedures along the way. Along with varying degrees of data input quality over the years, new processes, and adding/maintaining customizations along the way, it’s become in my mind a bit of a bloated and hard to maintain mess.
We don’t have any formalized record retention policies for ERP data at this point, but I would think at some point, some data should be able to be cleaned up and purged, right? Or does it just follow me around like a ghost? With PII and ePHI becoming more and more important to properly collect, secure, and destroy, how are other companies dealing with this? It doesn’t seem like ERP is built to handle the loss of data, given it’s many reliance’s on connections to other data.
The second part of this question is regarding re-implementing.
Sometimes it sounds amazing to be able to go back to the drawing board and decide what customizations are needed (hint, it’s probably not all of them), what data can be offloaded/summarized, and start with fresh configurations and new data.
I bet this is a pipe dream, but I am curious if anyone had done that and what they can share regarding the experience.
GREAT QUESTION. I’ve been asked it more than once.
The answer… it depends on A LOT of things, and the weight each of those things gets varies tremendously.
Probably most importantly is how long will it take “the committee” (because IT never gets to make those calls) to decide which foundational data (customers, suppliers, part numbers, part classes, customer groups, warehouse bins, et cetera ad infinitum) comes forward; which customizations/BPMs/Configurators/reports (also et cetera ad infinitum) come forward, and how long will it take to rewrite all that technical stuff?
What regulatory/customer requirements are you under? I had one customer that made parts for old (read: obsolete) aircraft… they were required by some of their customers to maintain records for 30 years. We were literally creating UD tables from 4x6 notecards.
What are you hoping to gain? In the (not so) good old days, storage space was expensive… now it’s (pardon the expression) virtually free. Processor time was expensive then too… but with the indexing capabilities of SQL Server, even that isn’t much of an issue.
The things in E9 and E10 that make me scream to reimplement are things like bad UOM settings, major company/site modifications, too many customers/suppliers have changed merged/created national accounts, things of that nature that mean you’re spending too much time fixing things due to bad foundations.
Just having “too much data” is something I haven’t found to be as much of a problem as it used to be.
Thank you for the response. I have much to learn in this area, and this is great to be able to speak with experts!
We are under FDA regulation, as we are in the tissue bank space (more like in a very strange grey-area of that space actually in that we are not a tissue bank but work as the distribution for specialty types of tissues). We do business domestically and internationally, so we would be obligated to comply with both individual state PII regulation (technically) and GDPR requirements.
The primary value to our company with the ERP data has been research, interestingly enough. We transact everything through Epicor, but the primary request I receive for data is around the usage of our products and analysis related to the attributes of the product itself. Of course Epicor continues to be a great platform for efficiency and extendibility (through integrations, customization, etc.) but in my experience, the most important data in it is the product usage data.
We consume inventory data from our suppliers via UD tables, then we consume it into Epicor inventory to transact it financially. In parallel, we transact the UD inventory data in a series of customizations. The UD inventory data is the primary asset for the usage data, obviously combined with who used the inventory. Things like wait times, lot sizing, lot attributes, patient to lot matching algorithms, etc. are all studied and have contributed much (in the way of data sources for scientific publications and internal efficiencies measures) to the evolving science of osteochondral, meniscus, and tendon allografts.
My concern is that it’s a bloated and complex mess and our institutional knowledge is extremely poor. If I hadn’t been the one involved all along the way, I am not sure how it would be useful.
Secondly, while space is not really an issue, poor configurations and too much data can certainly be a hard thing to get quality information from. That’s where I was thinking if there is a good way to summarize old data, it would be nice, but I don’t see how that would work from a functional perspective without breaking all sorts of links in the system.
As to your second question, when we went from 8.03 to E10, we did it as a virgin install of E10, then exported select data from V8, and used DMT to get it into E10. So a fresh start - leaving behind all the old (and “bad” data behind).
This forced us to recreate every BAQ, BPM, BAM, Report, Customization, etc… While that sounds like lots of repeated work, it soon became clear that a lot of those weren’t really necessary. The latest version of E10 now handled our business cases natively (or close enough), or we learned that those old customizations, BAM’s, BAQs, etc…were originally implemented because we didn’t fully understand how V8 worked. So they were “work-arounds” to problems that didn’t really exist.
I’d like to say it was all rainbows and unicorns. But it wasn’t. The biggest letdown wasn’t anything to do with V8, E10 or Epicor. It was our people that were supposed to do this “clean-up”. Things like trimming down the Customer records to just the ones we need. Ditto for Parts, BOM’s, Suppliers, etc… So a lot of our garbage in V8 has found its way into E10.
Full disclosure - some of that was my fault as well. I figured I could just copy all the Part files over, then use DMT delete the ones we didn’t need. That would have worked, had I not copied over the PartCosts too. Those PartCost records created a CST-ADJ for evry part in the system. They’re now forever chained around my neck.
I did a poor job answering your question initially.
What I hope to gain is a cleaner data set, be it configurations, customizations, customers, quotes, etc.
The second thing I didn’t get to resolve was knowing those regulatory entities we are under, I am unclear of how Epicor can support those requirements. It’s not like it’s a simple task to uncover our exposure to PII, at least not in my mind. Record destruction seems to be impossible as well, for most types of this data
The Idea was to “pick and choose”, and only bring over the “good data”. This was most clear when we had Part Numbers like XH-EYEBOLT-3/8".
The one thing that turned out harder than we anticipated was when one team determined that customer XYZ was not to be brought over, but then another wanted Orders related to that customer. And the old orders can’t be imported if the customer doesn’t exist. Likewise, purchasing wanted old PO’s so suppliers and parts that were to be left out of the migration, are now back in the mix.
There was a thread a few months back on PII and destruction/retention. One proposed method would be to copy the PII (names, addresses, phone numbers, etc…) to offline storage, using a hash as the key to relate it back to the original record. Then in the original record, store the hash, and overwrite the PII containing fields with “PII DELETED” (or something to indicate it’s not real.
Interesting…what I am taking away from that is “custom and complex” Wouldn’t that be cool to be able to flag fields as PII and then have Epicor give a OOTB mechanism to do that?
That would be cool. But I believe one of the sticking point with guarding PII is that any stored PII has to be offline. Which makes the process kind of convoluted. “I want to transfer this information electronically, but the receiving repository must be off line…”
I’m going to ignore the PII stuff for now… although that will certainly play a role, I would spend some time at the higher level first (and you may well already know the answers to these).
Who is responsible for the data cleansing? This is ALWAYS one of the most important parts of any re-implementation. Who gets to choose, and what are their parameters? Are you messing with any items that will have a financial impact (Part Class/Product Group/Sales Category, Customer/Supplier Group, things of that nature)? How much time are these people willing/able to commit to this project?
None of your “history” gets lost… as long as you can keep a virtual environment of your current system alive you’ll always have access to it. You can even write external BAQs to join them directly.
Please remember the expression, “Don’t start vast projects with half-vast ideas” (say it quickly).
That’s a great point; I seriously doubt we have the internal resources or stomach (as a company) to go through the pains and commitments of a full reimplementation. There is no one responsible for data cleansing (outside of myself and a few other capable individuals) and configurations were historically implemented by myself and the old implementation team, so the business is used to that paradigm.
Let me ask you this; storage space aside, and assuming that record retention and destruction policies apply, how is one supposed to be compliant with the PII we are avoiding talking about ?
how is one supposed to be compliant with the PII we are avoiding talking about ?
Right now, the only thing I can think of is to add a level of indirection that Calvin mentioned above. That way, you store a meaningless value in Epicor but the PII is stored separately in a CRM or a POD as Tim Berners-Lee is proposing. It keeps PII out of your database from the start so it never ends up in backups.
The first step of any project is defining the scope of the project. If you can’t actually perform the re-implementation you would like, what would a “stomach-able” project look like?
I’ve not yet had to worry about PII, so I really can’t advise you (sorry!). To the best of my knowledge, in the lack of acceptable federal standards, each state sets its own regulations. I know Massachusetts has a pretty stringent requirement (I live in MA, and my wife runs her own business that accepts credit cards… but since she uses a shopping cart program and Etsy, we are not personally handling that data so don’t have to worry about it).
Others here have battle scars from this war… but as you well know, Epicor does not have any real functionality for “record destruction”. I needs to either be added on third-party or built custom AFAIK.
I guess that’s what I am so confused about, since Epicor CRM is tightly coupled with basically all the places PII could exist (customers, suppliers, etc). Surely other companies are under these same regulations? PII itself seems to have multiple interpretations, for example when an individual is a business entity, their name/email/phone are required for their business, but if that same information was for a “contact”, now it’s PII
Need a good lawyer to argue that your customers aren’t “persons”, but rather the companies they work for. So the information about them isn’t really “personal”
Common sense (which is nowhere to be found in a court of law) would say that the law is intended for maintaining individuals privacy. Is my work email address and phone number “personal” information?
This site keeps my email address in DB. Is it bound by PII regulations?
That’s a good question. We are going though a cybersecurity audit and a big focus on it is this kind of info (although not all the focus) and there is just a ton of information that I’m learning right now regarding this stuff. Best way to be compliant is to not have any electronic or paper systems
Wouldn’t that be cool to be able to flag fields as PII and then have Epicor give a OOTB mechanism to do that?
?