We’re intending to move over to ECM for Epicor attachment management, and have exported the attachment table from Epicor (Ice.XFileRef) and verified that the files still exist.
Is there a good way to bulk upload the files from this list (.CSV file)?
Thinkin that I could locate the uploaded files in the AAAA0001 database and update their reference in Epicor via DMT. But looking for a reasonably quick method to upload 9900 files to ECM.
Unless there’s a better method - I’m open to that option too 
Benjamin, they (being Epicor) rumored having a tool to do this that you could pay Epicor to do for you… do you want the contact I have to talk about that?
This is the exact function the ECM Client was made for. Get it running and go to the Batch Import tab.
You can do this a handful of ways but what we did (for this exact same reason/process) is to create the FileAttach table data as an XLSX file with fields for meta data that we wanted filled in and/or key transaction data so a Datalink could be built to go get the rest of the meta data we wanted. We built a workflow for the DocType - this can be a temporary workflow just for the bulk import. This workflow simply processes the file using the datalink, puts it in the correct folder, and attached it back to the Epicor transaction.
It will consume the source file and all the image files into the ECM database and file store and can be set to move the original file to a ‘completed’ directory. Plus you get error logging. It works perfectly.
You can also run the DMT to remove the old FileStore attachment links, otherwise you’ll have two docs linked to the transaction but one will give you an error.
Also - if you want to keep the same DocType - you can only have one connection type for a given DocType. eg. You can’t have OrderDoc attachments using both FileStore and ECM at the same time. You’ll need a second Doctype or you’ll lose access to all of the docs attached by the previous method (but the links on the transaction will still be there to confuse the users)
They also have the ill named DMS Document Management System Export which also does bulk imports. I had PS try to import a ton of Altec documents, but it got messy, so I just made links to them since they were very old.
I would assume this has been used to do the exact process you want.
I was starting to play around with ‘Batch Import’, but was having difficulty getting it to play nice or parse the .csv file I was feeding it. Thanks for the kick in the right direction, I figured out that my Content Types didn’t match what was in my .CSV, so a ‘Test Import’ produced exactly -nothing- (no logs, no error, no blank results window).
Putting in a correct set of Content Types, yielded something closer to the results I was looking for. Thanks!
yes - sorry, CSV files not XLSX. And yes - the fields need to line up with the content type fields - named exactly the same. Plus you have to have a “PATH” column that points to the original filestore’d document.
Here’s the doc if you don’t have it. I can also provide some examples of SQL to generate XML and CSV files plus their related ECM Workflows.
DocStar Batch Import Guide [v2.1].pdf (1.2 MB)
And don’t forget to mark a solution for the benefit of others searching for answers.
I fully agree with @MikeGross. The Batch Import process using the DocStar ECM client works smoothly when everything is set up correctly!
No need for me to ramble, since I supported DocStar (ECM & IDC) in an environment that did not use Epicor ECM . . . We had Dynamics GP and, historically, I’m an Oracle EBS ERP specialist.
But I want to point out . . . I used the ECM Client’s Batch Import process to import over 100,000 PDFs with tab-delimited TXT text files generated from SQL scripts against Dynamics GP databases. Not all 100k files at once . . . I started each round with about 20-24k PDF files. I used the ECM Client running on my DocStar ECM Windows server (for the ECM Apps tier).
The files got imported fairly quickly – 80 to 90% successful; most failures were due to a weakness in my SQL logic, NOT anything to do with the ECM Batch Import process.
FYI - These were PDFs of historical AP payments with supporting invoices from prior years. And I started to load these PDFs into ECM before we implemented an AP Invoice Automation solution with DocStar & Dynamics GP. So it was important that these PDFs of historical payments with invoices had their own Content Type and Workflow (no approvals needed for historical invoices!) separate from the Content Type & Workflow for AP Invoice Automation.
Also, it was helpful for search retrieval to have some ECM field names in this custom Historical Payments/Invoices content type that matched the required ECM field names in our upcoming AP Invoice Automation solution.
@JerseyEric —^ that right there is my exact same experience. Clean SQL yields clean imports, and I did almost 100k as well. Could not agree with your details more - thanks for adding that.
WOWWW that’s amazing
Thanks, @MikeGross. It’s not that my SQL skills were flawed. But I had to use partial Vendor (payee) Name and Payment # (check #) values from each PDF’s filename for the WHERE criteria to join to transaction records in our Dynamics GP databases! Surprised it worked smoothly for 80 to 90,000 PDFs!
Exactly - and not implying that at all - but we know that SQL on old data is never perfect. - there is always some sort of ‘bad’ data for one reason or another, and you I’m only willing to do so many CASE statements in my SELECT. 
You can only ever get
That’s why you spit out bad data to another bin to look at later
I have a Proof of concept working - Imported a very small test batch of 3 files, and DMT’d the new path into Epicor, which seemed to work fairly well. Then I went to run a batch of 100 and ran into the next snag:
We have a few boiler-plate files (think “T’s and C’s”, and the like) that got attached to several Quotes in Epicor, it seems that the Batch Import ‘locks’ files as it’s importing, and will only allow one instance of the file to be imported per session. It barf’s that the second+ import of the same file within that .CSV file cannot be found.
Is there a work-around for this?
@klincecum, in the case of the ECM Batch Import, I have it do the opposite. The imported PDFs [“good data”] get copied / moved to a Processed subfolder. The remaining PDFs [“bad data”] are left there in the import folder, ready to taunt me!
Do the version/content of these change over time?
Yes - that’s going to make this difficult. Not sure how I would do that.
No - I wouldn’t bring them in.
But I have a different question - why did you
The import process will consume the CSV or XML file with the file path in there and get the file into ECM - then the ECM workflow should have a step using the provided datalink called “Attach to Epicor”. Actually, you may not have this and may need to reach out to Epicor Professional Services for it.
@JerseyEric did you use DMT or use the workflow/datalink combo? I did not use DMT at all.
They are historical, and rarely (if ever) change - the bigger issue is that people have a tendency to move/rename/delete historical data, so importing them is ideal. Being historical, I’m wondering if I can import one copy, write-protect it in ECM, and adjust the links in Epicor to all point to the same copy in ECM - is this a possibility?
I’m just swinging the hammer I know how to use 
(and I’m cheap)
We only have the bare ECM, no workflows or otherwise (and I’m ignorant of what functionality is included in the ‘base’ license of ECM).
The existing attachments are already linked in Epicor, presumably to whatever they should be linked to - it is fairly easy for me to just change the existing link, rather than rebuild them. Since we’re fully on-prem, I am able to mine the new links from SQL server and concatenate a new XFileName string to DMT into Epicor (from the same Excel file I’m generating the Batch Import from).
My new XFileName field that I DMT into Epicor is the “{document GUID};\ECM\Path\to\Document.pdf”, the same as Epicor generates in the Ice.XFileRef table when you attach/store a document into ECM from within Epicor. I imported a custom field with my Batch that included the current XFileRefNum, that I could use to make sure I was extracting the correct GUID from the AAAA0001 database, and VLOOKUP the GUID for my DMT file. There is a slight concern that I haven’t found all of the tables the ECM links are generated into, but initial tests seem to function as desired 
I expect there may be a better way - but this doesn’t feel all that hateful to me for what I expect to be used once.
Hi, @MikeGross. Sorry, I missed your question:
"Did you use DMT or use the workflow/datalink combo?
I did not use DMT at all."
If I understand your question correctly (and I might not), my response is that I used the workflow/datalink combo.
I’m not sure what DMT is. If it’s specific to Epicor ERP, keep in mind that I never used Epicor ERP. I was integrating DocStar ECM with Dynamics GP.
ok - you understood it correctly, I was really just asking to compare notes with @Stedman.
As for DMT - you should have it - check with your CAM, as I think it’s now part of the base license and not additional like it used to be. It’s the Data Management Tool, or data migration tool. It uses the API and has import templates for 100 or so data entities in Kinetic. Everything from loading the GL Chart to importing Sales Orders and Shipments.
I use it for doing mass inventory adjustments (qty and cost), and mass updates to code tables like custom territory, Indirect codes, etc. And mostly, we use it to import he GL journal entry from our payroll company using a powershell script.
Hey - totally understand. Vanilla ECM is missing a bit of the feature set that would really make this work nicely. But I agree with your synopsis - swapping the pointer links may work just fine for a one-time “retro-fit” of the document links.
I’d be careful, because you’re going to have to change a few other things perhaps, like the document type connection value - I think ECM is “5”. And you can’t have the same doc type connecting to two different storage methods at the same time. (We covered this earlier) If you can test it all out and it work for a few docs, then a full rewrite of the links may be the best bet for you in your situation.
But I feel I have to remind you that doing so is going to leave you without support if it goes sideways. Directly writing/altering the database tables (other than UD tables) is strictly verboten (forbidden) by your support agreement.
But I wish you luck, and if you want more on ECM, just reach out.