Any advice for ways to speed up the DMT processing?
Uploading Job Material now, (about 42,000 records) and the estimate is about 4 hours.
Our setup uses two servers. One as the App server, and the other for the SQL DB.
Currently running the DMT program on the App Server box (that’s where the DMT.EXE is).
Calvin
You can’t … DMT goes as fast as your epicor allows
-Jose Gomez
Imports that don’t require Group IDs can be split into multiple files and run simultaneously
I was going to ask about running multiple instances of DMT, but was afraid I’d get my wrist slapped for suggesting such a dangerous thing.
Can’t be any harm in running another instance of DMT for a totally unrelated table.
You don’t even have to do that…run another instance…although you can…but even within the same session you should be able to run an upload to another table.
M. Manasa Reddy
P: 703.471.7145 x454
manasa@euclidsys.commailto:manasa@euclidsys.com
Wow … I just assumed the processing was “modal”. Never even tried opening another tab while it was processing.
Thanks!
I have run up to 10 at a time without any issue.
When I was trained on DMT the method I was shown was to split into as many files as long as the records per minute did not drop. I used two very fast PCs with SSDs to process the files. I split most large files into 3 and did 6 on a few.
I have a macro that splits and renames spreadsheets if you want tit.
Greg
Note that the latest update to DMT includes the ability to split a single import file into multiple imports under a single DMT session.
I haven’t tested this feature yet myself but it is intended to speed up the import process by dividing the file into multiple single imports.
I agree.
Depending on the version of Epicor the bottleneck is usually the app servers. So multiple uploads is the way to go until the records per minute starts to drop.
There used to be a utility you could download which will split a CSV file into as many as you want. Unfortunately it no longer appears to be on Epicweb.
There is a free tool here: csv tool free download - SourceForge .
I’ve not tried it, but it appears to do the same thing.
Regards
Steve Johnson MBCS CITP CMA (Certified Epicor Consultant)
Quick question regarding the files. [quote=“gpayne, post:8, topic:35846”]
with SSDs to process the files.
[/quote]
Does the DMT program load the upload file into the PC’s memory? Or does it fetch records from the upload file while it is running?
If it does, then does the location of the file play a part? All my upload files are on the V8 server (the BAQ Exports put them there), which is a different server than the E10 App server or SQL box.
Do I need to stay hands off of the upload file while DMT is processing it? Is it okay to have it open in Excel while DMT is processing?
Calvin
It loads the file into memory.
Calvin Dekker
[CBIEMAIL]http://www.codabears.com/
(630) 672-7688 x1484
[Untitled-1] <https://www.facebook.com/CodaBearsInc> [Untitled-2] <http://www.linkedin.com/company/codabears-inc> [Untitled-3] <https://twitter.com/CodaBearsInc> [G+] <http://google.com/+CodaBearsIncRoselle>
[EPICOR]
Catch CodaBears Communications!http://eepurl.com/brRMQ5
The odd thing I’ve noticed about splitting big files (like BOMs) is that
the later instances run faster than the first, so I put more records in the
third or fourth than the first. Maybe the first is the one that has the
business objects open. Don’t know.
There must be diminishing returns there somewhere, but I’ve had six running
and didn’t notice a penalty in terms of RPM.
There is a penalty if you’re running a hosted system, or over Citrix,
Remote Desktop, etc. AND your source file is on your local PC. The error
and completion logging goes where the source file is. If that’s not on the
same network as Epicor, you’ll pay a time penalty there when DMT writes a
completion record for each record uploaded.
Joe