You can run DMT on the same computer many times....
You can even run the same table many times on the same session of DMT...
You don't need to split it into different work stations...
Marco Vissuet
Systems Engineering
Pacific Contours Corporation
Office (619) 670-3900
Fax (619) 670-1643
mvissuet@...<mailto:
marcov@...>
http://www.pacificcontours.com/
"The information contained herein may be subject to the International Traffic in Arms Regulations (ITAR) Warning: - This document contains data whose export is restricted by the Arms Export Control Act (Title 22, U.S.C., Sec 2751, et seq.) as amended, or the Export Administration Act (Title 50, U.S.C., App 2401 et seq.) as amended. Violations of these export laws are subject to severe criminal and civil penalties. Disseminate in accordance with provisions of DoD Directive 5230.25.
From:
vantage@yahoogroups.com [mailto:
vantage@yahoogroups.com] On Behalf Of Tom J. Christie
Sent: Monday, November 19, 2012 8:01 AM
To:
vantage@yahoogroups.com
Subject: RE: [Vantage] DMT
I was told by support that if you want to disable all the bpms you can rename that folder and they will be ignored. Just name it back and restart Epicor to reactivate them. I haven't confirmed it but it would be convenient!
From:
vantage@yahoogroups.com<mailto:vantage%40yahoogroups.com> [mailto:
vantage@yahoogroups.com<mailto:vantage%40yahoogroups.com>] On Behalf Of Brian Roberts
Sent: Monday, November 19, 2012 8:57 AM
To:
vantage@yahoogroups.com<mailto:vantage%40yahoogroups.com>
Subject: RE: [Vantage] DMT
One detail to add to this excellent list...
Disabling BPMs (via un-checking that enabled box) still leaves the BPM code on your server, and DMT wastes a bit of time loading that code before seeing it is disabled. You might get better performance by deleting the disabled code in the Epicor905\BPMExec\xxxBPM folder. Those files will get recreated when you re-enable the BPM.
Brian.
From:
vantage@yahoogroups.com<mailto:vantage%40yahoogroups.com><mailto:vantage%40yahoogroups.com> [mailto:
vantage@yahoogroups.com<mailto:vantage%40yahoogroups.com><mailto:vantage%40yahoogroups.com>] On Behalf Of Stephen Edginton
Sent: Friday, November 16, 2012 8:11 PM
To:
vantage@yahoogroups.com<mailto:vantage%40yahoogroups.com><mailto:vantage%40yahoogroups.com>
Subject: RE: [Vantage] DMT
Here are some general guidelines for loading with DMT.
If you are on SQL DB perform an Index Rebuild before you load.
Change SQL to Simple vs Full recovery mode
Stop Process and Task Agents while loading
Disable any BPMs or Data Directives (unless you need them for the validation and or automation).
If you do keep then enabled ensure you verify the impact of them on the DMT import process (some BPM wizards do generate memory leaks and degrade performance) watch Appserver memory it should not continue to increase.
If you have windows server 2008, ensure the power mode settings are High Performance
Perform the general Epicor tuning recommendations and on a virtual box ensure you allocate the correct number of vcpus and don't overcommit.
Load with multiple importers if required, this can be as high as 20 instances of DMT to get the required throughput, there is an upper limit on the number of licenses you have in Epicor.
You should stop adding importers when the CPU of the Appserver is at 80%.
When you load with multiple instances ensure you start the same number of Appserver instances as you have DMT import threads, if you don't do this
You will see degrading performance as you add importers. (this is don't in open edge explorer / asbman)
Use the DMT diff tools available from EpicWeb this will split your files for your so you can load into as many as required.
It can also compare an already extracted (and preloaded) CSV file with a newly extracted one and just provide the differences.
This is very useful when piloting and performing regular extracts as you can keep preload and keep your new system in sync before go live leaving only real dynamic data to load over the cut over.
Load in add only mode over add and update as this is faster
Sort data by Company, Plant, Part (primary keys) as this will reduce index fragmentation and allow for DMT to cache better.
Use the latest version of DMT, latest version of Epicor
Run DMT on the Appserver
Script DMT, this doesn't impact on performance but can allow you to script the full load waiting where required and running others in parallel.
Disable on access antivirus and perform a scheduled scan instead for Epicor client and server folders
Review DMT templates and documentation, this include notes such as when loading orders to turn off Ready to process and pass in the EDI flag,
This increases the performance of the base business object and increases the load speed.
if you need higher performance than you can get out of DMT with the standard business objects, Epicor can provide custom extensions that limit the business object validation
I.e. DMT deploys its own business objects dynamically and these are optimised for importing. An example is loading 6,000,000 part / cost and warehouse records @ 7600 RPM with 6 importers.
Kind Regards,
From:
vantage@yahoogroups.com<mailto:vantage%40yahoogroups.com><mailto:vantage%40yahoogroups.com><mailto:vantage%40yahoogroups.com> [mailto:
vantage@yahoogroups.com<mailto:vantage%40yahoogroups.com><mailto:vantage%40yahoogroups.com><mailto:vantage%40yahoogroups.com>] On Behalf Of Scott Litzau
Sent: 16 November 2012 22:25
To:
vantage@yahoogroups.com<mailto:vantage%40yahoogroups.com><mailto:vantage%40yahoogroups.com><mailto:vantage%40yahoogroups.com>
Subject: RE: [Vantage] DMT
Thanks for the info.
I split my extract into 3 different files to start and running it on 3
different workstations. Now it is really crawling, 14 for the first, 9 for
the second and 7 for the third. I think I have to have some
misconfiguration somewhere.
Scott
From:
vantage@yahoogroups.com<mailto:vantage%40yahoogroups.com><mailto:vantage%40yahoogroups.com><mailto:vantage%40yahoogroups.com><mailto:vantage%40yahoogroups.com> [mailto:
vantage@yahoogroups.com<mailto:vantage%40yahoogroups.com><mailto:vantage%40yahoogroups.com><mailto:vantage%40yahoogroups.com><mailto:vantage%40yahoogroups.com>] On Behalf Of
Tom J. Christie
Sent: Friday, November 16, 2012 11:06 AM
To:
vantage@yahoogroups.com<mailto:vantage%40yahoogroups.com><mailto:vantage%40yahoogroups.com><mailto:vantage%40yahoogroups.com><mailto:vantage%40yahoogroups.com>
Subject: RE: [Vantage] DMT
It varies dramatically but that sounds accurate. I have seen some drop to 6
RPM and others upwards of 200 RPMS. It depends on what the server has to do
with each record as they come in.
However, I have also noticed that if you were to load DMT onto another
computer and run it simultaneously you will find both run at the same speed
(essentially doubling your input). I think there is an upper limit but I
haven't found it and I have used 5 at the same time. Just split your CSV
file into chunks and boot it up.
Make sure to save off a backup on the SQL server after each DMT in case you
need to roll back. It is usually quicker to roll back and try again than
correct things that went wrong.
From:
vantage@yahoogroups.com<mailto:vantage%40yahoogroups.com><mailto:vantage%40yahoogroups.com><mailto:vantage%40yahoogroups.com><mailto:vantage%40yahoogroups.com> <mailto:vantage%40yahoogroups.com>
[mailto:
vantage@yahoogroups.com<mailto:vantage%40yahoogroups.com><mailto:vantage%40yahoogroups.com><mailto:vantage%40yahoogroups.com><mailto:vantage%40yahoogroups.com> <mailto:vantage%40yahoogroups.com> ] On
Behalf Of litzer1
Sent: Friday, November 16, 2012 9:25 AM
To:
vantage@yahoogroups.com<mailto:vantage%40yahoogroups.com><mailto:vantage%40yahoogroups.com><mailto:vantage%40yahoogroups.com><mailto:vantage%40yahoogroups.com> <mailto:vantage%40yahoogroups.com>
Subject: [Vantage] DMT
I have gone back and read most of the posts for DMT and just wondering how
many RPM is "normal" understanding everyone's hardware/system setup is
different. Besides what I have read if anyone has found a way to speed it
up. I am running a APP Server and SQL Server virtualized on the same
hardware with SSD Drives with a separate disk array for app, sql data, sql
temp and sql logs.
I am converting to E9 and my usrs want all the old data; Orders,Jobs,...
(Yes, I tried to talk them out of it). Specifically on Order Detail I am
getting anywhere between 30 to 50 RPM and with the amount of records we have
that calculates out to 4 1/2 days of DMTing for that alone then I need to
move onto SO Releases, Jobs (Head, Asm, Opr, OpDtl, Mtl) POs, etc... It will
take a good month I am guessing just to convert the data. That does not seem
right. I have already went through the virtualization tuning guide and made
some adjustments there and now I am going through the SQL Tuning Guides to
attempt to increase performance. I figured with my setup this should go much
faster.
Scott
[Non-text portions of this message have been removed]
[Non-text portions of this message have been removed]
[Non-text portions of this message have been removed]
[Non-text portions of this message have been removed]
[Non-text portions of this message have been removed]
[Non-text portions of this message have been removed]
