Logs that are located in <Epicor installation folder>\Server\Logs.
Look in *.server.log files.
Look in *.server.log files.
We are currently on Vantage 6.1, and planning an upgrade to Epicor 9.05
MSSQL. I have some questions about the data conversion process...
We hired Epicor & a consultant to convert our database by running
through all the various upgrades then the Progress -> MSSQL conversion.
A test conversion done this way worked very well, but took forever.
We're looking a live conversion of 4 days of downtime, and are
struggling with planning this time. For example, we have certain
shipment timeframes we must meet, our shipping process requires Vantage,
volume is 50-60 orders/day, and double entering transactions after the
conversion is done raises a bunch of concerns. Has anybody gone through
this much conversion downtime or double entry and could offer any
advice?
Another option suggested by our consultant is the Data Migration Tool
from Epicor. Reviewing the limited documentation available, it appears
to only come with templates for 60-ish tables, and any additional ones
are custom work. (Don't have the tool yet, or even a quote for it yet).
Has anybody used this tool & your opinion?
We are also looking at purging data in Vantage 6, to reduce the database
size & thus conversion time. We have never used the "Database Purge &
Summarize" utility in Vantage, and our consultant also suggested they
could do custom purging of tables not touched by the standard purge. We
have approx 5 years of historical data, and could probably purge
everything more than 18 months old. For anybody that has done purging,
has this made a big difference?
Any other options to consider?
Vantage 6 database size is 3GB, and the converted Epicor 9 MSSQL
database is 14.5GB. 40 office users & 30 MES stations.
Thanks,
Brian Roberts
Business Systems Analyst
Ground Effects Ltd
[Non-text portions of this message have been removed]
MSSQL. I have some questions about the data conversion process...
We hired Epicor & a consultant to convert our database by running
through all the various upgrades then the Progress -> MSSQL conversion.
A test conversion done this way worked very well, but took forever.
We're looking a live conversion of 4 days of downtime, and are
struggling with planning this time. For example, we have certain
shipment timeframes we must meet, our shipping process requires Vantage,
volume is 50-60 orders/day, and double entering transactions after the
conversion is done raises a bunch of concerns. Has anybody gone through
this much conversion downtime or double entry and could offer any
advice?
Another option suggested by our consultant is the Data Migration Tool
from Epicor. Reviewing the limited documentation available, it appears
to only come with templates for 60-ish tables, and any additional ones
are custom work. (Don't have the tool yet, or even a quote for it yet).
Has anybody used this tool & your opinion?
We are also looking at purging data in Vantage 6, to reduce the database
size & thus conversion time. We have never used the "Database Purge &
Summarize" utility in Vantage, and our consultant also suggested they
could do custom purging of tables not touched by the standard purge. We
have approx 5 years of historical data, and could probably purge
everything more than 18 months old. For anybody that has done purging,
has this made a big difference?
Any other options to consider?
Vantage 6 database size is 3GB, and the converted Epicor 9 MSSQL
database is 14.5GB. 40 office users & 30 MES stations.
Thanks,
Brian Roberts
Business Systems Analyst
Ground Effects Ltd
[Non-text portions of this message have been removed]
Hi Bruce,
the business objects at the .Net level and it can spawn multiple
processes to keep several appServers busy. Because it uses the BOs, it
also enforces all business logic.
Mark W.
>The DMT is different. It does not load the tables directly. It calls
> I'm only assuming the DMT is a lot like the tools that have preceded it. I'll apologize ahead of time if I'm mistaken.
>
the business objects at the .Net level and it can spawn multiple
processes to keep several appServers busy. Because it uses the BOs, it
also enforces all business logic.
Mark W.
The "kind of slow" is one of my (many) worries about this tool...
The documentation I was given has some sample load times (but no info on
hardware setup for those times):
Part 67030 records in 1224 minutes
Bill of Material 147482 records in 4349 minutes (no, that's not a
typo - 72 hours)
It implies you can use 6 clients for these tables, so cut those times by
6 I think. I am unclear on whether you can load both tables at the same
time, but suspect not because of the relationships between tables.
Our table sizes aren't that large, but still... <gulp>.
Brian.
________________________________
From: vantage@yahoogroups.com [mailto:vantage@yahoogroups.com] On Behalf
Of cooner_55421
Sent: Friday, December 03, 2010 4:12 PM
To: vantage@yahoogroups.com
Subject: [Vantage] Re: Data conversion options & Data Migration Tool
Thanks Mark,
Damn... that sounds kind of slow too?
Does anybody use a "kill your mama with an ax" approach and write to
tables directly in 9?
--- In vantage@yahoogroups.com <mailto:vantage%40yahoogroups.com> , Mark
Wonsil <mark_wonsil@...> wrote:
The documentation I was given has some sample load times (but no info on
hardware setup for those times):
Part 67030 records in 1224 minutes
Bill of Material 147482 records in 4349 minutes (no, that's not a
typo - 72 hours)
It implies you can use 6 clients for these tables, so cut those times by
6 I think. I am unclear on whether you can load both tables at the same
time, but suspect not because of the relationships between tables.
Our table sizes aren't that large, but still... <gulp>.
Brian.
________________________________
From: vantage@yahoogroups.com [mailto:vantage@yahoogroups.com] On Behalf
Of cooner_55421
Sent: Friday, December 03, 2010 4:12 PM
To: vantage@yahoogroups.com
Subject: [Vantage] Re: Data conversion options & Data Migration Tool
Thanks Mark,
Damn... that sounds kind of slow too?
Does anybody use a "kill your mama with an ax" approach and write to
tables directly in 9?
--- In vantage@yahoogroups.com <mailto:vantage%40yahoogroups.com> , Mark
Wonsil <mark_wonsil@...> wrote:
>it. I'll apologize ahead of time if I'm mistaken.
> Hi Bruce,
>
> >
> > I'm only assuming the DMT is a lot like the tools that have preceded
> >[Non-text portions of this message have been removed]
>
> The DMT is different. It does not load the tables directly. It calls
> the business objects at the .Net level and it can spawn multiple
> processes to keep several appServers busy. Because it uses the BOs, it
> also enforces all business logic.
>
> Mark W.
>
Good Day Brian:
Sun went live on 11/29
in Epicor 9, 9.05.601B SQL-Unidata
Sun was previously on M2K (Manage-2000) 7.0 SP 5 (Informix)
We used the DMT tool. I think it is faster then Service-Connect, but
slower then the other tool (Pervasive?); but DMT does better Data
checking ( Business logic).
SQL is immediate, but NO businesslogic checking so cannot be used
unless you spend many hours "testing" your import files.
Our run time - 24 hours a day - was 3.5 days. 292,000 Part#s,
400,000 records in BOM
We broke it up into static ( ran 11/18) and Active data ( ran
11/24). Static took 3 days - Active only 0.5 days.
Active consists of only AdjCost, AdjQty, OpenAR, OpenAP, so between
11/18 and 11/24, our old package, M2K ran as normal. Just need to Add
new Part#'s into LIVE before doing Active pass since some of those new
parts may have cost or Qty.
We were lucky in that they did the conversion on our box, so there
was no sending of files, and I was able to edit the error files and get
them ready for re-processing.
DMT tool is about $2,400.
Also, DMT does not like files over 50,000 records ( runs slower and
slower), so we had 12 PART files - but that is good because with 8
processors we could open up 8 sessions of the DMT tool and run at once.
If you try to manually clean up data, do it early - it takes a LOT
of time. Some one has to be really motivated to do it. It is difficult
because the data has to stay live in you current system so if you have
duplicate Customer#s, both need to stay active.
There was one option we did not try. To migrate, the Part-master is
four files. Part, PartRev, PartWhse, and PartPlant - each with 292,000
records. There is a DMT template for 9.04 ( not 9.05) that does all four
files at once and is a lot faster. Rumor had it that they were working
on a 9.05 version.
Don't loose tack of Part Pricelists, and Quotes from Vendors for
parts.
We went from M2K 9 manage 2000) to 9.05.601B. Transition from V6
might be smoother since db is more similar.
Do they still have to convert 6.01 to 9.04 first, then 9.05?
Biggest delay was that once it starts to run you have to be resdy to
fix the "Reprocess" files ( which are csv files with the records that
did not load) because you have to finish one set before beginning the
next. Since we needed to run 24 hours a day, errors had to be fixed as
soon as they occured - if you wait until morning, you will loose 8 hours
of run time.
Feek free to call
len.hartka@...
Leonard C. Hartka, IT Director\ERP Manager
Sun Automation Group
66 Loveton Circle
Sparks, Md. 21152
410-329-3560 ext. 120
410-329-3564 FAX
443-255-7192 Work Cell
len.hartka@...
www.Sunautomation.com <http://www.sunautomation.com/>
________________________________
From: vantage@yahoogroups.com [mailto:vantage@yahoogroups.com] On Behalf
Of Brian Roberts
Sent: Friday, December 03, 2010 3:34 PM
To: vantage@yahoogroups.com
Subject: [Vantage] Data conversion options & Data Migration Tool
We are currently on Vantage 6.1, and planning an upgrade to Epicor 9.05
MSSQL. I have some questions about the data conversion process...
We hired Epicor & a consultant to convert our database by running
through all the various upgrades then the Progress -> MSSQL conversion.
A test conversion done this way worked very well, but took forever.
We're looking a live conversion of 4 days of downtime, and are
struggling with planning this time. For example, we have certain
shipment timeframes we must meet, our shipping process requires Vantage,
volume is 50-60 orders/day, and double entering transactions after the
conversion is done raises a bunch of concerns. Has anybody gone through
this much conversion downtime or double entry and could offer any
advice?
Another option suggested by our consultant is the Data Migration Tool
from Epicor. Reviewing the limited documentation available, it appears
to only come with templates for 60-ish tables, and any additional ones
are custom work. (Don't have the tool yet, or even a quote for it yet).
Has anybody used this tool & your opinion?
We are also looking at purging data in Vantage 6, to reduce the database
size & thus conversion time. We have never used the "Database Purge &
Summarize" utility in Vantage, and our consultant also suggested they
could do custom purging of tables not touched by the standard purge. We
have approx 5 years of historical data, and could probably purge
everything more than 18 months old. For anybody that has done purging,
has this made a big difference?
Any other options to consider?
Vantage 6 database size is 3GB, and the converted Epicor 9 MSSQL
database is 14.5GB. 40 office users & 30 MES stations.
Thanks,
Brian Roberts
Business Systems Analyst
Ground Effects Ltd
[Non-text portions of this message have been removed]
This e-mail and any attachments may contain proprietary and/or confidential information. If you are not the intended recipient, please notify the sender immediately by reply e-mail or at 410-472-2900 and then delete the message without using, disseminating, or copying this message or any portion thereof. With e-mail communications you are urged to protect against viruses.
[Non-text portions of this message have been removed]
Sun went live on 11/29
in Epicor 9, 9.05.601B SQL-Unidata
Sun was previously on M2K (Manage-2000) 7.0 SP 5 (Informix)
We used the DMT tool. I think it is faster then Service-Connect, but
slower then the other tool (Pervasive?); but DMT does better Data
checking ( Business logic).
SQL is immediate, but NO businesslogic checking so cannot be used
unless you spend many hours "testing" your import files.
Our run time - 24 hours a day - was 3.5 days. 292,000 Part#s,
400,000 records in BOM
We broke it up into static ( ran 11/18) and Active data ( ran
11/24). Static took 3 days - Active only 0.5 days.
Active consists of only AdjCost, AdjQty, OpenAR, OpenAP, so between
11/18 and 11/24, our old package, M2K ran as normal. Just need to Add
new Part#'s into LIVE before doing Active pass since some of those new
parts may have cost or Qty.
We were lucky in that they did the conversion on our box, so there
was no sending of files, and I was able to edit the error files and get
them ready for re-processing.
DMT tool is about $2,400.
Also, DMT does not like files over 50,000 records ( runs slower and
slower), so we had 12 PART files - but that is good because with 8
processors we could open up 8 sessions of the DMT tool and run at once.
If you try to manually clean up data, do it early - it takes a LOT
of time. Some one has to be really motivated to do it. It is difficult
because the data has to stay live in you current system so if you have
duplicate Customer#s, both need to stay active.
There was one option we did not try. To migrate, the Part-master is
four files. Part, PartRev, PartWhse, and PartPlant - each with 292,000
records. There is a DMT template for 9.04 ( not 9.05) that does all four
files at once and is a lot faster. Rumor had it that they were working
on a 9.05 version.
Don't loose tack of Part Pricelists, and Quotes from Vendors for
parts.
We went from M2K 9 manage 2000) to 9.05.601B. Transition from V6
might be smoother since db is more similar.
Do they still have to convert 6.01 to 9.04 first, then 9.05?
Biggest delay was that once it starts to run you have to be resdy to
fix the "Reprocess" files ( which are csv files with the records that
did not load) because you have to finish one set before beginning the
next. Since we needed to run 24 hours a day, errors had to be fixed as
soon as they occured - if you wait until morning, you will loose 8 hours
of run time.
Feek free to call
len.hartka@...
Leonard C. Hartka, IT Director\ERP Manager
Sun Automation Group
66 Loveton Circle
Sparks, Md. 21152
410-329-3560 ext. 120
410-329-3564 FAX
443-255-7192 Work Cell
len.hartka@...
www.Sunautomation.com <http://www.sunautomation.com/>
________________________________
From: vantage@yahoogroups.com [mailto:vantage@yahoogroups.com] On Behalf
Of Brian Roberts
Sent: Friday, December 03, 2010 3:34 PM
To: vantage@yahoogroups.com
Subject: [Vantage] Data conversion options & Data Migration Tool
We are currently on Vantage 6.1, and planning an upgrade to Epicor 9.05
MSSQL. I have some questions about the data conversion process...
We hired Epicor & a consultant to convert our database by running
through all the various upgrades then the Progress -> MSSQL conversion.
A test conversion done this way worked very well, but took forever.
We're looking a live conversion of 4 days of downtime, and are
struggling with planning this time. For example, we have certain
shipment timeframes we must meet, our shipping process requires Vantage,
volume is 50-60 orders/day, and double entering transactions after the
conversion is done raises a bunch of concerns. Has anybody gone through
this much conversion downtime or double entry and could offer any
advice?
Another option suggested by our consultant is the Data Migration Tool
from Epicor. Reviewing the limited documentation available, it appears
to only come with templates for 60-ish tables, and any additional ones
are custom work. (Don't have the tool yet, or even a quote for it yet).
Has anybody used this tool & your opinion?
We are also looking at purging data in Vantage 6, to reduce the database
size & thus conversion time. We have never used the "Database Purge &
Summarize" utility in Vantage, and our consultant also suggested they
could do custom purging of tables not touched by the standard purge. We
have approx 5 years of historical data, and could probably purge
everything more than 18 months old. For anybody that has done purging,
has this made a big difference?
Any other options to consider?
Vantage 6 database size is 3GB, and the converted Epicor 9 MSSQL
database is 14.5GB. 40 office users & 30 MES stations.
Thanks,
Brian Roberts
Business Systems Analyst
Ground Effects Ltd
[Non-text portions of this message have been removed]
This e-mail and any attachments may contain proprietary and/or confidential information. If you are not the intended recipient, please notify the sender immediately by reply e-mail or at 410-472-2900 and then delete the message without using, disseminating, or copying this message or any portion thereof. With e-mail communications you are urged to protect against viruses.
[Non-text portions of this message have been removed]
Good Day:
Having just been through it, I think it would be near impossible to
have import files without errors, unless you have use 9.05 for years at
another company.
The inter-relationships are mind boggling.
You need for your import tool to thoroughly check you data - unless
you have relatively few records and simple files.
Just as I was proved totaly wrong when I thought that we could
implement E9 in 4 months ( it took 11) so I was equally un-prepared for
the Migration problems, and I have done this kind of thing before (
1994, 2001, 2004, 2006).
len.hartka@...
________________________________
From: vantage@yahoogroups.com [mailto:vantage@yahoogroups.com] On Behalf
Of cooner_55421
Sent: Friday, December 03, 2010 4:12 PM
To: vantage@yahoogroups.com
Subject: [Vantage] Re: Data conversion options & Data Migration Tool
Thanks Mark,
Damn... that sounds kind of slow too?
Does anybody use a "kill your mama with an ax" approach and write to
tables directly in 9?
--- In vantage@yahoogroups.com <mailto:vantage%40yahoogroups.com> , Mark
Wonsil <mark_wonsil@...> wrote:
[Non-text portions of this message have been removed]
Having just been through it, I think it would be near impossible to
have import files without errors, unless you have use 9.05 for years at
another company.
The inter-relationships are mind boggling.
You need for your import tool to thoroughly check you data - unless
you have relatively few records and simple files.
Just as I was proved totaly wrong when I thought that we could
implement E9 in 4 months ( it took 11) so I was equally un-prepared for
the Migration problems, and I have done this kind of thing before (
1994, 2001, 2004, 2006).
len.hartka@...
________________________________
From: vantage@yahoogroups.com [mailto:vantage@yahoogroups.com] On Behalf
Of cooner_55421
Sent: Friday, December 03, 2010 4:12 PM
To: vantage@yahoogroups.com
Subject: [Vantage] Re: Data conversion options & Data Migration Tool
Thanks Mark,
Damn... that sounds kind of slow too?
Does anybody use a "kill your mama with an ax" approach and write to
tables directly in 9?
--- In vantage@yahoogroups.com <mailto:vantage%40yahoogroups.com> , Mark
Wonsil <mark_wonsil@...> wrote:
>it. I'll apologize ahead of time if I'm mistaken.
> Hi Bruce,
>
> >
> > I'm only assuming the DMT is a lot like the tools that have preceded
> >This e-mail and any attachments may contain proprietary and/or confidential information. If you are not the intended recipient, please notify the sender immediately by reply e-mail or at 410-472-2900 and then delete the message without using, disseminating, or copying this message or any portion thereof. With e-mail communications you are urged to protect against viruses.
>
> The DMT is different. It does not load the tables directly. It calls
> the business objects at the .Net level and it can spawn multiple
> processes to keep several appServers busy. Because it uses the BOs, it
> also enforces all business logic.
>
> Mark W.
>
[Non-text portions of this message have been removed]
DMT uses the business logic because a lot of issues can occur when
trying to map to the tables, it runs as fast as your AppServer's can
process the requests and you can run multiple instances / threads to
speed up processing.
Writing into a SQL database directly will be very quick, but it lets you
import data that is incorrect and invalid.
Epicor's SQL schema has no constraints so all you really get is unique
index checking the rest is down to your own validations.
Take adding a Part to the system as an example, mapping to the SQL
tables requires different mappings depending on data elements, depending
on company and plant configuration options; this is a lot to track
especially if configuration options are being refined.
Using the business objects abstracts this detail but at a cost.
It is not impossible to load via SQL directly, but it takes skill and
time to check and can cause you issues later when processes don't work
as expected because an assumption based on perceived behaviour was
incorrect.
When you have lots of data to import it is always best to load the
static elements ahead of your go live window, then process and net
change of the difference as you get closer to the Go Live and in the Go
Live window. The benefit of this is it allows you to pilot and test with
recognisable data. It is a risk reduction strategy for the go live and
it means your data loading time for the go live window is reduced.
If it takes 73 hours to process some semi static data, you need to load
it ahead of time. You often would not be changing all of your static
items all the time and can process the changes as often as needed to
keep the window small.
In the cases where you require millions of records and the speed is too
slow, Epicor can create a custom business object (optimised for loading)
or you can look at using a direct map if the standard business objects
are too slow. During the initial stages of Data investigation you
establish what entities you will be migrating, the number of entries,
estimated time and the method to use.
It also comes down to internal skill sets, if you have very good SQL
developers able to write your mapping, profile the appservers, keep
track of schema differences between versions and patches and understand
Epicor internals then you may be best with SQL bulk copy or direct sql
importing methods.
If you want to export into CSV or excel files and load into the required
entities then get reprocess files for any items that failed to fix and
reload, script the process and leave to run then use a tool like DMT.
So business object importing is slower and dependant on server sizing,
version and entity being imported into, but it is safe and scriptable
and is done ahead of time in most cases.
Regards,
Stephen
From: vantage@yahoogroups.com [mailto:vantage@yahoogroups.com] On Behalf
Of cooner_55421
Sent: 03 December 2010 21:29
To: vantage@yahoogroups.com
Subject: [Vantage] Re: Data conversion options & Data Migration Tool
Well, I know the argument about using the business logic.
And believe me, I trashed a LOT of test databases before I got all the
related tables and fields correct.
But... 73 hours? Who wants to do that?
I've talked to at least two others this week that are having speed
issues getting data into Vantage.
One Service Connect, the other DMT. You make three.
[Non-text portions of this message have been removed]
trying to map to the tables, it runs as fast as your AppServer's can
process the requests and you can run multiple instances / threads to
speed up processing.
Writing into a SQL database directly will be very quick, but it lets you
import data that is incorrect and invalid.
Epicor's SQL schema has no constraints so all you really get is unique
index checking the rest is down to your own validations.
Take adding a Part to the system as an example, mapping to the SQL
tables requires different mappings depending on data elements, depending
on company and plant configuration options; this is a lot to track
especially if configuration options are being refined.
Using the business objects abstracts this detail but at a cost.
It is not impossible to load via SQL directly, but it takes skill and
time to check and can cause you issues later when processes don't work
as expected because an assumption based on perceived behaviour was
incorrect.
When you have lots of data to import it is always best to load the
static elements ahead of your go live window, then process and net
change of the difference as you get closer to the Go Live and in the Go
Live window. The benefit of this is it allows you to pilot and test with
recognisable data. It is a risk reduction strategy for the go live and
it means your data loading time for the go live window is reduced.
If it takes 73 hours to process some semi static data, you need to load
it ahead of time. You often would not be changing all of your static
items all the time and can process the changes as often as needed to
keep the window small.
In the cases where you require millions of records and the speed is too
slow, Epicor can create a custom business object (optimised for loading)
or you can look at using a direct map if the standard business objects
are too slow. During the initial stages of Data investigation you
establish what entities you will be migrating, the number of entries,
estimated time and the method to use.
It also comes down to internal skill sets, if you have very good SQL
developers able to write your mapping, profile the appservers, keep
track of schema differences between versions and patches and understand
Epicor internals then you may be best with SQL bulk copy or direct sql
importing methods.
If you want to export into CSV or excel files and load into the required
entities then get reprocess files for any items that failed to fix and
reload, script the process and leave to run then use a tool like DMT.
So business object importing is slower and dependant on server sizing,
version and entity being imported into, but it is safe and scriptable
and is done ahead of time in most cases.
Regards,
Stephen
From: vantage@yahoogroups.com [mailto:vantage@yahoogroups.com] On Behalf
Of cooner_55421
Sent: 03 December 2010 21:29
To: vantage@yahoogroups.com
Subject: [Vantage] Re: Data conversion options & Data Migration Tool
Well, I know the argument about using the business logic.
And believe me, I trashed a LOT of test databases before I got all the
related tables and fields correct.
But... 73 hours? Who wants to do that?
I've talked to at least two others this week that are having speed
issues getting data into Vantage.
One Service Connect, the other DMT. You make three.
[Non-text portions of this message have been removed]
Good Day:
Stephen is correct the DMT tool produces error messages and the
"Reprocess.csv's" are very helpful - just fix and re-run.
Also, the DMT tool would then be around for future use- if you buy
it.
At this point, Live on 11/29, I have no good tool to load data into
the files in the future, or to change all the records in one field to a
cetain value.
Sevice-Connect is to complex ( line code essentially), and as
Stephen said, SQL-stuff is Too dangerous because it does not check. For
various reason's, Sun did not buy the DMT tool and I have already
regretted it several times.
len.hartka@sunautom
________________________________
From: vantage@yahoogroups.com [mailto:vantage@yahoogroups.com] On Behalf
Of Stephen Edginton
Sent: Sunday, December 05, 2010 8:35 AM
To: vantage@yahoogroups.com
Subject: RE: [Vantage] Re: Data conversion options & Data Migration Tool
DMT uses the business logic because a lot of issues can occur when
trying to map to the tables, it runs as fast as your AppServer's can
process the requests and you can run multiple instances / threads to
speed up processing.
Writing into a SQL database directly will be very quick, but it lets you
import data that is incorrect and invalid.
Epicor's SQL schema has no constraints so all you really get is unique
index checking the rest is down to your own validations.
Take adding a Part to the system as an example, mapping to the SQL
tables requires different mappings depending on data elements, depending
on company and plant configuration options; this is a lot to track
especially if configuration options are being refined.
Using the business objects abstracts this detail but at a cost.
It is not impossible to load via SQL directly, but it takes skill and
time to check and can cause you issues later when processes don't work
as expected because an assumption based on perceived behaviour was
incorrect.
When you have lots of data to import it is always best to load the
static elements ahead of your go live window, then process and net
change of the difference as you get closer to the Go Live and in the Go
Live window. The benefit of this is it allows you to pilot and test with
recognisable data. It is a risk reduction strategy for the go live and
it means your data loading time for the go live window is reduced.
If it takes 73 hours to process some semi static data, you need to load
it ahead of time. You often would not be changing all of your static
items all the time and can process the changes as often as needed to
keep the window small.
In the cases where you require millions of records and the speed is too
slow, Epicor can create a custom business object (optimised for loading)
or you can look at using a direct map if the standard business objects
are too slow. During the initial stages of Data investigation you
establish what entities you will be migrating, the number of entries,
estimated time and the method to use.
It also comes down to internal skill sets, if you have very good SQL
developers able to write your mapping, profile the appservers, keep
track of schema differences between versions and patches and understand
Epicor internals then you may be best with SQL bulk copy or direct sql
importing methods.
If you want to export into CSV or excel files and load into the required
entities then get reprocess files for any items that failed to fix and
reload, script the process and leave to run then use a tool like DMT.
So business object importing is slower and dependant on server sizing,
version and entity being imported into, but it is safe and scriptable
and is done ahead of time in most cases.
Regards,
Stephen
From: vantage@yahoogroups.com <mailto:vantage%40yahoogroups.com>
[mailto:vantage@yahoogroups.com <mailto:vantage%40yahoogroups.com> ] On
Behalf
Of cooner_55421
Sent: 03 December 2010 21:29
To: vantage@yahoogroups.com <mailto:vantage%40yahoogroups.com>
Subject: [Vantage] Re: Data conversion options & Data Migration Tool
Well, I know the argument about using the business logic.
And believe me, I trashed a LOT of test databases before I got all the
related tables and fields correct.
But... 73 hours? Who wants to do that?
I've talked to at least two others this week that are having speed
issues getting data into Vantage.
One Service Connect, the other DMT. You make three.
[Non-text portions of this message have been removed]
This e-mail and any attachments may contain proprietary and/or confidential information. If you are not the intended recipient, please notify the sender immediately by reply e-mail or at 410-472-2900 and then delete the message without using, disseminating, or copying this message or any portion thereof. With e-mail communications you are urged to protect against viruses.
[Non-text portions of this message have been removed]
Stephen is correct the DMT tool produces error messages and the
"Reprocess.csv's" are very helpful - just fix and re-run.
Also, the DMT tool would then be around for future use- if you buy
it.
At this point, Live on 11/29, I have no good tool to load data into
the files in the future, or to change all the records in one field to a
cetain value.
Sevice-Connect is to complex ( line code essentially), and as
Stephen said, SQL-stuff is Too dangerous because it does not check. For
various reason's, Sun did not buy the DMT tool and I have already
regretted it several times.
len.hartka@sunautom
________________________________
From: vantage@yahoogroups.com [mailto:vantage@yahoogroups.com] On Behalf
Of Stephen Edginton
Sent: Sunday, December 05, 2010 8:35 AM
To: vantage@yahoogroups.com
Subject: RE: [Vantage] Re: Data conversion options & Data Migration Tool
DMT uses the business logic because a lot of issues can occur when
trying to map to the tables, it runs as fast as your AppServer's can
process the requests and you can run multiple instances / threads to
speed up processing.
Writing into a SQL database directly will be very quick, but it lets you
import data that is incorrect and invalid.
Epicor's SQL schema has no constraints so all you really get is unique
index checking the rest is down to your own validations.
Take adding a Part to the system as an example, mapping to the SQL
tables requires different mappings depending on data elements, depending
on company and plant configuration options; this is a lot to track
especially if configuration options are being refined.
Using the business objects abstracts this detail but at a cost.
It is not impossible to load via SQL directly, but it takes skill and
time to check and can cause you issues later when processes don't work
as expected because an assumption based on perceived behaviour was
incorrect.
When you have lots of data to import it is always best to load the
static elements ahead of your go live window, then process and net
change of the difference as you get closer to the Go Live and in the Go
Live window. The benefit of this is it allows you to pilot and test with
recognisable data. It is a risk reduction strategy for the go live and
it means your data loading time for the go live window is reduced.
If it takes 73 hours to process some semi static data, you need to load
it ahead of time. You often would not be changing all of your static
items all the time and can process the changes as often as needed to
keep the window small.
In the cases where you require millions of records and the speed is too
slow, Epicor can create a custom business object (optimised for loading)
or you can look at using a direct map if the standard business objects
are too slow. During the initial stages of Data investigation you
establish what entities you will be migrating, the number of entries,
estimated time and the method to use.
It also comes down to internal skill sets, if you have very good SQL
developers able to write your mapping, profile the appservers, keep
track of schema differences between versions and patches and understand
Epicor internals then you may be best with SQL bulk copy or direct sql
importing methods.
If you want to export into CSV or excel files and load into the required
entities then get reprocess files for any items that failed to fix and
reload, script the process and leave to run then use a tool like DMT.
So business object importing is slower and dependant on server sizing,
version and entity being imported into, but it is safe and scriptable
and is done ahead of time in most cases.
Regards,
Stephen
From: vantage@yahoogroups.com <mailto:vantage%40yahoogroups.com>
[mailto:vantage@yahoogroups.com <mailto:vantage%40yahoogroups.com> ] On
Behalf
Of cooner_55421
Sent: 03 December 2010 21:29
To: vantage@yahoogroups.com <mailto:vantage%40yahoogroups.com>
Subject: [Vantage] Re: Data conversion options & Data Migration Tool
Well, I know the argument about using the business logic.
And believe me, I trashed a LOT of test databases before I got all the
related tables and fields correct.
But... 73 hours? Who wants to do that?
I've talked to at least two others this week that are having speed
issues getting data into Vantage.
One Service Connect, the other DMT. You make three.
[Non-text portions of this message have been removed]
This e-mail and any attachments may contain proprietary and/or confidential information. If you are not the intended recipient, please notify the sender immediately by reply e-mail or at 410-472-2900 and then delete the message without using, disseminating, or copying this message or any portion thereof. With e-mail communications you are urged to protect against viruses.
[Non-text portions of this message have been removed]
How did you move your open Jobs? Did you use DMT for this or move them
manually?
We have around 2000 open jobs and we are trying to figure out the best
way to move them when we go from 6.1 to 9.x.
Thanks,
Jim Moore
From: vantage@yahoogroups.com [mailto:vantage@yahoogroups.com] On Behalf
Of Len Hartka
Sent: Friday, December 03, 2010 3:26 PM
To: vantage@yahoogroups.com
Subject: RE: [Vantage] Data conversion options & Data Migration Tool
Good Day Brian:
Sun went live on 11/29
in Epicor 9, 9.05.601B SQL-Unidata
Sun was previously on M2K (Manage-2000) 7.0 SP 5 (Informix)
We used the DMT tool. I think it is faster then Service-Connect, but
slower then the other tool (Pervasive?); but DMT does better Data
checking ( Business logic).
SQL is immediate, but NO businesslogic checking so cannot be used
unless you spend many hours "testing" your import files.
Our run time - 24 hours a day - was 3.5 days. 292,000 Part#s,
400,000 records in BOM
We broke it up into static ( ran 11/18) and Active data ( ran
11/24). Static took 3 days - Active only 0.5 days.
Active consists of only AdjCost, AdjQty, OpenAR, OpenAP, so between
11/18 and 11/24, our old package, M2K ran as normal. Just need to Add
new Part#'s into LIVE before doing Active pass since some of those new
parts may have cost or Qty.
We were lucky in that they did the conversion on our box, so there
was no sending of files, and I was able to edit the error files and get
them ready for re-processing.
DMT tool is about $2,400.
Also, DMT does not like files over 50,000 records ( runs slower and
slower), so we had 12 PART files - but that is good because with 8
processors we could open up 8 sessions of the DMT tool and run at once.
If you try to manually clean up data, do it early - it takes a LOT
of time. Some one has to be really motivated to do it. It is difficult
because the data has to stay live in you current system so if you have
duplicate Customer#s, both need to stay active.
There was one option we did not try. To migrate, the Part-master is
four files. Part, PartRev, PartWhse, and PartPlant - each with 292,000
records. There is a DMT template for 9.04 ( not 9.05) that does all four
files at once and is a lot faster. Rumor had it that they were working
on a 9.05 version.
Don't loose tack of Part Pricelists, and Quotes from Vendors for
parts.
We went from M2K 9 manage 2000) to 9.05.601B. Transition from V6
might be smoother since db is more similar.
Do they still have to convert 6.01 to 9.04 first, then 9.05?
Biggest delay was that once it starts to run you have to be resdy to
fix the "Reprocess" files ( which are csv files with the records that
did not load) because you have to finish one set before beginning the
next. Since we needed to run 24 hours a day, errors had to be fixed as
soon as they occured - if you wait until morning, you will loose 8 hours
of run time.
Feek free to call
len.hartka@... <mailto:len.hartka%40sunautomation.com>
Leonard C. Hartka, IT Director\ERP Manager
Sun Automation Group
66 Loveton Circle
Sparks, Md. 21152
410-329-3560 ext. 120
410-329-3564 FAX
443-255-7192 Work Cell
len.hartka@... <mailto:len.hartka%40sunautomation.com>
www.Sunautomation.com <http://www.sunautomation.com/>
________________________________
From: vantage@yahoogroups.com <mailto:vantage%40yahoogroups.com>
[mailto:vantage@yahoogroups.com <mailto:vantage%40yahoogroups.com> ] On
Behalf
Of Brian Roberts
Sent: Friday, December 03, 2010 3:34 PM
To: vantage@yahoogroups.com <mailto:vantage%40yahoogroups.com>
Subject: [Vantage] Data conversion options & Data Migration Tool
We are currently on Vantage 6.1, and planning an upgrade to Epicor 9.05
MSSQL. I have some questions about the data conversion process...
We hired Epicor & a consultant to convert our database by running
through all the various upgrades then the Progress -> MSSQL conversion.
A test conversion done this way worked very well, but took forever.
We're looking a live conversion of 4 days of downtime, and are
struggling with planning this time. For example, we have certain
shipment timeframes we must meet, our shipping process requires Vantage,
volume is 50-60 orders/day, and double entering transactions after the
conversion is done raises a bunch of concerns. Has anybody gone through
this much conversion downtime or double entry and could offer any
advice?
Another option suggested by our consultant is the Data Migration Tool
from Epicor. Reviewing the limited documentation available, it appears
to only come with templates for 60-ish tables, and any additional ones
are custom work. (Don't have the tool yet, or even a quote for it yet).
Has anybody used this tool & your opinion?
We are also looking at purging data in Vantage 6, to reduce the database
size & thus conversion time. We have never used the "Database Purge &
Summarize" utility in Vantage, and our consultant also suggested they
could do custom purging of tables not touched by the standard purge. We
have approx 5 years of historical data, and could probably purge
everything more than 18 months old. For anybody that has done purging,
has this made a big difference?
Any other options to consider?
Vantage 6 database size is 3GB, and the converted Epicor 9 MSSQL
database is 14.5GB. 40 office users & 30 MES stations.
Thanks,
Brian Roberts
Business Systems Analyst
Ground Effects Ltd
[Non-text portions of this message have been removed]
This e-mail and any attachments may contain proprietary and/or
confidential information. If you are not the intended recipient, please
notify the sender immediately by reply e-mail or at 410-472-2900 and
then delete the message without using, disseminating, or copying this
message or any portion thereof. With e-mail communications you are urged
to protect against viruses.
[Non-text portions of this message have been removed]
[Non-text portions of this message have been removed]
manually?
We have around 2000 open jobs and we are trying to figure out the best
way to move them when we go from 6.1 to 9.x.
Thanks,
Jim Moore
From: vantage@yahoogroups.com [mailto:vantage@yahoogroups.com] On Behalf
Of Len Hartka
Sent: Friday, December 03, 2010 3:26 PM
To: vantage@yahoogroups.com
Subject: RE: [Vantage] Data conversion options & Data Migration Tool
Good Day Brian:
Sun went live on 11/29
in Epicor 9, 9.05.601B SQL-Unidata
Sun was previously on M2K (Manage-2000) 7.0 SP 5 (Informix)
We used the DMT tool. I think it is faster then Service-Connect, but
slower then the other tool (Pervasive?); but DMT does better Data
checking ( Business logic).
SQL is immediate, but NO businesslogic checking so cannot be used
unless you spend many hours "testing" your import files.
Our run time - 24 hours a day - was 3.5 days. 292,000 Part#s,
400,000 records in BOM
We broke it up into static ( ran 11/18) and Active data ( ran
11/24). Static took 3 days - Active only 0.5 days.
Active consists of only AdjCost, AdjQty, OpenAR, OpenAP, so between
11/18 and 11/24, our old package, M2K ran as normal. Just need to Add
new Part#'s into LIVE before doing Active pass since some of those new
parts may have cost or Qty.
We were lucky in that they did the conversion on our box, so there
was no sending of files, and I was able to edit the error files and get
them ready for re-processing.
DMT tool is about $2,400.
Also, DMT does not like files over 50,000 records ( runs slower and
slower), so we had 12 PART files - but that is good because with 8
processors we could open up 8 sessions of the DMT tool and run at once.
If you try to manually clean up data, do it early - it takes a LOT
of time. Some one has to be really motivated to do it. It is difficult
because the data has to stay live in you current system so if you have
duplicate Customer#s, both need to stay active.
There was one option we did not try. To migrate, the Part-master is
four files. Part, PartRev, PartWhse, and PartPlant - each with 292,000
records. There is a DMT template for 9.04 ( not 9.05) that does all four
files at once and is a lot faster. Rumor had it that they were working
on a 9.05 version.
Don't loose tack of Part Pricelists, and Quotes from Vendors for
parts.
We went from M2K 9 manage 2000) to 9.05.601B. Transition from V6
might be smoother since db is more similar.
Do they still have to convert 6.01 to 9.04 first, then 9.05?
Biggest delay was that once it starts to run you have to be resdy to
fix the "Reprocess" files ( which are csv files with the records that
did not load) because you have to finish one set before beginning the
next. Since we needed to run 24 hours a day, errors had to be fixed as
soon as they occured - if you wait until morning, you will loose 8 hours
of run time.
Feek free to call
len.hartka@... <mailto:len.hartka%40sunautomation.com>
Leonard C. Hartka, IT Director\ERP Manager
Sun Automation Group
66 Loveton Circle
Sparks, Md. 21152
410-329-3560 ext. 120
410-329-3564 FAX
443-255-7192 Work Cell
len.hartka@... <mailto:len.hartka%40sunautomation.com>
www.Sunautomation.com <http://www.sunautomation.com/>
________________________________
From: vantage@yahoogroups.com <mailto:vantage%40yahoogroups.com>
[mailto:vantage@yahoogroups.com <mailto:vantage%40yahoogroups.com> ] On
Behalf
Of Brian Roberts
Sent: Friday, December 03, 2010 3:34 PM
To: vantage@yahoogroups.com <mailto:vantage%40yahoogroups.com>
Subject: [Vantage] Data conversion options & Data Migration Tool
We are currently on Vantage 6.1, and planning an upgrade to Epicor 9.05
MSSQL. I have some questions about the data conversion process...
We hired Epicor & a consultant to convert our database by running
through all the various upgrades then the Progress -> MSSQL conversion.
A test conversion done this way worked very well, but took forever.
We're looking a live conversion of 4 days of downtime, and are
struggling with planning this time. For example, we have certain
shipment timeframes we must meet, our shipping process requires Vantage,
volume is 50-60 orders/day, and double entering transactions after the
conversion is done raises a bunch of concerns. Has anybody gone through
this much conversion downtime or double entry and could offer any
advice?
Another option suggested by our consultant is the Data Migration Tool
from Epicor. Reviewing the limited documentation available, it appears
to only come with templates for 60-ish tables, and any additional ones
are custom work. (Don't have the tool yet, or even a quote for it yet).
Has anybody used this tool & your opinion?
We are also looking at purging data in Vantage 6, to reduce the database
size & thus conversion time. We have never used the "Database Purge &
Summarize" utility in Vantage, and our consultant also suggested they
could do custom purging of tables not touched by the standard purge. We
have approx 5 years of historical data, and could probably purge
everything more than 18 months old. For anybody that has done purging,
has this made a big difference?
Any other options to consider?
Vantage 6 database size is 3GB, and the converted Epicor 9 MSSQL
database is 14.5GB. 40 office users & 30 MES stations.
Thanks,
Brian Roberts
Business Systems Analyst
Ground Effects Ltd
[Non-text portions of this message have been removed]
This e-mail and any attachments may contain proprietary and/or
confidential information. If you are not the intended recipient, please
notify the sender immediately by reply e-mail or at 410-472-2900 and
then delete the message without using, disseminating, or copying this
message or any portion thereof. With e-mail communications you are urged
to protect against viruses.
[Non-text portions of this message have been removed]
[Non-text portions of this message have been removed]
Jim,
I have worked with a couple clients that we implemented with the DMT software.
Job were brought over - 3,500 of them. It was a matter of working carefully through the hirearchy of the tables.
DMT can be slow to bring the data over, I have run multiple sessions during conversion, which seemed to help some. Just be careful of writing to the same records.
As a practice we have gone with DMT versus using the upgrade path from 6.1 to 9.04 or 9.05 because it gives you more flexibility on how BOOs and BOMs come over.
Bruce Larson
Senior Solution Architect
alternative Technology Partners
Web Site: www.alttechpartners.com
Cell: (763) 486-0030
Fax: (763) 447-3571
Email: bruce.larson@...
I have worked with a couple clients that we implemented with the DMT software.
Job were brought over - 3,500 of them. It was a matter of working carefully through the hirearchy of the tables.
DMT can be slow to bring the data over, I have run multiple sessions during conversion, which seemed to help some. Just be careful of writing to the same records.
As a practice we have gone with DMT versus using the upgrade path from 6.1 to 9.04 or 9.05 because it gives you more flexibility on how BOOs and BOMs come over.
Bruce Larson
Senior Solution Architect
alternative Technology Partners
Web Site: www.alttechpartners.com
Cell: (763) 486-0030
Fax: (763) 447-3571
Email: bruce.larson@...
--- In vantage@yahoogroups.com, "Moore, Jim (Anniston)" <james.moore@...> wrote:
>
> How did you move your open Jobs? Did you use DMT for this or move them
> manually?
>
> We have around 2000 open jobs and we are trying to figure out the best
> way to move them when we go from 6.1 to 9.x.
>
>
>
> Thanks,
>
> Jim Moore
>
>
>
>
>
> From: vantage@yahoogroups.com [mailto:vantage@yahoogroups.com] On Behalf
> Of Len Hartka
> Sent: Friday, December 03, 2010 3:26 PM
> To: vantage@yahoogroups.com
> Subject: RE: [Vantage] Data conversion options & Data Migration Tool
>
>
>
>
>
> Good Day Brian:
>
> Sun went live on 11/29
> in Epicor 9, 9.05.601B SQL-Unidata
> Sun was previously on M2K (Manage-2000) 7.0 SP 5 (Informix)
>
> We used the DMT tool. I think it is faster then Service-Connect, but
> slower then the other tool (Pervasive?); but DMT does better Data
> checking ( Business logic).
> SQL is immediate, but NO businesslogic checking so cannot be used
> unless you spend many hours "testing" your import files.
>
> Our run time - 24 hours a day - was 3.5 days. 292,000 Part#s,
> 400,000 records in BOM
>
> We broke it up into static ( ran 11/18) and Active data ( ran
> 11/24). Static took 3 days - Active only 0.5 days.
> Active consists of only AdjCost, AdjQty, OpenAR, OpenAP, so between
> 11/18 and 11/24, our old package, M2K ran as normal. Just need to Add
> new Part#'s into LIVE before doing Active pass since some of those new
> parts may have cost or Qty.
> We were lucky in that they did the conversion on our box, so there
> was no sending of files, and I was able to edit the error files and get
> them ready for re-processing.
> DMT tool is about $2,400.
> Also, DMT does not like files over 50,000 records ( runs slower and
> slower), so we had 12 PART files - but that is good because with 8
> processors we could open up 8 sessions of the DMT tool and run at once.
> If you try to manually clean up data, do it early - it takes a LOT
> of time. Some one has to be really motivated to do it. It is difficult
> because the data has to stay live in you current system so if you have
> duplicate Customer#s, both need to stay active.
>
> There was one option we did not try. To migrate, the Part-master is
> four files. Part, PartRev, PartWhse, and PartPlant - each with 292,000
> records. There is a DMT template for 9.04 ( not 9.05) that does all four
> files at once and is a lot faster. Rumor had it that they were working
> on a 9.05 version.
>
> Don't loose tack of Part Pricelists, and Quotes from Vendors for
> parts.
>
> We went from M2K 9 manage 2000) to 9.05.601B. Transition from V6
> might be smoother since db is more similar.
> Do they still have to convert 6.01 to 9.04 first, then 9.05?
>
> Biggest delay was that once it starts to run you have to be resdy to
> fix the "Reprocess" files ( which are csv files with the records that
> did not load) because you have to finish one set before beginning the
> next. Since we needed to run 24 hours a day, errors had to be fixed as
> soon as they occured - if you wait until morning, you will loose 8 hours
> of run time.
>
> Feek free to call
>
> len.hartka@... <mailto:len.hartka%40sunautomation.com>
> Leonard C. Hartka, IT Director\ERP Manager
> Sun Automation Group
> 66 Loveton Circle
> Sparks, Md. 21152
> 410-329-3560 ext. 120
> 410-329-3564 FAX
> 443-255-7192 Work Cell
> len.hartka@... <mailto:len.hartka%40sunautomation.com>
>
> www.Sunautomation.com <http://www.sunautomation.com/>
>
> ________________________________
>
> From: vantage@yahoogroups.com <mailto:vantage%40yahoogroups.com>
> [mailto:vantage@yahoogroups.com <mailto:vantage%40yahoogroups.com> ] On
> Behalf
> Of Brian Roberts
> Sent: Friday, December 03, 2010 3:34 PM
> To: vantage@yahoogroups.com <mailto:vantage%40yahoogroups.com>
> Subject: [Vantage] Data conversion options & Data Migration Tool
>
> We are currently on Vantage 6.1, and planning an upgrade to Epicor 9.05
> MSSQL. I have some questions about the data conversion process...
>
> We hired Epicor & a consultant to convert our database by running
> through all the various upgrades then the Progress -> MSSQL conversion.
> A test conversion done this way worked very well, but took forever.
> We're looking a live conversion of 4 days of downtime, and are
> struggling with planning this time. For example, we have certain
> shipment timeframes we must meet, our shipping process requires Vantage,
> volume is 50-60 orders/day, and double entering transactions after the
> conversion is done raises a bunch of concerns. Has anybody gone through
> this much conversion downtime or double entry and could offer any
> advice?
>
> Another option suggested by our consultant is the Data Migration Tool
> from Epicor. Reviewing the limited documentation available, it appears
> to only come with templates for 60-ish tables, and any additional ones
> are custom work. (Don't have the tool yet, or even a quote for it yet).
> Has anybody used this tool & your opinion?
>
> We are also looking at purging data in Vantage 6, to reduce the database
> size & thus conversion time. We have never used the "Database Purge &
> Summarize" utility in Vantage, and our consultant also suggested they
> could do custom purging of tables not touched by the standard purge. We
> have approx 5 years of historical data, and could probably purge
> everything more than 18 months old. For anybody that has done purging,
> has this made a big difference?
>
> Any other options to consider?
>
> Vantage 6 database size is 3GB, and the converted Epicor 9 MSSQL
> database is 14.5GB. 40 office users & 30 MES stations.
>
> Thanks,
>
> Brian Roberts
>
> Business Systems Analyst
>
> Ground Effects Ltd
>
> [Non-text portions of this message have been removed]
>
> This e-mail and any attachments may contain proprietary and/or
> confidential information. If you are not the intended recipient, please
> notify the sender immediately by reply e-mail or at 410-472-2900 and
> then delete the message without using, disseminating, or copying this
> message or any portion thereof. With e-mail communications you are urged
> to protect against viruses.
>
> [Non-text portions of this message have been removed]
>
>
>
>
>
> [Non-text portions of this message have been removed]
>
I recently purchased DMT to assist in our conversion from Avante to E9. I am getting an error when it run stating "Unknown error - see appserver logs". Not sure what log it is talking about. Has anyone else seen this error.
Thanks in advance,
Bob Jessop
Misonix Inc.
Thanks in advance,
Bob Jessop
Misonix Inc.
I have seen that error if there are blank rows at the end of the file
--- In vantage@yahoogroups.com, "Robert Jessop" <rjj@...> wrote:
>
> I recently purchased DMT to assist in our conversion from Avante to E9. I am getting an error when it run stating "Unknown error - see appserver logs". Not sure what log it is talking about. Has anyone else seen this error.
>
> Thanks in advance,
>
> Bob Jessop
> Misonix Inc.
>