Chris,
I am not sure of your hardware but I have recently had a lot of issues with the app servers for Vantage slowing down the overall system too often. After several calls I have learned the following:
1) Bad BAQ or reports will drag down a system
2) Runaway/Hung processes
I felt a system should be able to run for a decent period of time w/o reboots or restarts..I may be old school but that is my belief.
I have learned from several support calls as of late that you need to monitor the PIDs that relate to the _proapsv.exe*32 of Vantage. Chances are they are the app servers that get hung and will effect system performance.
You can see them in the PET tool, DB app server, status, details. You should see requests for the suspicous PID/process change pretty often if the process is working normally. You will need to monitor them for a bit to get a feel for your particular system.
If any of those are running for more than say 60 minutes and taking constant excessive cpu cycles chances are good they are hosed..Again, that number is based on my hardware, you mileage may vary. I have dual quad 3.0Ghz with 16 gig of memory and RAID 10 Dell PE 2900 that is about 3 years old. For point of reference my MRP for the largest company takes 4 minutes to run)
One thing is certain, when you see these processes running for longer than expected you will see performance begin to take a hit and it just keeps taking performance down over a relatively short time.
It is normal for these processes to take up about 13%, my system, of cpu while they are processing requests from users but they should "free up" after a period of time and drop there cpu time and usage back to zero briefly. The average time for me is about 30 minutes before they free up for more requests. I know if I see one
running an hour chances are it has an issue.
So what do I do when I see them run longer?
I check the system monitor, all process display checked and search out the PID to ensure it is truly hung. (Also check the DB app server, status, detail to see if requests increment)
If I determine it's hung I kill it via task manager on the server..after trying to kill it from the user session..if possible. Sometimes the user can not kill it even though they initiated it...don't ask me why. ;)
If it(app server) is needed Vantage cranks it up again. Worse case the user running the process, BAQ or report will lose that.
It's been a long learning curve for me as I do not have the luxury of a dedicated DB person. I am a generalist and get to be bad at everything. ;)
I don't know if this is best practices or not but I am used to seeing DB servers run for months at a good performance level before a reboot.(SQL servers) I know there are memory leaks in Windows 2003 R2 and a reboot does some good from time to time.
I am aiming for at least a month for now and if my insight from Vantage tech support is correct I should be able to run it longer w/o going through kicking everyone off and rebooting more than once a month to start with. I know there are temp files that need to get cleaned out occassionaly but many are related to those runaway processes. My theory is if you can kill those early then the temp
files should stay small and managable.
The above is based on a couple deep conversation with some decent tech people I have engaged at Epicor..the above is my summation of those conversations.
Tom Bosko
Miner Enterprises
I am not sure of your hardware but I have recently had a lot of issues with the app servers for Vantage slowing down the overall system too often. After several calls I have learned the following:
1) Bad BAQ or reports will drag down a system
2) Runaway/Hung processes
I felt a system should be able to run for a decent period of time w/o reboots or restarts..I may be old school but that is my belief.
I have learned from several support calls as of late that you need to monitor the PIDs that relate to the _proapsv.exe*32 of Vantage. Chances are they are the app servers that get hung and will effect system performance.
You can see them in the PET tool, DB app server, status, details. You should see requests for the suspicous PID/process change pretty often if the process is working normally. You will need to monitor them for a bit to get a feel for your particular system.
If any of those are running for more than say 60 minutes and taking constant excessive cpu cycles chances are good they are hosed..Again, that number is based on my hardware, you mileage may vary. I have dual quad 3.0Ghz with 16 gig of memory and RAID 10 Dell PE 2900 that is about 3 years old. For point of reference my MRP for the largest company takes 4 minutes to run)
One thing is certain, when you see these processes running for longer than expected you will see performance begin to take a hit and it just keeps taking performance down over a relatively short time.
It is normal for these processes to take up about 13%, my system, of cpu while they are processing requests from users but they should "free up" after a period of time and drop there cpu time and usage back to zero briefly. The average time for me is about 30 minutes before they free up for more requests. I know if I see one
running an hour chances are it has an issue.
So what do I do when I see them run longer?
I check the system monitor, all process display checked and search out the PID to ensure it is truly hung. (Also check the DB app server, status, detail to see if requests increment)
If I determine it's hung I kill it via task manager on the server..after trying to kill it from the user session..if possible. Sometimes the user can not kill it even though they initiated it...don't ask me why. ;)
If it(app server) is needed Vantage cranks it up again. Worse case the user running the process, BAQ or report will lose that.
It's been a long learning curve for me as I do not have the luxury of a dedicated DB person. I am a generalist and get to be bad at everything. ;)
I don't know if this is best practices or not but I am used to seeing DB servers run for months at a good performance level before a reboot.(SQL servers) I know there are memory leaks in Windows 2003 R2 and a reboot does some good from time to time.
I am aiming for at least a month for now and if my insight from Vantage tech support is correct I should be able to run it longer w/o going through kicking everyone off and rebooting more than once a month to start with. I know there are temp files that need to get cleaned out occassionaly but many are related to those runaway processes. My theory is if you can kill those early then the temp
files should stay small and managable.
The above is based on a couple deep conversation with some decent tech people I have engaged at Epicor..the above is my summation of those conversations.
Tom Bosko
Miner Enterprises
--- In vantage@yahoogroups.com, "tbmuzic" <tombosko@...> wrote:
>
> That does NOT seem right..that is too frequent.
>
> Tom
>
> --- In vantage@yahoogroups.com, Chris Thompson <chriselectrix@> wrote:
> >
> > We used to have to kick everyone out of the system mid-day every 7-10 days
> > because the speed would be dramatically reduced and it would get to a point
> > where it wouldn't let us log in.
> >
> > 1- Each night we stop and start our Apps (our of hours) on a scheduled task.
> >
> > 2- Each weekend we stop the apps, stop the db, clear out any temp files, clear
> > out any old report .xml files (which can be upto 8GB each), defrag the server,
> > start the db, start the app.
> >
> > Since doing the above, we have never had any dramatic slow downs and we have
> > never had any situations where it wouldnt let us log in.
> >
> > I would recommend the above steps based on what we have found.
> >
> > It's interesting to hear that we were not alone with the issues.
> >
> > If you want any of the scripts, I will be happy to share them.
> >
> > Thanks.
> >
> >
> >
> >
> > ________________________________
> > From: tbmuzic <tombosko@>
> > To: vantage@yahoogroups.com
> > Sent: Wed, 1 September, 2010 16:59:17
> > Subject: [Vantage] How often do you restart/reboot your Vantage server?
> >
> > ÂÂ
> > Wondering what the consensus is. It seems I have to restart/reboot mine every 3
> > weeks before processes get "hung" or error messages like "Maximum number of
> > client connections has been reached." I've had multiple calls into Epicor and
> > they have gone over my system setting several times and are within their specs
> > for 100 full users and 100 MES users. I average 90 full and 60 MES with plenty
> > of reusable licenses available)
> > Yet still I am forced to restart the database during the middle of the day
> > occasionally to rid these issues. Very frustrating to say the least, they can
> > never really pinpoint why these issues keep reoccurring.
> >
> > So, do you schedule reboots/restarts of your Vantage servers/databases? If so
> > how often? Weekly? biweekly? Monthly?
> >
> > What is best practice here to keep the system up 99.99% w/o intraday issues?
> >
> > Thanks!
> >
> > Tom Bosko
> > Miner Enterprises
> >
> >
> >
> >
> >
> >
> >
> > [Non-text portions of this message have been removed]
> >
>