8.03 Performance Monitoring and Troubleshooting (2011 Edition)

Vantage 8.03.408A Progress

Using shop tracker as outlined took approximately 14 seconds for 10,000 Records.

We are running two Intel Xeon X5650 processors with 32Gb ram on Windows 2008R2
Enterprise Server (PowerEdge R710). The database is on a PERC H700 Raid 10
controller with four SSD drives. No virtualization.



________________________________
From: Zac Jason Woodward <zac@...>
To: "vantage@yahoogroups.com" <vantage@yahoogroups.com>
Sent: Tue, February 22, 2011 9:16:27 AM
Subject: RE: [Vantage] Re: 8.03 Performance Monitoring and Troubleshooting (2011
Edition)

Â
Using shop tracker as outlined below returning 10,000 rows for me takes
approximately 15 - 17 seconds.

That being said I would like to point out that I am using a completely virtual
environment with the data stored on a SAN. I am curiously if you guys are on
dedicated hardware or virtualized?

"Zac" Jason Woodward
Network Administrator
Intermountain Electronics, Inc.
O: 877-544-2291
M: 435-820-6515
F: 435-637-9601
www.ie-corp.com

Creating customer confidence through extraordinary service and experienced
industry experts.

From: vantage@yahoogroups.com [mailto:vantage@yahoogroups.com] On Behalf Of
cubcrafters_it
Sent: Monday, February 21, 2011 1:56 PM
To: vantage@yahoogroups.com
Subject: [Vantage] Re: 8.03 Performance Monitoring and Troubleshooting (2011
Edition)

Yeah, ~10.8 to return 10,000 rows to my client (Win 7 VM). Haven't yet tried
directly on the server.

I was just trying to find something that other folks would have data against
without getting too heavy into the customizations.

Anyone else feel like posting some figures? :)

--- In vantage@yahoogroups.com<mailto:vantage%40yahoogroups.com>, Waffqle
<waffqle@...> wrote:
>
> How many records were you returning on that search? If you were getting 10k
> records a 10sec search wouldn't be considered too bad.
>
> On Mon, Feb 21, 2011 at 11:42 AM, cubcrafters_it <
> jason.navarrete@...> wrote:
>
> >
> >
> > So is anyone feeling particularly generous on this wonderful Monday to try
> > running the previously outlined 'benchmark'?
> >
> > Or can someone think of another simple tests that might be repeatable
> > amongst various systems? It'd be handy to get some figures to compare the
> > speed of different setups.
> >
> >
> > --- In vantage@yahoogroups.com<mailto:vantage%40yahoogroups.com>,
>"cubcrafters_it" <jason.navarrete@>
> > wrote:
> > >
> > > So my perflogs didn't turn up anything completely out of the ordinary,
> > but there are a few small things I'll follow up on.
> > >
> > > Still need to schedule some time to restart the task and process agents,
> > so we'll see what that does.
> > >
> > > Here's what I timed:
> > > Production Management -> Job Management -> General Operations -> Shop
> > Tracker
> > > Picked a resource group with a lot of labor rows, then over on the labor
> > tab, clicked 'labor' to bring up Labor Detail Search. Changed the options to
> > return 10,000 rows max, and then clicked search. Timed the time it took to
> > return those rows on my test client. Just used a stopwatch rather than using
> > a trace, as I didn't want that additional overhead.
> > >
> > > Average over 5 runs: 10.8 seconds (on 8.03.409A, Progress 10.1B)
> > >
> > > If anyone feels like running this test, I'd greatly appreciate it. Or
> > please suggest another benchmark that would be repeatable on other systems.
> > >
> > >
> > >
> > > --- In vantage@yahoogroups.com<mailto:vantage%40yahoogroups.com>, Waffqle
><waffqle@> wrote:
> > > >
> > > > I'd bump it up a bit and see what happens.You'll need to restart the
> > task
> > > > agent and the process agent. This won't throw kick everyone off, but
> > it'll
> > > > stop print jobs and service connect and possibly other stuff.
> > > >
> > > > PERC should be good to go.
> > > >
> > > > Have you timed any non-print/submitted processes? Adding lines or
> > saving
> > > > orders/jobs/etc?
> > > > You can run a trace and use the time info to do a rough benchmark.
> > That's a
> > > > good way to tell whether it's the server or the client.
> > > >
> > > >
> > > >
> > > > On Thu, Feb 10, 2011 at 6:00 PM, cubcrafters_it <
> > > > jason.navarrete@> wrote:
> > > >
> > > > >
> > > > >
> > > > > You may be onto something important here: my Processing Delay is
> > currently
> > > > > set to 2 (default is 10?). I've never seen my server on any sort of
> > > > > sustained processor load, however, so could that still be an issue?
> > Can I
> > > > > bump up the delay without having to restart anything?
> > > > >
> > > > > And as far as the RAID is concerned, it's on a PERC 5i with a BBU, so
> > I
> > > > > would hope it's a halfway decent setup.
> > > > >
> > > > > Still waiting on my performance logs to analyze this afternoon.
> > > > >
> > > > > -Jason
> > > > >
> > > > >
> > > > > --- In vantage@yahoogroups.com<mailto:vantage%40yahoogroups.com>,
>Waffqle <waffqle@> wrote:
> > > > > >
> > > > > > Print Jobs other 'Submitted Jobs' aren't always the best gauge of
> > > > > > performance as they use a polling system.
> > > > > > It may take you 20 seconds to see the print job even if it only
> > took the
> > > > > > system 2 seconds to produce it. The other 18 seconds, you're just
> > waiting
> > > > > on
> > > > > > the System Agent to poll.
> > > > > >
> > > > > > You can reduce the polling time in the System Agent. Whatever
> > 'Processing
> > > > > > Delay' is set to is the maximum time it will take the system to
> > begin a
> > > > > > submitted job.
> > > > > >
> > > > > > Note, be careful not to set it too fast lest it start stepping on
> > itself.
> > > > > 10
> > > > > > seconds is probably a safe starting point if it's currently set to
> > a
> > > > > large
> > > > > > number. You might be able to get away with 5 seconds if your server
> > is
> > > > > under
> > > > > > a pretty low load. A bit of trial and error will get you to the
> > right
> > > > > > number.
> > > > > >
> > > > > > Also, if your system has been locking up and you find that the
> > processing
> > > > > > delay is set especially low ...you probably just found the culprit.
> > > > > >
> > > > > >
> > > > > > Lastly, I didn't think to ask, but you're not running a white box
> > or
> > > > > cheap
> > > > > > (god forbid software) RAID? If your RAID isn't being handled by a
> > proper
> > > > > > controller, all bets are off as far as performance goes.
> > > > > >
> > > > > >
> > > > > > On Thu, Feb 10, 2011 at 2:00 PM, cubcrafters_it <
> > > > > > jason.navarrete@> wrote:
> > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > @cooner
> > > > > > > Yeah, 15 regular/15 MES seems like it ought to be fairly
> > lightweight,
> > > > > and
> > > > > > > it's probably less than 10 regular users on a regular basis.
> > > > > > >
> > > > > > > What sort of transactions are you talking about and how would I
> > check?
> > > > > The
> > > > > > > bulk of any sort of interaction with the system would be around
> > > > > > > manufacturing/production, if that makes any sense.
> > > > > > >
> > > > > > > We're using the same box since I started 7 months ago; it's
> > probably
> > > > > been
> > > > > > > formatted once somewhere along the lines, but it's got its fair
> > share
> > > > > of
> > > > > > > updates. We've got a new server waiting for Epicor 9, but again,
> > I have
> > > > > a
> > > > > > > nagging feeling the current server might be underutilized.
> > > > > > >
> > > > > > > I didn't have too much trouble going through the db tuning guide,
> > and
> > > > > so
> > > > > > > far it looks like we had a few things over-configured, so I'm not
> > sure
> > > > > what
> > > > > > > that means for us either.
> > > > > > >
> > > > > > > @waffqle
> > > > > > > We're not virtualized.
> > > > > > >
> > > > > > > I stared at perlogs for a few minutes this morning and the max I
> > saw
> > > > > > > Current Disk Queue Length was 2. I'm running Microsoft's PAL tool
> > (has
> > > > > > > anyone else ever used this? I just found it today) this afternoon
> > and
> > > > > > > logging data for 5 hours, to see if that turns up anything
> > useful.
> > > > > > >
> > > > > > > No termserv. Most of our MES computers are old, but even some of
> > our
> > > > > new
> > > > > > > machines seem to take longer than it should for some operations
> > or
> > > > > > > particular reports (but what's normal?).
> > > > > > >
> > > > > > > Network issues are a whole other ball of wax, but should be
> > readily
> > > > > > > identifiable if I run a client instance directly on the server.
> > We've
> > > > > got
> > > > > > > entry-level server Gb switches.
> > > > > > >
> > > > > > > Out of curiosity, can someone tell me how long it takes them to
> > print
> > > > > > > preview a Menu Security Report for APRP4100 (1099 Processing),
> > > > > unchecking
> > > > > > > 'New Page Per User/Group' but leaving all the other defaults? I
> > get a
> > > > > 19
> > > > > > > page report, and it took an average of 9.3 seconds to run
> > directly on
> > > > > the
> > > > > > > server.
> > > > > > >
> > > > > > > Thanks for all the help so far. The cumulative information here
> > has
> > > > > been
> > > > > > > more beneficial than anything I've gotten elsewhere.
> > > > > > >
> > > > > > >
> > > > > > > --- In vantage@yahoogroups.com<mailto:vantage%40yahoogroups.com>,
>"cooner_55421" <cooner_55421@>
> > wrote:
> > > > > > > >
> > > > > > > > Hi,
> > > > > > > >
> > > > > > > > Looks like you have 15 regular users and 15 MES users?
> > > > > > > > That doesn't sound like it would be a big strain on your
> > system.
> > > > > > > >
> > > > > > > > Just wondering about transactions. Amount, type, etc...?
> > > > > > > > History of your install. Fresh or a long history of upgrades on
> > the
> > > > > same
> > > > > > > box?
> > > > > > > >
> > > > > > > > >the Progress DB tuning guide,
> > > > > > > > >but I should probably go back through that again
> > > > > > > > I find the guide a little confusing.
> > > > > > > > I called Epicor Support several times - Tech/Install (option 3
> > & 1)
> > > > > > > > I got lucky and connected with a guy who put in some extra
> > effort.
> > > > > > > > I don't have his name handy right now, anyway...
> > > > > > > > He was able to explain a couple of things to me I never would
> > have
> > > > > > > figured out from the guide.
> > > > > > > >
> > > > > > > --- In vantage@yahoogroups.com<mailto:vantage%40yahoogroups.com>,
>Waffqle <waffqle@> wrote:
> > > > > > > >
> > > > > > > > I'm assuming you aren't virtualized, correct me if I'm wrong.
> > > > > > > >
> > > > > > > > If your buffer hits are already at >98% then more RAM isn't
> > going to
> > > > > get
> > > > > > > you
> > > > > > > > anywhere.
> > > > > > > >
> > > > > > > > Perfmon should be able to tell you pretty definitively whether
> > or not
> > > > > > > your
> > > > > > > > disks are your issue. (Though I would tend to assume they're
> > the
> > > > > > > > bottleneck.) Load it up and monitor your Average and Current
> > Disk
> > > > > Queue
> > > > > > > > Length. If your queue length is higher than "1.5 x # of
> > spindles(4)"
> > > > > your
> > > > > > > > disks are holding you back.
> > > > > > > > It's tempting to look at Read/Write and other IO measurements,
> > but at
> > > > > the
> > > > > > > > end of the day if your queue is long, your disks are
> > overloaded.
> > > > > > > >
> > > > > > > > Also have you looked at your clients? The Vantage/Epicor client
> > is
> > > > > not a
> > > > > > > > lightweight piece of software. If you're running a term serv,
> > check
> > > > > the
> > > > > > > load
> > > > > > > > on that as well.
> > > > > > > >
> > > > > > > > Look at your network out as well. As you're only running 15
> > users, I
> > > > > > > imagine
> > > > > > > > your network isn't too huge or overloaded; but it's certainly
> > worth
> > > > > > > > checking. Check the load on your switches/routers.
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > On Thu, Feb 10, 2011 at 11:28 AM, cubcrafters_it <
> > > > > > > > jason.navarrete@> wrote:
> > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > We're only using 5 out of our 8 GB of RAM. Can I devote more
> > of it
> > > > > to
> > > > > > > the
> > > > > > > > > DB or App servers? Would that be beneficial? Buffer Hits are
> > > > > already at
> > > > > > > > > ~98%.
> > > > > > > > >
> > > > > > > > > Disks are 15k SAS 3Gb/s drives, but yes, everything Epicor is
> > on
> > > > > the
> > > > > > > same
> > > > > > > > > RAID 10. Does anyone happen to have documentation on moving
> > temp
> > > > > dirs
> > > > > > > to
> > > > > > > > > different spindles? I'll also try monitoring disk access to
> > see
> > > > > what
> > > > > > > that
> > > > > > > > > tells me.
> > > > > > > > >
> > > > > > > > > I'm a little hesitant to throw hardware at the problem when
> > I'm not
> > > > > > > > > completely convinced that it will solve the problem
> > (partially
> > > > > because
> > > > > > > I'm
> > > > > > > > > cheap).
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > --- In
>vantage@yahoogroups.com<mailto:vantage%40yahoogroups.com>, "markbetts66"
><markbetts66@>
> > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > Hi,
> > > > > > > > > >
> > > > > > > > > > You may want to try more RAM. What type of disks are you
> > using in
> > > > > > > your
> > > > > > > > > server? When you say RAID10(EPICOR) do you mean DB,
> > Appserver,
> > > > > progress
> > > > > > > Temp
> > > > > > > > > Dir and report Temp Dir on the same disk?
> > > > > > > > > >
> > > > > > > > > > --- In
>vantage@yahoogroups.com<mailto:vantage%40yahoogroups.com>, "cubcrafters_it"
> > > > > <jason.navarrete@>
> > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > First off, very glad to have found this group.
> > > > > > > > > > >
> > > > > > > > > > > Secondly:
> > > > > > > > > > > Vista 8.03.409A on Progress 10.1B
> > > > > > > > > > > Dual Core Xeon 5160 3GHz, 8GB RAM
> > > > > > > > > > > RAID 1 (OS) + 10 (Epicor)
> > > > > > > > > > > d1: 2.21GB, b1: 341MB
> > > > > > > > > > > ~15 Standard + ~15 MES Users
> > > > > > > > > > >
> > > > > > > > > > > We're currently plagued with performance/speed/lockup
> > issues,
> > > > > and
> > > > > > > would
> > > > > > > > > love to be able to mitigate as much of it as possible in our
> > > > > journey to
> > > > > > > > > Epicor 9.
> > > > > > > > > > >
> > > > > > > > > > > I ran a dump and load at the beginning of the year, and
> > while
> > > > > it
> > > > > > > took
> > > > > > > > > the scatter factor down significantly, it didn't appear to
> > make a
> > > > > > > noticeable
> > > > > > > > > different in the end user experience (but it made me feel a
> > whole
> > > > > lot
> > > > > > > > > better). I also did look at the Progress DB tuning guide, but
> > I
> > > > > should
> > > > > > > > > probably go back through that again.
> > > > > > > > > > >
> > > > > > > > > > > Right now I'm lucky if we go a couple of weeks without
> > > > > something
> > > > > > > > > bringing down the system. I feel like I don't have any
> > visibility
> > > > > into
> > > > > > > > > what's causing the issues.
> > > > > > > > > > >
> > > > > > > > > > > I'd like to be able to monitor the server with something
> > like
> > > > > > > Nagios to
> > > > > > > > > help identify if something is out of order, but it's
> > frustrating
> > > > > that I
> > > > > > > > > rarely see the CPU get higher than 25%. There are some simple
> > > > > processes
> > > > > > > > > (searching for all Report Styles or Data Defs) that on
> > occasion
> > > > > seem to
> > > > > > > take
> > > > > > > > > a whole lot longer than they ought to.
> > > > > > > > > > >
> > > > > > > > > > > We're trying to work with a big Epicor guru for some paid
> > > > > advice,
> > > > > > > but
> > > > > > > > > it's been a little tricky to coordinate scheduling so far.
> > > > > > > > > > >
> > > > > > > > > > > Any thoughts, ideas, or suggestions would be helpful at
> > this
> > > > > point.
> > > > > > > I'm
> > > > > > > > > only 6 months in supporting the system, so just about
> > everything is
> > > > > new
> > > > > > > > > information for me.
> > > > > > > > > > >
> > > > > > > > > > > Thanks in advance!
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > [Non-text portions of this message have been removed]
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > > [Non-text portions of this message have been removed]
> > > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > > >
> > > > [Non-text portions of this message have been removed]
> > > >
> > >
> >
> >
> >
>
>
> [Non-text portions of this message have been removed]
>

[Non-text portions of this message have been removed]







[Non-text portions of this message have been removed]
First off, very glad to have found this group.

Secondly:
Vista 8.03.409A on Progress 10.1B
Dual Core Xeon 5160 3GHz, 8GB RAM
RAID 1 (OS) + 10 (Epicor)
d1: 2.21GB, b1: 341MB
~15 Standard + ~15 MES Users

We're currently plagued with performance/speed/lockup issues, and would love to be able to mitigate as much of it as possible in our journey to Epicor 9.

I ran a dump and load at the beginning of the year, and while it took the scatter factor down significantly, it didn't appear to make a noticeable different in the end user experience (but it made me feel a whole lot better). I also did look at the Progress DB tuning guide, but I should probably go back through that again.

Right now I'm lucky if we go a couple of weeks without something bringing down the system. I feel like I don't have any visibility into what's causing the issues.

I'd like to be able to monitor the server with something like Nagios to help identify if something is out of order, but it's frustrating that I rarely see the CPU get higher than 25%. There are some simple processes (searching for all Report Styles or Data Defs) that on occasion seem to take a whole lot longer than they ought to.

We're trying to work with a big Epicor guru for some paid advice, but it's been a little tricky to coordinate scheduling so far.

Any thoughts, ideas, or suggestions would be helpful at this point. I'm only 6 months in supporting the system, so just about everything is new information for me.

Thanks in advance!
We had similar issues with 8.03 for almost 2 years and multiple calls to tech support didn't fix it. We upgraded our switches, the server (24gb memory), clients, did dump and load, etc. Nothing seemed to help.

Then one day I accidentally ran across the "Process Set" icon. I looked into our scheduled processes that were running intermittently. Per our implementation consultant's recommendation, we scheduled three processes to run every 15 minutes (Fill Shop Capacity Records, Production Planner Process, and Build Project Analysis). It turned out, by just scheduling all three process on a 15 minute interval, all three of those processes were running simultaneously. Also, the frequency was far too high and they were just finishing up when the next schedule was beginning. I switched them over to a Process Set (which launched them consecutively) and reduced the frequency to twice per day and the server started behaving normally.

--- In vantage@yahoogroups.com, "cubcrafters_it" <jason.navarrete@...> wrote:
>
> First off, very glad to have found this group.
>
> Secondly:
> Vista 8.03.409A on Progress 10.1B
> Dual Core Xeon 5160 3GHz, 8GB RAM
> RAID 1 (OS) + 10 (Epicor)
> d1: 2.21GB, b1: 341MB
> ~15 Standard + ~15 MES Users
>
> We're currently plagued with performance/speed/lockup issues, and would love to be able to mitigate as much of it as possible in our journey to Epicor 9.
>
> I ran a dump and load at the beginning of the year, and while it took the scatter factor down significantly, it didn't appear to make a noticeable different in the end user experience (but it made me feel a whole lot better). I also did look at the Progress DB tuning guide, but I should probably go back through that again.
>
> Right now I'm lucky if we go a couple of weeks without something bringing down the system. I feel like I don't have any visibility into what's causing the issues.
>
> I'd like to be able to monitor the server with something like Nagios to help identify if something is out of order, but it's frustrating that I rarely see the CPU get higher than 25%. There are some simple processes (searching for all Report Styles or Data Defs) that on occasion seem to take a whole lot longer than they ought to.
>
> We're trying to work with a big Epicor guru for some paid advice, but it's been a little tricky to coordinate scheduling so far.
>
> Any thoughts, ideas, or suggestions would be helpful at this point. I'm only 6 months in supporting the system, so just about everything is new information for me.
>
> Thanks in advance!
>
Bruce,

Thanks for the tips. I looked through the Process Set and associated scheduled tasks, and we've got one process (Backflush) that runs every 5 minutes. That looks to be the only thing that runs on a regular sub-daily interval.

It doesn't however, appear to take more than a second to run and it doesn't ever overlap with itself, but I suppose it's possible that it could still be causing problems.

Any other tips on general diagnosis? How can I turn up verbosity on various things to get more visibility on errors?

--- In vantage@yahoogroups.com, "brucewbrannan" <bruce.brannan@...> wrote:
>
> We had similar issues with 8.03 for almost 2 years and multiple calls to tech support didn't fix it. We upgraded our switches, the server (24gb memory), clients, did dump and load, etc. Nothing seemed to help.
>
> Then one day I accidentally ran across the "Process Set" icon. I looked into our scheduled processes that were running intermittently. Per our implementation consultant's recommendation, we scheduled three processes to run every 15 minutes (Fill Shop Capacity Records, Production Planner Process, and Build Project Analysis). It turned out, by just scheduling all three process on a 15 minute interval, all three of those processes were running simultaneously. Also, the frequency was far too high and they were just finishing up when the next schedule was beginning. I switched them over to a Process Set (which launched them consecutively) and reduced the frequency to twice per day and the server started behaving normally.
>
> --- In vantage@yahoogroups.com, "cubcrafters_it" <jason.navarrete@> wrote:
> >
> > First off, very glad to have found this group.
> >
> > Secondly:
> > Vista 8.03.409A on Progress 10.1B
> > Dual Core Xeon 5160 3GHz, 8GB RAM
> > RAID 1 (OS) + 10 (Epicor)
> > d1: 2.21GB, b1: 341MB
> > ~15 Standard + ~15 MES Users
> >
> > We're currently plagued with performance/speed/lockup issues, and would love to be able to mitigate as much of it as possible in our journey to Epicor 9.
> >
> > I ran a dump and load at the beginning of the year, and while it took the scatter factor down significantly, it didn't appear to make a noticeable different in the end user experience (but it made me feel a whole lot better). I also did look at the Progress DB tuning guide, but I should probably go back through that again.
> >
> > Right now I'm lucky if we go a couple of weeks without something bringing down the system. I feel like I don't have any visibility into what's causing the issues.
> >
> > I'd like to be able to monitor the server with something like Nagios to help identify if something is out of order, but it's frustrating that I rarely see the CPU get higher than 25%. There are some simple processes (searching for all Report Styles or Data Defs) that on occasion seem to take a whole lot longer than they ought to.
> >
> > We're trying to work with a big Epicor guru for some paid advice, but it's been a little tricky to coordinate scheduling so far.
> >
> > Any thoughts, ideas, or suggestions would be helpful at this point. I'm only 6 months in supporting the system, so just about everything is new information for me.
> >
> > Thanks in advance!
> >
>
I remember trapping some errors but it was through the help of Epicor's tech support; somewhere in the work folders a log file showed a job that wasn't completing it's scheduling process. This problem wasn't really degrading system performance, it was keeping the scheduling process from completing.

A quick test of your backflush process would be to disable its schedule for a while and see how the system performs. In our case, all users immediately saw a huge difference. It sounds like yours is something else altogether. When our processes run at 12:15, the system degrades for 5-10 minutes and then returns to normal. At least it happens when most people are at lunch. Sorry I can't be of more help.

--- In vantage@yahoogroups.com, "cubcrafters_it" <jason.navarrete@...> wrote:
>
>
>
> Bruce,
>
> Thanks for the tips. I looked through the Process Set and associated scheduled tasks, and we've got one process (Backflush) that runs every 5 minutes. That looks to be the only thing that runs on a regular sub-daily interval.
>
> It doesn't however, appear to take more than a second to run and it doesn't ever overlap with itself, but I suppose it's possible that it could still be causing problems.
>
> Any other tips on general diagnosis? How can I turn up verbosity on various things to get more visibility on errors?
>
> --- In vantage@yahoogroups.com, "brucewbrannan" <bruce.brannan@> wrote:
> >
> > We had similar issues with 8.03 for almost 2 years and multiple calls to tech support didn't fix it. We upgraded our switches, the server (24gb memory), clients, did dump and load, etc. Nothing seemed to help.
> >
> > Then one day I accidentally ran across the "Process Set" icon. I looked into our scheduled processes that were running intermittently. Per our implementation consultant's recommendation, we scheduled three processes to run every 15 minutes (Fill Shop Capacity Records, Production Planner Process, and Build Project Analysis). It turned out, by just scheduling all three process on a 15 minute interval, all three of those processes were running simultaneously. Also, the frequency was far too high and they were just finishing up when the next schedule was beginning. I switched them over to a Process Set (which launched them consecutively) and reduced the frequency to twice per day and the server started behaving normally.
> >
> > --- In vantage@yahoogroups.com, "cubcrafters_it" <jason.navarrete@> wrote:
> > >
> > > First off, very glad to have found this group.
> > >
> > > Secondly:
> > > Vista 8.03.409A on Progress 10.1B
> > > Dual Core Xeon 5160 3GHz, 8GB RAM
> > > RAID 1 (OS) + 10 (Epicor)
> > > d1: 2.21GB, b1: 341MB
> > > ~15 Standard + ~15 MES Users
> > >
> > > We're currently plagued with performance/speed/lockup issues, and would love to be able to mitigate as much of it as possible in our journey to Epicor 9.
> > >
> > > I ran a dump and load at the beginning of the year, and while it took the scatter factor down significantly, it didn't appear to make a noticeable different in the end user experience (but it made me feel a whole lot better). I also did look at the Progress DB tuning guide, but I should probably go back through that again.
> > >
> > > Right now I'm lucky if we go a couple of weeks without something bringing down the system. I feel like I don't have any visibility into what's causing the issues.
> > >
> > > I'd like to be able to monitor the server with something like Nagios to help identify if something is out of order, but it's frustrating that I rarely see the CPU get higher than 25%. There are some simple processes (searching for all Report Styles or Data Defs) that on occasion seem to take a whole lot longer than they ought to.
> > >
> > > We're trying to work with a big Epicor guru for some paid advice, but it's been a little tricky to coordinate scheduling so far.
> > >
> > > Any thoughts, ideas, or suggestions would be helpful at this point. I'm only 6 months in supporting the system, so just about everything is new information for me.
> > >
> > > Thanks in advance!
> > >
> >
>
Hi,

You may want to try more RAM. What type of disks are you using in your server? When you say RAID10(EPICOR) do you mean DB, Appserver, progress Temp Dir and report Temp Dir on the same disk?

--- In vantage@yahoogroups.com, "cubcrafters_it" <jason.navarrete@...> wrote:
>
> First off, very glad to have found this group.
>
> Secondly:
> Vista 8.03.409A on Progress 10.1B
> Dual Core Xeon 5160 3GHz, 8GB RAM
> RAID 1 (OS) + 10 (Epicor)
> d1: 2.21GB, b1: 341MB
> ~15 Standard + ~15 MES Users
>
> We're currently plagued with performance/speed/lockup issues, and would love to be able to mitigate as much of it as possible in our journey to Epicor 9.
>
> I ran a dump and load at the beginning of the year, and while it took the scatter factor down significantly, it didn't appear to make a noticeable different in the end user experience (but it made me feel a whole lot better). I also did look at the Progress DB tuning guide, but I should probably go back through that again.
>
> Right now I'm lucky if we go a couple of weeks without something bringing down the system. I feel like I don't have any visibility into what's causing the issues.
>
> I'd like to be able to monitor the server with something like Nagios to help identify if something is out of order, but it's frustrating that I rarely see the CPU get higher than 25%. There are some simple processes (searching for all Report Styles or Data Defs) that on occasion seem to take a whole lot longer than they ought to.
>
> We're trying to work with a big Epicor guru for some paid advice, but it's been a little tricky to coordinate scheduling so far.
>
> Any thoughts, ideas, or suggestions would be helpful at this point. I'm only 6 months in supporting the system, so just about everything is new information for me.
>
> Thanks in advance!
>
We're only using 5 out of our 8 GB of RAM. Can I devote more of it to the DB or App servers? Would that be beneficial? Buffer Hits are already at ~98%.

Disks are 15k SAS 3Gb/s drives, but yes, everything Epicor is on the same RAID 10. Does anyone happen to have documentation on moving temp dirs to different spindles? I'll also try monitoring disk access to see what that tells me.

I'm a little hesitant to throw hardware at the problem when I'm not completely convinced that it will solve the problem (partially because I'm cheap).


--- In vantage@yahoogroups.com, "markbetts66" <markbetts66@...> wrote:
>
> Hi,
>
> You may want to try more RAM. What type of disks are you using in your server? When you say RAID10(EPICOR) do you mean DB, Appserver, progress Temp Dir and report Temp Dir on the same disk?
>
> --- In vantage@yahoogroups.com, "cubcrafters_it" <jason.navarrete@> wrote:
> >
> > First off, very glad to have found this group.
> >
> > Secondly:
> > Vista 8.03.409A on Progress 10.1B
> > Dual Core Xeon 5160 3GHz, 8GB RAM
> > RAID 1 (OS) + 10 (Epicor)
> > d1: 2.21GB, b1: 341MB
> > ~15 Standard + ~15 MES Users
> >
> > We're currently plagued with performance/speed/lockup issues, and would love to be able to mitigate as much of it as possible in our journey to Epicor 9.
> >
> > I ran a dump and load at the beginning of the year, and while it took the scatter factor down significantly, it didn't appear to make a noticeable different in the end user experience (but it made me feel a whole lot better). I also did look at the Progress DB tuning guide, but I should probably go back through that again.
> >
> > Right now I'm lucky if we go a couple of weeks without something bringing down the system. I feel like I don't have any visibility into what's causing the issues.
> >
> > I'd like to be able to monitor the server with something like Nagios to help identify if something is out of order, but it's frustrating that I rarely see the CPU get higher than 25%. There are some simple processes (searching for all Report Styles or Data Defs) that on occasion seem to take a whole lot longer than they ought to.
> >
> > We're trying to work with a big Epicor guru for some paid advice, but it's been a little tricky to coordinate scheduling so far.
> >
> > Any thoughts, ideas, or suggestions would be helpful at this point. I'm only 6 months in supporting the system, so just about everything is new information for me.
> >
> > Thanks in advance!
> >
>
I'm assuming you aren't virtualized, correct me if I'm wrong.

If your buffer hits are already at >98% then more RAM isn't going to get you
anywhere.

Perfmon should be able to tell you pretty definitively whether or not your
disks are your issue. (Though I would tend to assume they're the
bottleneck.) Load it up and monitor your Average and Current Disk Queue
Length. If your queue length is higher than "1.5 x # of spindles(4)" your
disks are holding you back.
It's tempting to look at Read/Write and other IO measurements, but at the
end of the day if your queue is long, your disks are overloaded.

Also have you looked at your clients? The Vantage/Epicor client is not a
lightweight piece of software. If you're running a term serv, check the load
on that as well.

Look at your network out as well. As you're only running 15 users, I imagine
your network isn't too huge or overloaded; but it's certainly worth
checking. Check the load on your switches/routers.



On Thu, Feb 10, 2011 at 11:28 AM, cubcrafters_it <
jason.navarrete@...> wrote:

>
>
> We're only using 5 out of our 8 GB of RAM. Can I devote more of it to the
> DB or App servers? Would that be beneficial? Buffer Hits are already at
> ~98%.
>
> Disks are 15k SAS 3Gb/s drives, but yes, everything Epicor is on the same
> RAID 10. Does anyone happen to have documentation on moving temp dirs to
> different spindles? I'll also try monitoring disk access to see what that
> tells me.
>
> I'm a little hesitant to throw hardware at the problem when I'm not
> completely convinced that it will solve the problem (partially because I'm
> cheap).
>
>
> --- In vantage@yahoogroups.com, "markbetts66" <markbetts66@...> wrote:
> >
> > Hi,
> >
> > You may want to try more RAM. What type of disks are you using in your
> server? When you say RAID10(EPICOR) do you mean DB, Appserver, progress Temp
> Dir and report Temp Dir on the same disk?
> >
> > --- In vantage@yahoogroups.com, "cubcrafters_it" <jason.navarrete@>
> wrote:
> > >
> > > First off, very glad to have found this group.
> > >
> > > Secondly:
> > > Vista 8.03.409A on Progress 10.1B
> > > Dual Core Xeon 5160 3GHz, 8GB RAM
> > > RAID 1 (OS) + 10 (Epicor)
> > > d1: 2.21GB, b1: 341MB
> > > ~15 Standard + ~15 MES Users
> > >
> > > We're currently plagued with performance/speed/lockup issues, and would
> love to be able to mitigate as much of it as possible in our journey to
> Epicor 9.
> > >
> > > I ran a dump and load at the beginning of the year, and while it took
> the scatter factor down significantly, it didn't appear to make a noticeable
> different in the end user experience (but it made me feel a whole lot
> better). I also did look at the Progress DB tuning guide, but I should
> probably go back through that again.
> > >
> > > Right now I'm lucky if we go a couple of weeks without something
> bringing down the system. I feel like I don't have any visibility into
> what's causing the issues.
> > >
> > > I'd like to be able to monitor the server with something like Nagios to
> help identify if something is out of order, but it's frustrating that I
> rarely see the CPU get higher than 25%. There are some simple processes
> (searching for all Report Styles or Data Defs) that on occasion seem to take
> a whole lot longer than they ought to.
> > >
> > > We're trying to work with a big Epicor guru for some paid advice, but
> it's been a little tricky to coordinate scheduling so far.
> > >
> > > Any thoughts, ideas, or suggestions would be helpful at this point. I'm
> only 6 months in supporting the system, so just about everything is new
> information for me.
> > >
> > > Thanks in advance!
> > >
> >
>
>
>


[Non-text portions of this message have been removed]
@cooner
Yeah, 15 regular/15 MES seems like it ought to be fairly lightweight, and it's probably less than 10 regular users on a regular basis.

What sort of transactions are you talking about and how would I check? The bulk of any sort of interaction with the system would be around manufacturing/production, if that makes any sense.

We're using the same box since I started 7 months ago; it's probably been formatted once somewhere along the lines, but it's got its fair share of updates. We've got a new server waiting for Epicor 9, but again, I have a nagging feeling the current server might be underutilized.

I didn't have too much trouble going through the db tuning guide, and so far it looks like we had a few things over-configured, so I'm not sure what that means for us either.

@waffqle
We're not virtualized.

I stared at perlogs for a few minutes this morning and the max I saw Current Disk Queue Length was 2. I'm running Microsoft's PAL tool (has anyone else ever used this? I just found it today) this afternoon and logging data for 5 hours, to see if that turns up anything useful.

No termserv. Most of our MES computers are old, but even some of our new machines seem to take longer than it should for some operations or particular reports (but what's normal?).

Network issues are a whole other ball of wax, but should be readily identifiable if I run a client instance directly on the server. We've got entry-level server Gb switches.

Out of curiosity, can someone tell me how long it takes them to print preview a Menu Security Report for APRP4100 (1099 Processing), unchecking 'New Page Per User/Group' but leaving all the other defaults? I get a 19 page report, and it took an average of 9.3 seconds to run directly on the server.

Thanks for all the help so far. The cumulative information here has been more beneficial than anything I've gotten elsewhere.

--- In vantage@yahoogroups.com, "cooner_55421" <cooner_55421@...> wrote:
>
> Hi,
>
> Looks like you have 15 regular users and 15 MES users?
> That doesn't sound like it would be a big strain on your system.
>
> Just wondering about transactions. Amount, type, etc...?
> History of your install. Fresh or a long history of upgrades on the same box?
>
> >the Progress DB tuning guide,
> >but I should probably go back through that again
> I find the guide a little confusing.
> I called Epicor Support several times - Tech/Install (option 3 & 1)
> I got lucky and connected with a guy who put in some extra effort.
> I don't have his name handy right now, anyway...
> He was able to explain a couple of things to me I never would have figured out from the guide.
>
--- In vantage@yahoogroups.com, Waffqle <waffqle@...> wrote:
>
> I'm assuming you aren't virtualized, correct me if I'm wrong.
>
> If your buffer hits are already at >98% then more RAM isn't going to get you
> anywhere.
>
> Perfmon should be able to tell you pretty definitively whether or not your
> disks are your issue. (Though I would tend to assume they're the
> bottleneck.) Load it up and monitor your Average and Current Disk Queue
> Length. If your queue length is higher than "1.5 x # of spindles(4)" your
> disks are holding you back.
> It's tempting to look at Read/Write and other IO measurements, but at the
> end of the day if your queue is long, your disks are overloaded.
>
> Also have you looked at your clients? The Vantage/Epicor client is not a
> lightweight piece of software. If you're running a term serv, check the load
> on that as well.
>
> Look at your network out as well. As you're only running 15 users, I imagine
> your network isn't too huge or overloaded; but it's certainly worth
> checking. Check the load on your switches/routers.
>
>
>
> On Thu, Feb 10, 2011 at 11:28 AM, cubcrafters_it <
> jason.navarrete@...> wrote:
>
> >
> >
> > We're only using 5 out of our 8 GB of RAM. Can I devote more of it to the
> > DB or App servers? Would that be beneficial? Buffer Hits are already at
> > ~98%.
> >
> > Disks are 15k SAS 3Gb/s drives, but yes, everything Epicor is on the same
> > RAID 10. Does anyone happen to have documentation on moving temp dirs to
> > different spindles? I'll also try monitoring disk access to see what that
> > tells me.
> >
> > I'm a little hesitant to throw hardware at the problem when I'm not
> > completely convinced that it will solve the problem (partially because I'm
> > cheap).
> >
> >
> > --- In vantage@yahoogroups.com, "markbetts66" <markbetts66@> wrote:
> > >
> > > Hi,
> > >
> > > You may want to try more RAM. What type of disks are you using in your
> > server? When you say RAID10(EPICOR) do you mean DB, Appserver, progress Temp
> > Dir and report Temp Dir on the same disk?
> > >
> > > --- In vantage@yahoogroups.com, "cubcrafters_it" <jason.navarrete@>
> > wrote:
> > > >
> > > > First off, very glad to have found this group.
> > > >
> > > > Secondly:
> > > > Vista 8.03.409A on Progress 10.1B
> > > > Dual Core Xeon 5160 3GHz, 8GB RAM
> > > > RAID 1 (OS) + 10 (Epicor)
> > > > d1: 2.21GB, b1: 341MB
> > > > ~15 Standard + ~15 MES Users
> > > >
> > > > We're currently plagued with performance/speed/lockup issues, and would
> > love to be able to mitigate as much of it as possible in our journey to
> > Epicor 9.
> > > >
> > > > I ran a dump and load at the beginning of the year, and while it took
> > the scatter factor down significantly, it didn't appear to make a noticeable
> > different in the end user experience (but it made me feel a whole lot
> > better). I also did look at the Progress DB tuning guide, but I should
> > probably go back through that again.
> > > >
> > > > Right now I'm lucky if we go a couple of weeks without something
> > bringing down the system. I feel like I don't have any visibility into
> > what's causing the issues.
> > > >
> > > > I'd like to be able to monitor the server with something like Nagios to
> > help identify if something is out of order, but it's frustrating that I
> > rarely see the CPU get higher than 25%. There are some simple processes
> > (searching for all Report Styles or Data Defs) that on occasion seem to take
> > a whole lot longer than they ought to.
> > > >
> > > > We're trying to work with a big Epicor guru for some paid advice, but
> > it's been a little tricky to coordinate scheduling so far.
> > > >
> > > > Any thoughts, ideas, or suggestions would be helpful at this point. I'm
> > only 6 months in supporting the system, so just about everything is new
> > information for me.
> > > >
> > > > Thanks in advance!
> > > >
> > >
> >
> >
> >
>
>
> [Non-text portions of this message have been removed]
>
Print Jobs other 'Submitted Jobs' aren't always the best gauge of
performance as they use a polling system.
It may take you 20 seconds to see the print job even if it only took the
system 2 seconds to produce it. The other 18 seconds, you're just waiting on
the System Agent to poll.

You can reduce the polling time in the System Agent. Whatever 'Processing
Delay' is set to is the maximum time it will take the system to begin a
submitted job.

Note, be careful not to set it too fast lest it start stepping on itself. 10
seconds is probably a safe starting point if it's currently set to a large
number. You might be able to get away with 5 seconds if your server is under
a pretty low load. A bit of trial and error will get you to the right
number.

Also, if your system has been locking up and you find that the processing
delay is set especially low ...you probably just found the culprit.


Lastly, I didn't think to ask, but you're not running a white box or cheap
(god forbid software) RAID? If your RAID isn't being handled by a proper
controller, all bets are off as far as performance goes.


On Thu, Feb 10, 2011 at 2:00 PM, cubcrafters_it <
jason.navarrete@...> wrote:

>
>
>
>
> @cooner
> Yeah, 15 regular/15 MES seems like it ought to be fairly lightweight, and
> it's probably less than 10 regular users on a regular basis.
>
> What sort of transactions are you talking about and how would I check? The
> bulk of any sort of interaction with the system would be around
> manufacturing/production, if that makes any sense.
>
> We're using the same box since I started 7 months ago; it's probably been
> formatted once somewhere along the lines, but it's got its fair share of
> updates. We've got a new server waiting for Epicor 9, but again, I have a
> nagging feeling the current server might be underutilized.
>
> I didn't have too much trouble going through the db tuning guide, and so
> far it looks like we had a few things over-configured, so I'm not sure what
> that means for us either.
>
> @waffqle
> We're not virtualized.
>
> I stared at perlogs for a few minutes this morning and the max I saw
> Current Disk Queue Length was 2. I'm running Microsoft's PAL tool (has
> anyone else ever used this? I just found it today) this afternoon and
> logging data for 5 hours, to see if that turns up anything useful.
>
> No termserv. Most of our MES computers are old, but even some of our new
> machines seem to take longer than it should for some operations or
> particular reports (but what's normal?).
>
> Network issues are a whole other ball of wax, but should be readily
> identifiable if I run a client instance directly on the server. We've got
> entry-level server Gb switches.
>
> Out of curiosity, can someone tell me how long it takes them to print
> preview a Menu Security Report for APRP4100 (1099 Processing), unchecking
> 'New Page Per User/Group' but leaving all the other defaults? I get a 19
> page report, and it took an average of 9.3 seconds to run directly on the
> server.
>
> Thanks for all the help so far. The cumulative information here has been
> more beneficial than anything I've gotten elsewhere.
>
>
> --- In vantage@yahoogroups.com, "cooner_55421" <cooner_55421@...> wrote:
> >
> > Hi,
> >
> > Looks like you have 15 regular users and 15 MES users?
> > That doesn't sound like it would be a big strain on your system.
> >
> > Just wondering about transactions. Amount, type, etc...?
> > History of your install. Fresh or a long history of upgrades on the same
> box?
> >
> > >the Progress DB tuning guide,
> > >but I should probably go back through that again
> > I find the guide a little confusing.
> > I called Epicor Support several times - Tech/Install (option 3 & 1)
> > I got lucky and connected with a guy who put in some extra effort.
> > I don't have his name handy right now, anyway...
> > He was able to explain a couple of things to me I never would have
> figured out from the guide.
> >
> --- In vantage@yahoogroups.com, Waffqle <waffqle@...> wrote:
> >
> > I'm assuming you aren't virtualized, correct me if I'm wrong.
> >
> > If your buffer hits are already at >98% then more RAM isn't going to get
> you
> > anywhere.
> >
> > Perfmon should be able to tell you pretty definitively whether or not
> your
> > disks are your issue. (Though I would tend to assume they're the
> > bottleneck.) Load it up and monitor your Average and Current Disk Queue
> > Length. If your queue length is higher than "1.5 x # of spindles(4)" your
> > disks are holding you back.
> > It's tempting to look at Read/Write and other IO measurements, but at the
> > end of the day if your queue is long, your disks are overloaded.
> >
> > Also have you looked at your clients? The Vantage/Epicor client is not a
> > lightweight piece of software. If you're running a term serv, check the
> load
> > on that as well.
> >
> > Look at your network out as well. As you're only running 15 users, I
> imagine
> > your network isn't too huge or overloaded; but it's certainly worth
> > checking. Check the load on your switches/routers.
> >
> >
> >
> > On Thu, Feb 10, 2011 at 11:28 AM, cubcrafters_it <
> > jason.navarrete@...> wrote:
> >
> > >
> > >
> > > We're only using 5 out of our 8 GB of RAM. Can I devote more of it to
> the
> > > DB or App servers? Would that be beneficial? Buffer Hits are already at
> > > ~98%.
> > >
> > > Disks are 15k SAS 3Gb/s drives, but yes, everything Epicor is on the
> same
> > > RAID 10. Does anyone happen to have documentation on moving temp dirs
> to
> > > different spindles? I'll also try monitoring disk access to see what
> that
> > > tells me.
> > >
> > > I'm a little hesitant to throw hardware at the problem when I'm not
> > > completely convinced that it will solve the problem (partially because
> I'm
> > > cheap).
> > >
> > >
> > > --- In vantage@yahoogroups.com, "markbetts66" <markbetts66@> wrote:
> > > >
> > > > Hi,
> > > >
> > > > You may want to try more RAM. What type of disks are you using in
> your
> > > server? When you say RAID10(EPICOR) do you mean DB, Appserver, progress
> Temp
> > > Dir and report Temp Dir on the same disk?
> > > >
> > > > --- In vantage@yahoogroups.com, "cubcrafters_it" <jason.navarrete@>
> > > wrote:
> > > > >
> > > > > First off, very glad to have found this group.
> > > > >
> > > > > Secondly:
> > > > > Vista 8.03.409A on Progress 10.1B
> > > > > Dual Core Xeon 5160 3GHz, 8GB RAM
> > > > > RAID 1 (OS) + 10 (Epicor)
> > > > > d1: 2.21GB, b1: 341MB
> > > > > ~15 Standard + ~15 MES Users
> > > > >
> > > > > We're currently plagued with performance/speed/lockup issues, and
> would
> > > love to be able to mitigate as much of it as possible in our journey to
> > > Epicor 9.
> > > > >
> > > > > I ran a dump and load at the beginning of the year, and while it
> took
> > > the scatter factor down significantly, it didn't appear to make a
> noticeable
> > > different in the end user experience (but it made me feel a whole lot
> > > better). I also did look at the Progress DB tuning guide, but I should
> > > probably go back through that again.
> > > > >
> > > > > Right now I'm lucky if we go a couple of weeks without something
> > > bringing down the system. I feel like I don't have any visibility into
> > > what's causing the issues.
> > > > >
> > > > > I'd like to be able to monitor the server with something like
> Nagios to
> > > help identify if something is out of order, but it's frustrating that I
> > > rarely see the CPU get higher than 25%. There are some simple processes
> > > (searching for all Report Styles or Data Defs) that on occasion seem to
> take
> > > a whole lot longer than they ought to.
> > > > >
> > > > > We're trying to work with a big Epicor guru for some paid advice,
> but
> > > it's been a little tricky to coordinate scheduling so far.
> > > > >
> > > > > Any thoughts, ideas, or suggestions would be helpful at this point.
> I'm
> > > only 6 months in supporting the system, so just about everything is new
> > > information for me.
> > > > >
> > > > > Thanks in advance!
> > > > >
> > > >
> > >
> > >
> > >
> >
> >
> > [Non-text portions of this message have been removed]
> >
>
>
>


[Non-text portions of this message have been removed]
You may be onto something important here: my Processing Delay is currently set to 2 (default is 10?). I've never seen my server on any sort of sustained processor load, however, so could that still be an issue? Can I bump up the delay without having to restart anything?

And as far as the RAID is concerned, it's on a PERC 5i with a BBU, so I would hope it's a halfway decent setup.

Still waiting on my performance logs to analyze this afternoon.

-Jason

--- In vantage@yahoogroups.com, Waffqle <waffqle@...> wrote:
>
> Print Jobs other 'Submitted Jobs' aren't always the best gauge of
> performance as they use a polling system.
> It may take you 20 seconds to see the print job even if it only took the
> system 2 seconds to produce it. The other 18 seconds, you're just waiting on
> the System Agent to poll.
>
> You can reduce the polling time in the System Agent. Whatever 'Processing
> Delay' is set to is the maximum time it will take the system to begin a
> submitted job.
>
> Note, be careful not to set it too fast lest it start stepping on itself. 10
> seconds is probably a safe starting point if it's currently set to a large
> number. You might be able to get away with 5 seconds if your server is under
> a pretty low load. A bit of trial and error will get you to the right
> number.
>
> Also, if your system has been locking up and you find that the processing
> delay is set especially low ...you probably just found the culprit.
>
>
> Lastly, I didn't think to ask, but you're not running a white box or cheap
> (god forbid software) RAID? If your RAID isn't being handled by a proper
> controller, all bets are off as far as performance goes.
>
>
> On Thu, Feb 10, 2011 at 2:00 PM, cubcrafters_it <
> jason.navarrete@...> wrote:
>
> >
> >
> >
> >
> > @cooner
> > Yeah, 15 regular/15 MES seems like it ought to be fairly lightweight, and
> > it's probably less than 10 regular users on a regular basis.
> >
> > What sort of transactions are you talking about and how would I check? The
> > bulk of any sort of interaction with the system would be around
> > manufacturing/production, if that makes any sense.
> >
> > We're using the same box since I started 7 months ago; it's probably been
> > formatted once somewhere along the lines, but it's got its fair share of
> > updates. We've got a new server waiting for Epicor 9, but again, I have a
> > nagging feeling the current server might be underutilized.
> >
> > I didn't have too much trouble going through the db tuning guide, and so
> > far it looks like we had a few things over-configured, so I'm not sure what
> > that means for us either.
> >
> > @waffqle
> > We're not virtualized.
> >
> > I stared at perlogs for a few minutes this morning and the max I saw
> > Current Disk Queue Length was 2. I'm running Microsoft's PAL tool (has
> > anyone else ever used this? I just found it today) this afternoon and
> > logging data for 5 hours, to see if that turns up anything useful.
> >
> > No termserv. Most of our MES computers are old, but even some of our new
> > machines seem to take longer than it should for some operations or
> > particular reports (but what's normal?).
> >
> > Network issues are a whole other ball of wax, but should be readily
> > identifiable if I run a client instance directly on the server. We've got
> > entry-level server Gb switches.
> >
> > Out of curiosity, can someone tell me how long it takes them to print
> > preview a Menu Security Report for APRP4100 (1099 Processing), unchecking
> > 'New Page Per User/Group' but leaving all the other defaults? I get a 19
> > page report, and it took an average of 9.3 seconds to run directly on the
> > server.
> >
> > Thanks for all the help so far. The cumulative information here has been
> > more beneficial than anything I've gotten elsewhere.
> >
> >
> > --- In vantage@yahoogroups.com, "cooner_55421" <cooner_55421@> wrote:
> > >
> > > Hi,
> > >
> > > Looks like you have 15 regular users and 15 MES users?
> > > That doesn't sound like it would be a big strain on your system.
> > >
> > > Just wondering about transactions. Amount, type, etc...?
> > > History of your install. Fresh or a long history of upgrades on the same
> > box?
> > >
> > > >the Progress DB tuning guide,
> > > >but I should probably go back through that again
> > > I find the guide a little confusing.
> > > I called Epicor Support several times - Tech/Install (option 3 & 1)
> > > I got lucky and connected with a guy who put in some extra effort.
> > > I don't have his name handy right now, anyway...
> > > He was able to explain a couple of things to me I never would have
> > figured out from the guide.
> > >
> > --- In vantage@yahoogroups.com, Waffqle <waffqle@> wrote:
> > >
> > > I'm assuming you aren't virtualized, correct me if I'm wrong.
> > >
> > > If your buffer hits are already at >98% then more RAM isn't going to get
> > you
> > > anywhere.
> > >
> > > Perfmon should be able to tell you pretty definitively whether or not
> > your
> > > disks are your issue. (Though I would tend to assume they're the
> > > bottleneck.) Load it up and monitor your Average and Current Disk Queue
> > > Length. If your queue length is higher than "1.5 x # of spindles(4)" your
> > > disks are holding you back.
> > > It's tempting to look at Read/Write and other IO measurements, but at the
> > > end of the day if your queue is long, your disks are overloaded.
> > >
> > > Also have you looked at your clients? The Vantage/Epicor client is not a
> > > lightweight piece of software. If you're running a term serv, check the
> > load
> > > on that as well.
> > >
> > > Look at your network out as well. As you're only running 15 users, I
> > imagine
> > > your network isn't too huge or overloaded; but it's certainly worth
> > > checking. Check the load on your switches/routers.
> > >
> > >
> > >
> > > On Thu, Feb 10, 2011 at 11:28 AM, cubcrafters_it <
> > > jason.navarrete@> wrote:
> > >
> > > >
> > > >
> > > > We're only using 5 out of our 8 GB of RAM. Can I devote more of it to
> > the
> > > > DB or App servers? Would that be beneficial? Buffer Hits are already at
> > > > ~98%.
> > > >
> > > > Disks are 15k SAS 3Gb/s drives, but yes, everything Epicor is on the
> > same
> > > > RAID 10. Does anyone happen to have documentation on moving temp dirs
> > to
> > > > different spindles? I'll also try monitoring disk access to see what
> > that
> > > > tells me.
> > > >
> > > > I'm a little hesitant to throw hardware at the problem when I'm not
> > > > completely convinced that it will solve the problem (partially because
> > I'm
> > > > cheap).
> > > >
> > > >
> > > > --- In vantage@yahoogroups.com, "markbetts66" <markbetts66@> wrote:
> > > > >
> > > > > Hi,
> > > > >
> > > > > You may want to try more RAM. What type of disks are you using in
> > your
> > > > server? When you say RAID10(EPICOR) do you mean DB, Appserver, progress
> > Temp
> > > > Dir and report Temp Dir on the same disk?
> > > > >
> > > > > --- In vantage@yahoogroups.com, "cubcrafters_it" <jason.navarrete@>
> > > > wrote:
> > > > > >
> > > > > > First off, very glad to have found this group.
> > > > > >
> > > > > > Secondly:
> > > > > > Vista 8.03.409A on Progress 10.1B
> > > > > > Dual Core Xeon 5160 3GHz, 8GB RAM
> > > > > > RAID 1 (OS) + 10 (Epicor)
> > > > > > d1: 2.21GB, b1: 341MB
> > > > > > ~15 Standard + ~15 MES Users
> > > > > >
> > > > > > We're currently plagued with performance/speed/lockup issues, and
> > would
> > > > love to be able to mitigate as much of it as possible in our journey to
> > > > Epicor 9.
> > > > > >
> > > > > > I ran a dump and load at the beginning of the year, and while it
> > took
> > > > the scatter factor down significantly, it didn't appear to make a
> > noticeable
> > > > different in the end user experience (but it made me feel a whole lot
> > > > better). I also did look at the Progress DB tuning guide, but I should
> > > > probably go back through that again.
> > > > > >
> > > > > > Right now I'm lucky if we go a couple of weeks without something
> > > > bringing down the system. I feel like I don't have any visibility into
> > > > what's causing the issues.
> > > > > >
> > > > > > I'd like to be able to monitor the server with something like
> > Nagios to
> > > > help identify if something is out of order, but it's frustrating that I
> > > > rarely see the CPU get higher than 25%. There are some simple processes
> > > > (searching for all Report Styles or Data Defs) that on occasion seem to
> > take
> > > > a whole lot longer than they ought to.
> > > > > >
> > > > > > We're trying to work with a big Epicor guru for some paid advice,
> > but
> > > > it's been a little tricky to coordinate scheduling so far.
> > > > > >
> > > > > > Any thoughts, ideas, or suggestions would be helpful at this point.
> > I'm
> > > > only 6 months in supporting the system, so just about everything is new
> > > > information for me.
> > > > > >
> > > > > > Thanks in advance!
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > >
> > >
> > > [Non-text portions of this message have been removed]
> > >
> >
> >
> >
>
>
> [Non-text portions of this message have been removed]
>
I'd bump it up a bit and see what happens.You'll need to restart the task
agent and the process agent. This won't throw kick everyone off, but it'll
stop print jobs and service connect and possibly other stuff.

PERC should be good to go.

Have you timed any non-print/submitted processes? Adding lines or saving
orders/jobs/etc?
You can run a trace and use the time info to do a rough benchmark. That's a
good way to tell whether it's the server or the client.



On Thu, Feb 10, 2011 at 6:00 PM, cubcrafters_it <
jason.navarrete@...> wrote:

>
>
> You may be onto something important here: my Processing Delay is currently
> set to 2 (default is 10?). I've never seen my server on any sort of
> sustained processor load, however, so could that still be an issue? Can I
> bump up the delay without having to restart anything?
>
> And as far as the RAID is concerned, it's on a PERC 5i with a BBU, so I
> would hope it's a halfway decent setup.
>
> Still waiting on my performance logs to analyze this afternoon.
>
> -Jason
>
>
> --- In vantage@yahoogroups.com, Waffqle <waffqle@...> wrote:
> >
> > Print Jobs other 'Submitted Jobs' aren't always the best gauge of
> > performance as they use a polling system.
> > It may take you 20 seconds to see the print job even if it only took the
> > system 2 seconds to produce it. The other 18 seconds, you're just waiting
> on
> > the System Agent to poll.
> >
> > You can reduce the polling time in the System Agent. Whatever 'Processing
> > Delay' is set to is the maximum time it will take the system to begin a
> > submitted job.
> >
> > Note, be careful not to set it too fast lest it start stepping on itself.
> 10
> > seconds is probably a safe starting point if it's currently set to a
> large
> > number. You might be able to get away with 5 seconds if your server is
> under
> > a pretty low load. A bit of trial and error will get you to the right
> > number.
> >
> > Also, if your system has been locking up and you find that the processing
> > delay is set especially low ...you probably just found the culprit.
> >
> >
> > Lastly, I didn't think to ask, but you're not running a white box or
> cheap
> > (god forbid software) RAID? If your RAID isn't being handled by a proper
> > controller, all bets are off as far as performance goes.
> >
> >
> > On Thu, Feb 10, 2011 at 2:00 PM, cubcrafters_it <
> > jason.navarrete@...> wrote:
> >
> > >
> > >
> > >
> > >
> > > @cooner
> > > Yeah, 15 regular/15 MES seems like it ought to be fairly lightweight,
> and
> > > it's probably less than 10 regular users on a regular basis.
> > >
> > > What sort of transactions are you talking about and how would I check?
> The
> > > bulk of any sort of interaction with the system would be around
> > > manufacturing/production, if that makes any sense.
> > >
> > > We're using the same box since I started 7 months ago; it's probably
> been
> > > formatted once somewhere along the lines, but it's got its fair share
> of
> > > updates. We've got a new server waiting for Epicor 9, but again, I have
> a
> > > nagging feeling the current server might be underutilized.
> > >
> > > I didn't have too much trouble going through the db tuning guide, and
> so
> > > far it looks like we had a few things over-configured, so I'm not sure
> what
> > > that means for us either.
> > >
> > > @waffqle
> > > We're not virtualized.
> > >
> > > I stared at perlogs for a few minutes this morning and the max I saw
> > > Current Disk Queue Length was 2. I'm running Microsoft's PAL tool (has
> > > anyone else ever used this? I just found it today) this afternoon and
> > > logging data for 5 hours, to see if that turns up anything useful.
> > >
> > > No termserv. Most of our MES computers are old, but even some of our
> new
> > > machines seem to take longer than it should for some operations or
> > > particular reports (but what's normal?).
> > >
> > > Network issues are a whole other ball of wax, but should be readily
> > > identifiable if I run a client instance directly on the server. We've
> got
> > > entry-level server Gb switches.
> > >
> > > Out of curiosity, can someone tell me how long it takes them to print
> > > preview a Menu Security Report for APRP4100 (1099 Processing),
> unchecking
> > > 'New Page Per User/Group' but leaving all the other defaults? I get a
> 19
> > > page report, and it took an average of 9.3 seconds to run directly on
> the
> > > server.
> > >
> > > Thanks for all the help so far. The cumulative information here has
> been
> > > more beneficial than anything I've gotten elsewhere.
> > >
> > >
> > > --- In vantage@yahoogroups.com, "cooner_55421" <cooner_55421@> wrote:
> > > >
> > > > Hi,
> > > >
> > > > Looks like you have 15 regular users and 15 MES users?
> > > > That doesn't sound like it would be a big strain on your system.
> > > >
> > > > Just wondering about transactions. Amount, type, etc...?
> > > > History of your install. Fresh or a long history of upgrades on the
> same
> > > box?
> > > >
> > > > >the Progress DB tuning guide,
> > > > >but I should probably go back through that again
> > > > I find the guide a little confusing.
> > > > I called Epicor Support several times - Tech/Install (option 3 & 1)
> > > > I got lucky and connected with a guy who put in some extra effort.
> > > > I don't have his name handy right now, anyway...
> > > > He was able to explain a couple of things to me I never would have
> > > figured out from the guide.
> > > >
> > > --- In vantage@yahoogroups.com, Waffqle <waffqle@> wrote:
> > > >
> > > > I'm assuming you aren't virtualized, correct me if I'm wrong.
> > > >
> > > > If your buffer hits are already at >98% then more RAM isn't going to
> get
> > > you
> > > > anywhere.
> > > >
> > > > Perfmon should be able to tell you pretty definitively whether or not
> > > your
> > > > disks are your issue. (Though I would tend to assume they're the
> > > > bottleneck.) Load it up and monitor your Average and Current Disk
> Queue
> > > > Length. If your queue length is higher than "1.5 x # of spindles(4)"
> your
> > > > disks are holding you back.
> > > > It's tempting to look at Read/Write and other IO measurements, but at
> the
> > > > end of the day if your queue is long, your disks are overloaded.
> > > >
> > > > Also have you looked at your clients? The Vantage/Epicor client is
> not a
> > > > lightweight piece of software. If you're running a term serv, check
> the
> > > load
> > > > on that as well.
> > > >
> > > > Look at your network out as well. As you're only running 15 users, I
> > > imagine
> > > > your network isn't too huge or overloaded; but it's certainly worth
> > > > checking. Check the load on your switches/routers.
> > > >
> > > >
> > > >
> > > > On Thu, Feb 10, 2011 at 11:28 AM, cubcrafters_it <
> > > > jason.navarrete@> wrote:
> > > >
> > > > >
> > > > >
> > > > > We're only using 5 out of our 8 GB of RAM. Can I devote more of it
> to
> > > the
> > > > > DB or App servers? Would that be beneficial? Buffer Hits are
> already at
> > > > > ~98%.
> > > > >
> > > > > Disks are 15k SAS 3Gb/s drives, but yes, everything Epicor is on
> the
> > > same
> > > > > RAID 10. Does anyone happen to have documentation on moving temp
> dirs
> > > to
> > > > > different spindles? I'll also try monitoring disk access to see
> what
> > > that
> > > > > tells me.
> > > > >
> > > > > I'm a little hesitant to throw hardware at the problem when I'm not
> > > > > completely convinced that it will solve the problem (partially
> because
> > > I'm
> > > > > cheap).
> > > > >
> > > > >
> > > > > --- In vantage@yahoogroups.com, "markbetts66" <markbetts66@>
> wrote:
> > > > > >
> > > > > > Hi,
> > > > > >
> > > > > > You may want to try more RAM. What type of disks are you using in
> > > your
> > > > > server? When you say RAID10(EPICOR) do you mean DB, Appserver,
> progress
> > > Temp
> > > > > Dir and report Temp Dir on the same disk?
> > > > > >
> > > > > > --- In vantage@yahoogroups.com, "cubcrafters_it"
> <jason.navarrete@>
> > > > > wrote:
> > > > > > >
> > > > > > > First off, very glad to have found this group.
> > > > > > >
> > > > > > > Secondly:
> > > > > > > Vista 8.03.409A on Progress 10.1B
> > > > > > > Dual Core Xeon 5160 3GHz, 8GB RAM
> > > > > > > RAID 1 (OS) + 10 (Epicor)
> > > > > > > d1: 2.21GB, b1: 341MB
> > > > > > > ~15 Standard + ~15 MES Users
> > > > > > >
> > > > > > > We're currently plagued with performance/speed/lockup issues,
> and
> > > would
> > > > > love to be able to mitigate as much of it as possible in our
> journey to
> > > > > Epicor 9.
> > > > > > >
> > > > > > > I ran a dump and load at the beginning of the year, and while
> it
> > > took
> > > > > the scatter factor down significantly, it didn't appear to make a
> > > noticeable
> > > > > different in the end user experience (but it made me feel a whole
> lot
> > > > > better). I also did look at the Progress DB tuning guide, but I
> should
> > > > > probably go back through that again.
> > > > > > >
> > > > > > > Right now I'm lucky if we go a couple of weeks without
> something
> > > > > bringing down the system. I feel like I don't have any visibility
> into
> > > > > what's causing the issues.
> > > > > > >
> > > > > > > I'd like to be able to monitor the server with something like
> > > Nagios to
> > > > > help identify if something is out of order, but it's frustrating
> that I
> > > > > rarely see the CPU get higher than 25%. There are some simple
> processes
> > > > > (searching for all Report Styles or Data Defs) that on occasion
> seem to
> > > take
> > > > > a whole lot longer than they ought to.
> > > > > > >
> > > > > > > We're trying to work with a big Epicor guru for some paid
> advice,
> > > but
> > > > > it's been a little tricky to coordinate scheduling so far.
> > > > > > >
> > > > > > > Any thoughts, ideas, or suggestions would be helpful at this
> point.
> > > I'm
> > > > > only 6 months in supporting the system, so just about everything is
> new
> > > > > information for me.
> > > > > > >
> > > > > > > Thanks in advance!
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > > >
> > > > [Non-text portions of this message have been removed]
> > > >
> > >
> > >
> > >
> >
> >
> > [Non-text portions of this message have been removed]
> >
>
>
>


[Non-text portions of this message have been removed]
So my perflogs didn't turn up anything completely out of the ordinary, but there are a few small things I'll follow up on.

Still need to schedule some time to restart the task and process agents, so we'll see what that does.

Here's what I timed:
Production Management -> Job Management -> General Operations -> Shop Tracker
Picked a resource group with a lot of labor rows, then over on the labor tab, clicked 'labor' to bring up Labor Detail Search. Changed the options to return 10,000 rows max, and then clicked search. Timed the time it took to return those rows on my test client. Just used a stopwatch rather than using a trace, as I didn't want that additional overhead.

Average over 5 runs: 10.8 seconds (on 8.03.409A, Progress 10.1B)

If anyone feels like running this test, I'd greatly appreciate it. Or please suggest another benchmark that would be repeatable on other systems.



--- In vantage@yahoogroups.com, Waffqle <waffqle@...> wrote:
>
> I'd bump it up a bit and see what happens.You'll need to restart the task
> agent and the process agent. This won't throw kick everyone off, but it'll
> stop print jobs and service connect and possibly other stuff.
>
> PERC should be good to go.
>
> Have you timed any non-print/submitted processes? Adding lines or saving
> orders/jobs/etc?
> You can run a trace and use the time info to do a rough benchmark. That's a
> good way to tell whether it's the server or the client.
>
>
>
> On Thu, Feb 10, 2011 at 6:00 PM, cubcrafters_it <
> jason.navarrete@...> wrote:
>
> >
> >
> > You may be onto something important here: my Processing Delay is currently
> > set to 2 (default is 10?). I've never seen my server on any sort of
> > sustained processor load, however, so could that still be an issue? Can I
> > bump up the delay without having to restart anything?
> >
> > And as far as the RAID is concerned, it's on a PERC 5i with a BBU, so I
> > would hope it's a halfway decent setup.
> >
> > Still waiting on my performance logs to analyze this afternoon.
> >
> > -Jason
> >
> >
> > --- In vantage@yahoogroups.com, Waffqle <waffqle@> wrote:
> > >
> > > Print Jobs other 'Submitted Jobs' aren't always the best gauge of
> > > performance as they use a polling system.
> > > It may take you 20 seconds to see the print job even if it only took the
> > > system 2 seconds to produce it. The other 18 seconds, you're just waiting
> > on
> > > the System Agent to poll.
> > >
> > > You can reduce the polling time in the System Agent. Whatever 'Processing
> > > Delay' is set to is the maximum time it will take the system to begin a
> > > submitted job.
> > >
> > > Note, be careful not to set it too fast lest it start stepping on itself.
> > 10
> > > seconds is probably a safe starting point if it's currently set to a
> > large
> > > number. You might be able to get away with 5 seconds if your server is
> > under
> > > a pretty low load. A bit of trial and error will get you to the right
> > > number.
> > >
> > > Also, if your system has been locking up and you find that the processing
> > > delay is set especially low ...you probably just found the culprit.
> > >
> > >
> > > Lastly, I didn't think to ask, but you're not running a white box or
> > cheap
> > > (god forbid software) RAID? If your RAID isn't being handled by a proper
> > > controller, all bets are off as far as performance goes.
> > >
> > >
> > > On Thu, Feb 10, 2011 at 2:00 PM, cubcrafters_it <
> > > jason.navarrete@> wrote:
> > >
> > > >
> > > >
> > > >
> > > >
> > > > @cooner
> > > > Yeah, 15 regular/15 MES seems like it ought to be fairly lightweight,
> > and
> > > > it's probably less than 10 regular users on a regular basis.
> > > >
> > > > What sort of transactions are you talking about and how would I check?
> > The
> > > > bulk of any sort of interaction with the system would be around
> > > > manufacturing/production, if that makes any sense.
> > > >
> > > > We're using the same box since I started 7 months ago; it's probably
> > been
> > > > formatted once somewhere along the lines, but it's got its fair share
> > of
> > > > updates. We've got a new server waiting for Epicor 9, but again, I have
> > a
> > > > nagging feeling the current server might be underutilized.
> > > >
> > > > I didn't have too much trouble going through the db tuning guide, and
> > so
> > > > far it looks like we had a few things over-configured, so I'm not sure
> > what
> > > > that means for us either.
> > > >
> > > > @waffqle
> > > > We're not virtualized.
> > > >
> > > > I stared at perlogs for a few minutes this morning and the max I saw
> > > > Current Disk Queue Length was 2. I'm running Microsoft's PAL tool (has
> > > > anyone else ever used this? I just found it today) this afternoon and
> > > > logging data for 5 hours, to see if that turns up anything useful.
> > > >
> > > > No termserv. Most of our MES computers are old, but even some of our
> > new
> > > > machines seem to take longer than it should for some operations or
> > > > particular reports (but what's normal?).
> > > >
> > > > Network issues are a whole other ball of wax, but should be readily
> > > > identifiable if I run a client instance directly on the server. We've
> > got
> > > > entry-level server Gb switches.
> > > >
> > > > Out of curiosity, can someone tell me how long it takes them to print
> > > > preview a Menu Security Report for APRP4100 (1099 Processing),
> > unchecking
> > > > 'New Page Per User/Group' but leaving all the other defaults? I get a
> > 19
> > > > page report, and it took an average of 9.3 seconds to run directly on
> > the
> > > > server.
> > > >
> > > > Thanks for all the help so far. The cumulative information here has
> > been
> > > > more beneficial than anything I've gotten elsewhere.
> > > >
> > > >
> > > > --- In vantage@yahoogroups.com, "cooner_55421" <cooner_55421@> wrote:
> > > > >
> > > > > Hi,
> > > > >
> > > > > Looks like you have 15 regular users and 15 MES users?
> > > > > That doesn't sound like it would be a big strain on your system.
> > > > >
> > > > > Just wondering about transactions. Amount, type, etc...?
> > > > > History of your install. Fresh or a long history of upgrades on the
> > same
> > > > box?
> > > > >
> > > > > >the Progress DB tuning guide,
> > > > > >but I should probably go back through that again
> > > > > I find the guide a little confusing.
> > > > > I called Epicor Support several times - Tech/Install (option 3 & 1)
> > > > > I got lucky and connected with a guy who put in some extra effort.
> > > > > I don't have his name handy right now, anyway...
> > > > > He was able to explain a couple of things to me I never would have
> > > > figured out from the guide.
> > > > >
> > > > --- In vantage@yahoogroups.com, Waffqle <waffqle@> wrote:
> > > > >
> > > > > I'm assuming you aren't virtualized, correct me if I'm wrong.
> > > > >
> > > > > If your buffer hits are already at >98% then more RAM isn't going to
> > get
> > > > you
> > > > > anywhere.
> > > > >
> > > > > Perfmon should be able to tell you pretty definitively whether or not
> > > > your
> > > > > disks are your issue. (Though I would tend to assume they're the
> > > > > bottleneck.) Load it up and monitor your Average and Current Disk
> > Queue
> > > > > Length. If your queue length is higher than "1.5 x # of spindles(4)"
> > your
> > > > > disks are holding you back.
> > > > > It's tempting to look at Read/Write and other IO measurements, but at
> > the
> > > > > end of the day if your queue is long, your disks are overloaded.
> > > > >
> > > > > Also have you looked at your clients? The Vantage/Epicor client is
> > not a
> > > > > lightweight piece of software. If you're running a term serv, check
> > the
> > > > load
> > > > > on that as well.
> > > > >
> > > > > Look at your network out as well. As you're only running 15 users, I
> > > > imagine
> > > > > your network isn't too huge or overloaded; but it's certainly worth
> > > > > checking. Check the load on your switches/routers.
> > > > >
> > > > >
> > > > >
> > > > > On Thu, Feb 10, 2011 at 11:28 AM, cubcrafters_it <
> > > > > jason.navarrete@> wrote:
> > > > >
> > > > > >
> > > > > >
> > > > > > We're only using 5 out of our 8 GB of RAM. Can I devote more of it
> > to
> > > > the
> > > > > > DB or App servers? Would that be beneficial? Buffer Hits are
> > already at
> > > > > > ~98%.
> > > > > >
> > > > > > Disks are 15k SAS 3Gb/s drives, but yes, everything Epicor is on
> > the
> > > > same
> > > > > > RAID 10. Does anyone happen to have documentation on moving temp
> > dirs
> > > > to
> > > > > > different spindles? I'll also try monitoring disk access to see
> > what
> > > > that
> > > > > > tells me.
> > > > > >
> > > > > > I'm a little hesitant to throw hardware at the problem when I'm not
> > > > > > completely convinced that it will solve the problem (partially
> > because
> > > > I'm
> > > > > > cheap).
> > > > > >
> > > > > >
> > > > > > --- In vantage@yahoogroups.com, "markbetts66" <markbetts66@>
> > wrote:
> > > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > You may want to try more RAM. What type of disks are you using in
> > > > your
> > > > > > server? When you say RAID10(EPICOR) do you mean DB, Appserver,
> > progress
> > > > Temp
> > > > > > Dir and report Temp Dir on the same disk?
> > > > > > >
> > > > > > > --- In vantage@yahoogroups.com, "cubcrafters_it"
> > <jason.navarrete@>
> > > > > > wrote:
> > > > > > > >
> > > > > > > > First off, very glad to have found this group.
> > > > > > > >
> > > > > > > > Secondly:
> > > > > > > > Vista 8.03.409A on Progress 10.1B
> > > > > > > > Dual Core Xeon 5160 3GHz, 8GB RAM
> > > > > > > > RAID 1 (OS) + 10 (Epicor)
> > > > > > > > d1: 2.21GB, b1: 341MB
> > > > > > > > ~15 Standard + ~15 MES Users
> > > > > > > >
> > > > > > > > We're currently plagued with performance/speed/lockup issues,
> > and
> > > > would
> > > > > > love to be able to mitigate as much of it as possible in our
> > journey to
> > > > > > Epicor 9.
> > > > > > > >
> > > > > > > > I ran a dump and load at the beginning of the year, and while
> > it
> > > > took
> > > > > > the scatter factor down significantly, it didn't appear to make a
> > > > noticeable
> > > > > > different in the end user experience (but it made me feel a whole
> > lot
> > > > > > better). I also did look at the Progress DB tuning guide, but I
> > should
> > > > > > probably go back through that again.
> > > > > > > >
> > > > > > > > Right now I'm lucky if we go a couple of weeks without
> > something
> > > > > > bringing down the system. I feel like I don't have any visibility
> > into
> > > > > > what's causing the issues.
> > > > > > > >
> > > > > > > > I'd like to be able to monitor the server with something like
> > > > Nagios to
> > > > > > help identify if something is out of order, but it's frustrating
> > that I
> > > > > > rarely see the CPU get higher than 25%. There are some simple
> > processes
> > > > > > (searching for all Report Styles or Data Defs) that on occasion
> > seem to
> > > > take
> > > > > > a whole lot longer than they ought to.
> > > > > > > >
> > > > > > > > We're trying to work with a big Epicor guru for some paid
> > advice,
> > > > but
> > > > > > it's been a little tricky to coordinate scheduling so far.
> > > > > > > >
> > > > > > > > Any thoughts, ideas, or suggestions would be helpful at this
> > point.
> > > > I'm
> > > > > > only 6 months in supporting the system, so just about everything is
> > new
> > > > > > information for me.
> > > > > > > >
> > > > > > > > Thanks in advance!
> > > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > > > [Non-text portions of this message have been removed]
> > > > >
> > > >
> > > >
> > > >
> > >
> > >
> > > [Non-text portions of this message have been removed]
> > >
> >
> >
> >
>
>
> [Non-text portions of this message have been removed]
>
So is anyone feeling particularly generous on this wonderful Monday to try running the previously outlined 'benchmark'?

Or can someone think of another simple tests that might be repeatable amongst various systems? It'd be handy to get some figures to compare the speed of different setups.

--- In vantage@yahoogroups.com, "cubcrafters_it" <jason.navarrete@...> wrote:
>
> So my perflogs didn't turn up anything completely out of the ordinary, but there are a few small things I'll follow up on.
>
> Still need to schedule some time to restart the task and process agents, so we'll see what that does.
>
> Here's what I timed:
> Production Management -> Job Management -> General Operations -> Shop Tracker
> Picked a resource group with a lot of labor rows, then over on the labor tab, clicked 'labor' to bring up Labor Detail Search. Changed the options to return 10,000 rows max, and then clicked search. Timed the time it took to return those rows on my test client. Just used a stopwatch rather than using a trace, as I didn't want that additional overhead.
>
> Average over 5 runs: 10.8 seconds (on 8.03.409A, Progress 10.1B)
>
> If anyone feels like running this test, I'd greatly appreciate it. Or please suggest another benchmark that would be repeatable on other systems.
>
>
>
> --- In vantage@yahoogroups.com, Waffqle <waffqle@> wrote:
> >
> > I'd bump it up a bit and see what happens.You'll need to restart the task
> > agent and the process agent. This won't throw kick everyone off, but it'll
> > stop print jobs and service connect and possibly other stuff.
> >
> > PERC should be good to go.
> >
> > Have you timed any non-print/submitted processes? Adding lines or saving
> > orders/jobs/etc?
> > You can run a trace and use the time info to do a rough benchmark. That's a
> > good way to tell whether it's the server or the client.
> >
> >
> >
> > On Thu, Feb 10, 2011 at 6:00 PM, cubcrafters_it <
> > jason.navarrete@> wrote:
> >
> > >
> > >
> > > You may be onto something important here: my Processing Delay is currently
> > > set to 2 (default is 10?). I've never seen my server on any sort of
> > > sustained processor load, however, so could that still be an issue? Can I
> > > bump up the delay without having to restart anything?
> > >
> > > And as far as the RAID is concerned, it's on a PERC 5i with a BBU, so I
> > > would hope it's a halfway decent setup.
> > >
> > > Still waiting on my performance logs to analyze this afternoon.
> > >
> > > -Jason
> > >
> > >
> > > --- In vantage@yahoogroups.com, Waffqle <waffqle@> wrote:
> > > >
> > > > Print Jobs other 'Submitted Jobs' aren't always the best gauge of
> > > > performance as they use a polling system.
> > > > It may take you 20 seconds to see the print job even if it only took the
> > > > system 2 seconds to produce it. The other 18 seconds, you're just waiting
> > > on
> > > > the System Agent to poll.
> > > >
> > > > You can reduce the polling time in the System Agent. Whatever 'Processing
> > > > Delay' is set to is the maximum time it will take the system to begin a
> > > > submitted job.
> > > >
> > > > Note, be careful not to set it too fast lest it start stepping on itself.
> > > 10
> > > > seconds is probably a safe starting point if it's currently set to a
> > > large
> > > > number. You might be able to get away with 5 seconds if your server is
> > > under
> > > > a pretty low load. A bit of trial and error will get you to the right
> > > > number.
> > > >
> > > > Also, if your system has been locking up and you find that the processing
> > > > delay is set especially low ...you probably just found the culprit.
> > > >
> > > >
> > > > Lastly, I didn't think to ask, but you're not running a white box or
> > > cheap
> > > > (god forbid software) RAID? If your RAID isn't being handled by a proper
> > > > controller, all bets are off as far as performance goes.
> > > >
> > > >
> > > > On Thu, Feb 10, 2011 at 2:00 PM, cubcrafters_it <
> > > > jason.navarrete@> wrote:
> > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > @cooner
> > > > > Yeah, 15 regular/15 MES seems like it ought to be fairly lightweight,
> > > and
> > > > > it's probably less than 10 regular users on a regular basis.
> > > > >
> > > > > What sort of transactions are you talking about and how would I check?
> > > The
> > > > > bulk of any sort of interaction with the system would be around
> > > > > manufacturing/production, if that makes any sense.
> > > > >
> > > > > We're using the same box since I started 7 months ago; it's probably
> > > been
> > > > > formatted once somewhere along the lines, but it's got its fair share
> > > of
> > > > > updates. We've got a new server waiting for Epicor 9, but again, I have
> > > a
> > > > > nagging feeling the current server might be underutilized.
> > > > >
> > > > > I didn't have too much trouble going through the db tuning guide, and
> > > so
> > > > > far it looks like we had a few things over-configured, so I'm not sure
> > > what
> > > > > that means for us either.
> > > > >
> > > > > @waffqle
> > > > > We're not virtualized.
> > > > >
> > > > > I stared at perlogs for a few minutes this morning and the max I saw
> > > > > Current Disk Queue Length was 2. I'm running Microsoft's PAL tool (has
> > > > > anyone else ever used this? I just found it today) this afternoon and
> > > > > logging data for 5 hours, to see if that turns up anything useful.
> > > > >
> > > > > No termserv. Most of our MES computers are old, but even some of our
> > > new
> > > > > machines seem to take longer than it should for some operations or
> > > > > particular reports (but what's normal?).
> > > > >
> > > > > Network issues are a whole other ball of wax, but should be readily
> > > > > identifiable if I run a client instance directly on the server. We've
> > > got
> > > > > entry-level server Gb switches.
> > > > >
> > > > > Out of curiosity, can someone tell me how long it takes them to print
> > > > > preview a Menu Security Report for APRP4100 (1099 Processing),
> > > unchecking
> > > > > 'New Page Per User/Group' but leaving all the other defaults? I get a
> > > 19
> > > > > page report, and it took an average of 9.3 seconds to run directly on
> > > the
> > > > > server.
> > > > >
> > > > > Thanks for all the help so far. The cumulative information here has
> > > been
> > > > > more beneficial than anything I've gotten elsewhere.
> > > > >
> > > > >
> > > > > --- In vantage@yahoogroups.com, "cooner_55421" <cooner_55421@> wrote:
> > > > > >
> > > > > > Hi,
> > > > > >
> > > > > > Looks like you have 15 regular users and 15 MES users?
> > > > > > That doesn't sound like it would be a big strain on your system.
> > > > > >
> > > > > > Just wondering about transactions. Amount, type, etc...?
> > > > > > History of your install. Fresh or a long history of upgrades on the
> > > same
> > > > > box?
> > > > > >
> > > > > > >the Progress DB tuning guide,
> > > > > > >but I should probably go back through that again
> > > > > > I find the guide a little confusing.
> > > > > > I called Epicor Support several times - Tech/Install (option 3 & 1)
> > > > > > I got lucky and connected with a guy who put in some extra effort.
> > > > > > I don't have his name handy right now, anyway...
> > > > > > He was able to explain a couple of things to me I never would have
> > > > > figured out from the guide.
> > > > > >
> > > > > --- In vantage@yahoogroups.com, Waffqle <waffqle@> wrote:
> > > > > >
> > > > > > I'm assuming you aren't virtualized, correct me if I'm wrong.
> > > > > >
> > > > > > If your buffer hits are already at >98% then more RAM isn't going to
> > > get
> > > > > you
> > > > > > anywhere.
> > > > > >
> > > > > > Perfmon should be able to tell you pretty definitively whether or not
> > > > > your
> > > > > > disks are your issue. (Though I would tend to assume they're the
> > > > > > bottleneck.) Load it up and monitor your Average and Current Disk
> > > Queue
> > > > > > Length. If your queue length is higher than "1.5 x # of spindles(4)"
> > > your
> > > > > > disks are holding you back.
> > > > > > It's tempting to look at Read/Write and other IO measurements, but at
> > > the
> > > > > > end of the day if your queue is long, your disks are overloaded.
> > > > > >
> > > > > > Also have you looked at your clients? The Vantage/Epicor client is
> > > not a
> > > > > > lightweight piece of software. If you're running a term serv, check
> > > the
> > > > > load
> > > > > > on that as well.
> > > > > >
> > > > > > Look at your network out as well. As you're only running 15 users, I
> > > > > imagine
> > > > > > your network isn't too huge or overloaded; but it's certainly worth
> > > > > > checking. Check the load on your switches/routers.
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Thu, Feb 10, 2011 at 11:28 AM, cubcrafters_it <
> > > > > > jason.navarrete@> wrote:
> > > > > >
> > > > > > >
> > > > > > >
> > > > > > > We're only using 5 out of our 8 GB of RAM. Can I devote more of it
> > > to
> > > > > the
> > > > > > > DB or App servers? Would that be beneficial? Buffer Hits are
> > > already at
> > > > > > > ~98%.
> > > > > > >
> > > > > > > Disks are 15k SAS 3Gb/s drives, but yes, everything Epicor is on
> > > the
> > > > > same
> > > > > > > RAID 10. Does anyone happen to have documentation on moving temp
> > > dirs
> > > > > to
> > > > > > > different spindles? I'll also try monitoring disk access to see
> > > what
> > > > > that
> > > > > > > tells me.
> > > > > > >
> > > > > > > I'm a little hesitant to throw hardware at the problem when I'm not
> > > > > > > completely convinced that it will solve the problem (partially
> > > because
> > > > > I'm
> > > > > > > cheap).
> > > > > > >
> > > > > > >
> > > > > > > --- In vantage@yahoogroups.com, "markbetts66" <markbetts66@>
> > > wrote:
> > > > > > > >
> > > > > > > > Hi,
> > > > > > > >
> > > > > > > > You may want to try more RAM. What type of disks are you using in
> > > > > your
> > > > > > > server? When you say RAID10(EPICOR) do you mean DB, Appserver,
> > > progress
> > > > > Temp
> > > > > > > Dir and report Temp Dir on the same disk?
> > > > > > > >
> > > > > > > > --- In vantage@yahoogroups.com, "cubcrafters_it"
> > > <jason.navarrete@>
> > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > First off, very glad to have found this group.
> > > > > > > > >
> > > > > > > > > Secondly:
> > > > > > > > > Vista 8.03.409A on Progress 10.1B
> > > > > > > > > Dual Core Xeon 5160 3GHz, 8GB RAM
> > > > > > > > > RAID 1 (OS) + 10 (Epicor)
> > > > > > > > > d1: 2.21GB, b1: 341MB
> > > > > > > > > ~15 Standard + ~15 MES Users
> > > > > > > > >
> > > > > > > > > We're currently plagued with performance/speed/lockup issues,
> > > and
> > > > > would
> > > > > > > love to be able to mitigate as much of it as possible in our
> > > journey to
> > > > > > > Epicor 9.
> > > > > > > > >
> > > > > > > > > I ran a dump and load at the beginning of the year, and while
> > > it
> > > > > took
> > > > > > > the scatter factor down significantly, it didn't appear to make a
> > > > > noticeable
> > > > > > > different in the end user experience (but it made me feel a whole
> > > lot
> > > > > > > better). I also did look at the Progress DB tuning guide, but I
> > > should
> > > > > > > probably go back through that again.
> > > > > > > > >
> > > > > > > > > Right now I'm lucky if we go a couple of weeks without
> > > something
> > > > > > > bringing down the system. I feel like I don't have any visibility
> > > into
> > > > > > > what's causing the issues.
> > > > > > > > >
> > > > > > > > > I'd like to be able to monitor the server with something like
> > > > > Nagios to
> > > > > > > help identify if something is out of order, but it's frustrating
> > > that I
> > > > > > > rarely see the CPU get higher than 25%. There are some simple
> > > processes
> > > > > > > (searching for all Report Styles or Data Defs) that on occasion
> > > seem to
> > > > > take
> > > > > > > a whole lot longer than they ought to.
> > > > > > > > >
> > > > > > > > > We're trying to work with a big Epicor guru for some paid
> > > advice,
> > > > > but
> > > > > > > it's been a little tricky to coordinate scheduling so far.
> > > > > > > > >
> > > > > > > > > Any thoughts, ideas, or suggestions would be helpful at this
> > > point.
> > > > > I'm
> > > > > > > only 6 months in supporting the system, so just about everything is
> > > new
> > > > > > > information for me.
> > > > > > > > >
> > > > > > > > > Thanks in advance!
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > > [Non-text portions of this message have been removed]
> > > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > > >
> > > > [Non-text portions of this message have been removed]
> > > >
> > >
> > >
> > >
> >
> >
> > [Non-text portions of this message have been removed]
> >
>
How many records were you returning on that search? If you were getting 10k
records a 10sec search wouldn't be considered too bad.

On Mon, Feb 21, 2011 at 11:42 AM, cubcrafters_it <
jason.navarrete@...> wrote:

>
>
> So is anyone feeling particularly generous on this wonderful Monday to try
> running the previously outlined 'benchmark'?
>
> Or can someone think of another simple tests that might be repeatable
> amongst various systems? It'd be handy to get some figures to compare the
> speed of different setups.
>
>
> --- In vantage@yahoogroups.com, "cubcrafters_it" <jason.navarrete@...>
> wrote:
> >
> > So my perflogs didn't turn up anything completely out of the ordinary,
> but there are a few small things I'll follow up on.
> >
> > Still need to schedule some time to restart the task and process agents,
> so we'll see what that does.
> >
> > Here's what I timed:
> > Production Management -> Job Management -> General Operations -> Shop
> Tracker
> > Picked a resource group with a lot of labor rows, then over on the labor
> tab, clicked 'labor' to bring up Labor Detail Search. Changed the options to
> return 10,000 rows max, and then clicked search. Timed the time it took to
> return those rows on my test client. Just used a stopwatch rather than using
> a trace, as I didn't want that additional overhead.
> >
> > Average over 5 runs: 10.8 seconds (on 8.03.409A, Progress 10.1B)
> >
> > If anyone feels like running this test, I'd greatly appreciate it. Or
> please suggest another benchmark that would be repeatable on other systems.
> >
> >
> >
> > --- In vantage@yahoogroups.com, Waffqle <waffqle@> wrote:
> > >
> > > I'd bump it up a bit and see what happens.You'll need to restart the
> task
> > > agent and the process agent. This won't throw kick everyone off, but
> it'll
> > > stop print jobs and service connect and possibly other stuff.
> > >
> > > PERC should be good to go.
> > >
> > > Have you timed any non-print/submitted processes? Adding lines or
> saving
> > > orders/jobs/etc?
> > > You can run a trace and use the time info to do a rough benchmark.
> That's a
> > > good way to tell whether it's the server or the client.
> > >
> > >
> > >
> > > On Thu, Feb 10, 2011 at 6:00 PM, cubcrafters_it <
> > > jason.navarrete@> wrote:
> > >
> > > >
> > > >
> > > > You may be onto something important here: my Processing Delay is
> currently
> > > > set to 2 (default is 10?). I've never seen my server on any sort of
> > > > sustained processor load, however, so could that still be an issue?
> Can I
> > > > bump up the delay without having to restart anything?
> > > >
> > > > And as far as the RAID is concerned, it's on a PERC 5i with a BBU, so
> I
> > > > would hope it's a halfway decent setup.
> > > >
> > > > Still waiting on my performance logs to analyze this afternoon.
> > > >
> > > > -Jason
> > > >
> > > >
> > > > --- In vantage@yahoogroups.com, Waffqle <waffqle@> wrote:
> > > > >
> > > > > Print Jobs other 'Submitted Jobs' aren't always the best gauge of
> > > > > performance as they use a polling system.
> > > > > It may take you 20 seconds to see the print job even if it only
> took the
> > > > > system 2 seconds to produce it. The other 18 seconds, you're just
> waiting
> > > > on
> > > > > the System Agent to poll.
> > > > >
> > > > > You can reduce the polling time in the System Agent. Whatever
> 'Processing
> > > > > Delay' is set to is the maximum time it will take the system to
> begin a
> > > > > submitted job.
> > > > >
> > > > > Note, be careful not to set it too fast lest it start stepping on
> itself.
> > > > 10
> > > > > seconds is probably a safe starting point if it's currently set to
> a
> > > > large
> > > > > number. You might be able to get away with 5 seconds if your server
> is
> > > > under
> > > > > a pretty low load. A bit of trial and error will get you to the
> right
> > > > > number.
> > > > >
> > > > > Also, if your system has been locking up and you find that the
> processing
> > > > > delay is set especially low ...you probably just found the culprit.
> > > > >
> > > > >
> > > > > Lastly, I didn't think to ask, but you're not running a white box
> or
> > > > cheap
> > > > > (god forbid software) RAID? If your RAID isn't being handled by a
> proper
> > > > > controller, all bets are off as far as performance goes.
> > > > >
> > > > >
> > > > > On Thu, Feb 10, 2011 at 2:00 PM, cubcrafters_it <
> > > > > jason.navarrete@> wrote:
> > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > @cooner
> > > > > > Yeah, 15 regular/15 MES seems like it ought to be fairly
> lightweight,
> > > > and
> > > > > > it's probably less than 10 regular users on a regular basis.
> > > > > >
> > > > > > What sort of transactions are you talking about and how would I
> check?
> > > > The
> > > > > > bulk of any sort of interaction with the system would be around
> > > > > > manufacturing/production, if that makes any sense.
> > > > > >
> > > > > > We're using the same box since I started 7 months ago; it's
> probably
> > > > been
> > > > > > formatted once somewhere along the lines, but it's got its fair
> share
> > > > of
> > > > > > updates. We've got a new server waiting for Epicor 9, but again,
> I have
> > > > a
> > > > > > nagging feeling the current server might be underutilized.
> > > > > >
> > > > > > I didn't have too much trouble going through the db tuning guide,
> and
> > > > so
> > > > > > far it looks like we had a few things over-configured, so I'm not
> sure
> > > > what
> > > > > > that means for us either.
> > > > > >
> > > > > > @waffqle
> > > > > > We're not virtualized.
> > > > > >
> > > > > > I stared at perlogs for a few minutes this morning and the max I
> saw
> > > > > > Current Disk Queue Length was 2. I'm running Microsoft's PAL tool
> (has
> > > > > > anyone else ever used this? I just found it today) this afternoon
> and
> > > > > > logging data for 5 hours, to see if that turns up anything
> useful.
> > > > > >
> > > > > > No termserv. Most of our MES computers are old, but even some of
> our
> > > > new
> > > > > > machines seem to take longer than it should for some operations
> or
> > > > > > particular reports (but what's normal?).
> > > > > >
> > > > > > Network issues are a whole other ball of wax, but should be
> readily
> > > > > > identifiable if I run a client instance directly on the server.
> We've
> > > > got
> > > > > > entry-level server Gb switches.
> > > > > >
> > > > > > Out of curiosity, can someone tell me how long it takes them to
> print
> > > > > > preview a Menu Security Report for APRP4100 (1099 Processing),
> > > > unchecking
> > > > > > 'New Page Per User/Group' but leaving all the other defaults? I
> get a
> > > > 19
> > > > > > page report, and it took an average of 9.3 seconds to run
> directly on
> > > > the
> > > > > > server.
> > > > > >
> > > > > > Thanks for all the help so far. The cumulative information here
> has
> > > > been
> > > > > > more beneficial than anything I've gotten elsewhere.
> > > > > >
> > > > > >
> > > > > > --- In vantage@yahoogroups.com, "cooner_55421" <cooner_55421@>
> wrote:
> > > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > Looks like you have 15 regular users and 15 MES users?
> > > > > > > That doesn't sound like it would be a big strain on your
> system.
> > > > > > >
> > > > > > > Just wondering about transactions. Amount, type, etc...?
> > > > > > > History of your install. Fresh or a long history of upgrades on
> the
> > > > same
> > > > > > box?
> > > > > > >
> > > > > > > >the Progress DB tuning guide,
> > > > > > > >but I should probably go back through that again
> > > > > > > I find the guide a little confusing.
> > > > > > > I called Epicor Support several times - Tech/Install (option 3
> & 1)
> > > > > > > I got lucky and connected with a guy who put in some extra
> effort.
> > > > > > > I don't have his name handy right now, anyway...
> > > > > > > He was able to explain a couple of things to me I never would
> have
> > > > > > figured out from the guide.
> > > > > > >
> > > > > > --- In vantage@yahoogroups.com, Waffqle <waffqle@> wrote:
> > > > > > >
> > > > > > > I'm assuming you aren't virtualized, correct me if I'm wrong.
> > > > > > >
> > > > > > > If your buffer hits are already at >98% then more RAM isn't
> going to
> > > > get
> > > > > > you
> > > > > > > anywhere.
> > > > > > >
> > > > > > > Perfmon should be able to tell you pretty definitively whether
> or not
> > > > > > your
> > > > > > > disks are your issue. (Though I would tend to assume they're
> the
> > > > > > > bottleneck.) Load it up and monitor your Average and Current
> Disk
> > > > Queue
> > > > > > > Length. If your queue length is higher than "1.5 x # of
> spindles(4)"
> > > > your
> > > > > > > disks are holding you back.
> > > > > > > It's tempting to look at Read/Write and other IO measurements,
> but at
> > > > the
> > > > > > > end of the day if your queue is long, your disks are
> overloaded.
> > > > > > >
> > > > > > > Also have you looked at your clients? The Vantage/Epicor client
> is
> > > > not a
> > > > > > > lightweight piece of software. If you're running a term serv,
> check
> > > > the
> > > > > > load
> > > > > > > on that as well.
> > > > > > >
> > > > > > > Look at your network out as well. As you're only running 15
> users, I
> > > > > > imagine
> > > > > > > your network isn't too huge or overloaded; but it's certainly
> worth
> > > > > > > checking. Check the load on your switches/routers.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > On Thu, Feb 10, 2011 at 11:28 AM, cubcrafters_it <
> > > > > > > jason.navarrete@> wrote:
> > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > We're only using 5 out of our 8 GB of RAM. Can I devote more
> of it
> > > > to
> > > > > > the
> > > > > > > > DB or App servers? Would that be beneficial? Buffer Hits are
> > > > already at
> > > > > > > > ~98%.
> > > > > > > >
> > > > > > > > Disks are 15k SAS 3Gb/s drives, but yes, everything Epicor is
> on
> > > > the
> > > > > > same
> > > > > > > > RAID 10. Does anyone happen to have documentation on moving
> temp
> > > > dirs
> > > > > > to
> > > > > > > > different spindles? I'll also try monitoring disk access to
> see
> > > > what
> > > > > > that
> > > > > > > > tells me.
> > > > > > > >
> > > > > > > > I'm a little hesitant to throw hardware at the problem when
> I'm not
> > > > > > > > completely convinced that it will solve the problem
> (partially
> > > > because
> > > > > > I'm
> > > > > > > > cheap).
> > > > > > > >
> > > > > > > >
> > > > > > > > --- In vantage@yahoogroups.com, "markbetts66" <markbetts66@>
> > > > wrote:
> > > > > > > > >
> > > > > > > > > Hi,
> > > > > > > > >
> > > > > > > > > You may want to try more RAM. What type of disks are you
> using in
> > > > > > your
> > > > > > > > server? When you say RAID10(EPICOR) do you mean DB,
> Appserver,
> > > > progress
> > > > > > Temp
> > > > > > > > Dir and report Temp Dir on the same disk?
> > > > > > > > >
> > > > > > > > > --- In vantage@yahoogroups.com, "cubcrafters_it"
> > > > <jason.navarrete@>
> > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > First off, very glad to have found this group.
> > > > > > > > > >
> > > > > > > > > > Secondly:
> > > > > > > > > > Vista 8.03.409A on Progress 10.1B
> > > > > > > > > > Dual Core Xeon 5160 3GHz, 8GB RAM
> > > > > > > > > > RAID 1 (OS) + 10 (Epicor)
> > > > > > > > > > d1: 2.21GB, b1: 341MB
> > > > > > > > > > ~15 Standard + ~15 MES Users
> > > > > > > > > >
> > > > > > > > > > We're currently plagued with performance/speed/lockup
> issues,
> > > > and
> > > > > > would
> > > > > > > > love to be able to mitigate as much of it as possible in our
> > > > journey to
> > > > > > > > Epicor 9.
> > > > > > > > > >
> > > > > > > > > > I ran a dump and load at the beginning of the year, and
> while
> > > > it
> > > > > > took
> > > > > > > > the scatter factor down significantly, it didn't appear to
> make a
> > > > > > noticeable
> > > > > > > > different in the end user experience (but it made me feel a
> whole
> > > > lot
> > > > > > > > better). I also did look at the Progress DB tuning guide, but
> I
> > > > should
> > > > > > > > probably go back through that again.
> > > > > > > > > >
> > > > > > > > > > Right now I'm lucky if we go a couple of weeks without
> > > > something
> > > > > > > > bringing down the system. I feel like I don't have any
> visibility
> > > > into
> > > > > > > > what's causing the issues.
> > > > > > > > > >
> > > > > > > > > > I'd like to be able to monitor the server with something
> like
> > > > > > Nagios to
> > > > > > > > help identify if something is out of order, but it's
> frustrating
> > > > that I
> > > > > > > > rarely see the CPU get higher than 25%. There are some simple
> > > > processes
> > > > > > > > (searching for all Report Styles or Data Defs) that on
> occasion
> > > > seem to
> > > > > > take
> > > > > > > > a whole lot longer than they ought to.
> > > > > > > > > >
> > > > > > > > > > We're trying to work with a big Epicor guru for some paid
> > > > advice,
> > > > > > but
> > > > > > > > it's been a little tricky to coordinate scheduling so far.
> > > > > > > > > >
> > > > > > > > > > Any thoughts, ideas, or suggestions would be helpful at
> this
> > > > point.
> > > > > > I'm
> > > > > > > > only 6 months in supporting the system, so just about
> everything is
> > > > new
> > > > > > > > information for me.
> > > > > > > > > >
> > > > > > > > > > Thanks in advance!
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > [Non-text portions of this message have been removed]
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > > > [Non-text portions of this message have been removed]
> > > > >
> > > >
> > > >
> > > >
> > >
> > >
> > > [Non-text portions of this message have been removed]
> > >
> >
>
>
>


[Non-text portions of this message have been removed]
Yeah, ~10.8 to return 10,000 rows to my client (Win 7 VM). Haven't yet tried directly on the server.

I was just trying to find something that other folks would have data against without getting too heavy into the customizations.

Anyone else feel like posting some figures? :)

--- In vantage@yahoogroups.com, Waffqle <waffqle@...> wrote:
>
> How many records were you returning on that search? If you were getting 10k
> records a 10sec search wouldn't be considered too bad.
>
> On Mon, Feb 21, 2011 at 11:42 AM, cubcrafters_it <
> jason.navarrete@...> wrote:
>
> >
> >
> > So is anyone feeling particularly generous on this wonderful Monday to try
> > running the previously outlined 'benchmark'?
> >
> > Or can someone think of another simple tests that might be repeatable
> > amongst various systems? It'd be handy to get some figures to compare the
> > speed of different setups.
> >
> >
> > --- In vantage@yahoogroups.com, "cubcrafters_it" <jason.navarrete@>
> > wrote:
> > >
> > > So my perflogs didn't turn up anything completely out of the ordinary,
> > but there are a few small things I'll follow up on.
> > >
> > > Still need to schedule some time to restart the task and process agents,
> > so we'll see what that does.
> > >
> > > Here's what I timed:
> > > Production Management -> Job Management -> General Operations -> Shop
> > Tracker
> > > Picked a resource group with a lot of labor rows, then over on the labor
> > tab, clicked 'labor' to bring up Labor Detail Search. Changed the options to
> > return 10,000 rows max, and then clicked search. Timed the time it took to
> > return those rows on my test client. Just used a stopwatch rather than using
> > a trace, as I didn't want that additional overhead.
> > >
> > > Average over 5 runs: 10.8 seconds (on 8.03.409A, Progress 10.1B)
> > >
> > > If anyone feels like running this test, I'd greatly appreciate it. Or
> > please suggest another benchmark that would be repeatable on other systems.
> > >
> > >
> > >
> > > --- In vantage@yahoogroups.com, Waffqle <waffqle@> wrote:
> > > >
> > > > I'd bump it up a bit and see what happens.You'll need to restart the
> > task
> > > > agent and the process agent. This won't throw kick everyone off, but
> > it'll
> > > > stop print jobs and service connect and possibly other stuff.
> > > >
> > > > PERC should be good to go.
> > > >
> > > > Have you timed any non-print/submitted processes? Adding lines or
> > saving
> > > > orders/jobs/etc?
> > > > You can run a trace and use the time info to do a rough benchmark.
> > That's a
> > > > good way to tell whether it's the server or the client.
> > > >
> > > >
> > > >
> > > > On Thu, Feb 10, 2011 at 6:00 PM, cubcrafters_it <
> > > > jason.navarrete@> wrote:
> > > >
> > > > >
> > > > >
> > > > > You may be onto something important here: my Processing Delay is
> > currently
> > > > > set to 2 (default is 10?). I've never seen my server on any sort of
> > > > > sustained processor load, however, so could that still be an issue?
> > Can I
> > > > > bump up the delay without having to restart anything?
> > > > >
> > > > > And as far as the RAID is concerned, it's on a PERC 5i with a BBU, so
> > I
> > > > > would hope it's a halfway decent setup.
> > > > >
> > > > > Still waiting on my performance logs to analyze this afternoon.
> > > > >
> > > > > -Jason
> > > > >
> > > > >
> > > > > --- In vantage@yahoogroups.com, Waffqle <waffqle@> wrote:
> > > > > >
> > > > > > Print Jobs other 'Submitted Jobs' aren't always the best gauge of
> > > > > > performance as they use a polling system.
> > > > > > It may take you 20 seconds to see the print job even if it only
> > took the
> > > > > > system 2 seconds to produce it. The other 18 seconds, you're just
> > waiting
> > > > > on
> > > > > > the System Agent to poll.
> > > > > >
> > > > > > You can reduce the polling time in the System Agent. Whatever
> > 'Processing
> > > > > > Delay' is set to is the maximum time it will take the system to
> > begin a
> > > > > > submitted job.
> > > > > >
> > > > > > Note, be careful not to set it too fast lest it start stepping on
> > itself.
> > > > > 10
> > > > > > seconds is probably a safe starting point if it's currently set to
> > a
> > > > > large
> > > > > > number. You might be able to get away with 5 seconds if your server
> > is
> > > > > under
> > > > > > a pretty low load. A bit of trial and error will get you to the
> > right
> > > > > > number.
> > > > > >
> > > > > > Also, if your system has been locking up and you find that the
> > processing
> > > > > > delay is set especially low ...you probably just found the culprit.
> > > > > >
> > > > > >
> > > > > > Lastly, I didn't think to ask, but you're not running a white box
> > or
> > > > > cheap
> > > > > > (god forbid software) RAID? If your RAID isn't being handled by a
> > proper
> > > > > > controller, all bets are off as far as performance goes.
> > > > > >
> > > > > >
> > > > > > On Thu, Feb 10, 2011 at 2:00 PM, cubcrafters_it <
> > > > > > jason.navarrete@> wrote:
> > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > @cooner
> > > > > > > Yeah, 15 regular/15 MES seems like it ought to be fairly
> > lightweight,
> > > > > and
> > > > > > > it's probably less than 10 regular users on a regular basis.
> > > > > > >
> > > > > > > What sort of transactions are you talking about and how would I
> > check?
> > > > > The
> > > > > > > bulk of any sort of interaction with the system would be around
> > > > > > > manufacturing/production, if that makes any sense.
> > > > > > >
> > > > > > > We're using the same box since I started 7 months ago; it's
> > probably
> > > > > been
> > > > > > > formatted once somewhere along the lines, but it's got its fair
> > share
> > > > > of
> > > > > > > updates. We've got a new server waiting for Epicor 9, but again,
> > I have
> > > > > a
> > > > > > > nagging feeling the current server might be underutilized.
> > > > > > >
> > > > > > > I didn't have too much trouble going through the db tuning guide,
> > and
> > > > > so
> > > > > > > far it looks like we had a few things over-configured, so I'm not
> > sure
> > > > > what
> > > > > > > that means for us either.
> > > > > > >
> > > > > > > @waffqle
> > > > > > > We're not virtualized.
> > > > > > >
> > > > > > > I stared at perlogs for a few minutes this morning and the max I
> > saw
> > > > > > > Current Disk Queue Length was 2. I'm running Microsoft's PAL tool
> > (has
> > > > > > > anyone else ever used this? I just found it today) this afternoon
> > and
> > > > > > > logging data for 5 hours, to see if that turns up anything
> > useful.
> > > > > > >
> > > > > > > No termserv. Most of our MES computers are old, but even some of
> > our
> > > > > new
> > > > > > > machines seem to take longer than it should for some operations
> > or
> > > > > > > particular reports (but what's normal?).
> > > > > > >
> > > > > > > Network issues are a whole other ball of wax, but should be
> > readily
> > > > > > > identifiable if I run a client instance directly on the server.
> > We've
> > > > > got
> > > > > > > entry-level server Gb switches.
> > > > > > >
> > > > > > > Out of curiosity, can someone tell me how long it takes them to
> > print
> > > > > > > preview a Menu Security Report for APRP4100 (1099 Processing),
> > > > > unchecking
> > > > > > > 'New Page Per User/Group' but leaving all the other defaults? I
> > get a
> > > > > 19
> > > > > > > page report, and it took an average of 9.3 seconds to run
> > directly on
> > > > > the
> > > > > > > server.
> > > > > > >
> > > > > > > Thanks for all the help so far. The cumulative information here
> > has
> > > > > been
> > > > > > > more beneficial than anything I've gotten elsewhere.
> > > > > > >
> > > > > > >
> > > > > > > --- In vantage@yahoogroups.com, "cooner_55421" <cooner_55421@>
> > wrote:
> > > > > > > >
> > > > > > > > Hi,
> > > > > > > >
> > > > > > > > Looks like you have 15 regular users and 15 MES users?
> > > > > > > > That doesn't sound like it would be a big strain on your
> > system.
> > > > > > > >
> > > > > > > > Just wondering about transactions. Amount, type, etc...?
> > > > > > > > History of your install. Fresh or a long history of upgrades on
> > the
> > > > > same
> > > > > > > box?
> > > > > > > >
> > > > > > > > >the Progress DB tuning guide,
> > > > > > > > >but I should probably go back through that again
> > > > > > > > I find the guide a little confusing.
> > > > > > > > I called Epicor Support several times - Tech/Install (option 3
> > & 1)
> > > > > > > > I got lucky and connected with a guy who put in some extra
> > effort.
> > > > > > > > I don't have his name handy right now, anyway...
> > > > > > > > He was able to explain a couple of things to me I never would
> > have
> > > > > > > figured out from the guide.
> > > > > > > >
> > > > > > > --- In vantage@yahoogroups.com, Waffqle <waffqle@> wrote:
> > > > > > > >
> > > > > > > > I'm assuming you aren't virtualized, correct me if I'm wrong.
> > > > > > > >
> > > > > > > > If your buffer hits are already at >98% then more RAM isn't
> > going to
> > > > > get
> > > > > > > you
> > > > > > > > anywhere.
> > > > > > > >
> > > > > > > > Perfmon should be able to tell you pretty definitively whether
> > or not
> > > > > > > your
> > > > > > > > disks are your issue. (Though I would tend to assume they're
> > the
> > > > > > > > bottleneck.) Load it up and monitor your Average and Current
> > Disk
> > > > > Queue
> > > > > > > > Length. If your queue length is higher than "1.5 x # of
> > spindles(4)"
> > > > > your
> > > > > > > > disks are holding you back.
> > > > > > > > It's tempting to look at Read/Write and other IO measurements,
> > but at
> > > > > the
> > > > > > > > end of the day if your queue is long, your disks are
> > overloaded.
> > > > > > > >
> > > > > > > > Also have you looked at your clients? The Vantage/Epicor client
> > is
> > > > > not a
> > > > > > > > lightweight piece of software. If you're running a term serv,
> > check
> > > > > the
> > > > > > > load
> > > > > > > > on that as well.
> > > > > > > >
> > > > > > > > Look at your network out as well. As you're only running 15
> > users, I
> > > > > > > imagine
> > > > > > > > your network isn't too huge or overloaded; but it's certainly
> > worth
> > > > > > > > checking. Check the load on your switches/routers.
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > On Thu, Feb 10, 2011 at 11:28 AM, cubcrafters_it <
> > > > > > > > jason.navarrete@> wrote:
> > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > We're only using 5 out of our 8 GB of RAM. Can I devote more
> > of it
> > > > > to
> > > > > > > the
> > > > > > > > > DB or App servers? Would that be beneficial? Buffer Hits are
> > > > > already at
> > > > > > > > > ~98%.
> > > > > > > > >
> > > > > > > > > Disks are 15k SAS 3Gb/s drives, but yes, everything Epicor is
> > on
> > > > > the
> > > > > > > same
> > > > > > > > > RAID 10. Does anyone happen to have documentation on moving
> > temp
> > > > > dirs
> > > > > > > to
> > > > > > > > > different spindles? I'll also try monitoring disk access to
> > see
> > > > > what
> > > > > > > that
> > > > > > > > > tells me.
> > > > > > > > >
> > > > > > > > > I'm a little hesitant to throw hardware at the problem when
> > I'm not
> > > > > > > > > completely convinced that it will solve the problem
> > (partially
> > > > > because
> > > > > > > I'm
> > > > > > > > > cheap).
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > --- In vantage@yahoogroups.com, "markbetts66" <markbetts66@>
> > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > Hi,
> > > > > > > > > >
> > > > > > > > > > You may want to try more RAM. What type of disks are you
> > using in
> > > > > > > your
> > > > > > > > > server? When you say RAID10(EPICOR) do you mean DB,
> > Appserver,
> > > > > progress
> > > > > > > Temp
> > > > > > > > > Dir and report Temp Dir on the same disk?
> > > > > > > > > >
> > > > > > > > > > --- In vantage@yahoogroups.com, "cubcrafters_it"
> > > > > <jason.navarrete@>
> > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > First off, very glad to have found this group.
> > > > > > > > > > >
> > > > > > > > > > > Secondly:
> > > > > > > > > > > Vista 8.03.409A on Progress 10.1B
> > > > > > > > > > > Dual Core Xeon 5160 3GHz, 8GB RAM
> > > > > > > > > > > RAID 1 (OS) + 10 (Epicor)
> > > > > > > > > > > d1: 2.21GB, b1: 341MB
> > > > > > > > > > > ~15 Standard + ~15 MES Users
> > > > > > > > > > >
> > > > > > > > > > > We're currently plagued with performance/speed/lockup
> > issues,
> > > > > and
> > > > > > > would
> > > > > > > > > love to be able to mitigate as much of it as possible in our
> > > > > journey to
> > > > > > > > > Epicor 9.
> > > > > > > > > > >
> > > > > > > > > > > I ran a dump and load at the beginning of the year, and
> > while
> > > > > it
> > > > > > > took
> > > > > > > > > the scatter factor down significantly, it didn't appear to
> > make a
> > > > > > > noticeable
> > > > > > > > > different in the end user experience (but it made me feel a
> > whole
> > > > > lot
> > > > > > > > > better). I also did look at the Progress DB tuning guide, but
> > I
> > > > > should
> > > > > > > > > probably go back through that again.
> > > > > > > > > > >
> > > > > > > > > > > Right now I'm lucky if we go a couple of weeks without
> > > > > something
> > > > > > > > > bringing down the system. I feel like I don't have any
> > visibility
> > > > > into
> > > > > > > > > what's causing the issues.
> > > > > > > > > > >
> > > > > > > > > > > I'd like to be able to monitor the server with something
> > like
> > > > > > > Nagios to
> > > > > > > > > help identify if something is out of order, but it's
> > frustrating
> > > > > that I
> > > > > > > > > rarely see the CPU get higher than 25%. There are some simple
> > > > > processes
> > > > > > > > > (searching for all Report Styles or Data Defs) that on
> > occasion
> > > > > seem to
> > > > > > > take
> > > > > > > > > a whole lot longer than they ought to.
> > > > > > > > > > >
> > > > > > > > > > > We're trying to work with a big Epicor guru for some paid
> > > > > advice,
> > > > > > > but
> > > > > > > > > it's been a little tricky to coordinate scheduling so far.
> > > > > > > > > > >
> > > > > > > > > > > Any thoughts, ideas, or suggestions would be helpful at
> > this
> > > > > point.
> > > > > > > I'm
> > > > > > > > > only 6 months in supporting the system, so just about
> > everything is
> > > > > new
> > > > > > > > > information for me.
> > > > > > > > > > >
> > > > > > > > > > > Thanks in advance!
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > [Non-text portions of this message have been removed]
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > > [Non-text portions of this message have been removed]
> > > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > > >
> > > > [Non-text portions of this message have been removed]
> > > >
> > >
> >
> >
> >
>
>
> [Non-text portions of this message have been removed]
>
Using shop tracker as outlined below returning 10,000 rows for me takes approximately 15 - 17 seconds.

That being said I would like to point out that I am using a completely virtual environment with the data stored on a SAN. I am curiously if you guys are on dedicated hardware or virtualized?

"Zac" Jason Woodward
Network Administrator
Intermountain Electronics, Inc.
O: 877-544-2291
M: 435-820-6515
F: 435-637-9601
www.ie-corp.com

Creating customer confidence through extraordinary service and experienced industry experts.

From: vantage@yahoogroups.com [mailto:vantage@yahoogroups.com] On Behalf Of cubcrafters_it
Sent: Monday, February 21, 2011 1:56 PM
To: vantage@yahoogroups.com
Subject: [Vantage] Re: 8.03 Performance Monitoring and Troubleshooting (2011 Edition)



Yeah, ~10.8 to return 10,000 rows to my client (Win 7 VM). Haven't yet tried directly on the server.

I was just trying to find something that other folks would have data against without getting too heavy into the customizations.

Anyone else feel like posting some figures? :)

--- In vantage@yahoogroups.com<mailto:vantage%40yahoogroups.com>, Waffqle <waffqle@...> wrote:
>
> How many records were you returning on that search? If you were getting 10k
> records a 10sec search wouldn't be considered too bad.
>
> On Mon, Feb 21, 2011 at 11:42 AM, cubcrafters_it <
> jason.navarrete@...> wrote:
>
> >
> >
> > So is anyone feeling particularly generous on this wonderful Monday to try
> > running the previously outlined 'benchmark'?
> >
> > Or can someone think of another simple tests that might be repeatable
> > amongst various systems? It'd be handy to get some figures to compare the
> > speed of different setups.
> >
> >
> > --- In vantage@yahoogroups.com<mailto:vantage%40yahoogroups.com>, "cubcrafters_it" <jason.navarrete@>
> > wrote:
> > >
> > > So my perflogs didn't turn up anything completely out of the ordinary,
> > but there are a few small things I'll follow up on.
> > >
> > > Still need to schedule some time to restart the task and process agents,
> > so we'll see what that does.
> > >
> > > Here's what I timed:
> > > Production Management -> Job Management -> General Operations -> Shop
> > Tracker
> > > Picked a resource group with a lot of labor rows, then over on the labor
> > tab, clicked 'labor' to bring up Labor Detail Search. Changed the options to
> > return 10,000 rows max, and then clicked search. Timed the time it took to
> > return those rows on my test client. Just used a stopwatch rather than using
> > a trace, as I didn't want that additional overhead.
> > >
> > > Average over 5 runs: 10.8 seconds (on 8.03.409A, Progress 10.1B)
> > >
> > > If anyone feels like running this test, I'd greatly appreciate it. Or
> > please suggest another benchmark that would be repeatable on other systems.
> > >
> > >
> > >
> > > --- In vantage@yahoogroups.com<mailto:vantage%40yahoogroups.com>, Waffqle <waffqle@> wrote:
> > > >
> > > > I'd bump it up a bit and see what happens.You'll need to restart the
> > task
> > > > agent and the process agent. This won't throw kick everyone off, but
> > it'll
> > > > stop print jobs and service connect and possibly other stuff.
> > > >
> > > > PERC should be good to go.
> > > >
> > > > Have you timed any non-print/submitted processes? Adding lines or
> > saving
> > > > orders/jobs/etc?
> > > > You can run a trace and use the time info to do a rough benchmark.
> > That's a
> > > > good way to tell whether it's the server or the client.
> > > >
> > > >
> > > >
> > > > On Thu, Feb 10, 2011 at 6:00 PM, cubcrafters_it <
> > > > jason.navarrete@> wrote:
> > > >
> > > > >
> > > > >
> > > > > You may be onto something important here: my Processing Delay is
> > currently
> > > > > set to 2 (default is 10?). I've never seen my server on any sort of
> > > > > sustained processor load, however, so could that still be an issue?
> > Can I
> > > > > bump up the delay without having to restart anything?
> > > > >
> > > > > And as far as the RAID is concerned, it's on a PERC 5i with a BBU, so
> > I
> > > > > would hope it's a halfway decent setup.
> > > > >
> > > > > Still waiting on my performance logs to analyze this afternoon.
> > > > >
> > > > > -Jason
> > > > >
> > > > >
> > > > > --- In vantage@yahoogroups.com<mailto:vantage%40yahoogroups.com>, Waffqle <waffqle@> wrote:
> > > > > >
> > > > > > Print Jobs other 'Submitted Jobs' aren't always the best gauge of
> > > > > > performance as they use a polling system.
> > > > > > It may take you 20 seconds to see the print job even if it only
> > took the
> > > > > > system 2 seconds to produce it. The other 18 seconds, you're just
> > waiting
> > > > > on
> > > > > > the System Agent to poll.
> > > > > >
> > > > > > You can reduce the polling time in the System Agent. Whatever
> > 'Processing
> > > > > > Delay' is set to is the maximum time it will take the system to
> > begin a
> > > > > > submitted job.
> > > > > >
> > > > > > Note, be careful not to set it too fast lest it start stepping on
> > itself.
> > > > > 10
> > > > > > seconds is probably a safe starting point if it's currently set to
> > a
> > > > > large
> > > > > > number. You might be able to get away with 5 seconds if your server
> > is
> > > > > under
> > > > > > a pretty low load. A bit of trial and error will get you to the
> > right
> > > > > > number.
> > > > > >
> > > > > > Also, if your system has been locking up and you find that the
> > processing
> > > > > > delay is set especially low ...you probably just found the culprit.
> > > > > >
> > > > > >
> > > > > > Lastly, I didn't think to ask, but you're not running a white box
> > or
> > > > > cheap
> > > > > > (god forbid software) RAID? If your RAID isn't being handled by a
> > proper
> > > > > > controller, all bets are off as far as performance goes.
> > > > > >
> > > > > >
> > > > > > On Thu, Feb 10, 2011 at 2:00 PM, cubcrafters_it <
> > > > > > jason.navarrete@> wrote:
> > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > @cooner
> > > > > > > Yeah, 15 regular/15 MES seems like it ought to be fairly
> > lightweight,
> > > > > and
> > > > > > > it's probably less than 10 regular users on a regular basis.
> > > > > > >
> > > > > > > What sort of transactions are you talking about and how would I
> > check?
> > > > > The
> > > > > > > bulk of any sort of interaction with the system would be around
> > > > > > > manufacturing/production, if that makes any sense.
> > > > > > >
> > > > > > > We're using the same box since I started 7 months ago; it's
> > probably
> > > > > been
> > > > > > > formatted once somewhere along the lines, but it's got its fair
> > share
> > > > > of
> > > > > > > updates. We've got a new server waiting for Epicor 9, but again,
> > I have
> > > > > a
> > > > > > > nagging feeling the current server might be underutilized.
> > > > > > >
> > > > > > > I didn't have too much trouble going through the db tuning guide,
> > and
> > > > > so
> > > > > > > far it looks like we had a few things over-configured, so I'm not
> > sure
> > > > > what
> > > > > > > that means for us either.
> > > > > > >
> > > > > > > @waffqle
> > > > > > > We're not virtualized.
> > > > > > >
> > > > > > > I stared at perlogs for a few minutes this morning and the max I
> > saw
> > > > > > > Current Disk Queue Length was 2. I'm running Microsoft's PAL tool
> > (has
> > > > > > > anyone else ever used this? I just found it today) this afternoon
> > and
> > > > > > > logging data for 5 hours, to see if that turns up anything
> > useful.
> > > > > > >
> > > > > > > No termserv. Most of our MES computers are old, but even some of
> > our
> > > > > new
> > > > > > > machines seem to take longer than it should for some operations
> > or
> > > > > > > particular reports (but what's normal?).
> > > > > > >
> > > > > > > Network issues are a whole other ball of wax, but should be
> > readily
> > > > > > > identifiable if I run a client instance directly on the server.
> > We've
> > > > > got
> > > > > > > entry-level server Gb switches.
> > > > > > >
> > > > > > > Out of curiosity, can someone tell me how long it takes them to
> > print
> > > > > > > preview a Menu Security Report for APRP4100 (1099 Processing),
> > > > > unchecking
> > > > > > > 'New Page Per User/Group' but leaving all the other defaults? I
> > get a
> > > > > 19
> > > > > > > page report, and it took an average of 9.3 seconds to run
> > directly on
> > > > > the
> > > > > > > server.
> > > > > > >
> > > > > > > Thanks for all the help so far. The cumulative information here
> > has
> > > > > been
> > > > > > > more beneficial than anything I've gotten elsewhere.
> > > > > > >
> > > > > > >
> > > > > > > --- In vantage@yahoogroups.com<mailto:vantage%40yahoogroups.com>, "cooner_55421" <cooner_55421@>
> > wrote:
> > > > > > > >
> > > > > > > > Hi,
> > > > > > > >
> > > > > > > > Looks like you have 15 regular users and 15 MES users?
> > > > > > > > That doesn't sound like it would be a big strain on your
> > system.
> > > > > > > >
> > > > > > > > Just wondering about transactions. Amount, type, etc...?
> > > > > > > > History of your install. Fresh or a long history of upgrades on
> > the
> > > > > same
> > > > > > > box?
> > > > > > > >
> > > > > > > > >the Progress DB tuning guide,
> > > > > > > > >but I should probably go back through that again
> > > > > > > > I find the guide a little confusing.
> > > > > > > > I called Epicor Support several times - Tech/Install (option 3
> > & 1)
> > > > > > > > I got lucky and connected with a guy who put in some extra
> > effort.
> > > > > > > > I don't have his name handy right now, anyway...
> > > > > > > > He was able to explain a couple of things to me I never would
> > have
> > > > > > > figured out from the guide.
> > > > > > > >
> > > > > > > --- In vantage@yahoogroups.com<mailto:vantage%40yahoogroups.com>, Waffqle <waffqle@> wrote:
> > > > > > > >
> > > > > > > > I'm assuming you aren't virtualized, correct me if I'm wrong.
> > > > > > > >
> > > > > > > > If your buffer hits are already at >98% then more RAM isn't
> > going to
> > > > > get
> > > > > > > you
> > > > > > > > anywhere.
> > > > > > > >
> > > > > > > > Perfmon should be able to tell you pretty definitively whether
> > or not
> > > > > > > your
> > > > > > > > disks are your issue. (Though I would tend to assume they're
> > the
> > > > > > > > bottleneck.) Load it up and monitor your Average and Current
> > Disk
> > > > > Queue
> > > > > > > > Length. If your queue length is higher than "1.5 x # of
> > spindles(4)"
> > > > > your
> > > > > > > > disks are holding you back.
> > > > > > > > It's tempting to look at Read/Write and other IO measurements,
> > but at
> > > > > the
> > > > > > > > end of the day if your queue is long, your disks are
> > overloaded.
> > > > > > > >
> > > > > > > > Also have you looked at your clients? The Vantage/Epicor client
> > is
> > > > > not a
> > > > > > > > lightweight piece of software. If you're running a term serv,
> > check
> > > > > the
> > > > > > > load
> > > > > > > > on that as well.
> > > > > > > >
> > > > > > > > Look at your network out as well. As you're only running 15
> > users, I
> > > > > > > imagine
> > > > > > > > your network isn't too huge or overloaded; but it's certainly
> > worth
> > > > > > > > checking. Check the load on your switches/routers.
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > On Thu, Feb 10, 2011 at 11:28 AM, cubcrafters_it <
> > > > > > > > jason.navarrete@> wrote:
> > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > We're only using 5 out of our 8 GB of RAM. Can I devote more
> > of it
> > > > > to
> > > > > > > the
> > > > > > > > > DB or App servers? Would that be beneficial? Buffer Hits are
> > > > > already at
> > > > > > > > > ~98%.
> > > > > > > > >
> > > > > > > > > Disks are 15k SAS 3Gb/s drives, but yes, everything Epicor is
> > on
> > > > > the
> > > > > > > same
> > > > > > > > > RAID 10. Does anyone happen to have documentation on moving
> > temp
> > > > > dirs
> > > > > > > to
> > > > > > > > > different spindles? I'll also try monitoring disk access to
> > see
> > > > > what
> > > > > > > that
> > > > > > > > > tells me.
> > > > > > > > >
> > > > > > > > > I'm a little hesitant to throw hardware at the problem when
> > I'm not
> > > > > > > > > completely convinced that it will solve the problem
> > (partially
> > > > > because
> > > > > > > I'm
> > > > > > > > > cheap).
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > --- In vantage@yahoogroups.com<mailto:vantage%40yahoogroups.com>, "markbetts66" <markbetts66@>
> > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > Hi,
> > > > > > > > > >
> > > > > > > > > > You may want to try more RAM. What type of disks are you
> > using in
> > > > > > > your
> > > > > > > > > server? When you say RAID10(EPICOR) do you mean DB,
> > Appserver,
> > > > > progress
> > > > > > > Temp
> > > > > > > > > Dir and report Temp Dir on the same disk?
> > > > > > > > > >
> > > > > > > > > > --- In vantage@yahoogroups.com<mailto:vantage%40yahoogroups.com>, "cubcrafters_it"
> > > > > <jason.navarrete@>
> > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > First off, very glad to have found this group.
> > > > > > > > > > >
> > > > > > > > > > > Secondly:
> > > > > > > > > > > Vista 8.03.409A on Progress 10.1B
> > > > > > > > > > > Dual Core Xeon 5160 3GHz, 8GB RAM
> > > > > > > > > > > RAID 1 (OS) + 10 (Epicor)
> > > > > > > > > > > d1: 2.21GB, b1: 341MB
> > > > > > > > > > > ~15 Standard + ~15 MES Users
> > > > > > > > > > >
> > > > > > > > > > > We're currently plagued with performance/speed/lockup
> > issues,
> > > > > and
> > > > > > > would
> > > > > > > > > love to be able to mitigate as much of it as possible in our
> > > > > journey to
> > > > > > > > > Epicor 9.
> > > > > > > > > > >
> > > > > > > > > > > I ran a dump and load at the beginning of the year, and
> > while
> > > > > it
> > > > > > > took
> > > > > > > > > the scatter factor down significantly, it didn't appear to
> > make a
> > > > > > > noticeable
> > > > > > > > > different in the end user experience (but it made me feel a
> > whole
> > > > > lot
> > > > > > > > > better). I also did look at the Progress DB tuning guide, but
> > I
> > > > > should
> > > > > > > > > probably go back through that again.
> > > > > > > > > > >
> > > > > > > > > > > Right now I'm lucky if we go a couple of weeks without
> > > > > something
> > > > > > > > > bringing down the system. I feel like I don't have any
> > visibility
> > > > > into
> > > > > > > > > what's causing the issues.
> > > > > > > > > > >
> > > > > > > > > > > I'd like to be able to monitor the server with something
> > like
> > > > > > > Nagios to
> > > > > > > > > help identify if something is out of order, but it's
> > frustrating
> > > > > that I
> > > > > > > > > rarely see the CPU get higher than 25%. There are some simple
> > > > > processes
> > > > > > > > > (searching for all Report Styles or Data Defs) that on
> > occasion
> > > > > seem to
> > > > > > > take
> > > > > > > > > a whole lot longer than they ought to.
> > > > > > > > > > >
> > > > > > > > > > > We're trying to work with a big Epicor guru for some paid
> > > > > advice,
> > > > > > > but
> > > > > > > > > it's been a little tricky to coordinate scheduling so far.
> > > > > > > > > > >
> > > > > > > > > > > Any thoughts, ideas, or suggestions would be helpful at
> > this
> > > > > point.
> > > > > > > I'm
> > > > > > > > > only 6 months in supporting the system, so just about
> > everything is
> > > > > new
> > > > > > > > > information for me.
> > > > > > > > > > >
> > > > > > > > > > > Thanks in advance!
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > [Non-text portions of this message have been removed]
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > > [Non-text portions of this message have been removed]
> > > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > > >
> > > > [Non-text portions of this message have been removed]
> > > >
> > >
> >
> >
> >
>
>
> [Non-text portions of this message have been removed]
>



[Non-text portions of this message have been removed]