6

I propose a change to the FAQ and closing policy:

I'd like to see all of the benchmarking questions closed with prejudice; I'm tired of seeing them. In my mind, the answer is almost always the same:

Put your scenario in place, add monitoring, add traffic, evaluate results. Modify based on results. Rinse. Repeat.

Add to the FAQ that questions ranging from the awful

"how many users can a web server take"

to the specific

"I have a 42U rack with maxed-out HP Proliant DL360 G8's; how many concurrent users will that support running [app environment] on [platform]"

are going to be closed, because each application and environment is a unique and (more often ugly than beautiful) snowflake. You need to test your particular snow-flakey circumstances.

This is not to say that questions about poor performance shouldn't be allowed. "I'm running XYZ on ABC with CMYK and I'm RGB is slower than I'd expect. What should I check and what could I do to mitigate?" seems like a reasonable question to me...

1
  • 2
    How much wood would a wood chuck chuck if a wood chuck could chuck wood?
    – Mark Henderson Mod
    Commented Jan 16, 2012 at 3:46

3 Answers 3

Reset to default
5

Has someone done a canonical q/a for this?

If not, someone should make one explaining what they need to do to figure that out themselves, and we can point them all there.

10
  • I'm sure that there are approximately gigs of text on the internet describing people's capacity planning and scaling experiences. "Capacity planning" in Google yields About 17,700,000 results. O'Reilly's "Art of Capacity Planning" shop.oreilly.com/product/9780596518585.do, and a 2002 book called "System Performance Tuning" all come up with easy keyword searches...
    – gWaldo
    Commented Jan 16, 2012 at 3:58
  • Well, maybe we need a canonical Q/A and the answer is "Hey, check out these resources". We just need to be careful of link rot.
    – Mark Henderson Mod
    Commented Jan 16, 2012 at 4:02
  • I think that noting that theirs isn't a formulaic question is a good start, then offer suggestions that they begin by searching for 'Capacity planning,' 'load testing', etc with the specifics of their environment. I agree that link-rot is to be avoided, and I have mixed feelings about recommending books (especially when they get to be more than a couple of years old.)
    – gWaldo
    Commented Jan 16, 2012 at 5:05
  • This was exactly the comment I was scrolling down to make. Giving people the help they need to find out for themselves is a good thing, I think.
    – Rob Moir
    Commented Jan 16, 2012 at 10:22
  • 1
    I've answered this question a number of times, generally of the "this is how you figure it out for yourself," mode. It's been received favorably more of than I've had, "Whut?" or, "Just ^!&$ tell me."
    – sysadmin1138 Mod
    Commented Jan 16, 2012 at 12:33
  • @sysadmin1138 are any of the ones you've answered particularly shining examples of "How do I do capacity planning?" -- I think if we could close these as duplicates of a canonical "This is what you need to do" rather than "Not Constructive, here's a comment saying what you need to consider" it would be more efficient.
    – voretaq7
    Commented Jan 16, 2012 at 20:22
  • Question posted: serverfault.com/q/350454/50875
    – gWaldo
    Commented Jan 16, 2012 at 22:51
  • ...and one for Databases: serverfault.com/q/350458/50875
    – gWaldo
    Commented Jan 16, 2012 at 22:56
  • @gWaldo - ah, I kinda answered generically in the first question, and made it apply to databases. But someone else can clear up my answers :)
    – Mark Henderson Mod
    Commented Jan 16, 2012 at 23:21
  • Well done, sir! I came back from dinner to a masterpiece! I'll add some links for a couple other toos once I'm on at a full machine.
    – gWaldo
    Commented Jan 17, 2012 at 0:58
4

What about someone asking for tools to be used in benchmarking? Seems to me that would be perfectly on topic.

e.g. Someone might be looking for the tools and/or methods which might be employed to perform benchmarking on say various RAID configurations (there's more to the world than just web servers). It's not enough to just throw a load at it, you need to make measurements as well and the right tools can be a great help.

Edit: This question I saw today very nicely illustrates my point.

2
  • Asking about benchmarking / load-testing tools seems fine to me, but that question will only be asked for/about so many apps/environments before fairly definitive answers are available (and new ones can be closed as duplicates.)
    – gWaldo
    Commented Jan 16, 2012 at 22:28
  • @gWaldo, couldn't the same be said about just about any imaginable topic?
    – John Gardeniers
    Commented Jan 17, 2012 at 1:34
3

While we do get a lot of these, I think a FAQ-ban would be a bad idea. We tend to treat the FAQ as holy writ around here, and have zero tolerance attitudes towards things specifically excluded in it. There are enough useful edge-cases around this question that are quite topical that would be chucked right out if we had a FAQ ban that I think we'd be poorer for it.

I think this is better served with a canonical-question for all of the far-too-generic:

I have a webserver with $hardwareSpecs. How many users can that support?

questions. We have any number of answers to this one in the database (I've written a few of 'em), we just need to pick one. Questions a FAQ-ban would throw out are the more specific version of the above that read similar to:

I have a webserver with $hardwareSpecs, and running $appFramework with $appMods. how many users can that support?

This, unlike the first, is answerable. And by "answerable" I don't mean providing them with a number, I mean providing guidance on how to figure it out for themselves. We will continue to get some people who come back with, "stop prevaricating, just &@#! tell me," and we can continue to close those. But I've answered enough of these where the response was, "I had no idea how to do that, thanks," that there is value to such longer answers.

5
  • I myself think that the second question is better worded, but really the answer is the same for both: you need to test your environment yourself. I think that that is a reasonable general-purpose answer that fits every case (and so, meets FAQ criteria.) questions about tools and methods of benchmarking / capacity-planning are welcome, however.
    – gWaldo
    Commented Jan 16, 2012 at 22:36
  • 1
    @gWaldo The same generic answer applies to both yes. However, the second allows the answerer to give specific advice for tools to use, which the first doesn't. That's the key differentiators in my mind. The second is specific enough to not be Overly Broad or Too Localized.
    – sysadmin1138 Mod
    Commented Jan 16, 2012 at 22:39
  • Not being a Mod myself, is a periodic review of the FAQ occur, just to ensure that it still makes sense? The thinking being that is for instance this suggestion were applied, but found that it caused undue friction/pain it could be reviewed and rescinded?
    – gWaldo
    Commented Jan 16, 2012 at 22:39
  • Ok, I'm with you there, but the answerer has to preface their answer of helpful tools with 'you need to determine that yourself; only you know your environment, but I recommend that you use these: ...'
    – gWaldo
    Commented Jan 16, 2012 at 22:42
  • I'd agree with this, with the proviso that "what tools to use" for the second answer should be a canonical question & answer.
    – Rob Moir
    Commented Jan 24, 2012 at 15:49

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .