Canonical: "Help me size my wholly unspecified application" question

Question

We periodically get the "Help me size my wholly unspecified application" questions, like What capacities should have a hosting to support an average of 1000 registered users simultaneously? or Server requirement for running an application having huge network traffic.

Have we already worked up a canonical answer for these types of questions?

To a comment I made on the first question cited above, @squillman said 'Yeah, the canonical duplicate is "Not a real question"', but I'd argue that there's probably some value in a well thought out answer describing the need for a scientific process of capacity planning.

To my mind, anyone who would even think to ask such a question is clearly in need of some basic guidance (and, hopefully, can be made to understand that there is science to capacity planning and that "number of users" or some other catch-all value isn't the only data point used in planning). Closing these questions as "not a real question" doesn't point the OP in the right direction, and they end up still living in their happy fantasy world of cut-and-dried capacity planning via a single data point.

Answer 1

I did some writing about two weeks back that included me rambling on that... we can probably pick or even draft up a question on which I can talk about start small, measure, and scale well.

There is no shortage of folks running around the world trying to create the next Facebook. They crop up regularly on Serverfault asking what it takes to handle 10,000 connections per second for their next big idea website. I don’t have any good examples to reference because they’re bad questions; bad questions are closed and deleted. What those questions would show is a common problem in the world of system administration: the basic lesson unlearned.

...

Always Measure

There are an ample supply of questions on Serverfault consisting of, “Is this faster or why is that slower, etc.” The answer to those questions in most cases is, “we don’t know you hardware, measure it and find out.” A computer consists of multiple finite resources and in most cases bottlenecks on one of those resources. Most of the time that resource limitation is available memory, processor speed, or disk speed. There are others besides the big three, but the answer is always to measure, locate, and alleviate in a cost-effective way.

Thus, when it comes to adding services don’t tell me SSL is too slow, but rather tell me what it limits you to for speed. A good example I came across this past week was figuring out the cost of / most cost effective way to provide SSL on EC2. I was impressed that the asker had done a lot of homework before and took a look at comparable benchmarks to determine that things “didn’t feel right.” I find many questions where this task isn’t completed. So, whenever you start, consider if $whatever is worth the cost, and do it with a “de minimis” in mind — don’t calculate the small stuff, but if a few thousand in hardware or time is on the line, think it out. Sometimes cost isn’t the only factor, but be aware of it.

Finally, do the default to start. What works 90% of the time for 90% of of folks will probably work for you, or at least be a good starting point to compare against. Optimizing for a target before you’ve been able to measure it is, thanks to the laws of thermodynamics, comparable in efficiency to lighting money on fire. Both will generate heat, but I wouldn’t count on much more besides some ash and a waste of resources.

Answer 2

The closest one we have right now is probably this one:

How do you do load testing and capacity planning for web sites?

And another for databases.

Answer 3

This is what I was trying to address with my Meta question Can we weed out the benchmarking questions?

What I got from the answers was that we could do away from these if there were Canonical Questions, so I wrote How do you do Load Testing and Capacity Planning for Web Sites and How do you do Load Testing and Capacity Planning for Databases

I also believe that there was a section of the FAQ added in order to address these broadly-answerable questions. (I can't find it now. Perhaps it was part of a discussion...)