There’s admin cost, but it’s incremental and in the low percentage rate (per machine).

Optimizing 100% of your code won’t get you 100% more performance, maybe 20%. It will take several man month, and that’s not counting the fact that optimized (for CPUs, not developers or users) code is harder to maintain.

Should you optimize? For the small efforts that give you a lot of gain – certainly! I put indexes on my tables, choose good sorting algorithms, and pick up all the low hanging fruits.

But if you prefer an environment that’s optimized 100% of the time, for the 20% performance gain you get, you’re wasting a lot of resources.

If you follow what the costs are in the industry,
additions of machines imply additions of sysadmins to care and feed them.

You just traded the cost of development – perhaps
limited – for a guaranteed cost increase, both in hardware costs and in staffing to care for those extra machines.

—

The extra cost of development (optimize for the – vertical – scaling) may allow to reduce the ’slope’ of increase
for your horizontal scaling, leading to lower
costs in incremental hardware and associated sysadmin
staff.

a) You should learn what scalability is.
b) You should learn what a process is.
c) You should learn what a thread is.

After that go write blog entries.

All the best,
Tom

But for dedicated applications, this is less of an issue as they nominally don’t run potentially unsafe code (that’s vetted out in the development and QA process).

But the expenses of that security tends to outweigh the performance benefits of a threaded environment.

If you take the simplest case of a multithreaded server servicing stateless requests without the use of any shared resources, then the overhead is the cost of the OS context switching the threads vs switching the processes, and threads are undoubtedly much cheaper to switch than processes today.

Once you start sharing resources, then the threaded server does even better.

Process servers can’t share resources easily. One of the beauties of the process is the impenetrable wall that the OS throws up around it, giving us lots of security and privacy to do what we want, and how we want. At the naive level, a process server is most likely going to have to serialize any resource it doesn’t natively contain. It will load this data either from a DB or a filesystem, typically. It has hoist that data over that process “firewall” that the OS has built around the code.

If you do this once at startup, then all of the processes have a copy of the data and its simply a startup cost of the process. But now you’re “wasting space” because every process is duplicating data. If you do this to service each request, then you must pay the serialization costs.

You can share resources with shared memory, but as soon as you do that you get all of the coordination issues that are associated with threads. Not a problem with read only data, but it is for modifiable data. So you pay the synchronization prices that shared resourcse invoke in threaded environements, but without the benefits of a threaded environment.

Of course, with processes, the interprocess communication components are all going to be kernel space routines, not user space, and therefore more expensive. The kernel is also surrounded by a firewall of sorts, so anytime you need to talk to the kernel, data needs to be thrown back and forth across that boundary.

With threaded environments, most of these problems are mitigated. Sharing data is implicit in the design, you only need a single copy of the data, and all of your IPC is done in the user space, rather than the kernel. If you have an application sharing read only data among threads, your application isn’t written any differently than a process application, but you still all the lightweight benefits of the threaded app.

The biggest risk in a threaded environment is robustness. A single thread can destroy the entire process, including other “unrelated” clients. Processes are much safer than that.

But all that safety comes at a cost in performance for the application.

A well written threaded application will perform better than a well written process based application if for no other reason than simply demanding less of the encompossing operating system services and resources.

Thus a threaded application will provide better system utilization than a process based application.

While a process based developer doesn’t have to “worry about” resource sharing, the operating system DOES have to worry about that. It’s not like the problems go away, they’re simply delegated to the OS to manage them. So, you have to pay the same price as threaded applications (even if you’re paying that price unknowingly) but you don’t get the advantage of threaded environments in terms of performance and resource sharing.

Finally, regarding the containers and such, those are not required to write threaded applications. You can write a threaded application that doesn’t require any of the packaging or deployment issues et al.

The benefit of the containers is simply that we don’t have to reinvent that code when we can create a simple bundle for our application and delegate that low level plumbing to a tested and proven container.

So, feel free to assert that processes can scale, but threaded applications scale BETTER, provide better system utilization, and will reduce deployment infrastructure costs and long term operation costs.

In my experience, the only two metrics that matter are: how much will it cost you to serve (bandwidth, CPUs, etc)? and how much it will cost you to develop?

Generally, optimizing for processes is cheaper because hardware is cheaper than development. $3000 gets you a very nice server, but not even 2 man weeks of development.

I have not observed that multiple processes run faster, I observed that you get a lot more features in a lot shorter time and it scales very well. And for most Web applications, that’s the better tradeoff.

Try running Tomcat on Linux both with and without NPTL with a huge number of threads some time. You’ll see that things run a LOT better with threads instead of processes. Maybe you have observed multiple processes being faster, but were they using non-blocking I/O? Not a good comparison if so, which is why I suggest really comparing threads to processes 1-on-1.

This post is about scalability, as in throwing more boxes at the problem.