At its core, Passenger is a process manager and HTTP request router. In order to minimize response times, and to distribute load over multiple CPU cores for optimal performance, Passenger load balances requests over processes in an intelligent manner. This article explains the implications and details of our request load balancing mechanism.

With Passenger this is improved by running multiple processes of the application using pooled application groups. For each request from the incoming request queue, a non-completely-busy (free) application process is selected to handle the request.

For thread-safe Ruby apps it is also possible to enable [multithreading](<%= url_for "/references/config_reference/nginx/#passenger_concurrency_model" %>), which allows the application processes to concurrently handle multiple requests at the same time -- up to the amount of threads configured. In this case, Passenger forwards the request to the instance that meets the criteria described [below](#intelligent-routing).

If all application processes and threads are completely busy, Passenger spawns a new instance, up to the [process limit](<%= url_for "/references/config_reference/nginx/#passenger_concurrency_model" %>) for the group. The amount of processes of the groups combined may also not exceed the limit for the application pool. If either limit is reached, the request remains in the queue (which has its own [limit](#request-queue-overflow)).

### App Generations

Another core concept in the load balancing algorithm is that of the **app generations**. Every time Passenger is asked to [restart an app group](<%= url_for "/advanced_guides/troubleshooting/nginx/restart_app.html" %>), a counter is incremented. Processes created after the restart are labeled with a generation which is taken from this counter. Passenger usually performs a blocking restart, which means that there is only one generation of processes alive at a time, however if you initiate a [rolling restart](<%= url_for "/advanced_guides/deployment_and_scaling/nginx/zero_downtime_redeployments/ruby/index.html" %>), multiple generations can be alive at the same time, and Passenger will take this into account when load balancing requests.

## Intelligent routing

Passenger keeps a list of application processes. For each application process, Passenger keeps track of: the app generation that resulted in it being started, when it was started, and how many requests it is currently handling. When a new request comes in, Passenger routes the request to the process that meets the following criteria:

<li>is not completely busy,</li>

<li>is part of the newest available app generation¹ meeting the previous condition,</li>

<li>is the oldest process meeting the previous conditions,</li>

<li>is the least busy process meeting the previous conditions.</li>

[1] Unless blocked by <a href="<%= url_for "/advanced_guides/deployment_and_scaling/standalone/deployment_error_resistance.html" %>">Deployment Error Resistance</a>.

### Process with the highest app generation in the list has highest priority

If there are multiple processes that are not completely busy, then Passenger will pick one from the list with the highest app generation:

For example, suppose that there are 3 application processes:

Process A: gen 1, spawned 1s ago, handling 0 requests

Process B: gen 1, spawned 1s ago, handling 0 requests

Process C: gen 2, spawned 1s ago, handling 0 requests

Process C will be chosen for the next incoming request.

This speeds the rate at which requests are handled by the newer generation of app processes.

### Oldest available process in a generation has highest priority

If there are multiple processes from the newest generation, that not completely busy; then Passenger will pick the one that was started first, from the list.

For example, suppose that there are 3 application processes:

Process A: gen 2, spawned 30s ago, handling 0 requests

Process B: gen 3, spawned 20s ago, handling 0 requests

Process C: gen 3, spawned 10s ago, handling 0 requests

Process B will be chosen for the next incoming request.

This property is used by the [dynamic process scaling](<%= url_for "/advanced_guides/in_depth/ruby/dynamic_scaling_of_app_processes/index.html" %>) algorithm. Dynamic process scaling works by shutting down processes that haven't received requests for a while (processes that are "idle"). By routing to the oldest process that is not completely busy (instead of, say, a random one, using round-robin), Passenger gives other processes the chance to become idle and thus eligible for shutdown.

Another advantage of picking the oldest process is that it improves caching. Since the oldest process is the most likely candidate for load balancing, it will have the most chance to keep its cache warm, and therefore process requests more quickly.

### Least busy process as tie breaker

If there are multiple processes that have met all the previous conditions, then Passenger will pick the least busy one. For example, suppose that there are 3 application processes:

Process A: gen 3, spawned 10s ago, handling 1 requests

Process B: gen 3, spawned 10s ago, handling 0 requests

Process C: gen 3, spawned 10s ago, handling 2 requests

Process B will be chosen for the next incoming request.

Because Passenger [prefers to load balance to the oldest request in the generation](#algorithm_ordered), traffic may appear unbalanced between processes. Here is an example from `passenger-status`:

Suppose that you have 3 application processes, spawned in order, and each process's maximum concurrency is 1. When the application is idle, none of the processes are handling any requests:

Suppose that, while α is still in progress, a new request comes in (which we call β). That request will be load balanced to process B because it is the oldest not-completely-busy process:

This keeps process A's caches warm, and may allow process C to idle and be shutdown sooner.

Suppose that, while α is still in progress, 1 more request comes in (which we call β). That request will be load balanced to process A because it is the oldest not-completely-busy one:

Process A [αβ ]

Process B [ ]

Process A [αβɣ ]

Process B [ ]