How to Handle Multi-Tenant Performance Issues or "Noisy Neighbor Problem"?
Understanding How to Avoid Performance Degradation Caused by Noisy Neighbors in Multi-Tenant Systems
What is the Noisy Neighbor Problem?
Have you ever experienced a scenario where turning on a heavy electronic device at home resulted in shutting down the entire electric connection of your home? A similar situation happens when multiple services (multi-tenant systems) run together and share the underlying resources such as network bandwidth, memory, storage, etc., and one of the services consumes most of the resources, leaving the other services with insufficient resources. This leads to a situation where the services with depleted resources crash or run with degraded performance. This scenario is often termed the Noisy Neighbor Problem.
How to Handle the Situation?
Scaling the Resources
The most straightforward approach is to pour more resources into the system. However, this may not be financially feasible, as the added resources may not be fully utilized when the service is experiencing lower-than-usual traffic.
Service Isolation
Moving each service into its isolated environment (or using containers like docker) with an upper limit restriction on the resources each service can use can help avoid the noisy neighbor problem. In this scenario, services other than noisy service will not be impacted as their share of resources will be restricted from being used by the noisy neighbors.
Service Throttling
Noisy neighbors can also implement service throttling measures, such as rate-limiting incoming requests or clients using exponential jitter and retry, to avoid getting overwhelmed and keep resource utilization under the expected limit.
Proactive Monitoring
Proactively monitoring spikes in a service's resource utilization or incoming traffic can help trigger alerts or events. This approach can be used with the “Scalaing the Resources” approach, as triggers or events generated from monitoring can be used to scale up and down the service, making it more economical financially.
