Round Robin is Still Dumb: Load balancing has to grow up for AI inference

Round robin isn’t just “not ideal” for LLM inference. According to recent scheduling research, it’s actively harmful, because it treats inference like stateless, inter...