نسخة أولية وصول مفتوح
InferOpt: Constrained Multi-Objective Search for LLM Inference Configurations
Serving an LLM means setting dozens of inference-time knobs, from per-layer KV retention to per-layer expert counts. Practice sets them with mechanism-specific heuristics that return a single operating point and do not scale to layer-wise search spaces. We recast inference configuration as constrained multi-objective b …