Preprint Open access
FlexiFlow: Bandit-based Model Switching in ML Workflows
Model optimizations help improve inference performance and accuracy of ML workflows. However, relying on a single model to perform inference across all data batches often fails to maximize accuracy and thus overall performance. In many cases, alternate models could perform better on specific subsets of data where a pri …