Optimizing inference proxy for LLMs

Optimizing inference proxy for LLMs

GitHub – algorithmicsuperintelligence/optillm: Optimizing inference proxy for LLMs

“OptiLLM is an OpenAI API-compatible optimizing inference proxy that implements 20+ state-of-the-art techniques to dramatically improve LLM accuracy and performance on reasoning tasks – without requiring any model training or fine-tuning…”

Source: github.com/algorithmicsuperintelligence/optillm