Revolutionizing API Tool Retrieval with CoHyDE: A Game Changer for LLMs
CoHyDE addresses the challenge of tool retrieval in API catalogs by co-training dense encoders and rewriters. This innovative approach outperforms existing methods, enhancing both standard and vague query handling.
Tool retrieval in extensive API catalogs is no walk in the park for large language models (LLMs). User queries often arrive in plain, everyday language, while the API catalogs speak in technical jargon. Bridging this language gap has been a persistent challenge.
The Problem
Two main training strategies have emerged: contrastive encoder fine-tuning and HyDE-style query expansion. The fine-tuned encoder shines when the query already aligns with the catalog's language. But it falters with more ambiguous queries. On the flip side, zero-shot HyDE handles vague queries better but can stumble when the queries are well-formed, leading to hypothetical descriptions that don't quite fit.
Enter CoHyDE
CoHyDE is here to change the game. It's an iterative procedure that trains both the dense encoder and the LLM rewriter as a single, evolving system. The encoder gets retraining with InfoNCE on hypothetical descriptions generated by the rewriter. Meanwhile, the rewriter aligns its preferences using DPO against the encoder's retrieval scores. Both are prepped on the tool catalog before kicking off the loop.
Impressive Results
On a sample of approximately 10,000 tools from the ToolBench catalog, CoHyDE's three-round training improves performance over previous methods. It boosts NDCG@5 scores by 2.5 percentage points for standard queries and by a significant 6.3 percentage points on vague ones. In the most challenging scenarios, improvements reach as high as 8 percentage points.
The takeaway? Co-training is key. Isolating either component fails to match CoHyDE's performance, especially on vague queries, where losses can hit 8 percentage points.
Why It Matters
Why should developers care? Because CoHyDE isn't just a technical advancement. It's a practical solution that streamlines tool retrieval, increasing efficiency in real-world applications. With the growing complexity of API landscapes, having a reliable system to bridge the language gap is invaluable.
Clone the repo. Run the test. Then form an opinion. CoHyDE might be the breakthrough LLMs need to handle increasingly complex tool catalogs.
Get AI news in your inbox
Daily digest of what matters in AI.
Key Terms Explained
Direct Preference Optimization.
The part of a neural network that processes input data into an internal representation.
The process of taking a pre-trained model and continuing to train it on a smaller, specific dataset to adapt it for a particular task or domain.
Large Language Model.