GitHub’s engineering team discovered that giving Copilot too many tools within VS Code hindered its performance and reliability. A common frustration for myself and most developers. Their latest update proves that a streamlined toolset and semantic embeddings lead to faster, more effective code assistance.
While Copilot Chat could access hundreds of tools through the Model Context Protocol (MCP), this abundance often overwhelmed the AI. Too many choices meant slower responses, increased API failures, and a greater risk of mistakes. The team recognized that filtering and prioritizing tools was key to maintaining the agent’s flexibility without sacrificing speed or user experience.
Virtual Tools and Adaptive Clustering: Organizing the Chaos
To solve the overload, GitHub introduced virtual tools. Similar utilities are grouped into expandable categories, so Copilot only sees what’s needed at the right moment. They also leveraged adaptive clustering driven by a custom embedding model. Tools are grouped by semantic similarity, enabling stable clusters that don’t rely on the LLM to start from scratch every time.
- Each tool receives an embedding, and cosine similarity measures help organize them into logical groups.
- Clusters are summarized by the model and cached, providing quick access in future sessions.
NOTE: Semantic Tool Specification was recently added to vLLM, read more here.
Embedding-Guided Tool Routing: Smarter and Faster
Clustering alone wasn’t enough. The team developed Embedding-Guided Tool Routing, which compares the embedding of a user’s query with all tool cluster embeddings before expanding any group. This allows Copilot to pre-select the most relevant tools, minimizing unnecessary actions and slashing response times.
- This method raises the odds that Copilot surfaces the correct tool right when it’s needed, increasing success and reliability.
- Benchmarks reveal 94.5% tool use coverage with embeddings, compared to 87.5% for LLM-based selection and just 69.0% for a static list.
Distilling Copilot’s Arsenal to 13 Core Tools
By analyzing usage and outcomes, the team identified a core set of 13 essential tools covering repository navigation, file editing, search, and terminal functionality. Other tools are tucked into four virtual categories (Jupyter, Web Tools, Workspace, Testing) and are available only as needed. This change delivers:
- Up to 400ms faster response times
- More accurate and context-aware tool selection
- Fewer errors and simpler reasoning for Copilot
Future Directions: Long-Context Reasoning
The benefits of streamlined tool selection are just the start. The next challenge for Copilot is long-context reasoning: tracking prior actions, understanding user intent over time, and planning multifaceted tasks. Embedding and context-aware routing will be the foundation for transforming Copilot into a proactive development partner.
Takeaway
GitHub Copilot’s evolution shows that thoughtful reduction and intelligent organization can make AI smarter and more effective. By focusing on fewer, better-selected tools, Copilot is already delivering faster, more reliable assistance and the future holds even greater potential for AI-driven productivity in software development.
Source: The GitHub Blog
How GitHub Copilot Became Smarter by Using Fewer Tools