How to work with open AI in VS Code like a pro

Home » Python » How to work with open AI in VS Code like a pro

If you have been coding in VS Code for years but have barely touched modern AI assistants, you are in for a big change in your daily workflow. In just a short time, code editors have gone from simple text tools to full-blown AI workstations where you can chat with models, build agents, evaluate prompts and even fine‑tune your own models without leaving the editor. VS Code is at the center of this shift and, crucially, is embracing an open source, community‑driven approach to AI.

Working with open source AI inside VS Code is no longer a niche experiment but a realistic, production‑ready option for JavaScript/React developers, AI engineers and educators alike. Between the decision to open source the GitHub Copilot Chat extension and the arrival of the AI Toolkit for Visual Studio Code, you can mix commercial models (OpenAI, Anthropic, Google, GitHub Models) with local or open models (ONNX, Ollama, Windows‑optimized models) and control everything from discovery to deployment in one place. Let’s break down how this ecosystem works, what it enables and how you can take advantage of it without feeling left behind.

VS Code as an open source AI editor

The VS Code team has publicly committed to turning the editor into an open source AI‑first environment, not a closed black box. For over a decade, VS Code has been one of GitHub’s most successful OSS projects, with a large and active community that chose it precisely because its core is open. As AI becomes a default layer of the coding experience, Microsoft’s goal is to keep the same principles: open development, collaboration and strong community involvement.

A key move in that direction is the decision to release the GitHub Copilot Chat extension as open source under the MIT license. Instead of hiding the AI logic and UX behind proprietary code, the extension’s implementation becomes visible to everyone. The plan is not just to publish the repo and walk away: relevant parts of the extension will be carefully refactored and folded into the VS Code core so that AI capabilities feel like a natural, built‑in layer of the editor.

This integration is meant to reflect a simple reality: AI‑powered tools are now central to how we write code, not a side feature. By treating AI as a first‑class citizen in VS Code’s architecture, the team is reinforcing the idea that an open base plus extensibility produces better tools, richer ecosystems and more resilient workflows than any single closed assistant could deliver.

The Copilot Chat open sourcing is not happening in a vacuum but as part of VS Code’s longstanding OSS history. Over time, community issues and pull requests have helped the team quickly detect and fix bugs, security vulnerabilities and UX pain points. Extending that same open loop to AI features gives contributors a direct path to improve prompts, UI flows, telemetry transparency and more.

From a developer’s perspective, this means AI in VS Code is moving from “mysterious magic” to something you can inspect, debug and even fork if you really need to. You are not locked into a single vendor’s vision: you can choose Copilot, other assistants or your own open source agents, all living on top of a transparent AI‑ready editor.

Why VS Code is opening its AI stack now

The timing of this shift toward open AI in VS Code is driven by concrete changes in the broader AI landscape, not just ideology. Large language models have improved so much that a lot of the earlier “secret prompt sauce” has become less critical. Where previously a vendor could protect an edge by hiding clever prompt engineering, now many interactions converge around similar, well‑understood patterns.

At the UX level, the most effective interaction patterns with AI are now widely shared across editors, IDEs and tools. Things like inline suggestions, chat sidebars, explanation commands and refactoring prompts have become standard. Instead of each vendor reinventing the same widgets in isolation, the VS Code team wants these UI components to live in a stable, open codebase so the community can refine them, extend them and keep them evolving together.

An entire ecosystem of open source AI extensions and tools around VS Code has also emerged. Many of these extensions need to integrate deeply: they want to hook into chat flows, reuse UI components, debug prompt chains or mirror core behaviors. Doing this cleanly is very hard when key parts of the AI experience are sealed off in proprietary extensions with no public source.

Developer trust and transparency play another big role: people want to know what data their AI‑enabled editor is collecting and sending. By opening the Copilot Chat code, users and auditors can inspect exactly which events are logged, what is transmitted to backends, and how privacy decisions are implemented. That visibility goes far beyond a marketing FAQ and gives concrete, reviewable guarantees.

Security is also a strong argument for open AI in VS Code, as AI developer tools are becoming attractive targets for attackers. Malicious extensions, prompt injection paths or data exfiltration vectors can cause serious damage when your editor has deep access to repos and infrastructure (see extension threats and data theft). Historically, community contributions have helped VS Code patch issues quickly; applying that same model to AI features increases the odds of spotting problems early.

All these factors together made it natural for the VS Code team to switch their AI development from a closed to an open model. They are not just betting on open source because “it’s nice”, but because it demonstrably improves not only transparency and trust, but also extensibility, security and pace of innovation for AI inside the editor.

From Copilot Chat extension to AI features in VS Code core

Once the GitHub Copilot Chat extension code is published, the next step is to fold key AI features into the heart of VS Code. This is not a quick copy‑paste: the team will systematically refactor the extension’s components so that core AI primitives live alongside other editor capabilities, exposing well‑defined extension points.

Throughout this refactoring, VS Code’s long‑standing priorities remain unchanged: high performance, powerful extensibility and a clean, intuitive interface. AI features should feel fast, stable and predictable, not like a bolted‑on experiment. Extension authors should still be able to hook into the right layers without fighting hidden assumptions or undocumented APIs.

One of the trickiest aspects of AI features is testing, precisely because language models are stochastic. Outputs vary run to run, prompts evolve and model versions change. To avoid chaos, Microsoft plans to make its own prompt‑testing infrastructure open source as well. That way, community pull requests that touch AI logic can be validated against the same scenarios that internal teams use.

By sharing prompt tests and infrastructure, the project gives contributors a way to iterate on prompts and AI flows without blindly guessing whether they are breaking behavior. It also encourages a culture of treating AI logic as real, testable code, with metrics and regression checks, instead of magic strings sprinkled through the UI.

The ultimate goal is simple: contributing AI features to VS Code should be as straightforward as contributing any other part of the editor. You should not have to reverse‑engineer hidden APIs or guess what the “right” internal behaviors are. Instead, there will be a stable foundation and a shared test suite that let the community help shape AI behavior over time.

AI Toolkit for VS Code: your end‑to‑end AI workspace

On top of the open core, the AI Toolkit for Visual Studio Code acts as an all‑in‑one control center for building, testing and deploying AI‑powered applications. Rather than forcing you to juggle CLI tools, dashboards and scripts, the toolkit gives you a dedicated view in VS Code where you can manage your entire generative AI workflow.

The AI Toolkit integrates seamlessly with a wide range of model providers, both cloud‑based and local. Out of the box, it connects to services such as OpenAI, Anthropic, Google and GitHub Models, while also supporting local models through ONNX and Ollama. This makes it easy to mix open source models with commercial APIs depending on your needs and budget.

From a workflow perspective, the Toolkit covers the full lifecycle: model discovery, prompt experimentation, agent building, evaluation, conversion and deployment. Instead of one extension for autocomplete, another for evaluation and a third for local models, you get a coherent environment that understands how these pieces relate.

AI Toolkit is not limited to AI experts; it’s built for developers, AI engineers, data scientists, researchers and educators. If you are a JS/React developer returning to AI assistants after a few years away, the Toolkit is a very practical way to catch up quickly—everything you need to explore generative models and agents is in a single panel, with clear sections and documentation.

Installation is intentionally straightforward: you fetch the extension from the Visual Studio Marketplace and, once installed, a new AI Toolkit icon appears in the Activity Bar. There is also support for manual installation via VS Code’s standard extension workflow if you prefer to manage VSIX files yourself. After that, the Toolkit opens in its own view, organized into consistent sections.

Model Catalog and Playground: experimenting with open and hosted models

At the heart of AI Toolkit is the Model Catalog, which acts as a unified hub to discover and compare models from multiple sources. Within a single interface, you can browse models from Microsoft Foundry and Foundry Local, GitHub Models, ONNX, Ollama, OpenAI, Anthropic and Google. That means you can weigh open source and hosted models side by side before committing to one.

The catalog is designed to help you find the right fit for your specific use case, rather than blindly picking the biggest model. You can look at capabilities, performance considerations and usage patterns to decide whether you need a heavyweight LLM for reasoning, a smaller local model for privacy and latency, or something optimized for multimodal inputs.

Once you have a candidate model, the Playground provides an interactive chat environment for real‑time experimentation. In the Playground, you can type prompts, tweak parameters and even include multimodal inputs such as images or file attachments. This is ideal when you want to see how a model behaves with realistic developer workflows—reading code, summarizing logs or interpreting screenshots.

The Playground experience is crucial when you are comparing commercial and open models in VS Code. You can, for example, set up equivalent prompts across two providers and observe differences in code quality, latency and adherence to instructions. This kind of hands‑on testing is much more informative than reading static benchmarks.

By bundling catalog and playground in the editor, AI Toolkit shortens the feedback loop: you discover a model, try it immediately against your real tasks and decide if it earns a place in your stack. For open source AI workflows, this makes it dramatically easier to validate models like those running through ONNX or Ollama without setting up separate sandboxes.

Building and inspecting AI agents in VS Code

Beyond simple prompts, AI Toolkit introduces a structured way to build agents—reusable AI components that encapsulate behavior, tools and prompts. The Agent Builder guides you through creating sophisticated prompt structures, connecting tools (including MCP‑style interfaces) and generating production‑ready code with well‑defined outputs (see custom agents in VS Code).

Prompt engineering in this context becomes more than just fiddling with a single string; you define roles, instructions, tool usage and output formats in a repeatable way. This is particularly handy if you are building the same kind of assistant across different projects—say, a code reviewer agent, a documentation explainer or an image metadata analyzer.

Once an agent exists, the Agent Inspector lets you debug and visualize its behavior directly within VS Code. You can inspect intermediate steps, see how tools are called, and iterate on your design without leaving the editor window. This is invaluable when you are chasing subtle issues like hallucinated tool calls or brittle prompts.

For hosted environments, you can deploy agents to Microsoft Foundry right from AI Toolkit. A local agent that you have crafted in VS Code can become a hosted agent in Foundry, making it easier to integrate into larger systems or expose via APIs. A dedicated Hosted Agent Playground then offers a UI to interact with these hosted agents in the same familiar way.

This combination—Agent Builder, Agent Inspector, and hosted agent support—turns VS Code into a practical lab for designing and shipping agents, not just for testing standalone prompts. If you want to build an internal open source assistant tailored to your team’s stack, these tools give you the scaffolding you need.

Running and evaluating prompts at scale

When you are serious about AI in production, you need more than ad‑hoc chat sessions; you need systematic testing and evaluation. AI Toolkit addresses this through features like Bulk Run and Model Evaluation, both accessible from within VS Code.

Bulk Run lets you execute batch prompt tests across multiple models at the same time. You can define a set of test inputs and run them against different providers or configurations, which is ideal for comparing performance on real workloads or assessing how model updates impact your flows. This is especially useful if you want to understand how an open source local model stacks up against a paid API.

For deeper analysis, Model Evaluation provides tools to assess model behavior against datasets and standard metrics. You can measure things like F1 score, relevance, similarity or coherence, and you are free to add custom evaluation criteria tailored to your domain. This helps you move away from “it feels good” evaluations to quantifiable, reproducible metrics.

Evaluation is not limited to single models; you can compare several models or prompt variants using the same dataset. If you are doing prompt engineering for a code‑assistant agent, for example, you can measure how often each variant produces correct patches, high‑quality explanations or minimal hallucinations across a curated test set.

Because all of this runs inside VS Code, it integrates naturally into your development loop. You can edit prompts, rerun evaluations, adjust datasets and commit changes without context‑switching to separate UI dashboards. This is particularly beneficial when iterating on open source models that you are running locally, since you have tight control and fast feedback.

Fine‑tuning, model conversion and local deployment

If you prefer to work with open or custom models instead of exclusively relying on hosted APIs, AI Toolkit includes features tailored to that workflow. Two of the most important are Fine‑tuning and Model Conversion, both aimed at making models behave well in your specific environment.

The Fine‑tuning tool allows you to adapt pre‑trained models to your domain using your own datasets. You can run fine‑tuning jobs locally if you have GPU resources, or offload them to the cloud using Azure Container Apps with GPU acceleration. This is ideal when you want a model that better understands your internal codebase, documentation style or data schemas.

On the deployment side, Model Conversion helps you transform, quantize and optimize models so they run efficiently on local hardware. You can convert models from sources like Hugging Face into formats suited for ONNX or Windows‑accelerated runtimes, and tune them for CPU, GPU or NPU execution. This matters if you want fast, private inference on a laptop or workstation without paying per‑token API fees.

For Windows developers, the Toolkit’s profiling capabilities help diagnose how models use CPU, GPU and NPU resources. You can analyze ONNX models on different execution providers and inspect Windows Machine Learning events, which is essential when optimizing latency, throughput and cost for local AI workloads.

Together, fine‑tuning, conversion and profiling make VS Code a viable environment for serious open source AI work, not just for calling cloud APIs. You get a path from public checkpoints to optimized, domain‑specific models running locally or in your own infrastructure, all orchestrated from the editor you already use.

Managing resources and developer tools in AI Toolkit

The AI Toolkit UI is organized into clear sections—My Resources, Developer Tools, Monitor, and Help and Feedback—so you always know where to find each capability. This structure keeps things manageable even as you juggle multiple models, agents and deployments.

In the My Resources area, you see everything you have access to within the Toolkit. Local Resources lists items stored on your machine, such as local models, tools and agents. Your Foundry Project shows the Microsoft Foundry project linked to the Toolkit, including deployed models, prompt agents, hosted agents, tools, vector stores and classic agents. Connected Resources aggregates external connections, like GitHub Models, that are available to your workspace.

The Developer Tools section is where you find capabilities for discovering, building and deploying AI components. Under Discover, you have the Model Catalog and Tool Catalog, which help you browse and manage both models and tools. Under Build, you get access to features like Create Agent, Agent Inspector, Deploy to Microsoft Foundry, Hosted Agent Playground, Model Playground, Model Conversion and Fine‑tuning.

The Monitor section is focused on understanding how your AI applications behave in the wild. Tracing collects and displays detailed data about how your applications call models and agents, while Evaluation brings the model assessment tools mentioned earlier into one place. Profiling (for Windows ML) lets you dive into low‑level performance numbers across CPU, GPU and NPU.

Finally, Help and Feedback centralizes documentation links, release notes, issue reporting and community channels. From here you can quickly jump to AI Toolkit docs, see what is new in recent versions, file bugs on GitHub or join the community to share experiences and suggestions.

This structured layout turns AI Toolkit into more than just a collection of features; it becomes a coherent workspace for AI development in VS Code. Whether you are discovering new models, building agents, tracking performance or filing feedback, everything is only a couple of clicks away in a familiar environment.

Putting it all together, working with open source AI in VS Code today means much more than turning on an autocomplete plugin. You are getting an editor whose AI stack is moving into the open, a powerful Toolkit that covers the whole lifecycle from model discovery to deployment, and robust support for local and cloud models from multiple providers. For a seasoned JavaScript/React developer returning to AI tools, this ecosystem offers a pragmatic path to experiment, build and ship AI‑augmented features while keeping transparency, flexibility and community at the core of your workflow (see a complete Visual Studio Code tutorial).

Artículo relacionado:

VS Code Evolves: AI Integration, Open-Source Advances, and Fresh Extension Tools