Explainer
How AI coding assistants work: autocomplete, agents and evidence
AI coding assistants use language models to suggest code as you type, answer questions about a codebase and, increasingly, carry out multi-step tasks such as editing files and running commands.[1][2] 84% of developers in Stack Overflow's 2025 survey were using or planning to use AI tools, though more distrusted their accuracy than trusted it.[3] Studies disagree on productivity: one 2023 experiment found a task done 55.8% faster with Copilot, while a 2025 trial with experienced developers found that allowing AI tools increased their completion time by 19%.[4][5]
AI coding assistants are tools that use language models to help write, explain and change software.[1][6] They range from suggestions in the code editor to agents that plan and carry out multi-step tasks across a project.[7][2]
From suggestions to agents
GitHub describes its Copilot as an AI assistant that suggests code as you type, answers questions about a codebase, reviews changes and works on tasks assigned to it.[1] GitHub separates two ways of working. In assistive use, Copilot suggests or explains code and the developer reviews and applies each suggestion. In agentic use, it researches a repository, proposes a plan, edits files and runs tools, while the developer remains responsible for review and approval.[7]
Anthropic describes Claude Code as an agentic coding tool that reads a codebase, edits files, runs commands and integrates with development tools, and offers it in the terminal, code editors, a desktop app and the browser.[2] How AI agents use tools in general is covered in how AI agents work.
Context makes the difference
GitHub says Copilot gives better responses when it understands a project’s structure, conventions and tooling, which users can supply through instructions, reusable prompts, repository context and external tools.[8] How the underlying models work is explained in how large language models work.
How coding ability is measured
SWE-bench, introduced in 2023, tests whether language models can resolve real GitHub issues by editing the codebase. It contains 2,294 problems from 12 popular Python repositories, and at publication the best model, Claude 2, solved 1.96% of them.[9] Benchmarks like this and their limits are explained in how AI capabilities are measured.
What studies show about productivity
In a 2023 controlled experiment reported by Peng and colleagues, developers with Copilot implemented an HTTP server in JavaScript 55.8% faster than a control group.[4] In a 2025 randomised trial by METR, 16 experienced open-source developers worked on 246 tasks in projects they knew well; allowing AI tools increased completion time by 19%, although the developers estimated afterwards that AI had saved them 20%.[5]
In Stack Overflow’s 2025 Developer Survey, 84% of respondents were using or planning to use AI tools, up from 76% a year earlier, but more distrusted the accuracy of AI tools (46%) than trusted it (33%).[3]
Security and review
The authors of a 2021 study argued that because Copilot had processed large amounts of unvetted code, it would have learned from buggy, exploitable code.[6] Prompting Copilot with 89 security-relevant scenarios, they found about 40% of the 1,689 programs it produced to be vulnerable.[10] GitHub says the developer remains responsible for reviewing and approving agent work.[7] Models can also produce confident but wrong output, as explained in AI hallucinations.
Read next
How AI agents work
What an AI agent is and how it works: language models using tools in a loop, computer use, MCP connectors and long tasks.
Questions readers ask
How does GitHub Copilot work?
GitHub describes Copilot as an AI assistant that suggests code as you type, answers questions about a codebase, reviews changes and works on tasks assigned to it. In assistive use the developer reviews and applies each suggestion; in agentic use Copilot works through multiple steps, such as researching a repository, editing files and running tools.[1][7]
What is an agentic coding tool?
One that carries out multi-step coding work rather than single suggestions. Anthropic describes Claude Code as an agentic coding tool that reads a codebase, edits files, runs commands and integrates with development tools.[2]
Do AI coding assistants make developers faster?
The evidence is mixed. In a 2023 experiment, developers with Copilot implemented an HTTP server 55.8% faster than a control group. In a 2025 randomised trial with 16 experienced open-source developers, allowing AI tools increased completion time by 19%, even though the developers believed AI had made them faster.[4][5]
Is AI-generated code secure?
Not automatically. A 2021 study that prompted Copilot with 89 security-relevant scenarios found about 40% of the 1,689 programs it produced to be vulnerable.[10]
What is SWE-bench?
A benchmark introduced in 2023 that asks language models to resolve real GitHub issues by editing the codebase. It has 2,294 problems from 12 popular Python repositories; at publication the best model solved 1.96% of them.[9]
Do developers trust AI coding tools?
In Stack Overflow's 2025 survey, 84% of respondents were using or planning to use AI tools, but 46% distrusted the accuracy of AI tools against 33% who trusted it.[3]
Sources
· 10 claims from 7 publishers
Each numbered claim is a statement we checked against the sources listed with it. Tap a number in the text to see its sources in place.
- [1]
GitHub describes Copilot as an AI assistant that suggests code as you type, answers questions about a codebase, reviews changes and works on tasks assigned to it. confirmedas of 2026-11-14
- About GitHub Copilot · GitHub Docs · Primary source (retrieved 2026-11-14)
- [2]
Anthropic describes Claude Code as an agentic coding tool that reads a codebase, edits files, runs commands and integrates with development tools, available in the terminal, IDEs, a desktop app and the browser. confirmedas of 2026-11-14
- Claude Code overview · Anthropic · Primary source (retrieved 2026-11-14)
- [3]
In Stack Overflow's 2025 Developer Survey, 84% of respondents were using or planning to use AI tools in their development process, up from 76% a year earlier, but more developers distrusted the accuracy of AI tools (46%) than trusted it (33%). confirmedas of 2025-07-29
- 2025 Stack Overflow Developer Survey - AI · Stack Overflow · Primary source (retrieved 2026-11-14)
- 2025 Stack Overflow Developer Survey - AI · Stack Overflow · Primary source (retrieved 2026-11-14)
- [4]
In a 2023 controlled experiment reported by Peng and colleagues, developers with access to GitHub Copilot completed a task to implement an HTTP server in JavaScript 55.8% faster than a control group. confirmedas of 2023-02-13
- The Impact of AI on Developer Productivity: Evidence from GitHub Copilot · arXiv (Peng et al.) · Primary source · 2023-02-13 (retrieved 2026-11-14)
- The Impact of AI on Developer Productivity: Evidence from GitHub Copilot · arXiv (Peng et al.) · Primary source · 2023-02-13 (retrieved 2026-11-14)
- [5]
In a randomised controlled trial by METR of 16 experienced open-source developers working on 246 tasks in early 2025, allowing AI tools increased completion time by 19%, although the developers estimated afterwards that AI had cut their time by 20%. confirmedas of 2025-07-12
- Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity · arXiv (Becker et al., METR) · Primary source · 2025-07-12 (retrieved 2026-11-14)
- Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity · arXiv (Becker et al., METR) · Primary source · 2025-07-12 (retrieved 2026-11-14)
- Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity · arXiv (Becker et al., METR) · Primary source · 2025-07-12 (retrieved 2026-11-14)
- [6]
A 2021 security study described GitHub Copilot as a language model trained on open-source GitHub code, and its authors argued that because Copilot had processed large amounts of unvetted code, it would have learned from exploitable, buggy code. confirmedas of 2021-08-20
- Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributions · arXiv (Pearce et al., IEEE S&P 2022) · Primary source · 2021-08-20 (retrieved 2026-11-14)
- Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributions · arXiv (Pearce et al., IEEE S&P 2022) · Primary source · 2021-08-20 (retrieved 2026-11-14)
- [7]
GitHub separates assistive use, in which Copilot suggests or explains code and the developer reviews and applies each suggestion, from agentic use, in which Copilot works through multiple steps toward a goal - researching a repository, planning, editing files and running tools - while the developer remains responsible for review and approval. confirmedas of 2026-11-14
- About GitHub Copilot · GitHub Docs · Primary source (retrieved 2026-11-14)
- About GitHub Copilot · GitHub Docs · Primary source (retrieved 2026-11-14)
- [8]
GitHub says Copilot gives better responses when it understands a project's structure, conventions and tooling, which users can supply through instructions, reusable prompts, repository context and external tools. confirmedas of 2026-11-14
- About GitHub Copilot · GitHub Docs · Primary source (retrieved 2026-11-14)
- About GitHub Copilot · GitHub Docs · Primary source (retrieved 2026-11-14)
- About GitHub Copilot · GitHub Docs · Primary source (retrieved 2026-11-14)
- [9]
SWE-bench, introduced in 2023, tests whether language models can resolve 2,294 real GitHub issues from 12 popular Python repositories by editing the codebase; at publication the best model, Claude 2, solved 1.96% of them. confirmedas of 2023-10-10
- SWE-bench: Can Language Models Resolve Real-World GitHub Issues? · arXiv (Jimenez et al., ICLR 2024) · Primary source · 2023-10-10 (retrieved 2026-11-14)
- SWE-bench: Can Language Models Resolve Real-World GitHub Issues? · arXiv (Jimenez et al., ICLR 2024) · Primary source · 2023-10-10 (retrieved 2026-11-14)
- SWE-bench: Can Language Models Resolve Real-World GitHub Issues? · arXiv (Jimenez et al., ICLR 2024) · Primary source · 2023-10-10 · Abstract (2,294 problems, 12 repositories, 1.96% solved; figures typeset as maths in the abstract) (retrieved 2026-11-14)
- [10]
A 2021 study that prompted GitHub Copilot with 89 security-relevant scenarios found about 40% of the 1,689 programs it produced to be vulnerable. confirmedas of 2021-08-20
- Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributions · arXiv (Pearce et al., IEEE S&P 2022) · Primary source · 2021-08-20 (retrieved 2026-11-14)
Revision history (1)
- Page created.
Created Nov 14, 2026. Last reviewed by an editor on Nov 14, 2026. Next scheduled review: Apr 20, 2027.
Cite this page
"How AI coding assistants work: autocomplete, agents and evidence." ContentLora, updated Nov 14, 2026. https://contentlora.com/explain/how-ai-coding-assistants-work
Spotted an error? Suggest a correction or emailcorrections@contentlora.com.