AI Engineering Coach

★ 3.5kAgentic Engineering

#microsoft#coach#best-practice

Overview

Microsoft's open-source (community project, not an official product) AI coding 'coach': a VS Code extension that reads your local AI coding session logs and turns them into actionable improvement advice — its slogan is 'better agentic engineering'. It is harness-agnostic, analyzing logs from Claude Code, Copilot and more, with all parsing and analytics running locally, read-only, no telemetry. It breaks 'using AI engineering-grade' into an observe → measure → improve → level-up loop.

Who it's for

Developers actively using AI coding assistants who want to know whether their usage is truly 'engineering-grade' and are willing to improve habits based on evidence.

What you'll learn

01What it is: turning session logs into improvement advice
AI Engineer Coach reads your local AI coding session logs and produces actionable insights: track practice scores, weekly trends and daily activity; detect anti-patterns with 45 rules; measure AI-generated code volume by language, workspace, model and harness; discover repeated prompts and turn them into reusable skills; and score context health (agentic readiness checks, instruction-file audits, workspace context maps). The privacy promise is central: read-only on your session files, fully local analysis, no proprietary telemetry.
02Breaking 'engineering-grade AI coding' into a four-stage loop
Observe: Dashboard / Timeline / Coding Moments for practice scores, week-over-week trends and a Gantt-style per-day session timeline; Measure: Output / Burndown / Patterns for generated-code volume, token budget and a 7×24 activity heatmap; Improve: Anti-Patterns / Context Health that point out anti-patterns with concrete actions; Level Up: Learning Center / Achievements / Agentic SDLC that generate personalized quizzes and code-comparison rounds from your actual usage, plus an XP progression system.
03Anti-pattern detection: 45 editable rules
The Anti-Patterns page uses 45 markdown rules across five dimensions — prompt quality, session hygiene, code review, tool mastery and context management — each with severity ratings, concrete actions and example prompts. The Rule Editor lets you create, edit and tune rules visually or as raw markdown and live-test them against your own data; the Rule Playground is an interactive REPL for the rule DSL with a field browser, function catalog and metric list.
04Skill discovery and context health
Skill Finder detects patterns in your repeated prompts and matches community skills from the open-source catalog, turning habitual operations into reusable skills; Context Health delivers an overall context score, an agentic-readiness checklist, a workspace context map and AI-powered instruction-file review. These are the core levers of agentic engineering: context quality and reusable capability.
05How to install and use
The extension is not published to a marketplace or Releases page, so you build the .vsix yourself: clone, run npm ci && npm run package, then install via code --install-extension (or 'Install from VSIX' in the VS Code UI). After install, run 'AI Engineer Coach: Open Dashboard' from the command palette and filter by workspace or harness in the sidebar. The same dashboard also runs as a canvas inside the GitHub Copilot app, no VS Code needed.

Practice tips

Use Claude Code / Copilot normally for a week to accumulate session logs, then open the Dashboard to see your practice-score baseline before changing anything.
Open Anti-Patterns, pick the 2-3 rules with the highest hit rate, follow the 'concrete actions' to adjust your prompting habits, and compare again next week.
In Rule Playground, tune one detection rule with the rule DSL and live-test it against your real data to learn how rules are written.
Use Skill Finder to turn your high-frequency repeated prompts into reusable skills and track efficiency before vs. after.