用于Coding Agent 编程智能体的可维护性检测传感器
目录
Maintainability sensors for coding agents
用于Coding Agent 编程智能体的可维护性检测传感器
Overview of all sensors used所有所使用传感器的概览
Base harnesses and models基础型背带和款式
Static code analysis: Basic linting静态代码分析:基本的代码检查/错误检测
Rules for typical AI shortcomings典型人工智能缺陷的处理规则
Guidance for self-correction自我纠正的指导方针/自我修正的指导建议
Managing warnings - now more feasible?管理警告——现在更可行了吗?
Static code analysis: Dependency rules静态代码分析:依赖关系规则
Static code analysis: Coupling data静态代码分析:数据耦合问题
For AI consumption 用于人工智能相关应用/人工智能领域的使用
Gather evidence (run the CLI)收集证据(运行命令行界面)
Write the markdown report编写 Markdown 格式的报告
Static code analysis: AI modularity review静态代码分析:人工智能辅助的模块化审查
The test suite as a regression sensor测试套件作为回归检测工具的用途
Conclusions and open questions结论与未解决的问题
Appendix: About tooling 附录:关于工具的相关信息
Integration with the coding harness与编码工具的集成
Maintainability sensors for coding agents
用于Coding Agent 编程智能体的可维护性检测传感器
In a recent article about harness engineering for coding agent users, I laid out a mental model for expanding a coding agent harness: a system of guides and sensors that increase the probability of good agent outputs and enable self-correction before issues reach human eyes. This article is a more practical follow-up where I walk through my experience with using sensors that help keep the codebase maintainable.
在最近一篇关于如何为编程智能体用户提供有效辅助工具的文章中,我提出了一种扩展编程智能体辅助系统的思路:即构建一套由各种指导和监控机制构成的系统,从而提高智能体产生正确结果的概率,并能在问题出现之前自动进行修正。本文则是对该理念的进一步实践探讨,我分享了在使用各种监控机制来保持代码库可维护性方面的经验。
27 May 2026 2026 年 5 月 27 日
Birgitta is a Distinguished Engineer and AI-assisted delivery expert at Thoughtworks. She has over 20 years of experience as a software developer, architect and technical leader.
比吉塔是 Thoughtworks 公司的资深工程师,同时也是人工智能辅助开发领域的专家。她拥有超过 20 年的软件开发、架构设计及技术管理经验。
Contents 目录
- The application 该应用程序/该应用软件
- Static code analysis: Basic linting静态代码分析:基本的代码检查/错误检测
- Static code analysis: Dependency rules静态代码分析:依赖关系规则
- Static code analysis: Coupling data静态代码分析:数据耦合问题
- Static code analysis: AI modularity review静态代码分析:人工智能辅助的模块化审查
- The test suite as a regression sensor测试套件作为回归检测工具的用途
- Conclusions and open questions结论与未解之谜
There are multiple dimensions we usually want to achieve and monitor in our codebases: Functional correctness (works as intended), architectural fitness (is fast/secure/usable enough), and maintainability. I define maintainability here as making it easy and low risk to change the codebase over time - also known as “internal quality”. So I don't only want to be able to make changes quickly today, but also in the future. And I don't want to worry about introducing bugs or degradation of fitness every time I make a change - or have AI make a change. I usually see the first signs of cracks in the maintainability of an AI-generated codebase when the number of files changed for a small adjustment increases. Or when changes start breaking things that used to work.
在代码库中,我们通常需要实现并关注多个方面的指标:功能正确性(代码能按预期正常运行)、架构的合理性(代码是否足够快速、安全且易于使用),以及可维护性。在这里,我把可维护性定义为:能够以低风险、高效的方式对代码进行修改——这也被称为“内在质量”。因此,我不仅希望现在能够轻松地对代码进行修改,也希望将来也能如此。同时,我不想因为每次修改代码而引入错误或导致代码性能下降——当然,也不希望让人工智能在修改代码时带来类似的问题。通常,当为了进行微小调整而需要修改的文件数量增加时,或者当修改后的代码开始破坏之前正常运行的功能时,我就意识到人工智能生成的代码在可维护性方面出现了问题。
As always, my perspective is mainly focussed on writing code for applications like digital products or enterprise software. Risk profiles and goals for using AI coding agents vary depending on the software that is being built.
和往常一样,我的主要关注点还是为数字产品或企业软件等应用编写代码。在使用人工智能编程工具时,所面临的风险和目标因所开发的软件类型而异。
Internal quality problems affect AI agents in similar ways that they affect human developers. An agent working in a tangled codebase might look in the wrong place for an existing implementation, create inconsistencies because it has not noticed a duplicate, or be forced to load more context than a task should require.
内部质量方面的问题对人工智能智能体的影响,与对人类开发人员的影响类似。在结构混乱的代码库中工作的智能体,可能会在错误的位置寻找已有的实现代码;由于没有发现代码重复,从而导致逻辑上的不一致;或者不得不加载超出任务所需范围的额外信息。
In this article, I describe my experimentation with various sensors that help us and AI reflect on the maintainability of a codebase, and what I learned from that.
在这篇文章中,我介绍了自己使用各种传感器进行实验的过程。这些传感器有助于我们和人工智能更好地评估代码库的可维护性。同时,我也分享了从这些实验中得到的经验教训。
The application 该应用程序/该应用软件
I'm working on an internal analytics dashboard for community managers that reads chat space activity, engagement, and demographic data from a combination of APIs and presents the data in a web frontend.
我正在为社区管理员开发一个内部分析仪表板。该仪表板通过多种 API 获取聊天记录、用户参与度以及用户人口统计数据,然后以网页形式呈现这些数据。

Figure 1: The example app: web UI, service layer, and external APIs.
图 1:该示例应用程序的架构:网页用户界面、服务层以及外部 API。
The tech stack is a TypeScript, NextJS, and React. The backend reads and joins data from the APIs. The application has been around for a while, but for the sake of these experiments I rebuilt it with AI from scratch.
该应用程序所使用的技术栈包括 TypeScript、NextJS 和 React。后端部分负责从各种 API 中读取和整合数据。虽然这个应用程序已经存在了一段时间,但为了进行这些测试,我决定用 AI 从头开始重新构建它。
There are hardly any guides (e.g. markdown files) for AI about code quality and maintainability present, I wanted to see how well it can do just by relying on sensor feedback.
目前,几乎没有任何关于代码质量和可维护性的 AI 相关指南或模板可供参考。我想看看,仅依靠传感器反馈,AI 在处理这些问题时能表现得如何。
Overview of all sensors used
所有所使用传感器的概览

Figure 2: Where sensors can run: during the initial coding session, in the pipeline, on a schedule, and in production.
图 2:传感器可以运行的场景:在初始编码阶段、在处理流程中、按照预定时间表执行任务,以及在实际生产环境中使用。
This is an overview of the sensors I set up across the path to production.
以下是我在通往生产流程的各个环节中所安装的传感器的概览。
You can also find a video conversation about this application and most of these sensors on the Thoughtworks YouTube channel.
您还可以在 Thoughtworks 的 YouTube 频道上找到关于该应用程序以及这些传感器的视频介绍。
During coding session 在编程过程中
Sensors that run continuously alongside the agent to provide fast feedback.
这些传感器会持续与代理程序一起运行,从而能够提供快速的反馈。
- Type checker (computational)
类型检查器(计算领域) - ESLint (computational) ESLint(代码检查工具)
- Semgrep, SAST tool prescribed by our internal AppSec team (computational)
Semgrep 是一种 SAST 工具,是我们内部应用安全团队所推荐的工具。 - dependency-cruiser, runs structural rules to check internal module dependencies (computational)
Dependency-Cruiser 会运行各种结构规则,以检查各个模块之间的依赖关系(属于计算相关功能)。 - Test suite results including test coverage (computational - though the test suite is generated by AI, therefore created in an inferential way)
测试套件的结果,包括测试覆盖率(这些数据是通过计算得出的——不过,该测试套件是由人工智能生成的,因此其生成方式属于推理性质)。 - Incremental mutation testing (computational)
增量突变测试(计算方法) - GitLeaks runs as part of the pre-commit hook, I consider it to be a sensor as well, as it will give the agent feedback when it tries to commit (computational)
GitLeaks 作为预提交钩子的一部分来运行。我觉得它也可以被视作一种“传感器”:当代理程序试图进行提交操作时,GitLeaks 会向该代理程序反馈相关信息。
After integration - pipeline
集成完成后——进入管道处理阶段
The same computational sensors run again in CI. The in-session sensors give the agent early feedback during development. The CI pipeline confirms the result on clean infrastructure and after integration.
在持续集成过程中,同样会使用这些计算型传感器。这些传感器能在开发过程中及时向开发人员提供反馈。持续集成流程则会在环境稳定且所有组件都集成完成后,再次验证测试结果。
Repeatedly 反复地/一再地
Sensors that run on a slower cadence to detect drift that accumulates over time, rather than errors that occur in the moment.
这些传感器以较慢的频率进行检测,从而能够发现那些随着时间推移而逐渐积累的偏差,而不是那些瞬间出现的错误。
- A security review, prompt derived from our AppSec checklist for internal applications (inferential)
这是一次安全审查,其依据是我们针对内部应用程序制定的 AppSec 检查清单。 - A data handling review, prompt describes things like “no user names should ever be sent to the web frontend” (inferential)
在数据处理相关的要求中,有这样的规定:“绝不应将任何用户名发送到网页前端”(推论性要求) - Dependency freshness report, which runs a script first to get the age and activity of the library dependencies, and then has AI create a report with recommendations about potential upgrades, deprecations, etc (computational and inferential)
依赖项更新情况报告:该报告首先通过脚本来获取各个库依赖项的“年龄”和“活跃度”信息,然后利用人工智能技术生成报告,其中包含关于哪些依赖项需要升级、哪些应该被淘汰等方面的建议。 - Modularity and coupling review (computational and inferential)
模块性与耦合性分析(计算与推理方面)
With this context out of the way, let's dive into the first category of sensors.
在明确了上述背景之后,让我们来详细了解一下第一类传感器吧。
Base harnesses and models
基础型背带和款式
Throughout building the application, I used a mix of Cursor, Claude Code, and OpenCode (in that order of frequency). My default model was usually Claude Sonnet, for some of the planning and analysis tasks I used Claude Opus, and for implementation tasks I frequently used Cursor's composer-2 model.
在开发这个应用程序的过程中,我依次使用了 Cursor、Claude Code 和 OpenCode 这三种工具。通常情况下,我会使用 Claude Sonnet 作为默认模型;在需要进行某些规划或分析工作时,我会使用 Claude Opus;而在实际开发过程中,我则经常使用 Cursor 的 composer-2 模型。
Static code analysis: Basic linting
静态代码分析:基本的代码检查/错误检测
I'll start with my learnings from using ESLint in this application. Basic linting tools like ESLint mostly target maintainability risk at the level of individual files and functions.
我先来分享一下在使用 ESLint 进行代码检查时所获得的经验。像 ESLint 这样的基本代码检查工具,主要着眼于解决单个文件和函数层面的可维护性问题。
Rules for typical AI shortcomings
典型人工智能缺陷的处理规则
In my experience, the AI failure modes that are the most low-hanging fruit for static code analysis are
以我的经验来看,对于静态代码分析来说,最容易发现的 AI 分析错误类型就是那些显而易见的错误。
- Max number of arguments for functions
函数的参数最大数量 - File length 文件长度
- Function length 函数长度
- Cyclomatic complexity 圈复杂度
However, these weren't even active in ESLint's default preset, I had to configure maximums for them first. Hopefully, static analysis tools will evolve to provide better presets for usage with AI. A bit of research shows that people are also starting to publish ESLint plugins with rule sets that are specifically targeting known agent failure modes, like this one by Factory, with rules about things like requiring test files or structured logging.
不过,这些规则在 ESLint 的默认预设中甚至都没有被启用。我不得不先手动设置这些规则的最高限制值。希望静态分析工具能够不断发展,从而为与 AI 的结合使用提供更完善的预设。一些研究表明,人们也开始发布针对特定错误模式的 ESLint 插件。比如 Factory 开发的插件,其中包含了一些规则,比如要求必须要有测试文件或结构化的日志记录。
Guidance for self-correction
自我纠正的指导方针/自我修正的指导建议
A sensor is meant to give the agent feedback so that it can self-correct. Ideally, we want to give the agent extra context for that self-correction - a good kind of prompt injection. To do that, I built a custom ESLint formatter to override some of the default messages - with the help of AI of course, naturally.
传感器的作用就是向智能体提供反馈,使其能够自我修正。理想情况下,我们希望为智能体的自我修正提供更多的背景信息——这其实是一种有效的“提示”方式。为此,我开发了一个自定义的 ESLint 格式化工具,用来覆盖部分默认的提示信息。当然,这一过程当然也离不开人工智能的辅助。
Here is an example of my guidance for the no-explicit-any warning.
以下是我对 no-explicit-any 警告的提示示例。
We want things to be typed to make it easier to avoid errors, especially for key concepts.
But we also want to avoid cluttering our codebase with unnecessary types. Make a judgment
call about this. If you choose to not introduce a type, suppress it with:
// eslint-disable-next-line @typescript-eslint/no-explicit-any -- (give reason why)`,
Managing warnings - now more feasible?
管理警告——现在更可行了吗?
Static code analysis has been around for a long time, and yet, teams often didn't use it consistently, even when they had it set up. One of the reasons for that is the management overhead that comes with it. Effective use of this analysis requires a team to keep a “clean house”, otherwise the metrics just become noise. In particular warnings like the no-explicit-any example above are tricky, because you don't always want to fix them - it depends. And suppressing them one by one has always felt tedious, and like noise in the code.
静态代码分析技术早已存在,但即便相关工具已经配置好,各团队也往往不会持续使用它。其中一个原因就是这种分析方式会带来一定的管理成本。要有效利用这一分析工具,团队必须保持代码的整洁有序,否则各种分析结果就只会成为无用的数据而已。尤其是像上面 no-explicit-any 所示的警告信息,处理起来相当麻烦——因为并不总是需要去修复它们,具体情况需视实际情况而定。而逐一屏蔽这些警告信息则显得十分繁琐,而且这些警告信息本身也相当于代码中的“噪音”。
With coding agents, we might now have a chance at that clean baseline. In the guidance text above, the agent is told to make a judgment call, and allowed to suppress a warning in the code. This keeps the suppressions manageable, visible and reviewable.
借助编程智能体,我们现在或许有机会实现那个“干净的基准状态”。在上述指导说明中,智能体被要求自行做出判断,并可以被允许在代码中忽略某些警告信息。这样一来,这些被忽略的警告信息仍然可以被妥善管理、清晰可见,同时也便于后续审查。
For thresholds, like the maximum number of lines, or the maximum allowed cyclomatic complexity, I told the agent in the lint message that it may slightly increase the thresholds if it thinks that a refactoring is unnecessary or impossible in a particular case. This doesn't suppress the threshold forever, just increases it, so that the rule fires again if it gets even worse in the future. Constraints are preserved without forcing a binary suppress-or-comply choice.
对于各种阈值,比如最大行数或允许的最大圈复杂度,我在代码检查消息中告诉该工具:如果它认为在某种情况下进行重构是不必要或不可能的,那么可以适当提高这些阈值。这样做并不会永久性地取消这些限制,只是暂时提高阈值而已。这样一来,如果未来情况变得更糟,规则就会再次被触发。这样一来,各种约束条件仍然得到保留,而无需非得做出“要么遵守规则,要么完全忽略规则”的选择。
Observations 观察结果/观测数据
- Looking at the exceptions AI created (suppressed warnings, increased thresholds) was a good point to start my code review.
查看人工智能所生成的那些例外情况(被抑制的警告、被提高的阈值等),是开始代码审查时的好起点。 - AI frequently decided to increase the cyclomatic complexity threshold, but suggested good refactorings when I nudged it further. It was the only category where it did that, and I later discovered that I didn't have a self-correction guidance in place for this one, so there was no explicit instruction saying that a threshold increase should be the absolute exception. This is an indicator that the custom lint messages can indeed make quite a difference.
AI 经常决定提高循环复杂度的阈值,但当我进一步调整该阈值时,AI 会提出相应的重构建议。这是 AI 唯一会这样做的领域。后来我发现,对于这种情况,我没有设置任何自动修正机制,也就是说,没有明确的指示说明提高阈值应该是例外情况。这说明,自定义的代码检查提示确实能够产生很大的影响。 - Sometimes I want to treat rules differently in different parts of the code. Let's take
no-console, telling AI off when it usesconsole.log. In the backend, I want it to use a logger component instead. In the frontend, I might want to not use direct logging at all, or at the very least I need to use a different logging component. This is another example of the power of the self-correction guidance, and where AI can help with semantic judgment and management of analysis warnings.
有时候,我希望在代码的不同部分对规则进行不同的处理。以no-console为例:当 AI 使用console.log时,我希望系统能给出警告。在后端,我希望系统使用日志记录功能来处理这种情况;而在前端,我可能根本不想进行日志记录,或者至少希望使用不同的日志记录方式。这再次体现了自我修正机制的强大之处——AI 可以帮助我们进行语义判断,并妥善处理各种分析警告。 - I was watching out for examples of trade-offs between rules. The only one I've seen so far was created by the
max-linesandmax-lines-per-functionrules. I've seen AI do quite a bit of useful refactoring and breakdown into smaller functions and components as a result of this sensor feedback. However, in the React frontend, I'm seeing a worrying trend of components with lots and lots of properties as a result of passing values through a growing chain of smaller and smaller components. I haven't got useful observations yet about how good AI might be at making consistent decisions between tradeoffs like that.
我一直在留意各种规则之间所存在的权衡关系。到目前为止,我只看到max-lines和max-lines-per-function规则所导致的那种权衡情况。通过这种反馈机制,AI 能够有效地对代码进行重构,将复杂的代码拆分成更小的函数和组件。不过,在 React 前端开发中,我注意到一个令人担忧的趋势:由于数据需要通过越来越复杂的组件链来传递,导致某些组件的属性数量变得极其庞大。关于 AI 在处理这类权衡问题时的表现如何,我还没有得出有意义的结论。
Main takeaways 主要结论/要点
Overall, I was positively surprised by how many things I can cover with static analysis. I had to remind myself multiple times why it has been somewhat underused in the past, and what has changed: The cost-benefit balance. Cost is reduced because it's much cheaper to create custom scripts and rules with AI. And the benefit has also increased: the analysis results help me get a first sense of lots of hygiene factors that wouldn't even happen that much when I write code myself, so I can get common AI mistakes out of the way.
总体而言,静态分析能够处理的问题之多让我感到十分惊喜。我不得不多次提醒自己:为什么过去这种技术没有得到更广泛的应用?又是什么导致了这一变化呢?答案就是成本与收益的平衡。使用人工智能来编写自定义脚本和规则的成本大大降低了。同时,其带来的好处也有所增加:分析结果有助于我提前发现许多问题,而这些问题是我自己编写代码时很难发现的。这样一来,就能避免那些常见的由人工智能导致的错误。
However, I can't help but wonder if this can also lead to a false sense of security and an illusion of quality. After all, another reason why linters like this have been less used in the past is that they have limits, and we have been wary of using them as a simplified indicator of quality. There are lots of more semantic aspects of quality that static analysis cannot catch, it remains to be seen if AI can adequately fill that gap in partnership with those tools. I also discovered new supposed issues in the code every time I activated a new set of rules. It was always a mix of irrelevant things and things that actually matter. So I worry about feedback overload for the agent, sending it into a spiral of over-engineered refactorings.
不过,我不禁怀疑,这是否也会带来一种虚假的安全感以及质量上的错觉。毕竟,过去这类工具很少被使用的另一个原因就是,它们存在局限性。我们一直不愿将它们作为衡量质量的简单指标。静态分析无法捕捉到许多与代码质量相关的重要方面。目前还不清楚,人工智能能否与这些工具相结合,有效弥补这一缺陷。每次我启用新的规则后,都会发现一些所谓的“问题”。不过,其中既有无关紧要的内容,也有真正需要处理的问题。因此,我担心这会给相关工具带来过重的处理负担,导致不必要的过度重构。
Static code analysis: Dependency rules
静态代码分析:依赖关系规则
Basic linting is mostly focussed on quality and complexity within a file or function. Next I started looking into sensors that could give me and the agent feedback about maintainability concerns that cross file and module boundaries. Analysis tools in this area are historically even more underused than the basic linting.
基本的代码检查主要着眼于单个文件或函数的质量和复杂性。接下来,我开始研究那些能够让我和开发人员了解那些跨越文件和模块边界的、与代码可维护性相关的问题的人工智能工具。在这方面,分析工具的利用率其实比基本的代码检查还要低。
To learn about the potential of sensors that can help us and AI keep up good modularity inside of a codebase, I explored three things:
为了了解那些能够帮助我们和人工智能在代码库中保持良好模块化结构的传感器所具备的潜力,我研究了三个方面:
- Dependency rules (deterministic)
依赖关系规则(确定性规则) - Coupling analysis (deterministic and inferential)
耦合分析(确定性分析与推断性分析) - Modularity review (inferential)
模块性评估(推理层面)
Let's start with dependency rules. I worked with the agent to come up with a layered module structure for my application, about half way through implementing it. I asked it to help me write dependency-cruiser rules to enforce these layers.
让我们先从依赖关系规则开始吧。在开发过程中,我与该代理程序合作,为我的应用程序设计了分层模块结构。在开发进行到一半时,我请它帮助我编写 dependency-cruiser 规则,以确保这些分层结构得到妥善执行。

Figure 3: Layered module structure and dependency rules
图 3:分层模块结构与依赖关系规则
For example, one of the rules enforces that code in the clients folder never imports anything from the services folder:
例如,其中一条规则规定: clients 文件夹中的代码绝不能从 services 文件夹中导入任何内容。
{
name: “clients-no-services”,
comment:
“API clients must not depend on the orchestration layer above them. “ + LAYERS,
severity: “error”,
from: { path: “^server/clients/”, pathNot: “/__tests__/” },
to: { path: “^server/services/” },
},
As with the ESLint messages, I also expanded the error messages a bit to be self-correction guidance, recapping the layering concept as a whole:
与 ESLint 的提示信息类似,我也对错误提示进行了进一步细化,使其能起到自我纠正的作用。同时,我还重新阐述了整个分层结构的概念。
ERROR clients-no-services
API clients must not depend on the orchestration layer above them.
[Layers: routes -> services -> clients + domain; Services orchestrate: fetch data via clients, compute via domain -- no I/O, no SDKs, no knowledge of data fetching.]
Observations 观察结果/观测数据
- Without AI, I would not have gotten these rules in place quickly. The tool's configuration syntax has a steep entry cost, and AI absorbed that cost almost entirely.
如果没有人工智能,我根本无法如此迅速地建立起这些规则。该工具的配置语法相当复杂,学习成本很高,而人工智能几乎完全承担了这一成本。 - The agent violated the rules a handful of times after I introduced them, and then self-corrected based on
dependency-cruiserfeedback, so it did help keep my folder concepts.
在我向该智能体介绍了相关规则后,它有几次违反了这些规则。不过,根据dependency-cruiser的反馈,它及时进行了自我纠正。因此,该智能体确实有助于我更好地管理文件夹中的各种内容。 - I also used the same approach to introduce conventions for how React hooks should be structured in the frontend.
在定义前端中 React Hooks 的架构时,我也采用了同样的方法。 - I had to figure out how to catch things when AI starts creating new folders outside of this structure, with a rule that requires every new file to be somewhere in the predefined folder structure.
当人工智能开始在这个结构之外创建新的文件夹时,我必须找到一种方法来处理这些情况。根据规则,每个新文件都必须被保存在预定义的文件夹结构中。
Main takeaways 主要结论/要点
At the point when I introduced these rules, the structuring of code into folders had already become a little bit haphazard. I could see how the rules helped the agent clean that up, and then continue enforce these layers going forward. So I've found it quite a useful replacement for describing code structure in a markdown guide. However, tools like this are limited to what is expressible via imports, file names, and folder structure.
在我制定这些规则的时候,代码的组织方式已经变得有些杂乱无章了。我发现,这些规则有助于让代码结构更加有序,同时也能确保这种有序状态得以持续保持。因此,我觉得这些规则是用来描述代码结构的绝佳工具,完全可以替代传统的 Markdown 格式。不过,这类工具的局限性在于:它们只能通过导入语句、文件名和文件夹结构来体现代码的结构。
Static code analysis: Coupling data
静态代码分析:数据耦合问题
Next, I experimented with the extraction of typical coupling metrics from my codebase, i.e. the number of incoming and outgoing imports and calls per file.
接下来,我尝试从代码库中提取一些常见的耦合度指标,比如每个文件中对应的导入和调用的数量。
I didn't use any existing tools for this, instead I had a coding agent write an application that creates those metrics with the help of the typescript compiler, so that I could have maximum flexibility to play around with this as part of my experimentation. I had it add two interfaces: A web interface with a bunch of different visualisations of those metrics for my own human consumption. And a CLI that can provide those metrics to a coding agent.
在开发过程中,我没有使用任何现成的工具。相反,我让一个编程智能体来编写应用程序。该应用程序借助 TypeScript 编译器来生成各种指标数据。这样一来,我就能在实验过程中自由地调整各种参数,从而获得最理想的实验结果。该应用程序还包含了两个接口:一个是网页界面,用于以各种可视化方式展示这些指标数据,方便我直接查看;另一个是命令行接口,可以将这些指标数据提供给编程智能体使用。

Figure 4: Coupling metrics: web visualisations and CLI for agents.
图 4:耦合度指标:用于各智能体的网页可视化界面与命令行界面。
For human consumption 供人类食用
Most of these visualisations are well established concepts, like a dependency structure matrix (DSM). I found them tedious to interpret, and even though they were vibe coded and could most certainly be improved, I think that had more to do with the nature of the data. It's quite detailed data that needs a lot of context and experience to interpret it, and map it back to more high level good practices. So I have a feeling that these types of tools still won't really help reduce a human's cognitive load much when reviewing codebases that were changed by AI.
这些可视化工具所呈现的内容,大多都是些早已被广泛使用的概念,比如依赖结构矩阵之类的。不过,我觉得这些工具很难理解。虽然它们可以通过某种方式来优化,但我觉得这更多是因为数据的复杂性所致。这些数据非常详细,需要大量的背景知识和经验才能理解,进而将其转化为更易于理解的、有用的信息。因此,我觉得,当人们需要审查由人工智能修改过的代码库时,这类工具其实并不能有效减轻人类的认知负担。
For AI consumption 用于人工智能相关应用/人工智能领域的使用
I gave an agent access to this custom CLI (coupling-analyser) and asked it to create a report based on the data, including suggestions of how to improve the critical issues.
我授权该代理使用这个自定义的 CLI 工具( coupling-analyser ),并让它根据这些数据生成报告,同时提出改进那些关键问题的建议。
Here is an excerpt of what that prompt looked like - I'm mainly reproducing this to show you that I didn't actually give it much guidance on what good or bad modularity looks like, I mostly delegated to the model to interpret what good and bad looks like:
以下是该提示语的摘录——我之所以复制这段内容,是为了向您说明:其实我并没有给模型太多关于“什么是好的模块化结构、什么是不好的模块化结构”的指导。我基本上让模型自己去判断什么是好的、什么是不好的。
Produce a markdown report on modularity and coupling quality for the target TypeScript codebase, grounded in actual CLI output from npx coupling-analyser, not guesswork from static browsing alone.
针对目标 TypeScript 代码库,基于 npx coupling-analyser 提供的实际 CLI 输出结果来生成关于代码的模块化程度和耦合程度的分析报告,而非仅凭静态代码分析来推测结果。
Gather evidence (run the CLI)
收集证据(运行命令行界面)
Execute the CLI and capture stdout. Use the report subcommands—combine as useful for the question: …
运行 CLI 命令并捕获标准输出。使用 report 子命令——将它们组合起来以更好地回答这个问题:……
Write the markdown report
编写 Markdown 格式的报告
Use clear headings. Prefer concrete module IDs / paths and numbers quoted or paraphrased from CLI output.
请使用清晰的标题。最好使用具体的模块 ID/路径,以及从 CLI 输出中直接引用或转述的数字。
Suggested sections: 建议包含的章节:
-
Context — What was analyzed
背景——分析了什么内容 -
Executive summary — 2–5 bullets: overall modularity posture, top 1–3 systemic issues.
执行摘要——2-5 点:整体的模块化架构情况,以及最突出的 1-3 个系统性问题。 -
Findings from the tool — Summarize hotspots, top risks, notable cycles or mutual dependencies, and behavioural highlights as reported by the CLI.
该工具的分析结果包括:总结 CLI 所报告的热点问题、主要风险、值得关注的周期性现象或相互依赖关系,以及各种行为模式。 -
Interpretation (modularity lens) — Tie metrics to software design: cohesion vs. spread of change, stability vs. dependency direction, fan-in/fan-out intuition, cycle impact.
解读(模块化视角)——将各项指标与软件设计联系起来:内聚性与变更的扩散程度、稳定性与依赖关系的方向、扇入/扇出结构的直观理解、循环复杂度的影响。 -
Deep dives for each high and critical issue
对每一个重大且关键的问题都进行深入分析/详细探讨
- What it is — Module(s), role in the system, dependency neighbours (from CLI + minimal code peek if needed).
它到底是什么——包括所属的模块、在系统中的角色、以及与其相关的依赖关系(如有需要,还可以查看 CLI 命令和最小限度的代码示例)。 - Responsibilities today …
今天的责任…… - Why it hurts …
为什么这么疼…… - Design options (2+ where reasonable) …
设计选项(在合理的情况下,至少 2 种以上)…… - Why the new design is better — Fewer cycles, clearer dependency direction, smaller surfaces, test seams, align with likely change vectors.
为什么新设计更优:所需的循环次数更少,各部分之间的依赖关系更清晰,表面面积更小。此外,新设计还便于进行测试,且更符合可能发生的变化趋势。 - Future change risk — How each option reduces regression risk and makes safe evolution cheaper (concrete scenarios: “adding X”, “swapping Y”, “shipping Z independently”).
未来的变化风险——每种选择如何降低回归风险,从而让系统的安全演进变得更加容易实现(具体方案包括:“添加 X”、“替换 Y”、“单独部署 Z”)。
…
This LLM-led analysis actually pointed me to the same coupling hot spots that I would have found by looking through the visual diagrams, just in a format that was more digestible. And asking the LLM to ground its analysis in the results from the deterministic tool gave me a higher level of confidence, and probably also used less time and tokens than if the agent had scanned the codebase itself to find coupling problems.
这项由大语言模型主导的分析结果,实际上让我发现了与通过可视化图表所发现的相同的问题点。只不过,大语言模型呈现的结果更易于理解。此外,让大语言模型以确定性工具的分析结果作为依据,让我更加放心。与让智能体自行扫描代码库来查找问题相比,这种方式显然更省时、更高效。
Observations 观察结果/观测数据
What the LLM found based on this data was quite lackluster (I used Claude Opus 4.7 for this):
基于这些数据,大语言模型的表现相当糟糕(我使用的是 Claude Opus 4.7 版本):
- It said one of the biggest issues was a factory that initialises all the necessary components, but I had introduced that factory on purpose as a component that acts like a lightweight dependency injection framework.
文中提到,最大的问题之一就是有个负责初始化所有必要组件的工厂类。不过,我特意将这个工厂类设计成一个类似轻量级依赖注入框架的组件。 - Another issue it had was with a shared (
zod) schema between frontend and backend, declared a “god module” by the LLM. This is a common pattern though to create an explicit contract between backend and frontend, and is not as much of an issue when backend and frontend evolve together anyway, or even live together in the same repo, like in my case.
另一个问题是,前端和后端之间使用了共享的架构(zod)。LLM 将这个架构视为一个“核心模块”。不过,这种在前后端之间建立明确接口的方式其实很常见。只要前后端能够同步发展,或者像我这样,存储在同一个代码库中,那就不会有什么问题了。 - When legitimate patterns appear as high-coupling hubs, there would have to be a way to suppress those in future analyses, otherwise they create even more noise.
当那些属于正常现象的模式被当作“高耦合节点”来处理时,就必须在后续的分析中设法将它们剔除掉。否则,它们只会制造更多的干扰和混乱。 - The one kind of interesting finding it had: An
index.tsfile in the domain folder indiscriminately exposed all files in./domain, and is imported by lots of places. While that is also a common pattern to create explicit contracts for a layer, it does have its pros and cons, and is at least worth an investigation to see if it is appropriate for this codebase.
其中有一个有趣的发现:域名文件夹中的index.ts文件会无差别地暴露./domain中的所有文件,而且被许多地方所引用。虽然这种做法常被用来为某个层创建明确的接口规范,但它也有各自的优缺点。至少值得研究一下,看看这种做法是否适合当前的代码库。
Main takeaways 主要结论/要点
The examples above show that even more so than with the basic linting, good and bad does not have a clear definition, instead it is all about what is appropriate. And what coupling is appropriate depends on a lot of context, not just the raw call and import graph of a codebase. So based on this small experiment, I don't have the impression that this type of coupling data is useful to AI on its own.
上述例子表明,与基本的代码检查相比,所谓“好”与“坏”的标准并不明确。关键在于什么才是合适的做法。而什么样的耦合方式才是合适的,则取决于许多因素,而不仅仅是代码中的调用关系和导入关系。因此,通过这个简单的实验,我觉得这类关于耦合程度的数据本身并不太适合让人工智能来处理。
A more practical use I can imagine for this data is during risk triage for code review. When I review a code change made by AI, it seems useful to know what the impact radius of the changed files is, so that I can pay more attention when e.g. a file with 10+ callers is changed. Or an AI review agent could use the data to prioritise where it spends its tokens.
我能想到的一个更实际的应用场景是:在代码审查过程中进行风险评估。当我审查人工智能所做出的代码修改时,了解这些修改对其他文件的影响范围会很有帮助。比如,如果某个文件被修改了,而该文件又被其他 10 多个文件所引用,那我就需要格外注意这个修改可能带来的影响。或者,人工智能审查工具也可以利用这些数据来决定应将审查资源优先分配到哪些地方。
Static code analysis: AI modularity review
静态代码分析:人工智能辅助的模块化审查
The lackluster results from the coupling data experiment could have multiple reasons:
耦合数据实验结果不佳可能有多种原因:
- My prompt about what to analyse was not very specific
我关于分析内容的指示并不十分具体。 - The coupling data is not useful to AI
这些耦合数据对人工智能来说毫无用处。 - The coupling data only is too shallow and lacks context of the full code
仅有的耦合数据太过片面,缺乏对整个代码的全面理解。
So the final thing I did was to go fully down the inferential route and use Vlad Khononov's “Modularity Skills” to analyse the codebase design and find modularity issues. This proved to be very fruitful! It gave me lots of interesting pointers for refactorings that would obviously reduce the risk of future changes. I ran the skills a second time and gave them access to my coupling analysis CLI. The AI mostly found confirmation in the data, but not any additional findings. On the contrary, it pointed out lots of things that the CLI was missing. It's also worth noting that the second run of the analysis (without context of the first one) surfaced yet another issue that the first run did not find. A useful reminder that when it matters, it's often worth running an LLM-based analysis multiple times, to get a fuller picture.
最后,我采用了完全基于推理的分析方法,运用 Vlad Khononov 提出的“模块化分析技术”来评估代码架构中的模块化问题。这一做法取得了很好的效果!它为我提供了许多有用的改进建议,这些改进建议显然有助于降低未来进行代码修改时的风险。我又运行了一次该分析工具,并允许它访问我的代码耦合度分析工具。人工智能在数据分析中得到了大部分数据的验证,但并没有发现其他新的问题。相反,它指出了许多该分析工具所遗漏的问题。值得注意的是,第二次分析(在没有第一次分析结果作为参考的情况下)又发现了一些第一次分析时没有找到的问题。这提醒我们:在关键情况下,多次使用基于大语言模型的分析工具来获取更全面的分析结果是非常有必要的。
Observations 观察结果/观测数据
Here are some highlights from the results (model used was Claude Opus 4.7, same as for the coupling analysis):
以下是部分关键结果概要(所使用的模型与耦合分析时相同,均为 Claude Opus 4.7):
- Duplicate route code - all my three backend endpoints had their own route file, and each of those route implementations was almost identical. So whenever I would want to introduce a change to the general principles of the backend API (let's say introducing a request ID, or changing the error handling or logging approach), I'd have to do it in multiple files. I had only just introduced a third endpoint, so I think it's fair enough that this wasn't abstracted out yet. But in my experience, AI agents usually don't go ahead and start refactoring without an explicit nudge when they repeat a piece of code for the third or fourth time, they are quite happy to copy and paste.
路由代码重复——我的三个后端端点各自都有独立的路由文件,而这些路由的实现方式几乎完全相同。因此,每当我想对后端 API 的总体架构进行修改时(比如添加请求 ID、更改错误处理或日志记录方式),我都必须在多个文件中进行相应的修改。我刚刚才添加了第三个端点,所以目前还没有将相关代码进行抽象化处理,这也是可以理解的。不过以我的经验来看,当 AI 代理需要重复编写某段代码第三次或第四次时,它们通常不会主动去重构代码,而是直接选择复制粘贴即可。 - Inconsistency in calling the backend - or put another way, yet another form of semantic duplication. I have 3 pages in the application that need to call the backend with the same set of parameters (selected chat space, and which date range to analyse). Two of those pages were using the same hook and general approach to do this, but when AI introduced the third page, it deviated from that and reimplemented similar behaviour in its own way. This can e.g. lead to inconsistencies in error handling, or again the need to change multiple files when backend API principles change.
在调用后端接口时存在不一致性——换句话说,这又是一种语义上的重复。在该应用程序中,有 3 个页面需要使用相同的参数来调用后端接口(即所选的聊天空间以及需要分析的时间范围)。其中两个页面采用了相同的实现方式,但当 AI 功能被添加到第三个页面时,该页面采用了不同的实现方式。这可能会导致错误处理上的不一致性,或者当后端 API 的规范发生变化时,就需要修改多个文件。 - Inefficient handling of the core arguments - As just mentioned, all the pages in the application pass on a chat space ID and a date range to the backend. I had already noticed when I changed the way a user can specify a date range that AI had to change a lot of files for that change - over 40! So I was already aware that something was fishy here, and the analysis confirmed it: “Issue: Request parameters repeated at every level”. The recommendation was to introduce an object that wraps all of these parameters. AI had already done that in a way - but never fully followed through with the usage of that object, so it was an inconsistent mess.
对核心参数的处理方式效率低下——正如刚才所提到的,该应用程序中的所有页面都会将聊天空间 ID 和日期范围传递给后端。当我尝试改变用户指定日期范围的方式时,我发现 AI 不得不修改 40 多个文件才能实现这一变更。因此,我早就觉得有些不对劲了。进一步的分析也证实了这一点:“问题在于:各个层级上请求参数都重复出现”。解决方案是创建一个能够包含所有这些参数的容器对象。AI 其实已经尝试过这么做,但并没有彻底贯彻这一做法,因此整个系统显得杂乱无章。 - Responsibilities in the wrong place - The review found a bit of authentication code sitting inside our factory that was supposed to only be responsible for wiring up our modules. It implemented a fallback to mock data when the user is not authenticated. An unexpected location like that creates a risk of being missed when new routes are added.
职责分配不当——审查发现,我们的工厂里有一些本应只负责模块连接的组件,却也承担了身份验证的功能。当用户未经过身份验证时,这些组件会使用模拟数据来应对。这种安排意味着,当有新的路由被添加时,很容易被忽略,从而带来风险。 - Better interpretation of acceptable high-import-count “hubs” - Remember the “god classes” found by my previous coupling analysis? The modularity skills also noticed these, but in both cases nicely pointed out that they have a purpose in the context of this application. I assume that is either due to the good prompting in these skills, or due to the fact that this analysis actually read what was in the code, whereas I asked the other one to only rely on the coupling data.
对于那些具有较高导入次数的“核心组件”,现在的解释更为合理了。还记得我在之前的耦合性分析中发现的那些“关键组件”吗?模块化分析工具也注意到了这些组件,而且都指出了它们在该应用场景中的重要作用。我认为,这要么是因为这些分析工具的提示很到位,要么是因为该分析工具真正理解了代码中的含义,而另一项分析则仅仅依赖于耦合性数据来进行分析。
Main takeaways 主要结论/要点
- Dependency parsers like
dependency-cruisercan be effective live sensors to enforce some basic folder structures and dependency directions, but they can only go so far.
像dependency-cruiser这样的依赖关系解析器,可以作为有效的“实时监控工具”,用来确保文件夹结构的规范性以及各种依赖关系的正确性。不过,它们的作用毕竟有限。 - The AI modularity review is a great example of “garbage collection”, and worked quite well when given powerful prompts. Grounding it in actual coupling data didn't seem to make much difference. It would be great to find a way to apply this to the changed files in a commit, to have this earlier in the pipeline, but I did not explore this yet.
AI 模块化审查机制是“垃圾回收”机制的一个绝佳范例。当有恰当的提示信息时,该机制的效果相当不错。不过,将实际的数据关联起来似乎并没有带来太大效果。如果能找到一种方法,将这一机制应用到每次提交后产生的变更文件上,那就再好不过了。不过,我还没有深入研究这个问题。 - I ran the modularity review after building most of the codebase without applying that type of review myself - and it had some quite concerning and very valid findings that would have increased risk in the future. It shows that without human review and coupling expertise, AND without these extra AI reviews, the agent was definitely compounding inadvertent technical debt.
在编写了大部分代码之后,我进行了模块化审查。不过,我之前并没有亲自进行这种审查。审查结果相当令人担忧:其中发现的一些问题如果得不到妥善处理,将来可能会带来更大的风险。这说明,如果没有人工审查和专业的评估,再加上缺乏人工智能的辅助审查,那么该系统就会不断积累不必要的技术缺陷。
Overall, codebase design and modularity seems like a concern where computational sensors alone cannot help us much, AI is needed to add semantic interpretation, and consider trade-offs.
总体而言,代码架构和模块化设计是个需要重点考虑的问题。仅靠计算传感器难以解决这些问题,必须借助人工智能来提供语义解析,并权衡各种因素。
The test suite as a regression sensor
测试套件作为回归检测工具的用途
Tests have many purposes — they help us think about and drive our design, they document the wanted behaviour of the application (they are the ultimate specification!), and they help us detect regressions, i.e. they tell us when we break pre-existing functionality with a change. Effective regression tests play a big role in the maintainability of a codebase, they make it much safer to change it. So in the context of maintainability sensors, this section is about the test suite's role as a regression sensor.
测试有着多种用途:它们有助于我们思考和规划设计方案;它们能够记录应用程序应有的行为表现(实际上,测试结果就是最详细的规格说明);此外,它们还能帮助我们发现代码中的缺陷,也就是当某些更改导致现有功能出现故障时,测试能及时发现这些问题。有效的回归测试对代码的可维护性至关重要,它们能让代码的修改过程更加安全。因此,在讨论与可维护性相关的因素时,本节主要探讨了测试套件作为“回归检测工具”的作用。
When a pre-existing test fails, we have to ask ourselves a question: “Did I break something accidentally, so I need to change my implementation? Or am I changing the behaviour intentionally, so the tests have to change to adapt to this new specification?” A failing test gives AI the opportunity to ask that very question. It might not always take the right decision, mind you! But a good test suite decreases the probability that AI breaks wanted pre-existing behaviour.
当某个现有的测试失败时,我们必须问自己一个问题:“是我不小心弄坏了什么,因此需要修改代码吗?还是说,是我有意改变了程序的行为,所以测试也需要随之调整以适应这种新变化?”测试失败其实给了人工智能机会来思考这个问题。当然,人工智能并不总能做出正确的决策。不过,完善的测试机制可以降低人工智能破坏现有功能的概率。
In my chat analytics application, I had the agent write all the tests over time without much oversight other than manual testing and keeping an eye on the test coverage. I wanted to have a full AI-generated test suite to analyse its regression effectiveness in hindsight.
在我的聊天分析系统中,我让系统自动执行所有的测试。除了进行手动测试并监控测试的覆盖率之外,几乎没有其他人为的干预。我希望拥有一个完全由人工智能生成的测试套件,以便事后分析系统的稳定性如何。
There are two main risks with the approach of AI generating tests without review:
在人工智能生成测试且不经过人工审核的情况下,主要存在两大风险:
- Coverage is not a sufficient indicator of test effectiveness
覆盖率并不能充分反映测试的有效性。 - The tests might be testing faulty behaviour — this is a much more difficult problem than checking test effectiveness, and one for another time. This article focusses on test effectiveness only, i.e. assuming that our code implements the wanted behaviour, do we have tests that catch breaking code.
这些测试可能旨在检测那些有缺陷的行为——这比验证测试的有效性要困难得多,这个问题留待以后再讨论吧。本文仅关注测试的有效性,也就是说,假设我们的代码实现了预期的功能,那么我们是否有足够的测试来发现那些导致程序出错的代码呢?
What is in our toolbox?
我们的“工具箱”里有什么呢?
- Coverage ($) — tracks which parts of the code are executed by tests, giving an indication of which parts of the code are visible and invisible to tests.
覆盖范围($)——用于记录代码中哪些部分被测试所覆盖,从而判断代码的哪些部分可以被测试检测到,哪些部分则无法被检测到。 - Property-based testing ($) — can find missing logical test cases, by generating many input combinations from defined properties rather than hand-crafting examples.
基于属性的测试——通过根据既定的属性来生成各种输入组合,从而发现那些被遗漏的逻辑测试用例。这种方式无需手动创建测试用例。 - Fuzz testing ($$) — can find missing test cases for input resilience, by throwing unexpected or malformed inputs at the system.
模糊测试——通过向系统输入意外或格式错误的数据,从而发现与输入处理相关的测试用例的缺失之处。 - Mutation testing ($$) — can find missing assertions, by introducing small code mutations and checking whether the test suite catches them.
变异测试——通过引入小的代码变更,然后检查测试套件是否能检测到这些变更,从而发现那些被遗漏的断言。
In my application, I used coverage and mutation testing, as property-based testing and fuzz testing weren't as suitable to my use case.
在我的应用中,我采用了覆盖测试和变异测试。因为基于属性的测试和模糊测试并不适合我的应用场景。
Mutation testing 变异测试
Here is a small example from my codebase to illustrate how mutation testing can help us find gaps in assertions. The agent created this diagram for me during the analysis of mutation testing results:
以下是我代码库中的一个小例子,用于说明如何利用变异测试来发现断言中的漏洞。在分析变异测试结果时,该工具为我生成了这张图表:

Figure 5: Mutation testing example from the codebase.
图 5:代码库中的突变测试示例。
The mappers.ts file reported 100% statement coverage and 75% branch coverage — but it turned out to have no unit tests, and Stryker (the mutation testing tool I used) reported 13 survivors (i.e. after 13 of Stryker's code mutations the test suite was still green). The coverage in this case was high because the codebase has a big acceptance test that ultimately called these functions — coverage tells us that a line was executed, but not that its impact was verified. If this little mappers helper function dvpToSchema would be changed in the future, it could potentially break the display of a data graph in the UI.mappers.ts 文件显示,代码的覆盖率为 100%的函数覆盖率和 75%的分支覆盖率。但实际上,该代码没有任何单元测试。而我所使用的变异测试工具 Stryker 则检测到有 13 处代码在经过变异处理后,测试用例仍然能够正常通过。虽然这里的覆盖率看起来很高,但实际上,因为整个代码库中有一个大的验收测试,该测试会调用这些函数,所以才有了这么高的覆盖率。覆盖率只能说明某行代码确实被执行了,但无法确保其功能是否正常。如果将来 dvpToSchema 这个辅助函数发生变动,就有可能导致用户界面中的数据图表无法正常显示。
Observations 观察结果/观测数据
- AI was very helpful in analysing the mutation hot spots and making a prioritised plan where to increase test quality.
在分析突变热点以及制定提升检测质量的优先计划方面,人工智能发挥了极大的作用。 - Stryker writes results to a huge JSON file. To help with analysis and avoid accidentally clogging the context window, I generated a custom script to help the agent query Stryker's results efficiently. That is just one of many examples where AI helped me help AI.
Stryker 会将分析结果保存在一个庞大的 JSON 文件中。为了便于分析数据,同时避免意外占用过多的系统资源,我编写了一个自定义脚本,帮助智能体高效地获取 Stryker 的分析结果。这不过是人工智能在帮助我提升工作效率的众多例子之一罢了。
"""Query a Stryker mutation-testing JSON report from the command line.
Usage:
python query_stryker.py <report.json>; <command> [options]
Commands:
summary Overall status totals, mutation scores, thresholds.
files Per-file breakdown, default sorted by mutation score asc.
hotspots Lines with the most survivors / no-coverage mutants.
tests Test effectiveness: weak, unused, or top-killer tests.
Examples
# 1. Overall health — mutation score, status breakdown, threshold pass/fail
python ./query_stryker.py reports/mutation/mutation.json summary
# 2. Worst files first, with an action hint (strengthen assertions vs add tests)
python ./query_stryker.py reports/mutation/mutation.json files --top 10 -v
# 3. Same, but only for files you've changed in git (auto-detects the repo)
python ./query_stryker.py reports/mutation/mutation.json files --changed -v
# 4. Zoom into one file: every (line, actionable counts, sample mutators)
python ./query_stryker.py reports/mutation/mutation.json hotspots --file server/services/ai-summaries.ts --top 30
"""
Main takeaways 主要结论/要点
There currently seems to be a trend towards more end-to-end style acceptance tests. As mentioned in the beginning, AI has gotten really good at generating tests, so it has become quite normal for developers to just let AI generate lots of tests, without much review. Reviewing unit tests in particular can be very tedious. I'm not saying it's a good thing not to look at them at all — but I acknowledge the reality that it is unrealistic to think that human review of all tests is sustainable, and it's unrealistic to think that people will actually do it. So while we search for the appropriate testing pyramid/ice cream cone/muffin shape of the AI coding future, techniques like approved scenarios are becoming popular. As demonstrated above, acceptance tests increase coverage, but are often not very assertion-heavy, giving us a false sense of security in test effectiveness — mutation testing helps us monitor that gap.
目前,似乎有一种趋势,那就是更倾向于使用端到端的测试方式来进行验收测试。正如前面所提到的,人工智能在生成测试用例方面表现得相当出色。因此,开发者们通常会让人工智能自动生成大量测试用例,而无需进行过多的审核。不过,审核单元测试其实是一件相当繁琐的工作。我并不是说完全不审核测试用例就是好事——但现实情况是,让人类来审核所有的测试用例是不现实的,人们也不可能真的这么做。因此,在我们寻求适合人工智能编程未来的测试方式时,像“经过验证的测试场景”这样的技术正变得越来越受欢迎。正如上文所述,验收测试虽然能提高测试的覆盖率,但往往缺乏有效的验证机制,这会让我们误以为测试效果很好。而变异测试则有助于弥补这一缺陷。
Mutation testing has a practical limitation of course: It is quite resource intensive. In my setup I didn't run it continuously (like some of my other sensors), but triggered incremental runs manually.
当然,变异测试也存在着实际上的局限性:它需要大量的资源来运行。在我的系统中,我没有像处理其他传感器那样持续不断地运行变异测试,而是手动触发其执行。
Conclusions and open questions
结论与未解决的问题
Computational sensors impressed me most at the file and function level. Cross-file concerns like modularity and coupling were a different story, the raw data itself was very noisy and not that useful without semantic interpretation of an LLM, i.e. an inferential sensor. But I was very impressed by the outputs and advice I could get from that with a good prompt, and also by the potential to present this information in different ways, for different experience levels.
在文件和功能层面,计算型传感器给我留下了深刻印象。不过,涉及多个文件之间的协同问题时,情况就有所不同了:原始数据本身包含大量噪声,如果没有大型语言模型的语义解析,这些数据就没什么用处。不过,只要给出合适的提示,这些计算型传感器就能提供有用的输出和建议。此外,这些信息还可以以多种方式呈现出来,从而满足不同用户的需求。
What I haven't seen in my experiments, but suspect can become more of an issue, is conflicts between sensors. The max-lines and max-lines-per-function rules showed some signs of tension, the refactorings to smaller and smaller functions pushed complexity into component property chains instead. More trade-offs like that are probably lurking, and it will be interesting to see over time if and how that becomes a problem.
在我的实验中,虽然没有直接观察到,但我怀疑传感器之间的冲突可能会成为一个越来越严重的问题。关于“每行代码的最大长度”以及“每个函数中代码行的最大数量”的规则,都存在一些矛盾之处。而将代码拆分成更小的函数后,复杂性反而转移到了各个组件的属性链上。类似的问题可能还有很多,随着时间的推移,我们将会看到这些问题是否真的会变成问题,以及会以何种方式体现出来。
I did not bother with guides at all in this application, for the sake of seeing the effect of the sensors more purely. I'm curious about how the balancing of guides and sensors will evolve. Once we feel confident in a set of sensors, what guides can we delete? Do sensors make the use of weaker models more realistic? How do we keep guides and sensors consistent with each other, and will we find ways to bundle them together somehow, to make them easier to maintain?
在这个应用中,我完全没有使用任何引导线或辅助装置,目的是为了更纯粹地观察传感器的实际效果。我很好奇,引导线与传感器之间的平衡关系会如何发展。一旦我们对某组传感器有足够的信心,那么哪些引导线就可以被删除了呢?传感器是否能让那些性能较弱的模型也能得到更合理的利用呢?我们该如何确保引导线与传感器之间的协调性?另外,我们能否找到某种方式将它们结合起来,从而便于维护呢?
In the regression testing area, my eyes have really been opened to how crucial mutation testing becomes when we make the decision to leave most of the testing to AI... And I want to stress once more that there is a whole other conversation to be had about correctness of tests!
在回归测试方面,我深刻意识到:当我们决定将大部分测试工作交给人工智能来处理时,变异测试就显得至关重要了……我还要再次强调:关于测试的准确性,还有许多问题需要进一步探讨!
While some of these sensors really do increase my trust into the quality of the outcomes, they are not a magical solution to take the human totally out of the loop. But I definitely experienced an improvement in my review experience and trust level with both computational and inferential sensors as my partners.
虽然这些传感器确实提升了我对检测结果的信任度,但它们并不能完全取代人类的作用。不过,当把计算和推理功能作为辅助手段来使用时,我的检测体验和信任度确实有所提升。
Acknowledgements 致谢
Thanks to Chris Ford and Matteo Vaccari in particular for the conversations about sensors, and all my colleagues at Thoughtworks who engaged with this topic and gave me feedback and ideas.
特别要感谢 Chris Ford 和 Matteo Vaccari 就传感器相关问题与我进行的探讨。同时,也感谢 Thoughtworks 的所有同事们,他们积极参与了这一话题的讨论,并给了我很多宝贵的反馈和建议。
GenAI was used for research and polishing the language.
GenAI 被用于相关研究以及语言的优化处理。
Appendix: About tooling 附录:关于工具的相关信息
How did I do this practically then, how did I provide this sensor data to the agent? In all of the experiments described, I used a vibe-coded little “sidecar” application that could run all of my computational sensors. It's not a fully realised tool and I made some compromises for simplicity, but it helped me explore and speculate about what good tooling for this might look like.
那么,我究竟是如何实际实现这一点的呢?我又是如何将传感器数据传递给相关程序的?在所有实验中,我都使用了一个经过特殊设计的“辅助程序”来处理各种传感器数据。这并不是一个功能完备的工具,为了简化操作,我不得不做出一些妥协。不过,它确实帮助我思考了:究竟什么样的工具才最适合实现这一目标。
The sidecar is a CLI that can read a config file describing all sensor commands, and then run them continuously, in intervals. This is somewhat inspired by how we run test suites in watch mode while we're typing code.
Sidecar 是一种命令行工具:它可以读取描述所有传感器指令的配置文件,然后以定时间隔持续执行这些指令。这一设计理念,某种程度上借鉴了我们在编写代码时,通过“监视模式”来运行测试套件的方式。

Figure 6: Overview of the sidecar setup
图 6:侧车装置的总体结构示意图
The CLI can offer both situational awareness for the human in supervised sessions, and a token-efficient and guidance-enriched summary to the agent.
该命令行界面既能为人在监督模式下提供必要的情境感知信息,又能为智能体提供高效且富有指导意义的摘要信息。

Figure 7: Visual overview of sensor status
图 7:传感器状态一览图

Figure 8: Summary the agent gets when it checks for the status
图 8:代理在检查状态时得到的汇总信息
A few things to note in the agent summary:
在代理总结中需要注意的几点:
- “Follow scouting rule: ...” - this addition at the top is an example of a global guidance prompt I configured in the sensors config file.
“遵循侦察规则:……”——这一行文字是我在传感器配置文件中设置的通用指导提示的示例。 - “Worse than / Same as snapshot” - The CLI has the ability to persist a snapshot to communicate trends to the agent.
“比快照更差/与快照相同”:该命令行工具能够保存快照,从而向代理程序传递相关趋势信息。 - “higher / lower is better” - A description of what the score here is, and what direction is wanted. This might be a bit overkill though, as most models' training data should be well aware that low number of test failures is good, and high number for coverage is good.
“数值越高/越低越好”——这描述了这里的评分标准以及所期望的趋势。不过,这种描述可能有些多余,因为大多数模型的训练数据应该已经明白:测试失败次数越少越好,而覆盖率越高则越好。 - “(below target threshold of...)” - For some sensors, it needs to be defined what good looks like, it's not always 0 or 100.
“(低于目标阈值……)”——对于某些传感器而言,需要明确什么是“良好状态”。数值并不总是 0 或 100。
This example illustrates the opportunity we have in a setup like this to provide much more guidance for the self-correction than when we use the tools out of the box. And the agent summary could of course be minimised even more and only include information on things that have failed, similar to an “evergreen” type of CI monitor.
这个例子表明,在这样的架构中,我们完全有可能为系统的自我修正提供更多的指导,这比直接使用现成的工具要有效得多。当然,对代理程序的总结内容也可以进一步简化,只需包含那些出现故障的信息即可,就像那种“永久性”的持续集成监控系统一样。
Integration with the coding harness
与编码工具的集成
How to get the agent to check the sensor status regularly? There are multiple options for this.
如何让代理定期检查传感器的状态呢?有多种方法可以实现这一点。
- Via a guide - A skill, or a section in AGENTS.md that asks the agent to check the sensors regularly. This is the easiest way, and this is what I mostly did. But, it is also quite unreliable. I had to ask the agents many, many times why it had not run the sensors check a single time, or why it was running tests or linters directly instead of checking the sensors.
通过指南来操作——这是一种技能,或者说是 AGENTS.md 文档中的某一部分内容,它要求智能体定期检查传感器。这是最简单的方法,也是我通常采用的方式。不过,这种方式并不可靠。我不得不多次询问智能体:为什么它一次都没有执行传感器检查任务?或者,为什么它直接执行了测试或代码检查,而没有先检查传感器呢? - Via hooks - I imagine there's a way to force this more regularly with hooks, but it needs some consideration which hooks to use. After every file edit is probably a good starting point, but it would have to be monitored if it's too much distraction for the agent.
通过钩子来实现——我觉得可以用钩子来更频繁地执行这一操作,不过需要先确定使用哪些钩子。在每次文件被编辑后执行该操作应该是个不错的起点,不过也要注意,避免给相关程序带来过大的负担。 - Via git hooks - For users who like their agents to do frequent small commits, a
pre-commithook can be a good way to force a sensors check to take place before code even gets into a commit.
通过 Git 钩子实现——对于那些希望代理程序能够频繁进行小型代码提交的用户来说,使用pre-commit钩子可以确保在代码被提交之前,先对相关传感器进行检测。 - Custom extension - I wondered if providing the harness with a custom tool that does the check makes it more reliable. I tried this by building an extension for (and with) Pi. It seems quite promising, and I could easily do things like asking the user on startup if they want to start and snapshot a sensors session. But at the time of writing this, I haven't used the extension enough to tell if it's running more frequently than it would with an instruction in some markdown file.
自定义扩展程序——我在想,如果为该设备配备一个能够自动执行各项检测功能的自定义工具,会不会让其更加可靠。我尝试为 Pi 平台开发了这样的扩展程序。效果相当不错:例如,可以在系统启动时询问用户是否希望开始传感器数据的采集与存储。不过,在撰写这篇文章时,我还没有充分使用这个扩展程序,因此无法确定它的使用频率是否真的比单纯在 Markdown 文件中添加相关指令时更高。

Figure 9: Custom extension in Pi, asking the user if they want to run the sensors on startup, and showing sensors state in the status line.
图 9:Pi 系统中的自定义扩展功能。该功能会询问用户是否希望在系统启动时运行各传感器,同时还会在状态栏中显示各传感器的状态。
Adding a sensor 添加传感器
Every time I added a new sensor, I had to write a little parser that could get the results from whatever output a sensor provided (e.g., reading coverage-final.json, or the semgrep output). This became quite easy once I created a skill for it, but it does require changing the CLI's code every single time.
每次我添加新的传感器时,都不得不编写相应的解析程序,以便从传感器输出的各类数据中获取所需的信息(比如从 coverage-final.json 或 semgrep 输出的数据中获取信息)。不过,一旦我为这种操作创建了相应的脚本,整个过程就变得简单多了。不过,无论如何,每次添加新传感器时,都不得不修改 CLI 的代码。

Figure 10: A tool like this needs to know how to parse each sensor's outcomes
图 10:像这样的工具需要能够正确解析每个传感器的检测结果。
So I started moving to a default parser that expects a certain JSON schema as output from the schema. In this approach, instead of having the CLI run e.g. ruff directly, it would run python ./ruff_sensor.py -- check sensors/, where ruff_sensor.py would run ruff, and then turn its output into the JSON schema the CLI understands and can display.
于是,我开始改用那种默认的解析器来处理数据。这种解析器要求输出结果必须符合特定的 JSON 格式。在这种方式下,CLI 不会直接执行 ruff ,而是会先执行 python ./ruff_sensor.py -- check sensors/ 。 python ./ruff_sensor.py -- check sensors/ 又会调用 ruff 来处理数据,然后将处理后的结果转换为 CLI 能够理解并显示的 JSON 格式。
This is an example of how the decision-making for customisation of CLIs like this might change. In the past, building a plugin system for the most common sensor types would be an obvious decision. But now, for some things we can expect users to build a bit of that customisation for themselves, to keep the tool itself simpler.
这就是在定制此类用户界面时,决策方式可能发生的变化的典型例子。过去,为最常见的传感器类型开发相应的插件系统是理所当然的做法。但现在,对于某些功能,我们可以让用户自行进行定制,从而让工具本身更简单易用。
Performance 性能表现
Naturally, the more of these sensors we want to run, the more resources they are going to need. The difference in performance of build and dev tools across different tech stacks can be quite significant. Yet another thing that is good for agents as well as humans: fast and efficient build tools providing quick feedback loops are very important. I used this setup for a few Typescript codebases, as well as a Python application. And you might have guessed it, the Python setup (where many of the tools these days are “blazingly fast” Rust implementations) didn't give me any trouble at all. Running the sensors for a Typescript codebase on the other hand regularly started my M3's fan. But hold your Javascript prejudices - this was actually in part the result of my virus scanner kicking into overdrive, because those sensors were rewriting files all the time...
当然,我们想要运行的传感器越多,所需的资源也就越多。不同技术栈所对应的构建和开发工具在性能上可能存在很大差异。对智能体来说,同样重要的是:快速高效的构建工具以及能够及时提供反馈的机制。我曾在几个 TypeScript 代码库以及一个 Python 应用程序中使用了这种配置。不出所料,使用 Python 时没有任何问题——因为现在很多相关工具都是用 Rust 实现的,所以效率极高。而在处理 TypeScript 代码库时,M3 的散热风扇经常会因为传感器不断重写文件而开始运转。不过先别急着下结论——其实这部分是因为我的杀毒软件开始全力工作了……
Effectiveness 有效性
One of the frequently asked questions about harnesses and harness engineering is how to know if the setup is effective. In the context of sensors, this question is similar to that of “How effective are our CD pipelines?”. If they're always green, that's suspicious, because they never catch anything. If they're always red, that either means that we're really bad at producing code that is up to our standards, or that the pipeline is way too sensitive or flaky.
关于代码检测工具及相关技术的常见问题之一,就是如何判断其效果如何。在传感器的背景下,这个问题类似于“我们的代码检测流程到底有多有效?”如果检测结果一直为“绿色”,那就值得怀疑了,因为这意味着该检测工具根本无法发现任何问题。而如果检测结果一直为“红色”,那要么说明我们在编写符合标准的代码方面做得太差,要么就是该检测工具过于敏感或不可靠。
But even though this is an “it depends” question, we need data to answer it. So I started logging a history of sensor states every time they were checked, to get some data.
不过,虽然这是个“因情况而异”的问题,但我们还是需要数据来回答它。因此,我开始记录每次传感器被检测时的状态,以便收集相关数据。

Figure 11: History of sensor states
图 11:传感器状态的历史记录
This historical data could answer questions like these:
这些历史数据可以用来回答诸如以下这样的问题:
- Is there a trend in sensors failing less frequently? --> a signal that the guides or the models are improving
传感器的故障率是否有下降的趋势?这或许表明,相关的指导原则或模型正在得到改进。 - Which sensors are never failing? --> a signal that they are not necessary
哪些传感器永远不会出故障呢?→这说明这些传感器其实没有存在的必要。 - What sensors / rules are frequently failing? --> a signal that we could improve our guides there
哪些传感器/规则经常出现故障?→ 这表明我们可以在这些方面改进相关的指导说明。
Conclusions 结论
I have found this setup very convenient both as a head up display for myself in supervised coding sessions, and as a source of information for my agent.
我发现,这种设置非常实用:既可以作为我在接受指导时查看信息的显示屏,也可以作为我的智能体获取信息的来源。
The biggest gaps as of now are:
目前,最大的差距在于:
- If I wanted to effectively use the CLI in unsupervised sessions, I would have to invest more thinking into the harness integration, i.e. how to make sure that the agent actually calls the CLI before it wraps up its work.
如果我想要在无监督模式下有效使用 CLI,就必须花更多心思来处理其与其他系统的集成问题,也就是要确保代理在完成工作之前能够正确调用 CLI。 - I haven't spent effort yet on making the CLI available to a coding agent sandbox, probably by putting it into a registry. “Sandboxability” should be a prime concern for all tooling around coding harnesses, to reduce the number of inconveniences and excuses developers have for not using sandboxes.
我还没有努力将 CLI 集成到编程代理的沙箱环境中,或许可以将其放入某个注册表中来实现这一目标。对于所有与编程相关的工具来说,“能否在沙箱环境中使用”应该是一个重要的考虑因素,这样才能减少开发者不使用沙箱环境的种种借口和麻烦。
27 May 2026: published test suite as regression sensor and conclusion
2026 年 5 月 27 日:已发布用于回归测试的测试套件及测试结果分析。
20 May 2026: Published: dependency rules, coupling data, and AI modularity review
2026 年 5 月 20 日:已发布:依赖关系规则、数据耦合情况以及人工智能模块化方面的评估结果。
19 May 2026: Published basic linting
2026 年 5 月 19 日:发布了初步的代码检查结果。
© Martin Fowler | Disclosures
更多推荐



所有评论(0)