Until fairly recently, the only place to encounter a diff was in pull request as IDEs would rarely show colored or encoded diffs since you are still in composition mode. AI Coding tools like Cursor have put really diffs on the center stage. When developing in agent mode, diffs are commonly used to visualize what’s being changed. State-of-the-art models were trained on vast volumes of code so they recognize diffs and diff-related signatures like @@, +/- except they are not necessarily able to deduce what the impact of the diff would be on a given code snippet.
In this post, we walk through the pros and cons of working with various types of diffs, and why textual diffs lead to bad AI coding suggestions.
Text-diffing for fast single file comparison
Text-diffing is simple: a line-by-line comparison of two text files to identify code that’s been added, removed, and modified. Although the foundational diffing algorithm is fifty years old (introduced with Unix in the mid-70s), this type of diffing is still the standard, implemented (and powers Git Diffing) in Github, Gitlab, Bitbucket, and virtually every mainstream version control system and code review tool.
It is excellent for very lightweight change diffing that doesn't require version control. But beyond surface-level analysis comparing lines of text, there is no history or context.
Unsurprisingly, we’ve found this is the most AI-friendly format of a diff, possibly due to the availability of raw training data on the internet. Your recommendations in Cursor are usually a text-diff.
Git-based diffing for version control code bases
Git-based diffing uses text-based diffing under the hood, except it applies it in the context of commits, branches, and repositories. Git diff compares code changes between different commits, branches, or working directories. It compares the content of files line by line, marking additions, deletions, and modifications.
Git diff is an essential tool for code change analysis, providing targeted and context-aware insights into code modifications. But it does have limitations, especially as your code base becomes more complex across multiple languages.
Git diffs are usually what confuses ChatGPT or Claude when you input things like merge conflicts because they lack the code structure, semantics, and intent used to compose them and diff them in your versioning process. Git diff treats renamed functions, refactored code, and moved files as separate changes rather than related modifications, making it difficult to merge without losing context. They also fail to correlate changes across multiple files, meaning you must manually track dependencies and reconstruct intent.
One more major limitation with Git diff is that it does not indicate why a change was made, its impact on dependencies, or whether it introduces breaking changes. We weren’t really expecting it to, but this lack of context-awareness has an impact on everything you and your team care about like test coverage, code hygiene, conventions, type consistency and more.
In essence, traditional diffing tools make humans responsible for deciphering intent and dependencies, which is why you probably need all that error-handling code to figure out when things stopped working..
AST diffing for multi-repo, complex code bases
Whether we like it or not, the use of common diffing techniques has a direct impact on the quality of code generated by AI models. One way to better prescribe code changes is by using a diffing technique that more accurately depicts how code is really beyond the characters used to compose it. This is where AST-based diffing shows huge promise.
Where Git diffing focuses on the specified versions of files, AST diffing takes a different approach. Each node in the Abstract Syntax Tree (AST) represents a syntax element (such as a function, variable, or expression), while the edges capture the relationships and structure between those elements.
AST diffing identifies where nodes have been added, removed, or modified between the two versions of code, enabling far more precise change detection. Unchanged nodes are preserved, modifications are finely tracked, and only meaningful changes are highlighted.
Key Benefits of AST Diffing
By mapping out changes according to the underlying relationships between syntax elements, AST diffing gives a clearer picture of their real-world impact:
- Accurate detection of critical changes: Tracks changes at a deeper level, helping to spot breaking changes and subtle refactors.
- Precise identification of modifications: Focuses on changed syntax nodes, eliminating noise from non-essential edits.
- Improved impact analysis: Maps changes to real-world impacts like functionality and readability, rather than superficial formatting tweaks.
- Context-rich reviews: Provides a better understanding of the relationship between code elements, enhancing the reviewer's ability to interpret changes.
- Minimized false positives: By comparing actual syntax changes, AST diffing reduces the chances of incorrectly flagging irrelevant modifications.
Here’s a deeper dive overview of how this works sequentially in action from detecting vertices to impact analysis.
How Baz Leverages AST Diffing for Better Code Reviews
The Baz AI code reviewer leverages Difftastic, which parses code using Tree-sitter. By leveraging AST diffing, Baz can:
- Accurately detect breaking changes, ensuring critical modifications don’t slip through the cracks.
- Detect structural modifications with minimal false positives, giving reviewers confidence in their analysis.
- Supports multiple languages easily across complex code repositories
This aligns with the same approach we use for structured file prioritization. Both methods achieve an improved understanding of meaningful changes in code, enabling reviewers to prioritize what really matters. AST diffing makes code reviews more accurate and saves time.
This is the standard foundation for all code reviews in Baz with the option to switch to Git-based diffing when needed. Check out the docs here and get a fully featured free trial today.
.png)
.png)
