The reliable way to analyze or modernize RPG code is a deterministic parser that produces an abstract syntax tree plus symbol resolution. Anything that skips deterministic parsing, including LLM-only analysis, risks silent failures on the constructs that make RPG RPG: fixed-column semantics, implicit file operations, and DDS-linked field definitions. A production-grade rpg code parser gives your team a foundation you can build tooling on top of, not a best guess.
Once that foundation is in place, here’s what you should expect it to hand back:
- An abstract syntax tree (AST) representing every program’s structure
- A token stream for syntax highlighting and editor integration
- A symbol table linking variables, files, and DDS definitions
- A call graph mapping program-to-program and subroutine dependencies
- JSON export so downstream tools (diagram generators, dashboards, migration scripts) can consume the output without touching raw source
Golden Path Digital built AS/Forward specifically to deliver this output at enterprise scale, parsing real production RPG and RPGLE codebases rather than toy samples.
Key Takeaways
A reliable RPG code parser must combine deterministic AST generation with symbol resolution, since dependency maps and dead-code reports are only trustworthy once every reference actually resolves.
| Point | Details |
|---|---|
| Deterministic parsing beats LLM-only analysis | RPG’s column-sensitive and implicit constructs still trip up large language models, so a parser-first pipeline stays the reliable foundation. |
| Symbol resolution is the real bottleneck | Call graphs and dead-code reports are only as good as the resolver linking fields back to DDS definitions and external programs. |
| Match technology to your constraints | ANTLR suits expressive grammars, Tree-sitter suits editor integration, and custom lexers suit strict legacy column quirks. |
| Validate against your ugliest programs | Test parse success rate and symbol-resolution coverage on your oldest, messiest code, not a clean sample set. |
| Golden Path Digital offers a production option | AS/Forward delivers AST exports, dependency maps, and dead-code reports built for enterprise-scale RPG codebases. |
Table of Contents
- What Does an RPG Code Parser Actually Do?
- Which Parsing Technology Should You Build On?
- What Can You Actually Build on Top of a Parser?
- How Do You Choose and Validate an RPG Code Parser?
- Golden Path Digital’s Approach to Parser-Driven Modernization
- How Do Parsers Handle RPG Language Extensions and Dialects?
- How Should a Parser Handle Errors During Parsing?
- What Are the Leading RPG Parser Options?
- How Do You Test and Validate a Parser Before Trusting It?
- Why Deterministic Parsing Should Be Non-Negotiable
- Get Enterprise-Grade RPG Analysis Without Building a Parser In-House
- Sources
- FAQ
What Does an RPG Code Parser Actually Do?
A parser’s job sounds simple until you look at what RPG demands from it. The pipeline runs in stages: a lexer breaks raw source into tokens, a parser organizes those tokens into a tree structure following the language’s grammar, and a semantic analysis phase (sometimes called symbol resolution) links every reference back to its actual definition. Skip any stage and the output downstream tools receive is incomplete or wrong.
Here’s where RPG gets genuinely hard compared to most languages a parser generator was designed for:
- Fixed-format RPG is column-sensitive. A token’s meaning changes depending on which column it starts in and which spec type surrounds it. A generic lexer has no idea that column 6 means something different on a C-spec than an F-spec.
- Free-format RPG reads more like modern code but still coexists with fixed-format specs in the same source member, so a real-world rpg code parser has to handle both inside a single file, sometimes a single subprocedure.
- Symbol resolution has to reach outside the source file. RPG programs reference DDS-defined files, externally described data structures, and other programs. Without a resolver that follows those links, you cannot build an accurate call graph or answer the question “what breaks if I touch this field?”
RPG’s column-aware lexing problem is well documented in parser development circles. Fixed-format tokens change meaning by start column and surrounding spec, which means most implementations need an external scanner or dedicated lexer modes before a general-purpose grammar can even begin.
A parser’s outputs typically land in one of three formats. JSON is the default for anything feeding a script or a web dashboard. EMF (Eclipse Modeling Framework) shows up in tooling built on Eclipse-based IDEs. PlantUML or Mermaid-ready text is what you want when the goal is a diagram a human reviews rather than a machine consumes.
Why not just point an LLM at the source and ask it to explain the program? IBM has poured large volumes of RPG code into internal models, and the models still struggle with RPG-specific constructs like implicit MOVE operations and level-break logic. That’s not a knock on the models. It’s a structural limitation: an LLM pattern-matches against training data, while a deterministic parser applies the language’s actual grammar every time, on every file, with the same result. For dependency mapping and dead-code detection at production scale, that reliability difference is not optional.
Strumenta’s work on RPG parsing and symbol resolution makes the same point from the tooling side: without a resolver linking RPG source to DDS and external programs, you cannot generate a trustworthy call graph, full stop.
Which Parsing Technology Should You Build On?
Three broad approaches dominate RPG parser development, and each one trades something for something else.
ANTLR gives you a grammar-driven parser generator with real expressiveness. You write the grammar in ANTLR’s own notation, and it generates a lexer and parser in your target language. The tradeoff shows up immediately with fixed-format RPG’s column rules. ANTLR’s community discussions on parsing RPGLE walk through this directly. Because standard ANTLR lexing doesn’t natively track column position as a semantic signal, teams end up writing custom lexer modes or a pre-processing pass that normalizes column-sensitive code before the grammar ever sees it.
Tree-sitter takes a different bet. It’s built for incremental parsing, meaning it can re-parse just the changed portion of a file instead of the whole thing, which is exactly what an editor needs for real-time syntax highlighting. Community-maintained Tree-sitter grammars for RPG already exist, and projects like mayflower/rpg-explainer show a working pipeline: Tree-sitter parses source into a JSON program index, and that structured index feeds report and diagram generation while an LLM layer only handles the human-readable explanation on top. That’s the pattern worth copying. Deterministic parsing does the structural work; the LLM only narrates what the parser already extracted.
Custom lexer and parser pairs remain the right call when your codebase has legacy quirks no general grammar anticipates: nonstandard column usage from decades-old coding conventions, embedded SQL blocks, or shop-specific preprocessor macros. Building custom means owning every edge case yourself, indefinitely.
Most production pipelines end up hybrid: lexer, then parser, then a dedicated symbol resolver, then whatever tooling consumes the AST. That layered structure, sometimes called a Kolasu-style pipeline after the open-source language-engineering framework, keeps each stage testable in isolation. When the resolver breaks on a new DDS pattern, you fix the resolver without touching the lexer.
- Implementation language affects hiring and long-term maintenance more than raw performance
- A representative test corpus (hundreds of real programs, not a handful of samples) is what actually validates a parser
- CI coverage that re-runs the parser against your regression corpus on every change catches silent breakage before it reaches production
- Maintainability, not benchmark speed, is usually the deciding factor between build-your-own and adopting an existing parser
Pro Tip: Before committing to any parsing technology, run it against your ugliest 20 programs first, not your cleanest ones. The programs with embedded SQL, deeply nested subprocedures, and copy-book chains are where grammars quietly fall apart.
Long-term maintainability is consistently cited as the deciding factor when teams choose a commercial parser over an in-house build. The initial development cost is rarely the issue. It’s the years of grammar updates that follow.
What Can You Actually Build on Top of a Parser?
An AST by itself is inert. Its value shows up in what you build downstream, and the applications split into a few well-established categories.
- Diagrams generated straight from the AST. Tomassetti’s work on sequence diagram generation walks through converting AST nodes into PlantUML strings, turning a program’s call flow into a diagram a business analyst can actually read. Mermaid works the same way for simpler flowcharts and entity relationships.
- Dependency mapping and dead-code detection. This is where symbol resolution earns its cost. Once every reference resolves back to a real definition, you can trace which programs call which, which fields are actually read versus just declared, and which subroutines nothing calls anymore. That’s the backbone of any RPG code analysis engagement aimed at trimming a bloated codebase before a migration.
- Transpilation. Converting an RPG AST into a target-language AST (Java, for instance) sounds mechanical but runs into grammar gaps fast: RPG’s implicit file handling and level-break logic have no direct equivalent in most modern languages, so transpiled output usually needs a manual review pass around those constructs specifically.
- Editor tooling. Syntax highlighting, jump-to-definition, and refactoring aids all depend on the same token stream and symbol table a parser already produces. If your parser generates a symbol table, IDE integration is mostly plumbing from that point forward.
One caveat worth repeating: an LLM asked to explain a program is only as accurate as the input it receives. Feed it structured parser output, AST plus symbol table, and it can generate solid plain-language summaries. Feed it raw RPG source and ask it to reason about level breaks or indicator logic directly, and accuracy drops. The rpg-explainer pattern of parser-first, LLM-second is the one worth replicating, and it’s the same architecture behind good data flow mapping work.
How Do You Choose and Validate an RPG Code Parser?
Picking a parser is a decision you’ll live with for years, since ripping one out mid-migration is expensive. Run through this checklist before you commit.
Selection criteria:
- Does it handle fixed-format, free-format, and mixed-format source in the same file?
- Does it perform actual symbol resolution against DDS and external program references, or just tokenize?
- Has it been run against codebases in the millions-of-lines range, or only small samples?
- What’s the license, and does it fit your organization’s legal review process?
- Is there a real support channel, or does the maintainer disappear between releases?
- Can it slot into your CI pipeline to run automatically on every commit?
Integration checklist once you’ve picked one:
- Run a smoke parse across your full source tree and log every failure
- Spot-check the AST on a handful of programs you know well, by hand
- Generate a call graph for one subsystem and verify it against what your team already knows is true
- Produce one sample diagram or transpiled output and have a senior developer review it
Questions worth asking a vendor or an open-source maintainer directly: How does the tool handle mixed fixed and free format in one member? What happens when a DDS reference can’t be resolved, does it fail loudly or silently skip it? What’s the parse success rate on a corpus similar in size to yours?
Pro Tip: A parser that silently skips unresolved symbols instead of flagging them is a red flag you should not ignore. Silent skips look fine in a demo and cause missing dependencies in your dead-code report six months later.
Watch for parsers with no published test corpus, no CI integration story, or vague answers about symbol resolution coverage. Those gaps tend to surface exactly when you need the tool most.
Golden Path Digital’s Approach to Parser-Driven Modernization
Golden Path Digital built AS/Forward around one sequencing decision: dependency mapping and deterministic parsing come before any AI-assisted transformation touches the code. That order matters. Applying automation to a codebase you haven’t mapped is how modernization projects introduce the exact regressions they were meant to avoid.
AS/Forward is designed to hold up on enterprise-scale RPG codebases, where parse success rate and symbol-resolution coverage matter more than a clever demo. In practice, engagements produce:
- AST exports in JSON for downstream tooling
- Dependency maps linking programs, subroutines, and DDS-defined files
- Dead-code reports identifying subroutines and fields nothing references anymore
- A CI-integrated artifact-generation job so the analysis stays current as source changes
Teams typically bring these artifacts into developer review workflows before any phased migration begins, starting with an IBM i modernization assessment that establishes the baseline map.
How Do Parsers Handle RPG Language Extensions and Dialects?
RPG isn’t one language so much as a family of dialects layered over four decades. RPG III, RPG/400, and RPG IV each carry their own syntax quirks, and IBM keeps adding free-format extensions on top of RPG IV with every release. A parser built against a single dialect snapshot will choke the moment it meets an older program using RPG III-style calculation specs or a newer one using a syntax feature added after the parser’s grammar was last updated.
The practical fix is grammar versioning: a parser that tracks which RPG IV release introduced which syntax feature, and degrades gracefully (flagging an unrecognized construct rather than crashing) when it meets something outside its known grammar. Shop-specific extensions compound this. Some IBM i shops embed SQL directly in RPG source, others rely on copy-book conventions that predate free-format entirely. A parser evaluated only against clean, modern free-format samples will pass every demo and then fail on the 30-year-old core programs that actually run your business. Test against your oldest programs, not your newest ones, before trusting any dialect-handling claim.
How Should a Parser Handle Errors During Parsing?
A parser that halts on the first malformed line is close to useless against a production codebase, because production RPG always has at least a few programs with quirks no grammar fully anticipated. Error recovery is what separates a research-grade parser from one you can actually run against thousands of members overnight.
The standard technique is called panic-mode recovery: when the parser hits a token it can’t reconcile with the grammar, it skips forward to a recognizable synchronization point (the next spec boundary, the next statement terminator) and resumes, logging the skipped section rather than aborting the whole run. A well-built RPG syntax analyzer should report every recovery event with a file name and line number, so a developer can review exactly what got skipped instead of trusting a silent partial result.
The alternative, failing the entire file on one bad line, means one legacy quirk in a 5,000-line program blocks analysis of the other 4,999 lines around it. For dependency mapping across a codebase with decades of accumulated exceptions, graceful degradation with clear logging beats a strict all-or-nothing parse every time.
What Are the Leading RPG Parser Options?
The field splits cleanly into open-source community projects and commercial platforms, and the right pick depends on how much ongoing maintenance your team can absorb.
On the open-source side, the idk project offers rpgle-parser and dds-parser components with Python bindings, capable of handling mixed free and fixed formats and exporting ASTs. Tree-sitter-based grammars, including the pipeline behind rpg-explainer, give you incremental parsing well suited to editor plugins and produce JSON program indexes that other tools can consume. Both are genuinely useful, and both come with the standard open-source tradeoff: you’re responsible for keeping the grammar current as IBM ships new RPG IV syntax, and support depends on maintainer availability rather than a contract.
Commercial platforms, AS/Forward among them, exist specifically to remove that maintenance burden. The pitch isn’t that commercial parsing is technically superior to a well-maintained open-source grammar. It’s that long-term maintainability and dedicated support matter more once you’re running analysis against a production codebase on a schedule, not a research timeline. If your team has the bandwidth to own grammar updates indefinitely, open-source is a legitimate path. If that maintenance isn’t your team’s core job, a supported platform closes that gap.
How Do You Test and Validate a Parser Before Trusting It?
Validation has to happen before a parser touches anything you’d act on, not after. Start with parse success rate against a representative corpus, hundreds of real programs pulled from your own codebase, not a handful of clean samples chosen to make the demo look good. Anything below a high success rate on your actual source tells you the grammar has gaps you’ll hit again in production.
Next comes symbol-resolution coverage: what percentage of file and field references actually resolve back to a DDS definition or external program, versus falling through silently. This single metric determines whether a dependency map or dead-code report is trustworthy or decorative.
Finally, build regression tests into CI so every parser update runs against the same corpus and flags any AST that changed unexpectedly. Enterprise-scale RPG analysis depends on exactly these three metrics: parse success rate, symbol-resolution coverage, and CI-integrated regression testing, according to industry reporting on IBM i modernization tooling. Spot-checking a handful of ASTs by hand catches obvious errors; only a corpus-wide, repeatable test catches the ones that show up on program number 4,000.
Why Deterministic Parsing Should Be Non-Negotiable
The conventional advice floating around IBM i forums treats parser choice as a technical detail you settle once and move past. That’s backwards. Parser choice determines whether every downstream deliverable, your dependency map, your dead-code report, your migration plan, is built on solid ground or on guesswork wearing a nice diagram.
The overrated idea is that an LLM can shortcut this. It can’t, not yet, not for RPG’s column-sensitive quirks and implicit file logic. What’s underrated is symbol resolution specifically. Teams fixate on AST generation because it’s visible and demoable, while the resolver linking fields back to DDS definitions is the part that actually determines whether your call graph can be trusted.
If you take one thing from this: validate any parser against your own ugliest, oldest programs before you trust its output on anything that matters. A parser that handles clean free-format code beautifully and chokes on your 1998 core billing program hasn’t proven anything yet.
Get Enterprise-Grade RPG Analysis Without Building a Parser In-House
Building and maintaining your own RPG parser means owning grammar updates, symbol-resolution edge cases, and regression testing indefinitely, on top of whatever modernization work you actually set out to do. AS/Forward gives you that parsing and symbol-resolution layer already built and already validated against large production RPG codebases, so your team’s time goes toward the modernization decisions themselves instead of grammar maintenance.
Golden Path Digital’s approach starts with dependency mapping before any transformation touches your source, producing AST exports, dependency maps, and dead-code reports your developers can review before committing to a migration path. That sequencing is the difference between a modernization project that surfaces risk early and one that discovers it in production. If your team is evaluating a path off RPG or simply needs a clear map of what a legacy codebase actually depends on, start with a legacy code modernization engagement and get a concrete picture of your codebase before you plan the next step.
Sources
- Getting a parser: build it, use an open-source one, or a commercial one? – Tomassetti
- mayflower/rpg-explainer
FAQ
What Is a Parser in Programming?
A parser is a program that takes a stream of tokens (produced by a lexer) and organizes them into a structured tree, typically an abstract syntax tree, that reflects the source code’s grammar and can be analyzed or transformed programmatically.
What Is RPG as a Programming Language?
RPG (Report Program Generator) is a programming language built for IBM i systems, originally designed around fixed-column specification lines and now supporting a free-format style closer to modern languages while retaining decades of legacy syntax conventions.
Does RPG Maker Use the Same RPG Language?
No. RPG Maker is unrelated game-development software that uses its own scripting language (commonly Ruby-based in older versions), while IBM i’s RPG is a business-application language with no connection to the game engine despite the shared acronym.
How Do You Build a Parser for a Language Like RPG?
Building a parser generally means writing or generating a lexer to tokenize source, defining a grammar (via ANTLR, Tree-sitter, or a custom implementation) that matches the language’s structure, and adding a symbol-resolution layer that links references to their definitions, ideally validated against a large real-world code corpus like the pipeline behind AS/Forward.
Can an LLM Replace a Deterministic RPG Parser?
Not reliably. IBM’s own internal work shows large language models still struggle with RPG-specific constructs, which means LLMs work best as an explanation layer sitting on top of structured parser output rather than as a replacement for deterministic parsing.