🪙 Token::Resolver

if ci_badges.map(&:color).detect { it != "green"} ☝️ let me know, as I may have missed the discord notification.

if ci_badges.map(&:color).all? { it == "green"} 👇️ send money so I can do more of this. FLOSS maintenance is now my full-time job.

👣 How will this project approach the September 2025 hostile takeover of RubyGems? 🚑️

I've summarized my thoughts in [this blog post](https://dev.to/galtzo/hostile-takeover-of-rubygems-my-thoughts-5hlo).

🌻 Synopsis

Token::Resolver is a configurable PEG-based token parser and resolver for structured token detection and replacement in arbitrary text.

Detects structured tokens like {KJ|GEM_NAME} in any file format and resolves them against a replacement map. The token structure (delimiters, separators, segment count) is fully configurable.

# One-liner: parse and resolve
result = Token::Resolver.resolve(
  "Hello {KJ|NAME}, welcome to {KJ|PROJECT}!",
  {"KJ|NAME" => "World", "KJ|PROJECT" => "my-app"},
)
# => "Hello World, welcome to my-app!"

💡 Info you can shake a stick at

Tokens to Remember
Works with JRuby
Works with Truffle Ruby
Works with MRI Ruby 4
Works with MRI Ruby 3
Support & Community
Source
Documentation
Compliance
Style
Maintainer 🎖️
`...` 💖	🧊 🐙 🛖 🧪

Compatibility

Compatible with MRI Ruby 3.2.0+, and concordant releases of JRuby, and TruffleRuby.

🚚 Amazing test matrix was brought to you by	🔎 appraisal2 🔎 and the color 💚 green 💚
👟 Check it out!	✨ github.com/appraisal-rb/appraisal2 ✨

Federated DVCS

Find this repo on federated forges (Coming soon!)

Federated DVCS Repository	Status	Issues	PRs	Wiki	CI	Discussions
🧪 kettle-rb/token-resolver on GitLab	The Truth	💚	💚	💚	🐭 Tiny Matrix	➖
🧊 kettle-rb/token-resolver on CodeBerg	An Ethical Mirror (Donate)	💚	💚	➖	⭕️ No Matrix	➖
🐙 kettle-rb/token-resolver on GitHub	Another Mirror	💚	💚	💚	💯 Full Matrix	💚
🎮️ Discord Server		Let’s	talk	about	this	library!

Enterprise Support

Available as part of the Tidelift Subscription.

Need enterprise-level guarantees?

The maintainers of this and thousands of other packages are working with Tidelift to deliver commercial support and maintenance for the open source packages you use to build your applications. Save time, reduce risk, and improve code health, while paying the maintainers of the exact packages you use.

💡Subscribe for support guarantees covering all your FLOSS dependencies
💡Tidelift is part of Sonar
💡Tidelift pays maintainers to maintain the software you depend on!
📊@Pointy Haired Boss: An enterprise support subscription is “never gonna let you down”, and supports open source maintainers

Alternatively:

✨ Installation

Install the gem and add to the application’s Gemfile by executing:

bundle add token-resolver

If bundler is not being used to manage dependencies, install the gem by executing:

gem install token-resolver

🔒 Secure Installation

For Medium or High Security Installations

This gem is cryptographically signed, and has verifiable SHA-256 and SHA-512 checksums by stone_checksums. Be sure the gem you install hasn’t been tampered with by following the instructions below.

Add my public key (if you haven’t already, expires 2045-04-29) as a trusted certificate:

gem cert --add <(curl -Ls https://raw.github.com/galtzo-floss/certs/main/pboling.pem)

You only need to do that once. Then proceed to install with:

gem install token-resolver -P HighSecurity

The HighSecurity trust profile will verify signed gems, and not allow the installation of unsigned dependencies.

If you want to up your security game full-time:

bundle config set --global trust-policy MediumSecurity

MediumSecurity instead of HighSecurity is necessary if not all the gems you use are signed.

NOTE: Be prepared to track down certs for signed gems and add them the same way you added mine.

⚙️ Configuration

Token Config Options

Option	Default	Description
`pre`	`"{"`	Opening delimiter
`post`	`"}"`	Closing delimiter
`separators`	`["\\|"]`	Segment separators (sequential; last repeats)
`min_segments`	`2`	Minimum segments for a valid token
`max_segments`	`nil`	Maximum segments (`nil` = unlimited)

🔧 Basic Usage

Basic Token Resolution

require "token/resolver"

# Parse a document to inspect tokens
doc = Token::Resolver.parse("Deploy {KJ|GEM_NAME} to {KJ|GH_ORG}")
doc.token_keys  # => ["KJ|GEM_NAME", "KJ|GH_ORG"]
doc.text_only?  # => false

# Resolve tokens
result = Token::Resolver.resolve(
  "Deploy {KJ|GEM_NAME} to {KJ|GH_ORG}",
  {"KJ|GEM_NAME" => "my-gem", "KJ|GH_ORG" => "my-org"},
)
# => "Deploy my-gem to my-org"

Handling Missing Tokens

# Default: raise on unresolved tokens
Token::Resolver.resolve("{KJ|MISSING}", {})
# => raises Token::Resolver::UnresolvedTokenError

# Keep unresolved tokens as-is
Token::Resolver.resolve("{KJ|MISSING}", {}, on_missing: :keep)
# => "{KJ|MISSING}"

# Remove unresolved tokens
Token::Resolver.resolve("{KJ|MISSING}", {}, on_missing: :remove)
# => ""

Custom Token Structure

# Tokens like <<SECTION:NAME>>
config = Token::Resolver::Config.new(
  pre: "<<",
  post: ">>",
  separators: [":"],
)
Token::Resolver.resolve("Hello <<NS:NAME>>!", {"NS:NAME" => "World"}, config: config)
# => "Hello World!"

Multi-Segment Tokens with Sequential Separators

# Tokens like {KJ|SECTION:SUBSECTION}
config = Token::Resolver::Config.new(
  separators: ["|", ":"],  # First boundary uses |, second uses :, rest repeat :
)
doc = Token::Resolver.parse("{KJ|META:AUTHOR}", config: config)
doc.tokens.first.key       # => "KJ|META:AUTHOR"
doc.tokens.first.prefix    # => "KJ"
doc.tokens.first.segments  # => ["KJ", "META", "AUTHOR"]

Step-by-Step API

# Parse
doc = Token::Resolver::Document.new("Hello {KJ|NAME}!")

# Inspect
doc.nodes       # => [Text("Hello "), Token(["KJ", "NAME"]), Text("!")]
doc.tokens      # => [Token(["KJ", "NAME"])]
doc.token_keys  # => ["KJ|NAME"]
doc.to_s        # => "Hello {KJ|NAME}!"  (roundtrip fidelity)

# Resolve
resolver = Token::Resolver::Resolve.new(on_missing: :raise)
result = resolver.resolve(doc, {"KJ|NAME" => "World"})
# => "Hello World!"

Design

Grammar Never Fails

The parslet grammar is designed so that any input is valid. When the parser encounters
{ but it doesn’t start a valid token, the { is consumed as plain text. No input can
cause a parse failure.

Single-Pass Resolution

Replacement values are emitted as-is and are not re-scanned for tokens. This prevents
infinite loops and ensures predictable behavior when replacement values contain token-like strings.

Performance

If the input doesn’t contain the pre delimiter at all, the parser fast-paths and returns
a single Text node without invoking parslet.

🦷 FLOSS Funding

While kettle-rb tools are free software and will always be, the project would benefit immensely from some funding.
Raising a monthly budget of… “dollars” would make the project more sustainable.

We welcome both individual and corporate sponsors! We also offer a
wide array of funding channels to account for your preferences
(although currently Open Collective is our preferred funding platform).

If you’re working in a company that’s making significant use of kettle-rb tools we’d
appreciate it if you suggest to your company to become a kettle-rb sponsor.

You can support the development of kettle-rb tools via
GitHub Sponsors,
Liberapay,
PayPal,
Open Collective
and Tidelift.

📍 NOTE
If doing a sponsorship in the form of donation is problematic for your company from an accounting standpoint, we’d recommend the use of Tidelift, where you can get a support-like subscription instead.

Open Collective for Individuals

Support us with a monthly donation and help us continue our activities. [Become a backer]

NOTE: kettle-readme-backers updates this list every day, automatically.

No backers yet. Be the first!

Open Collective for Organizations

Become a sponsor and get your logo on our README on GitHub with a link to your site. [Become a sponsor]

NOTE: kettle-readme-backers updates this list every day, automatically.

No sponsors yet. Be the first!

Another way to support open-source

I’m driven by a passion to foster a thriving open-source community – a space where people can tackle complex problems, no matter how small. Revitalizing libraries that have fallen into disrepair, and building new libraries focused on solving real-world challenges, are my passions. I was recently affected by layoffs, and the tech jobs market is unwelcoming. I’m reaching out here because your support would significantly aid my efforts to provide for my family, and my farm (11 🐔 chickens, 2 🐶 dogs, 3 🐰 rabbits, 8 🐈‍ cats).

If you work at a company that uses my work, please encourage them to support me as a corporate sponsor. My work on gems you use might show up in bundle fund.

I’m developing a new library, floss_funding, designed to empower open-source developers like myself to get paid for the work we do, in a sustainable way. Please give it a look.

Floss-Funding.dev: 👉️ No network calls. 👉️ No tracking. 👉️ No oversight. 👉️ Minimal crypto hashing. 💡 Easily disabled nags

🔐 Security

See {file:SECURITY.md SECURITY.md}.

🤝 Contributing

If you need some ideas of where to help, you could work on adding more code coverage,
or if it is already 💯 (see below) check {file:REEK reek}, issues, or PRs,
or use the gem and think about how it could be better.

We so if you make changes, remember to update it.

See {file:CONTRIBUTING.md CONTRIBUTING.md} for more detailed instructions.

🚀 Release Instructions

See {file:CONTRIBUTING.md CONTRIBUTING.md}.

Code Coverage

🪇 Code of Conduct

Everyone interacting with this project’s codebases, issue trackers,
chat rooms and mailing lists agrees to follow the {file:CODE_OF_CONDUCT.md }.

🌈 Contributors

Made with contributors-img.

Also see GitLab Contributors: https://gitlab.com/kettle-rb/token-resolver/-/graphs/main

⭐️ Star History

📌 Versioning

This Library adheres to .
Violations of this scheme should be reported as bugs.
Specifically, if a minor or patch version is released that breaks backward compatibility,
a new version should be immediately released that restores compatibility.
Breaking changes to the public API will only be introduced with new major versions.

dropping support for a platform is both obviously and objectively a breaking change

—Jordan Harband (@ljharb, maintainer of SemVer) in SemVer issue 716

I understand that policy doesn’t work universally (“exceptions to every rule!”),
but it is the policy here.
As such, in many cases it is good to specify a dependency on this library using
the Pessimistic Version Constraint with two digits of precision.

For example:

spec.add_dependency("token-resolver", "~> 1.0")

📌 Is "Platform Support" part of the public API? More details inside.

SemVer should, IMO, but doesn’t explicitly, say that dropping support for specific Platforms is a breaking change to an API, and for that reason the bike shedding is endless.

To get a better understanding of how SemVer is intended to work over a project’s lifetime, read this article from the creator of SemVer:

“Major Version Numbers are Not Sacred”

See {file:CHANGELOG.md CHANGELOG.md} for a list of releases.

📄 License

The gem is available as open source under the terms of
the {file:LICENSE.txt MIT License} .
See {file:LICENSE.txt LICENSE.txt} for the official Copyright Notice.

© Copyright

🤑 A request for help

Maintainers have teeth and need to pay their dentists.
After getting laid off in an RIF in March, and encountering difficulty finding a new one,
I began spending most of my time building open source tools.
I’m hoping to be able to pay for my kids’ health insurance this month,
so if you value the work I am doing, I need your support.
Please consider sponsoring me or the project.

To join the community or get help 👇️ Join the Discord.

To say “thanks!” ☝️ Join the Discord or 👇️ send money.

💌 💌 💌

Please give the project a star ⭐ ♥.

Thanks for RTFM. ☺️