What tirekick checks, and why
tirekick started with a production incident. An analytics MCP server shipped a tool schema containing a regex that was perfectly valid ECMA-262, an unescaped [ inside a character class, which stricter regex engines reject. The API-side validator refused the entire tool list, every request in the session failed, and Claude Code sessions were bricked until the server was unloaded. The linter came first, to make sure that never happened again. The directory came after, because the same twenty-second check is worth running against every public server before anyone installs it.
The checks
Every finding has a severity and a category. Severities: error means a strict client can reject the server outright, warn means agents will use the server badly, and info is worth knowing but never moves the grade. Categories: interop, agent ergonomics, and safety. New checks always ship as info first, so a new rule can be observed against the whole directory before it is allowed to move anyone's grade.
Schema conformance (draft 2020-12) — interop
Every tool's inputSchema and outputSchema is compiled with Ajv against the JSON Schema draft 2020-12 meta-schema. This is the same bar the Claude API applies to input_schema, so a failure here means real clients can reject the tool. Input schema roots must also be type object. Violations are errors.
Strict-regex safety — interop
Every pattern in a schema position is linted for constructs that pass in JavaScript but fail in RE2/Rust-style engines: lookaround assertions, backreferences, and the unescaped [ inside a character class that caused the original incident. Pattern strings inside examples, default, const, or enum values are data, not regexes, and are not linted. Because one bad pattern can get a whole tool list rejected, these are errors too.
Description quality — agent ergonomics
Agents choose tools by reading descriptions. A missing description or one under 20 characters is a warning; 20 to 39 characters is informational (present, but worth expanding — say what the tool does and when to pick it). A tool with no input schema at all is also a warning: the server will run, but agents will use it badly.
Tool-count bloat — agent ergonomics
Servers exposing more than 60 tools get an informational finding. Every tool definition is loaded into the agent's context before the first message, so oversized toolsets cost real tokens and measurably degrade tool selection. Informational for now: it never moves the grade.
How grades work
Findings are deduplicated before they are counted: one broken pattern reused across a tool's schema is one problem, not five, so counting is per distinct (tool, rule) pair. Each category is graded on its own, and the overall grade is the worst category grade. The exact curve:
- Any error caps a category at D.
- Errors touching 25% or more of a server's tools make it an F.
- Zero errors and zero warnings is an A.
- With no errors, warnings up to max(2, 20% of tool count) is a B; more than that is a C.
- Info findings never move a grade.
Servers that require credentials before they list tools aren't graded; the directory marks them as auth-gated instead. Every report carries the validator version that produced it, so a grade is always reproducible against the open-source validator.
Run it yourself
The validator is open source (MIT, tirekick on npm) and runs locally against remote URLs, npm packages, or a saved tools/list response:
npx tirekick check https://example.com/mcpIt exits non-zero when errors are found, so the same command works as a CI check. Or paste a URL into the checker and read the report in the browser.