Lint a technote#

Documenteer provides a command-line linter, documenteer technote lint, that checks a technote’s metadata and structure before it is built and published. Its most important job is verifying that every author has an internal_id that resolves in the Rubin author database (authordb.yaml), since these IDs are needed to mint a DOI for the technote. The command also checks that the content declares an abstract, that requirements.txt installs Documenteer correctly, and that the technote’s citation metadata — its DOI, the metadata registered for that DOI with DataCite, and, where the repository has adopted one, its CITATION.cff — is in order.

Rubin’s technote CI (from the rubin-sphinx-technote-workflows repository) runs this command so that builds fail early when a technote’s metadata is incomplete.

Run the linter#

Run the linter from the root of a technote repository:

documenteer technote lint

If your technote uses the standard Makefile (see Migrate legacy reStructuredText/Sphinx technical notes), the linter also runs as part of the repository’s combined lint target, alongside the Pre-commit hooks and the link checker:

make lint

The command prints each issue it finds, prefixed with its stable rule code (for example, [R101]), followed by a summary and a link to the documentation page for every rule that fired:

[R101] Author Yusra AlSayyad is missing an internal_id. Did you mean 'alsayyady' (matched by ORCID)? Run 'documenteer technote sync-authors' to add it.
Found 1 error(s) and 0 warning(s).
Learn more:
  R101: https://documenteer.lsst.io/technotes/lint/r101.html

The command exits with a non-zero status when any error-level issue remains, which is what causes a CI build to fail. Warning-level issues are reported but do not fail the command unless you pass --strict.

Options#

--dir <path>, -d <path>: Path to the technote directory to lint. Defaults to the current directory (.). The command locates technote.toml, the content file (index.rst, index.md, or index.ipynb), requirements.txt, and CITATION.cff within this directory.
--strict, -s: Promote all warnings to errors. With this flag, any finding — including the warning-level rules below — causes the command to exit non-zero.
--ignore <code>: Switch off the rule with this code for this run, in addition to whatever technote.toml already ignores. Repeat the option to ignore several rules (--ignore TN105 --ignore TN106). See Ignoring a rule.

Rules#

Each rule has a stable code so that findings are easy to triage, and so that a technote can switch one off by name, and a landing page describing the rule, showing a failing technote, and walking through the fix. A code’s prefix names the rule set it belongs to, and the number that follows names the concern. TN rules check what any technote needs — its technote.toml, its content file, its abstract — while R rules check Rubin’s conventions and services, such as the Rubin author database and Rubin’s technote packaging. The linter runs both sets together, and a finding names its set so that a code never has to change when the rule that carries it moves: the TN rules are expected to move into the technote package, and the R rules to stay here. Within either prefix the hundreds mean the same thing: 0xx for structure and configuration, 1xx for metadata, and 2xx for content.

Technote lint rules#
Code	Group	Rule	Default severity
TN001	Structural	`technote.toml` conforms to the technote schema.	Error
R002	Structural	`requirements.txt` declares `documenteer` with the `[technote]` extra.	Warning
R003	Structural	`requirements.txt` does not pin Sphinx as a separate requirement.	Warning
TN004	Structural	`technote.toml` exists in the technote directory.	Error
TN005	Structural	`technote.toml` is syntactically valid TOML.	Error
TN006	Structural	The technote has a content file (`index.rst`, `index.md`, or `index.ipynb`).	Error
TN007	Structural	The `[technote.lint]` configuration names rules the linter has.	Warning
R101	Metadata	Every author declares an `internal_id`.	Error
R102	Metadata	Each author’s `internal_id` resolves in the Rubin author database.	Error
R103	Metadata	The author database is reachable so that IDs can be resolved.	Warning
TN105	Metadata	The metadata registered with DataCite for the DOI matches the technote.	Warning
TN106	Metadata	`CITATION.cff` matches what `technote.toml` generates.	Error
TN201	Content	The content file declares an abstract directive.	Error
TN202	Content	The abstract uses the abstract directive rather than a section heading.	Error
TN203	Content	Sphinx can read the technote’s document.	Error
TN204	Content	The abstract directive has body content.	Error

Ignoring a rule#

Occasionally a rule fires on a technote that cannot fix it, and will never be able to. A technote whose DOI was minted by some other service is the standing example: TN105 correctly reports that the registered title and creators do not match technote.toml, and none of its fixes apply, because only the owner of that deposit can edit the record (see When the registered record cannot be changed). Left alone, the warning is permanent and --strict is out of reach forever.

List such a rule’s code in the [technote.lint] table of the technote’s technote.toml:

technote.toml#

[technote.lint]
# Zenodo DOI minted in 2016; a Rubin release cannot update its metadata.
ignore = ["TN105"]

An ignored rule does not run at all. TN105 makes no DataCite request when it is ignored, and R101–R103 do not query the author database, so switching a network rule off makes the run faster as well as quieter.

The linter reports what it skipped, so that a rule that is off never looks like a rule that passed:

✅ Technote lint passed with no issues.
Ignored 1 rule: TN105 (technote.toml [technote.lint]).

Ignored rules are counted apart from errors and warnings and never affect the exit code. A technote whose only finding is ignored passes --strict, which is the point: an unfixable warning should not keep a repository from making every other warning fatal.

Write a comment above each entry saying why the rule is off. The next person to read the file cannot otherwise tell a finding that is unfixable from one that is merely unfixed, and an ignore that outlives its reason quietly stops checking something worth checking.

You can also ignore a rule for a single run, without editing the file, which is useful for trying --strict out:

documenteer technote lint --ignore TN105 --strict

The option is repeatable and adds to the file’s list rather than replacing it. In the file, a code is written exactly as the rule table lists it — an uppercase prefix and a number, such as TN105 — because the technote package validates the [technote.lint] table’s shape when it reads technote.toml; on the command line the case does not matter.

A code that no rule carries — a typo, or a rule that has been retired — is reported as TN007 rather than silently doing nothing. The valid entries around it still apply. An ignore written in the wrong shape — a bare string rather than an array, an entry that is not a string, or a lowercase code — fails technote’s schema and is reported as TN001; nothing is read from a table technote rejects, so the rule the technote meant to switch off stays on until the file is fixed.

Note

The [technote.lint] table is named for the thing it configures rather than for the tool that reads it. Today that tool is Documenteer’s documenteer technote lint; the technote package owns the table’s schema (since technote 0.11.0), and the linter itself is expected to move there too.

What a rule set cannot change is its codes, which repositories and CI configurations refer to; a rule set that documents its rules somewhere other than documenteer.lsst.io links its findings there instead.

Running the linter offline#

Most rules read files and the technote’s own Sphinx build alone; two kinds of check leave the machine. TN105 asks DataCite what a technote’s DOI is registered as, and the author checks (R101–R103) resolve author IDs against the Rubin author database.

The two behave differently when the network is not there, and deliberately so:

An unreachable author database is reported as R103, a warning, because an unresolved internal_id is the thing that blocks a technote’s DOI from being minted — silence would hide it.
An unreachable DataCite is silent. TN105 reports nothing at all, and neither does an unregistered DOI, so this check reads the same offline as it does with the network up.

The Sphinx read the content and title rules use (The linter builds the technote) is offline too: it switches off the bibliography cache and the intersphinx inventories that a real technote build fetches, since neither changes what the document says.

Suggested author IDs#

An author internal_id that is missing (R101) or unknown (R102) is a dead end on its own, so the linter tries to name the ID you probably want. It queries the author database twice at most and appends a suggestion when one entry matches confidently:

If the author entry declares an orcid, the linter asks the author database which author holds exactly that ORCID.
Otherwise — or when nobody holds it — the linter searches the database for the author’s name.

[R101] Author Yusra AlSayyad is missing an internal_id. Did you mean 'alsayyady' (matched by ORCID)? Run 'documenteer technote sync-authors' to add it.
[R102] Author Lynne Jones has internal_id 'lynnej', which is not in the author database. Did you mean 'jonesrl' (R. Lynne Jones, matched by name)?

The message states what the match is based on:

matched by ORCID — the author database holds the author with the orcid in your technote.toml author entry. ORCID is globally unique and author-supplied, so this is the strongest evidence there is: the lookup is exact, and it succeeds however differently your technote and the author database spell the name.
matched by name — a single near-exact name match, whose ORCID (if it has one) does not contradict yours.

Declaring an orcid for an author is therefore the surest way to get a usable suggestion, and the only one that works when the author database spells the name differently than you do. It also changes what an R101 finding asks of you: an ORCID match is one documenteer technote sync-authors can act on by itself — it resolves the same ORCID and writes the internal_id in for you — so the message reads Run ‘documenteer technote sync-authors’ to add it. A name match is only a suggestion for you to verify and type in, so its message reads … after adding it.

The suggestion is deliberately conservative and best-effort: an ambiguous search (several equally good matches, as a common family name gives) or a match contradicted by a differing ORCID adds nothing to the message, and a failed lookup leaves the finding exactly as it would otherwise read. A suggestion never changes whether the command passes or fails — verify it before you use it.

How the abstract is found#

The abstract rules (TN2xx) read the document Sphinx builds — see The linter builds the technote — rather than scanning the source themselves, so what they report is what the published page carries:

Both the reStructuredText directive (.. abstract::) and the MyST fenced directive (a triple-backtick or ::: fence opening with {abstract}) are recognized, in any letter case — docutils lowercases directive names, so a {Abstract} fence builds and passes.
In reStructuredText, text on the marker line itself (.. abstract:: The abstract.) is directive content and passes.
Directive options (for example :class: dropdown) are not content the directive publishes. The abstract directive declares no options, so docutils parses an option line as a field list, and a directive holding only options publishes nothing and is empty (TN204).
An abstract factored into another file and pulled in with .. include:: or a MyST {include} fence is found, wherever that file lives and however deeply the includes nest: Sphinx resolves the includes, and the abstract is in the document either way. An include Sphinx cannot resolve simply contributes nothing, and Sphinx reports it when the technote is built.
A finding’s file:line: prefix comes from the parsed document, so it names the file the markup is written in — the included file, where the abstract was factored out. Where the document records no line, the finding names the file alone. An index.ipynb notebook is always located by file alone: MyST-NB numbers a notebook’s lines per cell, so a line number there corresponds to nothing you could find in the file.

The linter builds the technote#

The rules about what a technote publishes — its abstract (TN2xx), and the title that TN105 and TN106 compare — get their answer by building the technote with Sphinx, once per run, using the dummy builder that reads the document and writes nothing. That is the technote’s own build: its conf.py, its extensions, its markup. A technote is titled by its document’s top-level heading unless technote.toml declares a title, and documenteer technote migrate never writes one, so reading the document is the only way to know what the technote is called.

Two things follow:

documenteer technote lint needs the technote extra: install documenteer[technote], which a technote’s requirements.txt declares anyway (R002).
A technote Sphinx cannot read — a missing index file, a notebook that is not valid JSON, a conf.py that raises — is reported as TN203 carrying Sphinx’s own message, rather than as a missing abstract.

The read is quiet, writes its output to a temporary directory that is removed afterwards, and forces the technote’s network-using extensions off, so it neither leaves anything behind in the repository nor breaks the linter’s offline behavior.

Non-Sphinx technotes#

Some technote-series repositories are not Sphinx projects at all: they publish through the shared technote CI with a custom build command (an Org-mode deck, for example). When a directory has no content file (index.rst, index.md, or index.ipynb) and no conf.py, the linter treats it as one of these and checks only that its technote.toml is well formed — TN004, TN005, TN001, and the metadata rules (TN1xx and R1xx), which read technote.toml alone. The requirements.txt rules (R002/R003), the content rules (TN2xx), and TN006 are skipped, since they describe a Documenteer/Sphinx build that these repositories do not have. A repository that does have a conf.py but no content file is a broken Sphinx technote, and is reported as TN006.