Overview of the pipelines.lsst.io build system#
This page is a high-level overview of the tools and processes for building the pipelines.lsst.io documentation.
To learn how to build the pipelines.lsst.io site in practice, see Building the pipelines.lsst.io site locally. For more background on contributing to pipelines.lsst.io documentation, see Overview of the Stack documentation system.
Documenteer is the Python package that provides tooling for LSST DM’s Sphinx-based documentation projects, including Stack user guides such as pipelines.lsst.io. Documenteer plays four roles:
Documenteer manages and versions the Python dependencies needed to build Sphinx documentation. The
requirements.txtfile in the pipelines_lsst_io repository refers to a specific release of Documenteer, which in turn specifies its own Python dependencies.
Documenteer provides Sphinx extensions built specifically for LSST DM needs.
Documenteer provides the stack-docs command-line app that runs the Sphinx documentation build process (Documenteer also provides the related package-docs app for developer:single-package documentation builds).
Stack build and set up#
The prerequisite for the Stack documentation build is that the Stack is already built and set up. Sphinx imports Python packages to generate documentation for their APIs.
The main documentation repository (pipelines_lsst_io) has an EUPS table file. This table file defines all the Stack packages that should be set up and that are available for inclusion in the Stack documentation build. See How to add a package to pipelines.lsst.io for details.
First, Documenteer identifies which packages contain a
Any package containing a Doxygen configuration stub file, even if empty, is presumed to potentially contain source material the C++ API reference.
Second, Documenteer constructs a Doxygen configuration.
By default, the
include directory of each relevant package is included in the
INPUT Doxygen configuration tag.
Individual packages can also add other paths to the
IMAGE_PATH tags, remove paths (
EXCLUDE_PATTERNS tags), or exclude symbols (
Finally, Documenteer uses this combined Doxygen configuration to run the doxygen command to generate an HTML site and tag file that exclusively documents the C++ API reference.
Sphinx copies the Doxygen site into the
cpp-api directory during its build so that the Doxygen-generated API reference effectively becomes an sub-site of the Sphinx-rendered site.
The pipelines.lsst.io documentation project has a special
lsstcc role, created through doxylink extension, using the Doxygen tag file, that allows reStructuredText content to link to C++ API reference pages in the Doxygen site.
For more information about Documenteer’s built-in Doxygen build, see the
documenteer.stackdocs.doxygen module, and the
DoxygenConfiguration APIs in particular.
From a documentation writer’s perspective, see also: Cross-linking the Doxygen-built C++ API reference with the lsstcc role.
The Sphinx build#
Next, stack-docs runs the Sphinx build on the pipelines_lsst_io repository.
This is different from most projects that use a
Makefile and Sphinx’s builtin sphinx-build app.
stack-docs is an all-in-one front-end designed and engineered for building the pipelines.lsst.io site.
Python API reference#
The automodapi directives in module homepages generate the Python API reference documentation.
Specifically, automodapi introspects the APIs in the given Python module and creates stub files in the
py-api directory that contain
autodoc directives that generate the documentation page for each API.
The numpydoc extension transforms docstrings before they are read by
autodoc from the numpydoc format to Sphinx’s native docstring markup.
py-api directory is entirely transient.
In fact, the stack-docs clean command will delete it.