From 35599f9a6671779a377443ae6e596367a7613e22 Mon Sep 17 00:00:00 2001 From: Nicholas Chin Date: Tue, 21 Feb 2023 19:41:06 -0700 Subject: Docs: Replace Recommonmark with MyST Parser Recommonmark has been deprecated since 2021 [1] and the last release was over 3 years ago [2]. As per their announcement, Markedly Structured Text (MyST) Parser [3] is the recommended replacement. For the most part, the existing documentation is compatible with MyST, as both parsers are built around the CommonMark flavor of Markdown. The main difference that affects coreboot is how the Sphinx toctree is generated. Recommonmark has a feature called auto_toc_tree, which converts single level lists of references into a toctree: * [Part 1: Starting from scratch](part1.md) * [Part 2: Submitting a patch to coreboot.org](part2.md) * [Part 3: Writing unit tests](part3.md) * [Managing local additions](managing_local_additions.md) * [Flashing firmware](flashing_firmware/index.md) MyST Parser does not provide a replacement for this feature, meaning the toctree must be defined manually. This is done using MyST's syntax for Sphinx directives: ```{toctree} :maxdepth: 1 Part 1: Starting from scratch Part 2: Submitting a patch to coreboot.org Part 3: Writing unit tests Managing local additions Flashing firmware ``` Internally, auto_toc_tree essentially converts lists of references into the Sphinx toctree structure that the MyST syntax above more directly represents. The toctrees were converted to the MyST syntax using the following command and Python script: `find ./ -iname "*.md" | xargs -n 1 python conv_toctree.py` ``` import re import sys in_list = False f = open(sys.argv[1]) lines = f.readlines() f.close() with open(sys.argv[1], "w") as f: for line in lines: match = re.match(r"^[-*+] \[(.*)\]\((.*)\)$", line) if match is not None: if not in_list: in_list = True f.write("```{toctree}\n") f.write(":maxdepth: 1\n\n") f.write(match.group(1) + " <" + match.group(2) + ">\n") else: if in_list: f.write("```\n") f.write(line) in_list = False if in_list: f.write("```\n") ``` While this does add a little more work for creating the toctree, this does give more control over exactly what goes into the toctree. For instance, lists of links to external resources currently end up in the toctree, but we may want to limit it to pages within coreboot. This change does break rendering and navigation of the documentation in applications that can render Markdown, such as Okular, Gitiles, or the GitHub mirror. Assuming the docs are mainly intended to be viewed after being rendered to doc.coreboot.org, this is probably not an issue in practice. Another difference is that MyST natively supports Markdown tables, whereas with Recommonmark, tables had to be written in embedded rST [4]. However, MyST also supports embedded rST, so the existing tables can be easily converted as the syntax is nearly identical. These were converted using `find ./ -iname "*.md" | xargs -n 1 sed -i "s/eval_rst/{eval-rst}/"` Makefile.sphinx and conf.py were regenerated from scratch by running `sphinx-quickstart` using the updated version of Sphinx, which removes a lot of old commented out boilerplate. Any relevant changes coreboot had made on top of the previous autogenerated versions of these files were ported over to the newly generated file. From some initial testing the generated webpages appear and function identically to the existing documentation built with Recommonmark. TEST: `make -C util/docker docker-build-docs` builds the documentation successfully and the generated output renders properly when viewed in a web browser. [1] https://github.com/readthedocs/recommonmark/issues/221 [2] https://pypi.org/project/recommonmark/ [3] https://myst-parser.readthedocs.io/en/latest/ [4] https://doc.coreboot.org/getting_started/writing_documentation.html Change-Id: I0837c1722fa56d25c9441ea218e943d8f3d9b804 Signed-off-by: Nicholas Chin Reviewed-on: https://review.coreboot.org/c/coreboot/+/73158 Reviewed-by: Matt DeVillier Tested-by: build bot (Jenkins) --- Documentation/Makefile.sphinx | 235 +++--------------------------------------- 1 file changed, 15 insertions(+), 220 deletions(-) (limited to 'Documentation/Makefile.sphinx') diff --git a/Documentation/Makefile.sphinx b/Documentation/Makefile.sphinx index 289c45bec7..c4984ece08 100644 --- a/Documentation/Makefile.sphinx +++ b/Documentation/Makefile.sphinx @@ -1,60 +1,20 @@ ## SPDX-License-Identifier: GPL-2.0-only -# Makefile for Sphinx documentation +# Minimal makefile for Sphinx documentation # -# You can set these variables from the command line. -SPHINXOPTS ?= -SPHINXBUILD = sphinx-build -SPHINXAUTOBUILD = sphinx-autobuild -PAPER = -BUILDDIR = _build +# You can set these variables from the command line, and also +# from the environment for the first two. +SPHINXOPTS ?= +SPHINXBUILD ?= sphinx-build +SPHINXAUTOBUILD = sphinx-autobuild +SOURCEDIR = . +BUILDDIR = _build -# Internal variables. -PAPEROPT_a4 = -D latex_paper_size=a4 -PAPEROPT_letter = -D latex_paper_size=letter -ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . -# the i18n builder cannot share the environment and doctrees with the others -I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . - -.PHONY: help +# Put it first so that "make" without argument is like "make help". help: - @echo "Please use \`make ' where is one of" - @echo " html to make standalone HTML files" - @echo " dirhtml to make HTML files named index.html in directories" - @echo " singlehtml to make a single large HTML file" - @echo " pickle to make pickle files" - @echo " json to make JSON files" - @echo " htmlhelp to make HTML files and a HTML help project" - @echo " qthelp to make HTML files and a qthelp project" - @echo " applehelp to make an Apple Help Book" - @echo " devhelp to make HTML files and a Devhelp project" - @echo " epub to make an epub" - @echo " epub3 to make an epub3" - @echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter" - @echo " latexpdf to make LaTeX files and run them through pdflatex" - @echo " latexpdfja to make LaTeX files and run them through platex/dvipdfmx" - @echo " text to make text files" - @echo " man to make manual pages" - @echo " texinfo to make Texinfo files" - @echo " info to make Texinfo files and run them through makeinfo" - @echo " gettext to make PO message catalogs" - @echo " changes to make an overview of all changed/added/deprecated items" - @echo " xml to make Docutils-native XML files" - @echo " pseudoxml to make pseudoxml-XML files for display purposes" - @echo " linkcheck to check all external links for integrity" - @echo " doctest to run all doctests embedded in the documentation (if enabled)" - @echo " coverage to run coverage check of the documentation (if enabled)" - @echo " dummy to check syntax errors of document sources" - -.PHONY: clean -clean: - rm -rf $(BUILDDIR) + @$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) -.PHONY: html -html: - $(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html - @echo - @echo "Build finished. The HTML pages are in $(BUILDDIR)/html." +.PHONY: help Makefile.sphinx .PHONY: livehtml livehtml: @@ -63,172 +23,7 @@ livehtml: @echo $(SPHINXAUTOBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR) -.PHONY: dirhtml -dirhtml: - $(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml - @echo - @echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml." - -.PHONY: singlehtml -singlehtml: - $(SPHINXBUILD) -b singlehtml $(ALLSPHINXOPTS) $(BUILDDIR)/singlehtml - @echo - @echo "Build finished. The HTML page is in $(BUILDDIR)/singlehtml." - -.PHONY: pickle -pickle: - $(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle - @echo - @echo "Build finished; now you can process the pickle files." - -.PHONY: json -json: - $(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json - @echo - @echo "Build finished; now you can process the JSON files." - -.PHONY: htmlhelp -htmlhelp: - $(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp - @echo - @echo "Build finished; now you can run HTML Help Workshop with the" \ - ".hhp project file in $(BUILDDIR)/htmlhelp." - -.PHONY: qthelp -qthelp: - $(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) $(BUILDDIR)/qthelp - @echo - @echo "Build finished; now you can run "qcollectiongenerator" with the" \ - ".qhcp project file in $(BUILDDIR)/qthelp, like this:" - @echo "# qcollectiongenerator $(BUILDDIR)/qthelp/coreboot.qhcp" - @echo "To view the help file:" - @echo "# assistant -collectionFile $(BUILDDIR)/qthelp/coreboot.qhc" - -.PHONY: applehelp -applehelp: - $(SPHINXBUILD) -b applehelp $(ALLSPHINXOPTS) $(BUILDDIR)/applehelp - @echo - @echo "Build finished. The help book is in $(BUILDDIR)/applehelp." - @echo "N.B. You won't be able to view it unless you put it in" \ - "~/Library/Documentation/Help or install it in your application" \ - "bundle." - -.PHONY: devhelp -devhelp: - $(SPHINXBUILD) -b devhelp $(ALLSPHINXOPTS) $(BUILDDIR)/devhelp - @echo - @echo "Build finished." - @echo "To view the help file:" - @echo "# mkdir -p $$HOME/.local/share/devhelp/coreboot" - @echo "# ln -s $(BUILDDIR)/devhelp $$HOME/.local/share/devhelp/coreboot" - @echo "# devhelp" - -.PHONY: epub -epub: - $(SPHINXBUILD) -b epub $(ALLSPHINXOPTS) $(BUILDDIR)/epub - @echo - @echo "Build finished. The epub file is in $(BUILDDIR)/epub." - -.PHONY: epub3 -epub3: - $(SPHINXBUILD) -b epub3 $(ALLSPHINXOPTS) $(BUILDDIR)/epub3 - @echo - @echo "Build finished. The epub3 file is in $(BUILDDIR)/epub3." - -.PHONY: latex -latex: - $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex - @echo - @echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex." - @echo "Run \`make' in that directory to run these through (pdf)latex" \ - "(use \`make latexpdf' here to do that automatically)." - -.PHONY: latexpdf -latexpdf: - $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex - @echo "Running LaTeX files through pdflatex..." - $(MAKE) -C $(BUILDDIR)/latex all-pdf - @echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex." - -.PHONY: latexpdfja -latexpdfja: - $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex - @echo "Running LaTeX files through platex and dvipdfmx..." - $(MAKE) -C $(BUILDDIR)/latex all-pdf-ja - @echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex." - -.PHONY: text -text: - $(SPHINXBUILD) -b text $(ALLSPHINXOPTS) $(BUILDDIR)/text - @echo - @echo "Build finished. The text files are in $(BUILDDIR)/text." - -.PHONY: man -man: - $(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(BUILDDIR)/man - @echo - @echo "Build finished. The manual pages are in $(BUILDDIR)/man." - -.PHONY: texinfo -texinfo: - $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo - @echo - @echo "Build finished. The Texinfo files are in $(BUILDDIR)/texinfo." - @echo "Run \`make' in that directory to run these through makeinfo" \ - "(use \`make info' here to do that automatically)." - -.PHONY: info -info: - $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo - @echo "Running Texinfo files through makeinfo..." - make -C $(BUILDDIR)/texinfo info - @echo "makeinfo finished; the Info files are in $(BUILDDIR)/texinfo." - -.PHONY: gettext -gettext: - $(SPHINXBUILD) -b gettext $(I18NSPHINXOPTS) $(BUILDDIR)/locale - @echo - @echo "Build finished. The message catalogs are in $(BUILDDIR)/locale." - -.PHONY: changes -changes: - $(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes - @echo - @echo "The overview file is in $(BUILDDIR)/changes." - -.PHONY: linkcheck -linkcheck: - $(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck - @echo - @echo "Link check complete; look for any errors in the above output " \ - "or in $(BUILDDIR)/linkcheck/output.txt." - -.PHONY: doctest -doctest: - $(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest - @echo "Testing of doctests in the sources finished, look at the " \ - "results in $(BUILDDIR)/doctest/output.txt." - -.PHONY: coverage -coverage: - $(SPHINXBUILD) -b coverage $(ALLSPHINXOPTS) $(BUILDDIR)/coverage - @echo "Testing of coverage in the sources finished, look at the " \ - "results in $(BUILDDIR)/coverage/python.txt." - -.PHONY: xml -xml: - $(SPHINXBUILD) -b xml $(ALLSPHINXOPTS) $(BUILDDIR)/xml - @echo - @echo "Build finished. The XML files are in $(BUILDDIR)/xml." - -.PHONY: pseudoxml -pseudoxml: - $(SPHINXBUILD) -b pseudoxml $(ALLSPHINXOPTS) $(BUILDDIR)/pseudoxml - @echo - @echo "Build finished. The pseudo-XML files are in $(BUILDDIR)/pseudoxml." - -.PHONY: dummy -dummy: - $(SPHINXBUILD) -b dummy $(ALLSPHINXOPTS) $(BUILDDIR)/dummy - @echo - @echo "Build finished. Dummy builder generates no files." +# Catch-all target: route all unknown targets to Sphinx using the new +# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS). +%: Makefile.sphinx + @$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) -- cgit v1.2.3