Alan's Q2 Update 2024

By Alan Freitas on
Jul 13, 2024

Summary

MrDocs

MrDocs is a tool for generating reference documentation from C++ code and javadoc comments. I have been overseeing and reviewing all contributions to the project.

  • Krystian has focused on metadata extraction and issues related to Clang.
  • Fernando has been tackling usability, critical bugs, and essential features.
  • I’m enhancing the CI setup and working on Antora extensions, which will be incorporated into website-v2-docs.

Currently, metadata extraction primarily involves identifying SFINAE techniques in the code, enabling MrDocs to natively comprehend C++ constructs.

In the last quarter, I authored several documents comparing MrDocs to its alternatives:

  • Gap Analysis: Compares MrDocs to alternatives, identifying and prioritizing missing features.
  • Minimum Viable Product (MVP): Outlines the features necessary for MrDocs to be competitive.
  • Product Pitch: Explains MrDocs’ value proposition.

The MVP aims to include features common to all alternative tools, preventing initial adoption blockers and ensuring a positive first impression. Additionally, the MVP includes unique features of MrDocs, such as native comprehension of C++ constructs. We identified numerous selling points for MrDocs, including maintainability, multiple output formats, and its understanding of C++ constructs.

We created a GitHub project to track MrDocs’ progress, with issues categorized by priority:

  • P0 - Boost Features: Critical bugs and usability issues, using Boost.URL as a proof of concept.
  • P1 - Main MVP Features: Essential MVP features requested by users and Boost projects.
  • P2 - Technical Debt: Technical debt from P0 and P1.
  • P3 - All MVP Features: All other MVP features.
  • P4 - Non-MVP Features: Features beyond the MVP outlined in the gap analysis.

P0 encompasses all issues necessary for MrDocs to be utilized by an Antora extension in the Boost Release Tools. Using Boost.URL as a proof of concept, our goal in P0 is to replace the Doxygen + Docca workflow with MrDocs. We completed P0 recently, with partial results available at https://843.urlantora.prtest2.cppalliance.org/site/url/index.html

As we transition to P1, I’m adapting the Boost.URL project with Antora extensions to include reference documentation generated by MrDocs in the website-v2-docs Antora project. P1 includes all features identified in our Gap Analysis for MrDocs to be competitive with alternatives. We are also improving usability and documentation based on feedback to ensure users can effectively utilize the tool. Completing P0 and P1 will accumulate enough technical debt to start addressing P2.

In Q2/2024, we implemented several features and fixes, including:

  • Detect SFINAE idiom
  • Extract using directives, declarations, namespace aliases
  • Extract pack expansions in base-specifiers
  • Extract variadic functions
  • Extract explicit specifiers
  • Detect implementation-defined and see-below namespaces
  • Support @pre, @post, and @n commands
  • Annotate templates with See Also sections, constructors, and destructors
  • Ensure HTML templates match AsciiDoc templates
  • Implement select handlebars helper
  • Link names in overload sets
  • Include full nested name specifiers in titles
  • Ensure safe names for section IDs
  • Provide warnings for duplicate commands
  • Propagate dependency modes
  • Extract dependencies for templates
  • Check for atomic shared pointer support
  • Update XML schema
  • Escape names of overload set members
  • Restore HTML demos
  • Normalize source-root
  • Ensure global namespace is always present
  • Remove symbol type from titles
  • Update LLVM version to 7a28a5b3 and adapt the project
  • Treat references to purposefully excluded symbols as non-errors
  • Emit error messages for invalid javadoc commands
  • Manage info and config tables as .inc files
  • Print diffs in golden tests
  • Create CI demo variants with command line variants
  • Build statically for MrDocs to be used in CI by other projects
  • Use default containers for each compiler
  • Update CI to use official releases for dependencies, enhancing stability
  • Compare generated documentation in PRs with a demo comparison artifact
  • Include ref_name in demo artifact names for proper PR comparison
  • Upload documentation to the website
  • Ensure template consistency with checks
  • Split test targets
  • Set default build type to RelWithDebInfo
  • Clean headers and footers in docs
  • Create a demo page in the documentation using a custom Antora plugin
  • Add an install page with direct links to GitHub releases
  • Provide more extensive usage examples
  • Complete the configuration file reference
  • Include design notes on rationale and philosophy
  • Create a Contributor Guide
  • Refer to official documentation in README.adoc
  • Add a new banner referring to Mr. Docs

Integrations

As Boost.URL continues to integrate MrDocs, it has inspired the necessary features in MrDocs and is temporarily generating documentation with both Doxygen+Docca and Antora+MrDocs. The same workflow is implemented in Boost.Buffers, inspiring new feature requests such as identifying Niebloids and SFINAE constraints in documentation.

The features implemented in Boost.URL are described in the Boost Libraries section.

General work

Overall, my responsibilities in MrDocs include:

  • Setting up and maintaining CI for the project;
  • MrDocs and LLVM release binaries;
  • Build scripts;
  • Setting up and integrating dependencies;
  • Setting up and deploying the Antora toolchains and documentation to the project website;
  • Working on supporting libraries;
  • Supervising and reviewing the work done by other contributors (Krystian and Fernando); and
  • Fixing bugs.

Boost Libraries

The Boost library I’ve dedicated the most time to is Boost.URL. The library is in maintenance mode, but there is a constant demand for bug fixes and documentation improvements. Recent commits focus on the Antora+MrDocs workflow:

  • MrDocs collector plugin identifies versioned compiler executables on the host machine
  • Documentation includes a manual reference table
  • Refactored source code to distinguish details, implementation-defined, and see-below symbols
  • Created a new MrDocs target that includes all headers, reducing MrDocs run time from ~8 minutes to ~3 seconds and ensuring all headers are included in the documentation

Other significant updates in CI and documentation include:

  • CI: Added support for Node.js 20 actions due to GitHub updating the base Node.js version to 20. The lack of static linking in Node.js caused errors related to glibc versions.
  • CI: Updated the Antora workflow to use Clang 18, addressing issues with standard library synchrony between the host machine and LLVM/Clang versions.
  • Docs: string_token exposition
  • CI improvements were coordinated with the C++ Github Actions project, which implemented new features for these scenarios.

Boost Release Tools

I have integrated the toolchains I developed into the Boost Release Tools, adding support for features desired for the new Boost website, including the Antora+MrDocs workflow.

Since the last report, we adapted the documentation workflow to relocate the Antora documentation in the release. Additionally, Boost.URL documentation in the Antora format will be included in the release once we enable it in website-v2-docs. We are delaying this to avoid disruptions during the current release cycle. Meanwhile, we continue using the Doxygen+Docca workflow and are developing the Antora UI for the next release.

Boost Website

Among support projects for the new Boost website, I have been particularly involved with website-v2-docs, which includes the Boost website documentation as an Antora project. Its components cover the “User Guide,” “Contributor Guide,” and “Formal Review” sections.

Since the project’s inception, I have been overseeing and reviewing contributions from other team members.

Overall, my responsibilities include:

  • Reviewing and merging pull requests
  • Setting up and maintaining CI
  • Coordinating with the website project on content uploaded to AWS buckets
  • Developing reusable build scripts for release tools and previews
  • Writing technical sections of the documentation
  • Developing custom Boost/Antora extensions, such as the Boost Macro extension
  • Maintaining the Antora toolchain and templates
  • Adjusting Boost libraries to match the expected formats

For the next quarter, we plan to update the Antora UI bundle to resemble the current Boost website style.

C++ Github Actions

C++ Github Actions is a project I have maintained since 2023. It is a collection of composable, independent, and reusable GitHub Actions for any C++ project needing testing on various compilers and environments.

MrDocs, Boost.URL, Boost.HTTP, and Boost.Buffers currently use these actions in their CI. These projects provide valuable feedback to improve the actions.

The project includes actions to:

  • Generate a Github Actions Matrix for C++ projects;
  • Setup C++ compilers;
  • Install and setup packages;
  • Clone Boost modules;
  • Run complete CMake and b2 workflows;
  • Generate changelogs from conventional commits;
  • Generate summaries; and
  • Generate time-trace reports and flame graphs

The actions are designed to be modular and interoperable. Each action has a specific role, such as configuring an environment or building a C++ project. They can be composed to create customized CI/CD workflows.

One notable problem with GitHub actions in Q1/2024 is that GitHub decided to update the base version of Node.js to 20. Because Node.Js is not statically linked, this change broke many actions that depended on it. All projects that depended on older containers were getting an error related to the appropriate glic version not being found. Not updating to Node.js 20 would also give users deprecation warnings.

We ended up implementing a solution where older containers suggested by the cpp-matrix action have a volumes key to create a directory where node can be installed in such a way that GitHub actions will use this node executable. If using an old container, the user is responsible for installing Node.js in that directory. Examples are provided in the repository.

Thus, a number of new features and fixes were added to the C++ Actions project in Q2/2024.

  • Feature: support clang >=18 (which is also critical for MrDocs workflows)
  • Feature: cpp-matrix force factor flags
  • Feature: cpp-matrix is-container auxiliary key
  • Feature: cpp-matrix older containers suggest volumes for node 20
  • Fix: cpp-matrix windows runner has MSVC 14.40.33810
  • Fix: cpp-matrix default macOS is macos-14
  • Fix: setup-gcc GCC 14 comes from Ubuntu 24.04
  • Fix: setup-gcc ensure software-properties-common
  • Fix: setup-clang verify LLVM repo release files to filter repositories that do not exist
  • Refactor: package-install set DEBIAN_FRONTEND to noninteractive
  • Refactor: cpp-matrix extract support sanitizer as boolean
  • Refactor: all actions use node20
  • Refactor: cpp-matrix pkg-config included in container default installs
  • CI: only create releases for tags
  • CI: external actions updated
  • CI: matrix.cxx not required
  • CI: b2-workflow variant2 does not test on windows
  • CI: older containers patch node
  • CI: update actions/checkout to v4
  • Test: cpp-matrix custom suggestions
  • Docs: cpp-matrix open ranges

Most work has been done to fix issues revealed by testing the actions in new environments.

All Posts by This Author