Skip to content

08 Metaprogramming

Version controls for dependencies

Take a look at the various ways to specify version requirements for dependencies in Rust’s build system. Most package repositories support similar syntax. For each one (caret, tilde, wildcard, comparison, and multiple), try to come up with a use-case in which that particular kind of requirement makes sense.

major.minor.patch Release increments number of:

  • patch: API hasn’t changed
  • minor: API has new, backwards-compatible changes
  • major: API has backwards-incompatible change

Dependency specification | Documentation | Poetry - Python dependency management and packaging made easy

Specifying Dependencies - The Cargo Book (rust-lang.org)

# Caret requirements
# An update is allowed if the new version number does not modify the left-most non-zero digit in the major, minor, patch grouping.
^1.2.3  :=  >=1.2.3, <2.0.0
^1.2    :=  >=1.2.0, <2.0.0
^1      :=  >=1.0.0, <2.0.0
^0.2.3  :=  >=0.2.3, <0.3.0
^0.2    :=  >=0.2.0, <0.3.0
^0.0.3  :=  >=0.0.3, <0.0.4
^0.0    :=  >=0.0.0, <0.1.0
^0      :=  >=0.0.0, <1.0.0

# Tilde requirements
# If you specify a major, minor, and patch version or only a major and minor version, only patch-level changes are allowed. If you only specify a major version, then minor- and patch-level changes are allowed.
~1.2.3  := >=1.2.3, <1.3.0
~1.2    := >=1.2.0, <1.3.0
~1      := >=1.0.0, <2.0.0

# Wildcard requirements
*     := >=0.0.0
1.*   := >=1.0.0, <2.0.0
1.2.* := >=1.2.0, <1.3.0

Make: Top (GNU make)

especially: Automatic Variables (GNU make)

TLDR:

Makefile Tutorial By Example

There's a nice list of options that can be run from make. Check out --dry-run, --touch, --old-file.

You can have multiple targets to make, i.e. make clean run test runs the clean goal, then run, and then test.

A Simple Makefile Tutorial (colby.edu)

Note that order of entries in Makefile matters - it begins the first rule by default, then the rest is dependencies oriented. That said, if some latter targets are not dependencies of previous rules, they might not be executed

Some mistake I made: <mark>c - How do I make Makefile to recompile only changed files? - Stack Overflow </mark>

In short, makefile treat target name as file name.

If no target files exist: it will recompile it.

If target files exist: it won't recompile it.

paper.pdf: paper.tex plot-data.png
    pdflatex paper.tex

plot-%.png: %.dat plot.py
    ./plot.py -i $*.dat -o $@

.PHONY: clean
# clean is a phony target that is not a file named clean.
clean:
    git ls-files -o | xargs rm -f

similar command from pre-commit

git ls-files -- '*.py' | xargs pre-commit run --files: run all hooks against all *.py files in the repository.

.git/hooks/pre-commit

#!/bin/sh
# https://jsinkers.github.io/notes/notebooks/missing_class/08_metaprogramming.html
# Pre-commit script to prevent commit if the make fails
# Redirect output to stderr.

exec 1>&2

if make

then
    echo "Make successful"
else
    cat << EOF
    Error: could not make pdf
    EOF
    exit 1
fi

# OR run spefic command only, in this case paper.pdf entry only

#!/bin/sh
if ! make paper.pdf ; then
    echo "Cannot make paper.pdf"
    exit 1
fi

Github workflows - GitHub Pages

Compare common stacks that are known to be good for rendering static websites.

Mkdocs

Mkdocs has some issues: Navigation not in alphanumeric order (when pages config is automatic) · Issue #638 · mkdocs/mkdocs (github.com) that can be solved via hard-coding the navigation in the mkdocs.yml:

einops/mkdocs.yml at master · arogozhnikov/einops

OR another solution to setup the navigation care-free:

lukasgeiter/mkdocs-awesome-pages-plugin: An MkDocs plugin that simplifies configuring page titles and their order

courses/update_navigation_order.sh

From my own experience, it supports Latex pretty well: Math expression does not render correctly. · Issue #1733 · mkdocs/mkdocs

Anyways, it orders pages based on lexicographical order by default (which can be fixed by the shell script above). This saves a lot of time compared to manually generating or writing specific indexing file in order to maintain the navigation bar.

Jekyll

Jekyll provides the most beautiful UI/themes from what I can tell - but its navigation bar needs to be manually maintained/hard-coded using Collections | Jekyll • Simple, blog-aware, static sites. To get something in quickly:
The Quickest Way to Blog on GitHub Pages. | ruhoh universal static blog generator
stanford-cs329s/reports: Final reports for CS 329S Winter 2021

Just the Docs

Based on Jekyll, Just the Docs also requires hard-coding the navigation bar, but it's doing it in a bit ?convenient way using in-page YAML front matter Navigation Structure | Just the Docs

rnnh/bioinfo-notebook: 🔬 Bioinformatics Notebook. Scripts for bioinformatics pipelines, with quick start guides for programs and video demonstrations.

Docsify

Docsify is installed through npm which is not installed in most of servers I use - don't want to install it on a industry server without sudo. Also, it needs to hard-code page layout using _sidebard.md/_navbard.md/_coverpage.md/etc. e.g. here: docsify/_sidebar.md at master · docsifyjs/docsify

Read the Docs

A commonly used framework

It needs to hard-code navigation bar using index.rst: cutadapt/doc at main · marcelm/cutadapt

Like Sphinx Tutorial — Sphinx Tutorial 1.0 documentation, the main advantage is that it provides version controls - really easy to check old versions of the docs.

Sphinx

Need to use _toctree.yml for table of content. But it has really awesome UI and version control with example here:
huggingface/accelerate: 🚀 A simple way to train and use PyTorch models with multi-GPU, TPU, mixed-precision
D2L-Book: A Toolkit for Hands-on Books — D2L-Book: A Toolkit for Hands-on Books 0.1.17 documentation
Basic Installation — trimesh 3.17.1 documentation

Recommended theme:

readthedocs/sphinx_rtd_theme: Sphinx theme for readthedocs.org

Overall, for documentation, I personally prefer to Sphinx with the example shown here: accelerate/docs at main · huggingface/accelerate OR just-the-docs/just-the-docs: A modern, high customizable, responsive Jekyll theme for documention with built-in search..