a three-line build-cache that has lasted me a year
I have a script that processes a folder of input files and produces a single artifact. The processing is slow. The folder changes about once a week. I tried two real caching libraries, a CI feature, and a Python decorator. None survived contact with the script for more than a fortnight.
What I ended up with is dumb and has not failed me since:
STAMP := .build.stamp SOURCES := $(shell find inputs -type f | sort) $(STAMP): $(SOURCES); ./build.sh && touch $@
That's the whole thing. The trick is that find ... | sort
inside a shell expansion gives Make the list of files at parse
time, and the stamp file is the only "output" Make tracks. Add a
file, change a file, remove a file — the stamp's older than the
inputs, the build runs.
The reason this is not a real cache: it can't tell you which input changed, can't do partial rebuilds, and falls over the moment two parallel jobs touch the same stamp. None of which I need.
Most caching is like this. You don't need a cache, you need a freshness signal. A stamp file is a freshness signal you can explain to a colleague in one breath.