TIL / File-hash change detection keeps a big Lambda fleet's CI fast

File-hash change detection keeps a big Lambda fleet's CI fast

AWS LambdaCI/CDCloudFormation

The problem

A CI/CD pipeline that redeploys a whole fleet of Lambda functions on every push works fine at small scale. Past a hundred-plus functions it gets slow, and most of that time is spent repackaging and redeploying code that didn’t change at all in this push.

The fix

Hash each function’s source directory, compare it against the hash recorded from the last successful deploy (stored alongside the stack, e.g. in SSM Parameter Store or a small DynamoDB table), and only redeploy the functions whose hash moved.

import hashlib
from pathlib import Path

def hash_dir(path: Path) -> str:
    h = hashlib.sha256()
    for f in sorted(path.rglob("*.py")):
        h.update(f.read_bytes())
    return h.hexdigest()

changed = [fn for fn in functions if hash_dir(fn.source_dir) != last_deployed_hash[fn.name]]

Gotcha

Hash the function’s actual dependency lockfile too, not just its own source - a shared library bump with no change to the function’s own code still needs a redeploy, and skipping that is a sneaky way to end up running stale code in production while CI reports green.