Module Superbol_cache.Types

Public type definitions for the SuperBOL dependency-aware cache.

Terminology: a witness records an observation made while producing a cached value --- the contents of a file that was read, the listing of a directory that was searched, the status of an environment variable that was consulted, or the key of another entry that was reused. A manifest gathers the witnesses observed for one entry, along with the digest of the value that was produced.

An entry is looked up via a shallow key, computed from what is known before the work is performed. The shallow key locates the manifest; a hit additionally requires every recorded witness to still hold.

type digest = string

Digest of some contents; holds the raw (16-byte) output of Digest.string, not its hexadecimal rendering.

type key = string

Hexadecimal rendering of the digest that addresses a manifest.

type witness =
  1. | File of {
    1. name : string;
      (*

      canonical path of the file that was read

      *)
    2. digest : digest;
      (*

      digest of its contents

      *)
    }
  2. | Dir of {
    1. name : string;
      (*

      canonical path of the directory searched

      *)
    2. digest : digest;
      (*

      digest of its sorted, canonical listing

      *)
    }
  3. | Env of {
    1. var : string;
      (*

      name of the variable that was consulted

      *)
    2. value : string option;
      (*

      None iff it was unset

      *)
    }
  4. | Sub of {
    1. key : key;
      (*

      entry whose value was reused

      *)
    }

An observation whose invalidation entails a miss on the entry that recorded it.

Paths carried by File and Dir are canonical in the sense of Superbol_cache.Canon; they need not be meaningful on any machine other than the one that recorded them.

type manifest = {
  1. manifest_label : string;
    (*

    caller-chosen; for monitoring only

    *)
  2. manifest_witnesses : witness list;
  3. manifest_object : digest;
    (*

    digest of the associated value

    *)
}

Everything recorded about one entry, besides its value.

type read_error =
  1. | Bad_version_tag of {
    1. got : string;
    2. expected : string;
    }
  2. | Truncated
    (*

    item shorter than its own header claims

    *)
  3. | Malformed of string
    (*

    item could not be decoded

    *)
  4. | Absent
    (*

    no such item in the cache

    *)

Why a stored item could not be read back.

Note that a Bad_version_tag is expected in normal operation: it is how entries left behind by a different producer are skipped.

type error =
  1. | Unreadable of read_error
  2. | Storage of string
    (*

    underlying storage reported a failure

    *)
  3. | Cycle of key list
    (*

    Sub witnesses form a cycle

    *)

Why a cache operation could not be carried out. Every variant denotes a condition under which the caller should proceed as though the cache were empty; none of them is an error the caller needs to handle.

type prune_policy =
  1. | Never
    (*

    no entry is ever removed automatically

    *)
  2. | Bounded of {
    1. max_age : float option;
      (*

      Reclaim entries that have not been hit for that many seconds.

      *)
    2. max_size : int64 option;
      (*

      Keep the whole cache --- entries and the values they hold --- under that many bytes, by reclaiming those least recently used.

      *)
    }

When entries are reclaimed.

The bounds of Bounded compose: an entry goes if any of them says so, and leaving them all unset reclaims nothing but the leftovers of instances that died mid-write.

type entry_info = {
  1. info_key : key;
  2. info_label : string;
  3. info_witnesses : witness list;
  4. info_disk_usage : int64;
    (*

    manifest and value, in bytes

    *)
  5. info_last_hit : float;
    (*

    seconds since the epoch

    *)
}

One row of the monitoring API.

type stats = {
  1. stats_entries : int;
  2. stats_objects : int;
  3. stats_disk_usage : int64;
  4. stats_format_version : string;
  5. stats_producer : string;
}

Aggregate figures reported by the monitoring API.