Cache

Cache (sillo.cache)

sillo ships an advanced, backend-agnostic caching subsystem. It is deliberately decoupled from the application object — you configure a backend at the domain level and use it from anywhere (handlers, services, plain functions), not just inside a request.

from sillo.cache import MemoryCache, configure_cache, cache

configure_cache(MemoryCache(default_ttl=300))

@cache(ttl=120, tags=["catalog"])
async def get_product(product_id: int):
    return await db.products.get(product_id)

Installation & backends

Two backends ship today:

Backend	Dependency	Notes
MemoryCache	none (stdlib)	In-process, thread-safe, full feature set.
RedisCache	redis>=5 (uv add "sillo[cache]")	Async Redis; mirrors the memory feature set.

uv add "sillo[cache]"   # installs the optional redis driver

If redis is not installed, RedisCache still imports — the redis.asyncio dependency is loaded lazily and only errors when you actually construct a RedisCache.

Configuring a default backend

Call configure_cache() once at startup to register a process-wide default. Any @cache() with no backend= argument uses it:

from sillo.cache import MemoryCache, configure_cache, RedisCache

# For production: a shared Redis instance.
configure_cache(RedisCache(url="redis://localhost:6379/0", default_ttl=600))

# Or keep it simple/in-process for tests and small apps:
# configure_cache(MemoryCache(default_ttl=300))

If you never call configure_cache(), the first @cache() call creates an implicit in-process MemoryCache for you. You can always override per function with backend=.

from sillo.cache import cache, MemoryCache

local = MemoryCache(namespace="session", default_ttl=60)

@cache(backend=local, ttl=30)
def expensive(x):
    return compute(x)

Domain-level, not app-level

Unlike response/cookie/session config which is tied to silloApp, caching is configured independently. This keeps cached functions testable and reusable outside the web layer (CLI commands, workers, background tasks).

The @cache decorator

Decorate any callable — sync or async, function or bound method.

Async function

from sillo.cache import cache

@cache(ttl=120, tags=["users"])
async def get_user(user_id: int):
    return await db.users.get(user_id)

Sync function

The decorator bridges to the async backend automatically, so you can use the same @cache on a plain function:

@cache(ttl=60)
def price_lookup(sku: str):
    return catalog.lookup(sku)

Bound methods

self / cls are excluded from the cache key, so all instances of a class share one cache for the same arguments. Be careful: the cached value does not depend on instance state.

class ReportService:
    @cache(namespace="reports", ttl=300)
    def build(self, quarter: str):
        return self.run_query(quarter)

Decorator options

Option	Effect
backend	Explicit backend; defaults to the configured default.
ttl	Seconds until expiry (absolute or sliding).
namespace	Groups keys; used by clear() and versioning.
version	Key version; bump to invalidate a whole namespace at once.
key_prefix	Extra literal in the key (e.g. a role name).
tags	Tuple of invalidation tags attached to every entry.
sliding	Refresh TTL on each read instead of fixed expiry.
skip_cache_if	Predicate (*args, **kwargs) -> bool; when True, runs the call without caching.
serializer	"json" (default) or "pickle" for this function’s values.
settings	A CacheSettings object providing shared defaults.

Manual invalidation

Every decorated function gains an .invalidate(*args, **kwargs) coroutine that deletes just that call’s key:

await get_user.invalidate(42)   # drop the cached entry for user 42

Skipping the cache conditionally

@cache(ttl=60, skip_cache_if=lambda x: x < 0)
async def compute(x):
    return slow(x)

When skip_cache_if returns True, the function runs and its result is returned without being stored.

Using a backend directly (on the fly)

You don’t need the decorator — the BaseCache API works as a key-value store:

from sillo.cache import MemoryCache

cache = MemoryCache(namespace="sessions", default_ttl=3600)

await cache.set("token:abc", user_id, tags=["token:abc"])
value = await cache.get("token:abc")      # returns _MISSING on miss
exists = await cache.exists("token:abc")
await cache.delete("token:abc")
await cache.touch("token:abc", ttl=7200)  # extend lifetime
await cache.clear()                        # drop keys in this namespace

get returns the module-level sentinel _MISSING on a miss or expiry (not None), so caching None as a real value is safe.

Advanced features

TTL — absolute vs sliding

• Absolute (default): the entry expires ttl seconds after it was written.
• Sliding: each read resets the expiry window to ttl seconds, so an entry that is read frequently stays alive while a forgotten one expires.

await cache.set("hot", data, ttl=300, sliding=True)

Tag-based invalidation

Attach one or more tags to entries, then drop every key with a tag at once:

await cache.set("u:1", user1, tags=["user:1", "directory"])
await cache.set("u:2", user2, tags=["user:2", "directory"])

await cache.invalidate_tags("directory")   # removes both entries

This is ideal for “invalidate all cached views of entity X” patterns:

@cache(tags=("product",))
async def get_product(product_id: int):
    ...

# after an admin edit:
await product_cache.invalidate_tags("product")

Versioning

Bump version to expire an entire key space instantly without tracking tags:

@cache(namespace="catalog", version="v2")
async def list_catalog():
    ...

Changing version="v2" to version="v3" produces different keys, so old entries are effectively dead (and eventually evicted by TTL).

LRU & max-size eviction (MemoryCache)

Set max_size to bound memory. When the store exceeds max_size, the least-recently-used entry is evicted first. Reading an entry marks it recently used, so hot keys survive.

cache = MemoryCache(max_size=1024)
await cache.set("a", 1)
await cache.set("b", 2)
await cache.set("c", 3)   # evicts "a" (oldest)

Serialization: JSON vs pickle

• serializer="json" (default) is safe and cross-language but only handles JSON-compatible data. Complex Python objects are best-effort converted.
• serializer="pickle" stores arbitrary Python objects, at the cost of being Python-only and unsafe for untrusted input.

pickle_cache = MemoryCache(serializer="pickle")
await pickle_cache.set("obj", {"set": {1, 2, 3}})

Hit/miss statistics

Every backend tracks hits, misses, sets, deletes, and evictions, exposed via .stats():

stats = cache.stats()
print(stats.hit_rate)        # 0.0–1.0
print(stats.as_dict())
cache.reset_stats()

Redis backend

The async Redis backend maps concepts onto Redis primitives:

• Keys are stored verbatim (already namespaced/versioned by make_key).
• TTL uses Redis SETEX / EXPIRE.
• Tags are Redis sets (tag:<ns>:<tag>) of member keys; invalidate_tags deletes all members and the set.
• Sliding TTL re-issues EXPIRE on each read.

from sillo.cache import RedisCache

redis_cache = RedisCache(
    url="redis://localhost:6379/0",
    namespace="myapp",
    default_ttl=600,
    serializer="json",
)

await redis_cache.set("k", value, tags=["t1"])
value = await redis_cache.get("k")
await redis_cache.invalidate_tags("t1")

Use RedisCache as a shared backend across multiple processes/workers so they cooperate on one cache.

Full example

from sillo import silloApp
from sillo.cache import MemoryCache, configure_cache, cache

configure_cache(MemoryCache(default_ttl=300))

app = silloApp()

@app.get("/products/{product_id:int}")
async def product(request, response, product_id: int):
    data = await get_product(product_id)   # cached
    return response.json(data)

@cache(namespace="catalog", ttl=600, tags=("catalog",))
async def get_product(product_id: int):
    # ... expensive lookup ...
    return {"id": product_id}

API reference

Symbol	Kind	Purpose
BaseCache	class	Abstract backend contract.
MemoryCache	class	In-process backend (TTL, LRU, tags, version).
RedisCache	class	Async Redis backend (optional redis).
CacheStats	dataclass	Hit/miss/eviction counters.
CacheSettings	dataclass	Reusable decorator defaults.
configure_cache(backend)	function	Register the default backend.
get_default_backend()	function	Return (or lazily create) the default.
reset_cache_config()	function	Clear the default (useful in tests).
cache(...)	decorator	Cache a function/method’s result.
build_key(...)	function	Deterministic key construction.
tag_key(...)	function	Storage key for a tag set.

Naming keys

A cache key is an interface between the code that writes and the code that reads, and unstructured keys become unmanageable at about the tenth one.

A workable convention has four parts: a namespace, an entity, an identifier, and a version — myapp:user:42:v2. The namespace prevents collisions with anything else sharing the store. The version lets you invalidate an entire class of entries by bumping it, which is far easier than finding and deleting them.

Include everything that changes the value. A key that omits the locale serves French content to English users; one that omits the user id on personalised data serves one user’s data to another, which is the worst outcome available in a cache and is always caused by an incomplete key.

Keep keys bounded. A key built from a free-text search query has unlimited cardinality, so the hit rate approaches zero while memory approaches the size of every query anyone has ever run. Hash long inputs and cap what you cache.

Invalidation, ranked by reliability

Expiry is the only strategy that cannot leave you permanently wrong. Everything else should be layered on top of a TTL, never instead of one — a cache entry with no TTL and a missed invalidation is stale forever.

Write-through invalidation — deleting the key when the underlying data changes — is the common approach and its failure mode is a write path you forgot. Every place that mutates the entity must invalidate, and the one in a migration script will not.

Versioned keys avoid deletion entirely: bump a version and old entries age out naturally. This survives missed invalidations and is much easier to reason about across multiple processes.

The failure modes worth designing for

Stampede. A popular key expires and a thousand concurrent requests all miss and all recompute. The fix is a lock so one request recomputes while the others wait or serve stale.

Penetration. Repeated requests for something that does not exist miss every time and hit the database every time. Cache the negative result, briefly.

Cache as a dependency. A cache outage should degrade you, not break you. Treat a cache error as a miss — read through to the source and log it — rather than letting it raise. An application that 500s when Redis restarts has made a cache into a hard dependency by accident.

What to cache, and what not to

Cache things that are expensive to produce, read far more often than they change, and identical for many callers. Reference data, rendered fragments shared across users, the result of an aggregate query, an upstream API response with a slow provider.

Do not cache things that are cheap to produce — a cache lookup is not free, and for a primary-key fetch it is often slower than the query. Do not cache per-user data with low reuse: one entry per user with a hit rate near zero is memory spent for nothing. And do not cache anything where serving a stale value is a correctness problem — permissions, balances, inventory at the point of sale.

The question that settles most cases: how wrong is a value that is sixty seconds old? If the answer is “not at all”, cache it. If the answer is “it could authorise something it should not”, do not.

Measuring whether it works

A cache with no hit-rate metric is a guess. Track hits, misses, and evictions, and read them together — a low hit rate with high eviction means the cache is too small; a low hit rate with no eviction means the keys are too specific.

Track the latency of the cached path against the uncached one too. A cache that saves two milliseconds on a query that takes three is overhead with extra failure modes, and knowing that early saves you maintaining it.

Layering caches

An application usually ends up with three tiers: a per-request memo, a per-process in-memory cache, and a shared store like Redis. Each is faster and less shared than the one below it.

Read through them in order and write back up. The per-request layer costs nothing and eliminates the duplicate lookup within one handler; the in-process layer avoids the network hop; the shared layer is the one that survives a restart and is consistent across workers.

The hazard is invalidation, which must reach every tier. A per-process cache with a long TTL will happily serve data that Redis knows is stale, and there is no mechanism to tell it otherwise short of a short TTL or a pub/sub invalidation channel. Keep in-process TTLs measured in seconds for anything that changes.

HTTP caching is a different tool

Everything above is server-side. Cache-Control, ETag, and Last-Modified push caching out to the browser and any CDN in between, where a hit costs you nothing at all — no request, no process, no database.

They solve different problems and compose well. A response cached at the CDN for sixty seconds removes most of your traffic; a server-side cache makes the requests that do arrive cheap. Use ETag with conditional requests for large responses that change rarely, so an unchanged resource costs a 304 rather than a body.

The two rules that prevent incidents: never set a public Cache-Control on an authenticated response, and always set Vary on anything that differs by header. Both mistakes have the same consequence — one user’s data served to another from a cache you do not control.