DuckDB
Embedded analytical SQL engine — I use it over Parquet to query large event histories without running a database server.
DuckDB is an in-process analytical database: columnar, vectorized SQL that runs inside the application, much like SQLite does for transactional workloads.
How I use it
In Real-time Data Systems, recorded event streams are stored as Apache Parquet & Arrow and queried with DuckDB for research and replay. One lesson I documented: nested list columns come back as Python objects with heavy per-element overhead, so large histories are converted once into flat, memory-mapped NumPy arrays before training jobs read them — turning out-of-memory crashes into jobs that fit on a small machine.
Why it matters
Embedded analytics removes a whole server from the architecture for read-heavy, single-user research workloads.