ML Engineering

The Best Way to Use Text Embeddings Portably is With Parquet and Polars

A deep dive into efficient storage and retrieval of text embeddings using Parquet files and polars library, demonstrated through Magic: The Gathering card analysis. The article explores alternatives to vector databases for smaller datasets, highlighting how combining Parquet files with polars offers zero-copy operations and fast similarity searches.