Fire-Flyer File System (3FS) is a high-performance distributed storage solution optimized for AI workloads, featuring strong consistency and disaggregated architecture. The system achieves impressive throughput of 6.6 TiB/s in read operations across 180 storage nodes, while supporting diverse workloads from data preparation to inference caching.
An analysis of distributed systems programming models reveals limitations in current approaches: external-distribution, static-location, and arbitrary-location paradigms. Despite advancements in distributed systems over the last decade, programming models haven't fundamentally improved, leading to ongoing challenges with concurrency, fault tolerance, and versioning.
A Mastodon server dedicated to computer systems research and professional discussions, focusing on operating systems, distributed systems, networks, and databases within the fediverse ecosystem.
A technical analysis reveals Kafka's limitations as a job queue, highlighting potential unfairness in job distribution among workers, especially at low volumes. The worst-case scenario formula shows how jobs can be unevenly distributed, leading to inefficient resource utilization. Traditional message brokers may be more suitable for low-volume job queuing until Kafka implements KIP-932.
TigerBeetle has introduced Vörtex, a non-deterministic testing harness designed to complement their existing Deterministic Simulation Testing by testing compiled binaries and client libraries under real-world conditions. The system injects network faults and simulates process failures through a supervisor-managed architecture, having already uncovered several significant bugs in its first four months of operation.
Explores patterns for implementing realtime features in applications, focusing on methods like Poke/Pull, Push State, Push Ops, and Event Sourcing to keep client-server data synchronized. Details various transport mechanisms for maintaining server-client connections and addresses scaling challenges in distributed systems.