DuckDB

In-process analytical SQL database built for OLAP workloads. Runs embedded in your application or notebook with no server to deploy, so you query data where it lives: Parquet, JSON, CSV, S3, and data lakes. Uses a columnar storage engine and vectorized execution for analytical queries on large datasets.

DuckDB is used by data teams across Big Tech, finance, and startups. Unlike SQLite, it targets analytical workloads with aggregations and joins over large tables, and unlike Spark or data warehouse clusters, it embeds directly in-process for zero operational overhead. Released under MIT by the DuckDB Foundation.

Core capabilities:

• Direct querying of Parquet, JSON, CSV, and cloud storage without loading into tables
• Native integrations with Python, R, Java, Go, Node.js, Rust, and Wasm in the browser
• Extensions for Postgres compatibility, spatial data, AWS and Azure, and Apache Iceberg
• Single-file deployment with no external dependencies, runs on all major OS and CPU architectures
• Deep integration with Pandas and dplyr for interactive analysis workflows

Data scientists use DuckDB to analyze local or remote Parquet files in notebooks without setting up a cluster. Engineers embed it in applications for fast analytical queries, ETL pipelines, and ad-hoc reporting. The CLI and language bindings share the same engine, so scripts and embedded usage behave identically.