DuckDB 2.0: Async I/O, Recursive CTEs, and VARIANT Improvements
DuckDB 2.0 introduces significant performance enhancements, particularly in async I/O, recursive CTEs, and VARIANT data handling. Async I/O speeds up querying over AWS S3 by separating download and decode tasks, allowing network and CPU to work simultaneously. This results in 2x to 3x faster performance with no query changes required. Recursive CTEs have been rewritten to drastically improve performance on deep parent/child datasets, such as git history, by reducing the need to re-read the entire table for each recursion level. VARIANT data type now supports shredding, which optimizes storage and query performance by extracting consistent fields from JSON data into separate columns.
Read the full article
Continue reading at Hacker News →This is an AI-generated summary. Read the original for the full story.