Full-day workshop, 13 listed talks/papers.Full-day workshop, 13 listed talks/papers. The Next Adventure in Data Systems Research: AI, Quantum Computing, and Lessons for the PhD Journey • Towards Expressive, Performant, and Correct Database Systems • Lessons Learned Building Cross-Architecture Analytical Engines • ORM-Lock: Source-Linked Diagnosis and Repair Guidance for ORM-Backed Database Deadlocks • Declarative Recall for Approximate Vector Search • XGAP: Ambiguity-aware Cross-Platform Planning for Natural Language Graph Queries • User-Guided Discovery of Patterns in Sequential Data • Model Lakes for Cross-Model Understanding • Indexing Long Documents for LLM-Based Analysis • Generative AI for Multi-Modal Data Management • Learning from Quality, Not for It: Conditioning Models on Explicit Data Quality • Principled Bias Detection and Mitigation across the ML Data Lifecycle • … +1 moreread more
☆
Mon Aug 31 · 8:45 AM - 5:15 PM
ADMS · Seventeenth International Workshop on Accelerating Analytics and Data Management Systems Using Modern Processor and Storage Architectures
● Workshops
📍 Grand Ballroom D
Full-day workshop, 12 listed talks/papers.Full-day workshop, 12 listed talks/papers. ADMS Welcome • Special Hash Values for Linear-time Sorting • The Free Lunch Has a Queue: Characterizing On-chip Compression Accelerators for Analytics • From MapD to Sirius: Co-Designing GPU-Native OLAP • OptFSST: Optimized FSST String Compression • HetMatch: GPU-accelerated Heterogeneous String Pattern Matching • Accelerating Presto with GPUs • Automated Operator Fusion for GPU-Accelerated SQL: Generation and Evolutionary Refinement • Rubberband: Memory-Elastic, Skew-Tolerant GPU Hash Joins • High-Throughput LLM-based Compression and What It Takes • Building General-Purpose Bitmap Indexes: A tale of CUBIT, RABIT, and beyond • Maintainable Low-latency, and High-Quality Query Compilation with MLIR and TPDEread more
☆
Mon Aug 31 · 8:45 AM - 5:15 PM
NOVAS · Novel Optimization for Visionary AI Systems
● Workshops
📍 Grand Ballroom E
Full-day workshop, 18 listed talks/papers. Welcome to NOVAS • Cortex AI SQL • Enterprise Text-to-SQL: Are Agents Ready to Scale?Full-day workshop, 18 listed talks/papers. Welcome to NOVAS • Cortex AI SQL • Enterprise Text-to-SQL: Are Agents Ready to Scale? • DoubleAgent: Towards Text-to-SQL on Enterprise Databases • Exploring the Semantic Gap in Agentic Data Systems: A Formative Study of Operationalization Failures in Analytical Workflows • Prompting Strategies for Enterprise Text-to-SQL: An Ablation Study • Self-designing AI • SemJoin: Semantic Join Optimization • SemCEB: A Cardinality Estimation Benchmark for Semantic Operators • Low-Latency Semantic Processing with Optimal Prompt-Level Batching • Poodle: Seamlessly Scaling Down Large Language Models with Just-in-Time Model Replacement • Building the Data Infrastructure for Agentic Organizations • … +6 moreread more
☆
Mon Aug 31 · 8:45 AM - 5:15 PM
QDB · 15th International Workshop on Quality in Databases
● Workshops
📍 Commonwealth Ballroom B
Full-day workshop, 11 listed talks/papers. TBD • Can LLMs Serve as a Data Error Detection Engine?Full-day workshop, 11 listed talks/papers. TBD • Can LLMs Serve as a Data Error Detection Engine? Trade-offs in Accuracy, Cost, and Hallucination Across Datasets • Assessing Data Quality in Relational Database Migration: The Data Quality Score (DQS) • Measuring Credibility in Social Media Platforms through Data Quality and Provenance • Automatic Consistency Assessment through Partial Functional Dependency Mining • Ad-hoc Data Integration in Data Lakes • Poster session • Scalable Goal-Oriented Source Selection • Improving Data Preparation for CSV Files with LLMs • Towards Inference-Aware Privacy Guidance for Data Preparation • KGpipe: Generation of Pipelines for Data Integration into Knowledge Graphsread more
☆
Mon Aug 31 · 8:45 AM - 5:10 PM
TPCTC · The 18th TPC Technology Conference on Performance Evaluation and Benchmarking
● Workshops
📍 Commonwealth Ballroom C
Full-day workshop, 10 listed talks/papers.Full-day workshop, 10 listed talks/papers. SECBench: A Benchmarking Framework for Stream Processing Systems in the Sensor-Edge-Cloud Continuum • The Reasoning Tax: Token Economics of LLM Reasoning Across Task Types and Deployment Contexts • DataGenX V2: Automated Privacy-Preserving Synthetic Relational Data Generation from Database Statistics • Agentic AI • Intel Linear Tool: Diagnosing Multicore Scaling Bottlenecks Through End-to-End Cache Line Contention Analysis for Microsoft SQL Workloads • From Queries to Trajectories: Benchmarking Closed-Loop Agentic AI Systems • BufBench: A Benchmarking Framework for Buffer Pool Analysis in PostgreSQL • TexBench: Harnessing LLMs for Efficient Key-Value Benchmarking • MLPerf Endpoints: AI Inference Performance Evaluation for the Age of Generative AI • DeepChainBench: A Multi-model Benchmark for Deep Recursive Lineage and Resource Elasticityread more
AIDB · The 7th Applied AI for Database Systems and Applications
● Workshops
📍 Commonwealth Ballroom A
Full-day workshop, 18 listed talks/papers. AIDB welcome • Towards Efficient Semantic Query Processing Engines • Stage-Level Executor Allocation in Apache Spark™ with Cost - Performance Trade-offs • Bespoke-Card: Why Tune When You Can Generate?Full-day workshop, 18 listed talks/papers. AIDB welcome • Towards Efficient Semantic Query Processing Engines • Stage-Level Executor Allocation in Apache Spark™ with Cost - Performance Trade-offs • Bespoke-Card: Why Tune When You Can Generate? Synthesizing Workload-Specific Cardinality Estimators • Generation of Adversarial Queries via Bayesian Optimization • Large Databases Need Small, Open-Weight Language Models • Two Surfaces of Ambiguity: Complementary Detection for Text-to-SQL • Towards Practical Latency SLOs on Cloud Data Warehouses • AI Query Compilation for Unified and Optimized Execution • MLSkip: Data Skipping for ML Filters via Lightweight Metadata • Breaking Database Lock-in: Agentic Regeneration of High Performance Storage Readers for Database Bypass • Selectivity Estimation for Semantic Filters on Image Data • … +6 moreread more
SafeQL: Search-based Refinement for Safe and Efficient LLM-based Text-to-SQL (Geonho Lee, Korea Advanced Institute of Science and Technology et al.) - Large language models (LLMs) have advanced Text-to-SQL by enabling natural language interfaces to databases without task-specific fine-tuning.SafeQL: Search-based Refinement for Safe and Efficient LLM-based Text-to-SQL (Geonho Lee, Korea Advanced Institute of Science and Technology et al.) - Large language models (LLMs) have advanced Text-to-SQL by enabling natural language interfaces to databases without task-specific fine-tuning. However, existing LLM-based systems remain unreliable, often generating SQL queries that are invalid under the database schema, referencing non-existent tables, attributes, functions, or values. Such errors persist because interactions with the database management system (DBMS) are typically limited to error messages, leaving it in a largely passive role during query refinement. This paper proposes SafeQL, \textit{a search-based refinement paradigm that redefines the role of the DBMS as an active guide in the refinement process}. Instead of regenerating entire queries after execution failure, SafeQL interprets DBMS feedback to incrementally repair only the erroneous components. Each refinement step is formulated as a guided search within a \textit{safe query space}, where candidate queries are progressively validated through DBMS execution, thereby converging to an executable query and preventing repeated regeneration of errors. Experiments on the Bird and Spider benchmarks show that SafeQL significantly improves execution accuracy and efficiency compared to regeneration-based methods. • Dial: A Knowledge-Grounded Dialect-Specific NL2SQL System (Xiang Zhang, Shanghai Jiao Tong University et al.) - Enterprises commonly deploy heterogeneous database systems, each of which owns a distinct SQL dialect with different syntax rules, built-in functions, and execution constraints. However, most existing NL2SQL methods assume a single canonical dialect (e.g., SQLite) and struggle to produce queries that are both semantically correct and executable on target engines. Prompt-based approaches tightly couple intent reasoning with dialect syntax, rule-based translators often degrade native operators into generic constructs, and multi-dialect fine-tuning suffers from cross-dialect interference. In this paper, we present Dial, a knowledge-grounded framework for dialect-specific NL2SQL. Dial introduces: (1) a Dialect-Aware Logical Query Planning module that converts natural language into a dialect-aware logical query plan via operator-level intent decomposition and divergence-aware specification; (2) HINT-KB, a hierarchical intent-aware knowledge base that organizes dialect knowledge into (𝑖)a canonical syntax reference, (𝑖𝑖)a declarative function repository, and (𝑖𝑖𝑖)a procedural constraint repository; and (3) an execution-driven debugging and semantic verification loop that separates syntactic recovery from logic auditing to prevent semantic drift. We construct DS-NL2SQL, a benchmark covering six major database systems with 2,218 dialect-specific test cases. Experimental results show that Dial consistently improves translation accuracy by 10.25% and dialect feature coverage by 15.77% over state-of-the-art baselines. The code is at https://github.com/weAIDB/Dial. • NL2SQLBench: A Modular Benchmarking Framework for LLM-Enabled NL2SQL Solutions (Shizheng Hou, National University of Singapore et al.) - Natural Language to SQL (NL2SQL) technology empowers non-expert users to query relational databases without requiring SQL expertise. While large language models (LLMs) have greatly improved NL2SQL algorithms, their rapid development outpaces systematic evaluation, leaving a critical gap in understanding their effectiveness, efficiency, and limitations. To this end, we present NL2SQLBench, the first modular evaluation and benchmarking framework for LLM-enabled NL2SQL approaches. Specifically, we dissect NL2SQL systems into three core modules: Schema Selection, Candidate Generation, and Query Revision. For each module, we comprehensively review existing strategies and propose novel fine-grained metrics that systematically quantify module-level effectiveness and efficiency. We further implement these metrics in a flexible multi-agent framework, allowing configurable benchmarking across diverse NL2SQL approaches. Leveraging NL2SQLBench, we rigorously evaluate ten representative open-source methods on two datasets, the BIRD development set and the ScienceBenchmark development set, using two LLMs, DeepSeek-V3 and GPT-4o mini. We systematically assess each approach across the three core modules and evaluate multiple critical performance dimensions. Our evaluation reveals significant gaps in existing NL2SQL methods, highlighting not only substantial room for accuracy improvements but also the significant computational inefficiency, which severely hampers real-world adoption. Furthermore, our analysis identifies critical shortcomings in current benchmark datasets and evaluation rules, emphasizing issues such as inaccurate gold SQL annotations and limitations in existing evaluation rules. By synthesizing these detailed insights into a unified, transparent, and reproducible benchmarking, our study not only establishes a clear reference point for fair comparison across approaches but also serves as essential guidance for future targeted innovation in NL2SQL technology, thus advancing the practical deployment and real-world applicability of NL2SQL technologies. • Developing and Benchmarking Verification Algorithms to Improve Text-to-SQL Generation (Tarfah Alrashed, Google et al.) - Large Language Models (LLMs) have achieved impressive performance in translating natural language queries into executable SQL. However, these systems remain prone to deceptive failures: generating syntactically valid SQL that executes but fails to capture the user's intent. In this work, we argue that further progress can be made by focusing specifically on verification. In this paper, we formalize Text-to-SQL verification as a standalone task: assessing whether a candidate SQL query semantically satisfies a natural language request, without access to ground-truth labels. We propose and compare two modular verification strategies: Round-Trip Critique, which reverse-translates SQL into natural language to detect semantic drift, and Synthetic Execution Consistency, which uses unit-test-like synthetic inputs to ground verification in execution results. Our evaluation shows that these methods provide a robust signal for identifying incorrect queries, successfully flagging 64% of errors from a state-of-the-art generator and outperforming standard error detectors. We demonstrate two critical applications: (1) auditing foundational benchmarks (Spider, BIRD and KaggleDBQA), revealing that over two-thirds of "generator failures" actually stem from flawed benchmark labels, and (2) enabling Selective Generation, where a system uses verification signals to abstain from answering when confidence is low. Our results show that this paradigm significantly improves the quality of deployed data interfaces by transforming silent failures into explicit abstentions that alert the user to the failure and give them a chance to correct it. • Pervasive Annotation Errors Break Text-to-SQL Benchmarks and Leaderboards (Tengjun Jin, UIUC et al.) - Researchers have proposed numerous text-to-SQL techniques to streamline data analytics and accelerate the development of data-driven applications. To compare these techniques and select the best one for deployment, the community depends on public benchmarks and their leaderboards. Since these benchmarks heavily rely on human annotations during question construction and answer evaluation, the validity of the annotations is crucial. In this paper, we conduct an empirical study that (i) benchmarks annotation error rates for two widely used text-to-SQL benchmarks, BIRD and Spider 2.0-Snow, and (ii) corrects a subset of the BIRD development (Dev) set to measure the impact of annotation errors on text-to-SQL agent performance and leaderboard rankings. Through expert analysis, we show that BIRD Mini-Dev and Spider 2.0-Snow have error rates of 52.8% and 62.8%, respectively. We re-evaluate all 16 open-source agents from the BIRD leaderboard on both the original and the corrected BIRD Dev subsets. We show that performance changes range from −7% to 31% (in relative terms) and rank changes range from −9 to +9 positions. We further assess whether these impacts generalize to the full BIRD Dev set. We find that the rankings of agents on the uncorrected subset correlate strongly with those on the full Dev set (Spearman's 𝑟_𝑠 =0.85, 𝑝 =3.26e-5), whereas they correlate weakly with those on the corrected subset (Spearman's 𝑟_𝑠 =0.32, 𝑝 =0.23). These findings show that annotation errors can significantly distort reported performance and rankings, potentially misguiding research directions or deployment choices. • A Comparative Evaluation of Schema Subsetting for LLM-based NL-to-SQL over Large-Schema Databases (Kyle Luoma, United States Military Academy - Army Cyber Institute et al.) - Large Language Models (LLMs) have become the standard for natural language interfaces to databases, but their effectiveness can be limited by context window constraints, especially for databases with large schemas. Schema subsetting or linking, which is the task of reducing the schema information provided to the LLM, has emerged as a strategy to address these limitations, yet its impact on NL-to-SQL performance remains unclear, particularly for very large schemas. In this paper, we systematically evaluate 7 real-world schema subsetting modules across 3 contemporary NL-to-SQL benchmarks, including Bird, Spider 2, and SNAILS, and we introduce BigBird--an expansion of the Bird benchmark datasets that provides additional data for evaluating subsetting of large schemas. We also introduce new subsetting-specific performance and efficiency metrics that enable in-depth evaluation of subsetting methods. Our analysis aligns with other recent work that suggests that most subsetting methods actually degrade NL-to-SQL execution accuracy from between 3% - 10% (model and method dependent) on smaller schemas, but also reveals that some subsetting methods can improve NL-to-SQL execution accuracy by up to 2% - 7% and others reduce token usage while generally maintaining the same execution accuracy performance as full-schema representations on large schemas. We also present SKALPEL, a prototype hybrid subsetting method that combines LLM-based question decomposition with semantic search, suggesting the potential for reduced token usage in NL-to-SQL workflows. These findings clarify the trade-offs of schema subsetting and motivate future research on scalable schema linking for large databases.read more
☆
Tue Sep 1 · 10:45 AM - 12:15 PM
Research 2 · Cloud-Native Data Systems
● Research
📍 Grand Ballroom D
CloudGlide: Deconstructing the Landscape of Cloud-Based Analytics (Michail Georgoulakis Misegiannis, Technical University of Munich et al.) - Cloud-based analytics now exposes an increasingly vast space of design choices. Key axes include provisioning (static vs.CloudGlide: Deconstructing the Landscape of Cloud-Based Analytics (Michail Georgoulakis Misegiannis, Technical University of Munich et al.) - Cloud-based analytics now exposes an increasingly vast space of design choices. Key axes include provisioning (static vs. ephemeral), caching (capacity, tiering), scheduling (admission thresholds, parallelism), and pricing (reserved, on-demand, spot); each choice materially affects cost and performance. To navigate this complexity without deploying large-scale infrastructure, we present CloudGlide, a white-box simulation framework for systematically exploring cloud data analytics trade-offs. CloudGlide pairs a queueing-theoretic model with a discrete-event simulator (DES), ingesting real-world workload traces to provide cost and latency predictions under diverse configurations. Validated on industry traces and standard benchmarks, CloudGlide approximates behavior across existing architectures and supports rapid what-if analyses along the above axes, all without the prohibitive costs of live deployments. • BtrLog: Low-Latency Logging for Cloud Database Systems (Maximilian Kuschewski, Technische Universität München et al.) - Cloud database systems cannot rely on instance-local disks for write-ahead logging (WAL) durability, forcing WAL onto remote storage. Existing options are unsatisfying: remote block storage like EBS is easy to adopt but adds substantial write latency and cost, while object storage offers excellent durability and low storage cost but is impractical for OLTP due to high latency and per-append cost. Many cloud-native databases, therefore, depend on purpose-built logging backends, which are typically proprietary and tightly coupled to engine-specific replication and recovery protocols, limiting reuse. We present BtrLog, a reusable cloud logging service that combines low-latency durable appends with low-cost archival for the common single-writer architecture. BtrLog replicates log records across a quorum of SSD-backed log nodes in a single network round trip, reducing sensitivity to stragglers in commit latency. To minimize storage cost, log nodes archive records to object storage as large segments, which are written asynchronously and off the latency-critical write path. In our evaluation, BtrLog achieves lower latency than EBS and enables higher end-to-end transaction throughput when integrated into a DBMS. • Bespoke OLAP: Synthesizing Workload-Specific One-size-fits-one Database Engines (Johannes Wehrstein, TU Darmstadt et al.) - Modern OLAP engines support arbitrary analytical workloads, but this flexibility incurs overhead from runtime schema interpretation, generic data representations, and abstraction layers, even in compiled-query systems. Workload-specific engines can eliminate these costs and exploit specialized data structures and algorithms for higher performance, yet have historically been too expensive to build manually. Recent advances in LLM-based code synthesis challenge this tradeoff, but naive prompting does not produce correct or efficient engines due to deep architectural dependencies and the need for systematic refinement. We present Bespoke OLAP, a fully autonomous synthesis pipeline that constructs high-performance OLAP engines tailored to a target workload through iterative performance evaluation and automated validation. Bespoke OLAP generates engines from scratch within minutes to hours and achieves order-of-magnitude speedups over DuckDB and Umbra, demonstrating that the generality tax extends beyond query compilation to storage layout and algorithmic design. • LiquidCache: Efficient Pushdown Caching for Cloud-Native Data Analytics (Xiangpeng Hao, University of Wisconsin Madison et al.) - We present LiquidCache, a novel pushdown-based disaggregated caching system that evaluates filters on cache servers before transmitting data to compute nodes. Our key observation is that data decoding, not filter evaluation, is the primary bottleneck in existing systems. To address this challenge, we transcode Parquet data into a lightweight "Liquid" format and cache it for efficient filter evaluation. The Liquid format resides solely in the cache layer, requiring no changes to existing deployments and enabling easy adoption of new encodings without breaking compatibility. Through integration with Apache DataFusion and evaluation with ClickBench and TPC-H, we demonstrate that LiquidCache reduces cache CPU time by up to 10× without increasing memory footprint, and reduces network traffic by two orders of magnitudes compared to non-pushdown systems. • Redbench: Workload Synthesis From Cloud Traces (Johannes Wehrstein, TU Darmstadt et al.) - Workload traces from cloud data warehouse providers reveal that standard benchmarks such as TPC-H and TPC-DS fail to capture key characteristics of real-world workloads, including query repetition and string-heavy queries. In this paper, we introduce Redbench, a novel benchmark featuring a workload generator that reproduces real-world workload characteristics derived from traces released by cloud providers. Redbench integrates multiple workload generation techniques to tailor workloads to specific objectives, transforming existing benchmarks into realistic query streams that preserve intrinsic workload characteristics. By focusing on inherent workload signals rather than execution-specific metrics, Redbench bridges the gap between synthetic and real workloads. Our evaluation shows that (1) Redbench produces more realistic and reproducible workloads for cloud data warehouse benchmarking, and (2) Redbench reveals the impact of system optimizations across four commercial data warehouse platforms. We believe that Redbench provides a crucial foundation for advancing research on optimization techniques for modern cloud data warehousesread more
☆
Tue Sep 1 · 10:45 AM - 12:15 PM
Research 3 · Data Discovery over Data Lakes
● Research
📍 Grand Ballroom C
Matryoshka: Uncovering Relevant Features in Data Lakes to Enhance Machine Learning Applications (Fedor Turchenko, BIFOLD & TU Berlin et al.) - Data coverage and completeness are crucial data quality dimensions for machine learning (ML) applications.Matryoshka: Uncovering Relevant Features in Data Lakes to Enhance Machine Learning Applications (Fedor Turchenko, BIFOLD & TU Berlin et al.) - Data coverage and completeness are crucial data quality dimensions for machine learning (ML) applications. Data discovery allows retrieving relevant features from unstructured data lakes to augment the training data. However, existing methods are fragmented: they often target only individual ML tasks, decouple table retrieval from feature selection, and neglect inter-feature relationships, resulting in redundant, high-dimensional data. In this paper, we introduce Matryoshka, a holistic system that discovers and selects relevant features from data lakes for downstream ML applications. Matryoshka replaces costly join materialization with compact Gram matrix sketches and performs feature selection via incrementally fitted linear proxy models with correlation-based pruning, yielding redundancy-free feature sets for regression and classification. We evaluate Matryoshka on multiple data lakes of varying size and heterogeneity. Matryoshka improves downstream prediction quality by 18.5% on average while achieving the lowest geometric mean runtime and up to 120x faster execution on join-intensive workloads. • FedAugment: Table Augmentation Search over Decentralized Data Repositories (Lennart Behme, Technische Universität Berlin et al.) - Dataset search often aims to identify joinable or unionable datasets to augment a given query table. State-of-the-art approaches rely on large language models (LLMs) to embed tables into vector representations and perform semantic similarity search. However, existing work assumes a centralized data repository with embeddings generated by a single, homogeneous pipeline. In contrast to this simplifying assumption, data repositories in the real world are decentralized across multiple data providers, each operating their own embedding pipelines. Given the rapid pace of LLM development and provider-specific fine-tuning, enforcing a standardized pipeline is unrealistic. We introduce FedAugment, a framework for table augmentation search over decentralized data repositories with heterogeneous embeddings. FedAugment constructs a representative set of training examples, embeds it using the individual providers' pipelines, and learns projection functions that align heterogeneous embeddings into a shared vector space via multi-view contrastive learning. Using these projections, all embeddings are mapped into a globally aligned space that supports unified vector similarity search. Compared to issuing independent top-k queries to each data provider, FedAugment enables the retrieval of a global top-k result across all repositories, avoiding redundant retrievals and enabling cost-efficient table augmentation search in decentralized settings. • MosaicJoin: Compact Semantic Sketches for Value-Level Join Discovery (Grace Fan, New York University et al.) - Join discovery is a core task in data discovery, enabling data scientists to find columns that can be joined with a given query column. Early approaches focused on equi-joins, but data lakes and open-data repositories often contain columns whose values refer to the same entity but use different syntactic representations. To address this challenge, recent approaches discover semantically joinable columns but face a fundamental trade-off: methods that perform value-level comparisons accurately identify joinable columns but scale poorly to columns with high cardinality; column-level methods that encode an entire column into a single embedding are efficient but do not capture the fine-grained value alignment that determines whether a join is possible. We present MosaicJoin, a value-level semantic join discovery method that balances this tradeoff. MosaicJoin achieves scalability through a novel sketching strategy that approximates the joinability of a column pair without having to compare all values. At query time, MosaicJoin scores each candidate sketch using a joinability score at a cost bounded by the sketch size, making retrieval efficient even for high-cardinality columns. A query subsampling operator further reduces online search time with provable accuracy guarantees, enabling robust retrieval for large query columns. Extensive experiments show that MosaicJoin outperforms previously published methods across all benchmarks while running up to 66 times faster than other value-level methods. MosaicJoin requires no training or fine-tuning, and it scales robustly to query columns containing up to 57K values and data lake columns containing up to 1M values. • IncreQueryFusion: On-demand Data Fusion Framework in Dynamic Data Lakes (Wenhao Liu, Zhejiang University et al.) - In data lakes, efficient data fusion is essential yet challenging in dynamic scenarios due to heterogeneous multi-source data. Batch-based methods ensure high accuracy but suffer from high latency, while emerging on-demand fusion approaches, despite offering real-time responsiveness, lack native support for incremental scenarios and often struggle with query evidence constraints. To overcome these issues, we propose IncreQueryFusion, an end-to-end on-demand fusion framework tailored for dynamic data lakes. First, it introduces a Temporal Index Evidence Retrieval module to incrementally retrieve and selectively expand evidence for subsequent fusion. Then, an adaptive aligned hierarchical fusion mechanism organizes data values and performs evidence-aware truth inference. Finally, a dynamic fusion truth maintenance algorithm refreshes fusion results efficiently without full recomputation. A comprehensive evaluation of established benchmarks and our newly constructed dynamic datasets demonstrates the effectiveness of IncreQueryFusion. It outperforms the state-of-the-art baseline in accuracy by 5\%-30\% on static workloads, while achieving speedups of over 10$\times$ against on-demand methods and over 100$\times$ against batch methods in dynamic scenarios. • Discovering Approximate Denial Constraints in Large Databases (Albert Martin, Universitat Politècnica de Catalunya et al.) - Denial Constraints (DCs) form a highly expressive integrity rule language that subsumes many used formalisms such as keys and functional dependencies, making them widely adopted in applications that require the manipulation of rich sets of data constraints. This expressiveness has motivated the development of numerous algorithms for automatically discovering DCs from data, with particular emphasis on the discovery of approximate DCs to improve robustness to erroneous data. However, existing DC discovery algorithms exhibit computational costs that are quadratic in the number of tuples and exponential in the number of attributes, and most cannot accommodate changes in the data. Moreover, they often produce thousands of uninformative DCs. These limitations make current DC discovery algorithms difficult to use effectively on very large and dynamic databases. In this paper, we present LIMA, an approximate DC discovery algorithm that efficiently discovers DCs on very large and dynamic databases. LIMA uses statistical methods to infer properties of DCs from reduced samples, and introduces a novel discovery framework that exploits a more restrictive definition of DC validity to substantially reduce the cost of searching for valid DCs. We experimentally demonstrate that LIMA achieves significantly better scalability than current algorithms with respect to both rows and attributes, while also discovering higher-quality sets of DCs with precisions several orders of magnitude higher than the state of the art, both in static and in dynamic datasets.read more
☆
Tue Sep 1 · 10:45 AM - 12:15 PM
Research 4 · Approximate Query Processing
● Research
📍 Commonwealth Ballroom C
BaCon: Efficient Batch Processing of Counting Queries (Yuxi Liu, Duke University et al.) - Counting queries are ubiquitous in database systems, particularly for driving internal system optimization.BaCon: Efficient Batch Processing of Counting Queries (Yuxi Liu, Duke University et al.) - Counting queries are ubiquitous in database systems, particularly for driving internal system optimization. Learned models for cardinality estimation rely heavily on large-scale training data, yet generating such data by executing massive batches of counting queries is expensive. We propose BaCon, an efficient algorithm for batch evaluation of counting queries on top of a database system, without modifying its internals. BaCon integrates the idea of factorized databases with a workload-aware domain quantization strategy, allowing it to evaluate batches of counting queries using compact data structures rather than materializing massive join results. BaCon's design is compatible with most database management system, and we have implemented it as a client-side application on PostgreSQL with a lightweight C-language UDF (user-defined function). This implementation delivers speedups between 2$\times$ and 178$\times$ over baselines and good performance across various workloads, making training and maintenance of learned cardinality estimation models significantly more practical. • Finding Non-Redundant Simpson's Paradox in Multidimensional Data (Yi Yang, Duke University et al.) - Simpson's paradox has broad impact across many scientific domains. Existing detection methods overlook a key issue: many detected paradoxes may be redundant, arising from equivalent data subsets, identical subpopulation partitions, or correlated outcome variables, thereby obscure insights and increase computational cost. In this paper, we present a framework for finding non-redundant Simpson's paradoxes by formalizing three sources of redundancy - sibling child, separator, and statistic equivalence - and showing that pairwise redundancy forms an equivalence relation. We further propose a concise representation that groups redundant paradoxes and develop efficient algorithms combining depth-first population materialization with redundancy-aware discovery. Experiments on real and synthetic datasets show that redundancy is prevalent (over 40% in some cases), while our methods scale to millions of records, achieve up to 6.72 times speedup over brute-force approaches and identify robust paradoxes, enabling efficient discovery, compact summarization, and clear interpretation in multidimensional data. • Scalable Grid-based Computation of Kendall's Tau Correlation (Nikolaos Koutroumanis, Archimedes, Athena RC et al.) - Computing the correlation of two attributes in a large dataset is an important problem, with many applications, including exploratory analytics and dimensionality reduction. Among the well-known correlation measures, Kendall's Tau is the most robust one, as it is immune from parametric assumptions and outliers. On the other hand, computing Kendall's Tau for large-scale data becomes challenging (i) due to the superlinear cost of the state-of-the-art algorithm and (ii) because all data need to be memory-resident for efficient processing. In this paper, we address the problem via a geometric approach that partitions the data in the cells of a grid, and exploits the relative position of the cells to compute correlation information en masse. Our approach facilitates parallel and distributed computation of Kendall's correlation; we propose a scalable algorithm in this direction. Finally, we propose an efficient approximate algorithm with a provable error bound, which derives accurate results by a single pass over the grid statistics. Our experimental evaluation demonstrates the efficiency and scalability of our grid-based techniques compared to the state-of-the-art algorithm. • Optimal Approximate Matrix Multiplication over Sliding Windows (Haoming Xian, The Chinese University of Hong Kong et al.) - Matrix multiplication is a core operation in numerous applications, yet its exact computation becomes prohibitively expensive as data scales, especially in streaming environments where timeliness is critical. In many real-world scenarios, data arrives continuously, making it essential to focus on recent information via sliding windows. While existing approaches offer approximate solutions, they often suffer from suboptimal space complexities when extended to the sliding-window setting. In this work, we introduce SO-COD, a novel algorithm for approximate matrix multiplication (AMM) in the sliding-window streaming setting, where only the most recent data is retained for computation. Inspired by frequency estimation over sliding windows, our method tracks significant contributions, referred to as ``snapshots'', from incoming data and efficiently updates them as the window advances. Given matrices \(\boldsymbol{X} \in \mathbb{R}^{d_x \times n}\) and \(\boldsymbol{Y} \in \mathbb{R}^{d_y \times n}\) for computing \(\boldsymbol{X} \boldsymbol{Y}^T\), we analyze two data settings. In the \emph{normalized} setting, where each column of the input matrices has a unit \(L_2\) norm, SO-COD achieves an optimal space complexity of \( O\left(\frac{d_x+d_y}{\epsilon}\right) \). In the \emph{unnormalized} setting, where the square of column norms vary within a bounded range \([1, R]\), we show that the space requirement is \( O\left(\frac{d_x+d_y}{\epsilon}\log R\right) \), which matches the theoretical lower bound for an \(\epsilon\)-approximation guarantee. Time complexity analysis further shows that our SO-COD has comparable update time and superior query time compared to existing methods. Extensive experiments on synthetic and real-world datasets demonstrate that SO-COD effectively balances space cost and approximation error, making it a promising solution for large-scale streaming matrix multiplication. • ConRAD: Conformal Risk-Aware Neural Databases (Sonia Horchidan, KTH Royal Institute of Technology et al.) - Querying incomplete knowledge graphs with neural predictors is powerful but dangerous. Errors compound across multi-hop pipelines with no formal bound on the completeness of results. We introduce ConRAD, the first framework to enforce declarative marginal recall guarantees natively within a neural graph database query engine. Given a user-specified risk budget, ConRAD automatically derives per-operator prediction thresholds that satisfy the recall target in expectation over the query distribution, with finite-sample, distribution-free statistical validity via Conformal Risk Control, while maximizing end-to-end precision. To scale calibration across multi-operator query topologies, we introduce a quantile-space scalarization that reduces intractable high-dimensional threshold searches to a single parameter. We further design the conformal gate, a novel physical operator that dynamically bypasses neural inference when local graph evidence suffices, eliminating unnecessary model inferences in dense graph regions. Evaluated across three benchmarks and eight query topologies, ConRAD satisfies all risk budgets, with empirical recall falling below the target by at most 0.0547 across all settings. It reduces neural invocations to zero in near-complete graph regions, and achieves precision that matches or exceeds best-case static baselines that offer no guarantees and require manual threshold search.read more
☆
Tue Sep 1 · 10:45 AM - 12:15 PM
Research 5 · Storage Engines
● Research
📍 Commonwealth Ballroom B
A Resource-centric Analysis and Optimization of NoSQL Workloads using Distressed Resource Volume Metric (Gunika Verma, Microsoft et al.) - Large-scale managed cloud databases leverage sophisticated load Packing and Migration (PAM) algorithms, which provide the efficiencies necessary for running these services...A Resource-centric Analysis and Optimization of NoSQL Workloads using Distressed Resource Volume Metric (Gunika Verma, Microsoft et al.) - Large-scale managed cloud databases leverage sophisticated load Packing and Migration (PAM) algorithms, which provide the efficiencies necessary for running these services at scale on cloud resources. Research into optimizing the resources and reliability of cloud databases at massive scales is limited by a lack of public NoSQL workloads. We address this in the context of Cosmos DB, Microsoft’s flagship cloud-hosted NoSQL database. We first propose open-source NoSQL workloads from real Cosmos DB clusters, and analyze these traces to derive a novel reliability metric, Distressed Resource Volume (DRV), which captures the quality of service experienced by the end user. We then develop an opensource policy simulation framework, LoadStar, powered by a nonparametric statistical model of estimating the QoS of real traffic patterns. These form a reusable benchmark pipeline for validating policies for resource-centric NoSQL workloads. We then define a resource optimization problem for placing Cosmos DB replicas onto VM nodes, develop the Luna model for forecasting future load distributions, and the Orbit PAM algorithm that uses these forecasts to trigger and rebalance stressed replicas, to reduce tail-errors. Our experiments, validated using LoadStar for these workloads, demonstrate Orbit's benefits over the existing Cosmos DB policy and a worst-fit optimized baseline, with higher load delivered at lower error rates and up to 35% reduction in resources. These have been deployed in production, with potential savings of $100M's/yr while improving service reliability for millions of customers. • Shard: A Scalable and Resize-optimized Hash Index on Disaggregated Memory (Hantian Zha, Renmin University of China et al.) - Disaggregated memory (DM) separates memory and computing resources into distinct pools, improving resource utilization, scalability, and data sharing in data centers and cloud environments. These systems utilize RDMA-capable networks, which provide high throughput and low latency, making them well suited for high-performance indexing in data storage systems. However, existing DM-optimized hash indexes face significant challenges in achieving the one RTT goal due to excessive remote read/write accesses, correctness issues in concurrent operations, high latency during resizing, and costly multi-node synchronization. This paper addresses these challenges by introducing a novel architecture called Shard, designed to enhance the performance of hash indexes in disaggregated memory. We leverage the structure of Iceberg Hashing to ensure that each key is mapped to fewer buckets. We propose the Ordered-CAS technique to minimize read/write accesses and ensure correctness when handling duplicate keys. To address the trade-offs between resizing and synchronization, we adopt a lazy-resizing strategy and propose the RDMA-combining and adaptive frequency synchronization (AFS) techniques. We implement Shard and conduct a comprehensive evaluation on DM. The results show that Shard outperforms state-of-the-art DM-optimized hash indexes by at most 6.7× (RACE), 3.6× (SepHash), and 1.8× (Outback) in YCSB workloads, respectively. • CIDER: Boosting Memory-Disaggregated Key-Value Stores with Pessimistic Synchronization (Yuxuan Du, Fudan University et al.) - Memory-disaggregated key-value (KV) stores suffer from a severe performance bottleneck due to their I/O redundancy issues. A huge amount of redundant I/Os are generated when synchronizing concurrent data accesses, making the limited network between the compute and memory pools of DM a performance bottleneck. We identify the root cause for the redundant I/O lies in the mismatch between the optimistic synchronization of existing memory-disaggregated KV stores and the highly concurrent workloads on DM. In this paper, we propose to boost memory-disaggregated KV stores with pessimistic synchronization. We propose CIDER, a compute-side I/O optimization framework, to verify our idea. CIDER adopts a global write-combining technique to further reduce cross-node redundant I/Os. A contention-aware synchronization scheme is designed to improve the performance of pessimistic synchronization under low contention scenarios. Experimental results show that CIDER effectively improves the throughput of state-of-the-art memory-disaggregated KV stores by up to 6.6× under the YCSB benchmark. • Efficient, Scalable, and Fair Locking on Disaggregated Memory with Decentralized Coordination (Hanze Zhang, Shanghai Jiao Tong University et al.) - Databases on disaggregated memory (DM) rely heavily on locking for concurrency control. However, we find that under contention, existing lock implementations can significantly degrade database performance because they overload the network interface controllers (NICs) of memory nodes (MNs) and provide poor fairness among competing clients on compute nodes (CNs). This paper presents DecLock, an efficient, scalable, and fair locking mechanism for DM. DecLock decouples centralized state maintenance on MNs from decentralized ownership transfer across CNs. Its cooperative queue-notify locking atomically queues waiters on MNs and then transfers lock ownership through direct message-based notifications between CNs, rather than repeated retries to MNs. This design preserves precious MN-NIC resources for data access while ensuring fair lock handoff. Evaluations show that DecLock improves throughput by up to 43.37x, 4.35x, and 1.81x over state-of-the-art RDMA-based spinlock, ticket lock, and MCS lock, respectively. Moreover, DecLock helps a NoSQL data store, a transaction engine, and a real-world database index avoid severe performance degradation under high contention, improving throughput by up to 1.48x, 1.59x, and 2.31x over prior solutions, respectively. • Breaking the Isolation-Freshness Trade-off: Joint Adaptive Storage Optimization for HTAP Systems (Zhenghao Ding, Renmin University of China et al.) - HTAP systems aim to support large-scale transaction processing while preserving real-time analytics over fresh operational data. Achieving this dual goal requires carefully ensuring workload isolation and data freshness. However, this is a challenge that existing systems often struggle to meet, as they rely on static, coarse-grained storage configurations, such as duplicating full data across dual storage model (row and column store) or storing data in a single model. These incur excessive synchronization overhead that degrades data freshness or compromises workload isolation under mixed workloads. We present Jasper, a joint adaptive storage mechanism that dynamically configures fine-grained storage layouts based on workload characteristics. Jasper performs workload-aware horizontal and vertical partitioning and selectively materializes column store replicas for update-sparse, query-intensive partitions. This design ensures strong workload isolation while minimizing unnecessary data redundancy, significantly reducing synchronization overhead and improving data freshness. We implement Jasper in TiDB and conduct extensive evaluations using both standard benchmarks and a real-world TiDB production workload. Extensive evaluations on both benchmarks and real-world TiDB production workloads show that Jasper cuts workload completion time by 20.43% - 40.59%, delivering state-of-the-art performance in balancing isolation and freshness for HTAP systems. • TVA: A Version-aware Temporal Graph Storage System for Real-time Analytics (Wenhao Li, Renmin University of China et al.) - Analyzing temporal graphs can reveal valuable insights that are typically hidden in static graphs. Unfortunately, existing graph storage systems either lack native temporal support or suffer from high latency when querying temporal graphs. This paper presents TVA, a new temporal graph storage system designed for efficient temporal query processing. First, TVA introduces a specialized multi-version storage architecture that separates version metadata from actual data, i.e., the property values associated with different versions of vertices and edges. This architecture enables efficient version retrieval for a vertex or edge by quickly locating valid version metadata and directly dereferencing it to access the corresponding property values. Second, we design tailored data structures, namely the temporal table and enhanced hopscotch-based hashing, to compactly organize the version metadata of adjacent vertices and edges, thus reducing random I/O for metadata lookups during the neighborhood scan initiated from a vertex. Finally, to further accelerate neighborhood scans over multiple vertices, we propose a version-kipping strategy that reuses temporal information obtained from prior scans, thereby avoiding redundant metadata lookups across scans. Empirical evaluations demonstrate that TVA achieves up to 9.9×lower temporal query latency and 2.2×lower storage overhead compared to state-of-the-art temporal graph storage systems. • Kirin: Efficient In-Storage Learned Compaction for LSM-Trees via System-Algorithm Co-Design (guifeng wang, Shanghai Jiao Tong University et al.) - The log-structured merge-trees (LSM-trees) are widely used in modern Key-Value (KV) stores, offering strong write performance but facing significant inefficiencies in compaction and indexing. While recent researches have integrated learned indexes with LSM-trees to address these inefficiencies, their integration remains hindered by excessive cold data movement, limited parallelism in model training, and the decoupled nature of compaction and training. In this paper, we present Kirin, a hybrid KV store that synergistically integrates LSM-tree and learned index, and leverages computational storage devices (CSDs) to offload data-intensive tasks. Kirin introduces a novel learned compaction approach that embeds model training directly into the compaction process to conceal training latency and enable timely model updates. Kirin also employs a collaborative approach between the host and CSD to parallelize compaction and minimize storage access during indexing. Our experiments with DaisyPlus OpenSSD demonstrate that Kirin outperforms existing solutions in both read and write throughput by a large margin, while maintaining low read latency under heavy write workloads. • Sampling-based Predictive Database Buffer Management (Mohammad Khalaji, University of Waterloo et al.) - Systems often need to support analytical (OLAP) workloads that perform concurrent scans of data on secondary storage. The buffer manager is tasked with fetching data into the database system's buffer pool and caching it there so as to increase the hit rate on these data, thereby lowering query latencies. This paper presents a database buffer caching policy that uses information about long-running scans to estimate future accesses. These estimates are used to approximate an optimal buffer caching policy that would otherwise infeasibly require knowledge about future accesses. Since a buffer caching policy must be efficient with low overhead, we present sampling-based predictive buffer management techniques where buffer eviction considers only a small random sample of buffers and access time estimates are used to select from the sample. This design is advantageous as it is easily tuned by adjusting the sample size, and easily modified to improve access time estimates and to expand the set of workload types that can be predicted effectively. We evaluate our techniques through both simulation studies on real Amazon Redshift workload traces and through implementation into the well-known open-source PostgreSQL database system on the popular TPC-H and YCSB benchmarks. We show that our approach delivers substantial performance improvements for workloads with scans, reducing I/O volume significantly by up to 40% over PostgreSQL's Clock-sweep policy and over prior predictive approaches for workloads using sequential scans and index accesses.read more
☆
Tue Sep 1 · 10:45 AM - 12:15 PM
Research 6 · Graph Queries and Analytics
● Research
📍 Commonwealth Ballroom A
Aquila: A High-Concurrency System for Incremental Graph Query (ziqi zou, Beijing Institute of Technology et al.) - Incremental querying of multiple concurrent patterns in dynamic graphs is essential for various real-world applications.Aquila: A High-Concurrency System for Incremental Graph Query (ziqi zou, Beijing Institute of Technology et al.) - Incremental querying of multiple concurrent patterns in dynamic graphs is essential for various real-world applications. However, existing solutions face two limitations, particularly in multi-core architecture. First, performance isolation deteriorates under concurrent queries due to coarse-grained scheduling strategies, where long-running queries block shorter ones. Second, these approaches struggle with generating high-quality query plans for multi-query graphs efficiently. To address these limitations, we introduce Aquila, a high-concurrency system designed for efficient multi-query processing in dynamic graphs on multi-core. First, Aquila decouples concurrent queries into a combination of operators with specific functionalities, and these operators transmit intermediate results to each other, forming a matching flow. Operator-level workload and resource scheduling strategies are employed to achieve performance isolation. Second, Aquila adopts the matching tree to represent the query plan. A greedy algorithm is designed to construct matching trees by jointly extracting common subgraphs and generating an efficient matching order, enhanced by subgraph relation optimizations with the subgraph relation graph. Extensive experiments demonstrate that Aquila outperforms existing approaches by 1-3 orders of magnitude in real-time query metrics. • CEMR: An Effective Subgraph Matching Algorithm with Redundant Extension Elimination (Linglin Yang, Peking University et al.) - Subgraph matching is a fundamental problem in graph analysis with a wide range of applications. However, due to its inherent NP-hardness, enumerating subgraph matches efficiently on large real-world graphs remains highly challenging. Most existing works adopt a depth-first search (DFS) backtracking strategy, where a partial embedding is gradually extended in a DFS manner along a branch of the search trees until either a full embedding is found or no further extension is possible. A major limitation of this paradigm is the significant amount of duplicate computation that occurs during enumeration, which increases the overall runtime. To overcome this limitation, we propose a novel subgraph matching algorithm, CEMR. It incorporates two techniques to reduce duplicate extensions: common extension merging, which leverages a black-white vertex encoding, and common extension reusing, which employs common extension buffers. In addition, we design two pruning techniques to discard unpromising search branches. Extensive experiments on real-world datasets and diverse query workloads demonstrate that CEMR outperforms state-of-the-art subgraph matching methods. • Efficient Temporal Subgraph Management: A New Interval Index (Dian Ouyang, Guangzhou University et al.) - Many research efforts have been conducted to mine various substructures in temporal graphs. Given a set of temporal subgraphs and an arbitrary time window, we aim to design an index structure to efficiently retrieve all subgraphs contained in (sub-valid) or containing (super-valid) the window. The problem falls in the category of fundamental interval range queries studying the relationship between a set of intervals and a query interval. We propose a novel data structure that is tailored for real-world temporal subgraphs with high volumes, great overlaps, and frequent updates. We design a lightweight linear size index structure with a linear index construction time. The index enables us to answer queries in near optimal time. We also propose algorithms to maintain the index. Our running time to insert a subgraph is bounded by the size of the changed values in the index, which is optimal in the context. Deleting a subgraph takes constant time. Experiments on real-world datasets with numerous subgraph instances demonstrate our significant advantages compared with existing baselines. • TRIM: An Efficient Framework for Exact Eccentricity Computation on Large-Scale Graphs (Dian Ouyang, Guangzhou University et al.) - In graph theory, the eccentricity of a vertex quantifies its centrality by measuring the maximum distance to any other vertex in the graph. This metric underpins important graph properties such as the diameter (maximum eccentricity) of the graph, which is defined by the minimum and maximum centrality values across all vertices. Due to the substantial time overhead caused by full-graph BFS traversals, researchers have focused on incorporating bounding techniques to accelerate algorithm execution. However, the state-of-the-art approach is unable to identify useless vertices and fails to terminate during the search since its bound update relies on complete traversals. In this paper, we propose a novel framework that uses vertex dominance to identify redundant vertices and introduces a new rule to ensure correct termination after skipping a vertex. In addition, we adopt a merging strategy to reduce the number of traversals. Our method achieves up to two orders of magnitude speedup in runtime compared to the state-of-the-art approach, while efficiently handling graph data at the 100-million scale. • GPU-Accelerated 𝜂-threshold Decomposition for Uncertain Graphs (Yu Chen, Zhejiang University et al.) - The 𝜂-threshold decomposition in uncertain graphs, which calculates the 𝜂-thresholds for each vertex, is a fundamental problem in graph analysis. However, the current CPU-based peeling algorithm suffers from prohibitive computational costs, making it infeasible for time-sensitive applications such as fraud detection and dynamic public opinion monitoring. To address this, we introduce Gatd, the first GPU-accelerated framework for 𝜂-threshold decomposition, co-designed with GPU architecture to enable efficient parallelization. Given that the problem is a computationally intensive per-vertex task dominated by probability computation, thereby constraining efficiency, Gatd incorporates three enhancement modules: (i) Redundancy reduction through lower-bound pruning, leveraging safety thresholds from prior iterations, and batch updating of vertices sharing the same 𝜂-threshold; (ii) Adaptive parallelization utilizing dynamically sized thread collaboration groups and hybrid scheduling to match computational resources with dynamic workloads; and (iii) Three-stage load balancing based on neighbor grouping, work stealing, and hierarchical merging to mitigate supernode-induced imbalance. Extensive experiments on diverse uncertain graphs demonstrate that the optimized Gatd achieves speedups of up to four orders of magnitude over state-of-the-art CPU-based methods and existing GPU-based graph processing frameworks, facilitating efficient decomposition even for large-scale networks. • Characterizing Parallel Subgraph Matching Performance: A Systematic Study of Interactions, Scalability, and Enumeration (Tao Yu, Fudan University et al.) - Subgraph matching is a fundamental yet NP-hard problem in graph algorithms. Modern multi-core shared-memory architectures present substantial opportunities to accelerate subgraph matching through parallelism. However, while several parallel subgraph matching algorithms have been proposed, it warrants a systematic empirical study to evaluate: (1) the interaction effect of different parallel strategies, (2) their scalability, (3) underlying performance factors, and (4) the potential for efficiently parallelizing existing sequential algorithms. In this paper, we present a comprehensive study of parallel subgraph matching by analyzing three key components: task splitting, task scheduling, and match enumeration. To investigate their interplay, we evaluate 100 feasible combinations of representative techniques for each component. We further assess scalability across varying thread counts and explore performance variations under diverse query and data graph characteristics. • Balancing the Blend: An Experimental Analysis of Trade-offs in Hybrid Search (Mengzhao Wang, Zhejiang University et al.) - Hybrid search, the integration of lexical and semantic retrieval, has become a cornerstone of modern information retrieval systems, driven by demanding applications like RAG. The design space for these systems is complex, yet a systematic understanding of the trade-offs among their retrieval paradigms, combination schemes, and re-ranking methods is still lacking. To address this, we present the first experimental analysis of advanced hybrid search architectures. Our framework integrates four retrieval paradigms - full-text search, sparse vector search, dense vector search, and tensor search - and evaluates their combinations and re-ranking strategies across 11 real-world datasets. Our results reveal three key findings: (1) A “weakest link” phenomenon, where a weak path can substantially degrade overall accuracy, highlighting the need for path-wise quality assessment before fusion. (2) A data-driven map of performance trade-offs, demonstrating that optimal configurations depend heavily on resource constraints and data characteristics, precluding a one-size-fits-all solution. (3) The identification of tensor-based re-ranking fusion as an alternative to mainstream fusion methods, offering the semantic power of tensor search at a fraction of the computational and memory cost. Our findings offer concrete guidelines for designing adaptive, scalable hybrid search systems and identify key directions for future research. • RT-RkNN: Reverse k Nearest Neighbor Queries as a Graphics Ray Casting Problem (Zhengyang Bai, RIKEN et al.) - Reverse k nearest neighbor (RkNN) queries are fundamental in spatial databases, location-based analytics, and recommendation systems. Existing state-of-the-art techniques rely on spatial pruning supported by R-trees and their variants. However, their pruning effectiveness degrades significantly in challenging scenarios where the number of facilities is small, the user population is dense, or the value of k is large. To overcome these limitations, we formulate the RkNN query in two-dimensional geometric spaces as a graphics ray casting problem, in which users are modeled as rays and facilities are represented as geometric primitives. Based on this formulation, we design the first algorithm and provide an implementation that exploit dedicated hardware ray tracing cores on modern GPUs. This new approach preserves strong filtering performance even for large values of k, dense user populations, and highly sparse facility distributions. Extensive experimental results demonstrate that our method outperforms state-of-the-art algorithms in diverse settings, especially in scenarios where traditional pruning strategies become inefficient.read more
☆
Tue Sep 1 · 10:45 AM - 12:15 PM
Tutorial 1 · Hardware Acceleration for Spatial Databases: Building Next-Generation Query Engines Using Ray Tracing Cores
● Tutorials
📍 Stone
Hardware Acceleration for Spatial Databases: Building Next-Generation Query Engines Using Ray Tracing Cores
Rubao LeeThe Ohio State University
Liang GengThe Ohio State University
Xiaodong ZhangThe Ohio State University
☆
Tue Sep 1 · 10:45 AM - 12:15 PM
Tutorial 2 · Instance-Optimal Acyclic Joins: From Theory to Systems
● Tutorials
📍 Webster
Instance-Optimal Acyclic Joins: From Theory to Systems
Paraschos KoutrisUniversity of Wisconsin--Madison
Stijn VansummerenUHasselt, Data Science Institute
Qichen WangNanyang Technological University
Yisu Remy WangUniversity of California, Los Angeles
Xiangyao YuUniversity of Wisconsin--Madison
☆
Tue Sep 1 · 10:45 AM - 12:15 PM
Industry Session 1 · Query Processing, Optimization & Tuning
● Industry
📍 Marina Ballroom I
Incremental Query Optimizer Statistics in Amazon Redshift (Pascal Pfeil, AWS et al.) - Accurate optimizer statistics are fundamental to query and ML-prediction performance in modern database systems, yet maintaining them poses a significant challenge for large-scale data warehouses.Incremental Query Optimizer Statistics in Amazon Redshift (Pascal Pfeil, AWS et al.) - Accurate optimizer statistics are fundamental to query and ML-prediction performance in modern database systems, yet maintaining them poses a significant challenge for large-scale data warehouses. Traditional statistics collection relies on full table scans, which become prohibitively expensive as tables grow to billions of rows and beyond. This creates a critical tension: statistics must be kept current to ensure high-quality query plans and accurate ML predictions, but the cost of collecting them grows with data volume. Amazon Redshift, a fully managed, petabyte-scale cloud data warehouse, exemplifies this challenge as customers continuously ingest data from sources such as application logs, IoT telemetry, clickstream data, or use zero-ETL to ingest transactional and operational data. Incremental statistics collection addresses this challenge by updating statistics based solely on modified data, avoiding full table scans while maintaining accuracy. In this paper, we present the design and implementation of incremental statistics collection and maintenance based on data sketches in Amazon Redshift. This technique reduced the fleet-wide weekly compute time spent on collecting statistics for large tables by 40%, while producing statistics that are as accurate as, or even more accurate than before. Our experiences offer practical insights for database practitioners seeking to adopt incremental statistics in production systems at scale, particularly those employing PostgreSQL-style optimizer statistics. • AutoLiquid: Autonomic Data Layout Optimization for the Databricks Lakehouse (Supun Nakandala, Databricks et al.) - Data layout recommendation is a classic problem in database literature, yet the downstream challenge of autonomously applying these changes in live production systems is less frequently studied. In Lakehouse architectures, selecting the right clustering key is critical for data skipping performance, but remains a manual process that cannot scale to the hundreds of millions of tables managed by platforms like Databricks. We present AutoLiquid, an autonomic system that solves this by continuously selecting, verifying, and applying clustering keys for Liquid Clustering tables, requiring only a single "CLUSTER BY AUTO" declaration from the user. To navigate the tension between adapting to unpredictable workloads and maintaining strong performance guarantees against regressions, AutoLiquid leverages the decoupled storage and compute architecture of the Lakehouse. It utilizes lightweight heuristics over collected scan telemetry to identify promising candidate keys, followed by a fast shadow verification process that measures actual pruning benefits on sampled data. This "verify before commit" approach ensures that only decisions with demonstrated improvement are applied. We present the algorithm and end-to-end verification flow, providing empirical results using real customer workload data. When compared to customer-selected clustering keys, AutoLiquid achieves similar or better performance on over 95% of evaluated tables. AutoLiquid is currently deployed in production at Databricks, managing clustering keys for millions of tables. • Evaluating the Practical Effectiveness of LLM-Driven Index Tuning on Microsoft SQL Server (Xiaoying Wang, Microsoft et al.) - Indexes are crucial for database performance. Index tuning, i.e., selecting appropriate indexes for a database workload, is an important problem. The state-of-the-art index tuning tools in the industry, e.g., Database Tuning Advisor (DTA) developed for Microsoft SQL Server, rely on a ``what-if'' API, which can estimate the cost of a query for a given index configuration. They take as input a SQL workload and constraints such as a storage bound, and search over the large space of index configurations to find one with low optimizer-estimated cost for the input workload. Large language models (LLMs) offer a different approach to index tuning, using knowledge they have learned from publicly available training data. However, the effectiveness of LLM-driven index tuning in comparison to today's index advisors, particularly on enterprise workloads, remains unclear. In this paper, we study the practical effectiveness of LLM-driven index tuning on Microsoft SQL Server using both industrial benchmarks and real-world enterprise customer workloads, and compare it with DTA. Our results show that while LLMs in several cases identify configurations that significantly outperform those found by DTA in terms of execution time, they suffer from high variance in index recommendation quality. Furthermore, index recommendations from the LLM are often substantially worse than DTA in terms of optimizer-estimated cost, making it challenging to extend cost-based index advisors such as DTA to leverage LLMs for index tuning. We point to some areas of future work that may be important for robustly leveraging LLMs for index tuning. • Real-time SQL Plan Management in Oracle (Sunil Chakkappen, Oracle et al.) - Consistent query performance is essential for mission critical database applications, yet SQL execution plans can change due to factors such as database upgrades, DML changes, new indexes, etc. While plan stability mechanisms such as stored outlines prevent regressions by freezing execution plans, they also inhibit performance improvements by disallowing plan evolution. We introduced SQL Plan Management (SPM) in Oracle 11g to address this trade-off by maintaining a set of accepted execution plans and allowing plan evolution only when new plans demonstrably outperform existing baselines. However, prior implementations of SPM primarily rely on background performance verification processes, delaying regression detection and recovery. This issue is amplified in autonomous cloud database systems, where several automatic actions that could cause plan change driven regressions are performed with limited customer control. Timely detection and remediation is paramount, but the constrained background resources on cloud may not keep pace. To overcome these limitations, we introduce Real-Time SPM in Oracle 26ai, a novel extension of SPM that performs foreground verification of new execution plans during user query execution. Real-Time SPM leverages runtime session context to immediately validate plan changes, enabling rapid adoption of superior plans while promptly detecting and preventing regressions. This paper presents the architecture and design of Real-Time SPM - including technical challenges like reliably comparing performance of previous plans - and contrasts it with traditional background plan evolution. Our experiments highlight tangible benefits of Real-Time SPM - delivering immediate performance boost while preserving plan stability in both cloud-native and on-premise environments. Real-Time SPM is successfully deployed in Oracle production, laying the groundwork for more adaptive and user-responsive SQL plan management in enterprise RDBMS platforms. • Towards Industrial-Scale Parametric Query Optimization (Songsong Mo, Nanyang Technological University et al.) - Parametric Query Optimization (PQO) is crucial for efficiently executing parametrized queries (PQ) in modern industrial database systems. This paper addresses two key challenges overlooked by existing PQO techniques: large-scale template generalization and online adaptability. For large-scale workloads, maintaining one model per template is impractical, while sharing a single model across all templates leads to degraded performance. To overcome this issue, we propose a representation-based clustering strategy coupled with hierarchical model training, which significantly reduces model cost while preserving accuracy. For online adaptability, we observe that query parameter distributions shift over time, rendering fixed plan caches suboptimal. To address this, we introduce a KL-divergence-driven model fine-tuning and plan updating strategy that dynamically adapts to workload changes. Our approach, deployed in OceanBase, is extensively evaluated on four benchmarks. Results show that it achieves up to 1.62$\times$ acceleration over the OceanBase optimizer and outperforms RankPQO, a state-of-the-art PQO method, by up to 1.23$\times$, demonstrating improved scalability and robustness for industrial-scale PQO. • Ultron: History-Based Query Optimization at Databricks (Supun Nakandala, Databricks et al.) - Databricks, a modern lakehouse data platform, operates at massive scale (over 50 million VMs, processing billions of queries per day) with limited data statistics, making traditional cost-based query optimization challenging. Databricks uses adaptive query execution (AQE) to mitigate many runtime inefficiencies, but certain decisions, such as join operator selection, could benefit significantly from accurate prior knowledge. We present Ultron, a production-grade history-based query optimization (HBO) framework recently deployed at Databricks. Since analytical workloads are often repetitive, Ultron can learn from prior executions of the same query and apply precise, subplan-level historical insights during optimization. To achieve this, Ultron introduces a three-tier architecture consisting of (1) a distributed cache for query history, (2) a low-latency local prediction layer with data-drift awareness, and (3) modular optimization applications that modify query plans without risky exploration. We demonstrate three applications of Ultron using data from the Databricks fleet and synthetic benchmarks, showing that Ultron improves the median latency of eligible joins by 25\% in production, optimizes approximately 23\% of all joins across the fleet, and achieves 96\% post-hoc decision accuracy. Ultron operates with modest overhead ($< 5\%$ median additional optimization time) and scales elastically with load, demonstrating that precise history-based optimization can safely complement adaptive execution in large-scale cloud data platforms.read more
☆
Tue Sep 1 · 10:45 AM - 11:45 AM
DEI Talks
● DEI Talks
📍 Grand Ballroom A+B
Publishing with Springer Nature - This workshop is designed to help authors develop successful journal submissions.Publishing with Springer Nature - This workshop is designed to help authors develop successful journal submissions. The workshop will cover topics including: logical manuscript structure (what should your submission include?), efficient publication strategy (how should you choose a journal to maximize impact?), successful journal submission (what should you keep in mind when preparing your submission?), and the lifetime of a journal submission (what should you do once a final decision on your paper is made?). There will also be alotted time for authors to ask questions.read more
Jessica GutowitzSpringer Nature
☆
Tue Sep 1 · 10:45 AM - 12:15 PM
Sponsor Talks 1
● Sponsor Talks
📍 Otis
Database Engines at Salesforce (Nat Wyatt, Salesforce) • Query Engines at Jane Street (Andrew Dulson, Jane Street) • Beyond One-Size-Fits-All: How eBay Builds Purpose-Built Databases at Scale (Viswa Vutharkar, eBay), Aditya Marella (eBay) • Axiom: Composable Query Engines Built on...Database Engines at Salesforce (Nat Wyatt, Salesforce) • Query Engines at Jane Street (Andrew Dulson, Jane Street) • Beyond One-Size-Fits-All: How eBay Builds Purpose-Built Databases at Scale (Viswa Vutharkar, eBay), Aditya Marella (eBay) • Axiom: Composable Query Engines Built on Velox (Maria Basmanova, Meta) • Accelerating Data Processing Across the Modern Hardware Stack (Todd Mostak, NVIDIA)read more
Built for you by(617)-PATRICK, Champion AI speaker
Harmonizing Efficiency and Accuracy in Filtered Vector Search (Zixiang Zhou, MIT et al.) - Approximate nearest neighbor search (ANNS) is increasingly essential for AI-driven applications.Harmonizing Efficiency and Accuracy in Filtered Vector Search (Zixiang Zhou, MIT et al.) - Approximate nearest neighbor search (ANNS) is increasingly essential for AI-driven applications. In practice, ANNS is often coupled with label filtering to improve accuracy, relevance, and efficiency, a variant known as filtered ANNS or FANNS. Existing FANNS methods fall short in efficiency or accuracy. This is because, first, searching on a label-mixed index often converges to local minima due to label interference. Second, isolating and duplicating labels in the index inflates the index and incurs prohibitive memory overhead. Third, current join-based strategies for multi-filter search perform substantial wasted computation on unpromising candidates. We present BigFANN, a label-aware FANNS framework that achieves high speed and space efficiency, while retaining high accuracy. Our framework features a hybrid indexing scheme that introduces heterogeneous edge types and flexible IVF-graph indexing. Specifically, we construct graph indices with tunable combinations of exclusive and shared edges based on label characteristics. This method effectively minimizes label interference for single-filter search under a certain memory budget, ensuring high search accuracy and speed. In addition to heterogeneous edges, we adopt a hybrid of graph and IVF indices to deal with single- and multi-filter queries. Particularly, for multi-filter searches, we propose a join-free search strategy to eliminate wasted computation in the existing join-based strategy. Experimental results on various datasets show that BigFANN significantly outperforms state-of-the art FANNS frameworks, UNG and ParlayIVF^2, by up to 786× and 4× respectively, while achieving the same or better accuracy. • CGIF: Combining Proximity Graphs and Inverted Files for Efficient Filtered Vector Search over Arbitrary Predicates (Jiarui Luo, Rutgers University et al.) - Modern retrieval systems increasingly require filtered vector search under arbitrary predicate constraints, where users filter results by attributes such as category, price, location, keywords, and their combinations. Existing solutions either specialize in a single predicate type (e.g., range or equality filters), rely on dense, high-overhead indexes, or fail to handle predicates with diverse selectivities. As a result, they fail to simultaneously achieve efficiency, scalability, and flexibility. In this paper, we propose CGIF, an index that efficiently supports approximate nearest neighbor search (ANNS) both with and without predicates, while preserving the lightweight and scalable structure of the widely adopted vector index HNSW. Our design builds on an observation from previous works that HNSW traversal naturally consists of two phases: (1) a navigation phase, where the search rapidly moves toward the query’s vicinity, and (2) a local exploration phase, where traversal expands locally to refine results. CGIF retains the original HNSW search strategy during navigation to efficiently reach the query region, and introduces a predicate-aware traversal during local exploration. When a neighbor does not satisfy the query predicates, CGIF replaces it with alternative candidates drawn via inverted-file (IVF) indexing, ensuring effective local exploration under diverse predicates. Extensive experiments on multiple real-world datasets show that CGIF consistently outperforms state-of-the-art filtered vector search methods, delivering up to 2× faster query performance while maintaining high recall across diverse predicate types and selectivities. • Elastic Index Selection for Label-Hybrid AKNN Search (Mingyu Yang, The Hong Kong University of Science and Technology (Guangzhou) et al.) - Real-world vector embeddings often carry additional label attributes, such as keywords and tags. In this context, label-hybrid approximate k-nearest neighbor (AKNN) search retrieves the top-k approximate nearest vectors to a query, subject to the constraint that their labels fully contain the query-label set. A naive solution builds a separate index for every query-label set, but the exponential growth of such sets makes this approach storage-prohibitive. To overcome this, we propose selectively indexing only a subset of query-label sets while still ensuring efficient processing for all queries. This is made possible by a key insight into label containment: an index built for a label set L can also serve any query whose label set L' is a superset of L, with query cost bounded by the elastic factor: the ratio between the number of vectors matching L and those matching L'. We formalize the index-selection task as a constrained optimization problem that chooses which label sets to index to satisfy space and query efficiency constraints. We prove the problem is NP-complete and propose efficient greedy algorithms for its efficiency- and space-constrained variants. Extensive experiments on real-world datasets show that our method achieves 10x - 800x speedups over state-of-the-art techniques. Moreover, our approach is index-agnostic and can be seamlessly integrated into existing vector database systems. • ANNiE: A Learned Query Cost Estimator for Graph-Based Approximate Nearest Neighbor Search (Zeyu Wang, Fudan University et al.) - Query cost estimation is a fundamental problem in data management with numerous applications in query execution, yet remains an open problem in vector Approximate Nearest Neighbor Search (ANNS). Cost estimation plays a critical role in ensuring the accuracy of ANNS results, reducing unnecessary search effort, and enabling cost-based optimization. In this paper, we define the problem of cost estimation in ANNS, analyze its challenges, and introduce ANNiE, a novel learned cost estimator designed for graph-based ANNS. ANNiE estimates the cost required to reach a specified recall target and couples its estimates with probabilistic quality guarantees. We show how ANNiE can be used to optimize search time by designing the first accuracy-guaranteed graph search algorithm. Our experimental evaluation with several workloads, demonstrates that ANNiE improves estimation accuracy by 6$\times$ over the baselines, while achieving the probabilistic guarantee. Moreover, the graph search of ANNiE, ANNiE-S, achieves a 2.3$\times$ speedup over the baselines, while automatically reaching each query's recall target. • An Experimental Evaluation of Hybrid Querying on Vectors (Jiaxu Zhu, Huazhong University of Science and Technology et al.) - Recent studies demonstrate the significant practical value of hybrid queries, which integrate vector search with structured filters (e.g., attribute and range filtering) for refined retrieval. However, current evaluations lack unified benchmarking standards and systematic assessment methodologies. Existing studies not only fail to cover mainstream algorithms but also omit systematic comparisons or in-depth analysis on different methods. To address this issue, we design a complete evaluation framework for hybrid queries. Our study introduces 15 hybrid query algorithms and systematically classifies them based on multiple dimensions, such as index organization and filtering strategy, providing a reference for the categorization of hybrid queries. In experiments, for attribute filtering, we construct standard attribute sets, enabling a unified comparison of algorithms in terms of index construction efficiency, query performance, and robustness. For range filtering, we also evaluate the algorithm performance across the 3 metrics through controlled variation of query ranges. Additionally, we conduct an in-depth analysis of the experimental results based on the underlying principles of algorithms. Extensive experimental results reveal the strengths and weaknesses of each algorithm. Based on the findings, we develop a set of practical guidelines for algorithm selection, offering reliable references for different application scenarios. Furthermore, we identify potential directions for improvement to address the current limitations of these algorithms. • Revisiting Filtered ANN Benchmarks: A Hardness-Controlled Benchmark Generator for Realistic Evaluation (Mintaek Lim, Seoul National University et al.) - Filtered approximate nearest neighbor (FANN) search must satisfy both vector similarity and structured predicates, yet evaluations remain brittle because real hybrid workloads are rarely shareable and existing benchmarks rely on ad-hoc synthetic or semi-real constructions. We argue that realism hinges on execution-driven query difficulty: failures in early filtering trigger over-fetching of additional candidates, shaping latency, throughput, and recall. Building on this insight, we propose Hardness, a query-level hardness metric that models the conditional execution chain via the over-fetch factor and extends naturally to strategy-conditioned settings. Across diverse datasets and hybrid strategies, 𝛼-Hardness exhibits strong monotonic alignment with empirical performance, while common proxies such as selectivity or attribute - vector correlation are frequently unstable or strategy-inconsistent. We further introduce HCBGen, a hardness-controlled benchmark generator that uses 𝛼-Hardness as an explicit control signal to synthesize workloads under coarse bias modes or to match a target hardness profile. Our experiments show that widely used benchmarks occupy a narrow, relatively easy portion of the hardness spectrum, masking robustness gaps that emerge under harder queries. Finally, we demonstrate that matching hardness distributions enables privacy-preserving proxy workloads that closely reproduce performance trends, bridging research benchmarks and real evaluation.read more
☆
Tue Sep 1 · 1:45 PM - 3:15 PM
Research 8 · Learned Indexes and Data Structures
● Research
📍 Grand Ballroom D
LiBox: A Learned Index as an Array to Minimize Last-Mile Search (Jian Zhou, University of Texas at Arlington et al.) - Learned index has received much attention due to its often dramatic performance and space-efficiency improvements over traditional index designs.LiBox: A Learned Index as an Array to Minimize Last-Mile Search (Jian Zhou, University of Texas at Arlington et al.) - Learned index has received much attention due to its often dramatic performance and space-efficiency improvements over traditional index designs. Its strength is rooted in its ability of using computation with models or functions, rather than key search, to obtain the position of a search key in a sorted key array. However, the models or functions cannot compute the correct position without errors. Instead, they can only help determine a key scope where last-mile local search is required to find the search key. Both model computation and the last-mile search can be expensive and limit the index’s performance potential. To unlock the full potential of learned index, we propose LiBox, which groups keys in a sorted array into boxes so that (1) the box that covers a search key can be identified without any error by a linear regression function; (2) the last-mile search in the box can be conducted using only one AVX-512 instruction. A learned index based on such a box structure can be highly efficient in which each key search takes a well-predictable number of instructions and memory accesses. In this paper, we present the design of a hierarchical box-based structure. Using moderately extra space in a box to accommodate irregular key distribution, LiBox can serve both read and write queries at the speed of array reference. Libox can also flexibly align its reorganization operation with read/write intensity distribution to support high-performance read by hiding the structural modification cost. We have implemented LiBox and extensively evaluated it. The experiment results show that it outperforms state-of-the-art learned indexes, such as ALEX and LIPP, and non-learned indexes such as ART by up to 1.4 - 4.5 times with similar or higher space efficiency. • Rethinking Learned Index and LSM-tree Integration (Guangxun Zhao, Dankook University et al.) - Learned indexes improve data access efficiency by accelerating lookups and reducing memory usage, but integrating them with write-optimized Log-Structured Merge-trees (LSM-trees) remains challenging due to frequent compactions and intensive updates. We analyze this integration and identify two key mismatches. First, learned indexes reshape the conventional SSTable sizing trade-off in LSM-trees. In conventional LSM-trees, SSTable size drives the read/write trade-off, and this trade-off is amplified by level asymmetry between write-intensive upper levels and read-intensive deeper levels. Learned indexes make lookups less sensitive to SSTable size, thereby changing the traditional trade-off. Second, learned indexes typically employ fixed error bounds that cannot adapt to key distribution shifts caused by LSM-tree compactions, resulting in inefficient index construction and degraded lookup performance across different levels. Wild Turkey addresses these mismatches with two complementary mechanisms. Level-Aware Compaction (LAC) introduces a level-specific SSTable sizing strategy that aligns compaction granularity with the distinct read/write characteristics of each level. Building on LAC, Wild-Learning is a reinforcement learning (RL)-based controller that adaptively tunes both the LAC-degree and the per-SSTable error bound in response to evolving data distributions and system conditions. Together, these mechanisms co-tune compaction behavior and learned index construction to balance read and write performance under changing workloads. On SOSD datasets and YCSB workloads, Wild Turkey achieves up to 2.01x higher write throughput, 1.52x higher read throughput, 36\% less write stall time, and 78.4\% fewer compactions compared to state-of-the-art LSM-tree and learned index integration. • Learned Static Function Data Structures (Stefan Hermann, Karlsruhe Institute of Technology et al.) - We consider the task of constructing a data structure for associating a static set of keys with values, while allowing arbitrary output values for queries involving keys outside the set. Compared to hash tables, these so-called static function data structures do not need to store the key set and thus use significantly less memory. Several techniques are known, with compressed static functions approaching the zero-order empirical entropy of the value sequence. In this paper, we introduce learned static functions, which use machine learning to capture correlations between keys and values. For each key, a model predicts a probability distribution over the values, from which we derive a key-specific prefix code to compactly encode the true value. The resulting codeword is stored in a classic static function data structure. This design allows learned static functions to break the zero-order entropy barrier while still supporting point queries. Our experiments show substantial space savings: up to one order of magnitude on real data, and up to three orders of magnitude on synthetic data. • STEM2: A Fast and Space-efficient Data Structure for Exact Multi-Set Membership Query (Yannian Niu, University of Connecticut et al.) - Multi-set membership queries are ubiquitous in networking and database systems. Current solutions force a difficult compromise: hash tables guarantee correctness but suffer from high memory footprints, while filter-based approaches optimize space at the cost of probabilistic errors. In this paper, we propose STEM2, a fast and space-efficient data structure that achieves 100% query accuracy and can support dynamic key updates for multi-set membership queries. STEM2 utilizes a balanced binary tree architecture where each non-leaf node incorporates a novel Exact Binary Set Separator (XBSS) to partition keys into two disjoint groups. A key innovation of our design is a minimized hashing scheme that requires only two hash computations per key lookup, significantly reducing computational overhead. Additionally, STEM2 separates the control plane and the data plane: the control plane handles construction and dynamic updates, while the data plane is dedicated to serving efficient membership queries. Extensive experiments show that STEM2 achieves over 120 million operations per second (Mops) in lookup throughput, outperforming the state-of-the-art Coloring Embedder by 20% and the Ludo hashing by up to 21.6×, while maintaining compact memory cost and exact correctness. • Index Intersection for High-Dimensional Range Queries (Maximilian Berens, TU Dortmund University et al.) - For high-dimensional analytics queries in scientific domains, a full table scan is often seen as the only feasible execution path, even if result cardinalities are known to be small. In this paper, we argue for intersecting multiple indices built over medium-sized attribute subsets (Teams) as a means to produce a list of (candidate) tuple IDs for further processing. While this strategy is compatible with various index structures, significant discriminative power lies in the combined selectivity of multiple predicates. Akin to bitmap indices and VA-files, adopting simple and lightweight index approaches for each Team, instead of highly accurate but costly ones, still enables high overall selectivity and precision. Thus, the focus shifts away from individual indices towards their efficient intersection. Teams with just 1 member/attribute (bitmap indices) are outperformed for selective queries due to the inability to avoid access to large parts of the index. For example, Teams with 5 members are up to 6-7 times faster for 85 dimensions and require 1.58-2.07 times less storage. Team-based Indexing is most useful for queries with high selectivity and dimensionality, such as the search for rare objects. • An Evaluation of N-Gram Selection Strategies for Regular Expression Indexing in Contemporary Text Analysis Tasks (Ling Zhang, University of Wisconsin Madison et al.) - Efficient evaluation of regular expressions (regex, for short) is crucial for text analysis, and n-gram indexes are fundamental to achieving fast regex evaluation performance. However, these indexes face scalability challenges because of the exponential number of possible n-grams that must be indexed. Many existing selection strategies, developed decades ago, have not been rigorously evaluated on contemporary large-scale workloads and lack comprehensive performance comparisons. Therefore, a uni"ed and comprehensive evaluation framework is necessary to compare these methods under the same experimental settings. This paper presents the "rst systematic evaluation of three representative n-gram selection strategies across "ve workloads, including real-time production logs and genomic sequence analysis. We examine their trade-o#s in terms of index construction time, storage overhead, false positive rates, and end-to-end query performance. Through empirical results, this study provides a modern perspective on existing n-gram based regular expression evaluation methods, extensive observations, valuable discoveries, and an adaptable testing framework to guide future research in this domain. We makeourimplementations of these methods and our test framework available as open-source at https://github.com/mush-zhang/RegexIndexComparison.read more
☆
Tue Sep 1 · 1:45 PM - 3:15 PM
Research 9 · Hardware-Accelerated Similarity Search
● Research
📍 Grand Ballroom C
GPU-Accelerated ANNS: Quantized for Speed, Built for Change (Hunter McCoy, Northeastern University et al.) - Approximate nearest neighbor search (ANNS) is a core problem in machine learning and information retrieval.GPU-Accelerated ANNS: Quantized for Speed, Built for Change (Hunter McCoy, Northeastern University et al.) - Approximate nearest neighbor search (ANNS) is a core problem in machine learning and information retrieval. GPUs offer a promising path to high-performance ANNS through massive parallelism and co-location with downstream applications, but current GPU indices face three limitations: inability to update without full rebuilds, lack of efficient quantization for high-dimensional vectors, and poor latency hiding due to data-dependent memory accesses. We present Jasper, a GPU-native ANNS system built on the Va- mana graph index that achieves both high query throughput and full updatability via three new techniques: (1) a batch-parallel con- struction algorithm enabling lock-free streaming insertions, (2) a GPU-efficient RaBitQ implementation that reduces memory foot- print up to 8×without random access penalties, and (3) an optimized search kernel with improved compute utilization and latency hiding. Across five datasets, Jasper achieves up to 1.93× higher throughput than CAGRA, the current state-of-the-art GPU index, while providing updatability that CAGRA lacks, constructs indices 2.4× faster on average, and delivers 19 - 131× faster queries than BANG, the previous fastest GPU Vamana implementation. • GPU-Native Approximate Nearest Neighbor Search with IVF-RaBitQ: Fast Index Build and Search (Jifan Shi, Nanyang Technological University et al.) - Approximate nearest neighbor search (ANNS) on GPUs is gaining increasing popularity for modern retrieval and recommendation workloads that operate over massive high-dimensional vectors. Graph-based indexes deliver high recall and throughput but incur heavy build-time and storage costs. In contrast, cluster-based methods build and scale efficiently yet often need many probes for high recall, straining memory bandwidth and compute. Aiming to simultaneously achieve fast index build, high-throughput search, high recall, and low storage requirement for GPUs, we present IVF-RaBitQ (GPU), a GPU-native ANNS solution that integrates the cluster-based method IVF with RaBitQ quantization into an efficient GPU index build/search pipeline. Specifically, for index build, we develop a scalable GPU-native RaBitQ quantization method that enables fast and accurate low-bit encoding at scale. For search, we develop GPU-native distance computation schemes for RaBitQ codes and a fused search kernel to achieve high throughput with high recall. With IVF-RaBitQ implemented and integrated into the NVIDIA cuVS Library, experiments on cuVS Bench across multiple datasets show that IVF-RaBitQ offers a strong performance frontier in recall, throughput, index build time, and storage footprint. For Recall≈0.95, IVF-RaBitQ achieves 2.3× higher QPS than the state-of-the-art graph-based method CAGRA, while also constructing indices 7.5× faster on average. Compared to the cluster-based method IVF-PQ, IVF-RaBitQ delivers on average over 2.9× higher throughput while avoiding accessing the raw vectors for reranking. • JHQ: Johnson-Lindenstrauss Enhanced Hierarchical Quantization for High-Dimensional Approximate Nearest Neighbor Search (Jiabao Han, Australian National University et al.) - High-dimensional approximate nearest neighbor (ANN) search is fundamental to modern AI applications including retrieval-augmented generation and recommendation systems. While vector quantization (VQ) methods excel at compressing vectors for efficient search, existing approaches face critical bottlenecks: prolonged indexing times due to expensive data-dependent training, slow query processing from quadratic distance computations, and poor scalability on large datasets. In this paper, we introduce a novel quantization framework that leverages the orthogonal Johnson-Lindenstrauss (JL) transformation to lay the foundation for resolving these bottlenecks. Our key insight is that the JL induces a predictable near-Gaussian distribution with independent dimensions, enabling quick codebook generation without expensive iterative training. Based on that, we propose two algorithms: JQ (JL-enhanced Quantization) achieves fast indexing through training-free codebook construction while maintaining provable distance error bounds; and JHQ (JL-enhanced Hierarchical Quantization) extends JQ with a two-level architecture that uses primary quantization for rapid candidate filtering and residual quantization for accurate refinement, achieving better query accuracy-speed tradeoff on large scale datasets. Finally, extensive experiments on six benchmark datasets with up to 3,072 dimensions demonstrate that our methods achieve 3 - 10$\times$ query speedup over state-of-the-art baselines at $\geq$95\% recall, with 10-30$\times$ index construction speedup. Particularly, JHQ excels on massive datasets, maintaining 2 - 10$\times$ higher queries-per-second at >90\% recall compared to JQ. • ConANN: Conformal Approximate Nearest Neighbor Search (Sonia Horchidan, KTH Royal Institute of Technology et al.) - Approximate Nearest Neighbor (ANN) search is widely used in applications such as recommendation systems, search engines, and natural language processing. Indexing techniques like the Inverted File (IVF) offer efficiency at the cost of accuracy, yet lack formal mechanisms to quantify or control approximation error. Existing approaches that attempt to provide such guarantees typically rely on restrictive assumptions about underlying data distributions, which limits their generalizability. We introduce ConANN, the first framework to provide formal, distribution-free error guarantees for IVF-based ANN search by leveraging recent advances in Conformal Risk Control. Empirical evaluation across five standard benchmarks demonstrates that ConANN: (1) tightly controls approximation error, achieving a worst-case False Negative Rate deviation within 0.03 percentage points of the target; (2) provides formal guarantees without requiring expansion of the search space, and in some cases even reduces the number of probed clusters; (3) dynamically adapts the cluster probes required per query; and (4) incurs negligible overheads when compared to existing state-of-the-art baselines. ConANN is integrated into the FAISS vector search library, facilitating adoption in real-world ANN systems.read more
☆
Tue Sep 1 · 1:45 PM - 3:15 PM
Research 10 · Data Provenance and Verifiability
● Research
📍 Commonwealth Ballroom C
Bolt-on, Verifiable Provenance for LLM-Powered Data Processing (Yiming Lin, University of California, Berkeley et al.) - Large Language Models (LLMs) are powerful tools for processing data.Bolt-on, Verifiable Provenance for LLM-Powered Data Processing (Yiming Lin, University of California, Berkeley et al.) - Large Language Models (LLMs) are powerful tools for processing data. However, LLMs are also complex black-boxes, returning answers to queries on data, without any indication for where the answer came from or whether it is trustworthy. We introduce the notion of provenance for data processing with LLMs. While existing heuristics (such as embedding similarity or directly asking an LLM) could provide some hints for where the answer was derived, they provide no guarantees that the answer can be derived using the identified provenance, and indeed, are often incorrect. Instead, we propose the notion of verifiable provenance wherein we identify a subset of the input text that reproduces the same (or equivalent) answer as that on the complete text, and introduce the notion of minimality, where the verifiable provenance is as small as possible. To identify such a provenance, a naive solution would require checking all possible subsets of the source data with the LLM, which is prohibitively expensive. We present BLIP, a bolt-on framework for efficiently inferring a small-sized verifiable provenance for any LLM-powered data processing task, with any LLM. As part of BLIP, we introduce eight strategies, each guaranteed to find a minimal verifiable provenance, as well as an adaptive strategy that combines their strengths to reduce cost further. We further extend BLIP to produce multiple minimal verifiable provenances. Experiments on seven datasets show that the provenance generated by BLIP is always guaranteed to reproduce the answer - achieving over 30% higher accuracy than the best-performing baseline with a comparable provenance size. Moreover, BLIP incurs a low cost, comparable to the original query on the original data. • V3DB: Audit-on-Demand Zero-Knowledge Proofs for Verifiable Vector Search over Committed Snapshots (Zipeng Qiu, Hong Kong University of Science and Technology et al.) - Dense retrieval services underpin semantic search, recommendation, and retrieval-augmented generation, yet clients typically see only a top-k list with no auditable execution evidence. We present V3DB, a verifiable, versioned vector-search service that checks on demand whether an untrusted provider's approximate nearest-neighbor (ANN) result was produced by executing published IVF-PQ semantics on a committed snapshot. V3DB commits to each corpus snapshot and standardizes IVF-PQ into a fixed-shape, five-step query semantics. Given a public commitment and query embedding, the service returns top-k payloads and, when challenged, produces a succinct zero-knowledge proof that the output follows these semantics on the committed snapshot, without revealing corpus embeddings or private index contents to the verifier during audit. To make proving practical, V3DB avoids costly in-circuit sorting and random access with multiset equality/inclusion checks plus lightweight boundary conditions. Our Plonky2 prototype proves up to 22x faster and uses up to 40% less peak memory than the circuit-only baseline, with millisecond verification. • Database Views as Explanations for Relational Deep Learning (Agapi Rissaki, Northeastern University et al.) - In recent years, there has been significant progress in the development of deep learning models over relational databases, including architectures based on heterogeneous graph neural networks (hetero-GNNs) and heterogeneous graph transformers. In effect, such architectures state how the database records and links (e.g., foreign-key references) translate into a large, complex numerical expression, involving numerous learnable parameters. This complexity makes it hard to explain, in human-understandable terms, how a model uses the available data to arrive at a given prediction. We present a novel framework for explaining machine-learning models over relational databases, where explanations are view definitions that highlight focused parts of the database that mostly contribute to the model's prediction. We establish such global abductive explanations by adapting the classic notion of determinacy by Nash, Segoufin, and Vianu (2010). In addition to tuning the tradeoff between determinacy and conciseness, the framework allows controlling the level of granularity by adopting different fragments of view definitions, such as ones highlighting whole columns, foreign keys between tables, relevant groups of tuples, and so on. We investigate the realization of the framework in the case of hetero-GNNs, and develop a model-specific approach via the notion of learnable masks. For comparison, we propose model agnostic heuristic baselines and show that our approach is both more efficient and achieves better explanation quality in most cases. Our extensive empirical evaluation on the RelBench collection across diverse domains and record-level tasks demonstrates both the usefulness of our explanations and the efficiency of their generation. • Toward Temporal Attribution Analytics in Dataflows (Chrysanthi Kosyfaki, Hong Kong University of Science and Technology, CSE Department et al.) - Data provenance (the process of determining the origin and derivation of data outputs) has applications across multiple domains including explaining database query results and auditing scientific workflows. Despite decades of research, provenance tracing remains challenging due to its high computational cost and storage requirements. In streaming systems such as Apache Flink, fine- grained provenance graphs can grow super-linearly with data volume, posing significant scalability challenges. We define temporal attribution, a new lightweight form of provenance, appropriate for certain tasks, such as monitoring dependencies between system components over time quantitatively. Temporal attribution enables time-focused analysis that does not require fine-grained, tuple-level dependency meta-data. Inspired by volume-based provenance tracking in Temporal Interaction Networks (TINs), we demonstrate TINs’ applicability in succinctly modeling quantified data exchanges between dataflow operators in stream data processing systems and in processing workflows, in general, over time. We classify data into discrete and liquid types, define five temporal provenance query types, and propose a state-based indexing approach. Our vision outlines research directions toward making this new form of temporal attribution a practical tool for large-scale dataflow analytics. • Programmable Dataflows: Abstraction and Programming Model for Data Sharing (Siyuan Xia, University of Chicago Division of the Physical Sciences Chris Zhu, University of et al.) - Data sharing is central to various applications such as fraud detection, ad matching, and improving patient care. However, each solution to data sharing is bespoke and cost-intensive, hampering value generation. We identify the lack of abstractions to control data release as the culprit of the problem. For example, it is common to have constraints on whether to share data that depend on the result of sharing, and evaluating these constraints requires sharing in the first place, leading to a standstill. To help people build solutions to a wide variety of data sharing applications, we propose programmable dataflows, which consist of two components. The first component is an abstraction, the contract, which agents use to communicate the intent of a data sharing action and evaluate its consequences before the dataflow takes place. This helps agents control the release of their data. The second component is a contract programming model (CPM), which allows agents to program data sharing applications catered to each problem’s needs with the contract abstraction. We describe how to deploy those applications on a data escrow to ensure data remains protected from unintended data releases. Our evaluation shows 1) the contract abstraction permits representing a wide range of sharing problems, 2) CPM permits writing programs for complex data sharing problems and 3) quantitatively, our improvements to CPM make sharing programs run efficiently. • Dinkel: State-Aware and Granular Framework for Validating Graph Databases (Celine Wüst, ETH Zurich et al.) - Graph database management systems (GDBMSs) have been powering many data-driven applications. To ensure GDBMS reliability, several testing approaches have been proposed. However, they all suffer from two key limitations: (1) insufficient support for generating complex and valid queries to exercise deep GDBMS code, and (2) lack of general oracles to validate the execution correctness of arbitrary queries. In this paper, we propose a novel and practical approach, Dinkel, for thoroughly testing GDBMSs. Our approach consists of two core techniques. First, to generate complex and valid queries, we model two kinds of graph state, query context and graph schema, to describe the Cypher variables and the manipulated graph labels and properties. We generate queries clause-by-clause, and modify the graph states on the fly to ensure each clause references the correct state information. Second, to generally validate query results, we introduce two fine-grained query transformations: clause-level and expression-level transformations. These transformations can operate on arbitrary queries while preserving their semantics. Dinkel validates GDBMSs by checking whether the transformed query produces the same results as the original. We evaluated Dinkel on three well-known GDBMSs. In total, we found 127 bugs, among which 113 were confirmed, 84 were fixed, and 33 were logic bugs. Compared to existing approaches, Dinkel can cover over 70% more code and find substantially more bugs within a 48-hour testing campaign. We expect Dinkel’s powerful bug detection to lay a practical foundation for GDBMS testing.read more
☆
Tue Sep 1 · 1:45 PM - 3:15 PM
Research 11 · Data Preparation for ML
● Research
📍 Commonwealth Ballroom B
Mil: Cost-guided Minimum Makespan Scheduling for Applications of Multiple LLMs (Jingzhi Fang, HKUST et al.) - Multi-LLM applications calling multiple LLMs per request are emerging. An important scenario is running these applications offline on a request set.Mil: Cost-guided Minimum Makespan Scheduling for Applications of Multiple LLMs (Jingzhi Fang, HKUST et al.) - Multi-LLM applications calling multiple LLMs per request are emerging. An important scenario is running these applications offline on a request set. This work aims to minimize the offline inference makespan of these applications to save time and cost. Specifically, we study minimum-makespan scheduling of multi-LLM applications (the MLAS problem), which requires GPU allocation, LLM parallelism selection, and LLM execution orchestration. MLAS is NP-hard, and it differs from existing multi-model frameworks and job scheduling problems due to LLMs’ unique properties (e.g., high memory demand, complex inference behavior), the offline inference setting, and relaxed execution precedence constraints. There is no existing work on MLAS and simple rules cannot handle all the problem instances. We propose a framework, Mil, for MLAS with three major components: (1) processing functions estimating LLM processing rates by output length sampling, inference process simulation, and per-generation-iteration latency estimation; (2) a greedy method that finds a good schedule with a theoretical guarantee on a simplified problem instance; (3) a runtime adjustment mechanism reducing GPU idleness. Experiments on various applications (ensembling, routing, chain summary, mixed) show that Mil can achieve up to 3.4× end-to-end speedups over current practice. • DeepPrep: An LLM-Powered Agentic System for Autonomous Data Preparation (Meihao Fan, Renmin University of China et al.) - Data preparation, which aims to transform heterogeneous and noisy raw tables into analysis-ready data, remains a major bottleneck in data science. Recent approaches leverage large language models (LLMs) to automate data preparation from natural language specifications. However, existing LLM-powered methods either make decisions without grounding in intermediate execution results, or rely on linear interaction processes that offer limited support for revising earlier decisions. To address these limitations, we propose DeepPrep, an LLM-powered agentic system for autonomous data preparation. DeepPrep constructs data preparation pipelines through iterative, execution-grounded interaction with an environment that materializes intermediate table states and returns runtime feedback. To overcome the limitations of linear interaction, DeepPrep organizes pipeline construction with tree-based agentic reasoning, enabling structured exploration and non-local revision based on execution feedback. To enable effective learning of such behaviors, we propose a progressive agentic training framework, together with data synthesis that supplies diverse and complex ADP tasks. Extensive experiments show that DeepPrep achieves data preparation accuracy comparable to strong closed-source models (e.g., GPT-5) while incurring 15x lower inference cost, while establishing state-of-the-art performance among open-source baselines and generalizing effectively across diverse datasets. • BRIEF: Bi-level Coreset Selection for Efficient Instruction Tuning in LLMs (Chaoyuan Shen, Beijing Institute of Technology et al.) - Instruction tuning is a key step in adapting large language models (LLMs) to effectively understand and follow human instructions. It enables LLMs to transform general knowledge into task-specific responses that align with user intent. Although many high-quality instruction tuning datasets have been released, efficiently utilizing these data sources during supervised fine-tuning (SFT) is important, as training on the full high-quality corpus can be computationally expensive. To address this inefficiency, we explore whether a compact, high-quality subset of instruction data can achieve comparable performance to full-dataset SFT, thereby reducing training cost without sacrificing effectiveness. To this end, this work proposes to select such a subset (a.k.a., coreset) of instruction examples that maintains comparable downstream performance while improving training efficiency. The key idea is inspired by our discovered decomposition that in instruction tuning, the training loss can be decomposed into two components that effectively quantify the contribution of an instruction to the two fundamental capabilities of LLMs, namely knowledge-related capability and instruction following capability. We then revisit the objective of the classical coreset approaches to balance the two capabilities when selecting instruction examples. Based on a bi-level formulation and a composite gradient distance that makes the objective submodular, we design an effective algorithm to achieve a bounded approximation error. Experiments on 4 datasets across 9 downstream tasks demonstrate that BRIEF reduces computational costs by 3× while improving accuracy by 5% on Llama-3.1-8B, Qwen3-4B and Mistral-Nemo-12B. • Data-efficient Online Training for Direct Alignment in LLMs (Chi Zhang, Beijing Institute of Technology et al.) - In recent years, online Direct Alignment from Preferences (DAP) has emerged as a popular alternative for Reinforcement Learning from Human Feedback (RLHF) due to its training stability and simplicity. In online DAP, training relies on preference data, each composed of a question and a pair of large language model (LLM) responses. However, annotating preference data, i.e., generating responses for questions, and using these data to train the RLHF model are computationally expensive. To address this, we propose DOTA, a data selection framework that minimizes the cost of generating preference data, while still ensuring the quality of training. First, we propose a theoretically grounded metric called Preference Perplexity (\texttt{PFP}) that enables us to design a low cost, gradient-based method to effectively estimate the contribution of each preference data point to model performance -- critical to data selection. Second, rather than first generating responses for all candidate questions and then selecting preference data points by measuring their \texttt{PFP}, we design an iterative end-to-end framework that only has to generate responses for a small subset of questions, without missing valuable data points. Experiments on UltraChat-200k and HH-RLHF across 13 downstream tasks demonstrate that DOTA reduces computation cost by a factor of three on LLaMA-3-8B, Qwen-3-4B, and Qwen-3-1.7B, without compromising training effectiveness. • Resilience-Aware Elastic Scaling for Cloud-Native Online DL Training on Multi-Tenant GPU Clusters (Qianhao Wu, Tencent et al.) - Online deep learning (DL) training has become pivotal in powering real-time applications. Yet tidal workload fluctuations leave GPU clusters significantly underutilized during off-peak periods. This not only wastes GPU capacity but also exacerbates scarcity for other GPU-intensive jobs on cloud-native GPU clusters. Cluster-wide resource leasing across different tenants enabled by elastic scaling offers a promising opportunity to enhance GPU utilization for cloud-native online DL training on multi-tenant GPU clusters. Existing solutions do not address the unique challenges of maintaining system stability during elastic scaling, including prolonged disruptions due to job reconstruction, failures arising from triggering dependency-unaware operations, and unreliable reclamation of loaned GPU resources. In this paper, we introduce WeFlex, a resilience-aware elastic scaling solution engineered for cloud-native online deep learning jobs in multi-tenant GPU clusters. WeFlex allows GPUs from online training jobs to be leased to other GPU-intensive jobs during low-demand periods while ensuring rapid reclamation as demand surges. It significantly reduces the duration of training disruptions by constructing an interruption mitigation pipeline, prevents dependency-unaware operation failures via topology-aware pod orchestration, and ensures reclamation of GPU resources through right-of-return GPU leasing. Evaluations on production GPU clusters at a 10,000-plus scale demonstrate that WeFlex enhances GPU utilization of online training by about 25\% while reliably maintaining continuous training performance. • Efficient Banzhaf-Based Data Valuation for $k$-Nearest Neighbors Classification (Guangyi Zhang, Shenzhen Technology University et al.) - Data valuation, the task of quantifying the contribution of individual data points to model performance, has emerged as a fundamental challenge in machine learning. Game-theoretic approaches, such as the Banzhaf value, offer principled frameworks for fair data valuation; however, they suffer from exponential computational complexity. We address this challenge by developing efficient algorithms specifically tailored for computing Banzhaf values in $k$-nearest neighbor ($k$NN) classifiers. We first establish the theoretical hardness of the problem by proving that it is \#P-hard. Despite this intractability, we exploit the locality properties of $k$NN classifiers to develop practical exact algorithms. Our main contribution is a dynamic programming framework that achieves significant computational improvements: we present a pseudo-polynomial algorithm with $O(Wkn^2)$ time complexity for weighted $k$NN classifiers, where $W$ is the maximum sum of top-$k$ weights, and a specialized algorithm for unweighted $k$NN that achieves $O(nk^2)$ time complexity, that is, linear in the number of data points. We also offer efficient Monte Carlo estimation methods. Extensive experiments on real-world datasets demonstrate the practical efficiency of our approach and its effectiveness in data valuation applications. • CaSh: Shapley Value Computation with Cache Optimization (Jiajun Tang, Zhejiang University et al.) - In recent years, the Shapley value has become the de facto standard for equitable attribution in data analytics, such as data valuation and model interpretability. Since exact computation entails an exponential complexity of O(2^n), sampling-based approximation algorithms are widely adopted. However, these methods treat utility functions as stateless black boxes, leading to a critical system-level inefficiency: the redundant and costly evaluation of identical coalitions that recur during sampling. To address this bottleneck, we propose CaSh, an algorithm-agnostic Caching framework that accelerates existing Shapley value approximation algorithms by strategically storing and reusing intermediate coalition utility computations. CaSh leverages a high-performance Direct Mapping architecture tailored for Shapley value approximation to cache coalition utility results, enabling significant speedups without introducing any additional approximation error. We integrate CaSh with major approximation algorithms and evaluate the performance across diverse data analytics tasks. Experimental results demonstrate that CaSh consistently accelerates widely used approximation algorithms, reducing total computation time by 8% to 29% depending on the underlying sampling strategy. This efficiency gain is achieved without introducing additional approximation error beyond the underlying estimator, improving the efficiency of Shapley value-based data analytics pipelines.read more
☆
Tue Sep 1 · 1:45 PM - 3:15 PM
Research 12 · Time Series and Data Streams
● Research
📍 Commonwealth Ballroom A
CLaP - State Detection from Time Series (Arik Ermshaus, Humboldt-Universität zu Berlin et al.) - The ever-growing amount of sensor data from machines, smart devices, and the environment leads to an abundance of high-resolution, unannotated time series (TS).CLaP - State Detection from Time Series (Arik Ermshaus, Humboldt-Universität zu Berlin et al.) - The ever-growing amount of sensor data from machines, smart devices, and the environment leads to an abundance of high-resolution, unannotated time series (TS). These recordings encode recognizable properties of latent states and transitions from physical phenomena that can be modelled as abstract processes. The unsupervised localization and identification of these states and their transitions is the task of time series state detection (TSSD). Current TSSD algorithms employ classical unsupervised learning techniques, to infer state membership directly from feature space. This limits their predictive power, compared to supervised learning methods, which can exploit additional label information. We introduce CLaP, a new, highly accurate and efficient algorithm for TSSD. It leverages the predictive power of time series classification for TSSD in an unsupervised setting by applying novel self-supervision techniques to detect whether data segments emerge from the same state. To this end, CLaP cross-validates a classifier with segment-labelled subsequences to quantify confusion between segments. It merges labels from segments with high confusion, representing the same latent state, if this leads to an increase in overall classification quality. We conducted an experimental evaluation using 405 TS from five benchmarks and found CLaP to be significantly more precise in detecting states than six state-of-the-art competitors. It achieves the best accuracy-runtime tradeoff and is scalable to large TS. We provide a Python implementation of CLaP, which can be deployed in TS analysis workflows. • CounterSnake: A lossless and generalized compression framework for diverse sketches (Xunpeng Liu, Peking University et al.) - Sketches are vital for large-scale stream analytics. However, they often use fixed-size counters, which remain underutilized, especially under skewed data distributions. Prior solutions to address this inefficiency compromise on accuracy, real-time operations, or generality, which limits their applicability. In this paper, we propose CounterSnake, a novel hierarchical compression framework that reduces the memory consumption of sketch counters. Compared with existing efforts, CounterSnake is the first one to fulfill four key requirements: (1) zero counter error, (2) bounded latency, (3) full counter interfaces, and (4) efficient multi-sketch optimization. The key idea is to dynamically link overflowing counters across layers to form variable-size logical counters. Besides, we design techniques such as tag-based linking, d-way mapping, sign-bit encoding, and virtual-counter abstraction to address the four requirements. We also theoretically derive its memory and time complexities under justified assumptions. Experiments against six SOTA solutions demonstrate up to several orders of accuracy improvements and comparable operation throughput. We also thoroughly evaluate CounterSnake and other frameworks, showing that CounterSnake is the only one that fulfills all the requirements. • KDSelector: A Framework of Knowledge-Enhanced and Data-Efficient Selector Learning for Anomaly Detection Model Selection in Time Series (Zhiyu Liang, Harbin Institute of Technology et al.) - Model selection has been raised as an essential problem in the area of time series anomaly detection (TSAD), because there is no single best TSAD model for highly heterogeneous time series in real-world applications. However, despite the success of existing model selection solutions, which usually learn (a.k.a. train) a classification model (especially neural network, NN) using historical data as a selector to predict the correct TSAD model for each time series to detect, the existing NN-based selector learning method cannot utilize the auxiliary knowledge in the historical data and requires iterating over all training samples, which limits the model selection ability and training speed of the selector. The latter data efficiency problem can be partially solved by existing data pruning methods designed for general NN training, but with suboptimal speedup or degraded selection ability due to disregarding intrinsic data properties in TSAD model selector training. To address these limitations, we propose KDSelector, to the best of our knowledge, the first framework customized for knowledge-enhanced and data-efficient learning of NN-based TSAD model selectors, of which we design three plug-and-play modules that are agnostic to NN architectures (e.g., ResNet and Transformer) and can be seamlessly integrated into the existing selector learning framework. Specifically, we propose two knowledge enhancement mechanisms to improve the selection ability of the selector with any architecture by integrating the auxiliary knowledge in a unified way. We further design a novel data pruning framework with theoretical guarantees to achieve state-of-the-art training acceleration for the NN-based selector with almost lossless selection ability. Extensive experiments demonstrate the superior performance of our proposals in terms of model selection ability and selector learning efficiency. • FB*: A Compact Index for Efficient and Exact Density-based Clustering (Bide Zhao, University of Sydney et al.) - Density-based clustering is a fundamental technique for discovering arbitrarily shaped clusters and handling noise, without requiring the number of clusters to be specified in advance. However, existing methods often struggle with efficiency and accuracy across varying query parameters, including the distance threshold $\varepsilon$ and size threshold $\mu$. In this paper, we propose a novel index-based algorithm for efficient and exact cluster extraction. We introduce FB, the first linear-size index that supports exact clustering with running time linear in the output size for any query $\varepsilon$ and a fixed $\mu$, along with an empirically compact variant, FB$^*$, for efficiently extracting density-based clusters. Due to the compactness of the index and the efficiency of the query algorithm, our index is well-suited for disk-based storage, enabling multiple versions of the index --- one for each distinct $\mu$ --- to support arbitrary $(\varepsilon,\mu)$ queries. We provide formal analyses of time and space complexity. Extensive experiments on 23 real-world datasets demonstrate that our method significantly outperforms existing approaches while guaranteeing exact clustering results. • DeXOR: Enabling XOR in Decimal Space for Streaming Lossless Compression of Floating-point Data (Chuanyi Lv, Zhejiang University et al.) - With streaming floating-point numbers being increasingly prevalent, effective and efficient compression of such data is critical. Compression schemes must be able to exploit the similarity, or smoothness, of consecutive numbers and must be able to contend with extreme conditions, such as high-precision values or the absence of smoothness. We present DeXOR, a novel framework that enables decimal XOR procedure to encode decimal-space longest common prefixes and suffixes, achieving optimal prefix reuse and effective redundancy elimination. To ensure accurate and low-cost decompression even with binary-decimal conversion errors, DeXOR incorporates 1) scaled truncation with error-tolerant rounding and 2) different bit management strategies optimized for decimal XOR. Additionally, a robust exception handler enhances stability by managing floating-point exponents, maintaining high compression ratios under extreme conditions. In evaluations across 22 datasets, DeXOR surpasses state-of-the-art schemes, achieving a 15% higher compression ratio and a 20% faster decompression speed while maintaining a competitive compression speed. DeXOR also offers scalability under varying conditions and exhibits robustness in extreme scenarios where other schemes fail. • Continuous Query for Top-K Maximal Sum Intervals over Streaming Data (Zhongshuai Zhang, Beijing Institute of Technolygy et al.) - The continuous identification of top-k maximal sum intervals using a sliding window over a data stream is a critical operation for applications in IoT and beyond. A maximal sum interval is a non-overlapping, contiguous subsequence with the maximal sum in a sequence of real numbers. Existing algorithms are ill-suited for streaming contexts: they either exhaustively enumerate all intervals even for small k values, or depend on indexes that require frequent and costly restructuring. We propose a novel partition-based strategy. Our core insight is a partitioning scheme that guarantees that any maximal sum interval is fully contained within a single partition, enabling independent and parallel processing. This design provides two key advantages: it enables early pruning of partitions that cannot contribute to top-k results, drastically narrowing the search space, and it enables efficient, incremental maintenance of the intervals in each partition. We develop algorithms for partition construction, incremental partition updates, and partition-based top-k maximal sum interval search. Extensive experiments on real and synthetic datasets demonstrate that our approach significantly improves efficiency. • PILOT-C: Physics-Informed Low-Distortion Optimal Trajectory Compression (Kefei Wu, Fudan University et al.) - Location-aware devices continuously generate massive volumes of trajectory data, creating demand for efficient compression. Line simplification is a common solution but typically assumes 2D trajectories and ignores time synchronization and motion continuity. We propose PILOT-C, a novel trajectory compression framework that integrates frequency-domain physics modeling with error-bounded optimization. Unlike existing line simplification methods, PILOT-C supports trajectories in arbitrary dimensions, including 3D, by compressing each spatial axis independently. Evaluated on four real-world datasets, PILOT-C achieves superior performance across multiple dimensions. In terms of compression ratio, PILOT-C outperforms CISED-W, the current state-of-the-art SED-based line simplification algorithm, by an average of 19.2%. For trajectory fidelity, PILOT-C achieves an average of 32.6% reduction in error compared to CISED-W. Additionally, PILOT-C seamlessly extends to three-dimensional trajectories while maintaining the same computational complexity, achieving a 49% improvement in compression ratios over SQUISH-E, the most efficient line simplification algorithm on 3D datasets. • A Topology-Aware Localized Update Strategy for Graph-Based ANN Index (Song Yu, Northeastern University et al.) - Graph-based indices are widely used for approximate nearest neighbor search (ANNS). Under dynamic workloads, existing ANNS systems amortize update overhead with large batches, but large batches degrade index quality. We identify two key limitations in existing systems when handling small-batch updates. First, they still scan the entire index to repair the affected graph topology and rebuild the index, causing heavy I/O. Second, their naive repair introduces many edges, repeatedly triggering costly neighbor pruning with expensive distance computations. To address these issues, we propose a topology-aware localized update strategy that exploits the locality of small-batch updates to reduce unnecessary I/O and computation. Specifically, we introduce a lightweight graph topology that quickly identifies affected nodes without full index scans, and a localized update mechanism that restricts modifications to the pages containing these nodes. Moreover, we design a similarity-aware localized connection method that links each affected node to a small set of highly similar neighbors, avoiding redundant edges and costly pruning. Extensive experiments show that our update strategy achieves 2.39-5.96X higher update throughput than the state-of-the-art graph-based streaming ANNS system FreshDiskANN while maintaining high search efficiency and accuracy.read more
☆
Tue Sep 1 · 1:45 PM - 3:15 PM
Tutorial 3 · From Human-Graph to Agent-Graph Interaction: State of the Art and Future Directions
● Tutorials
📍 Stone
From Human-Graph to Agent-Graph Interaction: State of the Art and Future Directions
Sourav S BhowmickNanyang Technological University
Byron ChoiHong Kong Baptist University
Arijit KhanBowling Green State University
☆
Tue Sep 1 · 1:45 PM - 3:15 PM
Tutorial 4 · CRDTs and Databases: A Hands-On Tutorial
● Tutorials
📍 Webster
CRDTs and Databases: A Hands-On Tutorial
Paulo Sérgio AlmeidaU. Minho & INESCTEC
Nuno FariaU. Minho & INESCTEC
José PereiraU. Minho & INESCTEC
☆
Tue Sep 1 · 1:45 PM - 3:15 PM
Industry Session 2 · Query Execution Engines & Hardware Acceleration
● Industry
📍 Marina Ballroom I
ZipFlow: a Compiler-based Framework to Unleash Compressed Data Movement for Modern GPUs (Gwangoo Yeo, KAIST et al.) - Modern Lakehouse architectures store data in open, columnar formats to enable cross-engine interoperability, and GPU-accelerated data analytics is increasingly becoming one of...ZipFlow: a Compiler-based Framework to Unleash Compressed Data Movement for Modern GPUs (Gwangoo Yeo, KAIST et al.) - Modern Lakehouse architectures store data in open, columnar formats to enable cross-engine interoperability, and GPU-accelerated data analytics is increasingly becoming one of these engines. In GPU-accelerated analytics, data transfer from CPU to GPU becomes a performance bottleneck due to limited PCIe bandwidth. Data compression is widely used to reduce transfer volume while leveraging GPUs for decompression. However, to optimize end-to-end query performance, the workflow of compression, transfer, and decompression must be holistically designed based on the compression strategy and hardware characteristics to balance I/O latency and computational overhead. In this work, we present ZipFlow, a compiler-based framework for optimizing compressed data movement in GPU-accelerated data analytics. ZipFlow classifies compression algorithms into three distinct patterns based on their inherent parallelism. For each pattern, ZipFlow applies generalized scheduling strategies to effectively exploit GPU computational resources across diverse architectures. Building on these patterns, ZipFlow provides flexible optimizations allowing to improve end-to-end query latency by 2.08× over the state-of-the-art GPU compression library (nvCOMP), and by up to 3.14× over a state-of-the-art CPU engines. • RayBooster: A Ray Tracing Engine to Accelerate SedonaDB (Liang Geng, the Ohio State University et al.) - Hardware acceleration is becoming increasingly critical for spatial databases, as their workloads are geometrically complex, data-intensive, and subject to growing real-time requirements. Building on our prior evidence that Ray Tracing (RT) cores can significantly accelerate spatial queries through dedicated hardware support, this paper presents RayBooster, the first solution that incorporates RT-core acceleration into a production-grade geospatial database system, Apache SedonaDB. Our approach delivers substantial performance improvements cost-effectively, focusing on spatial joins, which dominate execution time in spatial databases. To enable this integration, we bridge the mismatch between spatial query engines and RT hardware through three key systematic designs. First, to overcome the lack of random access in Well-Known Binary format, we design a GPU-optimized Structure of Arrays storage layout. Second, we eliminate indexing bottlenecks by constructing a monolithic Bounding Volume Hierarchy tree that encodes geometry IDs into the Z-axis, bypassing micro-index overhead. Third, to manage the diverse geometry types, our unified RelateEngine utilizes RT cores to compute the Dimensionally Extended 9-Intersection Model as a universal topological descriptor. Furthermore, we implement a memory-aware execution strategy to mitigate out-of-memory failures through robust resource management. Seamlessly integrated as an extension to SedonaDB, RayBooster delivers up to a 5.93X performance speedup on SpatialBench and provides a 59.02% reduction in operational costs, effectively transforming these idle RT units into an efficient engine for spatial analytics. • FastCompose: Eliminating Compilation Cold Starts in Query Execution with Composition (Venkatraman Govindaraju, Amazon et al.) - Compilation-based query execution produces optimized machine code per query but introduces a cold-start problem: when the compiled code is not cached, the query stalls during compilation, delaying data processing by up to orders of magnitude relative to the query's execution time. This overhead dominates short-running queries and creates latency variability for both interactive analytics and ETL pipelines. We introduce composition,a complementary technique in which query-time code generation emits only lightweight glue code to arrange pre-compiled operators into a query-specific execution plan, rather than re-emitting or compiling any operator logic, at a fraction of the cost of full compilation. Composition eliminates the cold-start stall while compilation catches up in the background and takes over for peak performance. We implement composition in Amazon Redshift through FastCompose, which enables both modes from a single code-base without relying on a separate fallback engine for cold runs. Compared to compilation-only cold starts (with caching and serverless compilation active), FastCompose achieves 7.0× speedup on TPC-DS 100GB, 2.0× on TPC-DS 3TB, 12.0× on TPC-H 100GB, and 1.6× on TPC-H 3TB. FastCompose is deployed across thousands of Redshift clusters. On production workloads, composition reduces cold-start dashboard load times by 3.5× and ETL duration by 1.9×, removing compilation as a bottleneck. • TQP++: Bridging ML Compilers and Analytical Query Processing on GPUs (Wei Cui, Microsoft et al.) - The convergence of unified, cloud-native Lakehouse platforms such as Microsoft Fabric, and the widespread deployment of AI-optimized hardware in datacenters is driving the rise of GPU-based analytical engines. However, building a GPU query engine that is both competitive with hand-tuned implementations and portable across vendors (NVIDIA, AMD, custom silicon) remains an open challenge, as existing systems either lock into a single vendor's toolchain or sacrifice performance for generality. In this paper, we present TQP++, an AI-native analytical query processor that repurposes ML compiler infrastructure to close this gap, achieving high performance and hardware portability from a single codebase. TQP++ integrates the Antares compilation framework with tiered GPU resource scheduling for SQL operators, a map-reduce-oriented fusion schema that eliminates intermediate materializations, and a multi-gated execution graph that adapts operator algorithms to runtime data characteristics. On TPC-H SF100, TQP++ executes all 22 queries in 1.1 second on an A100 (7x faster than HeavyDB, 15x faster than CPU baselines) and under 0.7 seconds on H100 and MI300, while targeting 9 devices across 3 vendors---including Xbox---without code changes. To our knowledge, this is the most hardware-diverse GPU query processor reported to date, achieving sub-second TPC-H SF100 on a single GPU, while delivering the deployment flexibility required by modern Lakehouse systems. • One Ring to Shuffle Them All: Scalable Intra-Process Data Redistribution with Ring-Buffer Shuffle in Redpanda Oxla (Adam Szymański, Red Panda et al.) - As server CPUs scale to dozens and now hundreds of cores per socket, parallel query engines must rethink how they redistribute data between threads. Partitioned operators such as hash joins and aggregations require frequent data redistribution across threads, yet existing intra-process shuffle designs fundamentally fail to scale with core count: batch partitioning avoids cross-thread synchronization in the hot path but materializes all intermediate data, introduces a global producer/consumer barrier, and requires a consumption approach with low cache locality, while channel-based streaming avoids materialization but incurs per-channel synchronizationhat scales poorly with core count. As core counts rise, these architectural tradeoffs increasingly prevent engines from fully utilizing modern hardware. We present a ring-buffer streaming shuffle design that addresses these shortcomings through lock-free atomic slot acquisition into fixed-size batch groups, achieving amortized O(1) synchronization cost per batch and O(M) memory independent of input size. Ring-buffer shuffle has been implemented in Redpanda's Oxla query engine for two years, where it currently powers production queries for Redpanda SQL users. We evaluate all three approaches on a 72-core NVIDIA GraceHopper, a 192-core dual-socket AWS Graviton4, and a 96-core (192-thread) AMD EPYC. On a 72-core single-socket system the ring buffer outperforms channel streaming by up to 44% and batch partitioning by up to 79%; at 192 cores the advantage over channel grows to over 100% and over 300% versus batch partitioning. Even so, on chiplet architectures with many partitioned L3 caches, the shared atomic counter becomes a cross-die bottleneck and channel-based streaming remains competitive. End-to-end evaluation on TPC-H and ClickBench shows the advantage is workload-dependent: the ring buffer wins TPC-H overall (1.07×), but consumer-heavy patterns - COUNT(DISTINCT), wide aggregates - swing ClickBench the other way. • From Presto to Prestissimo: A Velox-Powered Modernization Journey (Amit Dutta, Meta Platforms, Inc. et al.) - Presto is a widely adopted, open-source distributed SQL query engine that has powered Meta's exabyte-scale data analytics for over a decade. Over time, as data volumes and workload diversity consistently increased, the original Java-based Presto worker fleet started facing challenges in reliability, performance, and operational efficiency, despite continuous upgrades and enhancements. To overcome these challenges, we created Prestissimo (open-sourced under Presto), a more modern version of Presto, re-engineered to use the Velox open-source composable library; Prestissimo provides a drop-in C++ worker replacement, delivering substantial improvements in query latency, hardware efficiency, and reliability. In this paper, we outline the phased, in-place migration journey that resulted in a full and seamless transition of hundreds of millions of daily large-scale analytic queries, without service interruption or additional hardware. To ensure correctness and reliability, we developed a comprehensive suite of verification tools and testing frameworks, including randomized fuzzers, end-to-end verifiers, and production traffic shadowing. We discuss the challenges of achieving feature parity at scale across different programming languages, how we handled backward compatibility, how pipeline migration was prioritized, and how operational excellence was achieved in a multi-tenant, business-critical environment. The complete Prestissimo migration took about two years, alongside substantial development in both Prestissimo and Velox. With this journey, we have fully decommissioned the legacy Java worker fleet, achieving broad gains including 2-5x faster queries while at the same time cutting down the hardware footprint significantly, and reducing the number of reliability issues in production by an order of magnitude. Prestissimo is today fully rolled out at Meta, and has replaced Presto Java workers in many other deployments across the industry.read more
☆
Tue Sep 1 · 1:45 PM - 3:15 PM
DB Research Integrity (Part 1)
● Ethics
📍 Grand Ballroom A+B
DB Research Integrity Working Group (Angela Bonifati, Lyon 1 University & CNRS et al.)
☆
Tue Sep 1 · 1:45 PM - 3:15 PM
Demo Group A · Graph, Systems & Infrastructure
● Demos
📍 Marina Ballrooms II - IV
29 demos in this group, shown twice during the conference.29 demos in this group, shown twice during the conference. Includes: QueryBrew: System-Agnostic SQL-to-SQL Query Optimization • FedBridge: A Federated Query Engine over Embedding-Heterogeneous Vector Databases • HaDA: Empower Database Experts with Data Dependencies • OMBench: Taming Data Management Requirements in Cloud-Native Applications • How Out-of-Bounds Are Your Cardinality Estimates? • AtomSQL: Interactive Disambiguation of NL-to-SQL via User-Guided Atom-Level Alignment • Credo: Declarative Control of LLM Pipelines via Beliefs and Policies • SpatialSQL: A Multi-Agent System for Interactive and Observable Spatial Text-to-SQL • DataMagic: Transforming Tabular Data into Data Insight Video • Hamilton - Interactive Ontology Learning • … +19 moreread more
☆
Tue Sep 1 · 1:45 PM - 3:15 PM
Sponsor Talks 2
● Sponsor Talks
📍 Otis
Data Intelligence for the Agentic Era and Beyond LLMs: Optimizing the Systems Backbone of AI Engines (Wei Hong, Google), Fatma Özcan (Google) • Trusted Agentic AI with Oracle AI Database (Tirthankar Lahiri, Oracle) • Where Data Gravity Meets Intelligence: Building...Data Intelligence for the Agentic Era and Beyond LLMs: Optimizing the Systems Backbone of AI Engines (Wei Hong, Google), Fatma Özcan (Google) • Trusted Agentic AI with Oracle AI Database (Tirthankar Lahiri, Oracle) • Where Data Gravity Meets Intelligence: Building AI-Native Databases for Agentic AI (Hao Chen, Alibaba) • From Data to Decisions: Research-Driven Agentic AI in the Real World (Estevam Hruschka, Megagon Labs) • From Databases to Governed Enterprise Memory (Min-Soo Kim, GraphAI)read more
Built for you by(617)-PATRICK, Champion AI speaker
Unveiling Challenges for LLMs in Enterprise Data Engineering (Jan-Micha Bodensohn, DFKI & Technical University of Darmstadt et al.) - Large Language Models (LLMs) promise to automate data engineering on tabular data, offering enterprises a valuable opportunity to cut the high...Unveiling Challenges for LLMs in Enterprise Data Engineering (Jan-Micha Bodensohn, DFKI & Technical University of Darmstadt et al.) - Large Language Models (LLMs) promise to automate data engineering on tabular data, offering enterprises a valuable opportunity to cut the high costs of manual data handling. But the enterprise domain comes with unique challenges that existing LLM-based approaches for data engineering often overlook, such as large table sizes, more complex tasks, and the need for internal knowledge. To bridge these gaps, we identify key enterprise-specific challenges related to data, tasks, and background knowledge and extensively evaluate how they affect data engineering with LLMs. Our analysis reveals that LLMs face substantial limitations in real-world enterprise scenarios, with accuracy declining sharply. Our findings contribute to a systematic understanding of LLMs for enterprise data engineering to support their adoption in industry. • LLMs as Stratification Signals for KG Accuracy Evaluation (Stefano Marchesin, Università di Padova et al.) - Knowledge Graph (KG) accuracy assessment is essential for ensuring data quality in downstream applications, yet remains prohibitively expensive due to annotation costs and scale. Large Language Models (LLMs), trained on vast corpora, offer cheap fact validation but remain unreliable as direct accuracy estimators due to hallucinations and knowledge gaps. We propose a novel approach that exploits LLM capabilities without relying on their correctness: using aggregated LLM predictions as stratification signals for sampling-based accuracy estimation. By partitioning KGs into internally homogeneous strata guided by aggregated LLM outputs, we achieve statistically significant cost reductions ranging from 11% to 54% over unstratified and topology-based baselines on real-world KGs. To scale beyond LLM computational constraints, we introduce a knowledge distillation strategy that transfers stratification signals to efficient student models, requiring annotation of only 0.25% of facts while maintaining signal quality. Experiments on six KGs spanning 20M+ triples demonstrate consistent improvements over SotA methods, with statistical guarantees on accuracy estimates. • Auto-Fill: Learning to Predict Missing Values Accurately with Specialist Language Models (Yurong Liu, New York University et al.) - Predicting missing cell values in tabular data is a fundamental problem in data cleaning. While state-of-the-art reasoning models show great promise in predicting missing values in tables, by reasoning holistically across rows and columns, they are costly to deploy at scale and tend to be overconfident, often generating hallucinated or false-positive predictions. In this paper, we observe that achieving high precision missing-value prediction in tables requires a distinct combination of three capabilities: (1) world knowledge, (2) text-based reasoning, and (3) code-based reasoning. We systematically explore design choices for combining these capabilities, and propose an Auto-Fill approach that post-trains three specialist small language models (SLMs), each optimized for one capability. We develop a calibrated ensemble mechanism that either dynamically selects the most confident specialist or abstains, ensuring high accuracy. Extensive experiments on 11 benchmarks with 2200 real tables drawn from diverse domains show that Auto-Fill achieves superior accuracy compared to state-of-the-art reasoning models (e.g., o3-pro, Gemini 3 Pro, and DeepSeek R1), while operating at a fraction (less than 1%) of the cost of these frontier models. Our results highlight the effectiveness of specialization and calibrated abstention in the important domain of tabular data. • Accelerating String-Heavy Queries with LLM Token Tables (Tobias Schmidt, TUM et al.) - Strings are the most common data type in modern database systems, yet they are often treated as an afterthought in high-performance data formats. While numerical data benefits from specialized, light-weight compression schemes, text is typically handled by general-purpose algorithms such as Zstd, LZ4, or Snappy, which require full-block decompression before processing. In this paper, we explore the potential of repurposing Large Language Model (LLM) tokenizers as a lightweight string compression scheme for databases, similar to FSST, but with a global token table shared across all tables and columns. Operators such as joins and aggregations can exploit this consistent encoding to defer decompression and process encoded values directly. We implement a global token table based on GPT-4’s tokenizer in Umbra and demonstrate execution time improvements of up to 2× on string-heavy workloads, while reducing storage and memory consumption by up to 1.65×. Tokenizers integrate well with other compression algorithms, such as FSST, OnPair, or Zstd, while maintaining good compression ratios and high decompression throughput exceeding 6 GB/s on a single CPU core.read more
☆
Tue Sep 1 · 3:45 PM - 5:15 PM
Research 14 · Tabular Data Understanding
● Research
📍 Grand Ballroom D
Schuyler: Self-Supervised Clustering of Tables in Relational Databases (Lukas Laskowski, Hasso Plattner Institute, University of Potsdam et al.) - Databases are integral to modern applications.Schuyler: Self-Supervised Clustering of Tables in Relational Databases (Lukas Laskowski, Hasso Plattner Institute, University of Potsdam et al.) - Databases are integral to modern applications. They enable the efficient processing of vast amounts of data, making it possible to build services that support millions of users. However, these systems often comprise hundreds of interconnected tables, complicating maintenance and comprehension. To effectively operate them, having an overview of the database is of utmost importance. Database table clustering involves grouping semantically related tables, which simplifies many database management, analysis, and integration tasks. We present Schuyler, a system that clusters database tables by combining structural and semantic features of the database. Specifically, Schuyler fine-tunes a large language model in a self-supervised manner using triplet-loss to produce high-quality embeddings representing table semantics. Subsequently, these embeddings are clustered to achieve a database table clustering. Our approach requires no labeled training data and, thus, is applicable to arbitrary databases. To validate Schuyler and benchmark it against state-of-the-art competitors, we introduce a benchmark collection consisting of five real-world databases. These databases vary significantly in size (29 - 481 tables) and complexity (3 - 47 clusters) and reflect diverse real-world challenges. Our results demonstrate that Schuyler consistently achieves superior clustering performance, improving the state-of-the-art on average by 0.13 ARI (adjusted Rand index) and 0.10 AMI (adjusted mutual information). • Replacing Multi-Step Assembly of Data Preparation Pipelines with One-Step LLM Pipeline Generation for Table QA (Fengyu Li, Zhejiang University et al.) - Table Question Answering (TQA) aims to answer natural language questions over structured tables. Large Language Models (LLMs) enable promising solutions to this problem, with operator-centric solutions that generate table manipulation pipelines in a multi-step manner offering state-of-the-art performance. However, these solutions rely on multiple LLM calls, resulting in prohibitive latencies and computational costs. We propose Operation-R1, the first framework that trains lightweight LLMs (e.g., Qwen-4B/1.7B) via a novel variant of reinforcement learning with verifiable rewards to produce high-quality data-preparation pipelines for TQA in a single inference step. To train such an LLM, we first introduce a self-supervised rewarding mechanism to automatically obtain fine-grained pipeline-wise supervision signals for LLM training. We also propose variance-aware group resampling to mitigate training instability. To further enhance robustness of pipeline generation, we develop two complementary mechanisms: operation merge, which filters spurious operations through multi-candidate consensus, and adaptive rollback, which offers runtime protection against information loss in data transformation. Experiments on two benchmark datasets show that, with the same LLM backbone, Operation-R1 achieves average absolute accuracy gains of 9.55 and 6.08 percentage points over multi-step preparation baselines, with 79% table compression and a 2.2$\times$ reduction in monetary cost. • Human-Centered Exploration of Table Unionability (Nina Klimenkova, Worcester Polytechnic Institute et al.) - Table union search (TUS) identifies tables that can be meaningfully combined by row-wise union with a given query table and is a core operation in data discovery over data lakes. Yet what it means for two tables to be unionable is inherently ambiguous: domain experts disagree even on seemingly simple cases, and existing benchmarks collapse this disagreement into binary labels, omitting the behavioral context behind human decisions. We take a human-centered view of table unionability and study how humans, traditional TUS methods, and large language models (LLMs) interact on this task. We introduce TUNE (Table UNionability with human Evaluation), a benchmark of 464 expert judgments over 26 table pairs that records binary decisions, confidence scores, decision times, interaction traces, textual explanations, and post-survey reflections. Using TUNE, we (i) characterize human performance, overconfidence, and metacognitive quality (calibration and resolution); (ii) benchmark state-of-the-art TUS methods (Starmie, SANTOS, D3L), revealing complementary strengths and systematic misalignment with expert judgments; and (iii) evaluate four experimental scenarios that combine human behavioral signals and TUS features using classical ML models and LLMs. Our best configuration reaches 84% accuracy, improving over both human majority vote and the strongest standalone TUS method, while LLMs act as useful second opinions but are sensitive to conflicting signals. Overall, our results suggest that unionability labels reflect a structured yet imperfect human decision process and that hybrid human - model pipelines provide more reliable and interpretable unionability assessments. • SciTables : A Dataset and Evaluation Framework for Complex Table-to-Text Generation (Mehrnoush Alizade, Missouri S&T et al.) - Generating coherent and factually grounded text from structured data is a core challenge in natural language generation, with appli- cations in scientific communication, medical documentation, and automated reporting. Existing datasets primarily focus on open- domain or simplified table formats, limiting progress in more com- plex, high-stakes domains. We present SciTables, a new dataset and evaluation framework for scientific table-to-text generation, addressing the gap in existing resources that focus largely on open-domain or simplified tables. Our dataset is constructed from Computer Science papers on arXiv (2017 - 2023) and features complex tables rich in numeric, symbolic, and mathematical content paired with naturally occurring textual descriptions. We develop a scalable, semi-automated pipeline to extract, clean, and align tables with their associated text, preserving domain-specific language while minimizing annotation cost. The resulting benchmark poses realistic challenges for current models and supports evaluation beyond semantic similarity, including factual accuracy, relevance, and multiple forms of reasoning. We conduct extensive experiments with state-of-the-art generation models and show that while current models achieve strong semantic alignment with reference descriptions, they struggle with higher-order reasoning, aggregation, and factual grounding as table complexity increases. Our work provides a realistic and scalable benchmark for advancing faithful, informative, and reasoning-aware table-to-text generation in scientific domains. • Efficient Task Assignment for Multi-Workerset Crowdsourcing with Time and Expense Considerations (Atsuyuki Morishima, University of Tsukuba et al.) - Large-scale crowdsourcing with high-quality results, such as online surveys and data labeling tasks, is in wide demand. Existing techniques on task assignment result quality optimization, however, have addressed only a part of optimization space that does not necessarily reflect the real-world problem. For example, in reality, the proper worker set for task assignment and the workers’ submissions to the task are non-obvious in advance, which is not considered by existing techniques. Hence, this paper discusses a task assignment algorithm that dynamically probes the proper worker set(s) and worker submission models for the task to optimize task assignments in terms of quality, time, and expense. Specifically, we introduce CrowdBwO (Crowd Bandit with Optimization), a novel multi-armed bandit algorithm that is based on batched bandits and bandits with Knapsack and incorporates worker submission models. CrowdBwO dynamically determines and utilizes proper worker set(s) and worker submission models for each task under uncertainty to achieve high-performance crowdsourcing. We conducted extensive experiments with synthetic workers and real workers to evaluate CrowdBwO in two specific problem settings. Our extensive experimental results demonstrate that CrowdBwO is significant for real-world crowdsourcing and has a high performance.read more
☆
Tue Sep 1 · 3:45 PM - 5:15 PM
Research 15 · Performance Benchmarking
● Research
📍 Grand Ballroom C
Vodka: Rethink Benchmarking Philosophy in HTAP Systems (Zirui Hu, East China Normal University et al.) - For real-time analysis of up-to-date data, hybrid transaction/analytical processing (HTAP) systems have been extensively studied.Vodka: Rethink Benchmarking Philosophy in HTAP Systems (Zirui Hu, East China Normal University et al.) - For real-time analysis of up-to-date data, hybrid transaction/analytical processing (HTAP) systems have been extensively studied. In general, three techniques play a critical role in HTAP systems, which are resource isolation, consistency model, and data sharing. However, there still lacks a benchmark suite that could comprehensively cover the three techniques. The core challenges come from the requirements of: (a) consistent workload resource consumption (provide workloads with the same computational complexity); (b) query-oriented freshness evaluation (focus on the degree of version staleness in the range of queried data); (c) precise data sharing efficiency measurement (catch the synchronization status accurately). In this paper, we propose Vodka to address the above challenges. For resource isolation, we formalize the change of query cardinalities under dynamic modifications, and manipulate the cardinalities of various query operators to ensure consistent query complexity comparisons under any data size. For consistency model, we design a column value grained version management strategy based on which query-oriented freshness is calculated. For data sharing, we design a lightweight point query driven method to check the synchronization status accurately. We finally conduct extensive experiments on three representative systems to justify our designs and provide insights for future system developments. • TPCx-AI under the Microscope: A Benchmarking Debt Analysis (Ilin Tolovski, Hasso Plattner Institute, University of Potsdam et al.) - TPCx-AI is an industry standard benchmark for evaluating the end-to-end performance of machine learning systems and the underlying hardware configurations. In the database community, individual parts of the dataset and the workloads are used to evaluate preprocessing methods and systems for fast inference. In both of these cases, the datasets and workloads are used based on the characteristics defined in the specification. Upon analysis of TPCx-AI's dataset and use cases, we observe that the official implementation of TPCx-AI's kit diverges from the specification, does not evaluate the capabilities of the system under test, and impacts the overall performance in a benchmark run. In this paper, we investigate the benchmarking debt accumulated in the TPCx-AI dataset and the workloads. We identify properties that impact the benchmark's performance, including runtime and quality of use cases, the defined metrics and their thresholds, workload discrepancies, and data errors. Our analysis shows that all use cases and datasets contain benchmarking debts, impacting the training and serving runtimes by up to 350x and 800x, respectively. By addressing these debts, we observe an end-to-end throughput increase of up to 3.8x over the default TPCx-AI implementation. • SQL-Exchange: Transforming SQL Queries Across Domains (Mohammadreza Daviran, University of Alberta et al.) - We introduce SQL-Exchange, a framework for mapping SQL queries across different database schemas by preserving the source query structure while adapting domain-specific elements to align with the target schema. We investigate the conditions under which such mappings are feasible and beneficial, and examine their impact on enhancing the in-context learning performance of text-to-SQL systems as a downstream task. Our comprehensive evaluation across multiple model families and benchmark datasets---assessing structural alignment with source queries, execution validity on target databases, and semantic correctness---demonstrates that SQL-Exchange is effective across a wide range of schemas and query types. Our results further show that both in-context prompting with mapped queries and fine-tuning on mapped data consistently yield higher text-to-SQL performance than using examples drawn directly from the source schema. • RAGPerf: An End-to-End Benchmarking Framework for Retrieval-Augmented Generation Systems (Shaobo Li, University of Illinois Urbana-Champaign et al.) - We present the design and implementation of a RAG-based AI system benchmarking (RAGPerf) framework for characterizing the system behaviors of RAG pipelines. To facilitate detailed profiling and fine-grained performance analysis, RAGPerf decouples the RAG workflow into several modular components - embedding, indexing, retrieval, reranking, and generation. RAGPerf offers the flexibility for users to configure the core parameters of each component and examine their impact on the end-to-end query performance and quality. RAGPerf has a workload generator to model real-world scenarios by supporting diverse datasets (e.g., text, pdf, code, and audio), different retrieval and update ratios, and query distributions. RAGPerf also supports different embedding models, major vector databases such as LanceDB, Milvus, Qdrant, Chroma, and Elasticsearch, as well as different LLMs for content generation. It automates the collection of performance metrics (i.e., end-to-end query throughput, host/GPU memory footprint, and CPU/GPU utilization) and accuracy metrics (i.e., context recall, query accuracy, and factual consistency). We demonstrate the capabilities of RAGPerf through a comprehensive set of experiments and open source its codebase at GitHub. Our evaluation shows that RAGPerf incurs negligible performance overhead. • Benchmarking the Full Pipeline of Materialized-View-Based Query Rewriting (Xinjie Hu, Simon Fraser University et al.) - Materialized views (MVs) accelerate OLAP and data-warehouse workloads by precomputing reusable subexpressions, but practical MV-based query acceleration is a multi-stage pipeline: candidate enumeration, view selection under storage budgets, and query rewriting inside the optimizer. Existing evaluations typically study only parts of this pipeline and within a single system, leaving end-to-end trade-offs and cross-system behavior unclear. In this paper, we benchmark MV-based query rewriting by jointly evaluating enumeration, selection, and rewriting with a modular evaluation framework and by using controlled ablations. We also introduce a cross-engine protocol allowing us to compare systems that expose only execution plans by contrasting native optimizer-level rewriting with portable SQL rewriting baselines when available. Across representative academic methods and modern open-source and commercial systems, we find strong interaction effects between stages and large variability in rewriter MV usage and realized savings. We identify recurring failure modes that explain performance regressions after rewriting. Our results highlight which pipeline stages most often limit performance and provide evidence to guide future MV enumeration, selection, and rewriting designs.read more
☆
Tue Sep 1 · 3:45 PM - 5:15 PM
Research 16 · Entity Resolution
● Research
📍 Commonwealth Ballroom C
Near-Duplicate Text Alignment under Weighted Jaccard Similarity (Yuheng Zhang, Rutgers University et al.) - Near-duplicate text alignment is the task of identifying, among the texts in a corpus, all the subsequences (\ie substrings) that are similar to a given query.Near-Duplicate Text Alignment under Weighted Jaccard Similarity (Yuheng Zhang, Rutgers University et al.) - Near-duplicate text alignment is the task of identifying, among the texts in a corpus, all the subsequences (\ie substrings) that are similar to a given query. Traditional approaches rely on seeding - extension - filtering heuristics, which lack accuracy guarantees and require many hard-to-tune parameters. Recent methods leverage min-hash techniques under the following hash-based framework: index by grouping all the subsequences using their min-hash, and for any query, use the index to find all the min-hash sketches that are similar to the query's sketch. These methods guarantee to report all the subsequences whose estimated \textit{unweighted} Jaccard similarity with the query exceeds a user-provided threshold and are efficient. However, they fail to account for token importance, or even token frequency, which limits their applicability in real-world scenarios where tokens carry weights, such as TF-IDF. To overcome this limitation, in this paper, we propose \mono, an approach that supports near-duplicate text alignment under \textit{weighted} Jaccard similarity using consistent weighted sampling. \mono achieves optimality within the above hash-based framework. For example, when token weights are proportional to their frequencies, \mono generates $O(n + n\log f_T)$ groups (in expectation due to hashing) for a text of length $n$, where $f_T$ is the maximum token frequency in the text. Each group takes $O(1)$ space and represents a few subsequences sharing the same consistent weighted sampling. We further prove that this bound is asymptotically tight: any algorithm must produce $\Omega(n + n \log f_T)$ groups in expectation in the worst case. Such an optimality over a wide range of weight functions is achieved based on multiple key insights into the hashing structure under consistent weighted sampling and is thus non-trivial. Extensive experiments demonstrate that \mono outperforms the state-of-the-art by up to $26\times$ in index construction time, reduces index size by up to $30\%$, and improves query latency by up to $3\times$, while exhibiting superior scalability. • Featurized-Decomposition Join: Low-Cost Semantic Joins with Guarantees (Sepanta Zeighami, University of California Berkeley et al.) - Large Language Models (LLMs) are being increasingly used within data systems to process large datasets with text fields. A broad class of such tasks involves a semantic join - joining two tables based on a natural language predicate per pair of tuples, evaluated using an LLM. Semantic joins generalize tasks such as entity matching and record categorization, as well as more complex text understanding tasks. A naive implementation is expensive as it requires invoking an LLM for every pair of rows in the cross product. Existing approaches mitigate this cost by first applying embedding-based semantic similarity to filter candidate pairs, deferring to an LLM only when similarity scores are deemed inconclusive. However, these methods yield limited gains in practice, since semantic similarity may not reliably predict the join outcome - resulting in costs close to the naive approach. We propose Featurized-Decomposition Join (FDJ for short), a novel approach for performing semantic joins that significantly reduces cost while preserving quality. FDJ automatically extracts features and combines them into a logical expression in conjunctive normal form that we call a featurized decomposition to effectively prune out non-matching pairs. A featurized decomposition extracts key information from text records and performs inexpensive comparisons on the extracted features. We show how to use LLMs to automatically extract reliable features and compose them into logical expressions while providing statistical guarantees on the output result - an inherently challenging problem due to dependencies among features. Experiments on real-world datasets show up to 10 times reduction in cost compared with the state-of-the-art while providing the same quality guarantees. • SeDA: Bridging the Gap between Efficient Syntactic and Precise Semantic Search of Similar Passages in Large Text Corpora (Pranay Mundra, Yale University et al.) - A two-stage pipeline is commonly used to identify similar text passages in large document corpora: First, a fast approach produces potential matches, which are then examined in detail. Existing approaches for the first step consider only syntactic information and miss semantically similar passages that are syntactically dissimilar. To address this, we define the novel problem of semantic document alignment as a semantic set-similarity problem on k-width windows. For two documents S and T, an exhaustive baseline that evaluates all |S|x|T| window pairs is computationally infeasible since assessing the similarity of a single pair requires O(k³) time. We propose SeDA, which combines a sophisticated candidate generation technique with a bound cascade to drastically reduce the number of expensive window comparisons. It further exploits overlapping windows to efficiently compute both the bounds and the final similarity scores. Our empirical results on three large document corpora indicate that SeDA prunes over 99% of the window similarity computations, resulting in response-time improvements of 1.5-3 orders of magnitude over the baseline solution and 2-5 orders of magnitude over SBERT. Compared to purely syntactic competitors, SeDA provides competitive runtimes and achieves superior result quality, i.e., near-optimal F1-Score of precision/recall and matching the performance of purely semantic methods such as SBERT. • Can we trust LLM Self-Explanations for Entity Resolution? (Tommaso Teofili, Roma Tre University et al.) - Large Language Models (LLMs) have recently shown strong performance on Entity Resolution (ER). Additionally, akin to their prowess in providing accurate predictions, these models often generate self-explanations alongside their predictions through prompting. While such self-explanations are appealing due to their negligible computational cost, their actual reliability remains largely unexplored. In this paper, we conduct the first large-scale, systematic evaluation of LLM self-explanations for ER, focusing on saliency-based and counterfactual explanations at both the attribute and token levels. Across three LLMs, ten datasets, and multiple prompting strategies, we show that self-explanations are often unstable, weakly faithful, and poorly aligned with counterfactual evidence, revealing a substantial gap between plausibility and causal relevance. We further demonstrate that established post-hoc explanation methods provide significantly higher trustworthiness, but at a prohibitive computational cost when applied to LLMs. To bridge this gap, we introduce \uncerta{}, a hybrid explanation framework that leverages self-explanations as priors to guide post-hoc exploration. \uncerta{} achieves explanation quality comparable to post-hoc methods while reducing cost by up to an order of magnitude. • PINE: Extracting Correlated Token Pairs for Explainable Entity Matching (Hironori Mizuguchi, University of Tsukuba et al.) - Explanation techniques such as local interpretable model-agnostic explanation (LIME) provide reasons behind decisions made by machine-learning models. These methods typically use a set of features and their values as inputs and identify those that significantly influence the final decision. However, machine-learning models for entity matching operate on two sets of tokens or records, each representing an entity, to determine whether they refer to the same real-world entity. Explanations for entity-matching decisions are more convincing when they highlight contributing pairs of tokens within the pair of records, rather than focusing on individual tokens alone. In this sense, existing explanation techniques are insufficient for entity matching. Therefore, we propose a new method, Pair INterpretation for Entity matching (PINE), which takes two records as input, and outputs correlated token pairs as an explanation for an entity-matching decision. Our extensive experiments on public datasets demonstrate that the extracted token pairs exhibit strong correlations and serve as interpretable evidence for matching records.read more
☆
Tue Sep 1 · 3:45 PM - 5:15 PM
Research 17 · Key-Value Stores
● Research
📍 Commonwealth Ballroom B
Terark-DS: A High-Performance and Storage-Efficient Key-Value Separation Storage Engine on Disaggregated Storage (Jianshun Zhang, Huazhong University of Science and Technology et al.) - Log-structured merge-trees (LSM-trees) are widely adopted in modern storage systems for high write throughput, but suffer from...Terark-DS: A High-Performance and Storage-Efficient Key-Value Separation Storage Engine on Disaggregated Storage (Jianshun Zhang, Huazhong University of Science and Technology et al.) - Log-structured merge-trees (LSM-trees) are widely adopted in modern storage systems for high write throughput, but suffer from significant write amplification. Key-value (KV) separation mitigates this issue but introduces higher space overhead. To improve cost efficiency and resource elasticity, modern storage systems increasingly adopt compute-storage disaggregated architectures. However, disaggregation increases network overhead for data access, degrading write performance. It also prolongs garbage collection (GC), which increases the space cost of KV-separated LSM-trees. In this paper, we propose Terark-DS, a high-performance and storage-efficient KV separation storage engine on disaggregated storage. To achieve both high performance and low cost, Terark-DS employs differentiated redundancy based on LSM-tree access patterns, adaptive write-ahead logging that switches between serial and parallel modes for different batch sizes, and a network-efficient GC design to accelerate GC execution. Experiments show that Terark-DS outperforms existing disaggregated LSM-trees by 20.4%-63.9% in write throughput while reducing total costs by 22.7%-58.6%. • ArceKV: Towards Workload-driven LSM-compactions for Key-Value Store Under Dynamic Workloads (Junfeng Liu, Nanyang Technological University et al.) - Key-value stores underpin a wide range of applications due to their simplicity and efficiency. Log-Structured Merge Trees (LSM-trees) dominate as their underlying structure, excelling at handling rapidly growing data. Recent research has focused on optimizing LSM-tree performance under static workloads with fixed read - write ratios. However, real-world workloads are highly dynamic, and existing workload-aware approaches often struggle to sustain optimal performance or incur substantial transition overhead when workload patterns shift. To address this, we propose ElasticLSM, which removes traditional LSM-tree structural constraints to allow more flexible management actions (i.e., compactions and write stalls) creating greater opportunities for continuous performance optimization. We further design Arce, a lightweight compaction decision engine that guides ElasticLSM in selecting the optimal action from its expanded action space. Building on these components, we implement ArceKV, a full-fledged key-value store atop RocksDB. Extensive evaluations demonstrate that ArceKV outperforms state-of-the-art compaction strategies across diverse workloads, delivering around 3$\times$ faster performance in dynamic scenarios. • Dynamic read & write optimization with TurtleKV (Tony Astolfi, Tufts University and Mathworks et al.) - High read and write performance is important for generic key-value stores, which are foundational to modern applications and databases. Yet, achieving high performance for mixed and dynamic workloads is challenging due to fundamental trade-offs between memory use and I/O for retrieval and updates. Past work emphasizes the trade-off between read- and write-optimization as expressed through primary data structure, in combination with read-memory trade-off mechanisms like caching and filtering. This raises re-tuning costs as optimal trade-off targets change, due to restructuring of stored data. We show that write-memory trade-off mechanisms are under-developed in current designs, and propose a new approach to dynamic key-value store optimization using a novel read-/write-balanced on-disk structure, the TurtleTree, and flexible read-/write-memory tuning knobs. We describe how to avoid in-memory bottlenecks to achieve high performance across a wide tuning range. When evaluated using YCSB, our prototype TurtleKV matches state-of-the-art SplinterDB for inserts, and is 5x/12x faster than RockDB/WiredTiger. In mixed workloads, TurtleKV is 16-25% faster than SplinterDB, >4x RocksDB, and 3-6x WiredTiger. TurtleKV is 2-9x faster than the others for point-query workloads, and has the best scan rate of the write-optimized systems tested. • How Much Can RocksDB Chew? Achieving Near-Zero Write Stalls with Sustainable RocksDB (Hojin Shin, Dankook University et al.) - Modern data-intensive applications, from microservices to real-time AI serving, demand consistently low tail latency from backend storage. However, Log-Structured Merge-tree (LSM)-based Key-Value Stores (KVSs) like RocksDB are structurally prone to unpredictable write stalls. These stalls stem from a fundamental architectural decoupling of foreground write ingress and background data reorganization. By design, the system absorbs foreground writes at maximum speed without monitoring its actual time-varying compaction capacity. As a result, it accumulates internal pressure until rigid capacity thresholds are breached, triggering reactive safeguards that abruptly freeze all foreground writes. Relying on this reactive ``stop-and-go'' approach induces a persistent limit-cycle behavior, undermining long-run predictability and strict latency guarantees. We reframe write stalls as a continuous control problem. S-RocksDB is a sustainable admission controller that regulates foreground ingress to match the system's time-varying compaction capacity. Since this capacity varies at runtime, S-RocksDB employs online reinforcement learning to discover a sustainable admission rate. To ensure safe learning, a three-state operational model (SAFE, SEMI-SAFE, UNSAFE) confines exploration to stable conditions and deploys deterministic guardrails before stalls can occur. In 24-hour evaluations, S-RocksDB reduces over 64.3 million stall events to just 69, bounds P99.99 tail latency to sub-0.11 ms, and delivers predictable throughput with the lowest resource footprint among all compared systems. • Tidehunter: Large-Value Storage With Minimal Data Relocation (Andrey Chursin, Mysten Labs et al.) - Log-Structured Merge-Trees (LSM-trees) dominate persistent key-value storage but suffer from high write amplification from 10x to 30x under random workloads due to repeated compaction. This overhead becomes prohibitive for large values with uniformly distributed keys, a workload common in content-addressable storage, deduplication systems, and blockchain validators. We present Tidehunter, a storage engine that eliminates value compaction by treating the Write-Ahead Log (WAL) as permanent storage rather than a temporary recovery buffer. Values are never overwritten; and small, lazily-flushed index tables map keys to WAL positions. Tidehunter introduces (a) lock-free writes that saturate NVMe drives through atomic allocation and parallel copying, (b) an optimistic index structure that exploits uniform key distributions for single-roundtrip lookups, and (c) epoch-based pruning that reclaims space without blocking writes. On a 1,TB dataset with 1,KB values, Tidehunter achieves 830K writes per second, that is 8.4x higher than RocksDB and 2.9x higher than BlobDB, while improving point queries by 1.7x and existence checks by 15.6x. We validate real-world impact by integrating Tidehunter into Sui, a high-throughput blockchain, where it maintains stable throughput and latency under loads that cause RocksDB-backed validators to collapse. Tidehunter is production-ready and is being deployed in production within Sui.read more
☆
Tue Sep 1 · 3:45 PM - 5:15 PM
Research 18 · Graph Mining
● Research
📍 Commonwealth Ballroom A
gMatch: Fine-Grained and Hardware-Efficient Subgraph Matching on GPUs (Weitian Chen, Shanghai Jiao Tong University et al.) - Subgraph matching is a core operation in graph analytics, supporting a broad spectrum of applications from social network analysis to bioinformatics.gMatch: Fine-Grained and Hardware-Efficient Subgraph Matching on GPUs (Weitian Chen, Shanghai Jiao Tong University et al.) - Subgraph matching is a core operation in graph analytics, supporting a broad spectrum of applications from social network analysis to bioinformatics. Recent GPU-based approaches accelerate subgraph matching by leveraging parallelism but rely on a coarse-grained execution model that suffers from scalability and efficiency issues due to high memory overhead and thread underutilization. In this paper, we propose gMatch, a hardware-efficient subgraph matching approach on GPUs. gMatch introduces a fine-grained execution model that reduces memory consumption and enables flexible task scheduling among threads. We further design warp-level batch exploration and lightweight load balancing to improve execution efficiency and scalability. Experiments on diverse workloads and real-world datasets show that gMatch outperforms state-of-the-art subgraph matching methods, including STMatch, T-DFS, and EGSM, in both performance and scalability. We also compare against state-of-the-art systems for mining small patterns, such as BEEP and G$^2$Miner. While these systems achieve better performance on small datasets, gMatch scales to substantially larger queries and datasets, where existing approaches degrade or fail to complete. • Effective Durable Community Search in Large Temporal Graph (Yingli Zhou, The Chinese University of Hong Kong, Shenzhen et al.) - A temporal graph is an undirected graph where each edge is associated with a timestamp indicating when it occurs. As a fundamental topic in graph analysis, community search (CS) in temporal graphs has received much attention. Existing CS works on temporal graphs typically identify sets of vertices that form a $k$-core within a specific time window (temporal $k$-core). However, they overlook the duration of a temporal community, which is the continues time period that its members remain unchanged. Intuitively, the longer the duration of a temporal community, the higher its stability. Long-duration communities are useful in many areas, such as event detection and network analysis. In this paper, we introduce a novel community model, called temporal durable community (TDC), which is the temporal $k$-core with the longest duration in the temporal graph, and aim to efficiently find the TDC containing a query vertex. To solve this problem, we first propose a novel online algorithm based on binary search. We further develop two index structures that can quickly determine the duration of a given temporal $k$-core, followed by query algorithms. Experiments on ten real large temporal graphs show that our TDC model is effective for finding stable communities, and our index-based query algorithms are up to five orders of magnitude faster than the online algorithm. • Efficient Partition-based Approaches for Diversified Top-k Subgraph Matching (Liuyi Chen, Hunan University et al.) - Subgraph matching is a core task in graph analytics, widely used in domains such as biology, finance, and social networks. Existing top $k$ diversified methods typically focus on maximizing vertex coverage, but often return results in the same region, limiting topological diversity. We propose the Distance-Diversified Top-$k$ Subgraph Matching (DT$k$SM) problem, which selects $k$ isomorphic matches with maximal pairwise topological distances to better capture global graph structure. To address its computational challenges, we introduce the Partition based Distance Diversity (PDD) framework, which partitions the graph and retrieves diverse matches from distant regions. To further enhance efficiency, we develop two optimizations: embedding-driven partition filtering and densest-based partition selection over a Partition Adjacency Graph. Experiments on 12 real world datasets show our approach achieves up to four orders of magnitude speedup over baselines, with 95\% of results reaching 80\% of optimal distance diversity and 100\% coverage diversity. • Mix & Match: Subgraph Matching for Absolute Coverage (Konstantinos Skitsas, Aarhus University et al.) - The NP-hard problem of subgraph matching calls to detect all matchings of a smaller query graph within a larger data graph. The problem is fundamental in graph analysis and query answering, as it facilitates the understanding and analysis of the larger graph. Nevertheless, existing subgraph matching methods return results from one location of the graph before moving to another location, while the total results may be in the order of billions or even trillions; under these circumstances, existing methods may only present a portion of the results within reasonable time or space, which is not representative of the totality of results. This predicament leads to a biased representation of the data graph. In this paper, we study the problem of coverage in subgraph matching and propose Mix & Match (M&M) an algorithm that quickly returns results that are representative of the whole data graph. M&M achieves higher coverage employing a combination of global exploration, which prioritizes the exploration of nodes at the first level of backtracking that may enlarge coverage, and local exploration, which improves backtracking efficiency by pruning exploration paths that do not increase coverage. Our experimental study shows that M&M finds on average twice as many unique nodes as state-of-the-art algorithms in the same time. • Subgraph Enumeration: Beyond Tree Decomposition (Qiyan Li, The Chinese University of Hong Kong et al.) - We address the subgraph enumeration problem: given an unlabeled pattern graph 𝑝 and an unlabeled data graph 𝐺, find all subgraphs in 𝐺 isomorphic to 𝑝. Unlike labeled matching, the absence of label constraints creates exponentially larger search spaces with limited pruning opportunities. To address this challenge, we follow tree decomposition (TD) approaches that break complex patterns into smaller subgraphs (bags), compute matches for each bag, and join them to obtain final results. However, existing TD approaches suffer from suboptimal decomposition selection, incomplete symmetry-breaking usage, and expensive intermediate result materialization. We present MDSE (Minimal Decomposition-based Subgraph Enumeration) with three key contributions. We introduce minimal fractional hypertree decompositions (MinFHDs) that ensure compact bags and an efficient algorithm to explore all optimal-width decompositions. We develop new symmetry-breaking integration using complete rule sets with systematic selection for maximum pruning effect. To reduce materialization costs, we design MixJoin by embedding final result assembly within bag processing and formulate an enhanced cost model for attribute orders, incorporating both intersection and materialization overhead. Evaluation across 101 pattern graphs and 8 real-world datasets shows MDSE substantially outperforms existing algorithms.read more
☆
Tue Sep 1 · 3:45 PM - 5:15 PM
Industry Session 3 · Vector Search, Graph Analytics & In-Database AI
● Industry
📍 Marina Ballroom I
BiLink: Bidirectional Meta-paths for Link Discovery in Billion-Scale Heterogeneous Graphs (Jinquan Hang, Rutgers University et al.) - Link discovery in heterogeneous graphs, i.e., finding target edges between existing nodes, is a key task in many real-world applications.BiLink: Bidirectional Meta-paths for Link Discovery in Billion-Scale Heterogeneous Graphs (Jinquan Hang, Rutgers University et al.) - Link discovery in heterogeneous graphs, i.e., finding target edges between existing nodes, is a key task in many real-world applications. Since some path types between node pairs are related to target edges, the meta-path is widely used to represent these path types. However, the complexity of meta-path sampling grows exponentially with hop count, making long meta-paths impractical to sample on billion-scale graphs. Bidirectional search offers a natural way to split a long meta-path into two shorter meta-paths, each starting from one of its endpoints. However, different meta-paths have different structures, making it difficult to parallelize bidirectional meta-path sampling with uniform operations. Furthermore, existing models are not designed to jointly leverage the meta-path instances from bidirectional sampling. To address these challenges, we first introduce bidirectional meta-path, a unified path schema for bidirectional paths connecting node pairs. Based on this, we present BiLink, a framework for link discovery in billion-scale heterogeneous graphs. Specifically, BiLink represents all meta-paths in a unified table format and implements bidirectional meta-path sampling in distributed systems with uniform table operations. It also includes BiPathNN, a model that represents path instances as embedding sequences and jointly encodes them via the Transformer encoder for link prediction. Experiments show that bidirectional meta-path sampling is over 13 times faster than traditional meta-path sampling, while BiPathNN improves average precision by over 6% compared to state-of-the-art baselines. Moreover, BiLink has been deployed to discover companies' key personnel from 1.8 billion candidate pairs, increasing contact success rate by 86%. • Virtualizing Recursion: Just-In-Time Graph Analytics in a Hyperscale Relational Warehouse (Jason Arnold, Ocient et al.) - Modern hyperscale data warehouses (MPP) face increasing pressure to support complex, non-linear query paradigms. While the industry is moving toward standardized graph query languages like ISO GQL and SQL:2023 (SQL/PGQ), supporting these on shared-nothing architectures traditionally requires specialized, heavyweight physical operators (e.g., recursive union). This complicates the scheduler and often forces a trade-off between latency and scale. Furthermore, in high-availability environments where metadata operations require distributed consensus, the latency of creating ephemeral tables for recursion becomes prohibitive. This paper presents Ocient’s solution: decoupling the definition of recursion from the execution via Dataflows: a procedural superset of SQL. This architecture enables support for recursive SQL and is well-positioned as a transpilation target for emerging GQL and SQL:2023 workloads. We describe the implementation of a table virtualization runtime that inspects intermediate row counts to automatically switch between in-memory CTE injection (for low latency) and NVMe-backed storage (for massive scale). The system transparently transitions storage-backed tables back into virtual memory objects as they shrink or vice versa on growing tables. This unified substrate allows Ocient to execute hyperscale graph algorithms (such as Weakly Connected Components on graphs exceeding 100 billion edges) directly within the relational warehouse. • Benchmarking Native In-Database TPCx-AI at 100 Terabytes in Ocient Hyperscale Data Warehouse (Jason Arnold, Ocient et al.) - The growth of enterprise data has created a scalability gap in machine learning (ML) workflows. As data volumes climb into the hundreds of terabytes, extracting raw data to external frameworks such as Apache Spark imposes increasing costs in serialization latency, network transfer, and data governance. Audited TPCx-AI results, the standard benchmark for end-to-end ML pipelines, have been limited to scale factor 3,000 (3 TB) to date. This paper presents the architecture and performance of the Ocient Hyperscale Data Warehouse executing eight of the ten TPCx-AI use cases at scale factor 100,000 (100 TB). To our knowledge, this is the largest reported TPCx-AI execution, more than an order of magnitude beyond the largest audited result; the two omitted use cases require deep-learning model families our engine does not yet implement natively. For the eight executed use cases, we demonstrate linear or sub-linear scalability across six scale factors (1 GB to 100 TB), meeting the accuracy target at every scale except one minority-class classification case whose threshold is also missed by our algorithm-matched Spark implementation, indicating the gap is intrinsic to the synthetic dataset rather than to either engine. We detail a database architecture that feeds query operators directly from NVMe drives to CPU registers via userspace polling, and a memory system using 128 KB fragments within 1 GB hugepages. We achieve purely relational execution of iterative algorithms using native matrix types and operators in standard SQL: no UDFs, no embedded R/Python runtime, and no separate linear-algebra DSL. We further discuss a native C++ matrix library and an LLVM-based JIT compiler that translates decision tree models directly into machine code, and report an algorithm-matched comparison against Apache Spark across four scale factors from 100 GB to 100 TB. • SQL-Native Vector Search at Billion Scale in Presto (Zhichen Xu, Meta Platforms, Inc. et al.) - We present Presto Vector Search, a SQL-native distributed vector search system built on the observation that partition-based vector search decomposes naturally into relational algebra-partitioning as scalar functions, index construction as GROUP BY aggregation, search as equi-joins - with the local index type (FLAT, IVF, RaBitQ, HNSW) as a pluggable parameter within the aggregate. This decomposition enables pre-filter indexing, arbitrary joins and aggregations on results, and optimizer-driven zero-shuffle distributed execution as direct consequences of the relational mapping. A two-level API-a declarative table function rewritten by the optimizer into distributed SQL primitives - ensures both novice and expert users share the same optimized execution path. We validate across four workloads spanning three modalities (512- to 5,120-dim) and up to 2B+ vectors, achieving 95 - 99% recall on workloads with ground-truth labels, with 95 - 99% CPU reduction over brute-force baselines and ~7,000× network reduction via codec-driven co-located execution. On the public DEEP1B benchmark (1B vectors), end-to-end BUILD+SEARCH completes in under 2.5 minutes on 200 workers with 96.6% Recall@1 against official ground truth. • Nova: A Multi-Purpose Vector Engine for Low-Latency, Multi-Tenant, and Cross-Table Hybrid Retrieval (Dechuang CHEN, Alibaba Cloud et al.) - Vector retrieval underpins AI applications like recommendation, image search, and RAG. While similarity search is mature, production deployments struggle with high-concurrency ingestion, dense multi-tenancy, and complex cross-table hybrid queries bridging structured and unstructured data. Existing systems often fail to reconcile these demands, leading to resource contention and data staleness. We propose {\em Nova}, a cloud-native, multi-purpose vector engine integrated into Alibaba Cloud’s AnalyticDB-PG (ADB-PG). Nova introduces a {\em dual-layer delta-base architecture} that decouples real-time ingestion from indexing, ensuring low-latency ingestion, immediate data visibility, and lock-free queries even under bursty workloads. To support diverse storage hierarchies, Nova implements {\em Nova Memory} for ultra-low latency and {\em Nova Disk} for cost-effective, large-scale storage. Crucially, Nova treats vector search as a first-class citizen within the relational optimizer, enabling efficient cross-table hybrid retrieval via partial/covering indexes and cost-based adaptive joins. We further enhance multi-tenant isolation and stability through lightweight concurrent indexing and anytime query execution. Evaluations on standard benchmarks and production workloads show Nova significantly outperforms Milvus and pgvector, achieving lower latency, higher throughput, and stronger read/write stability. • TEngineDB-V: An OLAP-Native Vector Search System for Large-𝑘 Workloads at Tencent (Xufei Wu, Shanghai Jiao Tong University et al.) - Vector search systems are essential infrastructure for modern data-driven applications. Large-k analytical vector search, which retrieves k=10^3-10^5 results for analytics (e.g., aggregation, filtering, joins), is increasingly important for emerging workloads, including LLM data management and advertising analysis at Tencent. Existing systems remain inadequate: specialized vector databases often cap k (e.g., k <= 10^4) to satisfy tail-latency constraints and offer limited analytical support, while OLAP systems typically embed per-segment vector indexes as black boxes, causing severe read/compute amplification and preventing native query optimization. This paper presents TEngineDB-V, an OLAP-native vector search system for large-k workloads. TEngineDB-V makes vector search a first-class analytical primitive in Tencent's OLAP engine through a global segment-decoupled index materialized as relational tables, eliminating scatter--gather execution, reducing amplification, and enabling native storage optimizations. It decomposes IVFPQ-based search into relational operators, integrates OLAP optimizations, and introduces DPPQ, which combines direction-aware quantization with hierarchical residual refinement to improve recall while preserving relational efficiency. TEngineDB-V further incorporates index-aware query rewriting and a distributed-aware cost model for efficient distributed execution. Experiments show that TEngineDB-V achieves up to a 145x speedup over competitive systems such as StarRocks, and up to a 52x improvement in 10-billion-scale production deployments.read more
☆
Tue Sep 1 · 3:45 PM - 5:15 PM
Panel 1 · HW-SW Co-Design for Databases in the Age of AI-for-Systems
● Panels
📍 Grand Ballroom A+B
HW-SW Co-Design for Databases in the Age of AI-for-Systems
Monica ChiosaMIT
Jana GicevaTU Munich
Tae Jun HamGoogle
Matteo InterlandiMicrosoft Gray Systems Lab
Vasiliki (Vasia) KalavriBoston University
Clemens LutzNVIDIA
☆
Tue Sep 1 · 3:45 PM - 5:15 PM
Demo Group B · Query Processing, AI & Data Agents
● Demos
📍 Marina Ballrooms II - IV
31 demos in this group, shown twice during the conference.31 demos in this group, shown twice during the conference. Includes: Minimal Data Cleaning for Model Training by MinPrep • A Demonstration of Continuous Lifelong Conflict-Aware AGV Routing with Kinematic Constraints • FilterPilot: An Interactive Assistant for Adapting Filtering Predicate to Table Content • GraphAgent: An Effective Knowledge-Guided GNN Model Selection System • CADENZA in Action: Breaking the Monolith with Intent-Dependent Plan Spaces for Semantic Queries • MCAD: Multivariate Correlation Anomaly Data Generator • Carnot: Interpretable, Interactive, and Optimized Execution of Deep Research Queries • NiceT: Named Entity Cleaning and Enhancement with Human-in-the-loop • A Portable Middleware for Plan-Based Adaptive Query Processing • Policy-Aware Federated Query Orchestration Across Energy Data Spaces and Edge AI Services • … +21 moreread more
☆
Tue Sep 1 · 3:45 PM - 5:15 PM
Sponsor Talks 3
● Sponsor Talks
📍 Otis
Extreme-Scale Transaction Processing: Architectural Lessons from Mission-Critical Deployments (Jeff Fried, InterSystems) • Analytics on Special HW from Research to Product (Carlo Curino, Microsoft) • OceanBase Lakebase for the AI Era (Quanqing Xu, OceanBase) • Agent-Native Data Infrastructure: LakehouseRT, Lakebase, and...Extreme-Scale Transaction Processing: Architectural Lessons from Mission-Critical Deployments (Jeff Fried, InterSystems) • Analytics on Special HW from Research to Product (Carlo Curino, Microsoft) • OceanBase Lakebase for the AI Era (Quanqing Xu, OceanBase) • Agent-Native Data Infrastructure: LakehouseRT, Lakebase, and LTAP (Ippokratis Pandis, Databricks)read more
Built for you by(617)-PATRICK, Champion AI speaker
130 research papers presented as posters. Full list on the official program page (footer link) - the papers also appear in their research sessions on this schedule.
VLDB 2026 Special Event: Return to the Boston Area After 50+ Years: What can we learn from VLDB 1975?
● Keynotes & Plenary
📍 Grand Ballroom A+B
Panel marking VLDB's return to the Boston area after 50+ years, reflecting on lessons from VLDB 1975. • Welcome speech by Divesh Srivastava (AT&T, Plenary Session Organizer) • Session 1: Leading up to VLDB 1975 - Philip A.Panel marking VLDB's return to the Boston area after 50+ years, reflecting on lessons from VLDB 1975. • Welcome speech by Divesh Srivastava (AT&T, Plenary Session Organizer) • Session 1: Leading up to VLDB 1975 - Philip A. Bernstein (Microsoft Research), Michael Stonebraker (MIT), then Q&A • Session 2: Memories of a few VLDB 1975 Papers - David DeWitt (MIT), Sebastian Link (University of Auckland), C. Mohan (Hong Kong Baptist University), then Q&A • Session 3: Perspectives on the impact of VLDB - Renée J. Miller (University of Waterloo), Volker Markl (TU Berlin), Lei Chen (HKUST (Guangzhou)), then Q&Aread more
Abacus: A Cost-Based Optimizer for Semantic Operator Systems (Matthew Russo, MIT et al.) - LLMs enable an exciting new class of data processing applications over large collections of unstructured documents.Abacus: A Cost-Based Optimizer for Semantic Operator Systems (Matthew Russo, MIT et al.) - LLMs enable an exciting new class of data processing applications over large collections of unstructured documents. Several new programming frameworks have enabled developers to build these applications by composing them out of semantic operators: a declarative set of AI-powered data transformations with natural language specifications. These include LLM-powered maps, filters, joins, etc. used for document processing tasks such as information extraction, summarization, and more. While systems of semantic operators have achieved strong performance on benchmarks, they can be difficult to optimize. An optimizer for this setting must determine how to physically implement each semantic operator in a way that optimizes the system globally. Existing optimizers are limited in the number of optimizations they can apply, and most (if not all) cannot optimize system quality, cost, or latency subject to constraint(s) on the other dimensions. In this paper we present Abacus, an extensible, cost-based optimizer which searches for the best implementation of a semantic operator system given a (possibly constrained) optimization objective. Abacus estimates operator performance by leveraging a minimal set of validation examples, prior beliefs about operator performance, and/or an LLM judge. We evaluate Abacus on document processing workloads in the biomedical and legal domains (BioDEX; CUAD) and multi-modal question answering (MMQA). We demonstrate that, on-average, systems optimized by Abacus achieve 6.7%-39.4% better quality and are 10.8x cheaper and 3.4x faster than the next best system. • ReSequel: Robust LLM-assisted Query Rewriting and Optimization using Templatization and Sampling (Saeed Fathollahzadeh, Concordia University et al.) - Heuristic query rewriting has long complemented cost-based optimization to improve performance. Such rewrites transform SQL queries into semantically equivalent forms that are easier or faster to execute. Examples are standardizing expressions, eliminating redundancy, propagating constants, pushing down selections and projections, unnesting queries, and utilizing constraints. Modern DBMSs implement hundreds to thousands of such rules, but maintaining them is notoriously difficult. The interactions among rules are complex, and their static nature and application order prevent adaptation to specific query and database characteristics. Recent approaches that use large language models (LLMs) for query rewriting show promise but face challenges regarding the large search space, reliable query verification, and exploitation of metadata. We present ReSequel, an outer optimization layer on top of existing DBMSs to rewrite SQL queries using LLMs. ReSequel leverages catalog and statistical metadata to infer template-specific rules that guide the LLM toward effective query transformations. We generate, verify, and rank rewritten query variants on sampled data to ensure result correctness and runtime improvements. Our experiments cover eight benchmarks: JOB, TPC-H, Stats(-CEB), Public BI, IMDB, DSB, and StackOverflow; multiple DBMSs: PostgreSQL, MySQL, and DuckDB; as well as LLM-based query rewriting baselines. ReSequel yields workload-level speedups of up to 16x over native DBMSs and 22x over LLM-based systems, with individual queries exceeding 600x, across eight benchmarks and three DBMSs. • SemBench: A Benchmark for Semantic Query Processing Engines (Jiale Lao, Cornell University et al.) - We present a benchmark targeting a novel class of systems: semantic query processing engines. Those systems rely inherently on generative and reasoning capabilities of state-of-the-art large language models (LLMs). They extend SQL with semantic operators, configured by natural language instructions, that are evaluated via LLMs and enable users to perform various operations on multimodal data. Our benchmark introduces diversity across three key dimensions: scenarios, modalities, and operators. Included are scenarios ranging from movie review analysis to car damage detection. Within these scenarios, we cover different data modalities, including images, audio, and text. Finally, the queries involve a diverse set of operators, including semantic filters, joins, mappings, ranking, and classification operators. We evaluated our benchmark on three academic systems (LOTUS, Palimpzest, and ThalamusDB) and one industrial system, Google BigQuery. Although these results reflect a snapshot of systems under continuous development, our study offers crucial insights into their current strengths and weaknesses, illuminating promising directions for future research. • ELT-Bench: An End-to-End Benchmark for Evaluating AI Agents on ELT Pipelines (Tengjun Jin, UIUC et al.) - Practitioners are increasingly turning to Extract-Load-Transform (ELT) pipelines with the widespread adoption of cloud data warehouses. However, designing these pipelines often involves significant manual work to ensure correctness. Recent advances in AI-based methods, which have shown strong capabilities in data tasks, such as text-to-SQL, present an opportunity to alleviate manual efforts in developing ELT pipelines. Unfortunately, current benchmarks in data engineering only evaluate isolated tasks, such as using data tools and writing data transformation queries, leaving a significant gap in evaluating AI agents for generating end-to-end ELT pipelines. To fill this gap, we introduce ELT-Bench, an end-to-end benchmark designed to assess the capabilities of AI agents to build ELT pipelines. ELT-Bench consists of 100 pipelines, including 835 source tables and 203 data models across various domains. By simulating realistic scenarios involving the integration of diverse data sources and the use of popular data tools, ELT-Bench evaluates AI agents’ abilities in handling complex data engineering workflows. AI agents must interact with databases and data tools, write code and SQL queries, and orchestrate every pipeline stage. We evaluate four representative code agents with six popular Large Language Models (LLMs) on ELT-Bench. The highest-performing agent, OpenHands CodeActAgent Claude-3.5-Sonnet, correctly generates only 11.3% of data models, with an average cost of $1.41 and 72.2 steps per pipeline. Our experimental results demonstrate the challenges of ELT-Bench and highlight the need for a more advanced AI agent to reduce manual effort in ELT workflows. • stratum: A System Infrastructure for Massive Agent-Centric ML Workloads (Arnab Phani, TU Berlin et al.) - Recent advances in large language models (LLMs) transform how machine learning (ML) pipelines are developed and evaluated. LLMs enable a new type of workload, agentic pipeline search, in which autonomous or semi-autonomous agents generate, validate, and optimize complete data science pipelines. These agents predominantly operate over popular Python ML libraries and exhibit highly exploratory behavior. This results in thousands of executions for data profiling, pipeline generation, and iterative refinement of pipeline stages and hyperparameters. However, the existing Python-based ML ecosystem is built around libraries such as Pandas and scikit-learn, which are designed for human-centric, interactive, sequential workflows and remain constrained by Python¡¯s interpretive execution model, library-level isolation, and limited runtime support for executing large numbers of pipelines. Meanwhile, many high-performance ML systems proposed by the systems community either target narrow workload classes or require specialized programming models, which limits their integration with the Python ML ecosystem and makes them largely ill-suited for adoption by LLM-based agents. This growing mismatch exposes a fundamental systems challenge in supporting agentic pipeline search at scale. We therefore propose stratum, a unified system infrastructure that decouples pipeline execution from planning and reasoning during agentic pipeline search. Stratum integrates seamlessly with existing Python libraries, compiles batches of agent- or human-generated pipelines into optimized execution graphs, and efficiently executes them across heterogeneous backends, including a novel Rust-based runtime. We present stratum's architectural vision along with an early prototype, discuss key design decisions, and outline open challenges and research directions. Finally, preliminary experiments show that stratum can significantly speed up large-scale agentic pipeline search up to 16.6x.read more
☆
Wed Sep 2 · 10:45 AM - 12:15 PM
Research 20 · Learned Query Optimization
● Research
📍 Grand Ballroom D
OBELISK: Efficient Offline Query Planning with Bayesian Optimization-Informed Language Model Reasoning (Zhicheng Pan, East China Normal University et al.) - Query optimization (QO) remains a fundamental challenge in the database community.OBELISK: Efficient Offline Query Planning with Bayesian Optimization-Informed Language Model Reasoning (Zhicheng Pan, East China Normal University et al.) - Query optimization (QO) remains a fundamental challenge in the database community. Despite decades of research, cost-based QO (CQO) is still susceptible to performance regressions due to inherent inaccuracies in cardinality estimation, cost modeling, and plan enumeration. To mitigate the instability, modern databases employ SQL plan management (SPM), which reuses curated plans and bypasses CQO. However, there exists a fundamental issue in SPM: how can we efficiently identify the optimal plans to manage? The existing approach falls short due to low generalizability and poor interpretability. Thus, we argue for revisiting this problem from a novel perspective, where we intervene the sensitivity of CQO through well-designed cost scaling knobs. Nevertheless, this transformation poses three key challenges: (1) efficient search guidance, (2) comprehensive semantic utilization, and (3) cost-effective performance evaluation. To address these challenges, we propose OBELISK, an offline Bayesian optimization-informed query plannIng framework, with language model reasoning over cost scaling knobs. OBELISK is training-free and can efficiently find optimal query plan through a closed-loop process: a timeout-constrained Bayesian optimization technique to identify promising knob subspaces, thereby informing the search; a feedback-aware self-evolving reasoner to recommend knob configurations; and a lightweight evaluator with history-based admission gatekeeper to avoid redundant evaluations. Extensive experiments on well-established benchmarks demonstrate the effectiveness and superiority of OBELISK. • TATA: An Efficient Framework for Task Transfer in Query Plan Representation (Yue Zhao, Nanyang Technological University et al.) - Machine learning for database systems has achieved significant success in various database components, such as cost estimation, query optimization, index selection, view recommendation, and semantic equivalence detection. However, these solutions typically focus on a single task and normally need a large amount of labeled data for the task to train machine learning models. Even if a solution can be adapted for a different task, it will require recollecting labeled data for each new task, which is typically much more time-consuming than model training. While dataset collection is relatively easier for some tasks, it can be prohibitively expensive for others. A natural solution is to use transfer learning techniques to adapt learned knowledge from one task to another. However, we show that naive transfer learning methods perform poorly and are only as good as training from scratch. Their failures are mainly due to three challenges: (1) the source model is not robust as it is optimized to its task only; (2) the size of the target dataset is small; and (3) the inevitable distribution shift when changing tasks. To overcome these challenges, we first study the task transfer problem in query plan representation and propose a new framework TATA for the problem. Specifically, to address the lack of robustness in the source model, TATA incorporates a self-supervised component during the pretraining stage. Specifically, we design a query plan decoder to reconstruct the original query plan from its representation, ensuring the model preserves key features. This leads to more robust and transferable query plan representations. Next, to address the issues of small datasets and distribution shift, TATA generates an arbitrary number of query plans for the target task and assigns them realistic pseudo labels. This is achieved by utilizing both strong database domain knowledge and available datasets. Through extensive experiments, we show that TATA delivers substantial improvements on task transfer, achieving up to 5× reduction in dataset collection cost when transferring from cost estimation to two representative target tasks: query optimization and index selection. We demonstrate compatibility with three distinct query plan representation models, establishing broader applicability than prior transfer approaches. • Graph Transformers for Query Plan Representation: Potentials and Challenges (Chenghao Lyu, University of Massachusetts Amherst et al.) - Query Plan Representation (QPR) is central to workload modeling, with various deep-learning based architectures proposed in the literature. Our work is motivated by two key observations: (i) the research community still lacks clarity on which model, if any, best suits the QPR problem; and (ii) while transformers have revolutionized many fields, their potential for QPR remains largely underexplored. This study examines the strengths and challenges of Graph Transformers for QPR. We introduce a new taxonomy that unifies deep-learning based QPR techniques along key design axes. Our benchmark analysis of common QPR architectures reveals that Graph Transformer Networks (GTNs) consistently outperform alternatives, but can degrade under limited training data. To address this, we propose novel data augmentation techniques to enhance training diversity and refine GTN architectures by replacing ineffective language-model-inspired components with techniques better suited for query plans. Evaluation on JOB, TPC-H, and TPC-DS benchmarks shows that with sufficient training data, enhanced GTNs outperform existing models for capturing complex queries (JOB Full and TPC-DS) and enable the query embedder trained on TPC-DS to generalize to TPC-H queries out of the box. • AQD: Online Adaptive Query Dispatcher for HTAP Databases (Yang Wu, Tsinghua University et al.) - Hybrid Transactional-Analytical Processing (HTAP) has attracted growing attention from both academia and industry. Most HTAP systems adopt a dual-engine architecture, maintaining separate row and column engines to achieve workload isolation: row engines excel at transactional workloads, while column engines are optimized for analytical queries. For such systems, dispatching queries to the appropriate engine with ultra-low latency is highly desirable but remains challenging. Existing approaches often rely on traditional cost estimation, which is often inaccurate and fails to adapt to dynamic workload patterns. Moreover, they generally overlook resource balancing when dispatching workloads. In this paper, we present AQD, an online Adaptive Query Dispatcher framework. AQD operates in two phases: (1) in the offline phase, it trains a LightGBM classifier using self-paced, Taylor-weighted boosting that emphasizes costly mispredictions; (2) in the online phase, it employs a LinTS-Delta bandit to adapt to workload drift via execution feedback, while a Mahalanobis-based regulator ensures balanced CPU and memory utilization across the two engines. We integrate AQD into PolarDB and evaluate it on standard benchmarks as well as real-world datasets. Experimental results show that AQD reduces average query latency by over 90% compared to cost-threshold dispatching and improves HyBench score by 15% over the cost-threshold method and 9% over the current SOTA BRAD. • QDBO: A Real-time Quantum-augmented Database System Optimizer (Hanwen Liu, University of Southern California et al.) - Quantum annealers offer a promising sampling-based computing paradigm for large-scale combinatorial optimization. However, directly using them to solve database optimization problems in real-time scenarios remains challenging due to strict hardware constraints and the opacity of commercial hybrid solvers. To address these challenges, we present QDBO, a white-box quantum-augmented database optimizer that directly leverages low-level quantum sampling on a quantum annealer. It first performs offline reduction to produce embeddable QUBO instances via multiple mapping strategies that trade off efficiency and quality. It then applies an online iterative relaxation correction procedure that uses sampling feedback to correct the reduced QUBO across iterations, enabling controllable quality improvements under varying time budgets. We integrate QDBO with the PostgreSQL query optimizer to construct Q²O-QDBO for join order optimization. Evaluations on the Join Order Benchmark (JOB) and the Cardinality Estimation Benchmark (CEB) show that Q²O-QDBO improves 62 out of 113 JOB queries, and achieves up to 49.36× execution-time speedup and 34.65× end-to-end speedup on the CEB workload. Evaluations on synthetic queries with up to 50 relations show that Q²O-QDBO achieves significantly lower end-to-end latency while maintaining similar solution quality compared with black-box quantum solvers. To demonstrate generality, we further integrate QDBO with PostgreSQL for index selection on TPC-H, where QDBO achieves up to 2.64× workload speedup over the empty-index baseline and outperforms classical and black-box solver baselines. These results demonstrate that using QDBO to solve large-scale database optimization problems in real-time settings is promising.read more
☆
Wed Sep 2 · 10:45 AM - 12:15 PM
Research 21 · Data Systems on Modern Hardware
● Research
📍 Grand Ballroom C
Tux: Efficient Drop-in Networking for Database Systems (Xinjing Zhou, MIT et al.) - Modern high-performance OLTP systems are increasingly bottlenecked by kernel network stack and context switches arising from dispatching tasks between the network stack and DBMS worker threads.Tux: Efficient Drop-in Networking for Database Systems (Xinjing Zhou, MIT et al.) - Modern high-performance OLTP systems are increasingly bottlenecked by kernel network stack and context switches arising from dispatching tasks between the network stack and DBMS worker threads. While kernel-bypass TCP stacks improve performance, they often sacrifice compatibility and robustness, in addition to leaving performance opportunities on the table due to TCP's byte-stream interface. We present \textbf{Tux}, a kernel-bypass networking stack designed for database systems that achieves high performance without giving up compatibility and robustness. \tux addresses these challenges by (1) introducing a message‑based transport protocol that decouples reliability from in‑order delivery and natively preserves message boundaries without framing or copy overhead inherent in byte-stream interface; (2) providing a flexible pushdown abstraction that lets database engines execute DBMS-specific logic directly on the network cores to avoid context-switch overhead and to exploit message-based interface; and (3) leveraging eBPF/XDP to reuse well‑maintained kernel NIC drivers for compatibility and operational ease. We implement \tux in a library called \libtux, offering both zero‑change “compatibility” mode and minimal‑change “pushdown” mode. Our implementation, \libtux, evaluated on VoltDB, Redis, ScyllaDB, Memcached, and LeanStore, improves throughput up to 2.3x, reduces median and 99th percentile latencies by up to 2.6x and 4.7x, compared to existing kernel-bypass systems with minimal modifications. • dpKernels: Harvesting DPU Compute Resources for Data-path Efficiency in Cloud Data Processing (Jason Hu, University of Toronto et al.) - Data processing units, or DPUs, are equipped with hardware accelerators for compute-intensive data path tasks. Although DPUs’ SoC cores are wimpier than the host’s, hardware accelerators are typically orders of magnitude faster than CPUs. Harvesting DPU hardware accelerators for database systems could significantly increase throughput and save host CPU cycles. However, due to the heterogeneity of DPUs’ hardware configurations and performance characteristics, it is challenging to offer a unified and portable solution for cloud data processing systems to harvest the compute resources on DPUs across generations and vendors. Additionally, due to DPU resource constraints, offloaded compute tasks need to be carefully optimized and scheduled to achieve high efficiency and avoid performance regression. To address these challenges, we introduce two levels of abstraction: dpKernels, which are unified, efficient, and portable primitives that abstract DPU compute resources (i.e., hardware accelerators and SoC cores) for cloud data systems, and dpManager, an onboard management framework that abstracts specific DPU platforms for dpKernels to deliver their promises with optimized, scheduled, and cross-platform executions. The benefits of our proposal have been validated by the high performance of DPU-accelerated data processing for various workloads, systems, and DPU hardware. • MGI: A Communication Framework for Data Processing in Massive GPU Infrastructures (Di Wu, University of Toronto et al.) - This paper presents MGI, a general communication framework for performing data processing tasks in massive GPU infrastructures. Inter-GPU data transfer performance is crucial to multi-GPU data processing, and existing solutions repeatedly implement the same set of communication optimizations. MGI identifies these techniques and applies them judiciously behind a simple interface. Enabling MGI are (1) a central controller that models relevant hardware resources as an annotated graph and automates infrastructure- level optimizations to construct transfer plans and (2) a scalable data plane where buffers and executors are carefully designed to incorporate device- and link-level optimizations to execute data transfers efficiently. Our experiments on a variety of GPU infrastructures and workloads show that MGI significantly improves multi-GPU data processing performance compared to existing frameworks. • RayDB: Building Databases with Ray Tracing Cores (Xuri Shi, Fudan University et al.) - Ray tracing (RT) cores are a new type of hardware that have been actively integrated into modern GPUs. Recent studies have demonstrated that RT cores can deliver much higher performance than GPU CUDA cores and CPUs for general-purpose data processing tasks like table scan. In this paper, we propose RayDB, the first query engine that leverages RT cores to accelerate database query processing. RayDB transforms the core part of the query execution that involves multiple operators into one single ray tracing job. With a set of proposed encoding and ray launching techniques, RayDB effectively exploits RT cores to accelerate diverse workloads. Experimental results show that RayDB outperforms the state-of-the-art GPU-based query engine by up to 18.3×. • High-Performance DBMSs with io_uring: When and How to Use It (Matthias Jasny, TU Darmstadt et al.) - We study how modern database systems can leverage the Linux io_uring interface for efficient, low-overhead I/O. io_uring is an asynchronous system call batching interface that unifies storage and network operations, addressing limitations of existing Linux I/O interfaces. However, naively replacing traditional I/O interfaces with io_uring does not necessarily yield performance benefits. To demonstrate when io_uring delivers the greatest benefits and how to use it effectively in modern database systems, we evaluate it in two use cases: Integrating io_uring into a storage-bound buffer manager and using it for high-throughput data shuffling in network-bound analytical workloads. We further analyze how advanced io_uring features, such as registered buffers and passthrough I/O, affect end-to-end performance. Our study shows when low-level optimizations translate into tangible system-wide gains and how architectural choices influence these benefits. Building on these insights, we derive practical guidelines for designing I/O-intensive systems using io_uring and validate their effectiveness in a case study of PostgreSQL’s recent io_uring integration, where applying our guidelines yields a performance improvement of 14%. • Analyzing Near-Network Hardware Acceleration with Co-Processing on DPUs (Dimitrios Giouroukis, BIFOLD, TU Berlin et al.) - Data Processing Units (DPUs) are PCIe network cards (SmartNICs) equipped with specialized hardware accelerators for data processing. DPUs offer the opportunity to process data near the hardware network stack (near-network). By enabling near-network computation, DPUs reduce CPU load and improve end-to-end performance, an increasingly attractive approach to trends like compute-storage disaggregation and real-time data ingestion. However, existing research on DPU-based processing often overlooks hardware acceleration or relies on static offloading to the ARM subsystem, leaving open questions about how best to split work (or co-process) with the host CPU. In this paper, we analyze near-network hardware acceleration with co-processing on DPUs, revealing that DPU performance varies significantly depending on input data types, task and query-imposed configurations. Through our micro-benchmark experiments, we explore partial offloads and co-processing strategies that demonstrate the trade-offs between higher throughput against reconfiguration overhead on DPUs. Our findings offer practical insights for data systems practitioners seeking to leverage near-network accelerators in data processing pipelines.read more
☆
Wed Sep 2 · 10:45 AM - 12:15 PM
Research 22 · Property Graph Management
● Research
📍 Commonwealth Ballroom C
Repairing Property Graphs under PG-Constraints (Christopher Spinrath, Lyon 1 University, Liris CNRS et al.) - Recent standardization efforts for graph databases lead to standard query languages like GQL and SQL/PGQ, and constraint languages like Property Graph Constraints (PG-Constraints).Repairing Property Graphs under PG-Constraints (Christopher Spinrath, Lyon 1 University, Liris CNRS et al.) - Recent standardization efforts for graph databases lead to standard query languages like GQL and SQL/PGQ, and constraint languages like Property Graph Constraints (PG-Constraints). In this paper, we embark on the study of repairing property graphs under PG-Constraints. We identify a significant subset of PG-Constraints, encoding denial constraints and including recursion as a key feature, while still permitting automata-based structural analyses of errors. We present a comprehensive repair pipeline for these constraints to repair Property Graphs, involving changes in the graph topology and leading to node, edge and, optionally, label deletions. We investigate three algorithmic strategies for the repair procedure, based on Integer Linear Programming (ILP), a naive, and an LP-guided greedy algorithm. Our experiments on various real-world datasets reveal that repairing with label deletions can achieve a 59% reduction in deletions compared to node/edge deletions. Moreover, the LP-guided greedy algorithm offers a runtime advantage of up to 97% compared to the ILP strategy, while matching the same quality. • A Unified Query Planning Framework for Conjunctive Regular Path Queries (Yue Pang, Peking University et al.) - Conjunctive regular path queries (CRPQs) form a critical backbone of modern graph query languages, integrating subgraph matching with regular path queries (RPQs). Despite their ubiquity in applications like social networks, finance, and scientific data analysis, CRPQ optimization lacks a unified framework, relying instead on heuristic combinations of disjoint techniques for subgraph matching and RPQs. This paper bridges this gap by introducing a novel algebraic optimization framework for CRPQs. We propose a hypergraph query model that enables composability, addressing a longstanding challenge in graph query languages including GQL and SQL/PGQ, and define six core operators (TI, KC, SJ, UNION, INV, and SCAN) to abstract CRPQ semantics. Leveraging algebraic transformation rules, we enumerate a rich space of equivalent query plans and devise a cost-based optimizer to select near-optimal plans for execution. Implemented based on MillenniumDB and Neo4j, our framework achieves significant speedups on CRPQs extracted from the WDBench and LDBC SNB benchmarks. • Structural Normalization of Property Graphs (Maximilian Egger, Aarhus University et al.) - A property graph stores information in the form of a network of nodes and edges, annotated with labels and properties (i.e. attributes). Even though various forms of functional dependencies, uniqueness constraints, and keys have been proposed for graph data, no study has been hitherto undertaken that builds upon and utilizes these concepts to assemble a comprehensive proposal for graph normalization. An elementary proposal merely maps the nodes in the graph schema to a relational schema and deploys relational normalization techniques. Unfortunately, this proposal disregards the graph structure and thereby forfeits the opportunity for structural normalization. In this paper, we define five structural graph normal forms that extend relational normal forms building on concepts of keys and graph functional dependencies. Based on acyclic graph patterns, these normal forms apply to any data graph, address previously overlooked sources of inconsistency, and support structure-aware integrity preservation beyond node attributes. We showcase the applicability and effectiveness of these normal forms through experimentation. • Sankofa: Online Query-adaptive Dynamic Graph Summaries (Ama Bembua Bainson, Aarhus University et al.) - Knowledge Graphs (KGs) model semantic relationships between entities and are widely used in tasks such as question answering, recommendation, and large language model training. As KGs grow to billions of triples, storing and querying them on local devices becomes infeasible and also raises privacy concerns. KG summarization methods produce compact representations, yet existing methods are either static or not attuned to user interests. In this paper, we propose Sankofa, a query-aware, incremental, and online KG summarization method. Sankofa maintains personalized summaries on-device by incrementally updating them in response to new queries and data using a random-walk-based adaptation of node-centric Personalized PageRank, efficiently identifying the most relevant nodes while bounding summary size. We provide formal guarantees for our PPR approximation and demonstrate experimentally that Sankofa outperforms existing personalized KG summarization techniques in both speed and accuracy, enabling adaptive, privacy-preserving KG summaries that evolve with user interests and dynamic knowledge graphs. • Chipmink: Efficient Delta Identification for Massive Object Graphs (Supawit Chockchowwat, University of Illinois Urbana-Champaign et al.) - Ranging from batch scripts to computational notebooks, modern data science tools rely on massive and evolving object graphs that represent structured data, models, plots, and more. Persisting these objects is critical, not only to enhance system robustness against unexpected failures but also to support continuous, non-linear data exploration via versioning. Existing object persistence mechanisms (e.g., Pickle, Dill) rely on complete snapshotting, often redundantly storing unchanged objects during execution and exploration, resulting in significant inefficiency in both time and storage. Unlike DBMSs, data science systems lack centralized buffer managers that track dirty objects. Worse, object states span various locations such as memory heaps, shared memory, GPUs, and remote machines, making dirty object identification fundamentally more challenging. In this work, we propose a graph-based object store, named Chipmink, that acts like the centralized buffer manager. Unlike static pages in DBMSs, persistence units in Chipmink are dynamically induced by partitioning objects into appropriate subgroups (called pods), minimizing expected persistence costs based on object sizes and reference structure. These pods effectively isolate dirty objects, enabling efficient partial persistence. Our experiments show that Chipmink is general, supporting libraries that rely on shared memory, GPUs, and remote objects. Moreover, Chipmink achieves up to 36.5× smaller storage sizes and 12.4× faster persistence than the best baselines in real-world notebooks and scripts. • TurboLynx: Schemaless Graph Engine Strikes Back for General-Purpose Analytics (Taesung Lee, POSTECH et al.) - Graph database management systems (GDBMSes) are widely adopted for their efficient handling of graph traversal queries that capture complex relationships. Recently, a class of modern GDBMSes appeared that were designed to offer explicit support for schemaless property graph models (PGMs), providing users with a high degree of flexibility. However, GDBMSes in this class often suffer from performance bottlenecks in analytical database queries - typically involving operations such as group-by and aggregation. We argue that a major cause is that schemaless processing is not treated as a primary design requirement across the storage, query-processing, and optimization layers. To address this, we propose TurboLynx, a novel graph analytics engine that holistically integrates the schemaless property at every layer of the system - from storage to query processing and optimization. TurboLynx organizes graph data into cost-based clusters, called graphlets, and stores them in a columnar format. By adopting a graphlet-aware query optimizer and processor, TurboLynx efficiently handles both graph traversal and analytical workloads in a single system. Our comprehensive evaluation on LDBC SNB Interactive, TPC-H, and DBpedia demonstrates that TurboLynx outperforms state-of-the-art GDBMSes by up to 183.9x and leading RDBMSs by up to 41.27x.read more
☆
Wed Sep 2 · 10:45 AM - 12:15 PM
Research 23 · RAG and Text-to-SQL
● Research
📍 Commonwealth Ballroom B
Document-to-Database: Extraction Meets Relational Semantics (Zhengxuan Zhang, The Hong Kong University of Science and Technology(Guangzhou) et al.) - A vast amount of enterprise and scientific knowledge remains locked in unstructured documents, while analytics and decision-making rely on relational databases.Document-to-Database: Extraction Meets Relational Semantics (Zhengxuan Zhang, The Hong Kong University of Science and Technology(Guangzhou) et al.) - A vast amount of enterprise and scientific knowledge remains locked in unstructured documents, while analytics and decision-making rely on relational databases. Bridging this gap is challenging because document extraction operates on local text spans, whereas relational databases enforce global semantics through schemas, keys, and integrity constraints. As a result, simply prompting large language models (LLMs) to extract tables often yields outputs that are hard to audit and inconsistent at the database level. This tension motivates document-to-database (Doc2DB) construction, yet most existing approaches treat it as a one-shot extraction problem, focusing on individual tables while failing to reconcile extraction results with relational semantics. We present DataMosaic, a Doc2DB system that explicitly mediates between extraction and database semantics. Given an ER schema with integrity and business constraints, DataMosaic coordinates entity and relationship extraction with verification, repair, and targeted re-extraction in a closed extract-verify-iterate loop managed by an orchestrator. By separating intrinsic document ambiguity, correctable extraction errors, and structural constraint violations, DataMosaic incrementally constructs database instances that are accurate and semantically consistent. The system supports pluggable third-party LLM extractors, domain-specific verifiers, and repair operators. Experiments across financial, legal, academic, and student datasets show that DataMosaic substantially reduces constraint violations and improves database-level accuracy over strong Doc2DB baselines. • QA-GraphRAG: Query-Adaptive Plug-and-Play Retrieval Integration for Graph-based Retrieval-Augmented Generation (Zeang Sheng, Peking University et al.) - Large Language Models (LLMs) have demonstrated remarkable capabilities, yet they often suffer from hallucinations and lack up-to-date knowledge. Retrieval-Augmented Generation (RAG) addresses these limitations by grounding LLMs in external knowledge. While vector-based RAG is effective for simple queries, it struggles with complex queries that require multi-hop reasoning. Graph-based RAG frameworks have emerged to solve this by constructing knowledge graphs that capture global relationships and enable multi-hop reasoning. However, these graph-based approaches frequently underperform on simple fact-based queries compared to their vector-based counterparts, as they may lose detailed entity information. In this paper, we conduct dataset-level and framework-level analysis targeting graph-based RAG approaches. We find that existing QA benchmark datasets can be split to "Local" and "Global" queries that have different properties; and different RAG frameworks perform differently on these two kinds of queries. Concretely, existing graph-based RAG frameworks, including recent dual-branch ones, cannot consistently outperform vector-based RAG on "Local" queries. We attribute this phenomenon to the fact that graph-based RAG often employs a fixed retrieval strategy, leading to redundant information retrieval and unnecessary cost for simple queries. Based on the analysis, we propose QA-GraphRAG, a new query-adaptive plug-and-play retrieval integration for graph-based RAG frameworks. QA-GraphRAG incorporates a pre-trained router that predicts the optimal knowledge hierarchy from which to start retrieval based on the characteristics of the input query. Extensive experiments on conventional KGQA datasets and GraphRAG-Bench demonstrate that equipping existing graph-based RAG frameworks with our QA-GraphRAG leads to substantial performance improvements. • TACO: A Benchmark for Open-Domain Text-to-SQL with Ambiguous and Cross-Database Queries (Chao Deng, Renmin University of China et al.) - Text-to-SQL aims to translate natural language questions into executable SQL queries over structured databases. Existing benchmarks mainly focus on closed-domain settings with predefined database schemas and well-specified questions, but they fall short in addressing the challenges of open-domain scenarios, such as ambiguous questions, unspecified databases, and cross-database querying. To bridge this gap, we introduce TACO, a benchmark for open-domain Text-to-SQL with Ambiguous and CrOss-database queries. TACO consists of 1,500 real-world Text-to-SQL examples from a smart city data service and 13,000 high-quality synthetic examples generated from large-scale open data portals, covering diverse domains such as transportation, healthcare, and finance. To construct the synthetic examples, we develop an effective data synthesis pipeline that preserves the complexity of real-world queries. To demonstrate the utility of TACO, we introduce a baseline TACO-SQL composed of question rewriting, table linking, and query planning, to illustrate the challenges posed by TACO and to better understand the limitations of existing Text-to-SQL approaches. Extensive experiments on TACO using a variety of state-of-the-art Text-to-SQL approaches show that, while our proposed TACO-SQL achieves the best results, a significant gap remains between the existing approaches and human-written SQL. These findings highlight the difficulty of open-domain Text-to-SQL and position TACO as a valuable benchmark to drive future research. • In-depth Analysis of Graph-based RAG in a Unified Framework (Yingli Zhou, The Chinese University of Hong Kong, Shenzhen et al.) - Graph-based Retrieval-Augmented Generation (RAG) has proven effective in integrating external knowledge into large language models (LLMs), improving their factual accuracy, adaptability, interpretability, and trustworthiness. A number of graph-based RAG methods have been proposed in the literature. However, these methods have not been systematically and comprehensively compared under the same experimental settings. In this paper, we first summarize a unified framework to incorporate all graph-based RAG methods from a high-level perspective. We then extensively compare representative graph-based RAG methods over a range of questing-answering (QA) datasets - from specific questions to abstract questions - and examine the effectiveness of all methods, providing a thorough analysis of graph-based RAG approaches. As a byproduct of our experimental analysis, we are also able to identify new variants of the graph-based RAG methods over specific QA and abstract QA tasks respectively, by combining existing techniques, which outperform the state-of-the-art methods. Finally, based on these findings, we offer promising research opportunities. We believe that a deeper understanding of the behavior of existing methods can provide new valuable insights for future research. • BookRAG: A Hierarchical Structure-aware Index-based Approach for Retrieval-Augmented Generation on Complex Documents (Shu Wang, School of Data Science, The Chinese University of Hong Kong, Shenzhen et al.) - As an effective method to boost the performance of Large Language Models (LLMs) on the question answering (QA) task, Retrieval-Augmented Generation (RAG), which queries highly relevant information from external complex documents, has attracted tremendous attention from both industry and academia. Existing RAG approaches often focus on general documents, and they overlook the fact that many real-world documents (such as books, booklets, handbooks, etc.) have a hierarchical structure, which organizes their content from different granularity levels, leading to poor performance for the QA task. To address these limitations, we introduce BookRAG, a novel RAG approach targeted for documents with a hierarchical structure, which exploits logical hierarchies and traces entity relations to query the highly relevant information. Specifically, we build a novel index structure, called BookIndex, by extracting a hierarchical tree from the document, which serves as the role of its table of contents, using a graph to capture the intricate relationships between entities, and mapping entities to tree nodes. Leveraging the BookIndex, we then propose an agent-based query method inspired by the Information Foraging Theory, which dynamically classifies queries and employs a tailored retrieval workflow. Extensive experiments on three widely adopted benchmarks demonstrate that BookRAG achieves state-of-the-art performance, significantly outperforming baselines in both retrieval recall and QA accuracy while maintaining competitive efficiency. • MGRAG: Semantic Subgraph Matching and Graph-Aware Caching for Multimodal Retrieval-Augmented Generation (Yubo Wang, HKUST et al.) - Answering complex queries over large and heterogeneous multimodal document corpora is a central challenge in data management, requiring fine-grained, entity-level evidence retrieval and efficient context serving. Graph-based Retrieval-Augmented Generation (RAG) systems achieve promising effectiveness by organizing multimodal documents as knowledge graphs (KGs); however, they still face three limitations: (1) query-agnostic KG construction, where corpus-wide graphs overwhelm query-relevant entities with irrelevant noise; (2) inflexible graph matching, which relies on rigid topological matching and misses path-level semantic equivalences; (3) event-agnostic KV re-computation, which scores tokens independently of graph topology, failing to preserve event-level semantic structure. To address these issues, we propose MGRAG. First, MGRAG incrementally builds query-specific KGs on demand via a lazy, top-down construction strategy. Second, we formulate graph retrieval as a path-based semantic subgraph matching problem, prove it NP-hard, and design an efficient greedy algorithm for flexible, semantics-aware retrieval. Third, MGRAG employs an event-aware KV caching mechanism to selectively recompute tokens critical to query-related events. Experiments on seven real-world multimodal QA datasets show that MGRAG achieves superior effectiveness and efficiency compared to state-of-the-art RAG, subgraph matching, and KV caching baselines. • OpenSQL: Data-Efficient Text-to-SQL for Open-Source LLMs via Synthesized Intermediate Supervision (Ruilin Hu, Tsinghua University et al.) - The Text-to-SQL task enables non-expert users to query structured data through natural language. While recent methods based on closed-source large language models (LLMs) achieve strong performance, their high inference cost, data privacy concerns, and limited transparency hinder real-world deployment. Open-source LLMs are a promising alternative; however, training them for Text-to-SQL remains challenging due to scarce task-specific annotations and the difficulty of learning reliable grounding and reasoning solely from sparse end-to-end supervision. To address these challenges, we present OpenSQL, a data-efficient framework that improves Text-to-SQL performance of open-source LLMs via synthesized intermediate supervision. OpenSQL converts limited (Question, SQL) pairs into rich, task-decomposed training signals that guide the model to learn critical intermediate decisions. Concretely, (1) we train a global - local schema linking module with schema-aware learning to identify and refine relevant tables and columns; (2) we introduce reasoning-enhanced SQL generation, which produces diverse candidates along complementary reasoning paths and selects the best one through stepwise clause-level and semantic-level reasoning; and (3) we design a task-aware data augmentation pipeline that provides the intermediate supervision signals to support the entire training process. With the same 32B LLM backbone, OpenSQL achieves 70.0% accuracy on BIRD-dev using only 14K training samples, outperforming the advanced open-source Text-to-SQL model, OmniSQL, which uses 2.5M training samples. • PrepBench: How Far Are We from Natural-Language-Driven Data Preparation? (Jingzhe Xu, Tsinghua University et al.) - Data preparation is a central and time-consuming stage in data analysis workflows. Traditionally, commercial tools have relied on graphical user interfaces (GUIs) to simplify data preparation, allowing users to define transformations through visual operators and workflows. Recent advances in large language models (LLMs) raise the possibility of a paradigm shift toward natural language (NL)-driven data preparation, in which users can specify preparation intents in NL directly. However, it remains unclear how far current LLM-based agents are from this paradigm shift in practice. Existing code generation benchmarks do not capture key characteristics of data preparation, including ambiguous user intents, imperfect real-world data, and the need to translate code into interpretable workflows for validation. To bridge this gap, we present PrepBench, a benchmark designed to evaluate NL-driven data preparation along three core capabilities: interactive disambiguation, prep-code generation, and code-to-workflow translation. We crawl data from the Preppin' Data Challenges, and then extend it into a systematically designed benchmark. The benchmark covers diverse domains, and each task involves 3 to 18 data preparation steps. Nearly half of the tasks require over 100 lines of Python code, and the longest solutions approach 300 lines. Our evaluation shows that, despite recent progress, realizing this paradigm shift remains challenging for state-of-the-art LLMs. PrepBench provides a principled benchmark for measuring this gap and helps identify key challenges toward realizing NL-driven data preparation.read more
☆
Wed Sep 2 · 10:45 AM - 12:15 PM
Research 24 · Vector Indexing and Quantization
● Research
📍 Commonwealth Ballroom A
Quantization Meets Projection: A Happy Marriage for Approximate k-Nearest Neighbor Search (Mingyu Yang, The Hong Kong University of Science and Technology (Guangzhou) et al.) - Approximate k-nearest neighbor (AKNN) search is a fundamental problem with wide applications.Quantization Meets Projection: A Happy Marriage for Approximate k-Nearest Neighbor Search (Mingyu Yang, The Hong Kong University of Science and Technology (Guangzhou) et al.) - Approximate k-nearest neighbor (AKNN) search is a fundamental problem with wide applications. To reduce memory and accelerate search, vector quantization is widely adopted. However, existing quantization methods either rely on codebooks---whose sizes are flexible but whose query speed is limited by costly table lookups---or adopt dimension-wise quantization, which maps each vector dimension to a small quantized code for fast search. The latter, however, suffers from a fixed compression ratio because the quantized code length is inherently tied to the original dimensionality. To overcome these limitations, we propose MRQ, a new approach that integrates projection with quantization. The key insight is that, after projection, high-dimensional vectors tend to concentrate most of their information in the leading dimensions. MRQ exploits this property by quantizing only the information-dense projected subspace---whose size is fully user-tunable---thereby decoupling the quantized code length from the original dimensionality. The remaining tail dimensions are captured using lightweight statistical summaries. By doing so, MRQ preserves the high query throughput of dimension-wise quantization while achieving arbitrary compression ratios enabled by the projection step. Extensive experiments show that MRQ substantially outperforms the state-of-the-art method, achieving up to 3× faster search with only one-third the quantization bits for comparable accuracy • RNSG: A Range-Aware Graph Index for Efficient Range-Filtered Approximate Nearest Neighbor Search (Zhiqiu Zou, Beijing Institute of Technology et al.) - Range-filtered approximate nearest neighbor (RFANN) search is a fundamental operation in modern data systems. Given a set of objects, each with a vector and a numerical attribute, an RFANN query retrieves the nearest neighbors to a query vector among those objects whose numerical attributes fall within the range specified by the query. Existing state-of-the-art methods for RFANN search often require constructing multiple range-specific graph indexes to achieve high query performance, which incurs significant indexing overhead. To address this, we first establish a novel graph indexing theory, the range-aware relative neighborhood graph (RRNG), which jointly considers spatial and attribute proximity. We prove that the RRNG satisfies two crucial properties: (1) monotonic searchability, which ensures correct nearest neighbor retrieval via beam search; and (2) structural heredity, which guarantees that any range-induced subgraph remains a valid RRNG, thus enabling efficient search with a single graph index. Based on this theoretical foundation, we propose a new graph index called RNSG as a practical solution that efficiently approximates RRNG. We develop fast algorithms for both constructing the RNSG index and processing RFANN queries with it. Extensive experiments on five real-world datasets show that RNSG achieves significantly higher query performance with a more compact index and lower construction cost than existing state-of-the-art methods. • HEXA: A Disjoint-Subgraph-Based Indexing Framework for Approximate Nearest Neighbor Search at Billion Scale (Yifei Xu, Shanghai Jiao Tong University et al.) - Approximate Nearest Neighbor (ANN) search is a core operation in large-scale vector databases that support recommendation, semantic search, and retrieval-augmented generation. Graph-based indexes are widely used for their superior search performance at million scale, but scaling them to billion-scale datasets exposes a fundamental trade-off between construction efficiency and search performance. CPU-based methods produce high-quality graphs enabling fast search but require multiple days to construct, while GPU-based methods accelerate construction at the cost of degraded search performance. This trade-off stems from the traditional requirement of enforcing global connectivity during graph construction. We observe that graph-based search primarily depends on local connectivity between entry points and a query's nearest neighbors, making global connectivity unnecessary for high search performance. In this paper, we introduce HEXA, a disjoint-subgraph-based indexing framework that partitions the dataset into disjoint clusters and constructs refined proximity subgraphs independently on GPUs. At query time, HEXA employs a two-level routing structure to select relevant subgraphs and query-close entry points, then applies a budget-adaptive search algorithm that allocates exploration effort based on each subgraph's likelihood of containing nearest neighbors. Experiments on billion-scale datasets demonstrate that HEXA achieves up to 14.7 times higher query throughput at the same recall levels compared with various indexing methods, and reduces construction time to approximately one hour. • GAS: A Lightweight Framework for Filtered Search over Wide-table Vectors (Ziyuan He, Beihang University et al.) - Wide-table vectors, where each embedding is linked with numerous structured attributes, are prevalent in applications such as autonomous driving and multimodal data processing for large-model training. Efficiently retrieving semantically similar vectors under attribute filters is crucial for these tasks, a problem addressed by Filtered Approximate Nearest Neighbor Search (FANNS). Recent approaches follow two paradigms: (1) building per-attribute dedicated indexes that integrate attribute information, which incurs prohibitive build time and storage in wide-table settings; or (2) building an attribute-agnostic general index and applying predicates at query time, which often degrades search efficiency. Consequently, neither paradigm adequately supports wide-table scenarios. We aim to achieve good query performance with low upfront cost by incorporating information from many attributes into a single graph index, avoiding prohibitive overhead. Our key observation is that graph-traversal information from past queries can be reused to optimize future queries with the same filter attribute. Based on this insight, we devise Graph with Adaptive Shortcuts (GAS), a framework that leverages historical query logs to build lightweight auxiliary structures, enhancing search efficiency over a single base graph with minimal overhead. Extensive experiments on real-world datasets show that GAS consistently outperforms existing general indexes in wide-table scenarios, achieving up to 42.1× speedup on datasets with thousands of structured attributes. • Revisiting Task-Oriented Dataset Search in the Era of Large Language Models: Challenges, Benchmark, and Solution (Zixin Wei, The Chinese University of Hong Kong, Shenzhen et al.) - The search for suitable datasets is the critical "first step" in data-driven research, but it remains a great challenge. Researchers often need to search for datasets based on high-level task descriptions. However, existing search systems struggle with this task due to ambiguous user intent, task-to-dataset mapping and benchmark gaps, and entity ambiguity. To address these challenges, we introduce KATS, a novel end-to-end system for task-oriented dataset search from unstructured scientific literature. KATS consists of two key components, i.e., offline knowledge base construction and online query processing. The sophisticated offline pipeline automatically constructs a high-quality task-dataset knowledge graph by employing a collaborative multi-agent framework for information extraction, thereby filling the task-to-dataset mapping gap. To further address the challenge of entity ambiguity, a unique semantic-based mechanism is used for task entity linking and dataset entity resolution. For online retrieval, KATS utilizes a specialized hybrid query engine that combines vector search with graph-based ranking to generate highly relevant results. Additionally, we introduce CS-TDS, a tailored benchmark suite for evaluating task-oriented dataset search systems, addressing the critical gap in standardized evaluation. Experiments on our benchmark suite show that KATS significantly outperforms state-of-the-art retrieval-augmented generation frameworks in both effectiveness and efficiency, providing a robust blueprint for the next generation of dataset discovery systems.read more
☆
Wed Sep 2 · 10:45 AM - 12:15 PM
Tutorial 5 · Are We There Yet? A Grand Tour of ETA Queries
● Tutorials
📍 Stone
Are We There Yet? A Grand Tour of ETA Queries
Areeg MostafaUniversity of Minnesota
Mohamed HemdanUniversity of Minnesota
Mohamed MokbelUniversity of Minnesota
☆
Wed Sep 2 · 10:45 AM - 12:15 PM
Tutorial 6 · Bridging LLMs and Database Systems: A Deep Dive into Enhanced Relational Operators
● Tutorials
📍 Webster
Bridging LLMs and Database Systems: A Deep Dive into Enhanced Relational Operators
Tianjing ZengAlibaba Group
Yin LinAlibaba Group
Rong ZhuAlibaba Group
Yunxiang SuAlibaba Group
Zhongjun DingAlibaba Group
Bolin DingAlibaba Group
Jingren ZhouAlibaba Group
☆
Wed Sep 2 · 10:45 AM - 12:15 PM
Industry Session 4 · Cloud-Native OLTP & Distributed Transactions
● Industry
📍 Marina Ballroom I
Highly Scalable SQL Azure MVCC Version Cleaner (Jithendra Gandikota, Microsoft et al.) - SQL Azure utilizes Multi-Version Concurrency Control (MVCC) to support several features such as Read Committed Snapshot Isolation (RCSI), Snapshot Isolation (SI), Constant Time Recovery, and Metadata Versioning.Highly Scalable SQL Azure MVCC Version Cleaner (Jithendra Gandikota, Microsoft et al.) - SQL Azure utilizes Multi-Version Concurrency Control (MVCC) to support several features such as Read Committed Snapshot Isolation (RCSI), Snapshot Isolation (SI), Constant Time Recovery, and Metadata Versioning. MVCC maintains multiple versions of data rows to support these features. A background version cleaner lazily reclaims older versions that are not required by any of these features. The prior version cleaner algorithms suffered from several drawbacks which prevented it from cleaning older versions quickly leading to space bloat. Under conflicting workloads, the cleaner failed to acquire locks, delaying cleanup. Cleanup also slowed down significantly while handling very large databases and multi-database instances. When a consumer of some versions lagged, it held up many versions in the system from being cleaned up. This paper outlines the innovative solutions that SQL Azure implements to address these drawbacks, such as transaction-level version tracking, statement level cleanup, dynamic multi-threaded cleaner, and user-assisted cleanup. The paper also presents the impact of deploying these changes across millions of SQL databases in Azure. • The Live Database: Firestore’s Scalable and Consistent Realtime Queries (Matt Bell, Google et al.) - Google’s Firestore is a NoSQL serverless database service that scales to millions of queries per second and petabytes of storage, serving over 8 million databases and 1.5 billion monthly active end users. A key reason for its popularity is its unique realtime query feature, which greatly simplifies web and mobile application development. Realtime queries allow an application to present an up-to-date and strongly-consistent view of the state of the database to the end-user, and to receive fast notifications of changes to the database state. In this paper we present the architecture of the Firestore real-time stack, how it provides seamless scaling and high availability while supplying strongly-consistent responses to clients, including evaluations simulating workloads that represent large applications. We also explain how the Firestore SDK enables the application to deliver a smooth user experience through periods of network disconnection. • A Tree-Structured Two-Phase Commit Framework for OceanBase: Optimizing Scalability and Consistency (Quanqing Xu, OceanBase, Ant Group et al.) - Distributed transactional consistency over partitions is costly: classical two-phase commit (2PC) incurs high coordination and latency, and dynamic partition transfers complicate recovery. We introduce a tree-structured 2PC framework for OceanBase (OB) using single machine log streams in three ways. First, we use log streams as coalesced 2PC participants: co-located partitions that would each be separate in a partition-centric design share one log-stream participant, so a transaction spanning N such partitions interacts with one participant in commit, cutting coordination by orders of magnitude (e.g., 99% for N = 100). Second, a coordinator-rooted DAG recursively builds commit trees; migration contexts as leaves avoid explicit participant-list updates and circular dependencies while ensuring linearizable commits under topology change. Third, prepare unknown and trans unknown states prevent violations when participants lose context; they signal uncertainty during retries, avoiding erroneous aborts from “lying” participants while isolating users from ambiguity. Experiments approach single-machine transaction performance with lower latency and bandwidth consumption, supporting modern distributed databases. • Private Data Collections: Enabling Data Privacy in Permissioned Blockchains (Senthilnathan Natarajan, IBM Research et al.) - Blockchain technology provides a decentralized and immutable ledger, but its inherent transparency poses a significant barrier to enterprise adoption in domains requiring data privacy. Existing solutions like channels provide data isolation but are coarse-grained, requiring a separate ledger and consensus instance per channel. In this paper, we introduce private data collections, a mechanism for managing private data on permissioned blockchains. Our approach decouples private data storage from the replicated ledger: private data is shared only among authorized peers, while cryptographic hashes are recorded on the ledger for tamper-evidence and serializability verification. The key theoretical contribution is blind validation: we prove that on-chain hashes form a conflict-complete proxy for private state - OCC-based serializability checks on hashed state detect exactly the same conflicts as checks on the underlying private data, producing neither false negatives nor false positives. This enables the entire network to verify the correctness and serializability of private transactions without ever accessing the private data itself. We design a three-component dissemination protocol - proactive push during simulation, reactive pull at commit time, and background reconciliation post-commit - and we formalize the collection abstraction with seven design requirements. Our solution is designed for any blockchain platform that follows the simulate-order-validate-commit replication protocol. We implement it in Hyperledger Fabric and demonstrate that private data collections add only 3 - 14% overhead to transaction throughput while providing fine-grained data privacy. The mechanism has been in production since Hyperledger Fabric v1.2 and is deployed by organizations including Walmart, MediLedger, Honeywell, Oracle and GSBN. • Lakebase: Serverless Postgres over Open Lake Storage (Jasraj Dange, Databricks et al.) - Modern cloud applications and AI-driven workloads demand elastic compute, instant environment isolation, and open data interoperability. However, production databases remain largely based on monolithic architectures designed in the 1980s, tightly coupling storage and compute and creating operational fragility, cost inefficiencies, and vendor lock-in. This paper presents Lakebase, a new open database architecture that decouples transactional compute from storage while placing database data directly in low-cost cloud object stores in open format. Unlike second-generation cloud databases, such as Amazon Aurora, Microsoft Socrates and Google AlloyDB, which separate storage internally but retain proprietary formats and single-engine access, Lakebase exposes storage as an open, durable lake layer while running elastic, serverless PostgreSQL transactional engines independently on top. We describe the system design principles behind Lakebase, including open-format object storage persistence, compute elasticity down to zero, instant branching and cloning via copy-on-write semantics, and unified transactional-analytical access over shared lake storage, as deployed in the Databricks Lakebase service. We analyze how this architecture reduces operational complexity, enables Git-like database workflows, improves failure recovery semantics, and mitigates structural vendor lock-in. We further discuss production deployment characteristics, performance trade-offs, durability guarantees, and support for multicloud high availability. We evaluate Lakebase on standard OLTP and analytical benchmarks, alongside production telemetry on branching and elasticity. Lakebase represents a third generation of database architecture: transactional systems rebuilt natively for cloud object storage, elastic compute, and AI-era development workflows. • SalesforceDB: A Cloud-native Multi-tenant OLTP Database (Atish Agrawal, Salesforce et al.) - Salesforce Database (SalesforceDB) is a multi-tenant relational database built by Salesforce to power the transactional workload of its Customer Relationship Management applications and platform. It is the system of record and an integral part of the Salesforce stack. Using its own database allows Salesforce to innovate from the database layer up and integrate the application's multi-tenancy into the database, enables the creation of new capabilities, optimizes operations, and improves security. Designed from the ground up to be cloud-native with a focus on high reliability and performance, it allows for simple, reliable cloud operations and scales to Salesforce’s largest customers. The database manages customer data within the Salesforce infrastructure but is not directly accessible by customers. It is deployed in production across thousands of instances and dozens of regions around the world and holds many petabytes of data. This paper describes the motivation and innovative design of SalesforceDB. Like many modern databases, SalesforceDB is cloud-native and decouples compute and storage. Its design reflects Salesforce-specific requirements: it natively supports multi-tenancy, employs a shared, immutable, distributed storage system, and uses a tenant-clustered log-structured merge tree (LSM) for data access. This enables it to support high levels of availability and durability, including tolerating availability zone (AZ) failures. SalesforceDB’s architecture includes a novel LSM virtualization layer that facilitates tenant-level features that cannot be efficiently supported on existing database architectures. These include creating instant tenant copies, seamless tenant migration across database instances via metadata-only operations and efficient tenant-level encryption. Its shared-table multi-tenancy design minimizes per-tenant overhead. SalesforceDB’s scale-out architecture leverages its shared LSM storage to achieve high scalability with minimal coordination.read more
☆
Wed Sep 2 · 10:45 AM - 12:15 PM
DEI Panel
● DEI Talks
📍 Grand Ballroom A+B
Beyond Publications: Navigating the Realities of a Research Career - Our panelists, Ziawasch Abedjan, Carsten Binnig, Anna Fariha, and Nesime Tatbul, bring diverse experiences and perspectives from across the data management community.Beyond Publications: Navigating the Realities of a Research Career - Our panelists, Ziawasch Abedjan, Carsten Binnig, Anna Fariha, and Nesime Tatbul, bring diverse experiences and perspectives from across the data management community. The discussion will cover work - life balance, research funding, collaborations, and the implications of single- versus double-blind reviewing for fairness and inclusion. The session will encourage an open discussion with the audience on practical ways to foster diversity, equity, and inclusion in our community.read more
Brit Youngmann
Ziawasch Abedjan
Carsten Binnig
Anna Fariha
Nesime Tatbul
☆
Wed Sep 2 · 10:45 AM - 12:15 PM
Sponsor Talks 4
● Sponsor Talks
📍 Otis
Aurora DSQL Under The Hood - Talking About the Architecture of a Multi-Region Distributed Database (Vlad Vlasceanu, Amazon) • Couchbase: A Database Architected for Agentic Workloads (Keshav Murthy, Couchbase) • How AI and Database Work Together in Bytedance Database Products...Aurora DSQL Under The Hood - Talking About the Architecture of a Multi-Region Distributed Database (Vlad Vlasceanu, Amazon) • Couchbase: A Database Architected for Agentic Workloads (Keshav Murthy, Couchbase) • How AI and Database Work Together in Bytedance Database Products (Jianjun Chen, ByteDance) • Data Management Landscape at SAP in the Age of Business AI (Manoj Kumar Sharma, SAP)read more
Built for you by(617)-PATRICK, Champion AI speaker
Relational Deep Dive: Error-Aware Queries Over Unstructured Data (Daren Chao, University of Toronto et al.) - Unstructured data is pervasive, but analytical queries demand structured representations, creating a significant extraction challenge.Relational Deep Dive: Error-Aware Queries Over Unstructured Data (Daren Chao, University of Toronto et al.) - Unstructured data is pervasive, but analytical queries demand structured representations, creating a significant extraction challenge. Existing methods like RAG lack schema awareness and struggle with cross-document alignment, leading to high error rates. We propose ReDD (Relational Deep Dive), a framework that dynamically discovers query-specific schemas, populates relational tables, and ensures error-aware extraction with provable guarantees. ReDD features a two-stage pipeline: (1) Iterative Schema Discovery (ISD) identifies minimal, joinable schemas tailored to each query, and (2) Tabular Data Population (TDP) extracts and corrects data using lightweight classifiers trained on LLM hidden states. A main contribution of ReDD is SCAPE, a statistically calibrated method for error detection with coverage guarantees, and SCAPE-HYB, a hybrid approach that optimizes the trade-off between accuracy and human correction costs. Experiments across diverse datasets demonstrate ReDD's effectiveness, reducing data extraction errors from up to 30% to below 1% while maintaining high schema completeness (100% recall) and precision. ReDD's modular design enables fine-grained control over accuracy-cost trade-offs, making it a robust solution for high-stakes analytical queries over unstructured corpora. • Ken: An Execution Engine for Unstructured Database Systems (Ferdinand Kossmann, MIT et al.) - Unstructured database management systems (UDBMSes) leverage machine learning to apply the relational model to modalities beyond tables, such as documents, images and videos. Queries in a UDBMS consist of logical operators for which the UDBMS chooses physical implementations (e.g., different models) with the goal to optimize both query latency and accuracy. However, many operators only expose a coarse-grained set of implementations, forcing the UDBMS to excessively sacrifice either accuracy or latency without middle-ground options. For example, an entity matching operator can either be implemented through small, specialized models or large, general-purpose models (e.g., Large Language Models) --- while the former struggles on challenging inputs, the latter is more accurate but incurs orders of magnitude more computation. In this work, we aim to address this issue with model cascades, which seek to process “easy” inputs with small models and only resort to large models when necessary. However, cascades incur higher memory usage and additional data transfer between GPU memory and arithmetic units, which often slows queries compared to single models. To address this issue, we introduce Ken, a dedicated UDBMS execution engine that dynamically adapts its use of cascades to the query load, and optimizes the GPU placement and invocation scheduling of the cascade models. Compared to baselines, Ken achieves 1.7x - 3.3x latency reductions when combining similar models for a single operator, and 122x latency reductions when combining models with orders of magnitude size differences in a multi-operator query. • Multi-Objective Agentic Rewrites for Unstructured Data Processing (Lindsey Wei, University of Washington et al.) - One year ago, we open-sourced DocETL, a declarative system for LLM-powered data processing that, as of November 2025, has 3.2K GitHub stars and users across domains (e.g., journalism, law, medicine, policy, finance, and urban planning). In DocETL, users build pipelines by composing operators described in natural language, also known as semantic operators, with an LLM executing each operator's logic. However, due to complexity in the operator or the data it operates on, LLMs often give inaccurate results. To address this challenge, DocETL introduced rewrite directives, or abstract rules that guide LLM agents in rewriting pipelines by decomposing operators or data. For example, decomposing a single filter("is this email sent from an executive and discussing fraud?") into the conjunction of two separate semantic filters may improve accuracy. However, DocETL only optimizes for accuracy, not cost. How do we optimize for both? We present MOAR (Multi-Objective Agentic Rewrites), a new optimizer for DocETL. To target cost optimization, we introduce two new categories of directives and extend all three existing categories with new ones, bringing the total to over 30 directives - more than doubling what DocETL originally had. Moreover, since operators can interact with each other unpredictably due to LLM behavior, optimizing operators or sub-pipelines individually can yield suboptimal overall plans. Recognizing this, we design a new global search algorithm that explores rewrites in the context of entire pipelines. Since the space of rewrites is infinite - pipelines can be rewritten in many ways, and each rewritten pipeline can itself be rewritten - our algorithm adapts a multi-armed bandit framework to prioritize which pipelines to rewrite. Across six workloads, MOAR achieves 27% higher accuracy than ABACUS, the next-best optimizer, while matching its best accuracy at 55% of its cost. • RED-ANNS: An RDMA-Enabled Distributed Framework for Graph-Based Approximate Nearest Neighbor Search (Yue Chen, Fudan University et al.) - Unstructured data, such as text and images, are converted into high-dimensional vectors to capture their semantics for effective data retrieval. Approximate Nearest Neighbor Search (ANNS) over these vectors has become a fundamental technique in many domains, including retrieval-augmented generation and recommendation systems. With an ever-increasing volume of data, existing distributed solutions typically segment data across multiple machine nodes, handling query processing in a MapReduce-style approach. However, this approach suffers from reduced indexing efficiency and increased computational overhead, resulting in limited performance enhancement despite investing several times more resources. In this work, we propose RED-ANNS, a distributed ANNS approach on an RDMA network. The core idea is to maintain a logically full graph across a shared memory space of multiple nodes and utilize Remote Direct Memory Access (RDMA) to search the distributed graph, thereby avoiding the reduction in indexing efficiency caused by segmentation. The key to making this approach effective is to address the overhead associated with remote accesses. We reduce remote access frequency through locality-aware data placement and affinity-based query scheduling, while we hide remote access latency with a dependency-relaxed best-first search algorithm. Extensive experiments demonstrate that RED-ANNS achieves a performance improvement of up to 2.5× over MapReduce-style approaches and up to 5.3× over open source vector databases. • TablePuppet: Towards a Generic Framework for Learning over Relational Tables (Lijie Xu, Institute of Software Chinese Academy of Sciences Chulin Xie, UIUC: University o) - Learning over (distributed) relational tables (LRT) requires applying SQL queries that involve costly operations such as joins and unions to compose the training dataset, followed by model training atop the query results. This paradigm often introduces considerable computation, storage, and communication overhead that cannot be addressed by existing approaches. In this paper, we propose TablePuppet, a generic framework that can significantly reduce the overhead of LRT. We first formalize the LRT problem as learning over union of conjunctive queries (UCQ). We then decompose the learning process into two steps: (1) learning over join (LoJ), followed by (2) learning over union (LoU). In essence, LoJ pushes learning down to the individual tables being joined, while LoU further pushes learning down to the horizontal partitions/shards of each table. This two-step decomposition approach enables efficient distributed training without raw table sharing while preserving model accuracy. TablePuppet supports two standard ML optimization strategies, stochastic gradient descent (SGD) and alternating direction method of multipliers (ADMM), and can accommodate both centralized and distributed environments. In addition, TablePuppet introduces computation and communication optimizations to handle duplicate tuples introduced by joins, while further offering privacy guarantees for federated learning (FL) scenarios. Experimental evaluation results show that TablePuppet achieves comparable model accuracy to centralized baselines running directly on top of the SQL query results. Moreover, the SGD and ADMM algorithms implemented atop TablePuppet take less communication/training time to converge compared to the state-of-the-art approaches. • Computing Why-Provenance for Property Graph Queries (Koumudi Ganepola, TU Wien et al.) - Labeled Property Graphs (LPGs) have gained widespread adoption, supported by the ongoing standardization of the Graph Query Language (GQL). As LPGs are frequently constructed through the integration of heterogeneous data sources, inconsistencies and errors may propagate into query results. To explain and validate such results, it is essential to determine which parts of the input graph contributed to a given answer -- an aspect captured by data provenance. In this paper, we present a formal definition of why-provenance tailored specifically to GQL, characterizing the fine-grained subgraph that contribute to a query answer. Building on the Core-GQL algebraic foundation, we further propose an efficient query-rewriting-based algorithm for computing provenance, leveraging the native execution capabilities of existing GQL-compliant graph database systems. Our extensive experimental evaluation demonstrates the feasibility and scalability of our approach, showing that provenance can be computed with moderate overhead that scales effectively with graph size.read more
☆
Wed Sep 2 · 1:45 PM - 3:15 PM
Research 26 · ANN Search Systems
● Research
📍 Grand Ballroom D
CONDA: A Connectivity-Aware Dynamic Index for Approximate Nearest Neighbor Search over Evolving Data (Darae Lee, KAIST et al.) - Graph-based Approximate Nearest Neighbor Search (ANNS) indices must support continuous vector updates while maintaining both high search performance and low update...CONDA: A Connectivity-Aware Dynamic Index for Approximate Nearest Neighbor Search over Evolving Data (Darae Lee, KAIST et al.) - Graph-based Approximate Nearest Neighbor Search (ANNS) indices must support continuous vector updates while maintaining both high search performance and low update overhead for real-world applications such as RAG systems and streaming services. Existing in-place update methods often lose search accuracy due to graph connectivity loss and suffer high overhead from expensive deletion operations. We propose CONDA, a dynamic graph-based index featuring a topology-aware pruning rule with bidirectional link reinforcement and a lightweight lazy deletion scheme. Extensive experiments demonstrate that CONDA improves search recall by up to 24.5% over state-of-the-art methods while achieving 1.90x higher update throughput. • Aker: Density-Aware Approximate Caching for Vector Search (Sukjoon Oh, Korea Advanced Institute of Science and Technology et al.) - Disk-based approximate nearest neighbor search (ANNS) incurs high I/O overhead due to frequent disk accesses during index traversal. Approximate caching, which reuses the results of past queries to serve future similar queries, offers a promising approach to bypass disk searches. However, existing approaches suffer from two limitations. First, their hit predicates fail to simultaneously achieve high throughput and high accuracy, as they do not adapt to the varying local neighbor density. Second, they lack an effective refresh mechanism to maintain cache correctness during vector updates. We present Aker, an approximate cache for disk-based ANNS. Aker addresses these limitations through two core design choices. First, we introduce a per-query similarity threshold that each cache entry dynamically adjusts based on cache hit patterns. This design enables Aker to adapt to neighborhood densities to preserve both efficiency and accuracy. Second, we propose del-consistency, a consistency model that applies deletions eagerly and insertions lazily. Under this model, Aker implements a low-overhead refresh mechanism that bounds cache staleness while preserving search accuracy. We integrate Aker into pgvector and evaluate it on representative workloads. Aker improves recall by up to 64 percentage points over prior solutions and increases QPS by up to 3.2×, while using 0.6× the memory of pgvector's shared buffers. • PAIL: Efficient kNN Search on Set-Valued Attributes (Daniel Schmitt, University of Salzburg et al.) - We study the k-nearest neighbors (kNN) search problem on the domain of sets. Given a query set, the goal is to retrieve the k most similar sets from a collection according to a specified similarity function. Most existing solutions for set similarity queries focus on range search or top-k joins, which typically assume and exploit high similarity thresholds. We observe that existing approaches for kNN search - as well as adaptations of range search and top-k algorithms - exhibit poor performance due to low selectivity of their filtering techniques and high index traversal costs. To address these limitations, we propose PAIL, a kNN search algorithm for sets that supports a wide range of similarity functions. PAIL implements the positional filter - a filter that was previously used for post-filtering of candidates returned by an index - directly into a novel index structure to effectively prune candidates. To efficiently traverse only the necessary parts of the index, PAIL leverages the monotonicity of the similarity functions with respect to positional information. This traversal enables early termination by ensuring that the index is accessed in descending order of similarity upper bounds. To reduce index access overhead, we propose size grouping and eager reading of index entries that relax filter tightness for improved overall performance. Extensive experiments across diverse datasets demonstrate that PAIL consistently outperforms competing algorithms by up to three orders of magnitude. • QBAT: Model-based Query Budget Autotuner for Clustering-based Approximate Nearest Neighbor Search (Jonghyun Bae, Google et al.) - Approximate nearest neighbor search (ANNS) is a critical component in modern data-intensive applications, but its performance is often hindered by the use of a static query budget parameter. This one-size-fits-all approach, even if well-tuned, fails to account for the varying difficulty of individual queries, inevitably leading to suboptimal latency on easy queries and poor accuracy on hard ones. This paper introduces QBAT, a query-aware budget autotuner designed to resolve this dilemma. By analyzing query-specific features offline, QBAT dynamically allocates an appropriate budget for each query. We explore two predictive models: a highly accurate gradient-boosted decision tree and a simple, interpretable heuristic formula derived using the AlphaEvolve framework. These models can optimize budget allocation for both system performance or recall consistency priorities. Evaluations on large-scale datasets demonstrate that QBAT reduces total searched budget by up to 68.8% in the consistency mode on ScaNN, the state-of-the-art clustering-based ANNS method, while simultaneously enforcing a strict per-query recall target, a scenario where static budgets are notoriously inefficient and wasteful. • SVFusion: A CPU-GPU Co-Processing Architecture for Large-Scale Real-Time Vector Search (Yuchen Peng, Zhejiang University et al.) - Approximate Nearest Neighbor Search (ANNS) underpins modern applications such as information retrieval and recommendation. With the rapid growth of vector data, efficient indexing for real-time vector search has become rudimentary. Existing CPU-based solutions support updates but suffer from low throughput, while GPU-accelerated systems deliver high performance but face challenges with dynamic updates and limited GPU memory, resulting in a critical performance gap for continuous, large-scale vector search requiring both accuracy and speed. In this paper, we present SVFusion, a GPU-CPU-disk collaborative framework for real-time vector search that bridges sophisticated GPU computation with online updates. SVFusion leverages a hierarchical vector index architecture that employs CPU-GPU co-processing, along with a workload-aware vector caching mechanism to maximize the efficiency of limited GPU memory. It further enhances performance through real-time coordination with CUDA multi-stream optimization and adaptive resource management, along with concurrency control that ensures data consistency under interleaved queries and updates. Empirical results demonstrate that SVFusion achieves significant improvements in query latency and throughput, exhibiting a 20.9$\times$ higher throughput on average and 1.3$\times$ to 50.7$\times$ lower latency compared to baseline methods, while maintaining high recall for large-scale datasets under various streaming workloads. • I/O Optimizations in Graph-Based Disk-Resident Approximate Nearest Neighbor Search: A Design Space Exploration (liang li, China Telecom Cloud Computing Research Institute et al.) - Approximate nearest neighbor (ANN) search on SSD-backed indexes is increasingly I/O-bound (I/O accounts for 70--90% of query latency). We present an I/O-first framework for disk-based ANN that organizes techniques along three dimensions: memory layout, disk layout, and search algorithm. We introduce a page-level complexity model that explains how page locality and path length jointly determine page reads, and we validate the model empirically. Using consistent implementations across four public datasets, we quantify both single-factor effects and cross-dimensional synergies. We find that (i) memory-resident navigation and dynamic width provide the strongest standalone gains; (ii) page shuffle and page search are weak alone but complementary together; and (iii) a principled composition, OctopusANN, substantially reduces I/O and achieves 4.1--37.9% higher throughput than the state-of-the-art system Starling and 87.5--149.5% higher throughput than DiskANN at matched Recall@10=90%. Finally, we distill actionable guidelines for selecting storage-centric or hybrid designs across diverse concurrency levels and accuracy constraints, advocating systematic composition rather than isolated tweaks when pushing the performance frontier of disk-based ANN.read more
☆
Wed Sep 2 · 1:45 PM - 3:15 PM
Research 27 · GPU-Accelerated Query Processing
● Research
📍 Grand Ballroom C
GPU Acceleration of SQL Analytics on Compressed Data (Zezhou Huang, Microsoft et al.) - GPUs are uniquely suited to accelerate (SQL) analytics workloads when datasets fit in the GPU High Bandwidth Memory (HBM).GPU Acceleration of SQL Analytics on Compressed Data (Zezhou Huang, Microsoft et al.) - GPUs are uniquely suited to accelerate (SQL) analytics workloads when datasets fit in the GPU High Bandwidth Memory (HBM). Unfortunately, GPU HBMs remain typically small when compared with lower-bandwidth CPU main memory. Current solutions to accelerate queries on large datasets include multi-GPU execution, processing smaller data batches, and hybrid execution with a connected device (e.g., CPUs). Unfortunately, these approaches are exposed to the limitations of lower main memory and host-to-device interconnect bandwidths, introduce additional I/O overheads, or incur higher costs. This is a substantial problem when trying to scale adoption of GPUs on larger datasets. Data compression can alleviate this bottleneck, but to avoid paying for costly decompression/decoding, an ideal solution must include computation primitives to operate directly on data in compressed form. This is the focus of our paper: a set of new methods for running queries directly on light-weight compressed data using schemes such as Run-Length Encoding (RLE), index encoding, bit-width reductions, and dictionary encoding. Our novelty includes operating on multiple RLE columns without decompression, handling heterogeneous column encodings, and leveraging PyTorch tensor operations for portability across devices. Experimental evaluations show speedups of an order of magnitude compared to state-of-the-art commercial CPU-only analytics systems, for real-world queries on a production dataset that would not fit into GPU memory uncompressed. This work paves the road for GPU adoption in a much broader set of use cases, and it is complementary to most other scale-out or fallback mechanisms. • PystachIO: Efficient Distributed GPU Query Processing with PyTorch over Fast Networks & Fast Storage (Jigao Luo, TU Darmstadt et al.) - The AI hardware boom has led modern data centers to adopt HPC-style architectures centered on distributed, GPU-centric computation. Large GPU clusters interconnected by fast RDMA networks and backed by high-bandwidth NVMe storage enable scalable computation and rapid access to storage-resident data. Tensor computation runtimes (TCRs), such as PyTorch, originally designed for AI workloads, have recently been shown to accelerate analytical workloads. However, prior work has primarily considered settings where the data fits in aggregated GPU memory. In this paper, we systematically study how TCRs can support scalable, distributed query processing for large-scale, storage-resident OLAP workloads. Although TCRs provide abstractions for network and storage I/O, naive use often underutilizes GPU and I/O bandwidth due to insufficient overlap between computation and data movement. As a core contribution, we present PystachIO, a prototype of a PyTorch-based distributed OLAP engine that combines fast network and storage I/O with key optimizations to maximize GPU, network, and storage utilization. Our evaluation shows up to 3x end-to-end speedups over existing distributed GPU-based query processing approaches. • Bridging the Indexing Gap in Fused GPU Query Engines (Tianjun Bu, University of Chinese Academy of Sciences et al.) - GPU query backends achieve high throughput on analytical work- loads through massive parallelism, but lack indexing support that accelerates selective queries in CPU databases. Existing GPU index implementations face three limitations: (1) supporting conjunctive predicates only, (2) materializing intermediate results between in- dex access and query execution, and (3) assuming query boundaries align with pre-built index bins. We present a fused bitmap indexing approach that addresses these limitations. We introduce virtual query program that exe- cutes arbitrary boolean predicates with low overhead. We fuse index access with subsequent column lookups, joins, and aggre- gation, keeping intermediate results in registers and eliminating global-memory round-trips. To handle misaligned query bound- aries, we propose GPU friendly candidate checking that tracks three-valued row states (certain-in, certain-out, uncertain) through in-register boolean operations and verifies only the necessary can- didates, without accessing global memory. On Star Schema Benchmark SF=140 with RTX 5090 D, our fused bitmap index achieves up to 6.9× geometric-mean speed over our optimized non-indexed baseline built upon the Crystal GPU data- base query backend (Dense layout), and 4.3× with practical Sparse layout using less memory. Compared to current best compressed GPU bitmap implementation under perfect bin alignment (best case), our Sparse layout achieves 1.4× speed end-to-end. We show that generic elementwise-style GPU fusion achieves only 1.34× speed, while our pipeline reaches 3.19× with 0.8% overhead versus dedicated compile-time kernels. Results on an NVIDIA H800 server GPU show the approach remains stable across GPU architectures, with smaller but still consistent fusion benefits on server GPUs. • ThunderGNN: Unlocking Tensor Cores for Graph Neural Networks (YuAng Chen, The Chinese University of Hong Kong in Shenzhen et al.) - Graph Neural Networks (GNNs) have emerged as the state-of-the-art methodology for learning on graph-structured data, yet their performance is severely constrained by a fundamental mismatch between irregular graph sparsity and the rigid parallelism of modern hardware. While modern GPUs rely on Tensor Cores (TCs) to deliver massive computational throughput, these units demand strictly tiled, dense inputs - a requirement that conflicts with the extreme sparsity of real-world graphs. Existing frameworks fail to resolve this design conflict: they either fallback to legacy SIMT cores, leaving TCs underutilized, or they incur prohibitive memory bloat by forcing sparse data into dense tiles via excessive padding. To bridge this gap, we propose ThunderGNN, a hardware-aware acceleration system designed to reconcile graph irregularity with Tensor Core rigidity. ThunderGNN employs a unified co-design strategy comprising three key optimizations: (1) a sparsity-aware reordering algorithm that logically groups graph rows to maximize local density; (2) a Condensed Binarized Abstraction (CBA) storage layout that physically organizes the adjacency matrix into TC-aligned blocks without explicit zero-padding; and (3) a hardware-aware execution engine that efficiently streams compressed blocks directly into TCs. Extensive experiments on NVIDIA A100 GPUs demonstrate that ThunderGNN significantly outperforms state-of-the-art systems, achieving geometric mean speedups of 1.89X over DGL and 2.59X over PyG. • Terabyte-Scale Analytics in the Blink of an Eye (Bowen Wu, ETH Zurich et al.) - For the past two decades, the DB community has devoted substantial research to take advantage of cheap clusters of machines for distributed data analytics - we believe that we are at the beginning of a paradigm shift. The scaling laws and popularity of AI models lead to the deployment of incredibly powerful GPU clusters in commercial data centers. Compared to CPU-only solutions, these clusters deliver impressive improvements in per-node compute, memory bandwidth, and inter-node interconnect performance. In this paper, we study the problem of scaling analytical SQL queries on distributed clusters of GPUs, with the stated goal of establishing an upper bound on the likely performance gains. To do so, we build a prototype designed to maximize performance by leveraging ML/HPC best practices, such as group communication primitives for cross-device data movements. This allows us to conduct thorough performance experimentation to point our community towards a massive performance opportunity of at least 60x. To make these gains more relatable, before you can blink twice, our system can run all 22 queries of TPC-H at a 1TB scale factor!read more
☆
Wed Sep 2 · 1:45 PM - 3:15 PM
Research 28 · Query Optimization
● Research
📍 Commonwealth Ballroom C
Hybrid Mixed Integer Linear Programming for Large-Scale Join Order Optimisation (Manuel Schönberger, Technical University of Applied Sciences Regensburg et al.) - Finding optimal join orders is among the most crucial steps to be performed by query optimisers.Hybrid Mixed Integer Linear Programming for Large-Scale Join Order Optimisation (Manuel Schönberger, Technical University of Applied Sciences Regensburg et al.) - Finding optimal join orders is among the most crucial steps to be performed by query optimisers. Though extensively studied in data management research, the problem remains far from solved: While query optimisers rely on exhaustive search methods to determine ideal solutions for small problems, such methods reach their limits once queries grow in size. Yet, large queries become increasingly common in real-world scenarios, and require suitable methods to generate efficient execution plans. While a variety of heuristics have been proposed for large-scale query optimisation, they suffer from degrading solution quality as queries grow in size, or feature highly sub-optimal worst-case behavior, as we will show. We propose a novel method based on the paradigm of mixed integer linear programming (MILP): By deriving a novel MILP model capable of optimising arbitrary bushy tree structures, we address the limitations of existing MILP methods for join ordering, and can rely on highly optimised MILP solvers to derive efficient tree structures that elude competing methods. To ensure optimisation efficiency, we embed our MILP method into a hybrid framework, which applies MILP solvers precisely where they provide the greatest advantage over competitors, while relying on more efficient methods for less complex optimisation steps. Thereby, our approach gracefully scales to extremely large query sizes joining up to 100 relations, and consistently achieves the most robust plan quality among a large variety of competing join ordering methods. • One Join Order Does Not Fit All: Reducing Intermediate Results with Per-Split Query Plans (Yujun He, Southern University of Science and Technology et al.) - Minimizing intermediate results is critical for efficient multi-join query processing. Although the seminal Yannakakis algorithm offers strong guarantees for acyclic queries, cyclic queries remain an open challenge. In this paper, we propose SplitJoin, a framework that introduces split as a first-class query operator. By partitioning input tables into heavy and light parts, SplitJoin allows different data partitions to use distinct query plans, with the goal of reducing intermediate sizes using existing binary join engines. We systematically explore the design space for split-based optimizations, including threshold selection, split strategies, and join ordering after splits. Implemented as a front-end to DuckDB and Umbra, SplitJoin achieves substantial improvements: on DuckDB, SplitJoin completes 43 social network queries (vs. 29 natively), achieving 2.1x faster runtime and 7.9x smaller intermediates on average (up to 13.6x and 74x, respectively); on Umbra, it completes 45 queries (vs. 35), achieving 1.3x speedups and 1.2x smaller intermediates on average (up to 6.1x and 2.1x, respectively). • FlowLog: Efficient and Extensible Datalog via Incrementality (Hangdong Zhao, University of Wisconsin, Madison et al.) - Datalog-based languages are regaining popularity as a powerful abstraction for expressing recursive computations in domains such as program analysis and graph processing. However, existing systems often face a trade-off between efficiency and extensibility. Engines like Souffle achieve high efficiency through domain-specific designs, but lack general-purpose flexibility. Others, like RecStep, offer modularity by layering Datalog on traditional databases, but struggle to integrate Datalog-specific optimizations. This paper bridges this gap by presenting FlowLog, a new Datalog engine that uses an explicit relational IR per-rule to cleanly separate recursive control (e.g., semi-naïve execution) from each rule's logical plan. This boundary lets us retain fine-grained, Datalog-aware optimizations at the logical layer, but also reuse off-the-shelf database primitives at execution. At the logical level (i.e. IR), we apply proven SQL optimizations, such as logic fusion and subplan reuse. To address high volatility in recursive workloads, we adopt a robustness-first approach that pairs a structural optimizer (avoiding worst-case joins) with sideways information passing (early filtering). Built atop Differential Dataflow - a mature framework for streaming analytics - FlowLog supports both batch and incremental Datalog and adds novel recursion-aware optimizations called Boolean (or algebraic) specialization. Our evaluation shows that FlowLog outperforms state-of-the-art Datalog engines and modern databases across a broad range of recursive workloads, achieving superior scalability while preserving a simple and extensible architecture. • Robust Predicate Transfer with Dynamic Execution (Yiming Qiao, Tsinghua University et al.) - Efficient join query execution remains a key challenge in modern database systems. Although a recent method, Robust Predicate Transfer (RPT), improves robustness against suboptimal join orders, it introduces significant overhead from redundant filter creation and inefficient data scanning. We present RPT+ that addresses these issues through three key improvements. First, we propose asymmetric transfer plans to reduce redundant Bloom filter constructions. Second, we design cascade filters to improve data scanning efficiency by enabling both block-level skipping and tuple-level filtering. Third, we introduce dynamic pipelines to allow runtime filter creation and transfer plan adjustment. We implemented RPT+ in DuckDB (v1.3.0) and evaluated it across multiple benchmarks, including the Join Order Benchmark (JOB), SQLStorm, TPC-H, and Appian. Compared to the baseline DuckDB, RPT+ achieves speedups of 1.47x on JOB, 1.28x on SQLStorm, 1.10x on TPC-H, and 1.01x on Appian. Importantly, it avoids the significant performance regressions observed with the original RPT. These results demonstrate that RPT+ not only improves query performance but also maintains the robustness of RPT across diverse workloads. • Window Function Optimization: Co-Evaluation and Other Techniques (Daniel Lindner, Hasso Plattner Institute, University of Potsdam et al.) - Window functions are among the most expressive features of modern SQL. Surprisingly, relatively little has been written about their optimization. Some techniques exist, such as pushing predicates through a window under ideal conditions, but known optimizations no longer apply when those conditions are even slightly unmet. We show that these limitations are not fundamental, but persist because a reasoning framework for window function optimization has been missing. We provide such a framework, introducing techniques we call Frame Analysis, Partition Analysis, and a new execution strategy called Co-Evaluation. These clarify when and how optimizations can be applied. Co-Evaluation, in particular, allows early evaluation of predicates even when they depend on the window function’s result. We present each technique and organize the re- sults as a table of algebraic equivalences for window functions. We test these optimizations in an open-source engine, where they never hurt performance and make certain common queries up to 40.7× faster, with larger tables yielding larger gains.read more
☆
Wed Sep 2 · 1:45 PM - 3:15 PM
Research 29 · Database Configuration and Reliability
● Research
📍 Commonwealth Ballroom B
BBC: Improving Large-𝑘 Approximate Nearest Neighbor Search with a Bucket-based Result Collector (Ziqi Yin, Nanyang Technological University et al.) - Although Approximate Nearest Neighbor (ANN) search has been extensively studied, large-𝑘 ANN queries that aim to retrieve a large number...BBC: Improving Large-𝑘 Approximate Nearest Neighbor Search with a Bucket-based Result Collector (Ziqi Yin, Nanyang Technological University et al.) - Although Approximate Nearest Neighbor (ANN) search has been extensively studied, large-𝑘 ANN queries that aim to retrieve a large number of nearest neighbors remain underexplored, despite their numerous real-world applications. Existing ANN methods face significant performance degradation for such queries. In this work, we first investigate the reasons for the performance degradation of quantization-based ANN indexes: (1) the inefficiency of existing top-𝑘 collectors, which incurs significant overhead in candidate maintenance, and (2) the reduced pruning effectiveness of quantization methods, which leads to a costly re-ranking process. To address this, we propose a novel bucket-based result collector (BBC) to enhance the efficiency of existing quantization-based ANN indexes for large-𝑘 ANN queries. BBC introduces two key components: (1) a bucket-based result buffer that organizes candidates into buckets by their distances to the query. This design reduces ranking costs and improves cache efficiency, enabling high performance maintenance of a candidate superset and a lightweight final selection of top-𝑘 results. (2) two re-ranking algorithms tailored for different types of quantization methods, which accelerate their re-ranking process by reducing either the number of candidate objects to be re-ranked or cache misses. Extensive experiments on real-world datasets demonstrate that BBC accelerates existing quantization-based ANN methods by up to 3.8× at recall@𝑘 = 0.95 for large-𝑘 ANN queries. • Toward Drift-Aware Database Benchmarking (Guanli Liu, The University of Melbourne et al.) - Data and workload drift are critical to evaluating core database components such as caching, cardinality estimation, indexing, and query optimization, especially as AI-driven techniques increasingly permeate database systems. However, existing benchmarks remain largely static, offering little support for modeling drifts. This limitation arises from the absence of a shared vocabulary and practical tools for specifying and generating drift in both data and workloads. Guided by this vision of making drift a first-class concept, we propose a taxonomy of data and workload drift and design DriftSpec, a declarative specification that makes these drifts executable. Building on this, we present DriftBench, which instantiates DriftSpec to generate controlled drifts and enable drift-aware benchmarking. Together, the taxonomy, DriftSpec, and DriftBench form a first step toward a standardized, executable language for studying how data and workload evolution influence database behavior. They shift benchmarking from static, one-off tests to controlled, continuous evaluation under drift. • Storage-Centric Relation Design via High-Quality Approximate Functional Dependencies (Rui Ding, Northeastern University et al.) - As storage costs continue to rise, reducing redundancy has become increasingly important. In relational databases, classical normalization addresses redundancy through exact functional dependencies (FDs), but this rule-based design paradigm is not inherently cost-aware and does not necessarily minimize storage in practice. Moreover, much real-world redundancy follows FD+$\Delta$ patterns, where FDs hold for most tuples but are violated by a small fraction. To address this, we propose RelaxRD, a storage-centric relaxed schema design that leverages approximate functional dependencies (AFDs) to reduce redundancy in FD+$\Delta$. Rather than treating all AFDs as equally useful signals, we quantify the storage value of AFD subsets via duplicate gain and select a high-quality subset for decomposition. It decomposes tuples satisfying the selected AFDs while retaining violating tuples. The key issue is that selecting a high-quality subset is difficult due to conflicts and the exponential search space. To tackle this, we develop a family of efficient filtering techniques to eliminate low-value and unpromising candidates without exhaustive enumeration. Extensive experiments on real-world datasets demonstrate that RelaxRD consistently achieves substantial storage savings. • Why Database Manuals Are Not Enough: Efficient and Reliable Configuration Tuning for DBMSs via Code-Driven LLM Agents (Xinyi Zhang, Renmin University of China et al.) - Modern database management systems (DBMSs) expose hundreds of configuration knobs that critically influence performance. Existing automated tuning methods either adopt a data-driven paradigm, which incurs substantial overhead, or rely on manual-driven heuristics extracted from database documentation, which are often limited and overly generic. Motivated by the fact that the control logic of configuration knobs is inherently encoded in the DBMS source code, we argue that promising tuning strategies can be mined directly from the code, uncovering fine-grained insights grounded in system internals. To this end, we propose SysInsight, a code-driven database tuning system that automatically extracts fine-grained tuning knowledge from DBMS source code to accelerate and stabilize the tuning process. SysInsight combines static code analysis with LLM-based reasoning to identify knob-controlled execution paths and extract semantic tuning insights. These insights are then transformed into quantitative and verifiable tuning rules via association rule mining grounded in tuning observations. During online tuning, system diagnosis is applied to identify critical knobs, which are adjusted under the rule guidance. Evaluations demonstrate that compared to the SOTA baseline, SysInsight converges to the best configuration on average 7.11× faster while achieving a 19.9% performance improvement. • Scarf: Self-Adaptive Tuning via Multi-Objective Reinforcement Learning for Apache Flink (Liu Liu, Zhejiang University et al.) - Distributed stream processing systems (DSPSs) such as Apache Flink have become omnipresent for real‐time data processing in e-commerce, finance, telecommunications, etc. The execution behavior of Flink is controlled by a vast and complex space of configuration knobs, necessitating automatic knob tuning to economize resource usage while maintaining sufficient processing capabilities for a given workload. Existing automatic methods largely adjust limited configuration knobs, respond slowly to dynamic workloads, and have difficulty transferring knowledge between heterogeneous jobs with diverse knob spaces. To solve these problems, we present Scarf, a self‐adaptive configuration tuning framework using multi‐objective reinforcement learning (RL) for Apache Flink. Specifically, (1) we accelerate job‐specific knob selection by clustering historical workloads according to their parameter‐response patterns, dramatically reducing redundant sampling; (2) we formulate tuning as a multi‐objective RL problem that jointly optimizes throughput and resource usage, learning a forest of RL models offline representing the Pareto front of the configurations, and dynamically selecting configurations from the Pareto front under fluctuating online workloads; (3) we enable rapid adaptation to new job topologies via a transferable actor - critic architecture based on graph neural networks (GNNs), complemented with a progressive neural‐network (PNN) warm‐up strategy. We implement Scarf on Apache Flink and evaluate it on a diverse range of streaming applications. Our framework significantly outperforms state-of-the-art DSPS tuning approaches, achieving up to 62.5% savings in CPU resources, 68.3% savings in memory usage, 77.1% reduction in online tuning time, while maintaining sufficient processing abilities throughout workload fluctuations. • Detecting Data-Type-Related Logic Bugs in Relational DBMSs via Compatible Database Construction (Jiansen Song, Institute of Software Chinese Academy of Sciences et al.) - Relational Database Management Systems (DBMSs) serve as foundational systems for data storage and management, supporting a rich variety of data types to specify storage formats and value ranges. These data types play a critical role in both data storage and computation. However, complex data computation operations (e.g., explicit and implicit data type conversions) can introduce data-type-related logic bugs (typeBugs for brevity). Specifically, typeBugs can cause SELECT statements to return incorrect query results, which can easily be overlooked by DBMS developers. Unfortunately, existing DBMS testing approaches do not examine data type conversions, rendering them ineffective at detecting typeBugs. We observe that database columns with different data types (e.g., INT and BIGINT) can store identical data values within specific value ranges, and executing identical SQL operations on these database columns should produce consistent results. Inspired by this observation, we propose TypeCheck, a novel testing approach for effectively detecting typeBugs in relational DBMSs. We first identify storage-compatible data types T that can preserve identical data values and operation-oriented type compatibility rules opRules that define SQL operations producing consistent results when applied to different data types. Using T and opRules, we construct type-compatible databases and execute identical type-compatible SELECT statements on them. When these SELECT statements yield divergent execution results, we detect a typeBug. We implement and evaluate TypeCheck on six widely-used relational DBMSs, and have detected 37 typeBugs, of which 34 have been confirmed as previously unknown bugs, with 10 of these confirmed bugs already fixed. • E2ETune: End-to-End Knob Tuning via Fine-tuned Generative Language Model (Xinmei Huang, Renmin University of China et al.) - Database knob tuning is a significant challenge for database administrators, as it involves tuning a large number of configuration knobs with continuous or discrete values to achieve optimal database performance. Traditional methods, such as manual tuning or learning-based approaches, typically require numerous workload replays and are both time-consuming and resource-intensive. To address this challenge, we introduce E2ETune, an end-to-end knob tuner powered by a fine-tuned generative language model. The key idea is to leverage the exceptional sequence-to-sequence modeling capabilities of generative language models to capture the complex mapping between workloads (inputs) and their corresponding promising configurations (outputs). To achieve this goal, we propose a novel data generation framework to efficiently produce a large amount of training data, where each data sample consists of a workload and its promising configuration. Then, these data are used to fine-tune a generative language model, yielding an end-to-end knob tuner. This tuner offers out-of-the-box configuration recommendations for new workloads. We conduct extensive experiments to evaluate E2ETune's efficiency and effectiveness using 10 representative and 3 real-world benchmarks. Compared to state-of-the-art methods, E2ETune can identify competitive configurations in significantly less time.read more
☆
Wed Sep 2 · 1:45 PM - 3:15 PM
Research 30 · Scalable Graph Algorithms
● Research
📍 Commonwealth Ballroom A
A Practical Sublinear Approximation for Group Steiner Tree (Yuxuan Yang, Nanjing University et al.) - The Group Steiner Tree Problem (GSTP) is widely used in graph data management and mining, yet existing algorithms trade off practical efficiency against approximation quality:...A Practical Sublinear Approximation for Group Steiner Tree (Yuxuan Yang, Nanjing University et al.) - The Group Steiner Tree Problem (GSTP) is widely used in graph data management and mining, yet existing algorithms trade off practical efficiency against approximation quality: efficient methods offer only linear guarantees, while those with sublinear guarantees fail to scale to large graphs. In this paper, we present MonoGST+, a novel algorithm for GSTP that breaks this trade‑off by achieving a sublinear approximation while matching the running time of state‑of‑the‑art linear‑approximation solvers. Our approach extends a 2‑star‑based reduction to weighted set cover with a suspendable search and a monotonicity‑ and unimodality‑aware pruning strategy to eliminate redundant computation. Experiments on multiple real‑world datasets demonstrate the effectiveness and efficiency of MonoGST+, providing a practical, high‑quality solution for GSTP applications. • Theoretically and Practically Efficient Resistance Distance Computation on Large Graphs (Yichun Yang, Beijing Institute of Technology et al.) - The computation of resistance distance is pivotal in a wide range of graph analysis applications, including maximum flow computation, graph sparsification, and graph clustering. Despite its foundational importance, efficient algorithms for computing resistance distances on large graphs are still lacking. Existing state-of-the-art (SOTA) methods, including power iteration-based global algorithms and random walk-based local approaches, often struggle with slow convergence rates, particularly when the condition number of the graph Laplacian matrix, denoted by $\kappa$, is large. To tackle this challenge, we propose two novel and efficient algorithms inspired by the classic Lanczos method: Lanczos Iteration and Lanczos Push, both designed to reduce dependence on $\kappa$. Among them, Lanczos Iteration is a near-linear time global algorithm, whereas Lanczos Push is a local algorithm with a time complexity independent of the size of the graph. More specifically, we prove that the time complexity of Lanczos Iteration is $\tilde{O}(\sqrt{\kappa} m)$ ($m$ is the number of edges of the graph and $\tilde{O}$ means the complexity omitting the $\log$ terms) which achieves a speedup of $\sqrt{\kappa}$ compared to previous power iteration-based global methods. For Lanczos Push, we demonstrate that its time complexity is $\tilde{O}(\kappa^{2.75})$ under certain mild and frequently established assumptions, which represents a significant improvement of $\kappa^{0.25}$ over the SOTA random walk-based local algorithms. We validate our algorithms through extensive experiments on eight real-world datasets of varying sizes and statistical properties, demonstrating that Lanczos Iteration and Lanczos Push significantly outperform SOTA methods in terms of both efficiency and accuracy. • Efficient Locally h-Clique Densest Subgraph Discovery via Divide-and-Conquer (Yingli Zhou, The Chinese University of Hong Kong, Shenzhen et al.) - Finding the densest subgraph (DS) from a graph is a fundamental problem in graph databases. It has been extensively studied in the literature and has found many real applications in a wide range of fields, such as biology, finance, and social networks. This paper studies how to efficiently discover the locally $h$-clique densest subgraph (L$h$CDS), which is a recently-proposed variant of DS. An L$h$CDS is a subgraph which is the densest among the ``local neighbors''. Given a graph $G$, a number of L$h$CDSes can be returned, which reflect different dense regions of $G$ and thus give more information than DS. Existing L$h$CDS solutions suffer from low efficiency due to extensively redundant computations. To improve efficiency, in this paper, we propose a divide-and-conquer-based algorithm, which not only reduces the search space but also has an improved time complexity. Extensive experiments on 15 real large graph datasets show that our proposed algorithm is up to two orders of magnitude faster than the state-of-the-art. • Efficient Hyper-truss Decomposition over Hypergraphs (Haozhe Yin, University of New South Wales et al.) - Cohesive subgraph mining in hypergraphs has recently attracted increasing research attention due to its broad applicability in domains such as social networks, co-authorship networks, and recommendation systems. An important model, the hyper k-truss, is defined as a maximal cohesive subgraph in which each hyperedge is contained in at least (k-2) hyper-triangles (i.e., structures formed by three pairwise connected hyperedges). In this paper, we study the problem of hyper-truss decomposition, which aims to identify all hyper k-trusses for k ≥ 0. Due to the complex structure of hyper-triangles, the existing hyperedge-aware framework for hyper-truss decomposition incurs extra computational cost by traversing open hyper-triangles (i.e., hyper-triangles in which two hyperedges are not connected). Moreover, existing strategies enumerate all supporting hyper-triangles for each peeled hyperedge, which substantially limits overall efficiency. To address these issues, we propose a vertex-aware framework that leverages vertex-level connectivity among hyperedges. Under this framework, we design a vertex-oriented counting strategy to completely eliminate the traversal of open hyper-triangles during the counting phase and a vertex-based state propagation method to minimize the number of hyper-triangles enumerated in the peeling phase. Extensive experiments on eleven real-world datasets demonstrate the effectiveness and efficiency of our approach. • Anchored Maximum Communities over Large Directed Graphs (Yang Huang, Hunan university et al.) - User engagement is a powerful tool that analyzes the expansion or unraveling of social networks. There have been many researches on user engagement to anchor critical users for enhancing engagement. However, these researches neglect the inherent directed nature of real-world social networks, such as the unidirectional follower relationships on platforms like X (Twitter). Motivated by this, we introduce the Anchored (k, l)-Core Maximization (ADCM) problem over directed graphs for the first time. Given a directed graph G, degree constraints k and l, and a budget b, the goal is to find b vertices in G, whose sustained engagement can maximize the (k, l)-core. We prove the NP-hardness of the ADCM problem and propose the Greedy-based algorithm (Greedy-based) to process it effectively. After that, to mitigate the isolation effect caused by greediness, a vertex scoring function is designed to support core expansion. To boost the computational performance, we develop pruning techniques, candidate reduction strategies, and an upper-bound-based termination criterion, based on which we design the Fast Anchor D-core (FAD) algorithm. Extensive experiments on nine real-world datasets demonstrate the effectiveness and efficiency of our proposed algorithms. From experimental results, FAD achieves over 1000× speedup compared to Greedy-based when b > 3. • MDS-FSM: Coverage-Based Frequent Subgraph Mining in Single Graphs (Xiaozhen Guo, Tianjin University et al.) - Frequent subgraph mining (FSM) in a single large graph remains challenging because pervasive embedding overlap exposes a gap between rigor and tractability: MIS-style supports enforce strict de-duplication but are NP-hard and enumeration-dependent, whereas MNI-style supports are polynomial-time yet systematically inflate frequency under distributed overlap. We propose Minimum Density Support (MDS), a coverage-based measure that minimizes coverage density over vertex subsets, uniformly penalizes redundant overlap, and preserves anti-monotonicity. MDS is theoretically bounded between MIS and MNI and can be computed in polynomial time via submodular minimization. We further develop MDS-FSM with orbit compression, separability, and progressive bound tightening to avoid exhaustive embedding enumeration. Experiments on six real graphs show that MDS reduces overestimation and cross-topology estimation bias while scaling to million-node graphs. • Breaking Structural Isolation: Scalable Graph Clustering via Community-Aware Sampling and Structural Entropy (Jingyun Zhang, Beihang University et al.) - Unsupervised graph clustering is a fundamental technique for uncovering underlying semantic patterns in large-scale networks. Although Graph Contrastive Learning has demonstrated promising performance, existing methods often suffer from the "structural isolation" issue during mini-batch training, making it challenging to capture cohesive community structures that characterize the global topological distribution. To address these challenges, we propose SCISE, a Scalable unsupervised graph Clustering framework that preserves structural Integrity by synergizing community-aware sampling with constrained Structural Entropy. Specifically, we first introduce the Structural Entropy Community Constraint operator (SECC), which optimizes structural information within a constrained solution space to mitigate community fragmentation and enhance partition cohesion. Second, to prevent global information loss during batch training, we design a Community-Aware Sampling Expansion (CSampE) mechanism that incorporates the community context of target nodes into sampling batches, effectively breaking structural barriers and preserving topological integrity. Finally, we devise a Structural Contrastive Learning (StructCL) module that refines edge weights based on intra-batch structural similarity, guiding the encoder to learn representations in a higher-order structural space. Extensive experiments on six mainstream benchmark datasets demonstrate that SCISE significantly outperforms state-of-the-art algorithms, with ablation studies and robustness analyses further validating its effectiveness and reliability for real-world large-scale graphs. • Sparse Neighborhood Graph-Based Approximate Nearest Neighbor Search Revisited: Theoretical Analysis and Optimization (Xinran Ma, Academy of Mathematics and Systems Science, Chinese Academy of Sciences et al.) - Graph-based approaches to approximate nearest neighbor search (ANNS) enable fast, high-recall retrieval on billion-scale vector datasets. Among them, the Sparse Neighborhood Graph (SNG) is widely used due to its strong search performance. However, the lack of theoretical understanding of SNG leads to expensive tuning of the truncation parameter that controls graph sparsification. In this work, we present OPT-SNG, a principled framework for analyzing and optimizing SNG construction. We introduce a martingale-based model of the pruning process that characterizes the stochastic evolution of candidate sets during graph construction. Using this framework, we prove that SNG has a maximum out-degree of \(O(n^{2/3+\epsilon})\), where \(\epsilon>0\) is an arbitrarily small constant, and an expected search path length of \(O(\log n)\). Building on these insights, we derive a closed-form rule for selecting the optimal truncation parameter \(R\), thereby eliminating the need for costly parameter sweeping. Extensive experiments on real-world datasets demonstrate that OPT-SNG achieves an average \(5.9\times\) speedup in index construction time, with peak improvements reaching \(15.4\times\), while consistently maintaining or improving search performance.read more
☆
Wed Sep 2 · 1:45 PM - 3:15 PM
Tutorial 7 · An Extended Tutorial and Vocabulary for Relational Language Design in an Era of AI-Assisted Query Generation
● Tutorials
📍 Stone
An Extended Tutorial and Vocabulary for Relational Language Design in an Era of AI-Assisted Query Generation
Wolfgang GatterbauerNortheastern University
☆
Wed Sep 2 · 1:45 PM - 3:15 PM
Tutorial 8 · How Can Quantum Computing and Databases Meet "in Practice"? From Algorithms to Systems
● Tutorials
📍 Webster
How Can Quantum Computing and Databases Meet "in Practice"? From Algorithms to Systems
Hanwen LiuUniversity of Southern California
Yiran LiUniversity of Toronto
Ibrahim SabekUniversity of Southern California
Xuanhe ZhouShanghai Jiao Tong University
Hans-Arno JacobsenUniversity of Toronto
☆
Wed Sep 2 · 1:45 PM - 3:15 PM
Industry Session 5 · Storage, Indexing & I/O Governance
● Industry
📍 Marina Ballroom I
Overlay Bitmap Encoding for Efficient Consumption of Apache Parquet Files (Gunes Aluc, SAP Labs et al.) - Apache Parquet has significantly transformed big data processing with its efficient columnar storage capabilities, incorporating techniques such as the Partition Attributes Across (PAX)...Overlay Bitmap Encoding for Efficient Consumption of Apache Parquet Files (Gunes Aluc, SAP Labs et al.) - Apache Parquet has significantly transformed big data processing with its efficient columnar storage capabilities, incorporating techniques such as the Partition Attributes Across (PAX) layout and dictionary-based encoding to enhance data compression and query performance. Despite these advantages, Apache Parquet's limited indexing capabilities can lead to inefficiencies when users need to fetch only a subset of the data. This paper proposes an innovative extension to Apache Parquet by introducing a new encoding format where indexes form the foundational physical representation of a column. Specifically, we introduce \emph{overlay bitmaps}, a data structure that organizes column chunks into sequences of bitmap parts, each representing distinct column values that are further compressed using run-length encoding. Our implementation within the Apache Parquet C\texttt{++} libraries and integration into the SAP IQ relational engine demonstrates the dual use of overlay bitmaps as both a storage and indexing mechanism, significantly improving predicate evaluation speed and compression efficiency. Experimental evaluations highlight that overlay bitmap encoding surpasses traditional dictionary encoding in compression and reduces load execution times by up to two orders of magnitude by enabling efficient, row-level, predicate pushdown. This capability allows complex queries, which involve numerous joins and predicates, to be executed up to 10x faster. • Enhancing Database Write Performance with the Write Operation Pushdown Framework (Hao Wu, Bytedance.com et al.) - While ByteDance’s database infrastructure continues to serve bursty user traffic, emerging large-scale interactive applications generate increasingly write-intensive workloads spanning all stages of the data lifecycle. In production environments, we observe that these workloads are consistently bottlenecked by a common write-path inefficiency rooted in cache-miss-induced remote page fetches. In compute-storage disaggregated architectures, such remote page accesses introduce substantial I/O latency that dominates write performance. To address this bottleneck, we propose Write Operation Pushdown (WOP), a framework that eliminates Fetch-Before-Write (FBW) from the write path by pushing eligible writes to the storage layer for asynchronous execution. WOP leverages the redo-flush path to push down write operations and enhances the storage layer for semantic-aware execution. WOP supports high pushdown capacity with minimal memory footprint through Lightweight Metadata Tracking. WOP preserves correctness and consistency in distributed deployments via carefully designed mechanisms. To prevent read stalls introduced by asynchronous execution, WOP integrates a Fetch-After-Write (FAW) optimization with localized, on-demand replay. Across production workloads, WOP achieves up to 3.33× throughput improvement. On synthetic benchmarks, WOP improves throughput by up to 6.7× and reduces average latency by up to 85.2%. These results show that WOP significantly enhances write efficiency in industrial-scale workloads while preserving correctness in disaggregated deployments. • IORM: Hierarchical I/O Governance for Thousands of Consolidated Databases on Oracle Exadata (Rajarshi Chowdhury, Oracle America Inc et al.) - Oracle Exadata consolidates thousands of tenant databases onto shared storage infrastructure deployed at hundreds of customer sites worldwide. Oracle Multitenant architecture enables this extreme density, with thousands of tenant databases sharing a single Exadata storage system - but this creates a multi-level resource hierarchy (container databases, tenant databases, and workloads within tenants) that commodity block-layer schedulers cannot govern, as they lack visibility into database semantics and tenant boundaries.This paper presents the I/O Resource Manager (IORM), a storage-side scheduler built on three mechanisms: I/O Tagging, which propagates semantic context from the database kernel to the storage scheduler; Hierarchical Resource Profiles, which express compositional allocation policies across consolidation tiers using shares and limits; and Unified Storage Governance, which applies these policies consistently across all tiers of the storage hierarchy - persistent memory, flash, and hard disk - including cache placement decisions. IORM enables successful cloud deployments where thousands of tenants coexist on shared storage: production OLTP workloads run alongside concurrent analytical workloads from the same or different databases without noisy-neighbor interference. Evaluation on production Exadata systems demonstrates that IORM dramatically improves latency consistency, virtually eliminating tail latency outliers and delivering order-of-magnitude improvements in average read latency under mixed workloads. Hierarchical limits compose correctly across all three levels, and proportional share allocation tracks configured ratios closely even under highly skewed demand. • Tuning the Lookahead Distance for PostgreSQL Asynchronous IO (Wentao Wu, Microsoft Research et al.) - PostgreSQL (PG) recently introduced support for asynchronous IOs (PG-AIO). Under the hood, the PG-AIO subsystem adopts a classic producer/consumer model. The producer submits AIO requests to a global queue resident in shared memory, and the consumer, which by default includes a number of “IO workers,” grabs the AIO requests from the queue and performs the actual IO work. To hide the implementation details of this producer/consumer model, PG offers a “read stream” abstraction with a set of API functions for table access methods, such as sequential scan and bitmap heap scan, that rely on the producer to send in AIO requests. Although this streaming API significantly simplifies the implementation of the access methods, it poses a new challenge that is performance critical: every read stream needs to decide the “lookahead distance,” i.e., the number of AIO requests to submit at the same time. In this paper, we address this challenge by proposing an adaptive approach to automatically adjust the lookahead distance by only leveraging feedback information based on the IO completion time. Our approach is inspired by and draws a connection to the theory of optimal flow control in computer networking, which has a profound impact on the design of modern TCP congestion control mechanisms such as BBR. Experimental evaluation with a variety of real-world setups shows that our approach can often achieve the optimal lookahead distance that minimizes query latency. • The Evolution of Storage in Apache AsterixDB (Murtadha Al Hubail, Couchbase, Inc. et al.) - This paper describes the evolution and current state of the storage subsystem of Apache AsterixDB. An initial shared-nothing version of AsterixDB, with an LSM-based storage system, was open-sourced in 2013 after four years of R\&D; its storage system was described in a 2014 VLDB paper. In the dozen years since then, much has changed in the computing environment, including the move from magnetic storage to SSDs, the migration of database services to the cloud, and the availability and popularity of object storage. In this paper, we describe how AsterixDB storage has evolved in response, including the addition of column-oriented JSON storage, compute-storage separation with object store backing, and resource management policies targeting multi-user workloads over databases with many collections and indexes. Apache AsterixDB's storage system is also the storage system for the Capella Analytics and Enterprise Analytics product offerings from Couchbase, Inc. • Clock2Q+: A Simple and Efficient Replacement Algorithm for Metadata Cache in VMware vSAN (Yiyan Zhai, Carnegie Mellon University et al.) - Cache replacement is central to data system performance, yet metadata caches exhibit access patterns that differ substantially from data caches. In particular, metadata blocks often exhibit dense bursts of correlated references (e.g., multiple lookups that touch tuples that co-reside on the same metadata page). Treating such bursts as repeated independent hits can misclassify cold metadata as hot, polluting the cache and degrading hit ratio. We present Clock2Q+, a cache eviction algorithm deployed in the metadata cache of VMware vSAN, a production-grade distributed storage system. Clock2Q+ is the latest stage of nearly a decade of iterative evolution of vSAN's caching layer, shaped by real-world deployment, customer-driven feature requests, and an emphasis on simplicity as a first-class requirement for long-term maintainability. Clock2Q+ has a three-queue structure similar to S3-FIFO, and introduces a correlation window at the head of the Small FIFO in which accesses do not set the reference bit, preventing short-term correlated bursts from triggering premature promotion; accesses beyond this window set the bit so genuinely hot blocks are promoted efficiently. Clock2Q+ achieves the highest median and mean miss-ratio improvement among the state-of-the-art policies we evaluate on both metadata and data workloads, with the largest per-trace gap over S3-FIFO reaching a 28.5% relative miss-ratio reduction on metadata traces. To support production deployment, Clock2Q+ is engineered for low per-hit overhead and multi-core scalability, with allocation-free, array-based structures, fine-grained synchronization, dirty-page-aware eviction and promotion, bounded per-eviction work, and online cache resizing.read more
☆
Wed Sep 2 · 1:45 PM - 3:15 PM
Panel 2 · Publish or Ship? The Research-to-Production Gap in Data Systems
● Panels
📍 Grand Ballroom A+B
Publish or Ship? The Research-to-Production Gap in Data Systems
Subarna ChatterjeeIBM WatsonX
Natacha CrooksUC Berkeley
Carlo CurinoMicrosoft Gray Systems Lab
Sudipto DasDatabricks
Andreas KipfUniversity of Technology Nuremberg
Ryan MarcusUniversity of Pennsylvania
Yuanyuan TianMicrosoft Gray Systems Lab
☆
Wed Sep 2 · 1:45 PM - 3:15 PM
Demo Group C · Data Quality, Analytics & Applications
● Demos
📍 Marina Ballrooms II - IV
32 demos in this group, shown twice during the conference.32 demos in this group, shown twice during the conference. Includes: ImputePilot: A Graphical Model Selection Toolkit for Time Series Imputation • LLM-CER: An Interactive System for In-context Clustering-based Entity Resolution with Large Language Models • ECO-Hadoop: Energy Consumption Optimization System for Apache Hadoop • QueryCraft: A Natural Language-Driven NoSQL Database Querying System Powered by Large Language Models • ChatQPT: Towards Conversing with Relational Query Engines • LIMA: Denial Constraint Discovery in Large Dynamic Databases • DA-Studio: An Agentic System for End-to-End Data Analysis • Provlepsis4j: Querying Future Graphs in Neo4j • GALACTICA: An Interactive System for Local and Global Counterfactual Explanations for Time-series Clustering • The Case for Multi-Version Experimental Evaluation (MVEE) • … +22 moreread more
☆
Wed Sep 2 · 1:45 PM - 3:15 PM
Sponsor Talks 5
● Sponsor Talks
📍 Otis
Reimagining Databases for Agents (Aditya Parameswaran, Snowflake) • The Intelligence Equation: Everyone Has the Model, Nobody Has Your Context (Dattaraj Rao, Persistent) • Documents as Materialized Views: Open Challenges in Schema Optimization at MongoDB (Steve Tarzia, MongoDB) • Disrupting Data...Reimagining Databases for Agents (Aditya Parameswaran, Snowflake) • The Intelligence Equation: Everyone Has the Model, Nobody Has Your Context (Dattaraj Rao, Persistent) • Documents as Materialized Views: Open Challenges in Schema Optimization at MongoDB (Steve Tarzia, MongoDB) • Disrupting Data Management with Acceleration, Autonomy and Agent Services (Lisa Amini, IBM) • Transwarp GPU-Native Cognitive Database: A Unified Infrastructure for Enterprise Data and AI Workloads (Wang Bin, Transwarp)read more
Built for you by(617)-PATRICK, Champion AI speaker
Efficient GNN Training on Giant Graphs with Collective Batching and Scheduling (Xin Zhang, Hong Kong University of Science and Technology et al.) - Graph Neural Networks (GNNs) have achieved great success in many applications, and mini-batch training has become the...Efficient GNN Training on Giant Graphs with Collective Batching and Scheduling (Xin Zhang, Hong Kong University of Science and Technology et al.) - Graph Neural Networks (GNNs) have achieved great success in many applications, and mini-batch training has become the de facto standard for training GNNs on large-scale graphs. When developing mini-batch GNN training systems on CPU-GPU platforms, existing dedicated batching systems adopt a static workload-processor binding strategy, where the costly mini-batch preparation workload is exclusively assigned to either the CPU or the GPU. This leads to low utilization of CPU cores, PCIe bandwidth, and GPU computing capability, resulting in suboptimal training efficiency. To address this problem, we develop MorphGL, a novel GNN training system featuring a collective batching design. MorphGL adaptively dispatches the mini-batch preparation workload to both the CPU and GPU, ensuring that the workload distribution aligns with the CPU-GPU setup of the running machine for optimal efficiency. To maximize resource utilization, MorphGL employs the Dual-Buffer Scheduling algorithm to collectively schedule training stages across the CPU, PCIe, and GPU. Extensive experiments on three large real-world graphs with billions of edges and four machines with representative CPU-GPU configurations demonstrate that MorphGL consistently outperforms state-of-the-art GNN training systems, achieving up to 2.76x and 2.2x speedup over SALIENT and DUCATI, respectively. • FeLoG: Scalable and Efficient Distributed Graph Embedding with Feedback Loop Mechanism (Peng Fang, Huazhong University of Science and Technology et al.) - Graph embedding maps graph nodes into low-dimensional vectors to support applications such as recommendation, fraud detection, and graph-based retrieval-augmented generation (GraphRAG). As graphs scale to billions of edges, scalable and efficient graph embedding has become increasingly important. Existing frameworks commonly adopt a sampling-training paradigm, in which mini-batches are constructed by sampling nodes and their neighbors. However, sampling is typically decoupled from evolving embedding quality, causing redundant exploration of well-trained regions while under-sampling undertrained nodes. At the system level, such decoupling further leads to excessive communication, serialized execution, and low resource utilization in distributed environments. We present FeLoG, a feedback loop-driven system for scalable distributed graph embedding. (1) FeLoG introduces feedback-coupled sampling and training, dynamically prioritizing undertrained nodes according to real-time embedding-quality feedback, thereby reducing redundant computation and accelerating convergence. (2) It employs activity-aware communication that compresses frequently occurring node sequences to reduce intra-machine PCIe traffic and selectively synchronizes frequently updated embeddings to reduce inter-machine communication. (3) It adopts a round-interleaved pipeline that overlaps next-round sampling with current-round training to improve CPU-GPU utilization. Experiments against six state-of-the-art baselines on large-scale graphs show that FeLoG achieves an average speedup of 27.9×, reduces communication cost by more than 53.1%, and sustains over 80% CPU-GPU utilization. • Resource-Efficient FirmCore Decomposition on Billion-scale Multilayer Graphs (Cheng Huang, Aarhus University et al.) - Multilayer (ML) graphs offer a convenient paradigm for modeling complex node-to-node interactions, such as social or semantic connections, as layers of a graph. In such graphs, FirmCore decomposition represents an established technique to identify cohesive groups of nodes with strong ties across layers. Unfortunately, the fastest FirmCore decomposition method fails to fully harness the resources, leading to underutilized and idle threads. Our main observation is that FirmCores enjoy a grid structure we call FC-Grid, which we exploit to distribute work among threads. Building on this structure, we introduce serial and parallel algorithms for multi-core CPUs, as well as the first GPU-based algorithm. Owing to this new design, our solutions show greatly improved performance and resource utilization. Our experiments on 12 datasets show 9× speedup on average for our serial version FC-Grid compared to existing serial methods. Furthermore, our parallel algorithm achieves an average 100.3× speedup over the state-of-the-art parallel algorithm. For the challenging NP-hard densest subgraph mining problem in ML graphs, our algorithms achieve 15× speedup on average. • Scalable GNN Explanations with Distributed Shapley Values (Selahattin Akkas, Indiana University et al.) - With the growing adoption of graph neural networks (GNNs), explaining their predictions has become increasingly important. However, attributing predictions to specific edges or features remains computationally expensive. For example, classifying a node with 100 neighbors using a 3-layer GNN may involve identifying important edges from millions of candidate subgraphs. To address this challenge, we develop DistShap, a parallel algorithm that distributes Shapley value-based explanations across multiple GPUs. DistShap samples subgraphs in a distributed setting, executes GNN inference in parallel across GPUs, and solves a distributed least squares problem to compute edge importance scores. DistShap outperforms most existing GNN explanation methods in accuracy and is the first to scale to GNN models with millions of edges by using 128 GPUs. • Towards A Generalizable and Expressive Graph Neural Network for Graph-Level Tasks with Theoretical Guarantees (Haoyang Li, The Hong Kong Polytechnic University et al.) - Graph Neural Networks (GNNs) have become essential for solving graph-level tasks, such as classification and regression, across diverse domains including social networks and biology. However, existing GNNs struggle with the expressivity that captures complex structural patterns, and the generalization that ensures robust performance on diverse and noisy datasets. To address these challenges, we propose a novel GNN model that integrates a k-path rooted subgraph encoder, an adaptive graph contrastive learning approach, and a consistency-aware loss. The k-path rooted subgraph encoder enhances expressivity by capturing and distinguishing intricate substructures, with theoretical guarantees for counting paths and cycles. The adaptive graph contrastive learning framework improves generalization by generating domain-aware graph augmentations based on edge importance, while the consistency-aware loss ensures task-relevant properties are preserved across augmented views. Extensive experiments on 26 datasets spanning graph classification, regression, and realistic scenarios such as noise, class imbalance, and few-shot learning show that our model achieves superior performance against 18 state-of-the-art GNN models in both effectiveness and efficiency. The code is released in https://anonymous.4open.science/r/GEGNN.read more
☆
Wed Sep 2 · 3:45 PM - 5:15 PM
Research 32 · Learned Database Tuning
● Research
📍 Grand Ballroom D
DOT: Dynamic Knob Selection and Online Sampling for Automated Database Tuning (yifan wang, Orange / INRIA/ Univ.DOT: Dynamic Knob Selection and Online Sampling for Automated Database Tuning (yifan wang, Orange / INRIA/ Univ. Lille et al.) - Database Management Systems (DBMS) are crucial for efficient data management and access control, but their administration remains challenging for Database Administrators (DBAs). Tuning, in particular, is known to be difficult. Modern systems have many tuning parameters, but only a subset significantly impacts performance. Focusing on these influential parameters reduces the search space and optimizes performance. Current methods rely on costly warm-up phases and human expertise to identify important tuning parameters. In this paper, we present DOT, a dynamic knob selection and online sampling DBMS tuning algorithm. DOT uses Recursive Feature Elimination with Cross-Validation (RFECV) to prune low-importance tuning parameters and a Likelihood Ratio Test (LRT) strategy to balance exploration and exploitation. For parameter search, DOT uses a Bayesian Optimization (BO) algorithm to optimize configurations on-the-fly, eliminating the need for warm-up phases or prior knowledge (although existing knowledge can be incorporated). Experiments show that DOT achieves matching or outperforming performance compared to state-of-the-art tuners while substantially reducing tuning overhead. • Libra: One-Shot Parameter Sensitivity Estimation for Transfer Learning in Database Performance Prediction (Tatsuhiro Nakamori, Keio University et al.) - Accurate performance prediction is critical for database tuning, resource provisioning, and performance debugging. Recent work applies machine learning to predict DBMS performance, but these models often require expensive retraining when deployment contexts change. We present Libra, an end-to-end transfer learning framework that builds accurate performance models with minimal target-context sampling. Libra addresses two key challenges: (1) selecting source contexts based on performance-relevant similarity, and (2) leveraging source context data without negative transfer. We introduce a novel context retrieval method based on $\pi$-profiles, which capture parameter sensitivity. Libra uses a multilayer perceptron to infer the target $\pi$-profile in one-shot, and compares it with those of past contexts to retrieve the most similar one. Libra then selects important parameters based on percentile performance ratios and focuses sampling on high-impact parameters to efficiently train the model. Experiments across 161 contexts (combination of 7 hardware environments and 23 workloads) show that Libra outperforms state-of-the-art methods in terms of sampling efficiency (up to 32$\times$ speedup) and prediction accuracy (95.6\% error reduction). • MFTune: An Efficient Multi-fidelity Framework for Spark SQL Configuration Tuning (Beicheng Xu, Peking University et al.) - Apache Spark SQL is a cornerstone of modern big data analytics. However, optimizing Spark SQL performance is challenging due to its vast configuration space and the prohibitive cost of evaluating massive workloads. Existing tuning methods predominantly rely on full-fidelity evaluations, which are extremely time-consuming, often leading to suboptimal performance within practical budgets. While multi-fidelity optimization offers a potential solution, directly applying standard techniques - such as data volume reduction or early stopping - proves ineffective for Spark SQL as they fail to preserve performance correlations or represent true system bottlenecks. To address these challenges, we propose MFTune, an efficient multi-fidelity framework that introduces a query-based fidelity partitioning strategy, utilizing representative SQL subsets to provide accurate, low-cost proxies. To navigate the huge search space, MFTune incorporates a density-based optimization mechanism for automated knob and range compression, alongside an adapted transfer learning approach and a two-phase warm start to further accelerate the tuning process. Experimental results on TPC-H and TPC-DS benchmarks demonstrate that MFTune significantly outperforms five state-of-the-art tuning methods, identifying superior configurations within practical time constraints. • AXE: A Task Decomposition Approach to Learned LSM Tuning (Andy Huynh, Boston University et al.) - Log-Structured Merge (LSM) trees are used as the data structure of choice for key-value stores supporting a wide variety of applications. A common challenge for LSM-based systems is tuning them effectively, particularly as the complexity and number of tuning knobs increase. Prior work relies on expert-created cost models and expert-configured numerical solvers to produce high-quality tunings; however, these methods do not address tuning multiple instances at scale for various execution environments. On the other hand, using iterative learning, such as Bayesian Optimization (BO), relaxes the requirements for domain expertise and provides generalizability; however, it comes at a high cost, as it involves learning directly from database executions at deployment time. Furthermore, both approaches struggle with categorical tuning knobs that create a hard-to-navigate optimization space. To address these challenges, we introduce AXE, a novel learned LSM tuning paradigm that decomposes the tuning task into two steps. First, AXE trains a learned cost model using existing performance modeling or execution logs, acting as a surrogate cost function in the tuning process. Second, AXE efficiently generates arbitrarily many training samples for a learned tuner optimized to identify high-performance tunings using the learned cost model as its loss function. This task decomposition approach generalizes well for tuning simple and complex LSM designs and requires no retraining, allowing AXE to be used for tuning at scale. Compared to BO, AXE recommends higher performing tunings than BO 71% of the time while incurring 100× smaller tuning overhead. We further show that AXE requires less domain knowledge to produce optimal tunings than traditional expert-configured tuning pipelines. Lastly, we compare AXE to both state-of-the-art machine learning methods and analytical methods to show that AXE outperforms all other LSM tuning baselines. • DBAIOps: A Reasoning LLM-Enhanced Database Operation and Maintenance System using Knowledge Graphs (Wei Zhou, Shanghai Jiao Tong University et al.) - The operation and maintenance (O&M) of database systems is critical to ensuring system availability and performance, typically requiring expert experience (e.g., identifying metric-to-anomaly relations) for effective diagnosis and recovery. However, existing automatic database O&M methods, including commercial products, cannot effectively utilize expert experience. On the one hand, rule-based methods only support basic O&M tasks (e.g., metric-based anomaly detection), which are mostly numerical equations and cannot effectively incorporate literal O&M experience (e.g., troubleshooting guidance in manuals). On the other hand, LLM-based methods, which retrieve fragmented information (e.g., standard documents + RAG), often generate inaccurate or generic results. To address these limitations, we present DBAIOps, a novel hybrid database O&M system that combines reasoning LLMs with knowledge graphs to achieve DBA-style diagnosis. First, DBAIOps introduces a heterogeneous graph model for representing the diagnosis experience, and proposes a semi-automatic graph construction algorithm to build that graph from thousands of documents. Second, DBAIOps develops a collection of (800+) reusable anomaly models that identify both directly alerted metrics and implicitly correlated experience and metrics. Third, for any given anomaly, DBAIOps employs an automatic graph evolution mechanism that explores the relevant paths over the graph and dynamically explores potential gaps (missing paths) without human intervention. Based on the explored diagnosis paths, DBAIOps leverages reasoning LLM (e.g., DeepSeek-R1) that inputs the relevant pathways, identifies root causes, and generates clear diagnosis reports for both DBAs and common users. Our evaluation over four mainstream database systems (Oracle, MySQL, PostgreSQL, DM8) demonstrates that DBAIOps outperforms state-of-the-art baselines, 34.85% and 47.22% higher in root cause and human evaluation accuracy, respectively. DBAIOps supports 25 database systems and has been deployed in 20 real-world scenarios, covering domains like finance, energy, and healthcare (https:// www.dbaiops.com).read more
☆
Wed Sep 2 · 3:45 PM - 5:15 PM
Research 33 · Efficient LLM Systems
● Research
📍 Grand Ballroom C
RetroInfer: A Vector Storage Engine for Scalable Long-Context LLM Inference (Yaoqi Chen, University of Science and Technology of China et al.) - Recent large language models (LLMs) are rapidly extending their context windows, yet inference throughput lags due to increasing...RetroInfer: A Vector Storage Engine for Scalable Long-Context LLM Inference (Yaoqi Chen, University of Science and Technology of China et al.) - Recent large language models (LLMs) are rapidly extending their context windows, yet inference throughput lags due to increasing GPU memory and bandwidth demands. This is because the key-value (KV) cache, an intermediate structure storing token representations, grows linearly with context length and requires an iterative linear scan for attention computation. A promising direction to accelerate long context inference is to exploit attention's inherent sparsity by offloading the KV cache to CPU memory and retrieving only a small subset of tokens important to the current generation step. However, prior sparse attention approaches struggle to balance accuracy and retrieval cost due to varying sparsity patterns and inefficient GPU-CPU memory management. We present RetroInfer, a vector storage engine that realizes a sparsity-based KV cache for long-context inference. RetroInfer introduces an Attention aWare VEctor index (wave index) which fundamentally improves the tradeoff between attention accuracy and retrieval cost through tripartite attention approximation, accuracy-bound attention estimation, and segmented clustering. We also design the wave buffer, a GPU-CPU buffer manager that assigns computation and manages data across heterogeneous hardware. We evaluate RetroInfer across a range of models and workloads, demonstrating up to 4.4x higher decoding throughput over full attention at 120K context and up to 12.2x over sparse attention baselines at 1 million tokens - all while preserving full-attention-level accuracy. • OrbitFlow: SLO-Aware Long-Context LLM Serving with Fine-Grained KV Cache Reconfiguration (Xinyue Ma, POSTECH et al.) - Serving long-context LLMs is challenging because request lengths and batch composition vary during token generation, causing the memory footprint to fluctuate significantly at runtime. Offloading KV caches to host memory limits effective memory usage, but existing static and predetermined offloading strategies cannot adapt to the rapidly shifting memory demands of long-context serving. This often leads to excessive CPU-to-GPU KV transfers that translate into latency spikes and frequent SLO violations. To address these challenges, we introduce OrbitFlow, a fine-grained and adaptive KV cache management system that meets latency SLOs in long-context LLM serving. OrbitFlow employs a lightweight ILP solver to decide which layers’ KV caches to retain on the GPU for each request, within memory capacity constraints. It continuously refines KV placements based on runtime feedback when the active plan becomes suboptimal during token generation. Under heavy load, OrbitFlow invokes a fallback mechanism to temporarily defer in-flight requests with large memory footprints, preserving overall SLO attainment. Our experiments demonstrate that OrbitFlow improves SLO attainment for TPOT and TBT by up to 66% and 48%, respectively, while reducing the 95th percentile latency by 38% and achieving up to 3.3× higher throughput compared to existing offloading methods. • Efficient Cooperation-Aware Key and Value Management for LLM Inference (Qiheng Sun, Zhejiang University et al.) - Key-value (KV) caching is a widely used technique for boosting performance in database and storage systems. It keeps frequently accessed data in fast storage to minimize redundant data fetching and improve throughput. This same idea has been adopted in Large Language Models (LLMs), where it avoids recomputing the key and value states of previous tokens in attention heads during autoregressive decoding, thereby greatly accelerating inference. However, the KV cache in LLMs faces a significant challenge due to the substantial memory required to store these KV pairs in the inference process. This issue arises because each attention head in the LLM stores its own KV cache for all context tokens, leading to the cache size that grows linearly with sequence length. This has spurred research into efficient management of the KV cache of LLMs. One of the promising directions is KV cache budget allocation, with several approaches proposing head-level allocation as they recognize that different attention heads play distinct roles. However, these methods assess each head in isolation, overlooking their cooperative contributions within the model, which results in a deviation from their true impact. To address this limitation, we propose CoKV, a novel method that efficiently manages the KV cache in LLM inference by modeling the cooperation among attention heads as a cooperative game. By attributing the contribution of each head within the model in advance, CoKV can more effectively allocate the global KV cache budget in KV cache optimization techniques such as eviction and quantization. Extensive experiments demonstrate the effectiveness of CoKV on long-context benchmarks (e.g., LongBench, NIAH, and RULER) and mathematical reasoning benchmarks (e.g., GSM8K and MATH) across multiple model families, including Qwen, Llama, and Mistral. • Unified Static - Dynamic Pruning for Efficient LLM Inference (Jinhyeok Kim, Seoul National University et al.) - The increasing deployment of large language models (LLMs) has magnified the computational and memory bottlenecks of autoregressive decoding, where low arithmetic intensity and bandwidth-bound kernels dominate inference cost. Weight pruning offers a promising remedy, but existing methods remain confined to either static pruning (SP) - which permanently removes redundant weights but lacks adaptivity - or dynamic pruning (DP) - which adapts to input sparsity but introduces runtime irregularity. This paper presents SPDP, a unified sparse-inference framework that integrates unstructured SP with input-adaptive DP for efficient LLM inference on GPUs. SPDP co-designs a new Tiled-Column-wise Bitmap Compressed (Tiled-CBC) format and two complementary GPU kernels: (1) a CUDA-core spMspV kernel featuring Hybrid Activation-aware Dynamic Shared-Memory Bitmap Decoding (HAD-SMBD) for fine-grained, runtime activation skipping, and (2) a Tensor-Core SpMM kernel optimized for prefill computation. This joint format - kernel design harmonizes static and dynamic sparsity, maintaining bandwidth-efficient memory access and high arithmetic intensity under both phases of LLM inference. Comprehensive evaluations on inference-optimized GPUs demonstrate that SPDP achieves 1.24× - 1.37× average speedup (up to 2.51×) over state-of-the-art sparse frameworks such as SpInfer, while matching perplexity with up to 25\% higher sparsity. SPDP advances the inference efficiency - quality Pareto frontier, showing that unified static - dynamic pruning can deliver substantial throughput and performance-per-watt improvements in large-scale LLM serving. • Compass: SLO-aware Query Planner for Compound AI Serving at Scale (Banruo Liu Liu, University of Illinois Urbana Champaign et al.) - The rise of compound AI serving that integrates multiple operators in a pipeline enables end-user applications such as generative AI- powered meeting companions, autonomous driving, and immersive gaming. These workloads span diverse deployment spaces, from cloud-only queries to edge-assisted ones across infrastructure tiers, often including both within an application. Achieving high service goodput - i.e., meeting service level objectives (SLOs) for pipeline latency, accuracy, and costs-requires joint planning of operators’ placement, configuration, and resource allocation. However, diverse SLOs, varying runtime environments (e.g., heterogeneous device speeds), and a large volume of queries competing for shared infrastructure explode the planning space, making real-time serving and cost-efficient deployment intractable with existing advances. This paper presents Compass, the first SLO-aware query planner that optimizes large-scale compound AI workloads across diverse deployment spaces. Compass decomposes the many-query, multi-SLO planning problem into tractable subproblems while preserving global decision quality, exploiting plan similarities within and across queries to slash the search steps. It further improves per-step efficiency with a plan profiler that performs selective profiling to achieve high-fidelity performance estimates at a fraction of the profiling cost. At runtime, Compass performs query-plan bipartite matching to maximize SLO goodput under resource contentions. Real-world evaluations show that Compass improves service goodput by 2.4 - 5.1x, reduces deployment costs by 3.8 - 4.5x, and accelerates planning by 4.2 - 10.5x, achieving service responsiveness within seconds and near-optimal decision quality. • QStore: Quantization-Aware Compressed Model Storage (Raunak Shah, University of Illinois at Urbana-Champaign et al.) - Modern applications commonly leverage large, multi-modal foundation models. These applications often feature complex workflows that demand the storage and usage of similar models in multiple precisions. A straightforward approach is to maintain a separate file for each model precision (e.g., INT8, BF16), which is indeed the approach taken by many model providers such as HuggingFace and Ollama. However, this approach incurs excessive storage costs since a higher precision model (e.g., BF16) is a strict superset of a lower precision model (e.g., INT8) in terms of information. Unfortunately, simply maintaining only the higher-precision model and requiring every user to dynamically convert the model precision is not desirable because every user of lower precision models must pay the cost for model download and precision conversion. In this paper, we present QStore, a unified, lossless compression format for simultaneously storing a model in two (high and low) precisions efficiently. Instead of storing low-precision and high-precision models separately, QStore stores low-precision model and only the residual information needed to reconstruct high-precision models. The size of residual information is significantly smaller than the original high-precision models, thus, achieving high savings in storage cost. Moreover, Qstore does not compromise the speed of model loading. The low-precision models can be loaded quickly just like before. The high-precision models can also be reconstructed efficiently in memory by merging low-precision data and the residual with Qstore's lightweight decoding logic. We evaluate Qstore for compressing multiple precisions of popular foundation models, and show that Qstore reduces overall storage footprint by up to 2.2x (45% of the original size) while enabling up to 1.7x and 1.8x faster model saving and loading versus existing approaches.read more
☆
Wed Sep 2 · 3:45 PM - 5:15 PM
Research 34 · Temporal Graph Analytics
● Research
📍 Commonwealth Ballroom C
TIMEST: Temporal Information Motif Estimator Using Sampling Trees (Yunjie Pan, University of Michigan et al.) - The mining of pattern subgraphs, known as motifs, is a core task in the field of graph mining.TIMEST: Temporal Information Motif Estimator Using Sampling Trees (Yunjie Pan, University of Michigan et al.) - The mining of pattern subgraphs, known as motifs, is a core task in the field of graph mining. Edges in real-world networks often have timestamps, so there is a need for temporal motif mining. A temporal motif is a richer structure that imposes timing constraints on the edges of the motif. Temporal motifs have used to analyze social networks, financial transactions, and biological networks. Motif counting in temporal graphs is particularly challenging. A graph with millions of edges can have trillions of temporal motifs, since the same edge can occur with multiple timestamps. There is a combinatorial explosion of possibilities, and state-of-the-art algorithms cannot manage motifs with more than four vertices. In this work, we present TIMEST: a general, fast, and accurate estimation algorithm to count temporal motifs of arbitrary sizes in temporal networks. Our approach introduces a temporal spanning tree sampler that leverages weighted sampling to generate substructures of target temporal motifs. This method carefully takes a subset of temporal constraints of the motif that can be jointly and efficiently sampled. TIMEST uses randomized estimation techniques to obtain accurate estimates of motif counts. We give theoretical guarantees on the running time and approximation guarantees of TIMEST. We perform an extensive experimental evaluation and show that TIMEST is both faster and more accurate than previous algorithms. Our CPU implementation exhibits an average speedup of 28X over state-of-the-art GPU implementation of the exact algorithm, and 6X speedup over SOTA approximate algorithms while consistently showcasing less than 5% error in most cases. For example, TIMEST can count the number of instances of a financial fraud temporal motif on a graph with about 10M edges in four minutes with 0.6% error, while exact methods take more than two days. • Mayura: Exploiting Similarities in Motifs for Temporal Co-Mining (Sanjay Sri Vallabh Singapuram, University of Michigan, Ann Arbor et al.) - Temporal graphs serve as a critical foundation for modeling evolving interactions in domains ranging from financial networks to social media. Mining temporal motifs is essential for applications such as fraud detection, cybersecurity, and dynamic network analysis. However, conventional motif mining approaches treat each query independently, incurring significant redundant computations when similar substructures exist across multiple motifs. In this paper, we propose Mayura, a novel framework that unifies the mining of multiple temporal motifs by exploiting their inherent structural and temporal commonalities. Central to our approach is the Motif-Group Tree (MG-Tree), a hierarchical data structure that organizes related motifs and enables the reuse of common search paths, thereby reducing redundant computation. We propose a co-mining algorithm that leverages the MG-Tree and develop a flexible runtime capable of exploiting both CPU and GPU architectures for scalable performance. Empirical evaluations on diverse real-world datasets demonstrate that Mayura achieves substantial improvements over the state-of-the-art techniques that mine each motif individually, with an average speed-up of 2.4x on the CPU and 1.7x on the GPU, while maintaining the exactness required for high-stakes applications. • Worst-Case Optimal BGPs on Temporal Graphs (Diego Arroyuelo, Universidad Católica de Chile et al.) - We study how to evaluate basic graph patterns (BGPs) over temporal labeled graphs--where each edge is associated with an interval of temporal validity--in a worst-case-optimal manner. We adopt a flexible query language in which users specify m quads of the form (subject, property, object, time), using constants or variables. The time component denotes the instant at which a particular edge is valid, and users may also include order relations between temporal constants or variables. The answer to such extended BGPs is the set of all valid variable assignments, including those that involve time. We describe an index structure that, for a temporal graph with N edges, requires O(N) space and can evaluate extended BGPs in worst-case optimal time O(Q* m log N), where Q* represents the maximum number of solutions for query Q over any temporal graph with the same number of instants of edge validities. We use our index to adapt Leapfrog Triejoin to the temporal graph setting and can therefore consider any variable evaluation ordering. Moreover, the same index yields worst-case-optimal guarantees for related query types, including snapshot evaluation, version queries, and other temporal variants. Experiments on real-world datasets show that our approach answers realistic queries in milliseconds with minimal space overhead. • PRISM: A Training System to Unlock the Potential of Temporal Graph Learning Through Staleness Avoidance (Md Ashraful Islam, University of Massachusetts Amherst et al.) - Training memory-augmented Temporal Graph Neural Networks (M-TGNNs) efficiently and accurately remains challenging due to memory staleness, which arises when temporally dependent events are processed in the same batch and severely degrades accuracy at large batch sizes. We introduce PRISM, an M-TGNN training system that achieves staleness-freedom without giving up GPU parallelism by using multi-versioned memory vectors, so that each event in a batch can consume the memory version that is temporally consistent for it. PRISM formalizes a relaxed notion of staleness-freedom called lazy freshness, which allows for more parallelism than existing staleness-free approaches, and implements it through a multi-versioned memory refinement algorithm over a lightweight memory computation graph. On five temporal-graph benchmarks and three M-TGNN models (TGN, TNCN, APAN), PRISM improves the accuracy of existing models by up to 28% and surpasses the TGB leaderboard by 9.2%, while keeping training time competitive with parallel stale-memory systems (TGL, ETC) and consistently lower than stricter staleness-free baselines. PRISM thus provides a practical, staleness-free foundation for temporal graph learning. • AGIS: Fast Approximate Graph Pattern Mining with Structure-Informed Sampling (Seoyong Lee, Seoul National University et al.) - Approximate Graph Pattern Mining (AGPM) is essential for analyzing large-scale graphs where exact counting is computationally prohibitive. While there exist numerous sampling-based AGPM systems, they all rely on uniform sampling and overlook the underlying probability distribution. This limitation restricts their scalability to a broader range of patterns. In this paper, we introduce AGIS, an extremely fast AGPM system capable of counting arbitrary patterns from huge graphs. AGIS employs structure-informed neighbor sampling, a novel sampling technique that deviates from uniformness but allocates specific sampling probabilities based on the pattern structure. We first derive the ideal sampling distribution for AGPM and then present a practical method to approximate it. Furthermore, we develop a method that balances convergence speed and computational overhead, determining when to use the approximated distribution. Experimental results demonstrate that AGIS significantly outperforms the state-of-the-art AGPM system, achieving 28.5x geometric mean speedup and more than 100,000x speedup in specific cases. Furthermore, AGIS is the only AGPM system that scales to graphs with tens of billions of edges and robustly handles diverse patterns, successfully providing accurate estimates within seconds. We will open-source AGIS to encourage further research in this field.read more
☆
Wed Sep 2 · 3:45 PM - 5:15 PM
Research 35 · Secure and Trusted Query Processing
● Research
📍 Commonwealth Ballroom B
Secure Join Operations in Multi-Identifier Databases: Performance and Practicality (Wen-jie Lu, TikTok et al.) - In this work, we present an efficient and cryptographically secure protocol for multi-key inner-join computation that addresses the limitations of existing approaches.Secure Join Operations in Multi-Identifier Databases: Performance and Practicality (Wen-jie Lu, TikTok et al.) - In this work, we present an efficient and cryptographically secure protocol for multi-key inner-join computation that addresses the limitations of existing approaches. Our protocol leverages established Circuit Private Set Intersection (PSI) techniques to privately compute left-joins over individual key columns. These results are then securely aggregated into a final inner-join table using a novel private permutation protocol, which achieves a speedup of approximately 2× to 4× over prior methods. To enhance utility without compromising privacy, we introduce a deduplication mechanism based on ordered left-joins, enabling first-key deduplication while revealing no sensitive matching information. We formally analyze the security of our construction in the semi-honest model. Furthermore, we optimize the equality testing subroutine, a core component of Circuit PSI, reducing its round complexity without an increase in computational overhead. Empirically, our system demonstrates strong scalability, processing up to 1.8 × 104 records of 4 keys per second per CPU core. This represents a significant improvement over industry solutions such as Google’s [24] and Meta’s [4], which are not only slower but also reveal more information about the input databases. • Secure Multi-Party Sampling over Joins (Qiyao Luo, OceanBase, Ant Group et al.) - Secure multi-party computation (MPC) enables collaborative analytics over private datasets but faces critical efficiency barriers. State-of-the-art MPC protocols for query processing with joins incur prohibitive computational costs. While sampling-based approximate query processing has revolutionized plaintext analytics, its extension to secure settings remains unexplored. This paper proposes the first efficient and secure protocol for sampling over joins. The protocol achieves near-linear asymptotic complexity while preserving the confidentiality of input and metadata (e.g., degree and join sizes). It supports a wide range of queries, including multi-way joins, comparisons, and group-by operations, and is universally applicable across secure computation settings. Experiments demonstrate significant speedups over secure join-then-sample baselines. This work bridges the gap between theoretical secure computation and practical relational analytics, advancing scalable real-world secure collaborative analytics and learning scenarios. • Enabling Index-free Adjacency in Oblivious Graph Processing with Delayed Duplications (Weiqi Feng, University of Massachusetts Amherst et al.) - Obliviousness has been regarded as an essential property in encrypted databases (EDBs) for mitigating leakage from access patterns. Yet despite decades of work, practical oblivious graph processing remains an open problem. In particular, all existing approaches fail to enable the design of index-free adjacency (IFA), i.e., each vertex preserves the physical positions of its neighbors. However, IFA has been widely recognized as necessary for efficient graph processing and is fundamental in native graph databases (e.g., Neo4j). In this work, we propose a core technique named delayed duplication to resolve the conflict between IFA and obliviousness. To the best of our knowledge, we are the first to address this conflict with both practicality and strict security. Based on the new technique, we utilize elaborate data structures to develop a new EDB named Grove for processing expressive graph queries. The experimental results demonstrate that incorporating IFA makes Grove impressively outperform the state-of-the-art work across multiple graph-processing tasks, such as the well-known neighbor query and t-hop query. • SACK: Shielding Dynamic Attribute-based Access Control in Persistent Key-Value Stores (Yanjing Ren, The Chinese University of Hong Kong et al.) - Enforcing fine-grained access control is critical for secure key-value (KV) stores in cloud environments, yet classical attribute-based encryption incurs significant overhead. We present SACK, a shielded framework leveraging Intel SGX to enable efficient, dynamic attribute-based access control (ABAC) for KV stores in untrusted cloud environments, while ensuring confidentiality, integrity, and freshness. SACK decouples access control and data management by performing ABAC with hardware-assisted shielded execution and leveraging KV separation for secure, efficient, and crash-consistent KV storage. We implement SACK as a middleware system that can run atop general KV stores. Experiments show that SACK achieves high-performance KV operations and lightweight renewal of access rights.read more
☆
Wed Sep 2 · 3:45 PM - 5:15 PM
Research 36 · Semi-Structured, Scientific, and Multimedia Data
● Research
📍 Commonwealth Ballroom A
Blaze: Compiling JSON Schema for 10x Faster Validation (Michael Mior, Rochester Institute of Technology et al.) - JSON Schemas provide useful guardrails for developers of Web APIs to guarantee that the semi-structured JSON input provided by clients matches a predefined...Blaze: Compiling JSON Schema for 10x Faster Validation (Michael Mior, Rochester Institute of Technology et al.) - JSON Schemas provide useful guardrails for developers of Web APIs to guarantee that the semi-structured JSON input provided by clients matches a predefined structure. This is important both to ensure the correctness of the data received as input and also to avoid potential security issues from processing input that is not correctly validated. However, this validation process can be time-consuming and adds overhead to every request. Different keywords in the JSON Schema specification have complex interactions that may increase validation time. Since popular APIs may process thousands of requests per second and schemas change infrequently,we observe that we can resolve some of the complexity ahead of time in order to achieve faster validation. Our JSON Schema validator, Blaze, compiles complex schemas to an efficient representation in seconds to minutes, adding minimal overhead at build time. Blaze incorporates several unique optimizations to reduce the validation time by an average of approximately 10× compared existing validators on a variety of datasets. In some cases, Blaze achieves a reduction in validation time of multiple orders of magnitude compared to the next fastest validator. We also demonstrate that several popular validators produce incorrect results in some cases, while Blaze maintains strict adherence to the JSON Schema specification. • Streaming Validation of JSON Documents Against Schemas (Alexis Le Glaunec, Rice University et al.) - JSON is a popular data format for storing semi-structured data. We investigate the computational problem of JSON validation, which is the task of checking whether a JSON document adheres to a given schema. While there are several existing tools that support JSON validation, they implement offline algorithms that require loading the entire document in memory and creating the full parse tree before performing validation. This offline approach is constrained by the available system memory and is inappropriate when the data is presented as a stream. We propose an approach for performing streaming JSON validation that relies on a new class of pushdown automata that can process JSON documents in an online fashion. Our experimental results show that our approach uses substantially less memory and is faster than state-of-the-art tools. • Rhyme Native: Efficient Code Generation for Structured and Semi-Structured Workloads (Ran Guo, Purdue University et al.) - Modern data processing spans two worlds: flat relational tables, served by decades of database research producing highly optimized query engines, and nested semi-structured data such as JSON, for which expressive query languages exist but compilation and optimization techniques have been applied far less comprehensively. We ask whether a single query language can express both regimes naturally while compiling to efficient native code. We build on Rhyme, a declarative language whose object-notation syntax mirrors the structure of query results, and contribute on three fronts. We refine Rhyme's semantics for generator binding and missing values, allowing co-iteration, inner/outer joins, and nested-loop traversals to be expressed under different uses of generator symbols. We show that Rhyme's prior dependency-driven loop scheduler can generate incorrect code on hierarchical queries, and present a new scheduler based on finer-grained per-statement constraints that ensures correctness. We introduce a gradual type system and a C code generation backend that emits tag-less, statically typed code and specializes data loading and internal data structures for idiomatic SQL patterns. On TPC-H, JSONBench, and additional hierarchical queries, our system matches state-of-the-art compiled engines on SQL workloads and outperforms modern JSON-capable databases and DSLs on hierarchical queries. • Craw: A Unified and Efficient Querying Framework for Large-Scale Video Datasets (Ziqi Zhou, Nanchang University et al.) - The ubiquitous deployment of cameras has led to explosive growth of video data, creating an urgent need to explore valuable content. Single-level queries are insufficient to extract comprehensive information, raising the demand for multi-level queries (existence, dynamic, similarity) within a single unified system. However, limited by the high complexity and redundancy of video, existing systems usually support single-level queries, while Vision-Language Models that support multi-level queries incur prohibitive computational overhead, making them infeasible for large-scale video datasets. To address these issues, we propose Craw, a framework for efficient multi-level queries on large-scale video datasets. Specifically, Craw (1) designs the Video Semantic Unit to encapsulate video semantics, (2) develops a semantic-preserving video segmentation algorithm, and (3) constructs a hybrid index framework integrating an inverted index with a cluster index layer for efficient query execution. Experimental results show that Craw outperforms the state-of-the-art (SOTA) by reducing query latency up to two orders of magnitude, while effectively supporting multi-level queries. • Eureka: Enabling Fine-Grained Access and Range Queries on Compressed Scientific Data via Data-Index Co-Compression (Ning Yan, Georgia State University et al.) - Handling large-scale scientific data in high-performance computing (HPC) environments poses significant challenges, including excessive I/O, high storage costs, and slow query performance. Traditional approaches often require full data decompression and scans, making them impractical for real-time or interactive analysis. To address these limitations, we introduce Eureka, a unified data-index co-compression framework that enables fine-grained access and efficient range queries on compressed scientific datasets. Eureka integrates spatial domain decomposition with block-wise error-bounded lossy compression to support selective decompression. It constructs a hierarchical AVL-tree index during compression to capture block-level value ranges, enabling fast pruning during query execution. To reduce metadata overhead, the index itself is also compressed while ensuring recall-preserving results. Experiments on six diverse HPC simulation datasets show that Eureka achieves up to 25× data compression and over 300× index compression, surpassing state-of-the-art compressors such as SZ3 and ZFP in rate-distortion performance. Additionally, Eureka delivers over 30× speedup for low-selectivity range queries, making it a scalable and efficient solution for modern scientific data analysis.read more
☆
Wed Sep 2 · 3:45 PM - 5:15 PM
Tutorial 9 · Interoperability in Healthcare: A Primer and New Frontiers
● Tutorials
📍 Otis
Interoperability in Healthcare: A Primer and New Frontiers
Andra IonescuKTH Royal Institute of Technology
Paris CarboneKTH Royal Institute of Technology
Sebastiaan MeijerKTH Royal Institute of Technology
Christos KoutrasNew York University
Juliana FreireNew York University
Katja HoseTU Wien
Jayanth RaghothamaKTH Royal Institute of Technology
☆
Wed Sep 2 · 3:45 PM - 5:15 PM
Industry Session 6 · Distributed Analytics, Streaming & Resource Management
● Industry
📍 Marina Ballroom I
No Silver Bullet: Boosting GaussDB Performance on the 30TB TPC-H Workload (Tim Zeyl, Huawei et al.) - GaussDB is Huawei’s premier database system, designed for large-scale deployments and the most demanding workloads.No Silver Bullet: Boosting GaussDB Performance on the 30TB TPC-H Workload (Tim Zeyl, Huawei et al.) - GaussDB is Huawei’s premier database system, designed for large-scale deployments and the most demanding workloads. It is a distributed shared-nothing system, capable of handling all types of workloads. This paper outlines a series of modifications to GaussDB aimed at improving its performance on large-scale and complex analytical workloads. After these changes, its performance on the TPC-H workload exceeded the best published result by 40% at 30 TB. The key enhancements to achieve this elite performance include adopting a pipeline execution model, a faster and more scalable inter-node data shuffle, exploiting a unified bus and unified remote memory access. We also expanded the support of cost-based Bloom filter placement and implemented several Bloom filter streaming strategies, enabling their use across nodes. • ScaleSense: Cost-Intelligent Scaling Framework via Learned Resource Estimation in Alibaba AnalyticDB (Yifan Wu, Zhejiang University et al.) - Cloud-native serverless data warehouses achieve fine-grained elasticity by decoupling storage from compute, yet determining the optimal resource allocation for highly heterogeneous ad-hoc queries remains a formidable industrial challenge. Our analysis of production workloads in Alibaba AnalyticDB exposes a costly provisioning trap: the fear of catastrophic resource depletion drives users to blindly over-provision resources, wasting immense monetary budgets without alleviating non-CPU bottlenecks (e.g., I/O saturation). To break this impasse, we propose \ScaleSense, a proactive, query-level resource scaling framework. Specifically, it features a multi-faceted query encoder that jointly models plan topologies and hardware specifications. Crucially, a quantile-based resource predictor estimates multi-dimensional physical footprints, acting as a reliable safety net for optimal resource scaling. An auto-scaling controller then navigates the performance-cost Pareto frontier, dynamically tailoring allocations to specific business priorities without requiring model retraining. Evaluations on over 1.36 million production queries show that ScaleSense achieves state-of-the-art prediction accuracy with good prediction interval coverage. By achieving a 76.7% relative improvement in optimal resource configuration selection over the best baseline, this approach addresses the critical performance-cost trade-off while maintaining low-overhead inference latency, confirming its practical performance in production deployments. Under the performance-optimization policy, ScaleSense satisfies user-defined performance requirements while reducing monetary cost by up to 5.22x. • A Decade of Apache Spark Structured Streaming: How We Evolved The Architecture To Meet Real-World Needs (Anish Shrigondekar, Databricks et al.) - Apache Spark Structured Streaming is one of the most widely used streaming platforms in the world. It currently serves millions of weekly queries across thousands of Databricks customers. This paper describes the challenges we faced and the solutions we developed while enhancing Structured Streaming to be enterprise ready. Structured Streaming embraces the microbatch architecture, ensuring incremental processing via a series of discrete batch computations. We discuss four noteworthy challenges and solutions in the context of Spark and the microbatch architecture. First, to address the problem of under-utilized task slots - often due to planning time and straggler tasks - we introduced microbatch pipelining - a novel extension which safely incorporates concurrency in this architecture. It improved resource utilization and throughput by almost 3x. Second, many upstream sources such as Amazon Kinesis and Google Pub/Sub did not provide semantics that aligned well with the notion of discrete microbatches. We used techniques such as async pre-fetching, deduplication management, and decoupled metadata tracking to overcome this issue and were able to support a wide variety of streaming sources. Third, the previous version of state management was opaque and inflexible, making it difficult to express complex stateful computations and debug state updates. We evolved our state management offering by adding a new flexible, arbitrary stateful operator called transformWithState and also provided fine-grained visibility into the recorded state for enhanced debuggability and observability. Finally, to enforce fine-grained access control via a sandboxed environment, we had to address novel challenges unique to long-running, stateful streaming queries, specifically around persistent connection management, cross-boundary state access, and credential lifecycle. • A Unified Bandwidth Orchestration Framework for Hierarchical Storage Systems (Ji Zhang, Huawei Technologies et al.) - Hierarchical Data Storage Systems (HSSs) provide a cost-effective architecture that balances capacity and performance through internal data migration. Prior work has primarily focused on optimizing individual migration tasks, either within or across HSS tiers, or on exploiting device bandwidth to improve overall throughput. These approaches treat migration tasks in isolation, and the performance implications of executing heterogeneous migration tasks concurrently remain largely unexplored despite their prevalence in real-world HSS deployments. The growing adoption of Direct Data Access (DDA) architectures, in which accelerators access storage without CPU mediation, further amplifies this problem by removing a natural bandwidth arbiter from the I/O path. This paper presents an in-depth analysis of data migration behavior in commercial HSSs, uncovering substantial performance variability when multiple migration tasks execute concurrently. To mitigate this issue, we propose Pascal, a system-level bandwidth orchestration framework that improves performance robustness in production-grade HSSs. Inspired by hydraulic systems, Pascal adapts pressure/backpressure-style coordination to the multi-task migration setting: it treats each tier as a pressurized vessel and uses pressure gradients to allocate bandwidth across cache flush, tiering, garbage collection, and DDA flows. We evaluate Pascal on a commercial OceanStor HSS across three hardware configurations and eight workloads spanning database, AI training, AI inference, and production traces. Pascal achieves up to 20% higher throughput, 67% lower tail latency, and 79% reduced throughput jitter compared to local state-of-the-art controllers, while also stabilizing the performance jitter introduced by DDA architectures. • SOS: A High-Performance Distributed Key-Value Store for Large-Scale Online Services (Yingxin Li, Tencent et al.) - Large-scale online services - including web search, recommendation, and LLM inference workloads such as Retrieval-Augmented Generation (RAG) and KV-cache offloading - demand storage that handles petabyte-scale data under millisecond tail-latency SLAs. In-memory stores are cost-prohibitive at scale; disk-based systems sacrifice latency for capacity. We present SOS, a distributed key-value store that bridges this gap by guaranteeing at most one physical disk I/O per operation, enforced by three co-designed components: a flat in-memory hash index with ?? (1) lookup; a disk storage engine with a hierarchical variable-size layout that limits internal fragmentation to ?7.93%; and a write cache with asynchronous flush that fully decouples client-visible latency from device I/O. Distributed strong consistency is provided by Primary-Based Chain Replication with Two-Phase Commit and an in-chain retry mechanism that reduces write failures to near-zero under transient hardware jitter. Single-node benchmarks show SOS achieves 704 K QPS under uniform YCSB-B (256 client threads) at P99 = 2 ms, and outperforms RocksDB by 10.0× in throughput and 24.6× in P99 latency on write-heavy YCSB-A, with write amplification of 1.00× (vs. 14 - 16×). Deployed at Tencent for several years, SOS manages over 50 PB and 7 trillion records - peaking at 550 M QPS - powering Yuanbao RAG, Hunyuan KV-cache, user portrait and other services with P99.9 read latencies below 5 ms, at a fraction of in-memory cost. • PolarKV: Tier Locally, Serve Globally - A KV Cache over Cloud Memory and Storage (Yingqiang Zhang, Zhejiang University & Alibaba Cloud Computing et al.) - KV caching is essential for reducing redundant attention computation in LLM inference, yet existing memory-centric or storage centric approaches operate inefficiently in cloud environments due to the bandwidth - capacity - cost coupling of cloud resources. Furthermore, conventional hierarchical cache designs treat each tier as a global pool and often incur substantial cross-tier data movement, which in cloud deployments becomes inter-node network traffic - an extremely scarce and expensive resource. In this paper, we first analyze the bandwidth - capacity - cost characteristics of cloud memory and storage and derive practical guidelines for selecting and combining cloud resources for KV caching. Building on these insights, we present PolarKV , a cloud-native distributed KV cache pool deployed at scale in Alibaba Cloud. PolarKV organizes disaggregated memory and cloud block storage into a shard-paired, co-located hierarchy, treating memory as a bandwidth tier and storage as a capacity tier. PolarKV breaks the “tier-as-a-whole” abstraction: instead of migrating data between two global tiers, it partitions both the memory tier and the storage tier into aligned shards and performs tiering as shard-to-shard movement. By co-locating each memory shard with its corresponding storage shard on the same VM, PolarKV confines most tiering traffic to local paths rather than the network, removing network traffic from the critical tiering path. PolarKV is already commercially available as an independent KV cache service in Alibaba Cloud, enabling customers to connect inference engines such as vLLM and SGLang through a lightweight library. In our evaluation across controlled benchmarks and real production deployments, PolarKV reduces time-to-first-token (TTFT) by up to 80%. To our knowledge, it is the first cloud-native KV cache system that co-locates cloud memory and storage for scalable, cost-efficient LLM serving.read more
☆
Wed Sep 2 · 3:45 PM - 5:15 PM
Demo Group A · Graph, Systems & Infrastructure
● Demos
📍 Marina Ballrooms II - IV
29 demos in this group, shown twice during the conference.29 demos in this group, shown twice during the conference. Includes: QueryBrew: System-Agnostic SQL-to-SQL Query Optimization • FedBridge: A Federated Query Engine over Embedding-Heterogeneous Vector Databases • HaDA: Empower Database Experts with Data Dependencies • OMBench: Taming Data Management Requirements in Cloud-Native Applications • How Out-of-Bounds Are Your Cardinality Estimates? • AtomSQL: Interactive Disambiguation of NL-to-SQL via User-Guided Atom-Level Alignment • Credo: Declarative Control of LLM Pipelines via Beliefs and Policies • SpatialSQL: A Multi-Agent System for Interactive and Observable Spatial Text-to-SQL • DataMagic: Transforming Tabular Data into Data Insight Video • Hamilton - Interactive Ontology Learning • … +19 moreread more
Built for you by(617)-PATRICK, Champion AI speaker
129 research papers presented as posters. Full list on the official program page (footer link) - the papers also appear in their research sessions on this schedule.
Research 37 · Caching, Memory, and Storage Systems
● Research
📍 Grand Ballroom E
How to Write to SSDs (Bohyun Lee, Technische Universität München et al.) - This paper demonstrates that adopting out-of-place writes is essential for database systems to fully leverage SSD performance and extend SSD lifespan.How to Write to SSDs (Bohyun Lee, Technische Universität München et al.) - This paper demonstrates that adopting out-of-place writes is essential for database systems to fully leverage SSD performance and extend SSD lifespan. We propose a set of out-of-place optimizations that collectively reduce write amplification across both the DBMS and SSD layers. We redesign the in-place, B-tree-based LeanStore to write out-of-place and support these optimizations, and evaluate it on diverse OLTP benchmarks, dataset sizes, and SSDs. The final design improves throughput by 1.65 - 2.24× and reduces flash writes per operation by 6.2 - 9.8× on YCSB-A. On TPC-C with 15,000 warehouses, throughput improves by 2.45× while flash writes decrease by 7.2×. Finally, we show that the architecture can seamlessly support novel SSD interfaces such as ZNS and FDP. • Garnet: A Next-Generation Cache-Store for Accelerating Applications and Services (Badrish Chandramouli, Microsoft Research et al.) - Remote cache-stores have seen a dramatic rise in importance in recent years, fueled by a surge in data-driven applications. Most prior database research has focused on various aspects of traditional key-value stores with string values and a simple get/set based remote interface. However, modern cache-stores such as Redis offer a significantly richer interface that has witnessed unprecedented popularity and broad adoption across the developer community. The interface and use cases for such cache-stores in both end-user applications and large-scale services translate to new requirements on storage, scale, complex data type support, and durability. Garnet is a new cache-store that adopts the Redis wire protocol for compatibility, but rethinks from a database perspective how such a modern cache-store system should be designed from the ground up to meet these requirements. Research insights across the storage, network, and cluster stack allow Garnet to support the large Redis interface as a drop-in replacement, yet achieve stronger database features -- thread- and node-scalability, durability, transactions -- and better end-to-end performance (up to 100x higher throughput and 4x lower latency at high percentiles). These results translate to lower end-to-end costs for real-word applications and services. • SIDLE: Tree-structure Aware Indexes for CXL-based Heterogeneous Memory (Haoru Zhao, Shanghai Jiao Tong University et al.) - On heterogeneous memory (CXL-HM) where fast memory (i.e., CPU-attached DRAM) and slow memory (e.g., remote NUMA memory, RDMA-connected memory, Persistent Memory (PM)) coexist, optimizing the placement of tree-structure indexes (e.g., B+ tree) is crucial to achieving high performance while enjoying memory expansion. Nowadays, CXL-based heterogeneous memory (CXL-HM) is emerging due to its high efficiency and memory semantics. Prior tree-structure index placement schemes for HM cannot effectively boost performance on CXL-HM, as they fail to adapt to the changes in hardware characteristics and semantics. Additionally, existing CXL-HM page-level data placement schemes are not efficient for tree-structure indexes due to the granularity mismatch between the tree nodes and the page. In this paper, we argue for a CXL native, tree-structure aware data placement scheme to optimize tree-structure indexes on CXL-HM. Our key insight is that the placement of tree-structure indexes on CXL-HM should match the tree's inherent characteristics with CXL-HM features. We present CXL-HM, a tree-structure aware, node-grained data placement scheme for tree-structure indexes on CXL-HM. With SIDLE, developers can easily adapt existing tree-structure indexes to CXL-HM. We have integrated the B+ tree and radix tree with SIDLE to demonstrate its effectiveness. Evaluations show that SIDLE improves throughput by up to 71% and reduces P99 latency by up to 81% compared with state-of-the-art data placement schemes (e.g., MEMTIS) and HM-optimized tree-structure indexes (e.g., PACTree) in YCSB and real-world workloads. • Operation-Aware Hybrid Locking for Modern In-Memory Indexes (Vishal Gupta, EPFL et al.) - Achieving scalable performance in modern in-memory indexes is primarily limited by synchronization. Traditional synchronization approaches apply a single "one-size-fits-all" strategy, ignoring the diverse characteristics of different index operations. For instance, pessimistic lock coupling forces high atomic overhead on all tree traversals, even simple lookup operations. Meanwhile, optimistic queue-based locking, while efficient for lookups, suffers from performance collapse due to shared data movement during high-contention updates. This paper introduces OPAL, a hybrid operation-aware lock design for modern in-memory indexes. OPAL dynamically selects among three locking mechanisms within a single lock instance based on operation type: (i) optimistic version-based locking for read-only lookups; (ii) lightweight function-pointer-based batching for updates that eliminates shared data movement; and (iii) traditional MCS-based locking for structural modification operations (SMOs), such as node splits and merges, that naturally distributes contention across multiple index nodes. We evaluate OPAL on widely-used index structures: a B+ Tree and an Adaptive Radix Tree (ART). Compared to state-of-the-art optimistic locking, OPAL improves throughput by up to 2.43x and reduces latency by 80%. • Scalable GPU Acceleration of Scalar Functions in Analytical Databases: Compilation, Benchmarking and Optimization (Kaushik Rajan, Microsoft Research et al.) - Accelerating SQL query execution with GPUs is a central focus in database research. While prior systems have achieved notable speedups by offloading relational operators, the acceleration of the wide range of scalar functions that are supported by analytical engines remains unaddressed. Our analysis reveals that many scalar functions incur substantial computational overhead and often constitute the primary bottleneck in analytical queries on CPUs. This observation motivates a systematic exploration of the opportunities and challenges in accelerating scalar functions on GPUs. Unlike relational operators, which are few in number and standardized, production databases support hundreds of scalar functions. The absence of a standardized specification, combined with this diversity, renders manual GPU porting infeasible. To address this, we present an LLVM-MLIR-based compiler toolchain that automatically translates the CPU-based implementations of scalar functions from production databases into efficient GPU kernels while preserving their original semantics. Our approach lifts scalar functions to a high-level intermediate representation, applies resource-optimizing transformations, and generates GPU assembly code, supporting all relevant data types, parameters, and database context variables. As existing benchmarks do not sufficiently stress test scalar functions in analytical queries, we introduce a variant of TPC-H that utilizes scalar functions while preserving the original query intent. Integrating our GPU kernels into a state-of-the-art GPU database system, we demonstrate substantial performance gains over a leading CPU database that uses slightly more expensive hardware: 7.6× on enhanced TPC-H and 6.4× on production queries, further widening the gap between GPU and CPU databases. The generated kernels deliver performance comparable to hand-optimized GPU implementations, establishing our approach as a scalable and practical solution for accelerating scalar functions on GPUs. • Demystifying and Improving Lazy Promotion in Cache Eviction (Qinghan Chen, Carnegie Mellon University et al.) - Cache eviction algorithms play a critical role in the performance of modern data systems, yet their scalability is often limited by the high computational overhead associated with object promotions. Lazy Promotion techniques have emerged as relaxations of traditional Least-Recently-Used (LRU) methods, designed to alleviate lock contention and increase throughput. This work uses production traces from real-world systems to benchmark five Lazy Promotion strategies: Probabilistic-LRU, Batch-LRU, Delay-LRU, FIFO-reinsertion, and Random-LRU. We evaluate these techniques across miss ratio, scalability, promotion count, and a novel metric called promotion efficiency, which measures the number of hits per promotion. Our results reveal that Delay-LRU and FIFO-reinsertion significantly improve promotion efficiency, whereas Batch-LRU and Probabilistic-LRU struggle to reduce promotions without significantly increasing miss ratio. We further explore the impact of lazy promotion in advanced algorithms such as ARC and 2Q and make a similar observation. Moreover, we uncover substantial optimization potential, showing that most cache promotions are unnecessary when equipped with oracle knowledge. To further reduce promotions in LRU, we propose two novel enhancements - Delayed FIFO-reinsertion (D-FR) and Age-Guided Eviction (AGE) - that reduce promotions by 20 - 60% while achieving a similar or lower miss ratio.read more
☆
Thu Sep 3 · 10:45 AM - 12:15 PM
Research 38 · High-Performance Parallel Analytics
● Research
📍 Grand Ballroom D
CrocSort: Resource-Efficient, Skew-Resilient Parallel External Merge Sort (Riki Otaki, University of Chicago et al.) - Sorting is a core operator in large-scale data systems.CrocSort: Resource-Efficient, Skew-Resilient Parallel External Merge Sort (Riki Otaki, University of Chicago et al.) - Sorting is a core operator in large-scale data systems. As data increasingly exceeds main memory, external merge sort is essential, yet many implementations over-allocate memory and over-parallelize, decreasing efficiency. We present CrocSort, a byte-balanced parallel external merge sort with configurable memory and per-phase thread settings. Using analysis and experiments, we derive practical resource-configuration rules for selecting these settings from input size, memory budget, and thread cap. To balance parallel merge under skew, CrocSort reuses run sparse indexes for range partitioning to create a virtual total order over records. CrocSort also uses offset-value codes and related optimizations to reduce comparison work and, for prefix-redundant workloads, intermediate I/O volume. Across TPC-H and synthetic workloads on modern NVMe systems, CrocSort completes sorting at memory budgets where production systems abort, and the planner reduces unnecessary resource allocation compared to the greedy maximal approach across both tight- and ample-memory regimes. • One Pass to Parse Them All: Fused Parallel CSV Processing (Simon Ellmann, Technical University of Munich et al.) - CSV remains one of the most widely used formats for exchanging tabular data, making efficient CSV processing an important problem. Yet most CSV parsers are sequential, failing to exploit the parallelism of modern hardware. While parallel CSV parsing approaches have been proposed in the literature, none of these seem to be used in practice. Conversely, a simple idea for synchronization-free speculative parsing that is used for parallel parsing, e.g., in DuckDB, has never been described in the literature, nor has it been exploited efficiently. In this paper, we close this gap. We contribute a) a description of how real-world CSV files can be parsed in parallel on commodity multicore CPUs, b) a new programming model for general-purpose CSV parsers that unifies parallel parsing and parallel data processing into one pass over the data, and c) a new vectorization strategy with efficient index and zero-copy record construction to accelerate parsing. Our evaluation shows that csveee, our parser, outperforms widely-used CSV parsers in single- and multi-threaded performance, and scales near-linearly to achieve throughput of up to 180 GB/s - 22x faster than DuckDB - all while remaining practical for integration into real-world data processing systems. • Succinct and Fast Tiny Pointer Hash Tables (Xilin Tang, Cornell University et al.) - Hash tables sit on the critical path of many systems, yet modern designs still force a trade-off between fast operations and high memory overhead. We revisit this trade-off and present Tiny Pointer Hash Tables (TPHT), a family of practical hash tables that make two ideas from theory work at system scale: compressing pointers down to a byte, and encoding keys compactly so less metadata is needed. We engineer these into two complementary designs. Chained-TPHT targets maximal space savings, and is to our knowledge the first simple and practical succinct hash table, achieving a footprint smaller than the raw key-value payload size with constant-time operations. Flattened-TPHT targets latency, keeping the common case within a single cache miss while retaining strong space efficiency. Both variants support dynamic resizing without global pauses and integrate cleanly with 64-bit keys and values. Across YCSB and microbenchmarks, TPHT advances the latencyspace Pareto frontier: Chained-TPHT reaches 105.4% space efficiency, and Flattened-TPHT achieves 83.4% space efficiency with up to 89.3% higher throughput than strong baselines. Together, these results show that techniques primarily known in theory can be turned into systems-ready hash tables that meaningfully reduce memory use while delivering state-of-the-art performance. • Automated Tensor-Relational Decomposition for Large-Scale Sparse Tensor Computation (Yuxin Tang, Rice University et al.) - A \emph{tensor-relational} computation is a relational computation where individual tuples carry vectors, matrices, or higher-dimensional arrays. An advantage of tensor-relational computation is that the overall computation can be executed on top of a relational system, inheriting the system's ability to automatically handle very large inputs with high levels of sparsity while high-performance kernels (such as optimized matrix-matrix multiplication codes) can be used to perform most of the underlying mathematical operations. In this paper, we introduce upper-case-lower-case \texttt{EinSum}, which is a tensor-relational version of the classical Einstein Summation Notation. We study automatically re-writing a computation in Einstein Notation into upper-case-lower-case \texttt{EinSum} so that computationally intensive components are executed using efficient numerical kernels, while sparsity is managed relationally. • Global Hash Tables Strike Back! An Analysis of Parallel GROUP BY Aggregation (Daniel Xue, University of Pennsylvania et al.) - Efficiently computing group aggregations (i.e., GROUP BY) on modern architectures is critical for analytic database systems. Today, hash-based methods predominantly use a partitioned approach, in which incoming data is partitioned by key so that every row for a particular key is sent to the same partition. In this paper, we revisit a simpler strategy: a fully concurrent aggregation technique using a shared hash table. While approaches using general-purpose concurrent hash tables have generally been found to perform worse than partitioning-based approaches, we argue that the key ingredient is customizing the concurrent hash table for the specific task of group aggregation. Through experiments on synthetic workloads (varying key cardinality, skew, and thread count), we demonstrate that in morsel-driven systems, a purpose-built concurrent hash table can match or surpass partitioning-based techniques. We also analyze the operational characteristics of both techniques, including resizing costs and memory pressure. In the process, we derive practical guidelines for database implementers. Overall, our analysis indicates that fully concurrent group aggregation is a viable alternative to partitioning.read more
☆
Thu Sep 3 · 10:45 AM - 12:15 PM
Research 39 · ML Pipeline Management
● Research
📍 Grand Ballroom C
CAPS: Cost-Aware ML Pipeline Selection (A KONTAXAKIS, Université libre de Bruxelles et al.) - We present CAPS, a novel system for cost-aware pipeline selection in automated machine learning (AutoML).CAPS: Cost-Aware ML Pipeline Selection (A KONTAXAKIS, Université libre de Bruxelles et al.) - We present CAPS, a novel system for cost-aware pipeline selection in automated machine learning (AutoML). CAPS' approach is orthogonal to the search strategies used by existing AutoML frameworks, enabling seamless integration with them. This integration benefits AutoML systems by reducing computational waste---time spent evaluating inefficient pipelines or exceeding time or memory constraints---through lightweight performance and cost estimation. To incorporate cost-awareness into AutoML, \sys represents a set of pipelines as a directed hypergraph, estimates execution times for each function, and formulates the optimal subset selection as a constrained prize-collecting optimization problem. CAPS employs a simple yet effective greedy algorithm to approximate this optimal subset. Our evaluation shows that when CAPS prioritizes cost, it reduces waste up to 4x---saving up to 23 hours in 50 hours of pipeline evaluation. When CAPS balances the trade-off between performance and cost, it consistently achieves performance improvements of 5% to 15% over the state of the art within the same time budget across datasets. It also demonstrates that CAPS is compatible with complementary efficiency techniques such as early stopping and warm-starting, achieving speedups of up to 4.7x. • Stress-Testing ML Pipelines with Adversarial Data Corruption (Jiongli Zhu, University of California San Diego et al.) - Structured data-quality issues - such as missing values correlated with demographics, culturally biased labels, or systemic selection biases - routinely degrade the reliability of machine-learning pipelines. Regulators now increasingly demand evidence that high-stakes systems can withstand these realistic, interdependent errors, yet current robustness evaluations typically use random or overly simplistic corruptions, leaving worst-case scenarios unexplored. We introduce Savage, a causally inspired framework that (i) formally models realistic data-quality issues through dependency graphs and flexible corruption templates, and (ii) systematically discovers corruption patterns that maximally degrade a target performance metric. Savage employs a bi-level optimization approach to efficiently identify vulnerable data subpopulations and fine-tune corruption severity, treating the full ML pipeline, including preprocessing and potentially non-differentiable models, as a black box. Extensive experiments across multiple datasets and ML tasks (data cleaning, fairness-aware learning, uncertainty quantification) demonstrate that even a small fraction (around 5%) of structured corruptions identified by Savage severely impacts model performance, far exceeding random or manually crafted errors, and invalidating core assumptions of existing techniques. Thus, Savage provides a practical tool for rigorous pipeline stress-testing, a benchmark for evaluating robustness methods, and actionable guidance for designing more resilient data workflows. • PipeLens: Identifying Interventions for Resolving Malfunctioning Data Science Pipelines (Jahid Hasan, Purdue University et al.) - Data is a critical component of modern decision-making systems; system malfunctions (e.g., performance degradation and module failure) can often be traced back to a mismatch between the properties of the data and the assumptions of the system modules that process the data. For example, with the increasing use of open-source libraries to develop data science pipelines, common causes of system malfunctions include inappropriately configured data processing libraries for data cleaning tasks such as entity resolution or missing value imputation. Our objective is to resolve malfunctioning pipelines and improve their utility; we introduce PipeLens, a framework that leverages successful and unsuccessful runs of past pipelines for fixing pipeline malfunctions. PipeLens uses an acyclic graph representation of the pipeline and performs causal reasoning through interventions: when a system malfunctions with a given dataset, PipeLens modifies the pipeline (by changing its structure or the parameters of its modules) and observes the impact of this intervention on system behavior. To focus on useful interventions, we learn a proxy function that approximates the pipeline’s utility over a dataset and guides the search for the best intervention. Unlike traditional observational analysis that reports correlations between system parameters and their behavior, we provide causally verified root causes and suggest pipeline modifications that rectify malfunctions. Empirical evaluation on four data science tasks over four real-world datasets demonstrates that PipeLens consistently outperforms baselines in terms of interventions performed to repair malfunctions while maintaining practical running times. • Local Shapley: Model-Induced Locality and Optimal Reuse in Data Valuation (Xuan Yang, Duke university et al.) - The Shapley value provides a principled foundation for data valuation, but exact computation is #P-hard due to the exponential coalition space. Existing accelerations remain global and ignore a structural property of modern predictors: for a given test instance, only a small subset of training points influences the prediction. We formalize this model-induced locality through support sets defined by the model’s computational pathway (e.g., neighbors in KNN, leaves in trees, receptive fields in GNNs), showing that Shapley computation can be projected onto these supports without loss when locality is exact. This reframes Shapley evaluation as a structured data processing problem over overlapping support-induced subset families rather than exhaustive coalition enumeration. We prove that the intrinsic complexity of Local Shapley is governed by the number of distinct influential subsets, establishing an information-theoretic lower bound on retraining operations. Guided by this result, we propose LSMR (Local Shapley via Model Reuse), an optimal subset-centric algorithm that trains each influential subset exactly once via support mapping and pivot scheduling. For larger supports, we develop LSMR-A, a reuse-aware Monte Carlo estimator that remains unbiased with exponential concentration, with runtime determined by the number of distinct sampled subsets rather than total draws. Experiments across multiple model families demonstrate substantial retraining reductions and speedups while preserving high valuation fidelity. • Morphing-based Compression for Data-centric ML Pipelines (Sebastian Baunsgaard, Technische Universität Berlin et al.) - Data-centric ML pipelines extend traditional machine learning (ML) pipelines---of feature transformations and ML model training---by outer loops for data cleaning, augmentation, and feature engineering to create high-quality input data. Existing lossless matrix compression applies lightweight compression schemes to numeric matrices and performs linear algebra operations such as matrix-vector multiplications directly on the compressed representation but struggles to efficiently rediscover structural data redundancy. Compressed operations are effective at fitting data in available memory, reducing I/O across the storage-memory-cache hierarchy, and improving instruction parallelism. The applied data cleaning, augmentation, and feature transformations provide a rich source of information about data characteristics such as distinct items, column sparsity, and column correlations. In this paper, we introduce BWARE---an extension of AWARE for workload-aware lossless matrix compression---that pushes compression through feature transformations and engineering to leverage information about structural transformations. Besides compressed feature transformations, we introduce a novel technique for lightweight morphing of a compressed representation into workload-optimized compressed representations without decompression. BWARE shows substantial end-to-end runtime improvements, reducing the execution time for training data-centric ML pipelines from days to hours. • Fault Lines: Benchmarking the Impact of Label Data Quality on ML Robustness and Fairness (David Jackson, University of Amsterdam et al.) - Artificial intelligence systems depend critically on high-quality data, yet real-world datasets are often imperfect. Label noise, such as incorrect or biased labels, can lead to suboptimal model decisions. While label noise has garnered increasing attention, existing research primarily examines random noise, employs simpler models, or relies on limited evaluation criteria. To address this, we introduce Fault Lines, a comprehensive, model-agnostic benchmark comprising 15 datasets systematically corrupted with diverse types of label noise, paired with an evaluation framework. This resource supports the evaluation of data cleaning pipelines and guides the design of models that are robust, in both performance and fairness, to label noise. We benchmark the robustness to label noise of 22 state-of-the-art classification models, including gradient boosting, transformers, and fairness-oriented models. Our findings show that many models maintain strong performance under high random noise (e.g., up to 40% noise leads to only a modest reduction in Robust GBDT performance). However, these models are significantly less robust to even small amounts of biased noise (<10%), which can cause substantial performance drops (e.g., 7% noise reduces ResNet’s AUC by 4.4% on average) or maintain apparent stability at the expense of severe fairness degradation (e.g., MLP’s Predictive Parity difference increases by ~700% under 30% biased noise in the ACS Unemployment dataset). We investigate how different model architectures handle the impact of biased noise. Notably, transformer-based models appear more robust than boosting models when handling biased noise, though this advantage depends on tuning and comes with higher variance. Finally, we identify key factors for ML practitioners to mitigate the effects of label noise, including model selection, dataset analysis, and preprocessing.read more
☆
Thu Sep 3 · 10:45 AM - 12:15 PM
Research 40 · Scalable Graph Algorithms II
● Research
📍 Commonwealth Ballroom C
Efficient GPU-Accelerated Adaptive Minimum Cost Seed Selection (Gongyao Guo, The Hong Kong Polytechnic University et al.) - Efficient influence estimation and seed selection are crucial to social network advertising and are widely studied in data management.Efficient GPU-Accelerated Adaptive Minimum Cost Seed Selection (Gongyao Guo, The Hong Kong Polytechnic University et al.) - Efficient influence estimation and seed selection are crucial to social network advertising and are widely studied in data management. We focus on adaptive minimum cost seed selection (AMCSS), which selects seed nodes adaptively over multiple rounds, to reach a target number $\eta$ of influenced users while minimizing total seed cost. Prior work, notably ASTI with multi-root reverse reachable sets (mRR-sets), remains computationally expensive on large graphs, often taking hours to days even with CPU parallelism. We present GAAS, a GPU-accelerated method that integrates novel algorithmic and GPU-aware system designs to solve AMCSS in minutes. Processing mRR-sets involves irregular access patterns and variable-size samples, mismatching the GPU parallel architecture. Hence, we first develop a GPU-tailored mRR-set structure, GmRR, that assigns each thread block exclusive ownership of an equal-size segment with a circular layout, enabling efficient parallel mRR-set management while minimizing write contention. With GmRR, we design a GPU kernel ParallelGen to generate mRR-sets. Unlike prior work that regenerates mRR-sets from scratch in each round, we propose to update and reuse those from previous round, improving efficiency while requiring GPU-aware designs and rigorous theoretical analysis. Specifically, we design a ParallelUpdate kernel with theoretically grounded update rules that uses circular segment updates on GmRR for efficient mRR-set updates, together with a load-balancing scheme. We further devise a Select kernel for parallel seed selection. Integrating these together, GAAS efficiently solves AMCSS on GPUs with guarantees. Extensive experiments on large real-world graphs under different diffusion models show that GAAS is over an order of magnitude faster (up to 68.9$\times$) than parallel CPU and GPU baselines, while the seed cost is among the lowest. • Efficient GPU-Accelerated Local Subgraph Counting (Qiao He, The Hong Kong Polytechnic University et al.) - Local subgraph counting computes the exact number of occurrences of a query graph around every vertex in a data graph. By capturing local higher‑order structure, it supports extensive applications in network analysis and graph learning. The fastest existing method, SCOPE, accelerates counting through query graph decomposition, but it is designed for single‑threaded CPU execution. As a result, it struggles on large graphs and cannot take advantage of modern GPU hardware. A naïve GPU adaptation is also ineffective: as the number of parallel GPU threads grows, the memory footprint of their intermediate results quickly drains the device memory. We develop a high‑performance GPU solution for local subgraph counting that preserves SCOPE’s tree‑decomposition framework while explicitly resolving the tension between massive GPU parallelism and limited device memory. Our approach compresses the intermediate join-and-aggregate results and proposes an insert‑failure restart mechanism that guarantees correctness under bounded memory. We further design a key‑mapping strategy that enables lock‑free hash tables for higher throughput, eventually integrating these components into a complete GPU execution framework capable of handling arbitrarily complex queries. Experiments show that our GPU‑accelerated method achieves up to a \revise{35$\times$ speedup over a multi-threaded SCOPE implementation}, reducing the processing time for a million‑scale graph from days to about 20 minutes and making local subgraph counting practical at large scale. • Augmenting Social Influence of Uncertain Seeds via Probabilistic Link Insertion (Xiaolong Chen, The Hong Kong University of Science and Technology (Guangzhou) et al.) - The emergence of link recommendation systems has triggered a line of research on strategic link insertion to enhance information diffusion in social networks. Existing literature assumes a seed set where all seed users are deterministically activated at the start of the campaign. However, uncertain seeding is being increasingly prevalent and can be used to model more general scenarios like users' defaulting behavior or discount-based marketing. To investigate how to augment the influence of uncertain seeds by link recommendation, we formulate a problem named \textit{influence maximization with augmentation for uncertain seeds} (IMAUS), which aims to insert $k$ edges incident to the uncertain seeds so as to maximize the influence of the given seeds. Due to the NP-hardness of the problem and the non-submodularity of the objective function, solving IMAUS is technically challenging. To address this, we resort to the sandwich strategy and propose two submodular bounding functions for the optimization objective. To overcome the \#P-hardness of the bounding functions computation, we provide two unbiased estimators for the bounding functions via non-trivial usage of reverse influence sampling and devise greedy algorithms equipped with several principled accelerating techniques to return $(1-1/\mathsf{e}-\varepsilon)$-approximations for maximizing the bounding functions. With the above design, we instantiate the sandwich framework in a joint baking manner to reduce repeated sampling. Extensive experiments on $6$ real-world datasets are conducted to validate the effectiveness and efficiency of the proposed methods. Specifically, our algorithm consistently produces a higher influence increment than the baselines and is able to return a size-$100$ edge set for a billion-size graph within $10$ minutes. • Counting HyperGraphlets via Color Coding: a Quadratic Barrier and How to Break It (Marco Bressan, University of Milan et al.) - We study the problem of counting $k$-\emph{hyper}graphlets, an interesting but surprisingly ignored primitive, with the aim of understanding if efficient algorithms exist. To this end we consider \emph{color coding}, a well-known technique for approximately counting $k$-graphlets in graphs. Our first result is that, on hypergraphs, color coding encounters a \emph{quadratic barrier}: under the Orthogonal Vector Conjecture, no implementation of it can run in time sub-quadratic in the size of the input. We then introduce a simple property, $(\alpha,\beta)$-niceness, that hypergraphs from real-world datasets appear to satisfy for small values of $\alpha$ and $\beta$. Intuitively, an $(\alpha,\beta)$-nice hypergraph can be split into two sub-hypergraphs having respectively rank at most $\alpha$ and degree at most $\beta$. By applying different techniques to each sub-hypergraph and carefully combining the outputs, we show how to run color coding in time $2^{O(k)} \cdot \big(2^\beta |V| + \alpha^k |E| + \alpha^2 \beta \size{H}\big)$, where $H=(V,E)$ is the input hypergraph. Afterwards, we can sample colorful $k$-hypergraphlets uniformly in expected $k^{O(k)} \cdot (\beta^2+\ln |V|)$ time per sample. Experiments on real-world hypergraphs show that our algorithm neatly outperforms the naive quadratic algorithm, sometimes by more than an order of magnitude. • X-Wim: Massive Parallelization of Weighted Matching in Bipartite Graphs (Dayi Fan, The Ohio State University et al.) - The maximum weight perfect matching (MWPM) problem in bipartite graphs has extensive applications in database, machine learning, financial markets, and other data-intensive domains, and serves as a general formulation of weighted matching problems. The Hungarian algorithm is widely adopted for solving bipartite MWPM, and substantial research efforts have focused on improving its sequential time complexity. As data volumes grow and real-time processing demands escalate, parallel solutions become increasingly essential. However, efficient parallelization remains highly nontrivial due to the algorithm's intricate execution patterns, inherently sequential data dependencies, frequent phase switching, and the single-path-per-iteration search constraint. These critical issues motivate us to develop X-Wim, a massively parallel framework. It is built on a new phase-decoupled approach that breaks the strong interleaving between algorithmic phases, eliminates frequent global updates, enables concurrent search for multiple disjoint paths, and incorporates an adaptive search strategy. These algorithmic design efforts lead to substantial performance gains, even in the single-threaded setting. Extensive experiments on real-world datasets indicate that X-Wim surpasses state-of-the-art baselines, achieving up to a 9.93x speedup with 1 core and up to a 56.3x speedup with 8 cores. It also exhibits strong scalability. In tests up to 96 cores, it achieves an average 1.70x speedup each time the number of threads doubles. To the best of our knowledge, X-Wim is the fastest solution for this class of graph algorithms.read more
☆
Thu Sep 3 · 10:45 AM - 12:15 PM
Research 41 · LLMs for SQL and Data Preparation
● Research
📍 Commonwealth Ballroom B
Cleaning both Data Errors and Inaccurate Constraints on Numerical Sequential Data (Xiaoou Ding, Harbin Institute of Technology et al.) - Numerical sequence data from intelligent devices often have quality issues.Cleaning both Data Errors and Inaccurate Constraints on Numerical Sequential Data (Xiaoou Ding, Harbin Institute of Technology et al.) - Numerical sequence data from intelligent devices often have quality issues. While existing data cleaning methods focus on repairing data, we address the problem of repairing both data errors and inaccurate constraints. We propose two operations for modifying inaccurate constraints: expanding and compressing their value domains. Our solution includes constraint modification functions and algorithms to prevent under- and over-fitting in data cleaning. Theoretical evaluations demonstrate its reliability and effectiveness of the proposed solution, which achieves optimal repair with the distance no greater than |Σ′l| · ∊e + |Σ′r| · ∊s from the optimal repair. Experiments on real-life and synthetic datasets show that our bNDCRepair method improves F1-score by 17.6% compared to using the original constraints and performs best in MNAD. Results show high-level performance with the combination of our bNDCRepair and the state-of-the-art CVtRepair and Clean4TSDB in sequential data tasks. • SQL-Factory: A Multi-Agent Framework for High-Quality and Large-Scale SQL Generation (Jiahui Li, Zhejiang University et al.) - High quality SQL corpus is essential for intelligent databases. For example, Text-to-SQL requires SQL queries and corresponding natural language questions as training samples. However, collecting such a query corpus remains challenging in practice due to the high cost of manual annotation, which highlights the importance of automatic SQL generation. Despite recent advances, existing generation methods still face limitations in achieving both diversity and cost-effectiveness. Besides, many methods also treat all tables equally, which overlooks schema complexity and leads to under-utilization of structurally rich tables. To address these issues, this paper proposes a multi-agent framework for high-quality and large-scale SQL generation, dubbed SQL-Factory. It decomposes the generation process into three collaborative teams: the Generation Team explores diverse query structures using a powerful language model, the Expansion Team scales promising patterns via a lightweight language model, and the Management Team adaptively schedules the workflow and evaluates the quality of synthesized queries. This modular framework ensures a balanced trade-off between diversity, scalability, and generation cost. We apply SQL-Factory to four widely used benchmarks and generate over 300,000 SQL queries with less than $200 API cost. Our generated queries achieve higher diversity compared to other methods, and extensive experiments demonstrate that the generated queries significantly improve model performance in various downstream tasks. • LLM-AutoDP: Automatic Data Processing via LLM Agents for Model Fine-tuning (Wei Huang, antgroup et al.) - Large Language Models (LLMs) can be fine-tuned on domain-specific data to enhance their performance in specialized fields. However, such data often contains numerous low-quality samples, necessitating effective data processing (DP). In practice, DP strategies are typically developed through iterative manual analysis and trial-and-error adjustment. These processes inevitably incur high labor costs and may lead to privacy issues in high-privacy domains like healthcare due to direct human access to sensitive data. Thus, achieving automated data processing without exposing the raw data has become a critical challenge. To address this challenge, we propose \textbf{LLM-AutoDP}, a novel framework that leverages LLMs as intelligent agents to automatically generate and optimize data processing strategies. Starting from an initial prompt, our method generates multiple candidate strategies and iteratively refines them using feedback signals and comparative evaluations. This iterative in-context learning mechanism enables the agent to converge toward high-quality processing pipelines without requiring direct human intervention or access to the underlying data. To further accelerate strategy search, we introduce three key techniques: (1) \textit{Distribution Preserving Sampling}, which reduces data volume while maintaining distributional integrity; (2) \textit{Processing Target Selection}, which uses a binary classifier to identify low-quality samples for focused processing; and (3) \textit{Cache-and-Reuse Mechanism}, which minimizes redundant computations by reusing prior processing results. We evaluate LLM-AutoDP on five medical datasets across three model architectures. Results show that models trained on data processed by our framework achieve over 80\% win rates against models trained on unprocessed data. Compared to AutoML baselines based on LLM agents, LLM-AutoDP achieves approximately a 65\% win rate. Moreover, our acceleration techniques reduce the total searching time by up to $10\times$, demonstrating both effectiveness and efficiency. • SEMA: A High-performance System for LLM-based Semantic Query Processing (Kangkang Qi, Beijing Institute of Technology et al.) - The integration of Large Language Models (LLMs) into data analytics has unlocked powerful capabilities for reasoning over bulk structured and unstructured data. However, existing systems typically rely on either DataFrame primitives, which lack the efficient execution infrastructure of modern DBMSs, or SQL User-Defined Functions (UDFs), which isolate semantic logic from the query optimizer and burden users with implementation complexities. The LLM-powered semantic operators also bring new challenges due to the high cost and non-deterministic nature of LLM invocation, where conventional optimization rules and cost models are inapplicable for their optimization. To bridge these gaps, we present Sema, a high-performance semantic query engine built on DuckDB that treats LLM-powered semantic operators as first-class citizens. Sema introduces SemaSQL, a declarative dialect that allows users to seamlessly inject natural language expressions into standard SQL clauses, enabling end-to-end optimization and execution. At the logical level, the optimizer of Sema compresses natural language expressions and deduces relational constraints from semantic operators. At runtime, Sema employs Adaptive Query Execution (AQE) to dynamically reorder operators, fuse semantic operations, and apply prompt batching. This approach seeks a Pareto-optimal execution path that minimizes either token consumption or latency according to user preference, subject to accuracy constraints. We evaluate Sema on 26 semantic queries across classification, summarization, text extraction, and ranking tasks. Experimental results demonstrate that Sema achieves 2 − 10× speedup against three baseline systems while achieving competitive result quality. • Unstructured Data Analysis using LLMs: A Comprehensive Benchmark (Qiyan Deng, Beijing Institute of Technology et al.) - The explosion of unstructured data has immense analytical value. By leveraging large language models (LLMs) to extract table-like attributes from unstructured data, researchers are building LLM-powered systems that let users analyze documents as if querying a database. These unstructured data analysis (UDA) systems differ widely in query interfaces, optimization, and operators, making it unclear which works best in which scenario. However, no benchmark currently offers high-quality, large-scale, diverse datasets and rich query workloads to rigorously evaluate them. We present Bench-U, a comprehensive UDA benchmark that addresses this need. We curate 6 datasets from different domains and manually construct a relational database view for each using 30 graduate students. These relational databases serve as ground truth to evaluate any UDA system, regardless of its interface. We further design diverse queries over the database schema that evaluate various analytical operators with different selectivities and complexities. Using this benchmark, we conduct an in-depth analysis of key UDA components - query interface, optimization, operator design, and data processing - and run exhaustive experiments to evaluate systems and techniques along these dimensions. Our main contributions are: (1) a comprehensive benchmark for rigorous UDA evaluation, and (2) a deeper understanding of the strengths and limitations of current systems, paving the way for future work in unstructured data analysis. • ALER: An Active Learning Hybrid System for Efficient Entity Resolution (Dimitrios Karapiperis, International Hellenic Univeristy et al.) - Entity Resolution (ER) is a critical task for data integration, yet state-of-the-art supervised deep learning models remain impractical for many real-world applications due to their need for massive, expensive-to-obtain labeled datasets. While Active Learning (AL) offers a potential solution to this "label scarcity" problem, existing approaches introduce severe scalability bottlenecks. Specifically, they achieve high accuracy but incur prohibitive computational costs by re-training complex models from scratch or solving NP-hard selection problems in every iteration. In this paper, we propose ALER, a novel, semi-supervised pipeline designed to bridge the gap between semantic accuracy and computational scalability. ALER eliminates the training bottleneck by using a frozen bi-encoder architecture to generate static embeddings once and then iteratively training a lightweight classifier on top. To address the memory bottleneck associated with large-scale candidate pools, we first select a representative sample of the data and then use K-Means to partition this sample into semantically coherent chunks, enabling an efficient AL loop. We further propose a hybrid query strategy that combines "confused" and "confident" pairs to efficiently refine the decision boundary while correcting high-confidence errors. Extensive evaluation on large-scale datasets demonstrates ALER's superior efficiency: it consistently accelerates the training loop while drastically reducing resolution latency by a factor of 3.8 compared to the fastest baseline. • LEAD: Iterative Data Selection for Efficient LLM Instruction Tuning (Xiaotian LIN, The Hong Kong University of Science and Technology (Guangzhou) et al.) - Instruction tuning has emerged as a critical paradigm for improving the capabilities and alignment of large language models (LLMs). However, existing iterative model-aware data selection methods incur significant computational overhead, as they rely on repeatedly performing full-dataset model inference to estimate sample utility for subsequent training iterations. In this paper, we propose LEAD, a framework that LEArns to iteratively select Data by accurately estimating sample utility entirely within the standard training loop, eliminating the need for additional model inference. At its core, LEAD introduces Instance-Level Dynamic Uncertainty (IDU), a theoretically grounded utility function combining instantaneous training loss, gradient-based approximation of loss changes, and exponential smoothing of historical loss signals. To further scale efficiently to large datasets, LEAD employs a two-stage, coarse-to-fine selection strategy, adaptively prioritizing informative clusters through a multi-armed bandit mechanism, followed by precise fine-grained selection of high-utility samples using IDU. Extensive experiments across four diverse benchmarks show that LEAD significantly outperforms state-of-the-art methods, improving average model performance by 6.1%-10.8% while using only 2.5% of the training data and reducing overall training time by 5-10×.read more
☆
Thu Sep 3 · 10:45 AM - 12:15 PM
Research 42 · Privacy-Preserving Data Analytics
● Research
📍 Commonwealth Ballroom A
Doppio: Communication-Efficient and Secure Multi-Party Shuffle Differential Privacy (Wentao Dong, City University of Hong Kong et al.) - Modern database ecosystems increasingly process large-scale distributed user data, heightening the intrinsic tension between analytical utility and individual privacy.Doppio: Communication-Efficient and Secure Multi-Party Shuffle Differential Privacy (Wentao Dong, City University of Hong Kong et al.) - Modern database ecosystems increasingly process large-scale distributed user data, heightening the intrinsic tension between analytical utility and individual privacy. Shuffle differential privacy (shuffle DP) has recently emerged as a promising paradigm between the local and central models, offering favorable privacy-utility trade-offs by introducing a centralized, trusted shuffler. However, this architectural shift also poses new challenges in trust assumptions, system overhead, security risks, and workload limitations. To address them, we propose the augmented multi-party shuffle DP (AMP-SDP) model, which re-architects the data pipeline with a lightweight, versatile secret-shared intermediary layer. AMP-SDP (1) decentralizes trust while minimizing online communication costs; (2) provides structural security hardening against both shuffler compromise and user-side poisoning risks; and (3) augments shuffle DP for broader, more flexible workloads. Atop this model, we instantiate Doppio, a privacy-preserving crowdsourcing and data analytics framework. Our results show Doppio outperforms the state-of-the-art decentralized shuffle DP mechanism (Network Shuffling, SIGMOD'22) across many key metrics, affirming its effectiveness and efficiency in modern privacy-aware data management. • Bifrost: A Much Simpler Secure Two-Party Data Join Protocol for Secure Data Analytics (Shuyu Chen, Fudan University et al.) - Secure data join enables two parties with vertically distributed data to securely compute the joined table, allowing them to perform downstream Secure multi-party computation-based Data Analytics (SDA), such as analyzing statistical information or training machine learning models, based on the joined table. While Circuit-based Private Set Intersection (CPSI) can be used for secure data join, it inherently introduces redundant dummy rows in the joined table, which results in high overhead in the downstream SDA tasks. iPrivJoin addresses this issue but introduces significant communication overhead in the redundancy removal process, as it relies on the cryptographic primitive Oblivious Programmable Pseudorandom Function (OPPRF) and multiple rounds of oblivious shuffles. In this paper, we propose a much simpler secure data join protocol, Bifrost, which outputs (the secret shares of) a redundancy-free joined table. The highlight of Bifrost lies in its simplicity: it builds upon two conceptually simple building blocks, an ECDH-PSI protocol and a two-party oblivious shuffle protocol. The lightweight protocol design allows Bifrost to avoid the need for OPPRF. We also proposed a simple optimization named dual mapping that reduces the rounds of oblivious shuffle needed from two to one. Experiments on various datasets up to 100 GB show that Bifrost achieves $2.54 \sim 22.32\times$ speedup and reduces the communication by $84.15\% \sim 88.97\%$ compared to the state-of-the-art redundancy-free secure data join protocol iPrivJoin. In the two-step SDA pipeline (secure join and secure analytics) experiments, the redundancy-free property of Bifrost not only avoids the catastrophic error rate blowup in the downstream analytics caused by dummy rows introduced by CPSI, but also shows up to $2.80\times$ speed-up and up to $73.15\%$ communication reduction in the secure analytics process. • PrivSTD: Differentially Private Spatio-temporal Trajectory Density Data Publication (Shuzhan Ye, Zhejiang University et al.) - Spatio-temporal trajectory density data is used widely in, e.g., urban analytics, mobility, and epidemiology. While differential privacy guarantees are important enablers of the release and use of such data, it is challenging to offer privacy guarantees for high-resolution density data. Specifically, existing approaches inject noise into the spatial domain, failing to preserve inherent spatio-temporal correlations and suffering from severely reduced utility at fine granularities. We propose PrivSTD, a novel framework for differentially private release of spatio-temporal trajectory density data. PrivSTD leverages the Discrete Cosine Transform to project density data into the frequency domain, where spatial correlations and temporal smoothness are naturally captured by low-frequency components. To suppress noise-dominated frequencies, a Benjamini - Hochberg FDR - based adaptive truncation mechanism is introduced that preserves statistically significant structures without additional privacy cost. Furthermore, PrivSTD employs a control variate - enhanced Recorrupted-to-Recorrupted denoising model to reconstruct highquality density data without access to clean ground truth data. An experimental study shows that PrivSTD is capable outperforming existing methods, achieving 1.12X - 53.89X (6.12X on average) lower error across all datasets condisidered. • Efficient and Secure Range Counting over Distributed Geographic Data with Query Range Protection (Haoxin Yang, Xi'an Jiaotong University et al.) - Range counting is a core primitive in geographic information systems. When data is distributed across multiple organizations, conducting range counting raises substantial privacy concerns. Existing privacy-preserving protocols focus on protecting organizations’ datasets, but cannot simultaneously achieve efficiency, query privacy, and accuracy on overlapping data. Typical protocols process query range in plaintext for efficient point-in-range evaluation, since query-private designs rely on expensive secure comparisons. Moreover, most works assume non-overlapping datasets across organizations, which leads to huge errors in overlapping scenarios. In this paper, we propose \emph{PPRC}, the first protocol that jointly satisfies all the privacy, efficiency, and accuracy requirements. PPRC makes two key technical contributions. First, we design the \emph{Private Range Predicate (PRP)} technique that supports efficient point-in-range evaluation while protecting the query range. PRP reformulates range evaluation as encrypted membership tests, effectively replacing costly secure comparisons with faster secure multiplications. Second, we propose \emph{Oblivious Linear Counting (OLC)}, an aggregation scheme that efficiently and securely aggregates partial results from organizations with overlapping data. OLC involves only lightweight cryptographic operations and ensures that no information is leaked beyond the final range count. We theoretically analyze the accuracy, efficiency, and security of PPRC. Experiments on real-world and synthetic datasets show that PPRC achieves up to $55\times$ smaller errors and $37\times$ speedup compared to baseline protocols. • Highly-Efficient Large-Scale k-means with Individual Fairness (Shengkun Zhu, Wuhan University et al.) - Traditional k-means minimizes the sum of squared error (SSE) but may treat data points unequally, as some are assigned to significantly distant centroids. This leads to unfair outcomes in downstream tasks such as facility location planning, where each cluster corresponds to a specific share of limited resources. To address this, we modify the objective of k-means via exponential tilting, which emphasizes the impact of distant data points and yields a new objective: the tilted SSE. We propose TKM, which optimizes this objective via coordinate descent and stochastic gradient descent, and improves fairness by shifting centroids toward underrepresented groups. We adopt the within-cluster variance to quantify fairness among individuals within the same group, which provably reduces extreme disparities in outcomes. To improve large-scale data efficiency, we propose FastTKM, which uses stochastic dynamics to estimate the tilted SSE with lower computational cost. We theoretically demonstrate that, under our proposed methods, the variance decreases with t, a scaling factor that controls the degree of centroid deviation. Furthermore, our methods exhibit time and space complexities comparable to the classical Lloyd’s heuristic. Experimentally, our methods outperform six baselines in terms of clustering utility and fairness across twelve real-world datasets. In terms of efficiency, our methods achieve thousand-fold speedups in running time and reduction in memory usage, with this factor growing as the dataset size increases. • A Workload-Aware Encrypted Index for Efficient Privacy-Preserving Range Queries (Dong Wang, Chang'an University et al.) - Recent advances in workload-aware indexes have attracted growing attention for their ability to optimize index efficiency by learning query distributions. However, these architectures remain fundamentally incompatible with sensitive data scenarios that require encrypted index storage and privacy-preserving queries. Meanwhile, existing privacy-preserving solutions make it difficult for encrypted indexes to be workload-aware due to the complexity of cryptographic protocols, which prevents accurate cost estimation for a given workload. To address these limitations, this paper studies the workload-aware encrypted index for efficient privacy-preserving range queries. We propose \textsf{P$^3$RQ-Bitmap}, a \textsf{XOR}-encrypted bitmap index powered by a lightweight Pseudo Random Function (\textsf{PRF})-based comparison protocol. This index supports efficient privacy-preserving range queries while being workload-aware. Building upon this, we further propose \textsf{P$^3$RQ-WBTree}, a workload-aware encrypted tree index that optimizes query efficiency through adaptive data partitioning guided by a gradient descent-optimized cost model. The index comes with buffer and rebuild strategies to support dual updates for both data and workload. Extensive theoretical analysis and experiments demonstrate that \textsf{P$^3$RQ-WBTree} achieves at least 83$\times$ faster query performance compared to SOTA schemes.read more
☆
Thu Sep 3 · 10:45 AM - 12:15 PM
Tutorial 10 · Private LLM Inference with Homomorphic Encryption
● Tutorials
📍 Otis
Private LLM Inference with Homomorphic Encryption
Lawrence LimUniversity of California, Santa Barbara
Divyakant AgrawalUniversity of California, Santa Barbara
Amr El AbbadiUniversity of California, Santa Barbara
Graph-Based Retrieval-Augmented Generation: Applications, Challenges, Solutions, and Opportunities
Angela BonifatiLyon1 University, CNRS Liris
Yixiang FangThe Chinese University of Hong Kong, Shenzhen
Sibo WangThe Chinese University of Hong Kong
Yingli ZhouThe Chinese University of Hong Kong, Shenzhen
☆
Thu Sep 3 · 10:45 AM - 12:15 PM
Tutorial 12 · Graph Foundation Models: State of the Art and Future Directions
● Tutorials
📍 Webster
Graph Foundation Models: State of the Art and Future Directions
Alexander ZhouThe Hong Kong Polytechnic University
Haoyang LiThe Hong Kong Polytechnic University
Lei ChenHKUST, HKUST (GZ)
☆
Thu Sep 3 · 10:45 AM - 12:15 PM
Industry Session 7 · Disaggregated Memory, Shared Storage & Cloud DB Architecture
● Industry
📍 Marina Ballroom I
OceanBase Bacchus: A High-Performance and Scalable Cloud-Native Shared Storage Architecture for Multi-Cloud (Quanqing Xu, OceanBase, Ant Group et al.) - Although more databases embrace shared-storage architectures, current storage-disaggregated systems lack an optimal balance between cost and performance.OceanBase Bacchus: A High-Performance and Scalable Cloud-Native Shared Storage Architecture for Multi-Cloud (Quanqing Xu, OceanBase, Ant Group et al.) - Although more databases embrace shared-storage architectures, current storage-disaggregated systems lack an optimal balance between cost and performance. B+-tree - based shared storage struggles with frequent in-place updates under high concurrency; disaggregated designs still require a scalable shared-log path for multi-writer OLTP. This paper presents OceanBase Bacchus, a production grade disaggregated shared-storage architecture for OceanBase.We foreground two technical elements: (i) service-oriented PALF logging with sharded log streams and multiple Read-Write (RW) leaders, where partition-local ordering is composed with distributed 2PC across stream leaders, breaking the single-writer bottleneck of classic shared-log designs while preserving ACID, and (ii) a Shared Block Cache Service that pools macro-blocks within an availability zone, a less common alternative to purely per-node caches and motivated by stateless compute scaling. Cloud object storage backs the LSM-tree capacity tier for cost, but we do not treat “LSM + object storage” as the headline novelty (precedents exist). Bacchus’s value is integrating and hardening these mechanisms in a shipping system, demonstrated by credible OLTP/OLAP performance and substantial storage-cost reductions. The architecture decouples modules and enables elastic scaling of compute, cache, and storage independently. Experimental results show that OceanBase Bacchus matches or exceeds HBase and PolarDB in OLTP and significantly outperforms StarRocks in OLAP. With multi-cloud support, the system retains high availability and competitive performance while cutting storage costs by 59% (OLTP) and 89% (OLAP). • Why We Created Yet Another Memory Framework: Understanding MGA’s Role in Next-Gen Database Systems (Vikramraj Sitpal, Oracle et al.) - Despite the presence of multiple memory regions in modern database systems, supporting an efficient form of memory remains a challenge under production constraints. In enterprise-grade data systems, existing abstractions impose a trade-off between coarse-grained global sharing and strict process isolation, resulting in data copying, memory fragmentation, and limited support for controlled sharing. These challenges become more pronounced as workloads grow more diverse, and systems must tolerate process failures while maintaining predictable performance. This paper introduces the Managed Global Area (MGA), a scoped shared-memory abstraction in Oracle AI Database that addresses these limitations. MGA allows components to explicitly define allocation source, membership, and coordination semantics across selected processes while integrating with a production database engine. Unlike fully shared memory regions in Oracle, such as the System Global Area (SGA), MGA supports dynamic process membership and modular memory usage without imposing system-wide visibility. We evaluate MGA on analytical and AI workloads that stress shared-memory execution, including TPC-H hash joins and ONNX Runtime inference. Under concurrent execution, MGA reduces latency for join-intensive TPC-H queries by up to 35%. For ONNX-based inference, MGA-enabled model sharing reduces memory footprint by up to 90% and lowers large-model inference latency by up to 37%. These results demonstrate that dynamically scoped shared memory can improve both efficiency and predictability in production database systems. • K2+: A Multi-Region OLTP Database with Disaggregated Storage for High Availability and Strong Consistency (Haoze Song, The University of Hong Kong et al.) - Cloud computing has fundamentally reshaped database system architectures by enabling the decoupling of compute and storage resources and supporting global, multi-region deployments. However, effectively integrating storage-disaggregated architectures with multi-region OLTP processing remains challenging, particularly when maintaining transactional consistency and performing cross-region replication. This paper presents K2+, a multi-region transactional key-value store. K2+ achieves fine-grained data partitioning across regions by combining region-local storage pools with independent, region-specific commit logs. To support replication within and across regions, K2+ incorporates regional partition management with dedicated cross-region replication middleware. This architecture allows different regions to split, merge, and place physical partitions independently while preserving high reliability, high availability, and strong consistency. • SERON: Smart Query Router for Multi-Primary Cloud-Native Databases with Shared Storage (Yuwei Huang, Tsinghua University et al.) - Cloud-native databases have become a major deployment paradigm for commercial applications and services in recent years. One of the most promising development trends in cloud-native databases is to support multi-primary architecture with shared storage layers. However, when different writer nodes simultaneously access overlapping data ranges, query conflicts naturally arise, potentially leading to a significant performance drop. Therefore, it is of great importance to design query routers that can smartly separate incoming workloads to mitigate inter-node conflicts, which is a problem not fully discussed in the existing literature. To address this issue, in this paper, we present SERON, a smart query routing system customized for multi-primary cloud-native databases with shared storage. The system aims to minimize conflicts by directing each query to an optimal compute node. To this end, SERON generates workload-aware logical routing rules at different granularities using a two-stage framework, actively detects and handles physical data collisions through judicious conflict modeling, and constantly adapts its routing policies to provide the best runtime performance. The system has been integrated into GaussDB and deployed in real-world scenarios. Extensive experiments verify the effectiveness and cost-efficiency of SERON, which achieves up to 61.7\% execution time reduction compared to the best baseline. • Narwhal: Breaking the Local Boundary via Disaggregated Memory Scheduling for Alibaba AnalyticDB (Jian Zhou, Alibaba Cloud et al.) - Cloud OLAP workloads are bursty and memory-hungry. Per-machine DRAM caps and co-scaled CPU/memory provisioning are fragile. They lead to low utilization, slow autoscaling, and transient OOMs. Prediction-driven schedulers and bin-packing optimizers mitigate but struggle with rigid per-machine memory boundaries and forecasting errors, Existing RDMA or OS-level remote-memory approaches either assume single-tenant trusted clusters (lacking hardware isolation and QoS) or treat remote DRAM as opaque swap. We present Narwhal, an RDMA-co-designed analytical database. It jointly schedules local DRAM and a shared remote memory pool to trade consolidation against bounded remote borrowing. It enforces hardware-level multi-tenant isolation with per-tenant RDMA Protection Domains, quotas, and QoS. It uses a database-aware memory allocator with a three-tier cache and a pressure-threshold invariant to avoid OOMs and protect metadata. Integrated into a production engine, Narwhal eliminates OOMs, boosts memory overselling from 92.67% to 121.89% (+29.22) and node utilization from 78.16% to 96.63% (+18.47), while limiting average remote memory borrowing to just 1.63%. • Reaching the Pinnacle of TPC-DS: Co-design of Architecture, Executor, and Storage in TDSQL (Yiteng Chu, Renmin University of China et al.) - Enterprise data explosion and the urgent industry demand for real-time complex multidimensional analytics require OLAP databases to be highly scalable, efficient, and cost-effective. Though diverse solutions (shared-nothing MPP databases, cloud-native decoupled systems, in-process analytical engines) exist with respective strengths, they all have critical inherent flaws. In response, this paper presents TDSQL, a distributed OLAP database system developed by Tencent. We leverage its native architectural advantages, analyze the merits and drawbacks of state-of-the-art systems, and elaborate on the rationale behind our technical solution selection and proprietary innovations tailored for TDSQL. Specifically, built upon the traditional MPP execution framework, TDSQL incorporates the Forward Node mechanism to enhance scalability. By optimizing parallel execution, runtime filter strategies, and designing and implementing a vectorized execution engine, TDSQL achieves rapid response to large-scale complex queries. Experimental results based on the TPC-DS benchmark demonstrate that TDSQL ranks first among publicly reported systems. In a cluster configuration for 10,000 GB data size, TDSQL achieves a score of 72.6 million QphDS, which is 1.81 times and 3.82 times the scores of the second- and third-highest-performing database systems in public TPC-DS results, respectively, while offering a 79% and 37% lower cost per 1000 QphDS. • Hermes at Scale: Powering Distributed Queries with a Unified Memory Fabric (Tim Gubner, Huawei et al.) - MySQL is a widely used database management system. However, it has clear deficits for analytical workloads. To improve analytical performance, we introduced Hermes as an accelerator for MySQL. Since many workloads easily fit a single node/machine, we developed Hermes for this case. However, Hermes on a single node has a major limitation: Storage space is limited to a single-node, even worse, to main memory of a single node. In this paper, we address this problem by making Hermes distributed. New nodes can be added that bring additional resources, i.e. memory and compute, to the cluster. Hermes' storage can automatically expand to the new nodes. Our design operates as logically "Shared Memory" via an abstraction layer (Memcom). We argue that modern interconnects make shared memory feasible without extreme performance penalties. To further minimize sending data over the network, we annotate larger data blocks with localities, which allows reader nodes to locate and read local data. In addition, we extended Hermes' query execution to multiple nodes as well. This is implemented via a new Planner that splits the query plan into tasks and defines how query execution is parallelized (MPP intra-query parallelism). Each task is separated via data sharing operators (Portals) that write intermediates to a shared memory pool (Memcom). We extended scans to pick up row groups (our granularity of data storage) based on their locality. Therefore, distributed scans will preferably read local data (thanks to our predefined localities) and only when there is no more local data to be read consider remote data. We experimentally show that our approach is not only competitive in terms of performance but outperforms well-known open-source systems by more than 11x. Compared to industrial-grade cloud systems, Hermes also outperforms by 1.5x.read more
☆
Thu Sep 3 · 10:45 AM - 12:15 PM
Panel 3 · Data Agents: Rethinking Data Systems in the AI Agent Era
● Panels
📍 Grand Ballroom A+B
Data Agents: Rethinking Data Systems in the AI Agent Era
Guoliang LiTsinghua University
Yuyu LuoHKUST (Guangzhou)
Carsten BinnigTU Darmstadt
Samuel MaddenMIT
Fatma ÖzcanGoogle
Aditya ParameswaranUC Berkeley
Nan TangHKUST (Guangzhou)
Immanuel TrummerCornell University
Eugene WuColumbia University
Built for you by(617)-PATRICK, Champion AI speaker
Research 43 · Interactive Data Exploration and Debugging
● Research
📍 Grand Ballroom E
I-Rex: An Interactive Debugger for SQL (Yihao Hu, Duke University et al.) - SQL is declarative in nature and rich in its features.I-Rex: An Interactive Debugger for SQL (Yihao Hu, Duke University et al.) - SQL is declarative in nature and rich in its features. Writing semantically correct SQL queries and finding logical bugs in SQL are not easy, even for experienced programmers, who are often used to the mindset of working with general-purpose programming languages (GPLs). While there are many GPL debuggers, SQL debugging has received much less attention. In this paper, we present I-Rex, a SQL debugger that enables users to inspect the logical execution of SQL queries visually and interactively to identify and potentially fix logical bugs in the queries. I-Rex draws analogies to the debugging paradigm of GPLs (e.g., stepping, watchpoints, etc.), making it easier for programmers to adopt. However, unlike debugging GPLs, which involves executing the underlying program in full to the point of interest, I-Rex allows users to jump to arbitrary points of interest by leveraging the power of the database systems, through selective materialization and query rewrites. To simplify deployment, I-Rex acts as a lightweight middleware on top of the database system; it imposes no overhead to prepare a database for debugging and maintains no state in the database systems during debugging sessions. We demonstrate the effectiveness of I-Rex through performance experiments as well as a user study in an educational setting. • Efficient Query Repair for Aggregate Constraints (Shatha Algarni, University of Southampton et al.) - In many real-world scenarios, query results must satisfy domain-specific constraints, e.g., a minimum percentage of interview candidates selected based on their qualifications should be female. These requirements can be expressed as constraints over an arithmetic combination of aggregates evaluated on the result of the query. In this work, we study how to repair a query to fulfill such constraints by modifying its filter predicates. We introduce a novel query repair technique that leverages bounds on sets of candidate solutions and interval arithmetic to efficiently prune the search space. We demonstrate experimentally, that our technique significantly outperforms baselines that consider a single candidate at a time. • Testing Graph Databases via Transformations Between Fixed-Length and Variable-Length Queries (Jinxin Gui, Nanjing University et al.) - The ability of Graph Database Management Systems (GDBMSs) to efficiently store and query graph data has led to their widespread success. Unlike relational databases, GDBMSs model data as graphs and support expressive queries through graph traversal. Among the core functionalities, fixed-length and variable-length queries are particularly critical, as they underscore fundamental differences from traditional relational query execution. However, the correctness of such queries is notoriously difficult to ensure due to the intricate query semantics and the complexity of underlying optimizations like worst-case optimal joins. This paper presents a novel metamorphic testing approach named PATHTest that exploits result-equivalent transformations between fixed-length and variable-length queries. Specifically, PATHTest incorporates an iterative query generator that supports the generation of diverse and non-empty variable-length queries. During the mutation process, three transformation rules embedded within PATHTest help capture result-equivalent patterns between fixed- and variable-length queries, enhancing its capability to uncover both logical bugs and unexpected errors. Extensive evaluation on PATHTest across seven real-world, widely-used GDBMSs demonstrates the superiority of PATHTest, with 41 previously unknown bugs revealed, among which 24 are logic bugs, and 17 correspond to unexpected errors. To note, all 41 bugs are beyond the reach of the seven existing state-of-the-art testing approaches. By now, 29 of the 41 bugs have been confirmed, with 11 already fixed. Such evaluation results demonstrate the effectiveness and uniqueness of PATHTest in detecting bugs missed by existing testing approaches, contributing to the reliability of modern GDBMSs. • Exploring Exploratory Querying (Marcelo Arenas, Universidad Católica de Chile et al.) - We need to rethink how users understand and develop queries. The growing diversity of users, the increasing complexity of query languages and data architectures - now aided by tools like LLMs - are challenging the traditional view of a highly-trained user writing queries in a controlled environment. Query formulation has become a more exploratory endeavor that needs to be researched and supported: an iterative cycle of designing, debugging, and maintaining queries. To ground this vision, we present an empirical analysis of query logs from the Wikidata Query Service, revealing common patterns of iterative query modification. Based on these findings, we propose a concrete research program with hypotheses, user studies, and research questions for query languages, engines, and interfaces. Our contributions include a curated query session dataset, a classification of exploratory-query patterns, and a roadmap for building system-level support for exploratory querying. • Decisionhouse: Prescriptive Analytics in the Data Stack (Matteo Brucato, OSM Data et al.) - Data platforms have evolved by making data-intensive workloads native: SQL and query optimizers eliminated bespoke data-retrieval programs; Lakehouses added first-class support for ML training and serving over the same data. Prescriptive analytics (computing optimal actions subject to constraints over data) is equally data-intensive, yet remains outside the platform: every optimization problem requires a hand-built pipeline from data extraction to solver invocation, rebuilt from scratch whenever the data or the requirements change. We propose Decisionhouses, a new class of data infrastructure that makes prescriptive analytics native. A Decisionhouse provides (i) DeQL (Decision Query Language), a declarative SQL extension where users express decision problems over relational data; (ii) automatic formulation selection that exploits query and data semantics to pick the right problem class and solver - a choice that can change a query's complexity class from NP-hard to polynomial; and (iii) end-to-end integration of optimization into the data platform, from query parsing through solver execution. Decisionhouses can help address several challenges that have kept optimization outside data platforms, including pipeline brittleness, formulation expertise, structural blindness, and scalability cliffs, and make decision-making as accessible as querying data.read more
☆
Thu Sep 3 · 1:45 PM - 3:15 PM
Research 44 · Lakehouse and Open Table Formats
● Research
📍 Grand Ballroom D
FlatStor: An Efficient Embedded-Index Based Columnar Data Layout for Multimodal Data Workloads (Chi Zhang, Shanghai Jiao Tong University et al.) - Modern data lakes have become essential for storing, managing, and analyzing massive amounts of heterogeneous data.FlatStor: An Efficient Embedded-Index Based Columnar Data Layout for Multimodal Data Workloads (Chi Zhang, Shanghai Jiao Tong University et al.) - Modern data lakes have become essential for storing, managing, and analyzing massive amounts of heterogeneous data. As production data increasingly exhibits multimodal storage characteristics and multi-purpose access patterns, efficient management of such complexities becomes critical. However, current hybrid storage system-based data lakes face persistent challenges, including synchronization overhead, data correlation disruption, and escalating storage costs due to the involvement of multiple underlying storage systems. While columnar storage, central to data lakes, addresses hybrid-system inefficiencies, it struggles with the complexities of multimodal data storage and multi-purpose access. To tackle these challenges, we analyze access patterns across various scenarios and assess the issues in storing multimodal data. Based on these insights, we propose FlatStor, a FlatBuffers-based columnar Storage format with embedded indexing. It supports point access through indexing and handles multimodal data by vertically partitioning and treating each modality as a byte stream for storage. It also applies FSST compression, reducing storage overhead significantly. Benchmark evaluations reveal that FlatStor reduces the access latency by 99.6% and the storage overhead by 91.3% compared to Parquet in inference workloads. Furthermore, FlatStor outperforms LanceV2 with a 41.3% latency improvement, maintaining minimal additional overhead • Active Data Lakes: Regaining Physical Data Independence Without Losing Interoperability (Pascal Ginter, Technical University of Munich et al.) - Data lakes aim to avoid vendor lock-in and enable interoperability between different query engines on a single copy of data. While early data lakes were only collections of files in various formats, they have since evolved to incorporate some features traditionally associated with relational databases. Today, Apache Parquet is the de facto standard file format for relational data in data lakes. This standardization is fundamental to interoperability, but it comes at the cost of physical data independence because query engines integrate tightly with Parquet. As a result, adoption of novel approaches in the areas of file formats, access paths, and storage media has been limited. We propose the Active Data Lake architecture as a way to restore physical data independence and demonstrate its potential experimentally through three example optimizations. • Interoperable ACID Transactions for Open Table Formats (Tobias Goetz, Technical University of Munich et al.) - Open Table Formats (OTFs) enable interoperability across systems by using object stores as a shared storage layer. They support OTF HTAP, in which complex queries execute concurrently with updates on the object store. However, existing OTFs provide transactional guarantees only for single-table transactions. Supporting transactions that access multiple tables typically requires external coordination services, which compromise interoperability and deployment independence. In this paper, we show that this trade-off is unnecessary. We present OTF-native mechanisms for multi-table atomicity, consistency, and isolation that rely solely on object-store primitives. We realize them in LakeVilla, a prototype that provides full ACID transactions while remaining compatible with existing OTF deployments. Our evaluation combines formal verification, extensive benchmarks, and an interoperability case study with Trino, showing that LakeVilla delivers strong transactional guarantees with low overhead, scalable concurrency, and compatibility with existing analytics engines. • LakeHelm: Zero-Shot Lakehouse Advisor for Joint Engine-Format Selection and Configuration (Zhongwei Xu, university of michigan et al.) - Lakehouse systems unify the strengths of data lakes and data warehouses and are rapidly becoming a dominant architecture for analytic data management. The lakehouse architecture decouples system design into interoperable subsystems - execution engines(e.g., Spark, Trino, Presto) and table formats (e.g., Delta Lake, Iceberg, Hudi) - giving users flexibility to mix and match. However, jointly selecting and configuring these subsystems is hard: subsystem choices and configurations interact in complex ways, and online trial-and-error is costly (or infeasible when migration is required). Although there is extensive work on database tuning, most methods target a single subsystem and thus miss cross-dependencies; many also rely on iterative online tuning that is prohibitively expensive. In this work, we present LakeHelm, a zero-shot lakehouse advisor that jointly recommends an engine - format pair and its configuration without online feedback. LakeHelm uses a dual-gate Mixture-of-Experts model: separate gates specialize in engine and format choices, and experts learn configuration surrogates for each subsystem combination. To enhance generalization, we augment training data with generated SQL templates and synthesized workloads,layered atop collected runs that explore the configuration space. Evaluated across five standard benchmarks (TPC-DS, TPC-H, JOB, SSB, SSB-Flat), LakeHelm delivers competitive execution times - averaging 1.35× speedup over a fixed overall-best lakehouse configuration across a large number of workload variations. It achieves this via zero-shot inference on unseen workloads in seconds, without costly online experimentation. • Storing and Indexing Multiple Tables by Interesting Orderings: For Efficient Joins, Groupings, and Updates in Relational Databases (Wenhui Lyu, University of Wisconsin - Madison et al.) - Relational database systems often face a trade-off between supporting multi-table queries and frequent updates. Materialized join views can drastically speed up queries, but they slow down updates and may consume significant storage. Conversely, query-time joins over tables and their indexes optimizes update performance at the cost of query latency. Our recent study of two-table joins introduced “merged indexes” (a form of multi-table index) to break this trade-off, approaching the query performance of materialized views without sacrificing the update efficiency of traditional single-table indexes. This study generalizes this technique to “order-sharing pipelines” - multi-table joins and grouping operations on shared keys. By incorporating interesting orderings into the physical database design, merged indexes partially pre-compute these order-sharing pipelines, closely approaching the query performance of materialized views. Meanwhile, they match traditional indexes on update performance.read more
☆
Thu Sep 3 · 1:45 PM - 3:15 PM
Research 45 · Differential Privacy
● Research
📍 Grand Ballroom C
Understanding Disclosure Risk in Differential Privacy with Applications to Noise Calibration and Auditing (Patricia Guerra-Balboa, Karlsruhe Institute of Technology et al.) - Differential Privacy (DP) is widely adopted in data management systems to enable data sharing with formal disclosure guarantees.Understanding Disclosure Risk in Differential Privacy with Applications to Noise Calibration and Auditing (Patricia Guerra-Balboa, Karlsruhe Institute of Technology et al.) - Differential Privacy (DP) is widely adopted in data management systems to enable data sharing with formal disclosure guarantees. A central systems challenge is understanding how DP noise translates into effective protection against inference attacks, since this directly determines achievable utility. Most existing analyses focus only on membership inference---capturing only a threat---or rely on reconstruction robustness (ReRo). However, under realistic assumptions, we show that ReRo can yield misleading risk estimates and violate claimed bounds, limiting their usefulness for principled DP calibration and auditing. This paper introduces reconstruction advantage, a unified risk metric that consistently captures risk across membership inference, attribute inference, and data reconstruction. We derive tight bounds that relate DP noise to adversarial advantage and characterize optimal adversarial strategies for arbitrary DP mechanisms and attacker knowledge. These results enable risk-driven noise calibration and provide a foundation for systematic DP auditing. We show that reconstruction advantage improves the accuracy and scope of DP auditing and enables more effective utility-privacy trade-offs in DP-enabled data management systems. • Composition for Pufferfish Privacy (Jiamu Bai, Penn State University et al.) - When creating public data products out of confidential datasets, inferential/posterior-based privacy definitions, such as Pufferfish, provide compelling privacy semantics when the data have correlations. However, such privacy definitions are rarely used in practice because they do not always compose. For example, it is possible to design algorithms for these privacy definitions that have no leakage when run once but reveal the entire dataset when run more than once. We prove necessary and sufficient conditions that must be added to ensure linear composition for Pufferfish mechanisms, hence avoiding such privacy collapse. These extra conditions turn out to be differential privacy-style inequalities, indicating that achieving both the interpretable semantics of Pufferfish for correlated data and composition benefits requires adapting differentially private mechanisms to Pufferfish. We show that such translation is possible through a concept called the $(a^\star(b), b)$-influence curve, and many existing differentially private algorithms can be translated with our framework into a composable Pufferfish algorithm. We illustrate the benefit of our new framework by designing composable Pufferfish algorithms for Markov chains that significantly outperform prior work. • Measuring Database Unfairness via Dependency Quantification Under Differential Privacy (Mariia Vologdin, The Hebrew University et al.) - Differential privacy (DP) has become the de facto standard for protecting sensitive data, providing strong guarantees that published statistics or models reveal limited information about any individual. However, privacy noise and restricted data access make it increasingly difficult to assess the fairness and reliability of private datasets. In this paper, we propose a formal framework for quantifying data unfairness under DP. We identify three core desiderata for unfairness measures based on previous work: positivity, monotonicity, and DP computability. We further instantiate them through three complementary measures: (1) a mutual information - based measure with a total variation distance proxy suitable for DP, (2) a data-repair - based measure approximated via a reduction to weighted MaxSAT, and (3) a top-$k$ tuple contribution measure that isolates the most influential records in fairness violations. We design privacy-preserving algorithms and analyze their sensitivity, accuracy, and efficiency. Extensive experiments on multiple real-world datasets demonstrate that our proposed measures faithfully approximate their non-private counterparts, effectively quantify bias under privacy constraints, and provide insights for data management. • Fast and Private Max-Sum Diversification (Ron Zadicario, Tel Aviv University et al.) - Result diversification is crucial for generating informative, non-redundant data summaries and query outputs. Although its various formulations have been extensively studied across an array of data-driven disciplines, existing methods fail to address the privacy concerns that arise when the underlying data is sensitive. In this work, we initiate the study of result diversification under differential privacy, focusing on the max-sum diversification (MSD) problem, a widely adopted model with the objective of maximizing a linear combination of a submodular function, quantifying relevance, and the sum of pairwise distances between selected items, quantifying diversity. We propose differentially private algorithms for MSD under both cardinality and matroid constraints, achieving nearly optimal utility guarantees. At the same time, we design more efficient algorithms that maintain strong guarantees. Notably, the proposed algorithms are faster than existing non-private methods, making them appealing even in non-private settings. Experimental evaluations on real-world datasets demonstrate that the proposed approach achieves utility comparable to that of non-private baselines even under strong privacy guarantees, and significantly improves execution times for cardinality constraints. • PrivMDC: Leveraging Multi-Dimensional Correlations to Answer Differentially Private Range Queries (Serafim Costa, Universidade Federal do Ceará et al.)read more
☆
Thu Sep 3 · 1:45 PM - 3:15 PM
Research 46 · Transaction Management
● Research
📍 Commonwealth Ballroom C
Fast Verification of Strong Database Isolation (Zhiheng Cai, Tsinghua University et al.) - Strong isolation guarantees, such as serializability and snapshot isolation, are essential for maintaining data consistency and integrity in modern databases.Fast Verification of Strong Database Isolation (Zhiheng Cai, Tsinghua University et al.) - Strong isolation guarantees, such as serializability and snapshot isolation, are essential for maintaining data consistency and integrity in modern databases. Verifying whether a database upholds its claimed guarantees is increasingly critical, as these guarantees form a contract between the vendor and its users. However, this task is challenging, particularly in black-box settings, where only observable system behavior is available and often involves uncertain dependencies between transactions. In this paper, we present VeriStrong, a fast verifier for strong database isolation. At its core is a novel formalism called hyper-polygraphs, which compactly captures both certain and uncertain transactional dependencies in database executions. Leveraging this formalism, we develop sound and complete encodings for verifying both serializability and snapshot isolation. To achieve high efficiency, VeriStrong tailors SMT solving to the characteristics of database workloads, in contrast to prior general-purpose approaches. Our extensive evaluation across diverse benchmarks shows that VeriStrong not only significantly outperforms state-of-the-art verifiers on the workloads they support, but also scales to large, general workloads beyond their reach, while maintaining high accuracy in detecting isolation anomalies. • Pisco: An Isolation Bug Case Reduction and Deduplication Framework (Siyang Weng, East China Normal University et al.) - Practical implementations of Isolation levels (ILs) might deviate from their theoretical definitions, resulting in isolation bugs. Triggering isolation bugs usually requires concurrent test cases, which causes high debugging complexity. These test cases typically contain numerous irrelevant operations, resulting in large and redundant raw cases that violate the three essential criteria for bug report submission, i.e., reproducibility, conciseness, and uniqueness. Achieving these criteria involves three key challenges: 1) deterministically reproduce bugs despite high concurrency; 2) efficiently reduce cases without losing bug-triggering operations; 3) accurately distinguish unique bugs from the bug reports having complex transaction behaviors. To address these challenges, we propose an isolation bug case reduction and deduplication framework Pisco. First, we propose to simulate the DBMS's internal state to infer the order of conflicting operations for deterministic bug reproduction. Second, we introduce a dependency-aware divide-and-conquer strategy for efficient case reduction. Finally, we design a domain knowledge-driven, multi-agent collaboration framework for accurate bug deduplication. Extensive experiments show that Pisco reduces the cases to their minimal forms in just 20.0%/33.3% of the time required by C-Reduce/DDMin and has a deduplication ratio of up to 91.6%. • SunStorm: Geographically distributed transactions over Aurora-style systems (Cuong Nguyen, University of Maryland, College Park et al.) - There are two main approaches to scaling transactional database workloads: (1) a shared-nothing architecture with distributed transaction processing, or (2) an Aurora-style shared-storage architecture with separate compute and storage layers that scale independently. In option (2), the compute layer typically contains a single writer node and all other compute nodes are read-only. This may lead to scalability limits for write-intensive workloads, and introduces communication latency for write transactions that initiate far from the writer node. However, shared-nothing systems must pay the overhead of distributed coordination and commit protocols. In this paper, we discuss the design of a more scalable version of Aurora-style systems which supports multiple writer nodes managing geographically partitioned data. It yields many of the efficiency benefits of Aurora-style systems while removing the scalability bottleneck. Furthermore, geographic partitioning improves latency by over an order of magnitude for global applications in which clients from across the world can experience local write performance. • Orca: Flexible Quorums Meet Dynamic Quorums (Kevin Dharmawan, Stony Brook University et al.) - Large-scale data management systems rely on fault-tolerant protocols to ensure availability and consistency in the presence of node failures. However, standard protocols require majority quorums in every phase, making them impractical in environments with frequent failures, such as IoT networks. This paper presents Orca, a crash-fault-tolerant consensus protocol that integrates Raft-style log replication with Flexible Paxos quorum construction to maintain small replication quorums. Orca exploits the observation that node failures are often independent and temporally separated: by assuming that at most k nodes fail within a window too short for the system to detect the first failure, Orca fixes the replication quorum at k+1 nodes. As failures are detected, faulty nodes are temporarily excluded from the voting set and the election quorum is reduced accordingly, allowing the system to continue making progress as failures accumulate. Our evaluation across diverse workloads and fault scenarios demonstrates that Orca tolerates more failures than Raft and FlexiRaft while incurring negligible overhead. • Swan: Hybrid MVCC Management for Efficient Transaction Processing in LSM-Tree-Based Key-Value Stores (Yang Guo, The Chinese University of Hong Kong et al.) - Distributed NewSQL databases build SQL query engines on top of LSM-tree-based key-value stores to achieve fast and scalable online transaction processing. This requires efficient transaction support in the underlying storage engine, typically implemented using multi-version concurrency control (MVCC). However, existing in-memory MVCC schemes struggle with large write transactions due to high memory consumption and commit overhead, which can degrade insertion performance or even cause system crashes. Out-of-memory MVCC schemes alleviate these issues but suffer from expensive visibility checks under high concurrency, leading to performance degradation. In this paper, we propose a hybrid MVCC scheme that combines the fast visibility checks of in-memory MVCC with the low commit latency and memory usage of out-of-memory MVCC, enabling efficient transaction management. To integrate this design with LSM-tree-based key-value stores, we introduce several memory management techniques, including a transaction-aware data separation strategy, a stash region for uncommitted data, and a concurrent memtable flushing mechanism. We implement a prototype key-value store and extend YCSB to evaluate our design under various workloads. Experimental results demonstrate that our hybrid MVCC scheme combines the strengths of both approaches and delivers improved performance.read more
☆
Thu Sep 3 · 1:45 PM - 3:15 PM
Research 47 · Query Execution and In-Database Analytics
● Research
📍 Commonwealth Ballroom B
LIO: A lightweight and interpretable query optimizer based on an evolutionary forest (Chen Ye, Hangzhou Dianzi University et al.) - Learning-based query optimizers have shown significant advantages in generating high-quality query plans.LIO: A lightweight and interpretable query optimizer based on an evolutionary forest (Chen Ye, Hangzhou Dianzi University et al.) - Learning-based query optimizers have shown significant advantages in generating high-quality query plans. In these optimizers, query plans are represented at different level of granularity, and learning-based models are used to learn the relationship between query plans and execution times based on the past experience. Thus, efficient query plans can be generated for given queries. However, these optimizers often struggle to achieve a balance between model efficiency and prediction accuracy. In this paper, we propose a lightweight and interpretable query optimizer LIO based on an evolutionary forest. LIO employs a genetic programming algorithm to automatically explore optimal feature combinations for a random forest, balancing model usage costs, prediction accuracy, and interpretability. The outputs of the evolutionary forest serve as interpretability aids, guiding users in dynamically adding enhanced hint sets, which in turn improves optimization performance. Additionally, two pruning strategies are developed to reduce both the number and depth of the trees in the forest, significantly enhancing rule interpretability while maintaining an acceptable level of performance loss. Extensive experiments validate that LIO outperforms state-of-the-art optimizers in terms of prediction accuracy, total runtime, and interpretability. • The Data World Is Not Flat: Efficient Factorized Execution for Relational Systems (Stefan Lehner, TUM et al.) - Relational database management systems (RDBMSs) struggle with efficient execution of graph workloads, particularly those involving many-to-many joins with large intermediate results. Such joins are common in modern applications such as social networks, recommendation engines, and knowledge graphs. Traditional RDBMSs are optimized for selective joins, while the typical result blow-up in graph analytics leads to exceptionally long query times or even system crashes due to out-of-memory errors. Since graph and relational data are often intertwined, a unified solution can simplify system architecture and improve performance. To address this, we propose a novel code-generating engine with factorization, which represents tuples in a hierarchical form. Our approach enables intra-query-parallelized query execution on factorized representations and generates code to overcome their CPU-unfriendly layout. Additionally, we extend current factorized approaches by bottom-inserts to make joins reorderable. Our factorized approach can outperform traditional tuple representations and state-of-the-art RDBMSs by orders of magnitude on synthetic workloads and benchmarks as our experiments show. Moreover, its potential integrability into traditional RDBMSs enables efficient processing of complex, highly redundant queries, offering a unified and scalable solution for modern workloads. • Towards Efficient Random-Order Enumeration for Join Queries (Pengyu Chen, Harbin Institute of Technology et al.) - In many data analysis pipelines, a basic and time-consuming process is to produce join results and feed them into downstream tasks. Numerous enumeration algorithms have been developed for this purpose. To be a statistically meaningful representation of the whole join result, the result tuples are required to be enumerated in uniformly random order. However, existing studies lack an efficient random-order enumeration algorithm with a worst-case runtime guarantee for (cyclic) join queries. In this paper, we develop an efficient random-order enumeration algorithm for join queries with no large hidden constants in its complexity, achieving expected $O(\frac{\mathrm{AGM}(Q)}{|Res(Q)|}\log^2|Q|)$ delay, $O(\mathrm{AGM}(Q)\log|Q|)$ total running time after $O(|Q|\log|Q|)$-time index construction, where $|Q|$ is the size of input, $\mathrm{AGM}(Q)$ is the AGM bound, and $|Res(Q)|$ is the size of the join result. We prove that our algorithm is near-optimal in the worst case, under the combinatorial $k$-clique hypothesis. Our algorithm requires no query-specific preprocessing and can be flexibly adapted to many common database indexes with only minor modifications. We also devise non-trivial techniques to speed up enumeration and reduce memory usage, and present an experimental study of their impact on our algorithm. The experimental results show that our algorithm, enhanced with the proposed techniques, significantly outperforms existing state-of-the-art methods. • NeurIDA: Dynamic Modeling for Effective In-Database Analytics (Lingze Zeng, National University of Singapore et al.) - Relational Database Management Systems (RDBMS) manage complex, interrelated data and support a broad spectrum of analytical tasks. With the growing demand for predictive analytics, the deep integration of machine learning (ML) into RDBMS has become critical. However, a fundamental challenge hinders this evolution: conventional ML models are static and task-specific, whereas RDBMS environments are dynamic and must support diverse analytical queries. Each analytical task entails constructing a bespoke pipeline from scratch, which incurs significant development overhead and hence limits the wide adoption of ML in analytics. We present NeurIDA, an autonomous end-to-end system for in-database analytics that dynamically “tweaks” the best available base model to better serve a given analytical task. In particular, we propose a novel paradigm of dynamic in-database modeling to pre-train a composable base model architecture over the relational data. Upon receiving a task, NeurIDA formulates the task and data profile to dynamically select and configure relevant components from the pool of base models and shared model components for prediction. For a friendly user experience, NeurIDA supports natural language queries; it interprets user intent to construct structured task profiles and generates analytical reports with dedicated LLM agents. By design, NeurIDA enables ease-of-use and yet effective and efficient in-database AI analytics. Extensive experiment studies show that NeurIDA consistently delivers up to 12% improvement in AUC-ROC and 25% relative reduction in MAE across ten tasks on five real-world datasets. • EcoTable: Cost-effective Table Integration in Data Lakes for Natural Language Queries (Yuhui Wang, Beijing Institute of Technology et al.) - The diverse formats of CSV and Parquet files in data lakes pose a significant challenge to traditional ETL, which relies on data engineers to pre-define a target database schema and build a complex pipeline for data integration. Moreover, with this approach, the integrated data often cannot support various analytical needs, as the predefined schema does not necessarily satisfy the table format or join relationships required to answer unforeseen queries. To address this, we propose EcoTable, the first natural language-based data integration framework. Given a set of user-specified natural language queries, EcoTable automatically integrates the tables into a form that adequately supports the corresponding SQL queries. EcoTable achieves this by leveraging the semantic understanding and complex reasoning capabilities of Large Language Models (LLMs). Moreover, EcoTable addresses the scalability and cost issues introduced by expensive LLM inferences with a set of novel ideas. First, EcoTable introduces a graph to represent the overall search space, where nodes represent tables and edges carry weights indicating join likelihood produced by a lightweight deep learning model. On top of this graph data structure, EcoTable designs three components to achieve our goal: (1) the table identification layer aims to identify relevant tables via a two-stage schema linking based on user queries; (2) the graph-based validation layer aims to discover significant join paths, including necessary data transformations and bridging tables, by modeling the problem as Steiner tree searches; and (3) the table transformation layer generates transformation code to implement the joins using LLMs. We construct 4 real-world benchmark datasets with more than 200 queries. Extensive experiments demonstrate that EcoTable outperforms the state-of-the-art baselines, increasing accuracy by more than 30% and cutting LLM invocation costs by 5 times. • Remora: Scale-out Deterministic Execution for Smart Contracts (Zhengqing Liu, Imperial College London et al.) - Modern blockchains rely on a modular architecture that decouples consensus from execution. Recent advances in consensus algorithms have shifted the bottleneck to the execution layer, which must deterministically follow the consensus order and handle increasingly complex, compute-intensive smart contracts. We identify that single-node validators cannot keep up, motivating the need for a scale-out design. We design Remora, a scale-out smart contract execution engine. Remora adopts an efficient asymmetric architecture with centralized transaction dispatching and distributed execution, and depends on an object versioning scheme with a strict ownership model to guarantee deterministic scale-out execution. Remora achieves up to 3x throughput improvement compared to state-of-the-art deterministic execution schemes, scales up to 250k TPS matching modern consensus performance, and reduces latency by up to 5ms. We also show that Remora elastically adapts to bursty workloads and dynamic access patterns using real-world traces. Remora's main performance benefits come from a novel stateless-stateful separation during smart contract execution, which overlaps the execution of state-independent tasks with consensus, and a new locality-aware and load-balanced scheduling scheme.read more
☆
Thu Sep 3 · 1:45 PM - 3:15 PM
Research 48 · Dynamic Graphs and Graph Learning
● Research
📍 Commonwealth Ballroom A
Understanding Evolving Graph Structures for Large Discrete-Time Dynamic Graph Representation (Danni Wu, East China Normal University et al.) - Discrete-Time Dynamic Graphs (DTDGs) are commonly used to model and analyze systems evolving in discrete time steps (snapshots).Understanding Evolving Graph Structures for Large Discrete-Time Dynamic Graph Representation (Danni Wu, East China Normal University et al.) - Discrete-Time Dynamic Graphs (DTDGs) are commonly used to model and analyze systems evolving in discrete time steps (snapshots). For DTDG representation, existing approaches typically manage nodes' neighbors using an individual adjacency matrix for each snapshot, which provides neighbor information for structure learning based on neural networks. They either focus on the current snapshot, overlooking the evolution of temporal structures, or require preprocessing to access historical neighbors, resulting in significant computational overhead. In addition, the adjacency matrices for a DTDG consume O(T|V|²) memory, where T and |V| are the snapshot size and node size, respectively, restricting scalability on large DTDGs. To address these issues, in this paper, we propose a scalable and efficient framework (called UnderGS) with an efficient neighbor store, which can understand evolving graph structures for representation learning over DTDGs. Concretely, we first define a temporal influence score that helps identify influential temporal neighbors from current and previous snapshots. Upon it, we develop a temporal-cohesive neighbor store that maintains influential temporal neighbors for each node directly on the GPU, preserving evolving structural relationships across snapshots, which takes O(|V|K) memory for a DTDG (K is the neighbor size). Furthermore, our neighbor store enables seamless integration with message-passing graph neural networks and non-message-passing neural networks for temporal structure learning. Last, we introduce a lightweight training pipeline with a late-snapshot gradient aggregation mechanism, which enhances computational efficiency. Extensive experimental results on eight DTDGs show that UnderGS achieves up to 9× speed-up against the best competitors while achieving an average improvement of 31.36% in accuracy. • Efficient Temporal Edge-Core Maintenance in Streaming Graphs (Tongfeng Weng, National University of Singapore et al.) - Temporal graphs are critical for modeling dynamic systems where interactions evolve over time, with a central challenge being the characterization of structural cohesion. The temporal edge-core, defined under a temporal proximity constraint Δ, quantifies the stability and density of connections within subgraphs and is essential for applications such as anomaly detection and information diffusion. Existing edge-core decomposition methods, however, are designed for static graphs and are computationally prohibitive in streaming environments due to frequent edge arrivals and deletions. We present TECM, an efficient framework for streaming temporal edge-core decomposition that leverages the localized impact of edge updates within Δ-incident neighbors. TECM incrementally updates core values through Δ-aware traversals and localized H-index analysis, and incorporates batch processing to handle high-velocity streams. Extensive experiments on real and synthetic temporal networks demonstrate that TECM delivers speedups of several orders of magnitude over state-of-the-art static baselines, providing a scalable and principled solution for real-time structural analysis in evolving temporal graphs. • FlareDTDG: Harnessing Temporal Recency for Scalable Discrete-Time Dynamic Graph Training (Wenjie Huang, Zhejiang University et al.) - Discrete-time dynamic graphs (DTDGs), modeled as snapshot sequences, are widely used to capture temporal evolution in relational systems. Scaling DTDG training remains challenging: full-batch methods incur prohibitive memory and communication costs, while sampling or offloading often sacrifices accuracy or efficiency. A major limitation of existing frameworks is that they treat all snapshots equally, ignoring the temporal recency effect, where recent snapshots are typically far more predictive than older ones. We introduce FlareDTDG, a distributed framework that exploits temporal recency for efficient and scalable training. Its core is hybrid batching with temporal decay, which applies full-batch processing to recent snapshots, while progressively coarsely sampling older ones to form hybrid batches. We also integrate two co-designed optimizations: fast graph reconstruction via shrinking to eliminate cross-snapshot remapping, and adaptive comm-comp overlap scheduling to reduce synchronization overhead. Experiments show FlareDTDG achieves 1.4-2.5 times faster training and 10 - 85\% lower GPU memory usage than full-batch baselines, while preserving accuracy. It also scales to graphs with 100M nodes per snapshot, where existing systems fail due to memory limits or degraded performance. • Finding Time-Proximity Communities in Temporal Heterogeneous Information Networks (Yifu Tang, Swinburne University of Technology et al.) - Community search in heterogeneous information networks (HINs) often neglects temporal dynamics, yielding structures that poorly reflect real-world interactions. We introduce the Temporal HIN Community Search (THCS) problem and propose a novel (k, Tq, Pδ)-core model that captures both structural cohesiveness and temporal relevance. Our model uses a time span constraint δ to ensure interaction recency and a query interval Tq for flexible temporal exploration, filtering irrelevant connections while preserving structural density. We develop two efficient online algorithms - Center-based Sliding Window search and Incremental Center Expansion - that exploit meta-path symmetry and dynamic connectivity tracking. For frequent queries, we design a Temporal HIN Core Interval-Index (TCI-Index), organising minimal core intervals hierarchically with innovative compression techniques. Experiments on real-world datasets show our methods significantly outperform baselines, finding temporally meaningful communities with high efficiency. • A Semantics-aware Approach for Graph Edit Distance Estimation over Knowledge Graphs (Yingli Zhou, The Chinese University of Hong Kong, Shenzhen et al.) - Graph Edit Distance (GED) is a key metric for measuring the similarity between two Knowledge Graphs (KGs), defined as the minimum number of atomic operations required to transform one KG into another. It has broad applications in fields such as pattern recognition, biological analysis, and graph databases. The state-of-the-art approaches adopt Graph Neural Networks (GNNs) to predict GED, but they are limited to simple graphs and cannot be directly applied to the KGs, as they fail to capture the rich semantics and complex relationships present in KGs. To design a KG-native solution, in this paper, we propose a semantics-aware GNN model, SEABED, to capture local semantic dependencies and global semantic consistency between two KGs. Extensive experiments on four real-world KGs demonstrate that our proposed algorithm outperforms the state-of-the-art methods on all datasets. In particular, the mean absolute error is reduced by up to 66.7%, while the accuracy is improved by up to 70.5%, without increasing the computation time. • NeutronCloud: Resource-Aware Distributed GNN Training in Fluctuating Cloud Environments (Mingyi Cao, Northeastern University et al.) - Graph Neural Networks (GNNs) are widely employed to learn representations from graph-structured data. To support large-scale graph training, researchers use distributed techniques, partitioning the graph across multiple computing nodes and performing parallel training by exchanging dependency vertex information via cross-node communication. However, existing GNN training systems operate on statically partitioned subgraphs, making them difficult to adapt to resource fluctuations. In practice, resource fluctuations in cloud environments often cause variability in compute and communication resources, posing challenges for aligning each worker’s workload to its available resources during GNN training. In this paper, we propose NeutronCloud, a system designed for efficient GNN training in cloud environments. First, we adopt a resource-aware workload adjustment strategy. It builds on hybrid dependency handling by obtaining dependency information through both local computation and remote communication. During training, it dynamically adjusts the ratio between locally computed and remotely fetched dependencies based on each worker's available resources, ensuring workload-resource alignment. Second, we employ a dependency-aware partial-reduce approach reusing historical vertex embeddings and skipping the stragglers during gradient aggregation to address extreme resource fluctuations that cause some workers to lag significantly behind others in the cluster. Experimental results on the resource-fluctuating environment demonstrate that NeutronCloud achieves 1.83×-4.43× speedup compared to state-of-the-art distributed GNN systems. • Multimodal Knowledge Graph Completion via Relation-Aware Negative Sampling with Diffusion-Based Interpolation (Qian Ma, Dalian Maritime University et al.) - Multimodal Knowledge Graphs (MMKGs) enable structured reasoning across heterogeneous modalities and are essential infrastructure for data management and analytics. As MMKGs are inherently incomplete and generally contain noisy data, MMKG Completion (MMKGC) is a central task for improving data quality and semantic inference. Specifically, a crucial aspect of MMKGC is negative sampling, which impacts model discriminability and completion accuracy. However, existing negative sampling proposals often ignore the semantic properties of relation types and lack mechanisms for adaptive control of negative sample hardness, leading to suboptimal MMKGC performance. To address issues such as these, we propose RelDINS that improves semantic consistency and robustness by performing relation-type-aware negative sampling through diffusion-based interpolation. RelDINS incorporates two modules: (i) a Relation-type-aware Multimodal Embedding Learning (RMEL) module that adaptively injects relational semantics into entity representations based on cardinality constraints; (ii) and a Diffusion-based Interpolation Negative Sampling (DINS) module that dynamically generates hardness tunable negative samples via spherical linear interpolation in diffusion noise space. Extensive experiments on three public benchmarks show that RelDINS achieves state-of-the-art performance, with average improvements of 3.5% in MRR, 5.0% in Hit@1, 2.6% in Hit@3, and 1.4% in Hit@10 over leading baselines. Supported by a complexity analysis and an empirical study, RelDINS is a principled and scalable solution to enhancing semantic consistency and data reliability in MMKGC. • UniTG: A Unified System for Efficient and Seamless Textual Graph Learning (Meng Zhang, Nanyang Technological University et al.) - It has become critical to utilize language models (LMs) for representation learning on text-attributed graphs. They enhance the original graph neural networks (GNNs) by delicately modeling text attributes alongside graph structure learning. Despite these algorithmic breakthroughs, existing LM-based graph learning still fails in practical deployment due to several critical defects, namely time and resource inefficiency, inflexible decoupled architectures, limited model scale, and the omission of graph properties. In this paper, we propose UniTG, the first unified system that fuses the LM and GNN phases into a single end-to-end procedure through three co-designed components spanning the runtime, algorithm, and execution levels. At the runtime level, UniTG introduces Affinity-aware Flow Parallelism, exploiting graph affinity to scale the training of large graph neural networks. At the algorithm level, a novel Collaborative Learning strategy integrates both text and graph modalities to enable accurate joint training. At the execution level, the Streamlined Pipeline Schedule squeezes pipeline bubbles by interleaving LM fine-tuning into the GNN pipeline, boosting overall efficiency and resource utilization. Extensive experiments demonstrate that, compared with state-of-the-art LM-based graph learning systems, UniTG dramatically reduces learning makespan by up to 17.3x without compromising model quality. • Maximum Defective Biclique Search in Large Bipartite Graphs (Donghang Cui, Beijing Institute of Technology et al.) - The problem of identifying the maximum edge biclique in bipartite graphs has attracted considerable attention in bipartite graph analysis, with numerous real-world applications such as fraud detection, community detection, and online recommendation systems. However, real-world graphs may contain noise or incomplete information, leading to overly restrictive conditions when employing the biclique model. To mitigate this, we focus on a new relaxed subgraph model, called the $k$-defective biclique, which allows for up to $k$ missing edges compared to the biclique model. We investigate the problem of finding the maximum edge $k$-defective biclique in a bipartite graph, and prove that the problem is NP-hard. To tackle this computation challenge, we propose a novel algorithm based on a new branch-and-bound framework, which achieves a worst-case time complexity of $O(m\alpha_k^n)$, where $\alpha_k < 2$. We further enhance this framework by incorporating a novel pivoting technique, reducing the worst-case time complexity to $O(m\beta_k^n)$, where $\beta_k < \alpha_k$. To improve the efficiency, we develop a series of optimization techniques, including graph reduction methods, novel upper bounds, and a heuristic approach. Extensive experiments on 10 large real-world datasets validate the efficiency and effectiveness of the proposed approaches. The results indicate that our algorithms consistently outperform state-of-the-art algorithms, offering up to $1000\times$ speedups across various parameter settings.read more
☆
Thu Sep 3 · 1:45 PM - 3:15 PM
Tutorial 13 · Advances of Query Processing in Vector Databases
● Tutorials
📍 Otis
Advances of Query Processing in Vector Databases
Jiadong XieThe Chinese University of Hong Kong
Yingfan LiuXidian University
Jeffrey Xu YuThe Hong Kong University of Science and Technology (Guangzhou
☆
Thu Sep 3 · 1:45 PM - 3:15 PM
Tutorial 15 · Data Management for Agentic Memory: Foundations, Systems, and Challenges
● Tutorials
📍 Stone
Data Management for Agentic Memory: Foundations, Systems, and Challenges
Guoliang LiTsinghua University
Jiaqi TianTsinghua University
Xuanhe ZhouShanghai Jiaotong University
☆
Thu Sep 3 · 1:45 PM - 3:15 PM
Tutorial 16 · From Big Time Series Forecasting to Foundation Models for Structured Data
● Tutorials
📍 Webster
From Big Time Series Forecasting to Foundation Models for Structured Data
Xiyuan ZhangAWS
Abdul Fatir AnsariAWS
Christos FaloutsosCMU
George KarypisUMN
Yuyang WangAmazon Research
☆
Thu Sep 3 · 1:45 PM - 3:15 PM
Industry Session 8 · AI-Native Data Systems & Natural-Language Interfaces
● Industry
📍 Marina Ballroom I
Bridging NL2SQL for CAN Signal Analytics at NIO: From Externalized Schemas to CTE Pipelines (Wei Zhou, Shanghai Jiao Tong University et al.) - Large Language Models (LLMs) promise natural-language access to data, yet applying them to Controller Area Network (CAN)...Bridging NL2SQL for CAN Signal Analytics at NIO: From Externalized Schemas to CTE Pipelines (Wei Zhou, Shanghai Jiao Tong University et al.) - Large Language Models (LLMs) promise natural-language access to data, yet applying them to Controller Area Network (CAN) signal SQL analytics in the automotive industry is challenging. First, CAN data warehouses store tens of thousands of signals as opaque numeric columns. Multiple signals are often packed into a single column as bit segments, requiring external Database CAN (DBC) files beyond the native database schema for proper definitions. Second, engineers’ natural-language queries often use terms that map to multiple signals in the DBC files, with little direct indication of which signals are being referenced. Finally, signal analysis in production involves multi-stage SQL pipelines using layered Common Table Expressions (CTEs) for tasks such as temporal alignment, more complex than those handled by existing NL2SQL methods. To address these, we present NIO-Weaver, an end-to-end CAN signal analytics system within a graph-augmented multi-agent framework. First, to uncover signals hidden in opaque numeric columns, we introduce an automated graph-construction pipeline that leverages DBC files to build a domain-knowledge graph, associating signals with potential high-level terms. Second, to bridge the semantic gap between natural-language queries and multiple signals, we propose an optimized retrieval method that extracts terms from queries and performs hybrid retrieval. We combine regex matching, embedding search, and constrained graph traversal to identify candidate signals for further processing. Finally, to simplify the generation of multi-step CTE-based SQL pipelines, we introduce a hierarchical generation paradigm. We decompose queries into plans, incrementally generating executable CTEs, while self-correction mechanisms refine missing signals and incorrect SQLs. Experiments conducted on the real-world industrial CAN warehouse at NIO demonstrate that NIO-Weaver outperforms baselines by up to 67.2%. • OmniTable: A Unified Wide-Table System for Petabyte-Scale LLM Data Curation and Exploration (Yuzhuo Fu, AntGroup et al.) - Data curation is a critical bottleneck in industrial-grade LLM development, where petabyte-scale unstructured corpora are scattered across hundreds of physical tables, feature engineering relies on manual, table-centric pipeline orchestration, and data lineage is largely absent. We present OmniTable as an architecture blueprint for a unified wide-table layer built on Logical Unification, Physical Separation, targeting petabyte-scale LLM data curation and exploration. OmniTable makes four contributions: (1) a unified wide-table abstraction that consolidates multi-source heterogeneous data and thousands of derived features under a single logical schema via logical-physical mapping; (2) declarative feature lifecycle management that automates dependency resolution, execution planning, operator fusion, and lineage tracking, replacing manual pipeline orchestration with a "declare-and-execute" paradigm; (3) an adaptive execution engine with autonomous governance that achieves stable PB-scale feature backfill through heterogeneous compute routing (CPU/GPU), adaptive tuning, UDF-level fault tolerance, and automated storage layout optimization; and (4) hybrid-accelerated data exploration combining a global ID index, transparent OLAP offloading, and background materialized views to deliver second-level point lookups and filtered exports exceeding 20 TB/hour. In production, OmniTable manages over 35 PB of training data across web, code, PDF, and SFT domains, reducing the human-in-the-loop curation cycle from approximately 14 days to approximately 2.5 days (5.6x over the pre-OmniTable production workflow), with consistent feature versioning, auditable lineage, and minimal manual intervention. • IMLane: Composable Framework for Efficient AI Function Execution in Database Engine (Chenyang Zhang, East China Normal University et al.) - Efficient execution of artificial intelligence (AI) functions within the database engine has become an essential requirement for AI-driven data analysis workflows nowadays.During the development of supporting AI functions in OceanBase, we identified two performance bottlenecks incurred by the internal design of the database engine.First, the Python UDF-based AI function is executed via thread-level parallelism, which causes ineffective parallel execution due to limitations of CPython runtime.Second, the AI function is scheduled coupling with the database engine, which is oblivious to the diverse compute resource requirements of AI functions in different user scenarios, thus leading to the under-utilization of available compute resources.To address these common bottlenecks, we propose a composable framework, called IMLane, for equipping current database engines with efficient AI function execution.IMLane incorporates process-level AI function execution for effective parallel execution with little data transfer overhead, and decoupled AI function scheduling to match with the compute resource requirements.Our experiments on OceanBase and DuckDB show 7.89x and 6.53x improvements on average, respectively, after integrating IMLane.Moreover, IMLane demonstrates performance superiority over alternative solutions. • CoeusBI: A Comprehensive Interactive Business Intelligence System Powered by LLMs at Baidu (Jinqing Lian, Beijing University of Posts and Telecommunications et al.) - The advent of Large Language Models has catalyzed the emergence of interactive Business Intelligence (BI) systems. Although commercial BI products increasingly adopt semantic layers paired with natural language interfaces, they predominantly rely on manual configurations to define metrics and dimensions. Real-world deployments continue to face critical challenges: (a) frequent JOIN operations degrade the accuracy of SQL generation; (b) wide schemas exacerbate the challenge of schema linking; and (c) the generation of dialect-specific queries and the accurate support for multi-round dialogues incur high costs and yield limited accuracy. We introduce CoeusBI, an industrial-scale interactive BI system that addresses these barriers through a novel Dual-Agent Architecture paired with a Hierarchical Schema Linking module: (1) an offline View Generation Agent that utilizes error-feedback to autonomously convert complex JOIN queries into simple single-view queries, which eliminates the need for manual semantic modeling; (2) a Hierarchical Schema Linking module that leverages vector retrieval over views to handle exceptionally wide schemas efficiently; and (3) a dynamic Routing Agent that evaluates dialogue contexts to route queries, dynamically invoking either the synthesis of new intermediate representations or targeted modifications of existing ones, before compiling the unified representation via a deterministic SQL compiler that is agnostic to dialects. Extensive experiments on both public datasets and production datasets demonstrate that CoeusBI achieves significant improvements in query accuracy, token efficiency, and user satisfaction relative to existing methods. CoeusBI is currently deployed as a standalone service on the data platform of Baidu and is widely used across multiple business lines - including video, search, and advertising - supporting thousands of users daily. • VikingMem: A Memory Base Management System for Stateful LLM-based Applications (Jiajie Fu, Zhejiang University et al.) - Large Language Models have revolutionized interactive applications, however, their finite context windows pose a critical data management challenge for maintaining stateful, long-term interactions. Existing memory approaches, however, often rely on simplistic extraction that leads to incomplete memories or use rigid, single-use memory extract prompts tailored for one use case like Chatbots. Consequently, they lack generalizability and perform poorly on diverse downstream tasks. To bridge this gap, we introduce the Memory Base, a novel data management paradigm to manage the persistent state of long-term interactions. It is characterized by three core principles: selective extraction of high-value memories from raw information streams; inherent statefulness and evolution, where memory content is progressively summarized, corrected, and temporally weighted to prioritize recent interactions; and a generalizable abstraction paradigm designed for robust transferability across diverse applications, including education, recommendation, and agent memory. Building on this foundation, we present VikingMem, an end-to-end Memory Base Management System implemented on the VikingDB vector engine. VikingMem materializes this paradigm through interconnected event and entity abstractions. It features event-centric memory extraction to selectively handle complex information streams, while entities are dynamically updated by events to achieve stateful evolution. Using temporal compression via a topic-wise timeline and time-weighted recall, the system can progressively produce high-level summary memories, prioritize recent items, while compressing and fading older ones. Retrieval is further optimized through tensor-based reranking. Extensive evaluations on long-term memory benchmarks demonstrate that VikingMem achieves state-of-the-art performance, outperforming baselines by up to 30% in memory retrieval effectiveness while maintaining low latency essential for interactive applications. • DBAgent: An RL-Based Agent for Autonomous Database Operations and Maintenance (Xu Chen, University of Electronic Science and Technology of China et al.) - Database Operations and Maintenance (O&M) is a critical but complex and labor-intensive task. Recent LLM-based assistants promise to lower the barrier by reading manuals/tickets and exploring diagnostic search trees. However, existing LLM-based solutions fall short due to fundamental limitations in learning from expert demonstration. Such design fails to internalize domain dynamics (how interventions change plans, resources, etc.), and struggles under workload and statistics drift. To address this, we present DBAgent, an autonomous agent for Huawei Cloud Data Warehouse Service (DWS) integrated with Autopilot (DWS ’s production monitoring, alerting, and auto-remediation service). DBAgent consumes DWS telemetry (e.g., KPIs and execution plans) and Autopilot alerts to diagnose and remediate incidents in production clusters. DBAgent emulates an expert’s iterative Think - Act - Observe problem-solving loop with a policy trained via reinforcement learning (RL). It couples dynamic tool use for information gathering, a multimodal perception module for database-native signals, and an RL-based reasoning engine that plans, verifies, and executes remedies end to end. Extensive experiments show that DBAgent handles a broad range of complex O&M tasks. It surpasses the strongest baseline by +23% success rate and produces evidence-grounded reports with low hallucination (?5%).read more
☆
Thu Sep 3 · 1:45 PM - 3:15 PM
DB Research Integrity (Part 2)
● Ethics
📍 Grand Ballroom A+B
DB Research Integrity Working Group (Angela Bonifati, Lyon 1 University & CNRS et al.)
☆
Thu Sep 3 · 1:45 PM - 3:15 PM
Demo Group B · Query Processing, AI & Data Agents
● Demos
📍 Marina Ballrooms II - IV
31 demos in this group, shown twice during the conference.31 demos in this group, shown twice during the conference. Includes: Minimal Data Cleaning for Model Training by MinPrep • A Demonstration of Continuous Lifelong Conflict-Aware AGV Routing with Kinematic Constraints • FilterPilot: An Interactive Assistant for Adapting Filtering Predicate to Table Content • GraphAgent: An Effective Knowledge-Guided GNN Model Selection System • CADENZA in Action: Breaking the Monolith with Intent-Dependent Plan Spaces for Semantic Queries • MCAD: Multivariate Correlation Anomaly Data Generator • Carnot: Interpretable, Interactive, and Optimized Execution of Deep Research Queries • NiceT: Named Entity Cleaning and Enhancement with Human-in-the-loop • A Portable Middleware for Plan-Based Adaptive Query Processing • Policy-Aware Federated Query Orchestration Across Energy Data Spaces and Edge AI Services • … +21 moreread more
Built for you by(617)-PATRICK, Champion AI speaker
MH-GIN: Multi-scale Heterogeneous Graph-based Imputation Network for AIS Data (Hengyu Liu, Aalborg University et al.) - Location-tracking data from the Automatic Identification System, much of which is publicly available, plays a key role in a range of maritime safety and...MH-GIN: Multi-scale Heterogeneous Graph-based Imputation Network for AIS Data (Hengyu Liu, Aalborg University et al.) - Location-tracking data from the Automatic Identification System, much of which is publicly available, plays a key role in a range of maritime safety and monitoring applications. However, the data suffers from missing values that hamper downstream applications. Imputing the missing values is challenging because the values of different heterogeneous attributes are updated at diverse rates, resulting in the occurrence of multi-scale dependencies among attributes. Existing imputation methods that assume similar update rates across attributes are unable to capture and exploit such dependencies, limiting their imputation accuracy. We propose MH-GIN, a Multi-scale Heterogeneous Graph-based Imputation Network that aims improve imputation accuracy by capturing multi-scale dependencies. Specifically, MH-GIN first extracts multi-scale temporal features for each attribute while preserving their intrinsic heterogeneous characteristics. Then, it constructs a multi-scale heterogeneous graph to explicitly model dependencies between heterogeneous attributes to enable more accurate imputation of missing values through graph propagation. Experimental results on two real-world datasets find that MH-GIN is capable of an average 57% reduction in imputation errors compared to state-of-the-art methods, while maintaining computational efficiency. • FutureLight: An Efficient Future Traffic Data-Driven Reinforcement Learning Framework for Traffic Signal Controls (Zizhuo Xu, The Hong Kong University of Science and Technology et al.) - Reinforcement learning (RL) has become a promising approach for the Traffic Signal Control (TSC) problem, enabling agents (intersections) to learn control policies with dynamic traffic environments. However, existing RL-based methods typically rely solely on current traffic states and expected reward estimates, without leveraging predicted future traffic conditions for more effective decision-making. Therefore, we propose FutureLight, the first RL TSC framework that leverages future traffic data. Specifically, we design a macroscopic, signal-aware, and lane-level simulation FutureLight-RouteSys that efficiently and accurately estimates future traffic conditions. Then, the predicted results are embedded with FutureLight-Encoder into FutureLight-DQN through state augmentation, reward shaping, and hybrid value estimation, which combines simulated near-future rewards with bootstrapped near-future returns. Finally, we propose several pruning techniques to avoid redundant calculations and further improve overall training efficiency. Experimental results demonstrate that our proposed framework consistently improves traffic signal control performance, and also improves training efficiency by thirty times. • KAFY: An Extensible and Scalable Transformers-Based System for Trajectory Data Analysis (Youssef Hussein, University of Minnesota et al.) - Trajectory data analysis, e.g., trajectory summarization, imputation, prediction, and classification, has been fundamental to widely used applications. Even though several research efforts have been dedicated to develop numerous algorithms for trajectory analysis, there is an apparent lack of full-fledged systems that support a myriad of trajectory analysis tasks. The main reason is that each introduced algorithm employs new methods and data structures that are tailored to one specific trajectory analysis task. This paper presents KAFY; a full-fledged system that supports a myriad of trajectory data analysis tasks. KAFY leverages the recent advances in Natural Language Processing (NLP) where the transformer architecture is introduced as a system infrastructure to build large language models that can be fine tuned to support various NLP tasks. The main idea of KAFY is that instead of training a transformer architecture with a (spoken) language to produce (language) models, it trains it with the (unspoken) trajectory language to produce (trajectory) models. KAFY is an extensible system where its users can extend it with more transformers and/or trajectory operations. The first release of KAFY employs three transformers and supports five trajectory operations. Experimental results from a real deployment of KAFY show that it either outperforms or gives similar performance to existing baselines in all its supported trajectory operations. • MS-Index: Fast Top-k Subsequence Search for Multivariate Time Series under Euclidean Distance (Jens d'Hondt, Eindhoven University of Technology et al.) - Modern applications frequently collect and analyze temporal data in the form of multivariate time series (MTS) - time series that contain multiple channels. A common task in this context is subsequence search, which involves identifying all MTS that contain subsequences highly similar to a query time series. In practical scenarios, not all channels of an MTS are relevant to every query. For instance, airplane sensors may gather data on a plethora of components and subsystems, but only a few of these are relevant to a specific query, such as identifying the cause of a malfunctioning landing gear, or a specific flight maneuver. Consequently, the relevant query channels are often specified at query time. In this work, we introduce the Multivariate Subsequence Index (MS-Index), a novel algorithm for nearest neighbor MTS subsequence search under Euclidean distance that supports ad-hoc selection of query channels. The algorithm is exact and demonstrates query performance that scales sublinearly to the number of query channels. We examine the properties of MS-Index with a thorough experimental evaluation over 34 datasets, and show that it outperforms the state-of-the-art one to two orders of magnitude for both raw and normalized subsequences. • Error-bounded Point Cloud Compression Using Truncated Octahedron Quantization (Youyuan Liu, Temple University et al.) - With the rapid advancement of large-scale scientific simulations, the massive volume of point cloud data generated has increasingly become a critical bottleneck for modern storage systems. Existing point cloud compression techniques used in data storage systems are designed for sparse geometry and rely on quantization schemes whose optimality assumptions do not hold for dense data. When applied at the compression layer to point clouds, this representation mismatch leads to fundamentally sub-optimal rate-distortion trade-offs that cannot be addressed through parameter tuning or framework-level adaptations. This issue arises in scientific data management pipelines for applications such as molecular dynamics simulations, which generate massive particle datasets forming dense distributions. Consequently, state-of-the-art compression methods fail to fully exploit the redundancies inherent in such data. We address this limitation by developing a theory of point cloud compressibility for dense data, characterizing fundamental rate-distortion behavior at the representation layer. Guided by this analysis, we introduce XnYZip, an error-bounded lossy compressor based on provably optimal Truncated Octahedron quantization, combined with a locality-aware encoding pipeline using space-filling curves and run-length encoding. Experiments on large-scale scientific datasets demonstrate consistent storage and performance improvements, achieving up to 3x higher compression ratios, 2.2x faster compression, and 1.2x faster decompression compared to state-of-the-art point cloud compressors.read more
☆
Thu Sep 3 · 3:45 PM - 5:15 PM
Research 50 · Blockchain and Trusted Execution
● Research
📍 Grand Ballroom D
FairDAG: Consensus Fairness over Multi-Proposer Causal Design (Dakai Kang, University of California, Davis et al.) - The rise of cryptocurrencies like Bitcoin and Ethereum has driven interest in blockchain database technology, with smart contracts enabling the growth of decentralized finance...FairDAG: Consensus Fairness over Multi-Proposer Causal Design (Dakai Kang, University of California, Davis et al.) - The rise of cryptocurrencies like Bitcoin and Ethereum has driven interest in blockchain database technology, with smart contracts enabling the growth of decentralized finance (DeFi). However, research has shown that adversaries exploit transaction ordering to extract profits through attacks like front-running, sandwich attacks, and liquidation manipulation. This issue affects blockchains where block proposers have full control over transaction ordering. To address this, a more fair transaction ordering mechanism is essential. Existing fairness protocols, such as Pompe and Themis, operate on leader-based consensus protocols, which not only suffer from low throughput caused by single-leader bottleneck, but also give adversarial block proposers to manipulate transaction ordering. To address these limitations, we propose a new framework FairDAG that runs fairness protocols on top of DAG-based consensus protocols, which improves protocol performance in both throughput and fairness quality, leveraging the multi-proposer design and validity property of DAG-based consensus protocols. We conducted a comprehensive analytical and experimental evaluation of two FairDAG variants - FairDAG-AB and FairDAG-RL. Our results demonstrate that FairDAG outperforms prior fairness protocols in both throughput and fairness quality. • Fides: Secure and Scalable Asynchronous DAG Consensus via Trusted Components (Shaokang Xie, University of California, Davis et al.) - DAG-based BFT consensus has attracted growing interest in distributed data management systems for consistent replication in untrusted settings due to its high throughput and resilience to asynchrony. However, existing protocols still suffer from high communication overhead and long commit latency. In parallel, introducing minimal hardware trust has proven effective in reducing the complexity of BFT consensus. Inspired by these works, we present Fides, an asynchronous DAG-based BFT consensus protocol that, to our knowledge, is among the first to leverage TEEs to enhance both scalability and efficiency. Fides tolerates a minority of Byzantine replicas and achieves $O(\kappa n^2 + n^3)$ metadata communication complexity through a customized TEE-assisted Reliable Broadcast (T-RBC) primitive with linear communication complexity in one-step broadcast.Building on T-RBC, Fides redefines the DAG construction rules by reducing the reference requirement from $2f+1$ to $f+1$ between consecutive vertices. This new structure weakens DAG connectivity and invalidates traditional commit rules, so we formally abstract the problem and derive new theoretical bounds of liveness. We further propose a four-round commit rule that achieves the theoretically minimal commit latency. Besides, we design two additional primitives, T-RoundCert and T-Coin, to efficiently certify DAG references and replace the costly cryptographic common coin used in prior protocols.Comprehensive evaluations on geo-distributed and local testbeds show that Fides substantially outperforms state-of-the-art protocols, including Tusk, Bullshark, Mysticeti, Shoal++, RCC, Damysus, Achilles and HybridSet, achieving lower latency and higher throughput while preserving strong safety and liveness guarantees. • HarborMaster: Rollback Detection for Trusted Distributed Computing (Shubham Mishra, UC Berkeley et al.) - Trusted Execution Environments (TEEs) provide strong confidentiality and integrity guarantees to distributed data processing. Developers are increasingly using clusters of TEE-enabled workers for these applications. However, TEEs do not protect persistent state: a rollback attack may replace the current persistent state with a stale version. Conventional rollback detection systems interfere with the application’s steady-state performance either by adding high coordination overhead or log amplification. Our key insight is "Rollback attacks violate causality". Rollback attacks can be efficiently detected by causal logging, avoiding the coordination and log amplification overheads of prior solutions. We build HarborMaster, a high-performance rollback detection system for the TEE-based distributed computing clusters. HarborMaster offloads the job of rollback detection to a specialized auditing service that efficiently checks for rollback attacks using violations of causality in the TEE-based distributed computing workers. Running in AMD SEV-SNP Confidential VMs, HarborMaster only imposes 8 - 35% logging overhead and preserves the linear scaling properties of an unprotected distributed application. • Fugue: Online Elasticity for Distributed Stateful Stream Processing (Yuqiu Zhang, University of Toronto et al.) - Stateful stream processing engines are critical for real-time analytics but lack efficient mechanisms for runtime elasticity. The dominant "stop-the-world" model, used by systems like Apache Flink, requires halting applications globally for a long time, while recent on-the-fly protocols introduce severe trade-offs: proactive approaches impose a continuous resource tax by constantly replicating state, and existing reactive solutions suffer from architectural complexity and external dependencies. This paper introduces Fugue, a novel, self-contained reactive protocol that provides seamless and resource-efficient elasticity. The core of Fugue is a two-phase design that combines a pre-emptive background state transfer with an atomic, lightweight barrier-based cutover. By moving the bulk of an operator's state off the critical path and unifying the final ownership transfer with the system's native exactly-once synchronization mechanism, Fugue guarantees correctness with minimal disruption and steady-state overhead. We implemented Fugue in Apache Flink and our evaluation on realistic benchmarks shows it reconfigures a running query up to 40x faster than native Flink while maintaining over 95% of peak throughput. Compared to state-of-the-art baselines, Fugue reduces end-to-end migration latency by up to 14x, demonstrating a superior balance of robustness, performance, and operational simplicity. • Verifiable Authenticated Data Structure (V-ADS) for Analytic Queries (Masoud Nosrati, Iowa State University et al.) - An Authenticated Data Structure (ADS) enables a cloud service to generate a verification object (VO), allowing users to verify the correctness of query results. This is particularly valuable for data owners who wish to offload query processing to the cloud while remaining concerned about the trustworthiness of the results. In this paper, we address the problem of constructing ADS for a category of queries known as analytic queries, which operate over linear functions. Existing approaches require the data owner to build the ADS; however, the computational resources needed for this construction are often beyond the capabilities of many data owners - particularly those who are resource-constrained and thus choose outsourcing in the first place. Motivated by this challenge, we propose the concept of a Verifiable ADS (V-ADS), which delegates most of the ADS construction to the cloud while enabling the data owner to verify and sign the results with minimal effort. We present two techniques - baseline and advanced V-ADS - and demonstrate through theoretical analysis and experimental evaluation that these approaches substantially reduce the resource demands on the data owner.read more
☆
Thu Sep 3 · 3:45 PM - 5:15 PM
Research 51 · Stream Processing and IoT
● Research
📍 Grand Ballroom C
APEROL: Adaptive Parallel Edge-to-Cloud Runtime Optimization for Layered Workflow Execution (Dimitrios Banelas, Technical University of Crete et al.) - The execution of streaming analytics workflows across large-scale IoT infrastructures poses unique challenges.APEROL: Adaptive Parallel Edge-to-Cloud Runtime Optimization for Layered Workflow Execution (Dimitrios Banelas, Technical University of Crete et al.) - The execution of streaming analytics workflows across large-scale IoT infrastructures poses unique challenges. Central data collection depletes the available bandwidth and leaves IoT device resources unutilized. Therefore, workflow execution should be performed in-network, assigning workflow operator execution on devices across the cloud-to-edge continuum. However, the vast scale of devices results in an exponential number of possible combinations of workflow operator assignments. On top of that, workflows are executed on dynamic environments where volatile data stream distributions and device churn may render a deployed plan inefficient and, therefore, rapid adaptation decisions are crucial. To address these challenges, we present APEROL, the first suite of parallel optimization algorithms for timely and efficient workflow execution in IoT environments. APEROL introduces a novel conceptualization of the optimization search space, coupled with a signature-based execution plan enumeration scheme, that enable scalable, parallel plan exploration. The suite includes exhaustive, heuristic, greedy, and random sampling algorithms, which are complementary in algorithm speed vs. plan quality trade-offs under different setups. The current implementation examines up to 2M candidate plans per second on commodity hardware. Experiments with 5 challenging workflows from 2 streaming benchmarks, over real and simulated networks ranging from 10s to 1000s sites show APEROL's effectiveness and timeliness. • Incremental Stream Query Deployment under Continuous Infrastructure Changes in the Cloud-Edge Continuum (Ankit Chaudhary, Technische Universität Berlin et al.) - Distributed data stream processing engines (DSPEs) operating over the cloud-edge continuum must deploy data processing operators across a distributed infrastructure. However, the volatile nature of these infrastructure nodes - where devices frequently join, leave, or move - can invalidate existing query operator-to-topology node mappings, leading to interruptions in query execution and potential data loss. To ensure continuous processing while maintaining correctness, DPSEs must dynamically adapt these mappings and redeploy (part of) affected queries. In this paper, we introduce incremental stream query deployment (ISQD), a framework that efficiently redeploys queries affected by topology changes. ISQD employs a greedy strategy to identify and redeploy only affected operators. It uses ad-hoc queries to migrate operator state seamlessly, and leverages reconfiguration markers to synchronize the redeployment process. Our evaluation shows that \texttt{ISQD} achieves up to 7.5x lower deployment latency and up to 39x lower event time latency compared to state-of-the-art approaches, even under high-frequency topology changes. • Meerkat: Scalable, Network-Aware Failure Recovery for the Internet of Things (Anastasiia Kozar, TU Berlin et al.) - The Internet of Things (IoT) demands real-time, low-latency processing of data generated by thousands of heterogeneous, resource-constrained devices. In such dynamic environments, ensuring fault tolerance becomes critical, especially for safety-sensitive applications like disaster management or patient monitoring. However, existing centralized fault tolerance solutions face serious scalability challenges across large, hierarchically connected IoT topologies. In this paper, we present Meerkat, a network-aware fault-tolerance protocol explicitly designed for IoT environments. Meerkat achieves zero-downtime recovery via redundant operator execution on disjoint paths and efficient duplicate detection. It also includes dynamic load balancing that adapts operator placement to device volatility, ensuring fair resource use. Compared to state-of-the-art techniques, Meerkat sustains up to 70x higher throughput with only 28% network overhead. These results highlight Meerkat’s ability to deliver efficient fault tolerance with minimal overhead at IoT scale. • SHARP: Shared State Reduction for Efficient Matching of Sequential Patterns (Cong Yu, Aalto University et al.) - The detection of sequential patterns in data is a basic functionality of modern data processing systems for complex event processing (CEP), OLAP, and retrieval-augmented generation (RAG). In practice, the respective engines typically evaluate multiple shared patterns simultaneously, in order to improve the result quality for downstream applications. The evaluation of a large number of patterns under tight latency bounds is challenging, though, since matching needs to maintain state, i.e., intermediate results, that grow exponentially in the input size. Hence, systems turn to best-effort processing, striving for maximal recall under a latency bound. Existing techniques, however, consider patterns in isolation, neglecting the optimization potential induced by state sharing and corresponding interactions and interference across shared patterns. We describe Sharp, a state management library that employs state reduction for efficient best-effort pattern matching in shared patterns. To this end, Sharp incorporates state sharing between patterns through a new abstraction, coined pattern-sharing degree (PSD). At runtime, PSD facilitates the categorization and indexing of partial pattern matches. Once a latency bound is exceeded, Sharp realizes best-effort processing by using a cost model to select a subset of partial matches for further processing in constant time. In experiments with real-world data, Sharp achieves a recall of 95%, 93% and 72% for pattern matching in CEP, OLAP, and RAG applications, under a bound of 50% of the average processing latency. • SafeLoad: Efficient Admission Control Framework for Identifying Memory-Overloading Queries in Cloud Data Warehouses (Yifan Wu, Zhejiang University et al.) - Memory overload is a common form of resource exhaustion in cloud data warehouses. When database queries fail due to memory overload, it not only wastes critical resources such as CPU time but also disrupts the execution of core business processes, as memory-overloading (MO) queries are typically part of complex workflows. If such queries are identified in advance and scheduled to memory-rich serverless clusters, it can prevent resource wastage and query execution failure. Therefore, cloud data warehouses desire an admission control framework with high prediction precision, interpretability, efficiency, and adaptability to effectively identify memory-overloading queries. However, existing admission control frameworks primarily focus on scenarios like SLA satisfaction and resource isolation, with limited precision in identifying MO queries. Moreover, there is a lack of publicly available MO-labeled datasets with workloads for training and benchmarking. To tackle these challenges, we propose SafeLoad, the first query admission control framework specifically designed to identify MO queries. Alongside, we release SafeBench, an open-source, industrial-scale benchmark for this task, which includes 150 million real queries. SafeLoad first filters out memory-safe queries using the interpretable discriminative rule. It then applies a hybrid architecture that integrates both a global model and cluster-level models, supplemented by a misprediction correction module to identify MO queries. Additionally, a self-tuning quota management mechanism dynamically adjusts prediction quotas per cluster to improve precision. Experimental results show that SafeLoad achieves state-of-the-art prediction performance with low online and offline time overhead. Specifically, SafeLoad improves precision by up to 66% over the best baseline and reduces wasted CPU time by up to 8.09x compared to scenarios without SafeLoad.read more
☆
Thu Sep 3 · 3:45 PM - 5:15 PM
Research 52 · Knowledge Graphs and Graph Search
● Research
📍 Commonwealth Ballroom C
Love-at-First-Sight: First Answers Without the Awkward Silence in Big Knowledge Graphs (Giannis Vassiliou, HMU et al.) - The increasing number of large knowledge graphs (KGs) now available online requires methods for their efficient exploration.Love-at-First-Sight: First Answers Without the Awkward Silence in Big Knowledge Graphs (Giannis Vassiliou, HMU et al.) - The increasing number of large knowledge graphs (KGs) now available online requires methods for their efficient exploration. Most of these KGs offer online SPARQL endpoints for querying and exploring their data. In a typical scenario, the users issue coarse, exploratory queries at the beginning, refining them further in the sequel in order to find the answer to the question in mind. However, those coarse exploratory queries are costly to evaluate as they usually involve many results and take too much time to be answered, or even worse, they time out, limiting the exploration potential of the data they expose. In this paper, we present the LFS (Love-at-First-Sight) system, offering a unique solution to the aforementioned problem, enabling users to efficiently get the first answers to their queries. More specifically, we are the first to define the problem of constructing first-sight summaries (FSS), i.e., summaries able to provide rapidly, first answers to user queries, relying on existing query logs. We provide effective algorithms for constructing both exact and approximate FSS under budget constraints with theoretical guarantees. We analytically and experimentally demonstrate latency reductions of up to two orders of magnitude over SPARQL endpoints and one order of magnitude over relevant baselines. • Noisy Interactive Graph Search: An Uncertainty-Based Approach with Online Modeling of Latent Expertise and Difficulty (Han Linghu, The Hong Kong University of Science and Technology (Guangzhou) et al.) - Interactive graph search (IGS) has emerged as a powerful information retrieval paradigm for various applications. Given a hierarchy and an oracle that typically relies on human intelligence such as crowdsourcing, IGS aims to identify the most precise concept for an unknown object while minimizing interaction costs with the oracle. Most existing algorithms simplify the problem by assuming a perfect oracle that always provides correct answers. Others adopt an idealized noisy oracle that models noises as explicit error rates specified in advance and locate the target with Bayesian inference guided by a node-wise querying strategy. However, in real-world scenarios, the oracle inevitably makes mistakes and prior knowledge of the oracle is often limited. Moreover, the node-wise querying strategy that lacks holistic awareness of the search state and ignores the global hierarchical structure usually yields suboptimal queries. As a result, existing solutions suffer from significant accuracy degradation and prohibitive interaction costs. To address these challenges, we introduce IGS-RTA. We first formulate the problem based on search uncertainty, explicitly accounting for the randomness of the search state and hierarchical relations. We then propose a querying strategy that maximizes the expected uncertainty decrement. Our rigorous theoretical analysis establishes a logarithmic upper bound on the query complexity. In addition, to adapt to noisy settings with limited prior knowledge, we analyze oracle expertise and task difficulties, which characterize two groups of meta-factors that influence real query answering. We model their relationships using a probabilistic graphical model and design techniques to estimate these latent factors online. We evaluate IGS-RTA on two real-world datasets against six baselines. Results show that IGS-RTA improves search accuracy by up to 52% while reducing monetary costs by up to 8x. • Nav-Index: A High-Performance, Adaptive Index for Shortest Path Queries in RDBMS (Maximilian Reif, Technical University of Munich et al.) - Shortest path queries are a fundamental operation on graphs with numerous applications. Efficiently executing shortest path searches in RDBMS is challenging, as graphs can not only be static relations but might also occur as ad-hoc intermediate results of complex analytical queries. Especially single-pair shortest path queries are difficult to accelerate, because they require minimizing the search space and intermediate results cannot be shared e.g. for multiple destinations. To address these challenges, we present Nav-Index, an adaptive index for relational DBMS that accelerates large-scale shortest path queries with seamless integration into SQL and relational algebra. Nav-Index switches adaptively between three modes: an ad-hoc generic mode using Dijkstra's algorithm, an ad-hoc optimized mode for sparse graphs leveraging contraction hierarchies (CH), and a table index mode for static graphs also leveraging CHs. Graph DBMS typically provide specific implementations for such queries and are valuable for many workloads. In practice, however, existing systems show limited throughput for large-scale graphs such as road networks and offer limited optimization for analytical relational processing. Embedding Nav-Index into an RDBMS allows for a seamless composition with other relational operators, query decorrelation, cost-based optimization, morsel-driven parallelism, and pipelining. In the case of sparse graphs, Nav-Index's CH mode transforms the graph into a contraction hierarchy, allowing algorithmic speedups by orders of magnitude. We implemented Nav-Index into the compiling database system Umbra and evaluate Nav-Index extensively on road networks ranging from small cities to the whole planet and social graphs. In our experiments, Nav-Index outperforms all other database systems that we have tested by orders of magnitude due to algorithmic advantages and an efficient implementation. It can index the road network of Europe in under 12 minutes and Earth in less than 68 minutes, allowing planet-scale shortest path queries in milliseconds in a general-purpose RDBMS. • CRAFT: Corpus Relatedness Analysis Using Fourier Transforms (kaiwen chen, university of Toronto et al.) - A fundamental challenge in data management is the efficient discovery of term relationships from massive, unstructured text corpora, a critical first step in knowledge graph construction. This discovery task, however, faces prohibitive computational barriers: the quadratic $O(N^2)$ complexity of an all-pairs analysis and the intractability of processing the full term-document matrix. While dimensionality reduction via embeddings offers a partial solution, the resulting vector proximity often captures broad thematic similarity, failing to isolate the precise co-occurrence signals required for high-quality relation extraction. This paper introduces CRAFT, a system that overcomes these limitations by re-casting term relatedness discovery as a scalable signal processing problem. CRAFT's methodology decouples the discovery process from both the term-document matrix and quadratic-time comparisons. First, it employs a randomized Fourier transform to sketch term occurrence signals directly into a low dimensional complex space, a process that provably preserves the inner products essential for correlation analysis without materializing the underlying matrix. Second, to break the quadratic barrier, CRAFT leverages the inherent sparsity of term relationships by formulating discovery as a compressed sensing task. This enables the recovery of significant correlations for any given term directly from its compressed sketch via an efficient Orthogonal Matching Pursuit algorithm, obviating the need for an all-pairs comparison. Our end-to-end implementation and comprehensive experimental evaluation show that CRAFT significantly outperforms modern baselines in both efficiency and precision, enabling high-quality relation discovery at a previously infeasible scale. • Lower-Bound Distance Queries under Partial Information (Swastik Biswas, New Jersey Institute of Technology et al.) - Given a metric space graph $G(V,E)$ in which only a subset of edge distances is known, we study a query-based model for answering lower bound distance queries on unknown edges, without access to any black-box distance oracle. For a queried pair of objects $(u,v)$ with unknown distance, the lower bound corresponds to the minimum feasible value of $d(u,v)$ that satisfies all known distances and the triangle inequality constraints. We develop algorithmic techniques that operate directly on the partially observed graph to answer such queries efficiently. Our framework carefully balances preprocessing overhead, query processing time, and tightness of the produced bounds, while providing provable guarantees under metric assumptions. The proposed methods significantly improve both theoretical understanding and practical performance for lower bound inference, as validated through extensive experiments on multiple large-scale real-world datasets.read more
☆
Thu Sep 3 · 3:45 PM - 5:15 PM
Research 53 · Responsible Data Management
● Research
📍 Commonwealth Ballroom B
On Fair Epsilon Net and Geometric Hitting Set (Mohsen Dehghankar, University of Illinois Chicago et al.) - Fairness has emerged as a formidable challenge in data-driven decisions.On Fair Epsilon Net and Geometric Hitting Set (Mohsen Dehghankar, University of Illinois Chicago et al.) - Fairness has emerged as a formidable challenge in data-driven decisions. Many of the data problems, such as creating compact data summaries for approximate query processing, can be effectively tackled using concepts from computational geometry, such as $\varepsilon$-nets. However, these powerful tools have yet to be examined from the perspective of fairness. To fill this research gap, we add fairness to classical geometric approximation problems of $\varepsilon$-net, $\varepsilon$-sample, and geometric hitting set. We introduce and address two notions of group fairness: demographic parity, which requires preserving group proportions from the input distribution, and custom-ratios fairness, which demands satisfying arbitrary target ratios. We develop two algorithms to enforce fairness - one based on sampling and another on discrepancy theory. The sampling-based algorithm is faster and computes a fair $\varepsilon$-net of size which is only larger by a $\log(k)$ factor compared to the standard (unfair) $\varepsilon$-net, where $k$ is the number of demographic groups. The discrepancy-based algorithm is slightly slower (for bounded VC dimension), but it computes a smaller fair $\varepsilon$-net. Notably, we reduce the fair geometric hitting set problem to finding fair $\varepsilon$-nets. This results in a $O(\log \mathsf{OPT} \times \log k)$ approximation of a fair geometric hitting set. Additionally, we show that under certain input distributions, constructing fair $\varepsilon$-samples can be infeasible, highlighting limitations in fair sampling. Beyond the theoretical guarantees, our experimental results validate the practical effectiveness of the proposed algorithms. In particular, we achieve zero unfairness with only a modest increase in output size compared to the unfair setting. • Unbiased Binning for Fairness-aware Attribute Representation (Abolfazl Asudeh, University of Illinois Chicago et al.) - Discretizing raw features into bucketized attributes is a common step before sharing a dataset. However, this process can inadvertently introduce bias and amplify unfairness in downstream tasks. In this paper, we address this issue by formulating the unbiased binning problem, which seeks bucketized attributes that satisfy group parity. We develop an efficient dynamic programming algorithm to solve this problem for equal-size binning. In practice, however, an unbiased binning may incur a high price of fairness or may not exist at all, particularly when group distributions differ substantially. To accommodate settings in which small deviations from perfect parity are acceptable, we introduce the epsilon-biased binning problem, which restricts group disparities across buckets to at most epsilon. We first present a dynamic programming algorithm, DP, that computes the optimal solution in quadratic time. While polynomial, DP does not scale to large datasets. To address this, we propose a practically scalable algorithm based on local search (LS) for epsilon-biased binning. A central component of LS is a divide-and-conquer algorithm (D&C) that quickly finds a solution in near-linear time. We prove that D&C always returns a valid solution whenever one exists. The LS algorithm then initiates a local search, using the D&C solution as the upper bound, to find the optimal solution. Our LS and D&C algorithms are general, and not limited to equal-size binning. To complement our theoretical analysis, we conduct extensive experiments on real-world and synthetic datasets. Besides confirming the efficiency of the algorithms, our experiments verify that while fairness-unaware binning can generate biased attribute representations, this bias can be significantly reduced at a negligible price of fairness. • Auditing for Demographic Bias in Opaque Rankings (Antonio Ferrara, CENTAI et al.) - Auditing algorithmic fairness is a critical challenge in high-stakes domains like hiring and credit scoring, especially given the intrinsic opacity of algorithmic decision-making systems. In this paper, we tackle the following problem: given a ranking of individuals, how can we assess whether the order is driven by protected attributes (e.g., gender or race) rather than task-relevant features, under a strict black-box assumption where the ranking mechanism cannot be queried? Building on kernel conditional independence and partial distance correlation, we introduce Condor, a model-agnostic audit framework. Condor first residualizes the ranking and protected attributes with respect to observables in a reproducing kernel Hilbert space. It then quantifies the remaining association via distance correlation on the residualized embeddings, returning a normalized effect-size score. This procedure captures general nonlinear dependencies without assuming access to latent scores, requires no hyperparameter fine-tuning, and naturally accommodates mixed continuous and categorical data. From Condor's effect-size score, we derive a hypothesis test for conditional independence. By combining this test with an unconditional independence test, auditors can achieve a comprehensive causal understanding of the protected attributes' influence. We validate our proposal on real and semi-synthetic datasets with controlled influence of the protected attributes on the ranking: our method reliably detects the influence of protected attributes, outperforming established statistical auditing baselines. • Sample-based Distinct Cardinality Estimation for Multiple Attributes in Multi-Dataset Queries (Mehnaz Tabassum Mahin, University of California, Riverside et al.) - Estimating the number of distinct values in an attribute or a set of attributes is one of the classical and open problems of cost-based query optimizers (CBOs). Such estimations can be very difficult to make in the presence of query selection predicates without examining the complete dataset. It becomes even harder for a multi-dataset (i.e., join) query with selection predicates. Recent advances in CBOs have introduced sample-based approaches, which maintain stored samples on the underlying datasets to improve the accuracy of cardinality and selectivity estimation during query compilation. Leveraging these stored samples, this paper addresses the important yet challenging problem of estimating the number of distinct values in an attribute or a set of attributes in a multi-dataset query. We refer to our proposed sample-based approach as the MAMD (Multi-Attribute, Multi-Dataset) approach. The MAMD approach works for join queries with or without selection predicates and is also effective for estimating the number of distinct values in single-dataset queries. We present an experimental evaluation of the proposed MAMD approach with synthetic and real-world datasets, namely the TPC-H and the IMDB benchmark datasets. We demonstrate how it can estimate the number of distinct values with moderately low relative errors and with low storage overhead and execution time. We also investigate how the MAMD approach performs when we scale up the size of the database. • Algorithmic Data Minimization for Machine Learning over Internet-of-Things Data Streams (Ted Shaowang, University of Chicago et al.) - Machine learning can analyze vast amounts of data generated by IoT devices to identify patterns, make predictions, and enable real-time decision-making. This raises significant privacy concerns, necessitating the application of data minimization - a foundational principle in emerging data regulations, which mandates that service providers only collect data that is directly relevant and necessary for a specified purpose. Despite its importance, data minimization lacks a precise technical definition in the context of sensor data, where collections of weak signals make it challenging to apply a binary "relevant and necessary" rule. This paper provides a technical interpretation of data minimization in the context of sensor streams, explores practical methods for implementation, and addresses the challenges involved. Through our approach, we demonstrate that our framework can reduce user identifiability by up to 16.7% while maintaining accuracy loss below 1%, offering a viable path toward privacy-preserving IoT data processing.read more
☆
Thu Sep 3 · 3:45 PM - 5:15 PM
Research 54 · Cohesive Subgraph Discovery
● Research
📍 Commonwealth Ballroom A
Aggregating maximal cliques in real-world graphs (Noga Alon, Princeton University et al.) - Maximal clique enumeration is a fundamental graph mining task, but its utility is often limited by computational intractability and highly redundant output.Aggregating maximal cliques in real-world graphs (Noga Alon, Princeton University et al.) - Maximal clique enumeration is a fundamental graph mining task, but its utility is often limited by computational intractability and highly redundant output. To address these challenges, we introduce \emph{$\rho$-dense aggregators}, a novel approach that succinctly captures maximal clique structure. Instead of listing all cliques, we identify a small collection of clusters with edge density at least $\rho$ that collectively contain every maximal clique. In contrast to maximal clique enumeration, we prove that for all $\rho < 1$, every graph admits a $\rho$-dense aggregator of \emph{sub-exponential} size, $n^{O(\log_{1/\rho}n)}$, and provide an algorithm achieving this bound. For graphs with bounded degeneracy, a typical characteristic of real-world networks, our algorithm runs in near-linear time and produces near-linear size aggregators. We also establish a matching lower bound on aggregator size, proving our results are essentially tight. In an empirical evaluation on real-world networks, we demonstrate significant practical benefits for the use of aggregators: our algorithm is consistently faster than the state-of-the-art clique enumeration algorithm, with median speedups over $2.5\times$ for $\rho=0.1$ (and over $350\times$ in an extreme case), while delivering a much more concise structural summary. • Revisiting the Maximum Defective Clique Problem: Faster Branching and a Tighter Upper Bound (Kewu Yang, Harbin Institute of Technology, Shenzhen et al.) - The $k$-defective clique model relaxes the strict completeness constraint of the traditional clique by allowing up to $k$ missing edges, providing a robust formulation for detecting cohesive structures in noisy graphs. Consequently, the maximum $k$-defective clique problem has attracted significant attention. State-of-the-art exact algorithms predominantly adopt the branch-and-bound framework, which recursively partitions the current problem instance (or branch) into two sub-problems via a branching procedure, until each sub-problem becomes trivially solvable. However, this strategy often leads to excessive branching by overlooking intermediate sub-problems that are non-trivial yet efficiently solvable. While recent studies have attempted to refine branching procedures, they fail to address this structural redundancy. To address this, we propose BBRes, a framework that incorporates a novel early termination strategy into the recursive branching process. By employing a specialized polynomial-time solver to identify and resolve tractable sub-instances, BBRes effectively avoids redundant branching steps. Additionally, we design a tailored branching strategy that synergizes with this termination mechanism. As a result, BBRes achieves an improved theoretical worst-case time complexity. Furthermore, to enhance practical performance, we propose a tighter upper bound based on a novel double graph coloring method integrated with max-flow techniques, which is orthogonal to the branching framework. Extensive experiments demonstrate that BBRes achieves at least 2X speedup over state-of-the-art methods on a substantial fraction of the datasets. • CREST: Approximate k-Clique Counting in Real-World Networks via Refinement of Star-Based Sample Space (Yehyun Nam, Seoul National University et al.) - A k-clique, defined as the set of k pairwise adjacent vertices, plays a fundamental role in the analysis of real-world networks. Many downstream tasks require computing the number of k-cliques, yet listing or exact counting is often computationally prohibitive on massive networks, making approximate counting the only scalable option. Existing algorithms for approximate k-clique counting primarily use the Monte Carlo method. These algorithms construct a sample space, which is a collection of k-vertex sets including all k-cliques. They then perform sample trials, where each trial consists of selecting a k-vertex set uniformly at random from the sample space and checking whether it forms a k-clique. However, existing algorithms suffer from huge sample spaces and expensive sample trials. In this paper, we present CREST, an efficient Monte Carlo algorithm for approximate k-clique counting. We introduce a suite of novel techniques to address the two main objectives: (1) obtaining a small sample space, and (2) reducing the cost of sample trials. We propose a novel sample space refinement strategy to obtain a smaller sample space, and a star-based sampling approach that addresses both of the main objectives. We also develop a combinatorial method to obtain exact clique counts for certain subgraphs, effectively reducing their sample spaces to the extreme. Moreover, we present a new stopping criterion that satisfies the target accuracy requirement with fewer samples. Extensive experiments on real-world networks demonstrate that CREST outperforms the state-of-the-art algorithm by up to two orders of magnitude in running time, while maintaining the specified accuracy requirement. • Scalable Approximate Biclique Counting over Large Bipartite Graphs (Jingbang Chen, University of Waterloo et al.) - Counting \((p,q)\)-bicliques in bipartite graphs is crucial for a variety of applications, from recommendation systems to cohesive subgraph analysis. Yet, it remains computationally challenging due to the combinatorial explosion to exactly count the \((p,q)\)-bicliques. In many scenarios, e.g., graph kernel methods, however, exact counts are not strictly required. To design a scalable and high-quality approximate solution, we novelly resort to \emph{\((p,q)\)-broom}, a special spanning tree of the $(p,q)$-biclique, which can be counted via graph coloring and efficient dynamic programming. Based on the intermediate results of the dynamic programming, we propose an efficient sampling algorithm to derive the approximate $(p,q)$-biclique count from the \((p,q)\)-broom counts. Theoretically, our method offers unbiased estimates with provable error guarantees. Empirically, our solution outperforms existing approximation techniques in both accuracy (up to 8$\times$ error reduction) and runtime (up to 50$\times$ speedup) on nine real-world bipartite networks, providing a scalable solution for large-scale \((p,q)\)-biclique counting.read more
☆
Thu Sep 3 · 3:45 PM - 5:15 PM
Demo Group C · Data Quality, Analytics & Applications
● Demos
📍 Marina Ballrooms II - IV
32 demos in this group, shown twice during the conference.32 demos in this group, shown twice during the conference. Includes: ImputePilot: A Graphical Model Selection Toolkit for Time Series Imputation • LLM-CER: An Interactive System for In-context Clustering-based Entity Resolution with Large Language Models • ECO-Hadoop: Energy Consumption Optimization System for Apache Hadoop • QueryCraft: A Natural Language-Driven NoSQL Database Querying System Powered by Large Language Models • ChatQPT: Towards Conversing with Relational Query Engines • LIMA: Denial Constraint Discovery in Large Dynamic Databases • DA-Studio: An Agentic System for End-to-End Data Analysis • Provlepsis4j: Querying Future Graphs in Neo4j • GALACTICA: An Interactive System for Local and Global Counterfactual Explanations for Time-series Clustering • The Case for Multi-Version Experimental Evaluation (MVEE) • … +22 moreread more
Built for you by(617)-PATRICK, Champion AI speaker
97 research papers presented as posters. Full list on the official program page (footer link) - the papers also appear in their research sessions on this schedule.
CDMS · Keynote 1: Towards AI-Generated Database Management Systems
● CDMS
Keynote
📍 Grand Ballroom C
Recently, a new generation of large language models (LLMs), combined with powerful frameworks such as Claude Code, has made it practical to generate large code bases such as C and C++.Recently, a new generation of large language models (LLMs), combined with powerful frameworks such as Claude Code, has made it practical to generate large code bases such as C and C++. In my talk, I will discuss several recent and ongoing projects in the Cornell Database Group, aimed at leveraging LLMs for synthesizing highly efficient code for data processing. In particular, I will focus on GenDB, a multi-agent system that generates code for processing complex SQL queries that is specialized to the particularities of the target database, workload, and hardware. The resulting code often outperforms a variety of classical database systems by significant margins. I will conclude my talk by discussing open problems and avenues for future work.read more
Immanuel TrummerAssociate Professor of Computer Science, Cornell University, USA
☆
Fri Sep 4 · 8:45 AM - 9:00 AM
TaDA · TabClean: Scalable Tabular Data Cleaning via Reusable LLM-Synthesized Programs
● TaDA
📍 Grand Ballroom D
Yibo WangPurdue University
Riteng Zhang
Bharat Bhargava
Chunwei Liu
☆
Fri Sep 4 · 8:45 AM - 9:25 AM
VecDB · Keynote 1: Graph-Based Algorithms for Similarity Search: Challenges, Opportunities and Connections
● VecDB
Keynote
📍 Grand Ballroom E
Piotr IndykMIT
☆
Fri Sep 4 · 8:45 AM - 8:50 AM
SCCP · Opening Remark
● SCCP
📍 Commonwealth Ballroom C
☆
Fri Sep 4 · 8:50 AM - 9:00 AM
ADS · Opening Remarks & Keynote Introduction
● ADS
📍 Commonwealth Ballroom A
☆
Fri Sep 4 · 8:50 AM - 9:40 AM
SCCP · Keynote: Testing Consistency of Distributed Databases at AWS
● SCCP
Keynote
📍 Commonwealth Ballroom C
Checking whether a given database run is correct with respect to the intended isolation-level specification is often computationally hard.Checking whether a given database run is correct with respect to the intended isolation-level specification is often computationally hard. I will describe our ongoing work at Amazon Web Services on a tool that makes this problem tractable by exploiting database-internal information, such as transaction timestamps assigned by the concurrency control, and how we applied it to Aurora Limitless Database - a horizontally scalable variant of Aurora PostgreSQL with strongly consistent distributed transactions.read more
Alexey GotsmanResearch Professor, IMDEA Software Institute and Amazon Web Services
Built for you by(617)-PATRICK, Champion AI speaker
TaDA · LDI: Localized Data Imputation for Text-Rich Tables
● TaDA
📍 Grand Ballroom D
Soroush OmidvartehraniUniversity of Alberta
Davood Rafiei
☆
Fri Sep 4 · 9:00 AM - 10:00 AM
ADS · Keynote: The Post-Developer Era: From Generating Databases to Building Enterprise Applications Without Developers
● ADS
Keynote
📍 Commonwealth Ballroom A
Talk 9:00-9:45 AM (45 min) + Q&A 9:45-10:00 AM (15 min).
Tim KraskaProfessor, MIT
☆
Fri Sep 4 · 9:15 AM - 9:30 AM
TaDA · Predicting Table Joinability in Data Lakes using a Metadata Knowledge Graph
● TaDA
📍 Grand Ballroom D
Sola ShiraiIBM Research
Debarun Bhattacharjya
Oktie Hassanzadeh
Gaetano Rossiello
☆
Fri Sep 4 · 9:25 AM - 10:05 AM
CDMS · Keynote 2: The AI Frontier for Composable Data Systems
● CDMS
Keynote
📍 Grand Ballroom C
The rise of machine learning and agentic development has fundamentally reshaped the challenges facing database developers. These workloads have different query patterns, blending OLAP workloads with search and indexing access patterns more commonly associated with OLTP.The rise of machine learning and agentic development has fundamentally reshaped the challenges facing database developers. These workloads have different query patterns, blending OLAP workloads with search and indexing access patterns more commonly associated with OLTP. The users themselves are often now agents, requiring a new user experience, a heightened focus on API development, and more robust safeguards and boundaries. Finally, the development of these databases is often assisted by, if not entirely created by, coding agents. Composable data systems are ideally positioned to take advantage of the new environment. In this talk we will review the opportunities and challenges we’ve encountered when integrating AI into an open source database project, including architecture, business, and development challenges.read more
Weston PaceSoftware Engineer, LanceDB
☆
Fri Sep 4 · 9:25 AM - 10:15 AM
VecDB · Poster Session 1
● VecDB
📍 Galleria
Almost Navigable Graphs (Pratyush Avi, Christopher Musco) • Turbocharging PiPNN for Proximity and k-NN Graph Building (Tobias Rubel et al.) • Multi-Vector Embeddings are Provably More Expressive than Single Vector Embeddings (Rajesh Jayaram) • FastGANN: Marrying Quantization with Pruning for...Almost Navigable Graphs (Pratyush Avi, Christopher Musco) • Turbocharging PiPNN for Proximity and k-NN Graph Building (Tobias Rubel et al.) • Multi-Vector Embeddings are Provably More Expressive than Single Vector Embeddings (Rajesh Jayaram) • FastGANN: Marrying Quantization with Pruning for GPU ANNS (ZeAi Sun, Jinxu Liu) • Can Language Models Actually Retrieve In-Context? Drowning in Documents at Million Token Scale (Siddharth Gollapudi et al.) • Direct Path Optimization for Faster Ingestion and Sparser Graph Index in Vector Database (Tianji Yang, Xuhao Chen) • Characterizing Locality in Large-Scale Vector Search Workloads of Agentic AI Systems (Seungjin Lee et al.) • Learning Partition Trees for Nearest Neighbor Search (Ashwin Padaki et al.) • Metadata-Filtered Vector Search in the WebIQ Grounding Index (Fang Liu et al.) • Passing the Baton: High Throughput Distributed Disk-Based Vector Search with BatANN (Nam Anh Dang et al.) • ANNLib: A Development Framework for Efficient Approximate Nearest Neighbor Search (Zheqi Shen et al.) • Directional Beam Search: I/O-Efficient ANNS on GPU (Zikun Wang et al.) • In-Place Updates of a Graph Index for Streaming Approximate Nearest Neighbor Search (Haike Xu et al.) • Aggregate Recall Is Not Per-Slice Reliability: A Distribution-Free Coverage Contract for Embedding Retrieval (Rob Sneiderman) • SPI: Query-Depth-Adaptive Indexing for Streaming RAG in Vector Databases (Dong Liu et al.) • Mimicking Datasets at Billion Scale for Vector Search Benchmarking (Jinsol Park et al.) • Best Paper Award: A Bi-metric Framework for Efficient Nearest Neighbor Search (Haike Xu, Piotr Indyk, Sandeep Silwal) • The RaBitQ Library (Jianyang Gao et al.) • IVF² Index: Fusing Classic and Spatial Inverted Indices for Fast Filtered ANNS (Ben Landrum et al.) • Unified Multimodal Interleaved Document Representation for Retrieval (Jaewoo Lee et al.) • DistributedANN: Efficient Scaling of a Single DiskANN Graph Across Thousands of Computers (Philip Adams et al.) • Scalable k-Means Clustering for Large k via Seeded Approximate Nearest-Neighbor Search (Eliot W. Robson et al.) • FrugalRAG: Learning to retrieve and reason for multi-hop QA (Abhinav Java et al.) • Don’t Lag, RAG: Training-Free Adversarial Detection Using RAG (Roie Kazoom et al.) • ConvSearch-R1: Enhancing Query Reformulation for Conversational Search with Reasoning via Reinforcement Learning (Changtai Zhu et al.) • Graph-Based Vector Search: An Experimental Evaluation of the State-of-the-Art (Ilias Azizi et al.)read more
Extending Elle for Transaction Workloads with Duplicate Values (Zhiheng Cai, Si Liu, Hengfeng Wei, Yuxing Chen) • Efficient Black-Box Serializability Checking in the Presence of Range Predicates (Qikang Liu, Si Liu, Yuepeng Wang) • Specifying and Analyzing Transactional Consistency Models...Extending Elle for Transaction Workloads with Duplicate Values (Zhiheng Cai, Si Liu, Hengfeng Wei, Yuxing Chen) • Efficient Black-Box Serializability Checking in the Presence of Range Predicates (Qikang Liu, Si Liu, Yuepeng Wang) • Specifying and Analyzing Transactional Consistency Models with Predicates, Extended Abstract (Hengfeng Wei, Si Liu, Yuxing Chen)read more
☆
Fri Sep 4 · 9:45 AM - 10:15 AM
TaDA · Lightning Talks Session (3' talk, no Q&A)
● TaDA
📍 Grand Ballroom D
Predictive Query Language: A Domain-Specific Language for Predictive Modeling on Relational Databases (Vid Kocijan et al.) • Understanding Domain-Aware Distribution Alignment in Budgeted Entity Matching (Nicholas Pulsone et al.) • Representation, Retrieval, and Decision Space: A Case Study on Diagnosing...Predictive Query Language: A Domain-Specific Language for Predictive Modeling on Relational Databases (Vid Kocijan et al.) • Understanding Domain-Aware Distribution Alignment in Budgeted Entity Matching (Nicholas Pulsone et al.) • Representation, Retrieval, and Decision Space: A Case Study on Diagnosing LLM Failures in Column Type Annotation (Jianhao Cao, Rachel Pottinger) • TEmBed-T: A Multi-Dimensional Benchmark for Table-Level Embeddings (Ayeen Poostforoushan et al.) • PaperUnPlot: Benchmarking Chart-to-Table in the Wild (Daniele Bertillo et al.) • Towards Budget-Aware Dense Retrieval for Tables: Trade-offs, Alternatives and Future Directions (Inwon Kang et al.) • PFN-Syn: Generating Synthetic Tabular Data with Prior-Data Fitted Networks (Michael Zuo et al.) • Speeding Up Transformation Search with Lightweight Statistics (Rubab Sarfraz, Boris Glavic) • Curriculum Matters: Data-Efficient Relational PFN Pre-training with Synthetic Data (Mohammad Sadeq Abolhasani, Viswanath Ganpathy)read more
Built for you by(617)-PATRICK, Champion AI speaker
ADS · DemandPrep: Demand-Driven Data Preparation via Agentic Action Allocation and Operator-Grounded Execution
● ADS
📍 Commonwealth Ballroom A
Zekai QianHarbin Institute of Technology
Xiaoou DingHarbin Institute of Technology
Hongzhi WangHarbin Institute of Technology
Chen WangTsinghua University
☆
Fri Sep 4 · 10:45 AM - 11:25 AM
CDMS · Keynote 3: Synthesizing Database Kernels - The Next Frontier
● CDMS
Keynote
📍 Grand Ballroom C
Coding agents are reshaping how databases are built. We already see people starting to synthesize peripheral elements of their data stack: connectors, pipelines, and specialized solutions. Database kernels are the natural next step.Coding agents are reshaping how databases are built. We already see people starting to synthesize peripheral elements of their data stack: connectors, pipelines, and specialized solutions. Database kernels are the natural next step. What used to require scarce resources of database internals experts is becoming possible for wide audiences of developers. But synthesizing core database components is scary; there need to be strong guarantees around correctness, transactional consistency, performance, and scalability. This is where harnesses play an outsized role. What are the traits of such harnesses, and how can database systems be composed in this new world?read more
Benjamin WagnerCEO, Firebolt Analytics
☆
Fri Sep 4 · 10:45 AM - 11:15 AM
TaDA · Keynote: Ranking Algorithms for Semantic Sort and Top-K
● TaDA
Keynote
📍 Grand Ballroom D
Semantic operators abstract LLM calls in SQL clauses, gaining traction as an easy method to analyze semi-structured, unstructured, and multimodal datasets in addition to tabular data.Semantic operators abstract LLM calls in SQL clauses, gaining traction as an easy method to analyze semi-structured, unstructured, and multimodal datasets in addition to tabular data. In particular, semantic top-K operators have applications in search and information retrieval scenarios where the user’s intent is a ranking function expressed with a natural language query. In this talk I will describe a technique that advocates keeping SQL while extending it with first-class support for ML User-defined Functions (UDFs) as rankers, then present ListK, a solution for top-K search as a semantic operator that supports replacing the ranking function entirely with natural language prompts, and discuss a framework for modeling the noise of LLM output ranking.read more
Fatemeh NargesianAssistant Professor of Computer Science, University of Rochester
☆
Fri Sep 4 · 10:45 AM - 11:25 AM
VecDB · Keynote 2: Information Retrieval in 2030
● VecDB
Keynote
📍 Grand Ballroom E
Omar KhattabMIT
☆
Fri Sep 4 · 10:45 AM - 11:00 AM
ADS · “Skill Issues”: Data-Centric Optimization of Lakehouse Agents
● ADS
📍 Commonwealth Ballroom A
Nicole Rose SchneiderUniversity of Maryland
Davide GhilardiUniversità Milano Bicocca
Giacomo PiccininiBauplan Labs
Jacopo TagliabueBauplan Labs
☆
Fri Sep 4 · 10:45 AM - 11:15 AM
SCCP · Session 2: Who Broke Serializability and Snapshot Isolation? (Chair: Shuai Mu, Stony Brook University)
● SCCP
📍 Commonwealth Ballroom C
Diagnosing the Structure of Strict-Serializability Violations Across Spanner-like Read-Only Transaction Protocols (Sejong Kim, Yon Dohn Chung) • Verified Key-Value Stores Satisfying Snapshot Isolation, Extended Abstract (Arnaud Daby-Seesaram, Léon Ducruet, Lars Birkedal, Amin Timany)
Built for you by(617)-PATRICK, Champion AI speaker
Best Paper: Notified Serializability: A Consistency Model for Concurrent LLM Agents (Hongtao Lyu, Dingyan Zhang, Mingyu Wu, Xingda Wei, Haibo Chen) • Beyond Pass Rate: A Hierarchy of Behavioral Consistency for LLM Agents (Tian Lu, Zikai Wang, Cheng Tan)
☆
Fri Sep 4 · 11:25 AM - 11:45 AM
CDMS · Invited Talk: Axiom: Composable Query Engines Built on Velox
● CDMS
Invited Talk
📍 Grand Ballroom C
Over the past several years, Velox - an open-source C++ library - has become a shared foundation for data-intensive computation, providing reusable building blocks (aggregation, sort, join, a nested/complex type system, vectorized functions, and Arrow-compatible memory) that power engines across...Over the past several years, Velox - an open-source C++ library - has become a shared foundation for data-intensive computation, providing reusable building blocks (aggregation, sort, join, a nested/complex type system, vectorized functions, and Arrow-compatible memory) that power engines across the industry, from Presto (Prestissimo) and Spark (Gluten) to PyTorch data loading (TorchArrow). Velox unified the execution layer; Axiom completes the stack as the front-end: SQL and DataFrame parsing, logical planning, cost-based optimization, and query orchestration that together turn a query into an optimized, multi-fragment Velox plan.read more
Masha BasmanovaSoftware Engineer, Meta Platforms Inc., USA
☆
Fri Sep 4 · 11:25 AM - 12:05 PM
VecDB · Oral Session 1
● VecDB
📍 Grand Ballroom E
Almost Navigable Graphs (Pratyush Avi, Christopher Musco) • Turbocharging PiPNN for Proximity and k-NN Graph Building (Tobias Rubel et al.) • Best Paper Award: A Bi-metric Framework for Efficient Nearest Neighbor Search (Haike Xu, Piotr Indyk, Sandeep Silwal) • The...Almost Navigable Graphs (Pratyush Avi, Christopher Musco) • Turbocharging PiPNN for Proximity and k-NN Graph Building (Tobias Rubel et al.) • Best Paper Award: A Bi-metric Framework for Efficient Nearest Neighbor Search (Haike Xu, Piotr Indyk, Sandeep Silwal) • The RaBitQ Library (Jianyang Gao et al.) • IVF² Index: Fusing Classic and Spatial Inverted Indices for Fast Filtered ANNS (Ben Landrum et al.)read more
VecDB · Keynote 3: From Search to Research: Training Agents and Retrievers for Deep Research
● VecDB
Keynote
📍 Grand Ballroom E
Akari AsaiCMU
☆
Fri Sep 4 · 1:45 PM - 2:15 PM
Agents+Graphs · Keynote 1: Thinking in DAGs: How Humans and Agents Collaborate in Data Science
● Agents+Graphs
Keynote
📍 Grand Ballroom C
Chen LiUC Irvine
☆
Fri Sep 4 · 1:45 PM - 1:55 PM
BioDMS · Workshop Intro
● BioDMS
📍 Grand Ballroom D
☆
Fri Sep 4 · 1:45 PM - 2:30 PM
QCDKM · Keynote: Relational Database Engines on Quantum Platforms
● QCDKM
Keynote
📍 Commonwealth Ballroom C
Quantum computing is rapidly progressing from theory to practice, with fault-tolerant hardware expected within the coming decade.Quantum computing is rapidly progressing from theory to practice, with fault-tolerant hardware expected within the coming decade. This keynote explores the applicability of quantum computing to database management systems, examining recent advances in applying quantum algorithms to core DBMS tasks including query optimization and index configuration selection, the fundamental challenges in adapting database tasks to quantum hardware, and a roadmap of open research problems for practical quantum database systems.read more
Manish KesarwaniSenior Research Scientist, IBM Research, India
☆
Fri Sep 4 · 1:55 PM - 2:30 PM
BioDMS · Keynote: Hammers for Biomedical Nails: From Matching to Meaning in Data Harmonization
● BioDMS
Keynote
📍 Grand Ballroom D
Biomedical research is increasingly limited not by data availability, but by our ability to find, understand, and integrate it.Biomedical research is increasingly limited not by data availability, but by our ability to find, understand, and integrate it. In this talk, developed in the context of the ARPA-H Biomedical Data Fabric (BDF) program, I discuss systems we have built for biomedical data harmonization, combining classical algorithms, large language models, interactive visualization, and human validation - and reflect on design patterns for combining LLMs with algorithmic methods, keeping humans meaningfully in the loop, and interfaces that go beyond text-based chat.read more
Juliana FreireInstitute Professor, Tandon School of Engineering, New York University
Built for you by(617)-PATRICK, Champion AI speaker
Agents+Graphs · Keynote 2: Rethinking Graphs for the Agentic Enterprise
● Agents+Graphs
Keynote
📍 Grand Ballroom C
Yunyao LiAdobe
☆
Fri Sep 4 · 2:25 PM - 3:15 PM
VecDB · Poster Session 2
● VecDB
📍 Galleria
JAG: Joint Attribute Graphs for Filtered Nearest Neighbor Search (Haike Xu et al.) • VQ-bench: a composable vector quantization framework (Edo Liberty et al.) • VIBE: Vector Index Benchmark for Embeddings (Elias Jääsaari et al.) • MV-IVF: Multi-Vector Retrieval via...JAG: Joint Attribute Graphs for Filtered Nearest Neighbor Search (Haike Xu et al.) • VQ-bench: a composable vector quantization framework (Edo Liberty et al.) • VIBE: Vector Index Benchmark for Embeddings (Elias Jääsaari et al.) • MV-IVF: Multi-Vector Retrieval via Multi-Vector Clustering (Kishen N Gowda et al.) • LOCALE: Local-Alignment Embeddings for Noise-Robust DNA Search at SRA Scale (Ryan Synk et al.) • Quantization Beyond Uniform Bit Allocation (K. S. Sreeramji et al.) • DARTH+: Vector Search with Declarative Recall and Guarantees (Manos Chatzakis et al.) • Quantizing Sampling-Based Inner Product Sketches (Gabriel M. Silva et al.) • Beyond HyDE: Cross-Model Hypothesis Diversity for Robust Dense Retrieval (Siddhesh More, Kunal Jadhav) • Revisiting RaBitQ and TurboQuant: A Symmetric Comparison of Methods, Theory, and Experiments (Jianyang Gao et al.) • Plasma: A Layout-Aware Benchmark Reveals Memory Layout Matters for Graph-based ANNS on GPU (Yutaro Oguri et al.) • The PAG Library: Projection-Augmented Graph for Approximate Nearest Neighbor Search (Kejing Lu et al.) • Speakeasy: Billion-Scale Two-Server Private Semantic Search (Vihan Lakshman et al.) • Filtered Approximate Nearest Neighbor Search in Vector Databases: System Design and Performance Analysis (Abylay Amanbayev et al.) • Range Retrieval with Graph-Based Indices (Magdalen Dobson Manohar et al.) • Stop Indexing at Full Precision: Revisiting Clustering for Vector Embeddings (Leonardo Kuffo, Peter Boncz)read more
Mediverse: Multimodal Clinical Exploration and Search on a Single Graph (Andra Ionescu, Postdoctoral Researcher, KTH Royal Institute of Technology) - Lightning talk • MedSQLX: Translation of Medical Queries into UDF-Centric SQL (Catlynh Nguyen, Arizona State University) - Project talk •...Mediverse: Multimodal Clinical Exploration and Search on a Single Graph (Andra Ionescu, Postdoctoral Researcher, KTH Royal Institute of Technology) - Lightning talk • MedSQLX: Translation of Medical Queries into UDF-Centric SQL (Catlynh Nguyen, Arizona State University) - Project talk • An Analysis Graph for Statistical Genetics Agents (Stephen Dorn, PhD student, Computational Medicine Dept., UCLA) - Project talk • Right for the Wrong Reasons: A Benchmark for Hallucination and Clinical Safety in AI Health Triage (Sreeram Marimuthu, Worcester Polytechnic Institute) - Project talkread more
☆
Fri Sep 4 · 2:30 PM - 2:45 PM
QCDKM · Evaluating a Quantum Solution: How hard can it be?
● QCDKM
📍 Commonwealth Ballroom C
Daniel Aarao Reis Arturi
Emilie Fahber
Stefanie Scherzinger
Bettina Kemme
☆
Fri Sep 4 · 2:45 PM - 3:15 PM
Agents+Graphs · Keynote 3: Agentic Memory Through the Lens of Incremental View Updates
● Agents+Graphs
Keynote
📍 Grand Ballroom C
Eric LoThe Chinese University of Hong Kong
☆
Fri Sep 4 · 2:45 PM - 3:00 PM
QCDKM · What Quantum Encoding to Use for your Knowledge Graph?
Clinical research, operation and clinical trials rely on accurate and actionable data, yet manual chart review is slow and difficult to scale.Clinical research, operation and clinical trials rely on accurate and actionable data, yet manual chart review is slow and difficult to scale. This talk introduces Brim: an AI-Guided Chart Abstraction Platform, which combines foundational large language models with structured workflows to assist reviewers in extracting clinical variables from unstructured medical records.read more
Daniel FabbriAssociate Professor, Vanderbilt University
☆
Fri Sep 4 · 3:15 PM - 3:45 PM
Coffee Break
● Breaks & Social
📍 Grand Ballroom Foyer
Built for you by(617)-PATRICK, Champion AI speaker
VIBE: Vector Index Benchmark for Embeddings (Elias Jääsaari et al.) • Multi-Vector Embeddings are Provably More Expressive than Single Vector Embeddings (Rajesh Jayaram) • JAG: Joint Attribute Graphs for Filtered Nearest Neighbor Search (Haike Xu et al.) • VQ-bench: a...VIBE: Vector Index Benchmark for Embeddings (Elias Jääsaari et al.) • Multi-Vector Embeddings are Provably More Expressive than Single Vector Embeddings (Rajesh Jayaram) • JAG: Joint Attribute Graphs for Filtered Nearest Neighbor Search (Haike Xu et al.) • VQ-bench: a composable vector quantization framework (Edo Liberty et al.)read more
☆
Fri Sep 4 · 3:45 PM - 4:15 PM
Agents+Graphs · Keynote 4: Graph Memory for AI Agents: Design and Practice with MemoryBase
● Agents+Graphs
Keynote
📍 Grand Ballroom C
Cheng ChenByteDance
☆
Fri Sep 4 · 3:45 PM - 4:15 PM
BioDMS · Keynote: HuBMAP and the search for scalable biomedical data portals
● BioDMS
Keynote
📍 Grand Ballroom D
His research group develops visual interfaces and computational techniques that enable scientists and clinicians to efficiently interact with biomedical data.His research group develops visual interfaces and computational techniques that enable scientists and clinicians to efficiently interact with biomedical data. They developed the HuBMAP data portal for discovering, visualizing and downloading standardized multi-modal spatial and single-cell data from healthy human tissues. (Talk title as billed on the BioDMS program; no separate abstract published.)read more
Nils GehlenborgAssociate Professor, Department of Biomedical Informatics, Harvard University
☆
Fri Sep 4 · 3:45 PM - 4:30 PM
QCDKM · Keynote: Solving Hard Database Optimization Problems via Digital and Quantum Annealing
● QCDKM
Keynote
📍 Commonwealth Ballroom C
Database management systems give rise to a plethora of NP-hard optimization problems where solution quality has a dramatic impact on performance.Database management systems give rise to a plethora of NP-hard optimization problems where solution quality has a dramatic impact on performance. I will discuss over a decade of research leveraging quantum computing and quantum-inspired accelerators to make database optimization more efficient, from a VLDB 2016 paper on quantum annealers through our most recent SIGMOD 2026 work overcoming the limited qubit counts of current machines using domain-specific problem decomposition.read more
Immanuel TrummerAssociate Professor of Computer Science, Cornell University, USA
Agents+Graphs · Panel: Databases, Data Science, and AI Futures
● Agents+Graphs
Panel
📍 Grand Ballroom C
Chen LiUC Irvine
Cheng ChenByteDance
Da YanIndiana University Bloomington
☆
Fri Sep 4 · 4:15 PM - 4:50 PM
BioDMS · Submitted Talks
● BioDMS
📍 Grand Ballroom D
The Data Management Crisis Behind AI-Based Brain MRI Diagnosis: Heterogeneity, Governance, and Reproducibility (Tanmoy Debnath, Senior Lecturer in Engineering, Charles Sturt University) - Lightning talk • Scaling Population Genomics to a Million: Genomic Variant Store as a Variant Data Management...The Data Management Crisis Behind AI-Based Brain MRI Diagnosis: Heterogeneity, Governance, and Reproducibility (Tanmoy Debnath, Senior Lecturer in Engineering, Charles Sturt University) - Lightning talk • Scaling Population Genomics to a Million: Genomic Variant Store as a Variant Data Management Engine (Aaron Hatcher, Principal Software Engineer, Broad Institute of MIT and Harvard) - Project talk • Data Gatherer Revisited: Scalable Dataset Reference Extraction from Biomedical Literature (Pietro Marini, FAIR Data Engineer, DataTecnica) - Project talk • Similarity-Based Load-Balancing for Distributed Genomic Indices (Woodward (Buz) Galbraith, PhD student, Northeastern University) - Project talkread more
QCDKM · Improving Join Order Optimization on Gate-Based Quantum Computers via Structured Parameter Initialization
● QCDKM
📍 Commonwealth Ballroom C
Divya Shekar
Ruokun Wu
Dhanvi Bharadwaj
Gokul Ravi
Lin Ma
☆
Fri Sep 4 · 4:45 PM - 5:00 PM
QCDKM · Quantum-Ready Approximate Data Management using Tensor-Network Physical Pages
● QCDKM
📍 Commonwealth Ballroom C
Muzhi Chen
Hanwen Liu
Xuanhe Zhou
Hongming Xu
Zhenghao Li
Wei Zhou
Fan Wu
☆
Fri Sep 4 · 4:50 PM - 5:00 PM
BioDMS · Invited Talk: Embeddable OLAP for Variant Data: Why Warehousing, Search, and Research can share an Engine
● BioDMS
Invited Talk
📍 Grand Ballroom D
Population genomic variant data has an awkward dual nature between a numeric matrix of common variants and a sparse relational table of rare ones.Population genomic variant data has an awkward dual nature between a numeric matrix of common variants and a sparse relational table of rare ones. Phoebe is a system built around the open-source DataFusion and Vortex projects supporting all three usage patterns - warehousing, interactive search, and interchange - without interchange overhead, capable of running multi-gene cohorting search queries against 3.1 million simulated realistic exomes on S3 in under two seconds.read more
Tim PoterbaSoftware Engineer and Entrepreneur, Phoebe Genomics
Built for you by(617)-PATRICK, Champion AI speaker
Let AI help you determine which sessions to attend 🤖
The steps below are for copying all the session descriptions and putting them into your own AI model - Claude, Gemini, ChatGPT, Co-Pilot and so on. This is the best way to do it on your laptop or PC. If you don't have your laptop with you, use the phone feature in red at the top of this page.
Tip: This is a large session list. Claude (claude.ai) handles the full file reliably - if you don't have a window open, start one there. Other tools like ChatGPT and Gemini may silently cut off the data - the verification step in the prompt will catch that.
Paste one of these two prompts. The short one is enough; the longer one gives the AI a role, context about the data, and a clear task.
Pick the 10 best sessions for me and say why.
ROLE: Expert conference advisor for VLDB 2026 attendees CONTEXT: I'm pasting over 200 sessions for the VLDB 2026.
I'm a database systems researcher and engineer. I care about AI-for-systems and LLM+database work, vector search, and cloud-native engines, and I want to meet people building production data infrastructure. GOAL: Pick the top 10 sessions for me. OUTPUT FORMAT: List in chronological order. Do NOT number them sequentially.
Instead, start each entry with its relevance rank in bold, like: Rank #3 of over 200 - Session Title.
Follow with the day and time, then 2-3 sentences on why it fits my goals. VERIFICATION: Before answering, confirm how many sessions you received.
You should have at least 200. If you received significantly fewer,
tell me immediately - my paste was probably truncated.
How cool am I? 😎
Which sessions are right for you?
Describe yourself and what you want from VLDB 2026. The more specific you are, the better this works.
You can also say how many you want or when you're free - “just my top 3”, “give me 20”, “only Friday afternoon”. You get 10 unless you ask for more.
There is a daily limit on the phone version
so it can stay free for everyone at the event.
However, you can run it as many times as you want using your own AI account at home:
go to the bottom of the page
tap Copy this longer prompt
paste it into Claude, ChatGPT, or Gemini.
Powered by Claude. Your text is sent to Anthropic to generate the ranking, and is logged on this site so the feature can be improved. Please don't type anything confidential.
