Research Area
You can't fix what you can't measure. Chunhua's correctness research builds the standardized benchmark suites, analysis tools, and evaluation frameworks that the HPC community needs to reproducibly assess and improve data race detection, auto-parallelization, and parallel program correctness.
Parallel programs are notoriously difficult to get right. Data races — where two threads access the same memory location concurrently and at least one is a write — are subtle, non-deterministic bugs that can corrupt results, cause crashes, or silently produce wrong answers in scientific simulations. Detecting and eliminating these bugs requires both powerful analysis tools and rigorous benchmarks that can evaluate how well those tools work. Chunhua's correctness research addresses both sides of this challenge.
DataRaceBench is a community benchmark suite specifically designed to evaluate data race detection tools. It contains hundreds of microbenchmarks covering a comprehensive range of race patterns in OpenMP programs — from simple shared-variable races to complex patterns involving reductions, atomic operations, and nested parallelism. By providing ground-truth annotations (each microbenchmark is labeled as having or not having a race), DataRaceBench enables apples-to-apples comparison of different detection tools and serves as a regression suite for tool developers. It has been adopted by research groups worldwide and cited in dozens of publications.
AutoParBench extends this philosophy to automatic parallelization: a benchmark framework for evaluating how well compilers and tools can automatically identify and parallelize sequential loops. More recently, Chunhua's group has explored whether LLMs can detect data races — a fascinating intersection of AI and program correctness. The SC-W 2023 paper found that current LLMs show surprising capability on simple race patterns but struggle with the complex, context-dependent races that occur in real HPC codes — highlighting both the promise and the current limits of AI-based program correctness analysis.
The first systematic evaluation of LLMs as data race detectors, using DataRaceBench as the evaluation harness. Finds that GPT-4 achieves reasonable accuracy on simple races but misses subtle ones, pointing toward hybrid approaches combining compiler analysis with LLM reasoning. Published at the SC23 Workshop on Correctness.
The foundational DataRaceBench paper introducing the benchmark design philosophy, microbenchmark taxonomy, and evaluation methodology. The suite has been continuously expanded and is now the standard evaluation benchmark for OpenMP race detection tools in the research community.
Introduces AutoParBench, a benchmark framework for evaluating automatic parallelization tools. Provides ground-truth parallelization annotations and a correctness-checking methodology to reliably compare different auto-parallelization compilers on realistic loop patterns.
A comprehensive benchmark suite for evaluating data race detection tools on OpenMP programs. Provides hundreds of microbenchmarks spanning a systematic taxonomy of race patterns: loop races, pointer aliasing, reduction races, atomic races, and complex inter-procedural patterns. Each benchmark carries ground-truth race annotations. Widely adopted in academia and industry for tool evaluation and development.
A unified benchmark framework for evaluating automatic parallelization compilers and tools. Contains loops from real scientific codes alongside synthetic patterns, with ground-truth parallelizability annotations and an automated correctness-checking infrastructure.
You can't fix what you can't measure. Standardized benchmarks don't just evaluate tools — they define what correctness means and create shared ground truth for a research community.