CosmicAI Researchers explored how to build agentic systems for data science tasks critical to scientific discovery
CosmicAI researchers William Rudman, Abhishek Divekar, Kanishk Jain, Sebastian Joseph, Stella S. R. Offner , Matthew Lease, Kyle Mahowald, Greg Durrett, and Junyi Jessy Li explored how to build agentic systems that can solve the data science tasks critical to many scientific discovery pipelines. We introduce VESTA (Visual Exploration with Statistical Tool Agents), which proposes statistical models that fit data by using and creating tools that let agents explore data and refine their predictions.
Addressing a Key Weakness of Large Language Models
LLMs are great at writing code and conducting experiments. However, there’s a weakness in their ability to propose statistical models and evaluate their fit.
Enter VESTA: Visual Exploration with Statistical Tool Agents.
Visual Tools for Statistical Hypothesis
VESTA explores data and refines statistical models by creating visual tools that target specific statistical hypotheses. These tool outputs then inform the next proposed model, producing a more accurate fit to the data.
The team introduces DAWN: Dataset for Automated Workflows and Numerical Modeling. DAWN covers two tasks – time series modeling and distribution fitting – each with three splits: Easy, Hard, and Astro.
Comparing tool strategies
The team tests three variants of VESTA: no tools, dynamically created tools, and expert-written tools. VESTA significantly outperforms baselines, with the biggest gains on Hard and Astro. Dynamically generated tools land just behind expert tools.
Why it matters
While state-of-the-art LLMs excel at writing code, we find that without data exploration and iterative refinement, current systems fail to produce statistical models that accurately fit a given dataset. VESTA addresses this by equipping vision-language models with a "visualize -> propose -> explore -> refine" pipeline backed by a dynamically growing toolkit of data transformations, hypothesis-driven visualizations, and statistical tests. Tools such as VESTA can help scientists better understand their data and ease the creation of high-quality statistical models.