Active Slice Discovery in Large Language Models
LLMs fail systematically on hidden subsets of data — a demographic, a phrasing, a topic. Finding those "error slices" usually takes heavy manual labeling. We show you can find them cheaply.
We formalize Active Slice Discovery: actively grouping errors likely to share a failure pattern, then using limited annotator access to confirm the slice. On toxicity classification, uncertainty-based active learning recovers human-defined slices while significantly outperforming baselines.