Bloom by Safety Research

Bloom is an open-source tool for the automated evaluation of behaviors in Language Learning Models (LLMs). As a scaffolded evaluation system, Bloom uses an evaluation configuration, also known as a 'seed', which specifies a target behavior, exemplary transcripts, and the types of interactions the user wants to examine. The tool then generates a suite of interactions with the target model intending to expose the selected behavior. The evaluation suite's growth depends on how it is seeded, differing from other evaluations that might use a fixed elicitation technique and prompting pattern, which increases the uniqueness and adaptability of each evaluation run.To ensure reproducibility, Bloom evaluations should always be cited together with their complete seed configuration. The tool also allows users to add API keys from providers and lets them configure behavioral evaluative facets through behaviors.json and seed.yaml files. Additionally, Bloom also includes an interactive viewer that provides an intuitive interface to browse transcripts from the run, showcasing conversation flows with correct formatting.