MIT News  July 31, 2024 Recent studies have found that common interventions such as instruction tuning often result in poorly calibrated large language models (LLMs). Although calibration is well-explored in traditional applications, calibrating LLMs is uniquely challenging. The challenges stem as much from the severe computational requirements of LLMs as from their versatility, which allows them to be applied to diverse tasks. To address these challenges, researchers at MIT proposed THERMOMETER, a calibration approach tailored to LLMs. For calibrating the LLMTHERMOMETER learned an auxiliary model, using the data given from multiple tasks. According to the researchers it was computationally efficient, […]