Agent-based AI systems, commonly used in customer service bots and supply chain management, often require real-time processing, which can be costly. These systems sometimes introduce errors due to limited computational resources.
Researchers from MIT and the University of Washington propose a new method to enhance AI decision-making by setting a "budget" for computing resources. This approach, termed “latent inference budgets,” sets a cap on the computing power used for each task without compromising accuracy.
The concept allows AI agents to decide how deeply they should analyze a problem before generating a response. A lower budget prioritizes faster response times with potentially less accuracy, while a higher budget uses more computing power for a more accurate answer.
Businesses can set budgets based on task complexity. For instance, a customer support system could be allocated a lower budget for faster responses, while more complex tasks might have a higher budget.
The researchers applied this budgeting approach to tasks like maze navigation and chess. Systems with a set budget demonstrated improved decision-making and more accurate predictions compared to traditional models.
The paper highlighted the effectiveness of this approach across various domains, stating, “In maze navigation, pragmatic language understanding, and playing chess, [the method] outperformed classical models of bounded rationality while imputing meaningful measures of human skill and task difficulty.”
