Researchers at the University of Tsukuba have developed a theoretical model to evaluate the impact of diversity in machine learning models and input data on the reliability of a machine learning system's output. The study focuses on N-version machine learning systems, where multiple machine learning models and input data are combined to mitigate inference errors affecting the final output. While the empirical understanding indicates that diversity plays a role in reliability, a theoretical model had not been established until now.
The researchers introduced diversity metrics for machine learning models and input data concerning inference errors and constructed a theoretical model to assess the reliability of the machine learning system's output. The study explored configurations that utilize the diversity of models and input data, finding that such methods were the most stable for improving reliability under assumed situations.
N-version machine learning systems are designed to enhance reliability and safety in applications like autonomous driving and diagnostic medical imaging. While the study provides theoretical insights into optimizing diversity for improved reliability, researchers acknowledge the need to address practical challenges, such as reducing the cost, power consumption, and overhead associated with multiple inference processes. Ongoing investigations will aim to develop methods that achieve high reliability in N-version machine learning systems while addressing these practical considerations. The findings are published in IEEE Transactions on Emerging Topics in Computing.
