The Role of Benevolence in Trust of Autonomous Systems
Without reliable and robust methods for assessing the trustworthiness of intelligent, autonomous systems, the issue of trust has become one of the most significant obstacles to broad use of autonomy technology by DoD and other agencies. However, the impact of the research described in this report supports creation of computational methods that create a bridge to future engineering of trustworthy autonomous systems.
The core objectives of this research topic were
- Operationalize the quality of “benevolence” and understand how it contributes to well-calibrated trust of, and reliance upon, autonomous systems
- Investigate portrayal of trust-related attributes in the human-machine interface.
Key accomplishments included:
- Formulation of benevolence as a complex “belief structure” with antecedent beliefs having important semantic, temporal, causal and other interrelationships;
- Mapping of a portion of trust-related belief structures to measurable internal states of autonomous systems, thereby potentially creating new opportunities for assessment of trustworthiness of such systems;
- Empirical evidence in support of the proposition that previous psychological concepts of interpersonal human trust are applicable to trust in autonomous systems, including the role of personality and situation in modulating the role and importance of certain beliefs;
- Creation of a theory of a “Human Social Interface” which, when expressed in systems engineering terms, provides guidance for machine portrayal of trust-related qualities in human-machine social interaction;
- Design and implementation of a software prototype based on the Human Social Interface theory that provides a basis for future experimentation and evaluation.
This project resulted in eight peer-reviewed publications and sixteen presentations in scientific venues, meetings with distinguished visitors, and other in support of technology transition opportunities within DoD and to industry. Many new research questions were generated and there remains considerable work to do to fully understand the role of benevolence with respect to intelligent autonomous systems. Overall, the theoretical foundation for trustworthiness of autonomous systems is immature and remains an important area of focus for multiple disciplines.
Atkinson, David J. The Role of Benevolence in Trust of Autonomous Systems. Final Report Contract Number FA9550-12-1-0097. Air Force Office of Scientific Research. DOI: 10.13140/RG.2.1.4710.5127 (2015)
Methodology for Study of Human-Robot Social Interaction in Dangerous Situations
(Paper)(Presentation)