Open Postdoctoral position, faculty mentor Sanmi Koyejo

HAI Postdoctoral Fellowship with Professor Sanmi Koyejo and Professor Michael Bernstein, Department of Computer Science

Deployed AIs must navigate challenging tradeoffs. How much should a medical AI risk a false positive vs. a false negative? How much harm is allowable vs. the benefit provided by the AI? The research will develop algorithms, interfaces, and applications of metric elicitation from human feedback.
Metric elicitation is a framework designed to aid humans in identifying and balancing trade offs relevant to decision-making with AIs in settings where humans are the best judge of preferred outcomes, i.e., where a gold standard does not exist, and ideal outcomes depend on stakeholder preferences. Once identified, these preferences can be used for auditing decision systems, training automated decision-making models, and explaining decision-making processes. However, its success relies on developing effective interactive techniques for eliciting preferences and tradeoffs from people.
The postdoctoral scholar will work at the intersection of artificial intelligence and human-computer interaction to investigate some of the following questions:

• What are efficient algorithms to reduce the number (and increase the quality) of human queries?
• How do biases play into the elicitation procedure and outcomes?
• What new benchmarks and datasets might help measure progress in joint human-AI decision-making?
• What elicitation interfaces best capture peoples' intent in navigating these challenging tradeoffs?

The postdoctoral fellowship will be co-hosted by Professor Sanmi Koyejo (machine learning) and Professor Michael Bernstein (human-computer interaction). The PIs will support the postdoc fellow in the development of all the components of their IDP , career counseling, grant proposals, development of teaching and mentoring skills, and instruction in professional practices.

The success of the mentoring plan will be assessed by monitoring the postdoctoral researchers' personal progress by tracking their progress toward their career goals. An interview will be conducted towards the end of the postdoctoral program year to evaluate how well the program helped the researcher to achieve their career goals.

Stanford HAI is also committed to creating a diverse community of scholars who are engaged in contributing to the understanding and advancement of Human-Centered AI. Postdoctoral fellows will have the opportunity to engage with one another and with the broader Stanford HAI research community. They are also expected to participate in professional development, cohort-building, and other programmatic activities organized by HAI.
Evaluation Criteria

• Expertise or background in AI, algorithms, mechanism design, or related areas, as evidenced by degree, publications, research statement, or recommendation letters.
• Expertise or background in human-computer interaction, as evidenced by degree, publications, research statement, or recommendation letters.