NIST workshop on AI Measurement and Evaluation

NIST will hold a virtual workshop on Artificial Intelligence Measurement and Evaluation June 15-17, 2021. The three-day workshop aims to bring together stakeholders and experts to identify the most pressing needs for AI measurement and evaluation and to advance the state of the art and practice.

NIST is assigned responsibility by statute to advance underlying research for measuring and assessing AI technologies. That includes the development of AI data standards and best practices, as well as AI evaluation and testing methodologies and standards. NIST is working collaboratively with the private and public sectors to help prioritize and work on its AI activities.  


Workshop Goals: 

- Identify: 

  • the needs for and intended uses of AI measurement and evaluation 
  • the gaps in knowledge/practice preventing current AI measurement and evaluation activities from effectively meeting these needs/uses 

- Solicit guidance on which specific areas that NIST should focus its efforts  

- Identify specific users and applications in need of measurement and evaluation 

- Collate best practices for AI measurement and evaluation 

- Build community around AI measurement and evaluation: provide tools, resources, pointers to other sources, and continued engagement through periodic talks and seminars. 

Participants are requested to provide feedback on the read-ahead materials (overview document and TESTAIS presentation video recording) in order to help shape the program. These materials will be available in early June. 


Workshop Website/Registration: 

For details about the event, including the agenda, and to register, see: 


Please direct questions to: aime-workshop@nist.gov  


Schedule (daily 11am - 5pm EDT): 

  • June 15: Needs for AI Measurements and Evaluations and NIST/Community Engagement 
  • June 16: Gaps: Evaluation Design/Methods/Metrics/Data 
  • June 17: Gaps: Practices/Infrastructure/Challenges
Partagez :