Conducting NIST audits using the NIST AI Risk Management Framework

Understanding the NIST AI Risk Management Framework

The NIST AI Risk Management Framework is a general framework meant to be adapted to an organization’s specific requirements. Since the field of artificial intelligence is rapidly changing, the framework is still evolving, especially with the recent Executive Order and announcements from the U.S. Government.

NIST AI Risk Management Framework functions

The NIST AI Risk Management Framework and the NIST AI RMF Playbook are based on four core functions:

• Govern: Establish governance structures and processes to build a culture of AI risk management across the organization.
• Map: Identify and assess the risks associated with AI systems and people involved in using AI within the organization.
• Measure: Assess, analyze, and track the exposure from identified AI risks.
• Manage: Implement and maintain risk management controls to mitigate identified risks.

The NIST AI Risk Management Framework provides a simple method for identifying and assessing AI risk management across an organization. NIST compliance with the AI RMF does not require strict adherence to a defined set of controls; instead, the broad nature of the framework allows organizations to modify the guidance to fit their needs. According to the framework, the goal is not NIST compliance but “to help manage the many risks of AI and to promote trustworthy and responsible development and use of AI systems.” AI systems built with these core functions in mind would likely comply with the growing body of AI regulations and thereby earn the trust of stakeholders.

How to audit using the NIST AI Risk Management Framework

While the NIST AI Risk Management Framework is not prescriptive like FedRAMP and StateRAMP, we can perform a NIST audit using the NIST AI RMF as a guide. For example, the audit department could start by identifying the population of AI systems within the organization. Next, the team assesses the current state of AI risk management through interviews with key personnel, reviews of documentation, and observations of AI system development practices. The NIST audit fieldwork should focus on the risk management practices to govern, map, manage, and measure AI-associated risks. Finally, the team can test the design and effectiveness of the controls in place that are identified through fieldwork.

The NIST AI Risk Management Framework contains pervasive references to required documentation. Requiring clear, comprehensive documentation over the development lifecycle for AI has been a point of contention for several years. AI algorithms and training programs notoriously lack documentation and are often called black boxes. Part of any AI NIST audit should focus on the need for documentation on all critical processes, calculations, and models coming from AI.

Prioritize your NIST audit now

The artificial intelligence technology explosion is underway, and the NIST AI Risk Management Framework is well-suited to guide audit teams to understand the risk and control environment. If you have not added an AI audit to your plan, consider doing so now. NIST provides many resources to help organizations perform AI RMF audits, including the AI RMF Playbook and the AI RMF Assessment Guide.