Hugging Face Response to the U.S. NTIA Request for Comment on AI Accountability

TL;DR

On June 12, 2023, Hugging Face submitted a response to the U.S. NTIA’s request for comment on AI accountability, recommending that accountability mechanisms cover the full ML lifecycle, combine internal documentation with external transparency, and involve a broad range of stakeholders.

Context and Mission

Hugging Face frames its NTIA response within its mission to democratize good machine learning by emphasizing transparency and inclusion. As stated in the post, "Hugging Face’s mission is to "democratize good machine learning". We understand the term “democratization” in this context to mean making Machine Learning systems not just easier to develop and deploy, but also easier for its many stakeholders to understand, interrogate, and critique." The organization notes that it pursues this goal through education efforts, documentation focus, community guidelines, responsible openness, and no‑ and low‑code tools for analyzing datasets and models.

Core Recommendations

Hugging Face recommends three pillars for AI accountability: full lifecycle focus, internal‑external transparency, and broad stakeholder participation. The post lists the concrete recommendations:

• "Accountability mechanisms should focus on all stages of the ML development process. The societal impact of a full AI‑enabled system depends on choices made at every stage of the development in ways that are impossible to fully predict, and assessments that only focus on the deployment stage risk incentivizing surface‑level compliance that fails to address deeper issues until they have caused significant harm."
• "Accountability mechanisms should combine internal requirements with external access and transparency. Internal requirements such as good documentation practices shape more responsible development and provide clarity on the developers’ responsibility in enabling safer and more reliable technology. External access to the internal processes and development choices is still necessary to verify claims and documentation, and to empower the many stakeholders of the technology who reside outside of its development chain to meaningfully shape its evolution and promote their interest."
• "Accountability mechanisms should invite participation from the broadest possible set of contributors, including developers working directly on the technology, multidisciplinary research communities, advocacy organizations, policy makers, and journalists. Understanding the transformative impact of the rapid growth in adoption of ML technology is a task that is beyond the capacity of any single entity, and will require leveraging the full range of skills and expertise of our broad research community and of its direct users and affected populations."

Supporting Practices and Evidence

Hugging Face points to its existing tools and projects—documentation, education, no‑code analysis spaces, BigScience, and BigCode—as proof that its proposed mechanisms can work. The post states: "To that end, we have worked on fostering transparency and inclusion through our education efforts, focus on documentation, community guidelines and approach to responsible openness, as well as developing no- and low-code tools to allow people with all levels of technical background to analyze ML datasets and models." It adds that these approaches "have already proven their utility in promoting accountability, especially in the larger multidisciplinary research endeavors we’ve helped organize, including BigScience (see our blog series on the social stakes of the project), and the more recent BigCode project (whose governance is described in more details here)."

Implications for Policy

The response suggests that effective AI accountability policy should require documentation, enable external verification, and mandate inclusive governance processes. The post concludes: "We believe that prioritizing transparency in both the ML artifacts themselves and the outcomes of their assessment will be integral to meeting these goals." It also provides a link to the full PDF response for readers who want the complete detail.