Guide to Evaluation Perspectives on AI Safety; Guide to Red Teaming Methodology on AI Safety

Summary

Two companion guides from the Japan AI Safety Institute for AI developers and AI providers. The Guide to Evaluation Perspectives on AI Safety sets out concepts for evaluating the AI safety of LLM systems and AI agent systems: perspectives such as control of toxic output, fairness and inclusion, privacy protection, ensuring security, robustness and verifiability, with example risks and evaluation items, evaluators, timing and methods; version 1.20 adds 'observation and control' for AI agent systems. The Guide to Red Teaming Methodology on AI Safety covers evaluating the risk countermeasures of LLM systems from an attacker's perspective, from attack methods and roles to a 15-step process.

Language

English translation by Japan AI Safety Institute (J-AISI) Open it. The Japanese text is authoritative.

Safety topics

Status notes

Evaluation Perspectives Guide: first published 18 Sept 2024, version 1.10 on 28 Mar 2025, version 1.20 on 7 Jul 2026. Red Teaming Methodology Guide: first published 25 Sept 2024, version 1.10 on 31 Mar 2025. J-AISI has also issued domain-specific evaluation guides (healthcare, 2 Apr 2026; AI robotics, 23 Jul 2026) and annual AI safety reports; these are not recorded separately.

Sources

Open the official source https://aisi.go.jp/output/output_information/260707/

Secondary source: https://aisi.go.jp/output/output_information/250331_1/

Local copy

Verification

History

1. 2026-10-14related: [{"slug": "jp-aisi", "id": "JP-01"}, {"slug": "jp-ai-guidelines-for-business", " → [{"slug": "jp-aisi", "id": "JP-01"}, {"slug": "jp-ai-guidelines-for-business", " — two-way link to JP-09, JP-11, JP-12 added in run 2026-10-14-discover-leads-2026-10-142026-10-14-discover-leads-2026-10-14 · Claude Code (claude-opus-5-5)
2. 2026-10-14english_version: {"kind": "government translation", "url": "https://aisi.go.jp/output/output_info → {"kind": "government translation", "url": "https://aisi.go.jp/assets/pdf/ai_safe — Now points at J-AISI's English v1.10 of the evaluation guide, a stored source; it pointed at the red-teaming release page.2026-10-14-refresh-translation-fixes · Claude Code (claude-opus-5-5)
3. 2026-10-14summary: Two companion technical guides from J-AISI for AI developers and providers. One → Two companion guides from the Japan AI Safety Institute for AI developers and AI — Written from the Japanese texts of both guides: readers AI developers and AI providers (eval 1.3), scope LLM systems and AI agent systems (eval 2.4) and LLM systems (red-teaming 1.4); perspectives named as J-AISI's English v1.10 renders them (Control of Toxic Output, Fairness and Inclusion, ...); the red-teaming guide described from its section 1.1 (evaluating risk countermeasures from an attacker's perspective) and its contents (attack methods, roles, STEP 1 to STEP 15). 'Including LLM-based systems' from the earlier summary is gone.2026-10-14-refresh-translation-fixes · Claude Code (claude-opus-5-5)
4. 2026-10-14english_version: – → {"kind": "government translation", "url": "https://aisi.go.jp/output/output_info — English version found by the translation check in a stored copy (archive/JP-03/2026-10-11/secondary.html).manual-2026-10-14 · Claude Code (claude-opus-5-5)
5. 2026-10-14language: Japanese, English → Japanese — The translation check found the official text in this language only; the English texts are translations (recorded in english_version where stored).manual-2026-10-14 · Claude Code (claude-opus-5-5)

Related records

Japan AISI Japan · Published (non-binding, current)

AI Guidelines for Business Japan · Published (non-binding, current)

J-AISI Data Quality Management Guidebook Japan · Published (non-binding, current)

J-AISI Healthcare AI Safety Evaluation Guide Japan · Published (non-binding, current)

MIC Guidelines on Technical Measures for AI Security Japan · Published (non-binding, current)

Citation

Japan AI Safety Institute (J-AISI). Guide to Evaluation Perspectives on AI Safety; Guide to Red Teaming Methodology on AI Safety. 2024-09-18. https://aisi.go.jp/output/output_information/260707/. Status: Published (non-binding, current); verified 2026-10-14 (level A). Local copy archive/JP-03/2026-10-11/official.html retrieved 2026-10-11 14:00:52 UTC, SHA-256 f3e119316cce80e67cc83caec15ca8069a2aa07f94d5a439995e2ba6683b3e18. Every AI Policy Ever, version 1.18 (snapshot 2026-10-18), record JP-03.

Improve this record

Know of an amendment, a status change or an error? Report news or an error on GitHub; the form opens with this record filled in.