Tertiary Study

A tertiary study is a research method at the apex of the evidence hierarchy, defined as a systematic review of secondary studies (systematic reviews). Originating from evidence-based medicine, it is now widely used in fields like software engineering and AI to synthesize the highest level of evidence on a topic. While not an ISO standard itself, it is a critical tool for achieving compliance. For instance, the EU AI Act and ISO/IEC 42001 (AI Management System) require organizations to manage AI system robustness, fairness, and safety. By conducting a tertiary study, a company can comprehensively integrate findings from all secondary studies on topics like 'AI model explainability techniques' or 'adversarial attack defense strategies,' providing robust evidence to support its technology choices and risk controls.

In enterprise risk management, a tertiary study offers a structured approach to grasp the full scope of a complex technical domain. Implementation steps include: 1) Define the critical risk topic, e.g., 'A systematic review of techniques to evaluate and mitigate hallucinations in Large Language Models.' 2) Conduct a systematic search and screening of academic databases like IEEE Xplore, applying strict inclusion/exclusion criteria to find relevant secondary studies. 3) Perform evidence synthesis and gap analysis to extract and analyze the effectiveness and limitations of various mitigation techniques. A fintech company could use this method to assess risks before deploying a generative AI chatbot, aiming to increase its compliance audit pass rate to over 95% and reduce customer complaints from AI errors by 20%.

Taiwan enterprises face three main challenges. First, a scarcity of interdisciplinary talent skilled in AI, risk management, and rigorous research methodologies. The solution is to partner with expert consultants or train an internal team. Second, high resource and time commitment, which can be prohibitive for SMEs. This can be mitigated by focusing the study on high-risk AI applications and using AI-assisted tools to accelerate the process. Third, difficulty in translating academic findings into practical business applications. To overcome this, a cross-functional team should be formed at the project's outset to ensure the study's outputs are directly convertible into actionable internal policies and technical guidelines.

Winners Consulting specializes in tertiary studies for Taiwan enterprises, delivering AI governance systems compliant with international standards like ISO/IEC 42001 and the NIST AI RMF within 90 days. Our proprietary Evidence-Based Risk Assessment (EBRA) methodology bridges rigorous research with practical application, empowering clients to navigate the complexities of AI risk. We have served over 100 leading Taiwanese companies. Request a free AI governance maturity assessment: https://winners.com.tw/contact