HUNGARY: Using AI to Analyse Disruptions in the Healthcare System
Background
In 2025, the State Audit Office of Hungary (SAO) transitioned to a competency-based operational model. Each competency area required monitoring tools capable of highlighting not only the sector-wide big picture but also competency-specific risks to support data-driven audit activities.
The healthcare competency area considers analysing operational disruptions in the healthcare system to be a good proxy for an output-based approach to the operation and quality of the healthcare system. In Hungary, the competent authority issues decisions regarding operational disruptions, documenting key details such as the suspended organisational unit, the affected profession, capacity, the reason for the suspension, its duration, and the method of substitution.
The main obstacle to analysis to date has been the excessive manual effort required due to issues with the decisions. Even the competent authority lacks a standardised, normalised database. In practice, the decisions are only available as PDF image files, lacking a uniform format and structure. Key details can often only be inferred implicitly, requiring thorough review of the documents and understanding of related records and legislation.
Given the daily growth of the list and the thousands of documents involved, it could not be utilised as part of a monitoring system prior to the introduction of AI. The use of AI was first initiated by a Hungarian investigative journalism portal in this field; however, it lacked the depth required for a well-founded sectoral analysis, and the high degree of manual labour involved made integration into a monitoring system impossible.
Processing
To develop the analysable database, the SAO used its own in-house AI environment, with this project being one of its use cases. The implementation steps combined traditional pre-AI technologies with the use of AI.
1 Document processing
The fully automated processing pipeline was developed in Python, with AI calls made via the API. Downloading documents, converting them to text was done in code, and batch processing of the text documents was performed via AI API calls, with JSON output to facilitate processing. Finally, placing the results into a standardised, normalised database and adding geospatial elements were also performed via code. Furthermore, a significant portion of the necessary code was also generated by AI.
The quality assurance process involved the manual evaluation of a 50-item sample, which, based on experience, typically yielded an accuracy rate of over 98%. We found a somewhat lower accuracy rate in the very long lists regarding the affected municipalities, but the newer SAO AI model improved this rate as well.
Finally, OLAP views and reports were generated from the resulting database for analysts. These can be used to analyse trends and provide support when preparing for audits, consultations, and analyses.
Lessons learned
Based on the project’s experience, it is possible to process general types of documents to create an analysable database in a bulk and in an automated manner, even in cases where this was previously not feasible due to cost-benefit analysis.
