MALTA: IT audit of the online component of the 2021 Malta Census of Population and Housing

In December 2022, the Malta National Audit Office (NAO) published the report of its Information Technology (IT) audit of the online component of the 2021 Malta Census of Population and Housing. The 2021 Malta Census was the first census that could be completed through an online form. The audit assignment sought to analyse the planning and implementation phases of the Census, including the collection of data, data integrity and security, the technological challenges, and the costs of this project.

The NAO noted that the Malta National Statistics Office (NSO) carried out two local pilot studies, a thorough public consultation process and communicated very well with various stakeholders. The NSO also sought the help of Statistics Netherlands (CBS) and the Malta Information Technology Agency (MITA) during the development and implementation phases of this project.

The Malta Census 2021 data collection process consisted of four modes of data collection namely Computer Assisted Web Interviewing (CAWI), Computer Assisted Telephone Interviewing (CATI), Computer Assisted Personal Interviewing (CAPI) and Paper Assisted Personal Interviewing (PAPI) as outlined in the diagram below. All four modes used the same IT System but through two different interfaces. A web interface was used for the CAWI and PAPI data entry, whilst tablets were used for the CAPI and the CATI. Furthermore, a software application was developed to assist the daily operations of the call centre, monitor the cases that each enumerator had to deal with and manage the synchronisation of data.

Source: NSO

The NAO however noted that some documentation like signed off testing scripts, documentation showing how the technical specifications of tablets were determined, audit trails, backup logs etc. was not presented to the auditors.  The NAO observed that the tablets procured for this project were procured through an accelerated open procedure, which could have been avoided if the normal procurement process was started early enough. The NAO also observed that the selection process of Enumerators needed more attention and recommended that in future, the NSO considers seeking the help of other entities to assess the applicant’s IT skills.

During the course of this audit the NAO analysed the total costs incurred by other European countries that like Malta used the traditional data collection method for their 2011 census^[1]. The NAO noted that Malta’s cost per capita for the 2011 Census was of €3.54 (US$4.93 in 2011 US$) whilst the average cost per capita amongst other European Countries US$7.17. The NAO also analysed the Average cost per capita of 7 European countries that used the Combined data collection Method and the Average cost per capita of 7 European countries that used the Register Based Data Collection Method. Whilst the combined data collection method in 2011 had an average cost per capita of US$6.38 the register-based data collection method had an average cost per capita of US$0.47. The NAO also noted that the NSO had studied the option of the register-based data collection method however for this to be implemented the NSO needed the support of all the stakeholders involved in order to be able to collect the information needed for the Census from the administrative registers.

The NAO recommended that:

• the NSO identifies and documents anticipated risks vis-à-vis future census projects, assigning a probability assessment for each risk, identifying possible impact and ways to manage these risks;
• future plans for the procurement of one-off use hardware such as the tablets used in the 2021 Census, should include concrete plans for the continued use of such hardware once the survey is completed;
• the NSO analyses the demographics of online respondents by age, town/village, in order to better understand the challenges of these respondents, and devise plans how to combat these challenges in order to improve the percentage of online submissions further;
• the NAO recommended that the NSO conducts a comparative analysis of actual costs of the 2021 (once these become available) and 2011 Census surveys;
• the NSO considers the use of administrative registers for Census data collection once it is ensured that the format of data in administrative registers matches that required for the Census;
• an inter-Ministerial committee is set up to oversee the move towards implementing a register-based census and a population register.

The NAO considered the 56% online submission rate as one of this project’s success factors especially when considering the online submission rates achieved by other European countries who had just launched this functionality for the first time.

This report may be accessed through the National Audit Office website www.nao.gov.mt or Facebook page www.facebook.com/NAOMalta

^[1] Costs per capita incurred by other European Countries for the 2021 Census were not available at the time of this study.