Handle404 Quality Audit
Handle404 Quality Audit
Good Practices in
Managing Quality within SAIs
FOREWORD
About EUROSAI
EUROSAI (European Organisation of Supreme Audit Institutions) is one of the Regional Working Groups of
the International Organisation of Supreme Audit Institutions (INTOSAI). EUROSAI was established in 1990
with 30 members. Membership currently stands at 50 SAIs (the SAIs of 49 states and the European Court of
Auditors).
From its very beginning, EUROSAI has been active in organising fruitful and mutually beneficial cooperation in
the field of public audit. The objectives of the EUROSAI, as defined in Article 1 of its statutes, include to:
This document has been prepared by a EUROSAI working group which was given a mandate by the VII
EUROSAI Congress to look at ways of helping Supreme Audit Institutions to be more effective in achieving
high quality audits through the selection and communication of proven good practices. The document is
intended only as a guide for those running SAIs and not as being binding.
The working group was established in 2008 and is led by the State Audit Office of Hungary. Its members
include experts from the Supreme Audit Institutions of Denmark, Malta, Poland and the Russian Federation, as
well as from the European Court of Auditors.
The group would like to thank the following organisations for their support: the EUROSAI Secretariat; all
EUROSAI member SAIs; as well as the SAIs of Australia, Canada, India, New Zealand and USA.
This document is also available on the internet at http://www.eurosai.org. In addition, the working group
intends to establish a good practices database to ensure that SAI experts concerned with quality management
can have on-line access to related materials. It will contain good practices of SAIs submitted in a uniform
format and organized in line with the structure of this document. The electronic database of good practices will
be updated and amended on a regular basis.
3
Table of Contents
FOREWORD 3
Table of Contents 5
INTRODUCTION 7
I. GOVERNANCE 10
I.1 Risk Management System 10
I.2 Performance Indicators 12
I.3 Self-Assessment of the Organisation 14
I.4 Peer Review 15
IV. COMMUNICATION 30
IV.1 Internal Communication and Dialogue 30
IV.2 External Communication and Relationship with Stakeholders 31
CONCLUDING RECOMMENDATIONS 33
5
INTRODUCTION
The VII EUROSAI Congress held in Krakow between 2 and 5 June 2008 discussed the theme ‘Establishing an
Audit Quality Management System within a Supreme Audit Institution’ and in its Conclusions and
Recommendations supported the development of a good practices guide on audit quality. A working group
was established to produce the guide.
Although several of the good practices presented in this document relate to the principles set out in these
standards, the aim of this guide is inherently different. These good practices are complementary to
requirements of the standards and are aimed at providing practical proven ways of achieving quality. For ease
of reference the relation between the topics covered by the guide and the relevant element of ISSAI 40/ISQC 1
is set out in Annex I.
Basic Principles
It is vital that a Supreme Audit Institution (SAI) operates at high quality. In some ways the arguments for
achieving excellence are more compelling for SAIs than for other institutions because of the nature of their
work: judging the actions of others. The reputation of SAIs is based on the quality of their output. SAIs can
only achieve respect and authority if they can demonstrate that it itself is managed to high standards. This
means that SAIs should:
▪ adhere to professional standards of approach and evidence;
▪ achieve their objectives in the most efficient and effective way; and
▪ be - and be seen as - a well run organisation, operating to the highest administrative and financial
management standards.
Quality is rarely achieved spontaneously but needs to be managed into the organisation and should be based
on continuous improvement. Specific procedures should be applied at all levels using a quality management
system based on appropriate objectives, principles and strategy. The ultimate responsibility for establishing
and ensuring the running of the quality management system within an organisation lies with its leadership, and
should be one of their key priorities. A quality management system is most effective when it covers all aspects
1 Principal Paper, Country Papers, Discussion Paper, Congress presentations and discussions
2 The Lima Declaration of Guidelines on Auditing Precepts (ISSAI1), Code of Ethics (ISSAI30), INTOSAI Auditing Standards
(ISSAI 100, 200, 300, 400), Quality Control for Audits of Historical Financial Information (ISSAI 1220), [Draft] Quality Control for
SAIs (ISSAI 40)
3 International Standard on Quality Control (ISQC) 1, Quality Control for an Audit of Financial Statements (ISA 220)
4 The document was approved by the Contact Committee of Heads of the Supreme Audit Institutions of the Member States of the
European Union at its meeting in 2004 in Luxembourg
7
of SAIs’ activities, and integrates the various sub-systems through the application of common principles and
standards. Establishing an effective quality management system is an evolutionary process with SAIs currently
at different stages of development. Some SAIs may be at an initial stage with quality processes being
unstructured and undocumented. Others may be more advanced with quality processes in regular operation
as well as being regularly monitored, measured and continuously improved.
Quality is needed in both the professional work of the SAI, and its administration (giving it the authority to lead
by example). In order to be effective the following conditions are necessary:
▪ leadership sets strategy, acknowledges and communicates to all staff the importance of meeting
ethical standards and quality, sets the objectives of the quality management system and defines
roles and responsibilities;
▪ risks to meeting objectives are identified and managed.
▪ the organisation adopts the international standards on quality control, and establishes the
appropriate systems and practices to comply with them;
▪ formal rules and requirements (including review) are established within the organisation to help
ensure quality is achieved in professional and administrative (including financial management)
processes, as well as providing a standard against which the quality of implementation can be
judged;
▪ staff are recruited and trained to ensure they have adequate knowledge of professional standards
and adhere to ethical and legal requirements;
▪ sufficient financial resources are provided5;
▪ sufficient investment in information technology and communication is made to support the SAI;
▪ the operation of quality control procedures is documented, to ensure a clear record and trail;
▪ the implementation of the quality management system is regularly reviewed and evaluated both
by management through an effective quality assurance (monitoring and inspection) function, and
by external experts to provide independent assurance on its operation.
Furthermore, it is of interest for SAIs to consider obtaining an independent recognition of their quality, such as
an accredited quality standard.
One of the main strategic goals of SAIs is to contribute effectively to the transparency and accountability of the
management of public funds. This is achieved by carrying out high quality audits resulting in clear, reliable and
useful reports.
▪ Clarity of audit reports is ensured by clear and accurate drafting; setting out the audit objectives
and criteria; clearly describing the findings, conclusions and recommendations; and presenting
easily distinguishable main messages.
▪ Reliability of audit reports is ensured by complying with professional standards including
independence and objectivity; as well as providing findings and conclusions based on sufficient,
relevant and reliable audit evidence.
▪ Utility of audit reports is ensured by covering topics of relevance to stakeholders, presenting up-
to-date findings; timing audits to contribute to upcoming changes in the legislation or budget
execution; and recommending cost-effective remedial action.
The good practice guide covers 14 separate topics based on submissions received from EUROSAI members
and selected by the working group as likely to make a useful contribution to challenges currently facing SAIs.
5
Although this will not generally be within the direct control of the SAI.
8
They are presented under the following headings:
▪ GOVERNANCE – how the organisation and its work is organised and managed.
▪ AUDIT MATTERS – how the organisation undertakes its audit work.
▪ HUMAN RESOURCES – how the organisation manages its main resource.
▪ COMMUNICATION – how the organisation establishes and manages internal and external
communication.
Each topic is presented using the following format to facilitate reading and comprehension:
▪ CHALLENGE – description of the issue addressed;
▪ RESPONSE – description of the way(s) the challenge can be addressed;
▪ GOOD PRACTICES – proven ways the response can be implemented effectively.
The good practices guide is aimed at senior management of Supreme Audit Institutions. Its use is not
compulsory and it does not intend to make a complete or detailed presentation of all good practices but rather
an overview of specific approaches which may be useful or of interest.
9
I. GOVERNANCE
I.1 Risk Management System
CHALLENGE
Like any other organisation, SAIs face a number of risks in fulfilling their mandate, such as:
▪ failure to achieve their strategic goals (strategic risks);
▪ inadequacies or deficiencies in the management of internal processes and resources, as well as
risks arising from external events that could negatively impact on their operations (operational
risks);
▪ failure to fulfil judicial responsibilities or other legal requirements (legal risks)
▪ failure to maintain effective financial management and accountability arrangements (financial
risks); and
▪ risks that could impact negatively on the credibility and reputation of the organisation
(reputational risks).
RESPONSE
A risk management system should be established as a strategic and operational management tool
in order to identify, measure, monitor and control the key risks that the organisation faces in
pursuing its mission and objectives. The system should cover all risks, from high-level corporate
issues down to the risks related to individual audit tasks.
The SAI can determine the levels of risk exposure it is willing to tolerate for different areas, as well
as establish appropriate controls to manage risk to the required level. The tolerance levels may vary
between different risks and circumstances. Whenever there are changes to the identified risks, or
when controls are found to be inadequate, the risk management system should be adapted
accordingly.
GOOD
PRACTICES
1. It is good practice to embed risk management into the operation and culture of SAIs, and to
assign clear responsibilities for and within the risk management system.
2. SAIs can develop a risk management policy. The policy should identify the different types of risks
that the organisation faces, what can be done to mitigate these risks, and how the responsibility for
risk management is to be allocated. It is recommended that the policy is communicated to
stakeholders.
3. An internal risk management committee can be established by SAIs to facilitate and oversee the
introduction, implementation and monitoring of the risk management process. The committee
should contribute to major decisions affecting the organisation’s risk profile and exposure, as well as
include senior managers representing the different functions of the organisation. In order to ensure
consistency and continuity it is good practice to minimise frequent changes to the composition of the
committee. Furthermore, the risk management committee should be independent from other
organisational units and sufficiently empowered to exercise its functions effectively.
10
4. A risk register can be created to document and keep track of high priority risks. It can contain:
▪ a description of the nature of each identified risk;
▪ details of the risk monitoring system, such as information on the early warning mechanism in
place to raise the alert that a risk is increasing, as well as details on how improvements are to
be reported;
▪ a risk assessment rating of the possible impact of an event, should it actually occur, as well as
the likelihood of its occurrence with existing controls;
▪ an overall assessment of residual risk based on the combination of likelihood and impact;
▪ a list of the agreed controls or appropriate responses established to manage the risk;
▪ identification of the risk owner who is given the responsibility for assessing and managing
specific risks.
5. It is also good practice to review periodically the effectiveness of the risk management
arrangements, as well as identifying and assessing new or additional risks that the SAI faces.
Externally facilitated workshops can be held as necessary to support the review. Planned actions
resulting from this process can be incorporated into the SAI’s standard business planning cycle.
11
I. GOVERNANCE
I.2 Performance Indicators
CHALLENGE
SAIs need to measure achievement of key strategic objectives in order to track performance,
identify problems or weaknesses and, where necessary, propose corrective action.
RESPONSE
Relevant, practical and reliable performance indicators aim to provide SAIs with a timely and
balanced view of the organisation’s performance in undertaking audit tasks and running
administrative processes. The number and type of indicators required depends on the complexity
of the result being measured, the level of resources available for monitoring performance and the
amount of information required. Performance indicators can relate to inputs, processes, outputs
and impact and can be either quantitative (numerical) or qualitative (descriptive observations or
opinions).
GOOD
PRACTICES
1. Selecting appropriate and relevant indicators requires careful preparation, iterative refinement
and collaboration involving all levels of the organisation. It is good practice to link the
development of performance indicators to the objectives and/or targets of the organisation's
strategic planning process. The indicators should be clearly defined and cover the critical aspects
of the SAI activities. Established models (such as the Balanced Scorecard) can be used to guide
SAIs to develop an appropriate framework. It is important that such models are adapted to the
specific mandates, objectives and structures of the SAI.
Development of specific indicators depends, to a considerable extent, on the ability of the SAI’s
information system to provide reliable, complete and accurate information at a reasonable cost.
Changes may need to be made to SAIs’ information system in order to better support the
collection of data for the selected indicators.
2. Performance indicators should be accompanied by a definition of the expected results and the
respective strategic objectives to be measured. Indicators should also outline details on the
methods to be used to collect information, on who will be responsible for collecting the
information for each specific area, as well as on how and when the indicators will be reported and
to whom.
3. Performance indicators work best when they address single issues, thereby ensuring clarity of
what is being measured. This simplifies the collection of information for each indicator and
facilitates the allocation of responsibilities.
When developing performance indicators, an SAI should select a range of indicators which
provide a balanced assessment of the overall performance of the organisation. Also, when
introducing performance indicators, attention should be given towards avoiding the development
of perverse incentives.
12
Indicators should focus on the achievement of objectives and/or targets as well as on the relevant
aspects of performance such as inputs, processes, outputs and impact. The following are examples
of different types of indicators that can be used by SAIs to monitor and measure progress in
achieving objectives. These can be prepared for individual audits or classes of similar audits.
▪ Input and process indicators:
– Time-related measures, such as the average time spent to complete specific audits or types
of audits, the proportion of audits completed within the planned timeframe, and timeliness of
internal decisions;
– Cost factors, such as the cost of individual audit tasks, and/or the average cost of each type
of audit; and
– Human resource issues, including staff turnover rate, time allocated to training and
employee satisfaction levels.
▪ Output indicators:
– Quantity, such as the percentage of audited expenditure on executing each state function or
the number of audit reports published in a year; and
– Quality, for example the results of internal and external quality assessments of audit work
and published reports, as well as post-audit review.
▪ Impact indicators:
– views of stakeholders on the contribution and value-added of audit work;
– percentage of audit recommendations that have been implemented within specified time
periods;
– number of times SAIs are featured in the media and the type of coverage ;
– number of audit reports that were discussed in Parliament in a given year; and
– improvements and/or monetary savings arising from audits;
– level of auditees’ satisfaction with the quality of audits.
4. A performance indicator is a measure of the level of achievement of an objective against targets.
The results of a performance indicator need to be analysed in order to determine if any remedial
action is needed, including a revision of objectives and/or targets. It is good practice to have
general agreement within the organisation over the interpretation of results. SAIs need to take
into consideration that the achievement of some indicators (such as the duration of an audit) may
depend on circumstances entirely or partly beyond their control. For example, auditees can
exceed the deadlines set by SAIs for their written comments on management letters and reports.
5. It is good practice that SAIs report to stakeholders on the progress made in the achievement of
their objectives.
▪ Internally: progress on the achievement of key indicators should be reported to the appropriate
levels within the organisation using standard forms, graphs, scorecards and other visual
techniques. The reports should be timely and compiled at regular intervals depending on the
requirements of the organisation and the need to take corrective action.
▪ Externally: it is good practice to report progress on key indicators in annual reports or in
communications to the principal stakeholders (such as Parliament). The information can also be
communicated in other publications and placed on the SAI’s website. External communication of
performance indicators enhances transparency and accountability.
6. The development of performance indicators is a continuous process. The measures should be
periodically reviewed and adjusted to reflect new requirements or developments in the audit
field or to the SAI mandate.
13
I. GOVERNANCE
I.3 Self-Assessment of the Organisation
CHALLENGE
SAIs are required to judge the management of other institutions through their audits, yet rarely
come under close scrutiny themselves. This creates a risk that the SAI is not as effective or efficient
as it should be, which in turn risks undermining its credibility in the eyes of its stakeholders.
RESPONSE
The SAI can undertake a self-assessment. This involves a structured analysis of an organisation’s
strengths and weaknesses by its staff and management, which allows areas for improvement to be
identified, as well as recommendations for doing so. Self-assessment can take many different forms
depending on the purpose or context. It can either be wide-ranging, or focused on specific issues or
aspects of the organisation.
Any organisation’s human resources are a key source of informed insights to its operations. In SAIs,
this resource is particularly strong as many staff members will be trained auditors.
GOOD
PRACTICES
1. It is good practice to use an established approach such as the Common Assessment Framework
(CAF6) based on the EFQM Excellence Model. Examples of assessment topics under this model
include: leadership; strategy and planning; people; partnership and resources; processes; auditees
and citizens/customer-oriented result; society results; and key performance results.
2. The objectives of the self-assessment should be clearly defined and can be communicated to
the participants, together with the criteria to be applied. This allows the key points to be
addressed and the results presented in a systematic and balanced way.
3. The self-assessment can be undertaken by relatively small teams representing the different
levels of staff and management. This allows the evaluative issues to be discussed in detail and
a balanced consensus formed, thereby increasing the robustness of the results. Parallel
assessment carried out by a number of teams working on the same issues helps reduce the risk
of bias in the process. Including staff of all levels in the self-assessment exercise not only makes
use of their skills and experience but also sends a visible message about their importance to the
organisation and provides a sense of empowerment. The resulting visibility can help underline the
legitimacy of the process and encourage the acceptance of recommendations.
4. The main strength of self-assessments is in the identification of weaknesses and the definition of
the recommendations to correct them. If the recommendations are not implemented the process
will ultimately be ineffective. In order to facilitate the implementation of the recommendations, an
action plan can be established.
5. It is good practice to perform a self-assessment before embarking on a peer review
(see topic I.4). This gives the organisation the opportunity to identify areas for improvement and
allow changes to be made before the peer review takes place. When undertaken together in this
way, the two review processes make a more effective contribution to the promotion of
improvement.
6
Developed by the European Institute of Public Administration (EIPA).
14
I. GOVERNANCE
I.4 Peer Review
CHALLENGE
SAIs perform a difficult high profile task within a fast moving professional environment but in the
absence of competition. While they all carry out comparable work very few have a comparable
institution in their home country against which to benchmark their activities.
RESPONSE
A peer review is a process of subjecting the organisation and methods of SAIs to the scrutiny of
recognised experts from other SAI(s). It provides assurance to the outside world on the high
standards met by the SAI and identifies where improvements can be made to procedures and
output, thereby contributing to the overall effectiveness of the organisation. The SAI may decide
to limit the peer review to specific aspects of management or activities.
GOOD
PRACTICES
1. It is good practice to carry out a self-assessment (see topic I.3) before embarking on a peer review
in order to identify weaknesses and how improvements can be made. The peer review can then
take place once the recommendations of the self-assessment have started to be implemented. This
gives the possibility for the peer review to assess the adequacy of the measures taken.
2. The objectives and scope (terms of reference) of the peer review should be clearly defined and
documented before the decision to carry it out is taken. Peer review objectives may be
comprehensive, for example compliance of SAIs’ audit activity with professional standards, or
limited to specific types of audit (performance, compliance or financial) or area of activity. They may
also cover cross-sectional issues such as the system of quality control applied to audit work.
3. The process is likely to be effective if the selected peers are well respected, have the necessary
skills and experience, as well as sufficient resources for conducting the review. In peer reviews
performed on court-type SAIs, teams should include peers from similar organisations.
4. The peer review is normally covered by a written agreement, typically including: objectives and
scope, timetable, staffing, procedural matters, reporting issues, cost and practical support.
5. When embarking on a peer review, the selected reviewing team needs to be adequately prepared
for the task. They should be provided with full information on the applicable legal principles,
organisation charts, glossary of the terms and concepts used and the major procedures necessary
for an effective review. Members of the reviewing team should either be familiar with the working
language of the reviewee organisation or be provided with sufficient linguistic support.
6. The reviewee SAI should establish an internal support team to assist the peer team in its work,
including explaining all aspects, structure, scope, approach and methods of the organisation.
7. For the process to be effective it is necessary to analyse the peer review findings, conclusions
and recommendations in the light of the objectives that were initially set, as well as other issues
that may have emerged during the process. Internal discussion within the reviewee SAI on the peer
review findings can help to establish the best way to follow up on recommendations and prepare
an action plan.
15
8. The staff of the SAI can be informed about the peer review and its progress throughout the
process.
9. Peer review reports as well as the action plan for improvement can be disseminated to
Parliament, the media, and/or made public through the organisation’s website to promote
accountability and transparency.
10. A subsequent peer review with similar scope can be undertaken after an appropriate interval
(such as three years) to ensure that the identified weaknesses have been addressed completely
and effectively.
16
II. AUDIT MATTERS
II.1 Selection of Audit Tasks
CHALLENGE
SAIs undertake both obligatory (following legal requirements) and discretionary (left to the choice
of the organisation) audit tasks. The challenge is to carry out the obligatory tasks as efficiently
and effectively as possible in order to maximise the resources available for undertaking the
discretionary tasks. The latter should be selected in a way which address important issues and
thereby optimises the impact of the resources available.
RESPONSE
SAIs should establish a sound audit activity planning process (both long- and medium-term),
taking into account legal obligations as well as considerations of risk, materiality and the time
since the last audit. In this process particular attention should be given to the selection of
discretionary audit tasks, which are highly relevant to stakeholders and have good potential for
impact. Properly designed and coordinated planning process should ensure the most effective
use of SAI’s resources.
GOOD
PRACTICES
1. Carrying out risk assessments within the specific audit fields allows the identification of high risk
areas and therefore audit tasks which are likely to have the highest impact. A standardised risk
assessment approach can be used to rank and prioritise potential audit tasks.
2. Lessons learned from previous audits as well as action taken on recommendations can be highly
valuable when selecting and planning future audits. SAIs can use monitoring system (see topic
II.4) to collect this information in a structured and regular manner. In addition, communication with
the auditee (see topic IV.2) during this process can also increase efficiency and help to improve the
selection of audit tasks.
3. SAIs can also carry out regular analysis of macroeconomic issues and trends and prepare
preliminary studies on the more relevant topics. These studies can be used by SAIs to identify
those audit tasks that are more relevant and potentially of a higher impact. In these analyses results
of audit activities of other SAIs could be taken into account.
4. It is good practice for SAIs to monitor public interest and stakeholder expectations and use
these insights to contribute to the audit selection process. Issues that can be collected through
monitoring can include those of current interest to parliament and government, those resulting from
media monitoring as well as those raised by the general public, including complaint letters sent to
SAIs.
5. The planning of audit tasks requires cooperation between the audit departments of SAIs. A
planning unit responsible for coordinating activities can ensure efficiency and effectiveness in
planning throughout the organisation and minimise any overlap in the selection of audit tasks.
6. It is good practice to prepare planning guidelines or principles to serve as the basis for
establishing relevant selection criteria for use when planning audits. In this document, appropriate
weightings can be given to different overall issues or concerns according to priority, such as
materiality, auditability (feasibility), risks, timeliness, potential impact, overall balance of different
topics in the overall plan, and added value. The resulting audit plan should contain a list of audits
to be carried out, an indicative timetable, monitoring indicators, allocated responsibilities for each
audit, and resource requirements.
17
II. AUDIT MATTERS
II.2 Supporting the Audit Process
CHALLENGE
Auditors work in a complex environment and are required to face different and varied professional
situations depending on the type of tasks and nature of the audit target. SAIs need to identify the
type and level of support required for each audit task and define ways of how and when to
provide these tools, resources and technical support.
RESPONSE
In order for an audit team to have all the expertise, skills and resources necessary to carry out an
audit to the highest standard, the SAI should identify before each audit task the level of professional
and technical support required. Moreover, the SAI should determine whether the required support
will be provided in-house or outsourced. It is also essential to achieve and maintain high quality
throughout the audit cycle by ensuring that auditors follow professional standards and appropriate
guidelines, procedures and methods.
GOOD
PRACTICES
1. IT-based systems can help to structure the documentation and analysis of audit evidence, facilitate
the documenting of the audit work including the scanning and recording of audit evidence. These
systems can guide the auditor throughout the audit process, providing access to support (guidelines,
standards etc.) at all stages.
2. It is important that the audit evidence is obtained using reliable tests and methods. A good practice is
to establish/develop IT-platformed libraries of standardised and suggested tests, designed to ensure
compliance with audit methodology and standards as well as improving efficiency. Teams can use
such systems to access relevant material to support the audit process, including standardised forms
for recording judgments and conclusions.
3. It is good practice to provide auditors with access to experts such as a legal advisers, accountants,
actuaries, methodologists, statisticians, data analysts, and economists. In addition, working co-
operation with auditors from other SAIs could be useful.
4. Experts can advise the audit team when developing audit methods, criteria and approaches to help
ensure audit design is legally, technically, methodologically, and analytically sound. Furthermore, they
can provide specialised expertise for implementing individual audits when needed. In addition,
drafting experts can assist in reporting the audit results.
5. A strategy can be established for determining when and how to use different types of expertise.
Depending on the SAI, the required expertise can be either sourced in-house or outsourced (in
which case the SAI continues to retain responsibility for the quality of the work produced). SAIs can
also build a register of external experts recording the different areas in which they have been used
successfully in the past. This can help the audit team to quickly identify reliable experts who are
appropriate for the provision of specific expertise.
6. It is good practice to offer regular training (see topic III.2) and updates to all staff to ensure that
auditors have the required knowledge and skills to carry out planned audit tasks and identify where
experts could or should be used.
7. In order to ascertain that the audit team follows professional standards and guidelines throughout
the audit process, it is good practice to have available standards, guidelines, checklists and other
methodological support documents in the respective national language. Moreover, SAIs
can compile a glossary containing all relevant professional terms to further guide audit staff.
18
II. AUDIT MATTERS
II.3 Cooperation with the Auditee during the Audit Process
CHALLENGE
In order to be objective an auditor needs to remain independent of the auditee. Nonetheless,
achieving efficient and successful completion of an audit requires the cooperation of the auditee,
in particular to facilitate access to required data and information. In addition, such cooperation
can increase the likelihood that the auditee acts upon the audit findings, conclusions and
recommendations.
RESPONSE
The SAI needs to identify how to cooperate effectively with the auditee to facilitate the different
stages of the audit process and to increase the likelihood that audit findings, conclusions and
recommendations are followed up. In addition, continuous and close coordination through
ongoing dialogue with the auditee can help minimise interference with the day-to-day work of the
auditee.
GOOD
PRACTICES
1. In order to increase auditee’s acceptance of the audit, it is good practice to present the audit
subject, method and criteria to the auditee at an opening meeting. By doing so, the auditee can
facilitate the audit team’s efforts to identify ways of how to obtain the relevant audit evidence.
2. SAIs can also give the auditee the possibility to respond to the findings, conclusions and
recommendations of the audit during the process and prior to finalising the audit report so as to
avoid errors of fact and misinterpretations. This is sometimes referred to as the contradictory
procedure. For some SAIs the views of the auditees are required to be reflected in the report or
provided as a formal reply. An appropriate deadline should be set for the auditee to provide
comments on the audit report, taking into account the opportunity already given to the auditee to
comment.
3. As auditees are important stakeholders, follow-up action can be taken by SAIs to collect
feedback from them on the quality of the audit report and the audit process. A questionnaire
can be circulated to the auditee after the finalisation of the audit report. Constructive feedback
(see topic IV.2) can be used as input when assessing the quality of the audit and identifying
improvements that could be made. However, the auditor should be aware that the auditee might
object to the fact that weaknesses or errors were identified by the audit, despite these being
justified.
19
II. AUDIT MATTERS
II.4 Monitoring Audit Impact
CHALLENGE
It is important for SAIs to be informed on how their work has contributed to achieving good
governance and efficiency in auditees and the extent to which it adds value for stakeholders.
SAIs need to systematically keep track of how their findings are being used and their
recommendations implemented.
RESPONSE
SAIs can establish a comprehensive system for monitoring the implementation of audit
recommendations and assessing their impact using reliable sources of information. The results
can be presented to top management on a regular basis, as well as periodically to Parliament and
the general public. Monitoring of audit impact can be done through appropriate indicators (see
topic I.2).
GOOD
PRACTICES
1. It is good practice for SAIs to carry out periodic reviews of the degree of implementation of
audit recommendations or judicial activities. SAIs can also include the information gathered
from these reviews in a database of audit recommendations and their implementation for
access by auditors. The database should be updated regularly. This can be of use in audit
planning and formulation of future recommendations. The database may include various details
such as: whether or not the auditees accepted the recommendations and the feedback received;
which areas of activity were affected by the audit and how; indications about the level of difficulty
that auditees encountered in adopting the recommendations; the level of Parliamentary support;
and the period of time required for the respective recommendations to be taken up.
Relevant extracts of the database can also be made available to managers of the auditees or
parliamentary committees.
2. The practice of collecting feedback from auditees about the implementation of audit
recommendations can take a variety of forms, such as:
▪ written official responses from auditees on audit reports and opinions, with information on
whether, and how, they plan to adopt the audit recommendations;
▪ questionnaires (see topic IV.2) to auditees asking for their reaction to various aspects of the
audit findings, conclusions and recommendations;
▪ interviews with the employees of the auditee; and
▪ meetings with various management levels of the auditee.
3. It is good practice to verify the reliability of information received from auditees. This can be
done by:
▪ carrying out specific follow-up audits of findings, conclusions and recommendations
presented in previous reports and which are of continuing interest and/or pose a significant
risk;
▪ checking on the implementation of audit recommendations during subsequent audits with the
same auditee;
▪ using other sources of information, such as Parliament, media, and the general public.
20
4. In most cases SAIs present the results of their audits to Parliament and take into account the
level of interest that their audit work raises and the opinions it generates. SAIs can use
appropriate methods to solicit feedback from Parliament. Examples include:
▪ presenting the audit results to standing committees, particularly to the committee
responsible for public accounts;
▪ participating in, or supporting, Parliamentary debates;
▪ establishing direct contacts with MPs in order to provide them with explanatory information
on specific audit reports; and
▪ publishing a list of unimplemented audit recommendations in the SAI’s annual reports
and, where possible, disclosing the reasons why.
5. SAIs can also obtain feedback from the general public on the outcomes of their audits. This
can be done in the form of surveys, monitoring of media reaction to audits and enabling
stakeholders to comment via the SAI’s website (see topic IV.2).
6. It is good practice to prepare and publish periodic reports on the impact of audit findings,
conclusions and recommendations, including information on financial and other benefits
resulting from the SAI’s audit activity for the state and the general public.
21
II. AUDIT MATTERS
II.5 Quality Review of Completed Audits
CHALLENGE
It is important for the credibility of the SAI that its audits are of the highest standard and that this can
be demonstrated to stakeholders. Review procedures on completed audits are needed to provide
assurance on the quality of the audit process and its output, and contribute to improvement when
required complementing the supervision and review during the audit process.
RESPONSE
Reviews of completed audits can lead to the achievement of higher audit quality and therefore
enhance the credibility of audit results. Moreover, reviews of completed audits can be used to
identify possible improvements, corrective measures or changes to working procedures and/or
manuals. Finally, these types of review can be used to measure and document audit quality, and to
compile information for performance indicators (see topic I.2).
The reviews can be undertaken by external or internal reviewers, as long as the latter are not part of
the audit team. Some SAIs might choose to carry out internal reviews of particular issues while
allocating other issues to external reviewers.
GOOD
PRACTICES
1. The SAI may choose to undertake reviews of completed audits for all or a sample of audit tasks.
In the latter case, the selection of tasks might be based on criteria intended to ensure the coverage
of key strategic activities, including different audit types.
2. The post-audit reviews can focus on general systematic issues or specific issues related to
particular audits. Reviews can also target individual elements of the audit procedure rather than
the whole process.
3. The objectives of the reviews may vary. The review can cover issues such as whether the audit
approach and the applied methodology complied with professional standards, whether the methods
chosen were appropriate for the audit subject(s) and objective(s); and whether the audit findings
were based on solid, well argued and clearly presented evidence.
4. In the case of internal reviews it is good practice to focus on consistency with internal procedures,
professional standards and recommended methodology. Internal reviews can take the form of:
▪ cross-reviews, carried out by a group of experienced auditors working in a cross section of
organisational units, who did not participate in the particular audit to be reviewed; or
▪ reviews carried out by a specialised quality assurance function (unit) established within
SAIs independent from the execution of audit tasks. This organisational unit should report
directly to the Head of SAI or to those with responsibility for audit quality and/or methodology.
5. In case of external reviews of audits SAIs can engage a suitably qualified expert from financial
management associations, academic institutions, external audit firms or other SAIs. In the particular
case of performance audits, the SAI may establish focus or expert groups, representing different
disciplines, to undertake regular reviews of its audits.
22
6. It is important that top management is made aware of the results of the reviews. If these contain
recommendations regarding future audit work, it is good practice to also communicate these results
to all staff members so that they can apply them when carrying out their work.
7. It is also good practice to publish the results of reviews in order to increase the transparency and
credibility of the SAI. This could be done, for example, by asking reviewers to rate the particular
audits and by publishing these ratings on a regular basis, thus informing stakeholder of progress
made.
23
III. HUMAN RESOURCES
III.1 Staff Performance Appraisal
CHALLENGE
One of the main resources of SAIs is their employees. SAIs need to align the skills and knowledge
of the staff to the objectives of the organisation. They should therefore fairly and appropriately
assess how far their employees are adequately knowledgeable, skilled, satisfied and motivated.
RESPONSE
Performance appraisal is a management tool by which the performance and ability of employees
are evaluated (in terms of output quality and quantity, efficiency and timeliness). It is an important
tool for use in managing career development. Performance appraisal addresses institutional needs,
as well as the needs, abilities, motivation, and expectations of staff members.
Staff performance appraisal is a system of highly interactive processes aimed at:
▪ making an objective assessment of staff performance;
▪ increasing staff motivation and self-esteem;
▪ providing opportunities for organisational development particularly through the definition of
human resource strategy and goals;
▪ identifying training needs and developing training programmes;
▪ developing and facilitating effective communication; and
▪ distributing rewards and other incentives on a fair and transparent basis.
GOOD
PRACTICES
1. The performance appraisal process typically starts with setting the objectives for each staff member
for a given period. These targets should be based on the objectives of the organisation and the
personal development needs of the individual. There are several practical approaches to
performance appraisal, such as the results-focused and the behaviour-based models. In the first
case, performance plans are prepared and compared with achieved performance objectives at the
end of the appraisal period. In case of the behaviour-based approach, performance reviews focus
on an assessment of the knowledge and skills of employees.
2. It is good practice to establish appropriate appraisal criteria against which to assess performance.
Appraisal criteria should comprise clearly defined responsibilities for all functions, written job
descriptions and expectations. Performance appraisals using these criteria can be undertaken by
applying standardised checklists, templates and forms.
24
includes self-assessment, reviews by other colleagues and assessment of hierarchical
superiors.
4. The timing of the performance appraisal can vary. It can be carried out on a regular basis,
covering tasks completed during a particular period (usually annually) or upon completing a
particular task. The main benefit of the latter is that it provides focused and immediate feedback. It
is however more time consuming and disruptive than annual performance appraisals.
5. In order to facilitate the appraisal of staff, a rating scale method can be introduced to provide a
basis for a more objective and comparable evaluation of performance derived from established
principles. The rating scale can also serve as a basis for promotion and remuneration decisions.
25
III. HUMAN RESOURCES
III.2 Professional Training
CHALLENGE
All organisations depend on the knowledge, skills, expertise and motivation of their human
resources to perform effectively. Training and staff development are critical for creating the work
environment and culture that is conducive to achieving high levels of professionalism and quality.
SAIs can face several challenges when organising training for staff including what training to
provide for the development of the required competences, who will deliver the training, how and
when. The organisation also needs to ensure that the training it provides is relevant, cost-effective
and useful in reaching the organisation’s objectives.
RESPONSE
SAIs can develop human resource development policies, systems, strategies and plans to
implement integrated training and organisational learning activities. These initiatives should be
justified and clearly linked to development strategy and/or operational requirements. Training is not
effective unless it has a purpose and meets the needs of the SAI and its employees.
GOOD
PRACTICES
1. Training needs analyses address how SAIs are performing and what is needed to improve
performance. They also assess the skills and abilities of their work force and whether they are
sufficient to meet the organisation's needs. Training needs should firstly be determined for the
organisation as a whole, secondly for groups or units with particular functions, and thirdly for the
individual employees. Various sources can be used to determine training needs including staff
performance appraisals, quality control processes, audit plans, employee suggestions as well as
human resources development and organisational strategies. The results of the training needs
analysis can then serve as the basis for establishing the required areas and priorities for training, as
well as the form it should take.
2. Once training needs have been identified, the SAI can establish a training policy and use a
standard process to plan and design the learning events, including defining learning objectives
and drawing up an annual training plan. In addition, appropriate learning methods, training materials
and techniques need to be developed.
3. A range of training methods can be used by SAIs. If appropriately applied, these measures can
have a powerful impact on the individual and ultimately on organisational performance and
behaviour. They can increase motivation, prepare employees for promotion to more senior positions
and encourage staff retention. The following are examples of good practices in this area:
▪ A structured programme can be designed by SAIs to provide induction training to new staff on
the organisation, its culture, as well as on the internal procedures and work practices. It can take
place soon after an individual is employed in order to maximise the benefit of the training.
▪ SAIs can require staff to participate in specific in-house or external training events, including
training courses aimed at reinforcing skills and competences in audit techniques and
procedures, courses on the use of information technology, as well as training on communication
skills and personal effectiveness. Training sessions can also be held to familiarise staff with new
methodologies. The SAI can use various training methods including learning on the job,
lectures, workshops, seminars, secondments, as well as online learning and support through the
26
SAI’s intranet. It is good practice to require participants of training courses to actively share the
knowledge gained with their colleagues.
▪ SAIs aiming to raise their number of qualified staff can support employees who wish to study
for a relevant University degree or obtain professional audit qualifications.
▪ Coaching can be used to assess and improve a particular area of an individual’s ability or
performance. It is a ‘time bound’ relationship aimed at meeting specific goals.
▪ Mentoring is a more open-ended approach through which experienced colleagues can use
their knowledge and understanding of the work and workplace to regularly support, guide and
advise less experienced staff. SAIs can create specific mentoring programmes to enhance
workplace support.
▪ SAIs can also strengthen the training process by drawing up personal development plans for
staff members. These should be reviewed annually during staff appraisal interviews. The plans
can also be used to match employees’ interests and career aspirations with the present and
future needs of the SAI, including the identification of potential leaders/managers.
▪ SAIs can take a further step and develop their own cadre of internal trainers through ‘train the
trainers’ programmes.
4. Evaluation is important to the training process as a means of understanding where training has
succeeded, and where it has failed. It also allows the SAI to pinpoint areas for improvement.
Moreover, evaluation serves to assess whether the organisation achieved its objectives and
whether the investment in training was effective. There are several methods that can be used to
collect and analyse data on the outcome of training and to determine whether the training led to the
development of new skills and desired changes in work practices or behaviour. These include
holding post-event meetings (such as group discussions or one-to-one interviews),
questionnaires, and systematic monitoring of improvements in the workplace.
27
III. HUMAN RESOURCES
III.3 Staff Satisfaction
CHALLENGE
To operate effectively, SAIs need to ensure that staff members have a high degree of job
satisfaction and work towards achieving the objectives and interests of the organisation. High
levels of staff satisfaction have a positive impact on both the organisation and its employees.
Satisfied employees tend to be more productive, creative and committed to the organisation
making it indispensable for SAIs to consider satisfaction and motivation levels of their staff as a
priority.
RESPONSE
Achieving a high quality product requires motivated and adequately remunerated employees who
act in the organisation's interests. Satisfaction levels usually increase when employees know that
they are listened to, their issues of concern are being addressed and they are offered adequate
career opportunities and incentives.
SAIs should offer an appropriate range of flexible schemes and work-life benefits that pay due
attention to the promotion of gender equality, give each individual the opportunity to fulfil their
personal potential and make the best possible contribution towards the organisation.
GOOD
PRACTICES
1. It is good practice for SAIs to make sure that they have appropriate policies related to salaries,
incentives and career opportunities. Employees at all levels of the organisation appreciate
being recognised and acknowledged for their commitment, contribution and output. It is therefore
important for the organisation to identify and show appreciation for the individual and collective
contribution of employees in the achievement of its goals.
2. Financial rewards and non-monetary incentives can help keep the workforce motivated and
satisfied, and raise staff morale and increase loyalty. SAIs can provide family-friendly measures,
including childcare and healthcare support. To attract and retain staff, some SAIs also offer
other benefits such as meal vouchers and the possibility to participate in a supplementary
pension scheme.
3. In order to contribute to staff satisfaction, the work environment needs to be safe, friendly,
ergonomically comfortable and keep stress to an acceptable level. It is important to have modern
and adequate office equipment and facilities, as well as easy access for the disabled.
4. Interpersonal relations are also a key element of overall staff satisfaction. Promoting a good
working relationship among colleagues, positive thinking, trust and cooperation encourages a sense
of teamwork among employees and increases job satisfaction.
5. It is good practice for staff representatives (e.g. unions) to be involved from the outset in the design
and introduction of staff-related schemes such as flexible working arrangements, which can be in
the form of:
▪ Flexi-time, which offer staff flexibility in the times for starting and finishing work while respecting
core hours and the length of lunch break.
28
▪ Compressed working hours to enable staff to work their total number of hours over fewer
days. This working pattern can be limited to a maximum number of hours worked in one day
(e.g. ten hours) and to ensure that a minimum time is allocated for lunch break.
▪ Time off in lieu or unpaid leave to allow the extra time away from work to coincide with
personal needs.
▪ Reduced hours allowing staff members to work less hours in a week.
▪ Telework or a Work From Home arrangement can be offered to those employees who have
jobs that can be carried out equally well from alternative location. Tasks that require a high level
of concentration and minimum interruption, such as report writing, can be performed efficiently
from home.
6. It is good practice to carry out regular staff satisfaction surveys. The survey should be
anonymised to encourage honest feedback and to obtain a better indication of staff satisfaction.
▪ The questionnaire can cover all aspects of employment, e.g. organisational issues, work
environment, culture and management leadership styles. It may also focus on particular
aspects on which feedback is required, such as on the level of assistance provided by internal
support services to facilitate their work or improve the quality of their working life.
▪ It is good practice to provide quick feedback about the overall results to employees, both
positive and negative. This helps to demonstrate management's commitment to the process.
▪ Holding exit interviews of staff leaving the organisation is a good opportunity to obtain
feedback and identify specific problems.
29
IV. COMMUNICATION
IV.1 Internal Communication and Dialogue
CHALLENGE
An organisation which has insufficient internal communication and dialogue risks isolating and de-
motivating its staff, which in turn can jeopardise audit quality and lead to inefficiencies.
RESPONSE
SAIs should disseminate to all employees, through appropriate channels, sufficient and
appropriate information about their operations, policies, programmes, and work-life activities.
Developing various means to communicate and share information encourages and strengthens
the quality of internal communication and dialogue.
SAIs should establish strategy and procedures aimed at the creation of a friendly and
constructive communication environment and encourage management and staff to actively use
these mechanisms.
GOOD
PRACTICES
1. It is good practice to hold regular and frequent meetings between top management and staff.
These meetings can provide the opportunity to discuss points of common interest and share
information, as well as make proposals for improvement.
3. The intranet is a key platform for providing internal information and dialogue. It is good practice
to provide information on organisational structure, strategic plans, methodological guidance, work
in progress and published reports, administrative and building services, internal vacancies,
internal social news and items of personal interests, government and research links, and links to
various news services.
5. Professional forums could be organised to provide adequate orientation to newly recruited staff,
enhance their job satisfaction and encourage their retention.
6. SAIs can also distribute a regular newspaper covering topics of interest to management and
staff including, for example, details of staff suggestions that have been taken up, information on
recently issued reports, technical articles, recruitment of staff, information on awards, as well as
details of staff transfers and promotions.
30
IV. COMMUNICATION
IV.2 External Communication and Relationship with Stakeholders
CHALLENGE
SAIs are most effective if their work is known, read and understood outside the organisation.
They need to identify effective external communication channels to ensure sufficient transparency
and accountability, including enhancing their authority, credibility and reputation.
RESPONSE
Information on the results of SAIs` activities are made available to auditees, Parliament, media,
public, academics and research institutions. SAIs can also establish fruitful relationships with
national control bodies, other SAIs and international organisations.
It is important to establish and maintain continuous, positive and constructive dialogue with main
stakeholders through press conferences, press releases, statements to parliamentary
committees, the availability of sufficient information on the website and other forms of
communication. Effective use of constructive feedback from stakeholders can lead to improved
audit quality and the development of the professional activity of the SAI.
GOOD
PRACTICES
1. It is good practice for SAIs to focus on relations with Parliament as a principal stakeholder. This
can be done through various means, including the organisation of conferences, working groups,
committees, as well as initiatives to promote the role and work of the SAI. A special parliamentary
liaison unit can be established to facilitate communication between the SAI and Parliament. These
measures are important to ensure that SAIs` work remains relevant, credible and useful to
parliamentarians
2. SAIs can forward questionnaires to auditees (see topic II.4) to elicit their opinion on the way
audits have been conducted and to identify areas where improvements can be made.
3. An external advisory panel or board can be established to advise SAIs on professional issues.
The board, under the chairmanship of the SAI, could be comprised of recognised professionals from
different areas of expertise and background representing key stakeholders.
4. It is good practice for SAIs to establish a media centre with primary responsibility for managing
relations with the media and other organisations. The SAI can use this as a focal point to ensure a
policy of fair and equal treatment of the press, as well as to enhance credibility of the SAI through
increased transparency in its relations with Parliament, press and the public.
The media centre can ensure the day-to-day coordination of the public release of reports and
statements. It can also be used to build strong working relationships with the media, assist
parliamentary press staff on SAIs’ work and monitor press coverage of audit reports. Moreover, the
centre can issue guidance to staff and conduct media training for SAI officials involved in
responding to press inquiries.
5. The SAI’s website can also include comprehensive information on the organisation’s mission, core
values, strategy, activities, audit reports, careers opportunities, as well as speeches and professional
events. It could also provide links to other SAIs and other organisations.
31
6. It is useful for SAIs to keep track of public opinion (see topic II.4) on their activities through
monitoring information from publications, articles, and the media. In addition, some SAIs provide
feedback forms on their websites to encourage the public to give their comments on the SAI activities.
7. SAIs can cooperate with national control and law enforcement bodies (or other relevant
ministries, state committees and entities), in a way that does not compromise their independence.
32
CONCLUDING RECOMMENDATIONS
This document sets out various good practices covering many topics that are considered highly relevant to
improve quality. It is recommended for SAIs to take account of the good practices presented in this document
when establishing and operating a quality management system. The selected good practices reflect the
current situation, but are likely to develop and change over time. Within the framework of a step-by-step and
continuous learning process, SAIs should strive to extend the good practices related to an effective quality
management system to all aspects of their activities.
An effective quality management system covers enablers that are concerned with how the organisation
undertakes its activities (such as leadership; human, financial, and information resources, operational
processes), as well as results that are concerned with what results are being achieved (with special regard to
the needs of stakeholders and the general public).
These good practices could be applied individually. However – as the Introduction sets out – they will be more
effective if applied within the context of an integrated system, whereby they apply to all SAI activities and the
different elements complement each other based on common standards and principles.
Achieving quality in SAIs is important in order to provide credible, reliable and useful results, to allow the
organisations to lead by example, to provide a rewarding working environment, as well as to make efficient
use of resources. The overall purpose is to work effectively in the interests of the general public.
Quality within any organisation is not achieved spontaneously but requires a specific approach led by
management. They should be aware of the importance of, and be committed to, achieving the highest quality
standards.
The commitment to quality is important enough to be included in the SAI’s mission statement, making clear
that quality is required during the performance of all tasks of the organisation, and at all stages of those tasks.
It gives appropriate weight to the significance of the establishment or further development of a quality
management system, and increases the probability that all necessary measures – defined in internal
regulatory documents – are taken.
As a further step, it is recommended that the top management of SAIs establishes a plan to achieve high
quality, in the course of which they should:
▪ take stock of current quality arrangements and identify the measures still needed to achieve an
effective quality management system;
▪ define objectives and principles;
▪ set priorities;
▪ programme actions to be taken along with a defined timetable and details of the persons
assigned with responsibilities.
In the course of executing quality related plans, it is proposed to
▪ identify and assess the risks related to the implementation of actions and to be addressed for
within professional and administrative activities, as well as to establish control measures to
mitigate these risks;
▪ set up the required processes and procedures;
▪ ensure the allocation of appropriate resources and the availability of information necessary to
support the operation and monitoring of these processes and procedures.
33
Plans are recommended to be published in the interest of transparency and as an incentive for those required
to implement the measures. This helps underline the credibility (and authority) of the SAI by demonstrating
good practice in its own management.
34
ANNEX 1
Link between Good Practice Topics and Related ISSAI 40/ISQC 1 Element
II.3 Cooperation with the Auditee during the Audit Element 5: Engagement performance
Process
IV.1 Internal Communication and Dialogue Element 1: Leadership responsibilities for quality
within the firm
IV.2 External Communication and Relationship with Element 1: Leadership responsibilities for quality
Stakeholders within the firm
35