Language selection

Search


JOINT AUDIT AND EVALUATION SUMMARY PLAN 2018-2021

Table of Contents

Preamble

On December 4, 2017, the Audit Branch and the Strategic Evaluation Division were amalgamated into the Audit and Evaluation Branch (AEB).  This was done to capitalize on the natural strengths of each function, leveraging the synergies between them while maintaining a high degree of rigour and professionalism.

Development of the Risk-Based Audit Plan and Evaluation Plan provided an opportunity for joint planning within this new Branch. Joint planning has resulted in the development of a joint plan, even though the Audit and Evaluation plans each have different drivers and information requirements, as per their respective TB policies- i.e. Policy on Internal Audit and Policy on Results.  It has also successfully resulted in a more comprehensive review of the Audit and Evaluation requirements that exist within Departmental sectors and their programs allowing for joint engagements, where possible, through improved planning, including consultations with senior management on the full suite of planned audits and evaluations. 

The audit and evaluation plans outline the organizational priorities, risks and the tools used to identify those risks, and the opportunities to mitigate them. These priorities and risks help guide the selection of audit and evaluation projects for the Department during the period covered by these plans.

For the audit risk-based planning process, NRCan’s universe of auditable entities includes various programs, activities, processes, policies, and initiatives, which collectively contribute to the achievement of the Department’s strategic objectives. Priorities and Core Responsibilities, such as those identified in the NRCan Minister’s mandate letter, NRCan’s Departmental Results Framework (DRF), and the Department’s inventory of external legislated services, help to assess the completeness of the audit universe, which includes 24 groupings of auditable entities.

For Evaluation, planning is guided by the requirements of the TB Policy on Results introduced in 2016. Priority is given to a) ongoing programs of grants and contributions with average annual expenditures of $5M or more or b) programs with evaluation commitments made as part of Treasury Board submissions. Evaluations of other programs are determined by risk, with the overarching goal being that all departmental programs should be evaluated every five years unless they are low risk. A key factor influencing this year’s planning is the government-wide transition from the Program Alignment Architecture to the DRF.

Under this new structure, the Departmental Audit Committee (DAC) will continue to review all audit reports from AEB prior to their approval by the Deputy Minister. Similarly, all evaluation reports will also continue to be reviewed by the Performance Measurement, Evaluation and Experimentation Committee (PMEEC) prior to Deputy Minister approval. Future joint A&E reports, combining features of both audits and evaluations, will be reviewed by both the DAC and PMEEC.

In developing the plans, AEB considered four priorities as critical to the Department meeting its Core Responsibilities and supporting the Government of Canada’s priorities:

  • Working with partners to restore public confidence in developing and bringing natural resources to market;
  • Innovating to optimize resource potential and to support Canada’s transition to a low-carbon economy;
  • Leveraging NRCan’s expertise to maintain public safety and security; and,
  • Enabling NRCan to be a high-performing organization that is innovative and collaborative in delivering on its mandate.  

Summary

Natural Resources Canada’s (NRCan) Joint Audit and Evaluation (A&E) Plan, also referred to as the “Plan”, is prepared by the Audit and Evaluation Branch (AEB). It contains details on the role of internal audit (IA), evaluation, the AEB’s planning methodology for both functions, and the planned projects for the next three year cycle: 2018-21. Under the new AEB structure, which came into effect on December 4, 2017, the Departmental Audit Committee (DAC) will continue to review all audit reports and the Performance Measurement, Evaluation and Experimentation Committee (PMEEC) will continue to do the same for all evaluation reports from AEB, prior to their approval by the Deputy Minister (DM).

The Plan is developed in accordance with the requirements of the Treasury Board of Canada (TB) Policy on Internal Audit and the TB Policy on Results, along with related directives, guidelines and as it specifically pertains to IA, the Institute of Internal Auditors’ (IIA) International Standards for the Professional Practice of Internal Auditing.

Risk-Based Audit Process

Each year, NRCan’s Chief Audit and Evaluation Executive (CAEE) is required to prepare a risk-based audit plan, which sets out the priorities of the internal audit activity that are consistent with the organization’s goals and priorities. The audit planning process ensures that all internal audit activities are relevant, timely, and strategically aligned with NRCan’s Corporate Risk Profile (CRP) to support the achievement of the Department’s strategic objectives. The input from NRCan’s DAC, along with NRCan’s senior management, is sought and taken under advisement in setting internal audit activity priorities.

The starting point for the risk-based planning process is the identification of the audit universe. This audit universe document was developed by the AEB and is updated annually to reflect the Department’s most current priorities. The audit universe characterizes the array of possible audit activities and is made up of auditable entities identified as relevant to NRCan and its operating context. Auditable entities commonly include programs, processes, policies, management activities and control systems, along with departmental and government-wide initiatives, which collectively contribute to the achievement of NRCan’s strategic objectives. NRCan’s audit universe is made up of 24 groupings of auditable entities.

All auditable entities are subjected to a risk assessment and risk ranking exercise to select audit projects in order of priority. Criteria used for selecting audit projects include- past audit coverage and results; materiality; significance to management; level of risk; auditability; audit projects carried forward from the previous year’s Plan; organizational priorities; high priority areas identified by central agencies, such as the Office of the Comptroller General (OCG) and Office of the Auditor General (OAG), among others; and, legislated or other mandated obligations.

Prioritization of the audit universe is a two-step process. The first step includes management consultation, review, and consideration of the following available documentation: NRCan minister’s mandate letter; the former Program Alignment Architecture (PAA) and its replacement the Departmental Results Framework (DRF), departmental risk information, including NRCan’s CRP; the latest Management Accountability Framework (MAF) assessment; recent departmental-wide assessments of  IT and fraud risks, which lead to the identification of audits as part of the AEB’s continuous audit framework; business planning documentation; NRCan’s Departmental Plan (DP); Government priorities; and previous audit results (both internal and external), along with the Department’s most recent financial information and statements.

Other factors are also considered, such as collaboration with NRCan’s Evaluation function to identify opportunities to carryout joint work/efforts in order to improve efficiency and minimize duplication of efforts. 

The second step to prioritize the audit universe involves consideration of several factors, including significance to departmental core responsibilities and operational objectives; senior management requests and priorities; the DAC’s advice and recommendations; external audit activities and planned evaluations; readiness of the entity for audit activities; and availability of internal resources to complete the audit on time. Following this step, professional judgement is still required to risk-assess and rank the auditable entities. This is performed through collaborative discussions with NRCan senior management and the DAC, where emphasis is placed on projects planned for 2018-19 (the first year of the three-year plan), given that future projects are reassessed annually. Government and departmental priorities are also validated with senior management and the DAC to ensure planned audits align with higher priority areas. In addition, preliminary audit objectives are developed for each audit selected for the Plan. The final plan is then reviewed by the DAC and approved by the DM.

The following diagram highlights the four key phases used in the selection process for the development of a robust risk-based audit plan.

image
Text version

The Four Key Phases Used in the Selection Process for the Development of a Robust Risk-Based Audit Plan

This figure highlights the four key phases used in the selection process for the development of a robust Risk-based audit plan. It covers the starting point of the selection process that determines potential NRCan auditable entities covering a 3 year period to its final recommendation. The first large block represents the potential range of auditable components, which include departmental programs, activities, processes, structures, and initiatives that collectively contribute to the achievement of the Department’s core responsibilities. It is called the audit universe. The Audit and Evaluation Branch uses the Departmental Results Framework (DRF) as well as NRCan's inventory of external legislated services to ensure the audit universe identified is complete. There are approximately 24 groupings of auditable entities based on the DRF and NRCan’s sectors.

The next stage is to prioritize the audit universe based on a risk-based assessment. This is a two-step process that involves a preliminary and final prioritization based on a number of factors such as likelihood of risk and impact. The final 2 steps are to rank the priority of the proposed projects and to recommend them for approval in the 3 year audit plan (as in the final 2 large blocks).

Evaluation Process

The CAEE also oversees the development of an evaluation plan that outlines the policy and operational context for evaluation services delivered at NRCan. The 2017-18 PAA was replaced with the 2018-19 DRF and associated Program Inventory, which consists of all possible programs that might be considered for evaluation over the course of the next three years. In selecting programs to be evaluated, priority was first given to all mandatory evaluations, including:

  • all ongoing programs of grants and contributions with average annual expenditures of $5M or more;Footnote 1 and
  • evaluation commitments made as part of Treasury Board submissions.

Once all mandatory evaluations are accommodated within the evaluation plan, a risk assessment is undertaken of the remaining programs within the evaluation universe. The risk assessment exercise involves joint consultations with the IA function of the AEB to solicit senior management feedback on program risks and priorities for evaluation, as well as a review of departmental documentation (e.g., NRCan’s CRP, business planning documentation, NRCan’s Report on Plans and Priorities, departmental audit plans, 2017-18 departmental evaluation plan). Risks are assessed according to various program characteristics (e.g., upcoming renewal, materiality, visibility), management practices (e.g., structures, delivery complexity, performance measurement), past research findings, and upcoming audits. Evaluations are then planned for some of the remaining units of the evaluation universe according to the availability of evaluation resources and the entity’s relative risk ranking.

The final plan also considers feedback and/or requests from the Treasury Board Secretariat (TBS) and makes accommodation for non-project related advisory and other activities to support the department in responding to broader federal initiatives related to experimentation, cross-cutting program reviews (e.g., Innovation, Science), or special studies (e.g., prospective evaluations to support program design).

For operational reasons, this Plan has been prepared using a 3-year planning horizon. Given that the TB Policy on Results requires departments to submit a 5-year evaluation plan, an evaluation schedule (see appendix) has been included identifying the complete list of planned evaluations. These evaluations will cover approximately 88% of NRCan’s total planned program spending, which are subject to change during the actual scoping of the evaluations and exclude amounts for Internal Services and Statutory Payments.

Planning Results

The following table summarizes the number of new audits, evaluations, and joint A&E projects selected for each year, along with the number of OCG-led horizontal audits. In total, 42 of the highest priority engagements are planned for the next three years. Projects that have been carried forward from FY 2017-18 can be found in the respective detailed plans.

Table 1 – Number of New Projects by FY
New AEB Projects 2018-19 2019-20 2020-21
Audits 5 7 5
Evaluations 2 5 3
Joint A&E Projects (see Table 2) 3 2 3
Horizontal Audits (OCG-led) 1 1 1
Horizontal Evaluations (OCG-led) 3 - 1
TOTAL 14 15 13

Joint Audit & Evaluation Projects

The IA and Evaluation functions have held joint consultations with senior management and staff to ensure the most effective and efficient planning process, allowing the AEB to provide coordinated advice. As a result, this year’s update to the Plan includes eight potential future joint projects where collaboration is possible. Table 2 provides a listing of joint A&E projects for FYs 2018-19 to 2020-21. It should be noted that collaborative efforts will range from conducting joint interviews, the collection and sharing of information, to conducting hybrid audit and evaluation engagements.

Table 2 – Joint A&E Projects
2018-19 2019-20 2020-21
  • Management Control (MC) & Performance Measurement (PM) Framework of the Forest Sector Competitiveness Program (CFS)
4. MC & PM Framework of the Green Infrastructure Initiative (IETS, ES, CFS) 6. NRCan Environment
Studies & Assessment
Process (OCS – All
Science Sectors)
  • ESTMA (LMS)
5. Transition to the
Departmental Results
Framework
(SPRS – All Sectors)
7. MC & PM Framework of the Clean Growth in Natural Resources Sector Program
(IETS)   
  • Electric Vehicle and Alternative Fuel Infrastructure Deployment Initiative (ES) 

 

  8. MC & PM Framework of
Electricity – Emerging
Renewables Program (ES)

Audit of Public Accounts 2017-18

The AEB has received confirmation that the OAG will not perform detailed audit testing of NRCan’s Offshore Revenue activities as part of their annual public accounts audit program. The IA function will continue to perform audit testing on the controls related to this activity in FY 2018-19, given the visibility and importance to the provinces. The audit of Offshore Revenue will be carried out as one element of AEB’s annual continuous audit projects, which is described later on in this Plan.

Follow-up on Previous Audit & Evaluation Recommendations

As per the TB Policy on Internal Audit and International Standards for the Professional Practice of Internal Auditing and the new TB Policy on Results, the CAEE “must establish a follow-up process for each audit and evaluation recommendation and ensure that management actions have been effectively implemented or that senior management has accepted the risk of not taking action.”

The follow-up process at NRCan is a two-phase process, which begins with a management self-assessment of the level of implementation for each recommendation, which is accompanied with a Management Action Plan (MAP). MAPs receive senior management approval and assign detailed tasks to responsible parties that address audit recommendations in a timely manner. In the fall of each FY, the AEB reports on the status of the implementation of recommendations based on management’s self-assessment. Each spring, as part of the second phase, the AEB performs a validation of the full implementation of recommendations, confirming that the MAPs are indeed fully implemented. The validation approach includes the following procedures: conducting interviews; reviewing supporting evidence; and performing analysis and testing based on risk. Once completed, a Follow-Up Report is produced and approved by the DM. Once approved, the portion that is related to the IA function is sent to the OCG, where applicable.

Continuous Auditing of Core Controls

The AEB will continue to undertake assurance-based continuous auditing at NRCan to proactively identify potential control issues and report annually on various processes. This work will be performed in accordance with the IIA Standards (i.e. provide reasonable assurance). The work carried out will address key risks associated with significant departmental expenses that have been identified in part, as a result of NRCan’s updated Fraud Risk Assessment’s MAPs.

The 3 areas selected for continuous audit in 2018-19 are:

  • Travel;
  • Offshore Revenues; and
  • Grants and Contributions Processes.

NRCan’s annual report on continuous audit activities will be completed for the DAC’s fall 2018 meeting for continuous audits completed for FY 2017-18.

Tables 4 and 5, below, present the new “highest priority” audits, evaluations and joint A&E projects for fiscal years 2018-19, 2019-20 and 2020-21.

Table 4 – Audit Projects for FYs 2018-21
2018-19 2019-20 2020-21
1. NRCan’s Emergency
Management Activities
(CMSS – All Sectors)
7. Management of
Infrastructure within
Remote Areas
(CMSS – All Science Sectors)
15. NRCan’s Experimentation and    
Innovation Strategy
(SPRS – All Sectors)
2. NRCan’s Strategic & 
Operational Planning Process
(SPRS, CMSS – All Sectors)
8. Lower Churchill Falls Loan 
Guarantees (ES)
16. Capital Asset Management
(CMSS - All Sectors)
3. Federal Geospatial Platform
(SPRS – All Sectors)
9. Clean Technology Stream
(IETS)
17. Management of Fed-Prov.  
Offshore Agreements (ES)
4. Management of Aging IT
Systems (CMSS – All Sectors)
10. Open Government
(CMSS – All Sectors)
18. Implementation of NRCan’s IT
Strategy (CMSS - All Sectors)
5. Canada Land Surveys Program
(LMS)
11. Management of NRCan’s  
Communication Function   
(CPS)
19. Transition to Social Media &  
Information Sharing (CPS)
6. Horizontal Audit of Project
Management (CMSS –
OCG-led)
12. Horizontal Audit of
Information for Decision  
Making (CMSS – OCG-led)
20. TBD (CMSS – OCG-led)
  13. Aligning NRCan’s Culture to  
its Mandate (SPRS –
All Sectors)
 
  14. Classification (CMSS)  
*Annual Continuous Audit Projects (CMSS)*
Table 5 – Evaluation Projects for FYs 2018-21
2018-19 2019-20 2020-21
1. Geoscience for Sustainable
Development and Natural
Resources: Targeted
Geoscience 5 (LMS)
6. Forest Sector Competitiveness:
Forest Sector Products Market
Access and Diversification
(CFS)
11. Lower Carbon Transportation: Deployment of Electric  Vehicles and Alternative Transportation Fuels (ES/IETS)
2. Electricity Resources:
Renewable Energy (ES)
7. Climate Change Adaptation
(LMS)
12. Electricity Resources: Smart Grid (ES/IETS)
3. Energy Innovation Program:
World Class Tanker Safety
System-Phase II (IETS)
8. Polar Continental Shelf (LMS) 13. Energy Innovation Program: Clean Energy Science and Technology  (IETS)
4. Explosives Safety and Security:
Single Window Initiative (LMS)
9. Clean Growth in Natural
Resources (IETS)
14. Climate Change Adaptation and Clean Growth (OGD-led)
5. Clean Growth and Climate:
International Cooperation (ES)
10. Geoscience to Keep Canada
Safe: Canada Hazards
Information Service (LMS)
 

Central Agency Audit Projects for 2018-19 – 2020-21

The Department is also subject to audits by other assurance providers. Table 6 provides a listing of known external audit projects planned for FYs 2018-19 to 2020-21, with the expected tabling dates.

Table 6 – Planned External Audit Projects for FYs 2018-19 to 2020-21

Office of the Comptroller General

Horizontal Audit of Project Management Winter 2019
Horizontal Audit of Information for Decision Making Spring 2019

Office of the Auditor General

Audit of Building and Implementing Phoenix Spring 2018
Coordination of the 2017-18 OAG Audit of Public Accounts Fall 2018
Audit of Financial Assurances (2012) Spring 2019

Commissioner of the Environment and Sustainable Development

Audit of Oil Spills in Marine Waters Fall 2018
Audit of Departmental Progress in Implementing Sustainable Development Strategies Fall 2018
Audit of Corporate Social Responsibility in Natural Resources Sector Fall 2019
Audit of Mitigation of Severe Weather Events / Search and Rescue Fall 2019
Public Service Commission System-Wide Staffing Audit Fall 2018

Advisory Projects for FYs 2018-21

As an adjunct to the assurance role, the TB Policy on Internal Audit indicates that as part of the departmental Plan, the CAEE can consider “other oversight engagements, including, where the necessary expertise and capacity are in place, the option to provide consulting services to the organization.” Provided this, the AEB estimates 2-3 advisory projects per FY, which will be based on senior management needs and the availability of AEB’s resources. It is important to note that the AEB will always continue to maintain its independence, when providing both advisory and assurance types of services. The inclusion of advisory projects into NRCan’s Plan helps to ensure additional value is provided to senior management to complement our regular audit activities.

Appendix

NRCan Five-Year Evaluation Schedule (FYs 2018-19 to 2020-23)
Departmental Results Framework Program Planned Evaluations (next 5 years) Planned Approval (FY) 2018-19 Estimated Program Spending covered by Evaluation ($) 2018-19 Total Planned Program Spending ($) Evaluation Driver/Rationale
Geological Knowledge for Canada’s Onshore and Offshore Land Geo-Mapping for Energy and Minerals 2018-19 35,028,007 36,748,755 TB Commitment
Canada’s Extended Continental Shelf (Horizontal Evaluation led by GAC) 2018-19 501,788 TB Commitment
Core Geospatial Data Core Geospatial Data/Innovative Geospatial Solutions 2022-23 14,046,011 14,921,984 NRCan Priority
Canada Lands Survey System Canada Lands Survey System program 2019-20 9,560,082 12,172,017 TB Commitment
Geoscience for Sustainable Development of Natural Resources Targeted Geoscience Initiative Program 2019-20 11,087,543 26,548,273 TB Commitment
Climate Change Adaptation Climate Change Adaptation Program 2020-21 9,578,265 11,026,199 TB Commitment
Climate Change Adaptation Theme (Horizontal led by ECCC) 2021-22     TB commitment
Explosives Safety and Security Single Window Initiative (Horizontal Evaluation led by CBSA) 2018-19 1,532,812 6,328,863 TB Commitment
Explosives Safety and Security Program 2022-23 4,796,051   NRCan Priority
Geoscience to Keep Canada Safe Canadian Hazards Information Systems 2019-20 14,126,747 17,175,143 TB Commitment
Targeted Geoscience Initiative 2019-20 115,495   TB Commitment
Climate Change Adaptation (NRCan) 2020-21 2,932,901   TB Commitment
Climate Change Adaptation Theme (Horizontal led by ECCC) 2021-22     TB Commitment
Polar Continental Shelf Program Polar Continental Shelf Program 2019-20 7,392,750 7,392,750 NRCan Priority
Clean Energy Technology Policy, Research and Engagement World Class Tanker Safety System (Horizontal Led by TC) 2018-19 5,528,894 77,517,143 TB Commitment
Clean Energy Technology Policy, Research and Engagement program 2019-20 71,988,249   NRCan Priority
Clean Growth in Natural Resource Sectors Clean Growth in Natural Resource Sectors 2019-20 82,486,581 82,563,058 FAA Requirement
Energy Innovation Program Energy Innovation Program 2018-19 41,605,238 97,828,054 FAA Requirement
Deployment of Electric Vehicles and Alternative Transportation Fuels  2020-21 10,876,444 FAA Requirement
Smart Grids Program 2021-22 14,036,755 FAA Requirement
Electric Vehicles Infrastructure Demonstration Program 2021-22 22,344,000 FAA Requirement
Clean Energy for Rural and Remote Communities 2022-23 3,957,720 FAA Requirement
Energy Efficient Buildings Programs 2022-23 4,038,225 TB Commitment
Green Mining Innovation Green Mining Innovation Program 2022-23 15,713,887 15,713,887 NRCan Priority
Fibre Solutions Forest Sector Innovation Program 2018-19 3,442,733 11,717,311 TB Commitment
Electricity Resources Regional Electricity Cooperation and Strategic Infrastructure (RECSI) 2018-19 0 158,778,058 TB Commitment
Renewable Energy Deployment 2020-21 108,852,705 TB Commitment
Smart Grids Program 2021-22 10,869,837 FAA Requirement
Emerging Renewables Program 2022-23 24,939,778 FAA Requirement
Clean Energy for Rural and Remote Communities 2022-23 4,729,135 FAA Requirement
Energy Efficiency Energy Efficiency Program 2018-19 37,547,263 53,003,633 TB Commitment
Energy Efficient Buildings Programs 2022-23 15,456,370 TB Commitment
Energy and Climate Change Policy Energy and Climate Change Policy Capacity 2018-19 7,563,627 7,563,627 NRCan Priority
 Lower Carbon Transportation Energy Efficiency Program 2018-19 5,743,040 24,735,147 TB Commitment
Deployment of Electric Vehicles and Alternative Transportation Fuels  2020-21 18,992,107   FAA Requirement
Innovative Geospatial Solutions Core Geospatial Data/Innovative Geospatial Solutions 2022-23 14,367,311 14,367,311 NRCan Priority
Forest Sector Competitiveness Management Control  & Performance Measurement  Framework of Forest Sector Competitiveness 2018-19 98,742,643 98,742,643 NRCan Priority
Forest Sector Innovation 2018-19 FAA Requirement
Expanding Market Opportunities 2019-20 FAA Requirement
Clean Energy for Rural and Remote Communities 2022-23 FAA Requirement
Provision of Federal Leadership in Minerals and Metals Sector Extractive Sector Transparency (ESTMA) 2019-20 10,068,295 10,743,532 NRCan Priority
International Energy Engagement Energy and Climate Change Policy Capacity 2018-19 1,233,153 4,626,787 NRCan Priority
Clean Growth and Climate Change - International Cooperation Theme (Horizontal led by ECCC) 2020-21 3,146,731 TB Commitment
Indigenous Engagement Office – West West Coast Energy initiative / Indigenous Partnerships Office - West 2018-19 20,139,828 20,139,828 TB Commitment
Major Projects Management Office Major Projects Management Office Initiative (Horizontal led by NRCan) 2018-19 6,101,598 6,101,598 TB Commitment
Youth Employment Strategy Youth Emplyment Strategy     (Horizontal  led by ESDC) 2019-20 9,298,120 9,298,120 TB Commitment
Canadian Geodetic Survey: Spatially Enabling Canada Not Applicable (NA) NA NA 5,625,504 NA
Canada-US International Boundary Treaty NA NA NA 1,980,413 NA
Pest Risk Management NA NA NA 23,724,920 NA
Wildfire Risk Management NA NA NA 6,929,115 NA
Forest Climate Change NA NA NA 10,584,648 NA
Cumulative Effects NA NA NA 6,282,791 NA
Sustainable Forest Management NA NA NA 14,211,829 NA
Energy Safety and Security and Petroleum Resources NA NA NA 7,902,284 NA
Total -   784,508,719 895,092,941  
Percentage of 2018-19 Total Planned Program Spending
Covered by 5-year Evaluation Plan
87.6%

Table Notes: N/A- Programs denoted Not Applicable (NA) indicate no planned evaluations, being low risk and having no other driver. 


 

Page details

Date modified: