Summary of the Joint Audit and Evaluation of the Transition to the Departmental Results Framework
About the Transition to the Departmental Results Framework (DRF)
The transition to the DRF was driven by the government’s objective to change the reporting framework that was established by the Policy on Management, Resources and Results Structures (2010) and the Policy on Evaluation (2009). The new requirements of the Policy on Results (2016) require all federal departments and agencies to move to an improved performance measurement approach centred on the DRF. For the purposes of this joint engagement, the DRF transition is a continuum comprised of three phases: implementation, roll out and update of the DRF.
Text Description
Natural Resources Canada's Departmental Results Framework for 2019-20
This image illustrates the various components that make up NRCan's Departmental Results Framework and Program Inventory for 2019-20.
Across the top of the image is a row of three boxes identifying NRCan’s Core Responsibilities and an additional box representing Internal Services. Reading from left to right, the three Core Responsibilities are: Natural Resource Science and Risk Mitigation, Innovative and Sustainable Natural Resources Development, and Globally Competitive Natural Resource Sectors.
In the column under Natural Resource Science and Risk Mitigation is a description of this Core Responsibility, which is, “Lead foundational science and share expertise for managing Canada’s natural resources, reducing the impacts of climate change and mitigating risks from natural disasters and explosives.” Also in the column are the three Departmental Results associated with this Core Responsibility, and several Departmental Indicators associated with each Departmental Result.
The first Departmental Result under Natural Resource Science and Risk Mitigation is, “Canadians have access to cutting-edge research to inform decisions on the management of natural resources.” The five Departmental Indicators associated with this Departmental Result are:
- Number of times scientific products related to natural resources are accessed by Canadians
- Percentage of environmental assessment processes for which NRCan provided scientific and technical expertise
- Number of times stakeholders acknowledge using NRCan’s scientific and technical products in making their decisions
- Number of training and development initiatives that enable NRCan to incorporate Indigenous traditional knowledge in conjunction with NRCan science
- Quality index of geographic and locational data on Canada’s land resources, water and infrastructure
The second Departmental Result under Natural Resource Science and Risk Mitigation is, “Communities and officials have the tools to safeguard Canadians from natural hazards and explosives.” The three Departmental Indicators associated with this Departmental Result are:
- Percentage of hazardous natural events within Canada for which a notification was issued in a timely manner
- Number of enhanced wild fire monitoring tools using remotely sensed information
- Percentage of inspections of explosives rated safe
The third Departmental Result under Natural Resource Science and Risk Mitigation is, “Communities and industries are adapting to climate change.” The two Departmental Indicators associated with this Departmental Result are:
- Percentage of Canadian communities and industries that have taken steps to adapt to climate change
- Number of times NRCan products and expertise on adaptation are accessed by communities and industry
At the bottom of the Natural Resource Science and Risk Mitigation column is a list of NRCan programs associated with this Core Responsibility, which includes the following programs:
- Canadian Geodetic Survey: Spatially Enabling Canada
- Geological Knowledge for Canada’s Onshore and Offshore Land
- Core Geospatial Data
- Canada-US International Boundary Treaty
- Canada Lands Survey System
- Geoscience for Sustainable Development of Natural Resources
- Pest Risk Management
- Forest Climate Change
- Climate Change Adaptation
- Explosives Safety and Security
- Geoscience to Keep Canada Safe
- Wildfire Risk Management
- Polar Continental Shelf program
In the column under Innovative and Sustainable Natural Resources Development is a description of this Core Responsibility, which is, “Lead the transformation to a low-carbon economy by improving the environmental performance of Canada’s natural resource sectors through innovation and sustainable development and use.” Also in the column are the three Departmental Results associated with this Core Responsibility, and several Departmental Indicators associated with each Departmental Result.
The first Departmental Result under Innovative and Sustainable Natural Resources Development is, “Natural resource sectors are innovative.” The four Departmental Indicators associated with this Departmental Result are:
- Percentage of NRCan-funded innovation projects that result in new intellectual property, standards or regulations
- Percentage of NRCan-funded clean energy innovation projects advancing along the innovation scale
- Number of NRCan-funded green mining technologies, including waste and water management, proven through demonstrations
- Number of new forestry products developed that are informed by NRCan tools and knowledge
The second Departmental Result under Innovative and Sustainable Natural Resources Development is, “Clean technologies and energy efficiencies enhance economic performance.” The three Departmental Indicators associated with this Departmental Result are:
- Success of NRCan-funded clean technology demonstrations in terms of economic performance
- Ratio of leveraged investments in energy innovation projects funded by NRCan
- Total annual energy savings resulting from adoption of energy efficiency codes, standards and practices
The third Departmental Result under Innovative and Sustainable Natural Resources Development is, “Canada’s natural resources are sustainable.” The six Departmental Indicators associated with this Departmental Result are:
- Percentage of Canadian electricity generated from non-GHG emitting sources
- Number of renewable energy projects in remote communities and off-grid industrial operations
- Amount of wood harvested compared to the sustainable supply
- Change in greenhouse gas emissions resulting from NRCan-funded clean technology demonstrations
- Number of low-carbon recharging and refueling stations under development or completed
- Number of policies and initiatives developed collaboratively with Indigenous groups and communities
At the bottom of the Innovative and Sustainable Natural Resources Development column is a list of NRCan programs associated with this Core Responsibility, which includes the following programs:
- Clean Energy Technology Policy, Research and Engagement
- Clean Growth in Natural Resource Sectors
- Energy Innovation Program
- Green Mining Innovation
- Fibre Solutions
- Sustainable Forest Management
- Cumulative Effects
- Lower Carbon Transportation
- Electricity Resources
- Energy Efficiency
- Energy and Climate Change Policy
- Innovative Geospatial Solutions
In the column under Globally Competitive Natural Resource Sectors is a description of this Core Responsibility, which is, “Advance and promote market access, inclusiveness and competitiveness for Canada’s natural resource sectors, in support of jobs and economic growth.*” The asterisk indicates that this also includes statutory payments for offshore petroleum. Also in the column are the three Departmental Results associated with this Core Responsibility, and several Departmental Indicators associated with each Departmental Result.
The first Departmental Result under Globally Competitive Natural Resource Sectors is, “Access to new and priority markets for Canada’s natural resources is enhanced.” The four Departmental Indicators associated with this Departmental Result are:
- Canada’s share of U.S. and global imports of natural resources
- Number of Canadian-owned resource companies operating abroad
- Number of NRCan-led trade and promotion missions supporting the development or expansion of market access for natural resources
- Average number of companies, provinces/territories and Indigenous leaders participating in trade and promotion missions
The second Departmental Result under Globally Competitive Natural Resource Sectors is, “Canadians are engaged in the future of the new and inclusive resource economy.” The three Departmental Indicators associated with this Departmental Result are:
- Percentage of policy, regulatory and legislative changes with formal mechanisms for broad public engagement
- Number of joint analytical products with provinces and territories
- Number of Indigenous groups and communities implicated in economic development projects
The third Departmental Result under Globally Competitive Natural Resource Sectors is, “Enhanced competitiveness of Canada’s natural resource sectors.” The two Departmental Indicators associated with this Departmental Result are:
- Percentage of resource development project decisions on target as per timelines
- Number of times NRCan’s economic and investment data are accessed
At the bottom of the Globally Competitive Natural Resource Sectors column is a list of NRCan programs associated with this Core Responsibility, which includes the following programs:
- Forest Sector Competitiveness
- Provision of Federal Leadership in the Minerals and Metals Sector
- Energy Safety and Security, and Petroleum Resources
- International Energy Engagement
- Statutory Offshore Payments
- Major Projects Management Office – West
- Major Projects Management Office Initiative
- Youth Employment Strategy
What the Engagement Found
Relevance
The DRF transition is aligned with the Minister of Natural Resources mandate letter and the Treasury Board (TB) Policy on Results. The DRF transition planning process was relatively responsive to existing departmental reporting needs and priorities prior to the release of the Policy on Results (July 2016).
Compliance of DRF Tools
The DRF tools developed by NRCan comply with Policy on Results requirements. They were implemented, rolled out, and updated within the timeframe set by the Treasury Board Secretariat (TBS). The work of the Strategic Policy and Innovation sector (SPI) has systematically guided the DRF transition toward compliance.
Planning Process
Program Officials indicated relative satisfaction with the adequacy of resources allocated to the process. The planning process was based on internal and external knowledge and best practices, but the planning strategy could have been better documented and more comprehensive, and introduced earlier in the roll-out of the transition project.
Governance
In general, the roles and responsibilities for the DRF transition have been carried out in accordance with the requirements of the Policy on Results and approved by the Executive Committee and the Performance Measurement, Evaluation and Experimentation Committee (PMEEC). The roles of the different designated functions, especially those of the Chief Information Officer (CIO), the Chief Financial Officer (CFO), and the Chief Human Resources Officer (CHRO), appear to be largely unknown as they apply to the transition and Program Officials do not appear to be sufficiently informed about the different functional roles.
Engagement Strategy
Significant efforts were made to engage all NRCan sectors and external stakeholders, including TBS, in the transition to the DRF. However, the joint engagement found that the engagement strategy could also have been better documented, more comprehensive, and introduced earlier in the roll-out of the project. The engagement also noted that the effectiveness of the engagement strategy was assessed differently by the SPI and the other sectors. Most Program Officials and members of the Corporate Planners Network are satisfied with the role they have played in the DRF transition. However, many expressed a desire to know more about the consideration given by managers of the DRF transition to their perspectives and contributions.
Performance Measures
To some extent, the DRF transition has resulted in a better definition and understanding of Departmental Results. In terms of opportunities for improvement, several stakeholders stated that there was a need to better integrate the performance indicators (including long-term ones) required by program managers into the DRF. The need for alignment with financial indicators (efficiency) was also highlighted as an important element of program performance measurement.
About the Engagement
The objective of this joint audit and evaluation engagement is to examine the compliance of Natural Resources Canada’s (NRCan’s) transition to the Departmental Results Framework (DRF) with the Treasury Board (TB) Policy on Results and to assess the effectiveness of this transition, as well as the impacts of DRF planning and implementation on program management and reporting. The joint engagement covers the period from April 1, 2016, to March 31, 2020. The analysis excludes improvements to the DRF via annual amendments made after March 31, 2020. The 2022–23 DRF amendment process is currently the fourth completed iteration made to continue improvements to the DRF.
NRCan’s Audit and Evaluation Branch (AEB) conducted this engagement in accordance with the TB Policy on Results (2016) and the Institute of Internal Auditors’ International Standards for the Professional Practice of Internal Auditing and the Government of Canada’s Policy on Internal Audit. Details on specific engagement questions, methods and limitations are found in the full report.
Recommendations
Management Response and Action Plan
- It is recommended that the ADM Strategic Policy and Innovation (SPI) notify PMEEC of the sectors’ proposed improvements to the DRF based on consultations with Program Officials, members of the Corporate Planners Network, and members of the Planning and Reporting Committee, and then share the PMEEC’s feedback with these groups.
Management response: Management agrees with Recommendation #1.
Communication between PMEEC and other levels of governance that currently take place as part of the annual DRF amendment process will be strengthened. Moving forward, the SPI Planning, Delivery and Results Branch (PDR) will present feedback received from Program Officials, the Corporate Planning Network (CPN), and the Planning and Reporting Committee (PRC) in DRF amendment presentations.
Position Responsible: Head of Performance Measurement and DG, SPI-PDR
Due Date: November 2022
- It is recommended that the ADM Strategic Policy and Innovation (SPI) communicate to Program Officials, the Planning and Reporting Committee, and the Corporate Planners Network, the roles of various designated functions (as part of the DRF), especially those of the CIO, the CFO, and the CHRO, as well as those of the Head of Performance Measurement and the Head of Evaluation.
Management response: Management agrees with Recommendation #2.
SPI-PDR will update guidance and ensure the roles of designated functions as outlined in the Policy on Results are clarified. Updated guidance will be circulated and referenced in presentations delivered to Program Officials, Sectors, CPN and PRC, to increase understanding of key roles and responsibilities under the Policy and Directives on Results.
Position Responsible: Head of Performance Measurement and DG, SPI-PDR
Due Date: November 2022
- It is recommended that the ADM Strategic Policy and Innovation (SPI) formalize and continue to improve its engagement process for DRF implementation and base it on effective collaboration and consensus with stakeholders. This engagement process should be re-evaluated annually to ensure that it continues to reflect the expectations of the sectors, and that it adapts to changes in expectations.
Management response: Management agrees with Recommendation #3.
Through the annual DRF amendment process, SPI-PDR will lead a comprehensive review to ensure:
- the relevance and usefulness of the DRF for planning and resource allocation decision-making; and
- the framework aligns with recent policy directions and government priorities.
The annual review will entail:
- consultation meetings with Sector planners and Program Officials to identify proposed amendments, review and confirm proposed changes; and
- validation and approvals through briefings to PRC and PMEEC.
SPI-PDR will develop an engagement plan that will be integrated into the DRF amendment process. SPI-PDR will also provide progress updates to PMEEC and apply necessary adjustments accordingly.
Position Responsible: Head of Performance Measurement and DG, SPI-PDR
Due Date: November 2022
- It is recommended that the ADM Strategic Policy and Innovation (SPI) work with sectors and Program Officials to continue to improve the various elements of the DRF, particularly to increase their usefulness for direct program management including the financial aspect of their performance. To achieve this, SPI must, in concert with other sectors and their planners, develop and roll out an effective strategy for improving program performance indicators and their integration into the DRF. The strategy should take into account indicators related to government-wide policy considerations such as gender-based analysis plus and official languages, where appropriate
Management response: Management agrees with Recommendation #4.
SPI-PDR, with Program Officials and Sector Planners, will formalize an annual review process that will require Sectors to review and validate the availability, quality, and usefulness of performance indicators and ensure that key priorities (e.g., equity, diversity and inclusion, GBA Plus, official languages) are reflected in their Performance Information Profiles (PIP) and in the DRF. This process will be conducted as part of the implementation of the new PIP Webform.
Program Officials will also be invited to present their PIPs periodically to PMEEC. SPI-PDR will work with AEB to ensure evaluation recommendations about improvements to program performance measurement are considered by Program Officials in PIPs.
Position Responsible: Head of Performance Measurement and DG, SPI-PDR
Due Date: Formalized annual review process presented to PMEEC April 2023 and implemented annually thereafter.
The full version of the report for this engagement is available on Natural Resources Canada's website.
Page details
- Date modified: