Summary of the Horizontal Evaluation of the of the Major Projects Management Office Initiative

Program Information

Established in 2007, the Major Projects Management Office Initiative (MPMOI) is a federal horizontal initiative involving 12 federal regulatory departments/ agencies and led by Natural Resources Canada (NRCan). It consists of two main components:

• Providing funding to partner departments/ agencies to build their internal scientific, technical and Indigenous consultation capacity to support timely and robust technical analyses and Indigenous consultations; and
• The Major Projects Management Office (MPMO), housed within NRCan, which serves as a central coordinating body for reviews of major projects across Canada.

Funded activities of the MPMOI include, scientific and technical expert capacity, Indigenous consultations, proponent support and system integration, policy and reporting, and legal support. Five departments/agencies received funding to deliver these five activities.

In addition to examining the funded activities of the MPMOI, the evaluation covered activities funded under the Interim Strategy for Pipelines and other National Energy Board (NEB) Reviews (Interim Strategy). The objective of the Interim Strategy was to support Government commitments to restore public confidence in environmental and regulatory review processes while a broader review of those processes took place. It consisted of specific measures to ensure that reviews of major resource projects subject to NEB review aligned with principles announced by the Government of Canada in January 2016. These principles were:

• No proponent would be asked to return to the starting line;
• Decisions would be based on science and traditional knowledge of Indigenous peoples;
• The views of the public and affected communities would be sought and considered;
• Indigenous peoples would be meaningfully consulted and where appropriate, impacts on their rights and interests will be accommodated; and
• Direct and upstream greenhouse gas (GHG) emissions linked to the projects under review would be assessed.

Resources: Over this time period, departments/agencies were allocated approximately $274.6 million (M), which consisted of:

• MPMOI. $81.0M in new funding (in addition to $164.9M in funding from existing departmental/ agency budgets); and
• Interim Strategy. $28.7M in new funding (in addition to $13.0M in funding from existing departmental/agency budgets).

About the Evaluation

The evaluation covers the period from 2015-16 to 2017-18 and includes both the MPMOI and the Interim Strategy. Data collection was conducted between July 2018 and February 2019.

Objective: to assess the relevance, effectiveness, and efficiency of the MPMOI, including the Interim Strategy. It includes an assessment of the continued relevance and role of the MPMO.

Methods: Document/data review and 52 stakeholder interviews with 68 individuals.

Evaluation Governance:The evaluation was supported by a Horizontal Evaluation Advisory Committee, chaired by NRCan’s Audit and Evaluation Branch. It comprised representatives from the evaluation functions at: Fisheries and Oceans Canada; Crown-Indigenous Relations and Northern Affairs Canada-Indigenous Services Canada (CIRNAC-ISC); Canadian Environmental Assessment Agency; Environment and Climate Change Canada (ECCC); Transport Canada (TC); and the NEB. NRCan contracted PRA Inc. to conduct the evaluation.

Legislative context: Since the MPMOI was established in 2007, the legislative and regulatory regime for major resource projects in Canada has evolved considerably.

• Implementation of Canadian Environmental Assessment Act, 2012 (CEAA 2012) to improve the timeliness and predictability of the review process, reduce duplication of project reviews, strengthen environmental protection, and enhance Indigenous consultation. It also reduced the number of responsible authorities from 40 to 3.
• The proposal for a new impact assessment and regulatory process, specifically Bill C-69 and Bill C-68, fundamentally changing the legislation underpinning the MPMOI (i.e., CEAA 2012).

What the Evaluation Found

Relevance

The MPMOI and Interim Strategy align with:

• Priorities of the federal government, including those related to environmental protection, action on climate change, and reconciliation with Indigenous peoples; and
• Priorities and responsibilities of departments/agencies.

There is a continued need for departments/agencies to provide scientific, technical, and Indigenous consultation capacity to support major resource project reviews.

There is a continued need for functions currently delivered by the MPMO, but disagreement about which entity should be responsible for these in the future.

Effectiveness

No definitive conclusions on the extent to which progress has been made towards outcomes due to a lack of baseline and performance measurement data, and diverse perspectives from key informants.

Nevertheless, there is some evidence of progress related to the three outcomes assessed:

• Coordination of the regulatory review process for major resource projects;
• Timeliness, predictability, quality, transparency and accountability of regulatory reviews; and
• More coordinated, consistent and meaningful Indigenous and public consultations.

Multiple factors, internal and external to the Initiative, influence the ability of the Initiative to achieve its expected outcomes: role of the MPMO; capacity and resources; public expectations; economic/market conditions; and government priorities.

Efficiency

To the extent it provided support to federal departments/agencies to fulfill their statutory obligations related to major project reviews, the design was appropriate. However, given the evaluation could not conclusively determine progress towards expected outcomes, the design may not be optimal.

Due to limited objective information on resource use, outputs and expected outcomes, the evaluation cannot conclude on whether it is operating efficiently. However, there is no evidence to suggest an alternative design would be more efficient or effective.

Deputy Minister and Assistant Deputy Minister (ADM) committees, the key elements of the governance structure, are seen as a main strength of the Initiative. However, there may be a need to clarify the roles and responsibilities of these committees, their involvement in issues resolution, and the nature of NRCan participation.

Some performance measurement information is often insufficient to demonstrated effectiveness and efficiency.

Recommendation 1

NRCan should work with participating departments and agencies to ensure continuation of relevant activities currently performed by the MPMO. This process should be informed by a comprehensive review of the Office’s mandate, roles and responsibilities relative to the roles of other participating departments and agencies and/or external stakeholders, and should include the following task:

• Determine what entity or entities (including the MPMO itself) should carry out roles and responsibilities currently performed by MPMO going forward, with explicit reference to any pertinent legislative (i.e., Bill C-69) or other developments occurring or anticipated to occur at the time the determination is made, and update the mandate as required.

Management Response and Action Plan

NRCan management agrees.

With the coming-into-force of Bill C-69 and introduction of the new Impact Assessment Act (IA Act), many of the Office’s roles and responsibilities related to coordination of the federal regulatory process for major projects now reside with the Impact Assessment Agency of Canada (IAAC). Other roles, for example, with respect to coordination of crown consultation, now lie with IAAC, lifecycle regulators, or regulators.

Regarding responsibility for horizontal policy issues, the mandate and authorities of IAAC as set out in the IA Act, is to administer, coordinate, and develop policy directly related to the IA Act. 

NRCan has undertaken a comprehensive review of the Office’s former functions and horizontal coordination requirements and identified which entity or entities have responsibility for these roles in the new legislative and regulatory context. 

NRCan, working with participating departments and agencies, is pursuing a revised mandate for the Office, which takes into consideration the new legislation, the results of the review, as well as the broader context of Government priorities related to Canada’s natural resource sector. 

Responsible Manager – ADM, MPMO
Due Date – September 30, 2020

Recommendation 2

NRCan should collaborate with participating departments and agencies to improve performance measurement for the MPMOI. To ensure responsible departments and agencies can demonstrate the effectiveness and efficiency of the MPMOI and Impact Assessment and Regulatory Processes (IARP) the following steps should be taken (at a minimum):

• Revise the logic model for the Initiative to ensure that it clearly articulates activities, outputs and expected outcomes (immediate, intermediate, and long-term). This may include revising, removing, or adding activities, outputs and expected outcomes to ensure clarity and avoid repetition.
• Develop and implement a set of performance measures corresponding to the outcomes identified in the revised logic model. The performance measures should enable monitoring of progress towards outcome achievement; process indicators should not be relied upon as evidence of outcome achievement, and indicators should be meaningful and relevant to Initiative stakeholders and partners.
• Strengthen monitoring and reporting tools or practices, to facilitate consistent collection of performance information across departments and agencies, and enable a clear understanding of progress towards outcomes or enable course correction if the desired change is not seen. This should include establishing baselines or benchmarks for expected outcomes, and identifying clear targets and dates to achieve these targets.
• Improve tracking of financial and human resource information. This should include:
• Actual resource use from new and existing department and agency budgets for major resource project reviews (including trends over time); and
• Allocation of financial and human resources to various activities, corresponding to the activities identified in the revised logic model.
• Improve public-facing project monitoring systems to ensure that stakeholders and interested members of the public can obtain and understand information about major resource projects under federal review. This should include providing easier access to information about project assessments (including the rationale behind review decisions), as well as improving the functionality and timeliness of public-facing monitoring systems to ensure accessibility of project-level and aggregated information about major resource projects progressing through the federal review system.

With the passing of Bill C-69 and the implementation of the IARP, IAAC should also consider these improvements to performance measurement and collaborate with participating departments and agencies to ensure they are addressed for this new horizontal initiative.

Management Response and Action Plan

NRCan management agrees.

NRCan acknowledges the importance of quality performance measurement to inform program management, improve performance and provide transparent, clear and useful information on results achieved. 

As the horizontal MPMO Initiative led by NRCan sunsetted at the end of FY 2019-20, it is no longer relevant to make revisions to the performance measurement for the existing MPMOI. 

NRCan will ensure that measures to support the evaluation of effectiveness and efficiency are a key component of any new program proposal put forward.

With the coming into force of Bill C-69, IAAC has assumed responsibility for horizontal results tracking and reporting on the new impact assessment and regulatory processes, including monitoring and reporting on performance measures, and tracking of financial and human resources. 

Responsible Manager – ADM, MPMO
Due Date – Not applicable

Impact Assessment Agency management agrees.

In 2018, IAAC (previously the Canadian Environmental Assessment Agency) collaborated with eight other federal departments and agencies to develop a Horizontal Results Framework (HRF) for the IARP horizontal initiative. This HRF was approved by Deputy Heads from each of the nine federal departments and agencies, including: IAAC, Canada Energy Regulator (previously the NEB), NRCan, ECCC, CIRNAC, ISC, Health Canada, TC and Fisheries and Oceans Canada. The HRF was approved by the Treasury Board of Canada Secretariat in December 2018.

The Agency collects financial and non-financial performance information bi-annually, to be published on the Agency’s website as part of the Departmental Results Report (DRR) and Departmental Plan (DP) processes. The first publication of performance information will be with the 2018-19 DRR.

In fiscal year 2019-20, with the coming-into-force of the IAA on August 28, 2019, the Agency further collaborated with the above-mentioned departments and agencies to review targets, dates to achieve targets, and where necessary, to strengthen performance indicators. These updates will be published with the 2020-21 DP.

Moving forward, the Agency will continue to collaborate with partner organizations to ensure ongoing monitoring of performance of the horizontal initiative, and the availability of financial and non-financial performance information to support annual reporting in the DRR and DP, as well as to support future evaluations.

The Agency also continues to update the Canadian Impact Assessment Registry (previously the Canadian Environmental Assessment Registry) to: ensure the public has access to scientific and impact assessment information; provide greater transparency in the decision-making process; and provide public access to compliance and enforcement actions taken.

Responsible Manager - Vice-president, Operations, Vice-president, External Relations and Strategic Policy Sector, Vice-president - Corporate Services Sector
Due Date – March 31, 2020