Summary of the Evaluation of the Forest Disturbances Science and Applications Sub-program

Evaluation Findings

Relevance
There is an ongoing need for scientific knowledge, tools, advice, and collaboration to help stakeholders address forest disturbances related to pests, fire, and climate change. NRCan has a legitimate and appropriate role to play in conducting research, providing advice, collaborating and facilitating cooperation amongst stakeholders.

Performance - Efficiency and Economy
The sub-program demonstrates a high level of collaboration with other organizations, which reduces duplication of effort and facilitates sharing of resources and uptake of outputs. While the sub-program generally responds effectively to the needs of stakeholders, several opportunities for improvement were also identified, primarily with respect to: ensuring all appropriate target audiences are engaged; improving alignment between CFS priorities and the specific needs of different regions; increasing CFS resources or research in specific areas (e.g., FIRE); and improving or updating key datasets and decision support tools. A lack of performance monitoring hindered the ability of the sub-program to monitor progress towards intended outcomes and make adjustments to improve program delivery.

Performance – Effectiveness
Forest sector players are using CFS outputs to address risks related to forest disturbances, and are working collaboratively nationally and internationally to address emerging issues. There is evidence that CFS products and advice have influenced the development of stakeholder policies and practices intended to improve their ability to manage forest disturbances more effectively, however it was not possible to fully assess the extent to which these have in turn improved the effective management of forest disturbances. Although the intended ultimate outcome could not be assessed at the time of the evaluation, a review of the results chain confirms that the sub-program is reasonably expected to contribute to this outcome.  Factors affecting the achievement of outcomes include: collaboration (a key facilitating factor); stakeholders’ ability to participate in key forums, their awareness of potential issues or solutions, and their capacity to use CFS-produced outputs.  

Recommendations

1. CFS should develop and implement a client engagement and communication strategy that identifies or confirms target audiences for all project areas and for key products/outputs (e.g., for pests activities/solutions).
2. CFS should develop and implement specific research strategies that address both client-driven research requirements to meet stakeholder needs and the need for national-level, more strategic research in each of the project areas.
3. CFS should develop a human resources strategy to ensure there is sufficient capacity to address both client-driven research requirements and national-level, more strategic research. It should also include succession planning to identify strategies to recruit the necessary expertise to address capacity gaps due to attrition.
4. CFS should develop and implement a systematic performance monitoring and reporting framework/strategy for outputs, outcomes and leveraged resources, and which also includes a strategy for ongoing monitoring that would facilitate the ability to correct course, if necessary, to ensure the effective, efficient and economic delivery of the program.

Program Description

The sub-program conducts research and analysis, provides advice to stakeholders, and facilitates national and international collaboration among forest sector stakeholders to minimize the negative economic, environmental, and public safety impacts of forest disturbances caused by pests, wildfire, and climate change.

Scope and Methodology, Constraints and Limitations

The evaluation used a theory-based approach with multiple lines of evidence (e.g. document review, end-user and collaborator survey, case studies, interviews). A key limitation of the evaluation was the paucity of performance information available for the sub-program. The survey and case studies focused on the extent to which the sub-program contributed to expected outcomes and thus served to mitigate this constraint.