Evaluation of the Energy Innovation Program - Summary

What the Evaluation Found

Relevance

• There is a need for federal government support to energy innovation to reduce the risks associated with the high costs and uncertainty of developing and demonstrating new clean technologies in support of reducing GHG emissions.
• The role of NRCan in conducting and managing energy research, development and demonstration (RD&D) is appropriate given its legislated mandate, expertise and long-standing experience. However, the recent creation of several new innovation programs dispersed among several departments and agencies has contributed to confusion among external stakeholders as to NRCan’s role vis-à-vis other federal government players delivering energy innovation programs.

Effectiveness

• Outcomes pertinent to collaborations, leveraging, codes and standards, and technical advancements have met or exceeded reasonable historical expectations.
• The Program plays an important enabling role in addition to its direct support of technology advancement.
• While too early to realize longer-term outcomes, several technology areas demonstrate promise. For more advanced clean energy technologies, external factors such as low oil prices and unfavourable regulatory environments have hindered more widespread adoption.
• NRCan’s consistent and long-term support and broad approach were identified as key factors in advancing innovation.  

Efficiency

• There are good indications that EIP is a well managed program demonstrating good due diligence practices. The Program also regularly reviews its performance to improve its design and effectiveness.
• While a good performance measurement system is in place there is a need to streamline the system and to address some gaps in the tracking of outcomes, particularly knowledge transfer and uptake.
• Certain contribution funding rules and repayability requirements are not aligned with innovation programming and hamper efficiency and effectiveness.
• The Program conducted regular stakeholder consultations and environmental scans to inform strategic planning. In 2016, strategic planning was influenced by the short-time frames in which to plan and implement the energy innovation program and did not fully reflect long-term needs.

Program Information

This evaluation assessed two phases of RD&D programs – the ecoENERGY Innovation Initiative (ecoEII) and the subsequent Energy Innovation Program. These programs cover a range of technology areas such as renewables, electric vehicles and buildings which focus on reducing greenhouse gas emissions and enhancing competitiveness. Total expenditures from 2014-15 to 2017-18 were $268.6M.

The EIP was launched in April 2016 as an interim two-year renewal, pending development of the Pan Canadian Framework. In Budget 2017 EIP was renewed, receiving a total of $211.6M over four years and then ongoing funding of $52.9m per year.

About the Evaluation

The evaluation covers the period from 2014-15 to June 2018. Data collection was conducted between March 2018 and January 2019.

Methods: Document/data review, 6 case studies,
and 67 stakeholder interviews, review of 30 randomly selected project files.

Recommendations and Management Action Plan

Recommendation 1. In light of new core funding for EIP, OERD should strengthen its strategic planning process:

• Consider a broader systems based approach to planning for each technology area
• Set clear and realistic strategy, targets and milestones

Management Action Plan – Recommendation 1

Management agrees

Actions:
1.a In 2018-19 OERD developed a new strategic planning framework centred around objective-based missions that focus RD&D activities to address specific technology gaps to meet government of Canada priorities using a systems based approach.

OERD and IETS labs have already used the framework to develop their integrated 3-year work plan (2019-20 to 2022-23) for internal research and development programs.  Currently the framework is informing the planning and design of the relaunch of Energy Innovation Program in 2019-20.  (DG, OERD, Completed March 31, 2019)

1b.As part of the new strategic planning framework, work plans include a clear delineation of how each project contributes to one or more missions, as well as, targets and milestones that will enable tracking of results.  (DG, OERD, March 31, 2020)

Recommendation 2. OERD should communicate the role of EIP to external stakeholders to clarify its role.

Management Action Plan – Recommendation 2

Management agrees

Actions:
2. OERD is currently preparing to launch the revamped Energy Innovation Program during FY 2019-20. As part of the relaunch, OERD is working with NRCan’s Communications and Portfolio Sector to develop a new communications plan and suite of products and tools to help clarify the objectives, scope and focus of EIP for external and internal stakeholders.

OERD will continue to collaborate with its federal government partners, particularly the Clean Growth Hub, to ensure that intended stakeholders and target groups for EIP and other OERD administered programs are able to find, understand, and have the information they need to apply for the appropriate program (DG, OERD march 31, 2020)

Recommendation 3. OERD should further enhance elements of its program design and delivery as follows:

• Consider new, more flexible models of funding intake, especially given more stability in funding allocations.
• Work with Treasury Board to eliminate contribution repayment clauses and to identify and implement strategies to minimize funding lapses and reprofiling.

Management Action Plan – Recommendation 3

Management agrees

Actions:
3a. OERD has taken steps to integrate new, flexible funding models and calls for proposals into its program design. The launch of the revamped Energy Innovation Program will incorporate effective practices and lessons from OERD programs, such as Clean Growth Program and Impact Canada Clean Tech Initiative. These could include accelerated approval processes through trusted partnerships; enhanced collaboration through the Science and Technology Assistance for Cleantech (STAC) model, and/or innovation prizes and challenges. (DG, OERD, March 31, 2020)

3b.  OERD strongly concurs with the recommendation to remove the repayability requirement.  OERD will continue to pursue this course of action, working with NRCan’s Corporate Management and Services Sector and the Treasury Board Secretariat regarding the elimination of the contribution repayment clause from OERD programs. 

Furthermore, OERD will continue to make representation to Finance regarding appropriate funding profiles for innovation programs (i.e. less funding in start up years). (DG, OERD, March 31, 2020)

Recommendation 4. OERD should improve its performance measurement system:

• Strengthen its performance measurement processes to ensure systematic monitoring and reporting of other aspects of effectiveness (e.g. codes and standards, uptake of information products, intellectual property, etc.)
• Streamline reporting requirements (e.g. identifying the most critical information, using appropriate format and structure for follow-on recipient reporting)

Management Action Plan – Recommendation 4

Management agrees. 

Actions:
4a. OERD will take a phased approach to enhance its performance measurement processes. Phase I will include a review of performance metrics and the development of new tools and methods for data collection and analysis. By June 2020, OERD will review its performance indicators and metrics as part of a broader NRCan and Innovation and Energy Technology Sector exercise to review performance measurement frameworks (please see 4b). (DG, OERD June, 2020)

Additionally, in Phase II, which will occur over the next two years, OERD will develop, test and implement methods and tools (2-4) to measure uptake and diffusion of IP, information and influence on codes and standards, as well as, measure the long-term outcomes of OERD investments in energy innovation programs.  (DG, OERD, March 31, 2021)

OERD is also exploring opportunities to automate data collection methods and analysis that will enable more systematic monitoring and reporting of performance information. 

4b.   OERD is working with other branches in the Innovation and Energy Technology Sector and relevant groups within NRCan to review performance measurement frameworks and reporting requirements (ex. Departmental Results Framework, Performance Information Profiles) in an effort to reduce and streamline reporting requirements.   

Additionally, OERD is streamlining reporting requirements for proponents as part of the new strategic framework. OERD is also revising its structure for follow-on recipient reporting.  Further, the automation of data collection and adoption of new performance measurement methods and tools will improve the quality of performance data, while reducing the reporting burden for proponents.  (DG, OERD, March 31, 2020)