Audit of the ecoENERGY Innovation Initiative (AU1421)
TABLE OF CONTENTS
EXECUTIVE SUMMARY
INTRODUCTION
The ecoENERGY Innovation Initiative (ecoEII) received funding in Budget 2011, the Next Phase of Canada’s Economic Action Plan, for a comprehensive suite of research and development (R&D) and demonstration projects. The program’s objective is to support energy technology innovation to produce and use energy in a cleaner and more efficient way. This Initiative is a component of the Government of Canada’s actions to achieve real air emissions reductions, while maintaining Canada’s economic advantage. The ecoEII aims to help in the search for long-term solutions to reducing and eliminating air pollutants from energy production and use.
EcoEII will provide $268 million over five years (2011-2016) to fund clean energy research, development, and demonstration (RD&D) activities to support energy technology innovation to produce and use energy more cleanly and efficiently.
The audit of ecoEII was approved by the Deputy Minister on April 26, 2013 as part of the Department’s Risk-Based Audit Plan.
The objective of the audit is to provide reasonable assurance on the effectiveness and efficiency of the program’s management control framework and on compliance with Treasury Board approved terms and conditions.
STRENGTHS
The Office of Energy Research and Development (OERD) has established an effective and efficient management control framework for the delivery of the ecoENERGY Innovation Initiative, in compliance with Treasury Board approved terms and conditions as policies and directives. Program governance and processes are in place to ensure fair and transparent decision making, including the selection of projects. Roles, responsibilities and accountabilities are clearly defined, communicated and understood.
AREAS FOR IMPROVEMENT
Opportunities for improvement were identified with respect to clarifying and documenting costing guidance for eligible expenditures related to internal and other government department proposals and agreements. In addition the audit noted opportunities for improvement for future program planning and design.
INTERNAL AUDIT CONCLUSION AND OPINION
Overall, the Audit Branch can provide reasonable assurance that NRCan has a management control framework in place to effectively administer and manage the ecoEII program. Opportunities were identified to further improve areas related to project costing guidance and long-term planning.
STATEMENT OF CONFORMANCE
In my professional judgement as Chief Audit Executive, the audit conforms with the Internal Auditing Standards for the Government of Canada, as supported by the results of the internal Quality Assurance and Improvement Program.
Christian Asselin, CPA, CA, CMA, CFE
Chief Audit Executive
INTRODUCTION
The ecoENERGY Innovation Initiative (ecoEII) received funding in Budget 2011, the Next Phase of Canada’s Economic Action Plan, for a comprehensive suite of research and development (R&D) and demonstration projects. The program’s objective is to support energy technology innovation to produce and use energy in a cleaner and more efficient way. This Initiative is a component of the Government of Canada’s actions to achieve real emissions reductions, while maintaining Canada’s economic advantage. The ecoEII aims to help in the search for long-term solutions to reducing and eliminating air pollutants from energy production and use.
EcoEII will provide $268 million over five years (2011-2016) to fund clean energy research, development, and demonstration (RD&D) activities to support energy technology innovation to produce and use energy more cleanly and efficiently.
Specifically, the ecoEII program is intended to:
- Create high quality jobs for Canadians;
- Support energy technology innovation to produce and use energy cleanly and efficiently;
- Search for long-term solutions to eliminate air pollutants and greenhouse gas (GHG) emissions from energy production and use; and,
- Foster collaboration among industry, academia and government in the field of energy technology.
The Initiative consists of two separate funding streams: one for Research and Development (R&D) projects, and one for demonstration projects. Both were launched in August 2011. It should be noted that these funding streams include funding for external projects (Demonstration and R&D projects) through the use of contributions and funding for internal R&D projects (i.e. funding for federal laboratory R&D). Primary recipients of the program funding are industry, academia and federal government departments - which include projects within NRCan.
The Demonstration projects are focused on the “integration of renewable energy into both the electricity grid and the built environment” delivered by external organisations. The R&D component funds projects from industry, provincial research organizations and academia, as well as federal labs in the following priority areas:
- Energy Efficiency in Buildings and Communities;
- Clean Electricity and Renewables;
- Bioenergy;
- Electrification of Transportation; and,
- Unconventional Oil and Gas.
Within NRCan, the ecoEII falls under the Innovation and Energy Technology Sector, and is managed by the Office of Energy Research and Development (OERD). OERD is the Government of Canada's co-ordinator of energy research and development (R&D) activities.
AUDIT OBJECTIVES
The objective of the audit was to provide reasonable assurance on the effectiveness and efficiency of the program’s management control framework and on compliance with Treasury Board approved terms and conditions.
Specifically, the audit assessed whether:
- effective and efficient governance processes are in place for the ecoEII;
- the design and delivery of the ecoEII is in accordance with approved terms and conditions and compliance with the Policy on Transfer Payments and the Financial Administration Act (FAA); and,
- program planning, monitoring, and reporting, activities are in place and are operating effectively.
DEPARTMENTAL RISK
An internal audit of the ecoEII was initially identified as part of the Deputy Minister approved Risk-Based Audit Plan for 2014-15. Following additional consultation with senior management, it was decided to begin the audit in 2013-14 to provide assurance on its activities prior to the program’s anticipated renewal.
SCOPE AND METHODOLOGY
The scope of the audit considered the ecoEII program activities from June 2011 to March 2014.
The audit methodology was based on Treasury Board Policy on Internal Audit and Government of Canada Internal Audit Standards and included:
- interviews with key personnel with respect to the management of ecoEII and related activities;
- review of key project documents, as well as relevant policies, directives, and guidance; and,
- a detailed examination of a select sample of funding agreements along with related governance and management processes.
CRITERIA
Please refer to Appendix A.
FINDINGS AND RECOMMENDATIONS
EFFECTIVE AND EFFICIENT GOVERNANCE PROCESSES
Summary Finding
Effective and efficient governance processes are in place for the ecoEII.
Supporting Observations
As part of its mandate with respect to sustainable development, NRCan is responsible for establishing policies, priorities and programs primarily related to federal responsibilities for energy Science and Technology (S&T). The Department fulfils these responsibilities through its energy S&T activities, managed by OERD. The OERD is primarily responsible for the management of the ecoEII program.
The governance framework for the ecoEII program relies on the collaboration between key participants from other government departments, which allows departments to more effectively pool resources, coordinate research activities, and improve the decision making in federal energy S&T investment. This structure involves program management and various committees that have a role in determining which potential ecoEII projects get selected for funding. Key interdepartmental committees with an important role in delivery of the ecoEII include an Assistant Deputy Minister (ADM) Panel on Energy S&T, a Director General (DG) Committee on Energy S&T and Portfolio Committees.
The ADM Panel on Energy S&T focuses on the strategic perspective with responsibilities such as reviewing and approving the program’s strategic plans; reviewing and approving the distribution of effort across the innovation spectrum; considering and approving the most appropriate approach(es) to program delivery for each priority technology area in each Portfolio; and, reviewing and approving / rejecting all project proposals involving ecoEII federal funding totalling $1M or above. The Panel is comprised of S&T ADMs or their equivalents, representing seven federal government departments and agencies.
The DG Committee is responsible for considering and representing the operational level views and interests of its members’ department or agency to OERD and the ADM Panel on Energy S&T. Its main focus is to advise on strategic plans and implementation plans, program delivery mechanisms and operations of the overall program; monitoring progress and sharing information on departmental / agency energy S&T activities. The committee communicates its views and advice to the ADM Panel via OERD, and is intended to provide a transparent liaison link between the Panel and the Committee.
The Portfolio Committees are divided into R&D categories. These committees are responsible for managing the planning and delivery of the ecoEII and direct efforts to selected high-priority areas. They assess project proposals and approve those that are acceptable which involve funding under $1M or, recommend for approval to the ADM Panel those that involve ecoEII federal funding of $1M and overFootnote 1.
After a detailed review of documentation and through interviews, the audit concluded that Management has implemented effective and efficient governance processes to enable the achievement of program objectives and priorities and that roles, responsibilities and accountabilities are clearly defined, documented, and understood by stakeholders.
DESIGN AND DELIVERY
Summary Finding
The design and delivery of the ecoEII is in accordance with approved terms and conditions and compliance with the Policy on Transfer Payments and the Financial Administration Act (FAA). However, the audit found that costing guidance for eligible expenses provided to NRCan and other government departments (OGDs) is not clear.
Supporting Observations
The audit found the solicitation and recipient application processes were conducted in a manner that was fair and accessible. Management established and communicated procedures, along with roles and responsibilities to program staff to ensure the solicitation process was conducted consistently and within the approved terms and conditions. Calls for proposals, along with applicant guides were created and disseminated, providing potential applicants with the necessary information to put forward project proposals. In addition, potential applicants had ready access to information about the program which was made public via the internet.
The audit also found that control processes were in place to ensure the consistent assessment and approval of projects to meet the objectives of the program. Assessment tools and templates containing assessment instructions, including instruction related situations of possible conflict of interest were developed and used for the assessment of projects. An examination of a sample of agreements also confirmed that control processes were in place to ensure the consistent assessment and approval of both internal and external projects to meet the objectives of the program.
Furthermore, the audit found that the program’s objectives, established design and the manner in which the program is being delivered is in accordance with approved terms and conditions and compliance with the Policy on Transfer Payments and the Financial Administration Act (FAA).
Regarding external funding agreements, the audit found that they were consistently developed through a process that addressed risk, recipient requirements, and met TB and departmental guidelines and requirements. Agreements examined were administered in accordance with approved terms and conditions and in compliance with the requirements of the Policy on Transfer Payments and the Financial Administration Act. Any changes to funding levels were documented along with the rationale for the change and all external funding agreements were disclosed in accordance with proactive disclosure requirements.
For internal NRCan R&D funding agreements and R&D funding agreements with other government departments (OGDs), the audit found, however, that they were developed through a process that did not clearly set out project costing guidance with respect to eligible expenditures. Through a review of a sample of project files related to these agreements and interviews the audit noted a variety of methodologies being applied to the costing of internal R&D projects - including different approaches related to costing of overhead for projects.
MANAGEMENT INITIATIVE
The audit found that program delivery incorporated an innovative use of technology. To help potential project proponents enhance the quality of the project proposal, and to promote collaboration, the program had incorporated a web based networking tool during the project solicitation phase of the program that allowed proponents to find other proponents working in similar areas, essentially linking technical experts in order to enhance proposals and increase chances of project success.
RISK AND IMPACT
There is a risk that inconsistent costing approaches for internal projects may result in ineligible costs.
RECOMMENDATIONS
1. The Director General - Office of Energy Research and Development (OERD) should develop comprehensive project costing guidance with respect to eligibility of expenditures for internal and other government departments’ funding agreements.
MANAGEMENT RESPONSE, ACTION PLAN AND TIME FRAME
Management agrees.
OERD will develop project costing guidance for the eligibility of expenditures for internal research and development (R&D) projects and other government departments (OGD) funding agreements. This guidance will be tested in the context of the new Oil Spill Response Science (OSRS) Program, currently being designed. Based on experience with OSRS, the costing guidance will either be adopted or improved for future C-base programs.
Position responsible: Director General - Office of Energy Research and Development
Timing: Guidance document for the OSRS will be prepared by October 31, 2014.
PROGRAM PLANNING AND DESIGN
Summary Finding
Operational planning, monitoring, and reporting, activities are in place and are operating effectively, however; it was noted that the program design did not include resource requirements for the 5 year period following the program's end, during which time deliverables such as monitoring and managing repayable contributions and monitoring and analyzing reports on project outcomes from proponents, as required under reporting requirements outlined in contribution agreements. It was also noted that the program’s design, specifically its planned delivery approach for the initial phases of the program, did not adequately forecast and take into account the time required to carry out all planned program activities during the first fiscal year ending March 2012.
Supporting Observations
The audit found that operational planning processes for the management of the program/projects have been implemented for all key activities, and that these are documented and linked to strategic objectives and priorities. It was also found that the program has implemented risk based monitoring of recipient performance and reporting.
In order to assess level of risk associated with individual recipients, the program conducted desk audits on all initial claims. The desk audits served to identify issues and were used to further inform the level of monitoring required. For higher risk projects this could include increased monitoring or full recipient audits if required. Project monitoring is carried out by the use of a project tracking database incorporating an assessment of project risk and weekly project monitoring meetings. Briefings are also provided on program/project status to all levels of program management on a periodic basis.
The audit also found that performance measurement processes are in place to support program review and the departmental performance strategy. It was noted that management monitors and assesses performance against planned results on an ongoing basis and has made efforts to enhance performance monitoring with the introduction of additional measures designed to identify and assess the maturity of evolving technologies.
Operational planning, monitoring, and reporting activities are in place and are operating effectively once project agreements are put in place. The audit found; however, that the program design did not include resource requirements for the 5 year period following the end of program funding, during which time deliverables, such as reports from proponents, are due to be submitted. The resources required for the post-program activities are minimal in comparison to the period during which funding is distributed to proponents. With that said, these activities are important as they allow the program to report on medium and longer-term outcomes and success stories as well as monitoring re-payability of contributions. Not accounting for the resources required for ‘post-program’ activities could put a strain on available resources and possibly compete with other priorities and deliverables existing at that time.
Furthermore, the audit found that the program’s design, specifically its planned delivery approach for the initial phases of the program, did not adequately forecast and take into account the time required to carry out all planned program activities during the first fiscal year ending March 2012. The planned activities for the first year of the program included the establishment of the program, review and selection of project proposals, notifying applicants of their success or otherwise, completing contribution agreements, distribution of funding to recipients, and allowing sufficient time for proponents to carry out planned activities before fiscal year end March 2012. The additional time required to complete these activities resulted in delays of the final funding allocation decisions for both Demonstration and R&D projects.
It should be noted that although delays were incurred within the initial phases of the program delivery in 2012, information provided by management indicated that the program had met its service standards approximately 87% of the time.
MANAGEMENT INITIATIVE
The audit found that program monitoring included the introduction of innovative performance metrics. The program has recently introduced performance metrics known as Technology Readiness Levels (TRLs) in its ecoEII reporting templates for annual reporting on the status of projects and programs. TRLs are a methodology, which provides an accurate measure of a new innovation’s maturity or position along the innovation spectrum.
RISK AND IMPACT
There is a risk that the program may not meet its objectives in monitoring re-payability of contributions to the Department and collecting information on outcomes in support of future S&T policy development and initiatives due to lack of resource allocation planning.
There is also a risk that current program design and planning practices could affect the delivery of future OERD programs due to a lack of proper alignment between planning and program design and may result in continued lapsed funding.
RECOMMENDATIONS
2. The Director General - Office of Energy Research and Development (OERD) should ensure proper planning is undertaken to determine the resource required to fulfil the long-term program requirements extending from 2016 to 2021.
3. The Director General - Office of Energy Research and Development (OERD) should ensure that initial program design and planning for future programs take into consideration time required to deliver and achieve planned program activities and milestones.
MANAGEMENT RESPONSE, ACTION PLAN AND TIME FRAME
2. Management agrees.
OERD will ensure that resourcing requirements for the monitoring and reporting period are included as part of its annual business planning process.
For any future C-base programs, funding proposals will include a request for an additional two years of operating funding to ensure responsible management and wrap-up of the program, recognizing that the decision to approve such funding is made through the federal Budget process.
Position responsible: Director General - Office of Energy Research and Development
Timing: Annual business planning process. Proposal to central agencies to be included in a future program design (timing to be determined).
3. Management agrees.
OERD has already implemented this recommendation for the new Oil Spill Response Science (OSRS) Program. The OSRS Program design does not include vote 10 funding in year 1, in order to allow for sufficient time to deliver and achieve program activities and milestones. OERD will recommend that the above approach be used for any new future program design.
Position responsible: Director General - Office of Energy Research and Development
Timing: The above has already been incorporated into the program design (February 2014) for the OSRS Program.
APPENDIX A – AUDIT CRITERIA
The audit criteria were derived from widely recognized control models (e.g. Management Accountability Framework, CICA Criteria of Control – CoCo) and relevant policies, acts and legislation. Actual performance will be assessed against the audit criteria resulting in either a positive finding or the identification of an area of improvement.
The objective of the audit is to provide reasonable assurance on the effectiveness and efficiency of the program’s management control framework and on compliance with Treasury Board approved terms and conditions as well as policies and directives.
|
Sub-Objectives |
Audit Criteria |
|---|---|
|
Audit Sub-Objective 1: To determine whether effective and efficient governance processes are in place for the ecoENERGY Innovation Initiative.
|
1.1 Management has implemented effective and efficient Governance processes to enable the achievement of program objectives and priorities. |
|
1.2 A human resources (HR) plan is developed to address capacity and meet the requirements of the program. |
|
|
1.3 Roles, responsibilities and accountabilities are clearly defined, documented, and understood by all stakeholders. |
|
|
Audit Sub-Objective 2: To determine whether the design and delivery of ecoENERGY Innovation Initiative is in accordance with approved terms and conditions and compliance with the Policy on Transfer payments and the Financial Administration Act (FAA). |
2.1 Solicitation process and recipient application is conducted in a manner that is fair and accessible. |
|
2.2 Control processes are in place to ensure the consistent assessment and approval of projects to meet the objectives of the program. |
|
|
2.3 Funding agreements are consistently developed through a process that is tailored to address risk, recipient requirements, and meet TB and departmental guidelines and requirements. |
|
|
2.4 Grants and contribution over $25,000 are disclosed in accordance with proactive disclosure requirements. |
|
|
2.5 Agreements are administered in accordance with approved terms and conditions and in compliance with the requirements of the Policy on Transfer payments and the Financial Administration Act (FAA) related to entering into agreements, including amendments, and payments to recipients. |
|
|
2.6 The program has implemented departmental service standards for transfer payment programs. |
|
|
Audit Sub-Objective 3: To determine whether program planning, monitoring, and reporting, activities are in place and are operating effectively. |
3.1 Effective planning, monitoring, reporting processes are in place and are performed proportionately to their risk level. |
|
3.2 There are performance measurement processes in place to support program review and the departmental performance measurement strategy. |
Page details
- Date modified: