Audit of the Program of Energy Research and Development (AU1419)

TABLE OF CONTENTS

EXECUTIVE SUMMARY

INTRODUCTION

As part of its mandate, Natural Resources Canada (NRCan) is responsible for conducting research and developing federal resource policies, science, and technologies to support the sustainable development and competitiveness of the energy sector. The Program of Energy Research and Development (PERD or the Program) is a federal interdepartmental program, which was created as a result of the 1970s oil crisis, to carry out energy research and development, in order to increase the security of Canada's energy supply. In the 1996 Program Review, Ministers decided to continue PERD as an ongoing interdepartmental energy research and development (R&D) program, and centralized PERD resources within NRCan’s A-base. In support of this decision, a governance model was put in place that includes an interdepartmental ADM Panel, as the vehicle to advise on strategic direction and on program-related issues such as funding allocations and reductions, and to ensure that PERD remained an interdepartmental program. PERD resources were set at that time at $58M annually. Since then, PERD has supported a wide range of research and development activities in oil and gas, clean electricity, and energy end use, such as energy-efficient buildings, industry, and communities.   

The Program is designed to promote a sustainable energy future for Canada through its funded activities, both within NRCan as well as within other federal departments and agencies. As an interdepartmental program, PERD is expected to foster and develop synergies among departments and agencies whose mandates include energy technology issues.

PERD is managed by the Office of Energy Research and Development (OERD), in the Innovation and Energy Technology Sector (IETS) of NRCan. The Program has always been administered by OERD, which was originally created as the Secretariat for PERD. Historically, OERD was part of the Energy Sector at NRCan. Effective May 27, 2013, OERD and all programs it managed was moved to IETS within NRCan.

Over time, PERD resources have been reduced in various budgetary review exercises (e.g. strategic reviews). In Fiscal Year (FY) 2013-14, PERD expenditures totalled $43M, consisting of approximately $39M in Vote 1 (Operating Expenditures) funding for R&D to federal researchers and laboratories in up to 13 federal departments and agencies, and $4M primarily for program management, including a portion of that towards NRCan’s Intellectual Property Office and Library. Following a competitive process for project funding, the majority (approx. 51%) of PERD funding is allocated to three laboratories within NRCan’s IETS, which conduct a significant amount of energy R&D relevant to the Program. Main estimates for FY 2014-15 include a reduction in Program funding, resulting in total estimated expenditures of approximately $40M. Historically, a significant amount (approx. 73%) of PERD funding has been allocated to projects within NRCan, mostly allocated within IETS. The vast majority of PERD funds are allocated to projects via departmental transfer agreements with IETS and other NRCan laboratories, while Memorandum of Understanding (MOU) agreements are used for projects with other government departments (OGDs).

The objective of the audit was to provide reasonable assurance on the overall adequacy of the management controls supporting PERD operations, including compliance with the Program’s authorities.

STRENGTHS

Overall, the audit found that governance processes are established for PERD; and that roles, responsibilities, and accountabilities were generally defined, documented, and communicated to funding proponents. Management exercises oversight to enable the achievement of PERD objectives and priorities through various interdepartmental committees and strategic planning for each of its three technology area portfolios. In addition, the Program has established diverse planning, monitoring, and reporting activities, including proposal assessments and annual project status reporting.

AREAS FOR IMPROVEMENT

The audit identified opportunities for PERD to review the adequacy and efficiency of its management framework and funding structure to better reflect the limited risks related to departmental and interdepartmental transfers, which could increase efficiency by reducing the reporting burden on funding recipients; and mitigate any perceived conflicts of interest. Opportunities also exist to improve information management and performance measurement. 

INTERNAL AUDIT CONCLUSION AND OPINION

In my opinion, overall, PERD has governance and monitoring mechanisms in place to provide oversight on the achievement of program objectives. Opportunities exist to review the adequacy and efficiency of the Program’s management control framework and funding structure to better reflect risks related to departmental and interdepartmental transfers; and to improve information management and performance measurement.

STATEMENT OF CONFORMANCE

In my professional judgement as Chief Audit Executive, the audit conforms with the Internal Auditing Standards for the Government of Canada, as supported by the results of the Quality Assurance and Improvement Program.

Christian Asselin, CPA, CA, CMA, CFE
Chief Audit Executive
March 10, 2016

ACKNOWLEDGEMENTS

The audit team would like to thank those individuals who contributed to this project and particularly employees who provided insights and comments as part of this audit.

INTRODUCTION

As part of its mandate, Natural Resources Canada (NRCan) is responsible for conducting research and developing federal resource policies, science, and technologies to support the sustainable development and competitiveness of the energy sector. The Program of Energy Research and Development (PERD or the Program) is a federal interdepartmental program, which was created as a result of the 1970s oil crisis, to carry out energy research and development, in order to increase the security of Canada's energy supply. In the 1996 Program Review, Ministers decided to continue PERD as an ongoing interdepartmental energy research and development (R&D) program, and centralized PERD resources within NRCan’s A-base. In support of this decision, a governance model was put in place that includes an interdepartmental ADM Panel, as the vehicle to advise on strategic direction and on program-related issues such as funding allocations and reductions, and to ensure that PERD remained an interdepartmental program. PERD resources were set at that time at $58M annually. Since then, PERD has supported a wide range of research and development activities in oil and gas, clean electricity, and energy end use, such as energy-efficient buildings, industry, and communities.   

The Program is designed to promote a sustainable energy future for Canada through its funded activities, both within NRCan as well as within other federal departments and agencies. As an interdepartmental program, PERD is expected to foster and develop synergies among departments and agencies whose mandates include energy technology issues.

PERD is managed by the Office of Energy Research and Development (OERD), in the Innovation and Energy Technology Sector (IETS) of NRCan. The Program has always been administered by OERD, which was originally created as the Secretariat for PERD. Historically, OERD was part of the Energy Sector at NRCan. Effective May 27, 2013, OERD and all programs it managed was moved to IETS within NRCan.

Over time, PERD resources have been reduced in various budgetary review exercises (e.g. strategic reviews). In Fiscal Year (FY) 2013-14, PERD expenditures totalled $43M, consisting of approximately $39M in Vote 1 (Operating Expenditures) funding for R&D to federal researchers and laboratories in up to 13 federal departments and agencies, and $4M primarily for program management, including a portion of that towards NRCan’s Intellectual Property Office and Library. The majority (approx. 51%) of PERD funding is allocated to three laboratories within NRCan’s IETS, which conduct a significant amount of energy R&D relevant to the Program. Main estimates for FY 2014-15 include a reduction in Program funding, resulting in total estimated expenditures of approximately $40M. Historically, a significant amount (approx. 73%) of PERD funding has been allocated to projects within NRCan, mostly allocated within IETS. The vast majority of PERD funds are allocated to projects via departmental transfer agreements with IETS and other NRCan laboratories, while Memorandum of Understanding (MOU) agreements are used for projects with other government departments (OGDs).

PERD provides funding to federal departments and agencies, and not directly to parties outside of the federal government. Following a competitive process for project funding, PERD funding is allocated to IETS laboratories at NRCan and other Sectors within the Department as well as Other Government Departments (OGDs) such as Environment & Climate Change Canada (ECCC), the National Research Council (NRC), Innovation Science & Economic Development Canada (ISED), Agriculture & Agri-Foods Canada (AAFC), and Fisheries and Oceans Canada (DFO). Participating departments and agencies may collaborate with private sector organizations, universities, research organizations, provinces, and municipalities. The ultimate beneficiaries of PERD are both internal (federal government) and external. Internal beneficiaries include policy and regulation makers, codes and standards developers and implementers as well as federal scientists. External beneficiaries are from the Canadian energy technology industries, including utilities, spanning production (e.g. energy producers), distribution (e.g., pipelines industry), and use (e.g., the built environment, industry and transportation sectors). As an example of the benefits derived from PERD funding, ECCC has used this funding to generate knowledge on energy efficiency and emissions of electric vehicles in a Canadian context.

The Audit of the Program of Energy Research and Development was included in the Department’s Risk-Based Audit Plan, approved by the Deputy Minister on February 27, 2014.

AUDIT PURPOSE AND OBJECTIVES

The objective of the audit was to provide reasonable assurance on the overall adequacy of the management controls supporting PERD operations, including compliance with the Program’s authorities.

Specifically, the audit assessed whether:

• Adequate governance processes are established for PERD;
• The design and delivery of PERD is appropriate and in accordance with the Program’s authorities and compliance with the Policy on Transfer Payments and the Financial Administration Act (FAA); and
• Program planning, monitoring, and reporting activities are established and operating as intended to inform senior management decision making.

AUDIT CONSIDERATIONS

A risk-based approach was used in establishing the objectives, scope, and approach for this audit engagement. A summary of the underlying risks that were identified during the Planning Phase of the audit is included below.

There is a risk that:

• The PERD governance structures, roles, responsibilities, and accountabilities may not effectively support the Program;
• The PERD funding model may not be an appropriate model for the achievement of program objective and priorities;
• Processes and related control frameworks may not be in place to ensure the effective management of Program activities;
• Perception of lack of fairness and transparency in the administration of the Program could have a negative impact on the Program; and
• Information necessary for internal decision making and reporting may not be available or provided to NRCan management in a timely manner.

SCOPE

The audit focussed on relevant activities of PERD, including the governance and funding structures; project management and monitoring processes; and oversight activities.

Although the audit considered PERD’s allocations of funds over the past 10 years, the primary focus was on the period from May 2013 to June 2015, in order to reflect the organizational changes that occurred within OERD during that period.

APPROACH AND METHODOLOGY

The audit was conducted in accordance with the Treasury Board Policy on Internal Audit and Government of Canada Internal Audit Standards and entailed: 

• Interviews with key personnel (key stakeholders including PERD program management, process owners, and funding recipients) with respect to the management of PERD and related activities;
• Review of key documents including the Program authority documents, funding agreements, Memoranda of Understanding (MOUs), policies, directives, and guidance;
• Detailed examination of a select sample of funding agreements along with related project governance and management processes; and
• Analysis of budgets and review of the funding models used by the Program.

The conduct phase of this audit was substantially completed in December 2015.

CRITERIA

Please refer to Appendix A for the detailed audit criteria. The criteria guided the audit fieldwork and formed the basis for the overall audit conclusion.

FINDINGS AND RECOMMENDATIONS

Program Management

Summary Finding

Overall, governance processes are established for PERD. Management exercises oversight to enable the achievement of PERD objectives and priorities through various interdepartmental committees and strategic planning. Opportunities exist to review the adequacy of the Program’s management control framework and related processes to better reflect risks related to departmental and interdepartmental transfers; strengthen information management; and further improve performance measurement.  

Supporting Observations

Unlike a traditional Grant and Contribution program, PERD is entirely funded through A-base funding. A-base funds are provided to departments through Parliamentary Votes to support ongoing programs, in contrast with Grants and Contributions (C-base) funds, which are used to accomplish specific tasks over a specified period of time. PERD is a research and development program designed to support the development of new energy technologies as well as the creation of knowledge to support codes, standards, and regulations. PERD’s A-base funds are allocated through a competitive process to federal government departments and agencies to carry out projects intended to achieve Program objectives and priorities. Activities and outputs within PERD are intended to lead to advancing the development and uptake of new energy technologies, with the objectives of reducing the environmental impact of energy production, distribution, and use, and of supporting competitiveness.

In traditional federal Grants and Contributions programs to external entities, such as academia and industry, funds are transferred by a department to an external entity through an agreement for specified uses, and the responsible department must ensure that funds are used accordingly. As such, these programs normally require regular monitoring, a high level of oversight, and detailed expenditure information to ensure that public funds are used in accordance with the contribution agreement. Alternatively, when funding is transferred internally to NRCan laboratories and OGDs, as is the case for PERD, compensatory controls are in place to ensure that federal departments and agencies use resources in due regards to public funds. For instance, all public servants, as trustees of public resources, have a responsibility through their employment to deliver federal programs and services with probity. In addition, both NRCan and OGDs are subject to periodic examinations from the Office of the Comptroller General and Office of the Auditor General.

As such, the audit expected PERD to have established planning, monitoring, and reporting processes reflecting the limited risks associated with departmental and interdepartmental transfers and aligned with Program objectives. The audit examined these processes with respect to the governance and administration of funding agreements; performance measurement; and services standards. Management’s perspective is that the majority of the risks involved in PERD projects are not financial, but rather related to uncertainties inherent in R&D projects such as the timeliness of research projects or need for additional resources, or that the technologies developed may not result in the expected improvements (to cost and performance) within the expected timeframes. These risks can affect project timelines, budgets, and the achievement of expected outcomes. It should be noted that management has established planning, monitoring, and reporting processes to reflect the risks they have identified for R&D. The audit also found opportunities to simplify planning, monitoring, and reporting processes to better reflect risks related to departmental and interdepartmental transfers, since most PERD transfers are made to NRCan and OGDs.

Governance and Administration of Funding Agreements

Effective governance processes allow for management to exercise oversight and enable the achievement of program objectives and priorities. The audit found that PERD established governance processes through various committees and strategic planning. This included defining, documenting, and communicating roles, responsibilities, and accountabilities to the Program proponents; and establishing various planning, monitoring, and reporting activities.

The audit also found that PERD has control processes in place for the solicitation, assessment, and approval of project proposals, and the development of funding agreements. Many of the processes and controls currently in place, such as solicited proposals being subject to a challenge function involving OGD stakeholders and departmental staff as well as detailed progress achievement reports requested at the activity level for all projects, are similar to competitive funding competitions used for programs involving transfers of funding to external parties. Competitive programs with funding to external parties would normally include additional administrative processes, documentation, and review, while the PERD process requires detailed project information through proposals and reporting on project activities, budgets, and expected outcomes. Given the relatively limited risks involved with transferring funds to NRCan and to OGDs, there may be an opportunity to further reduce PERD processes and information requirements, to better reflect the risks related to departmental and interdepartmental transfers.

With regards to the administration of PERD agreements, the audit found that the sample of 12 funding agreements examined, in the form of Memoranda of Understanding (MOUs) and departmental transfer agreements, were generally administered in compliance with the Program Authorities. It is worth noting that the TB Policy on Transfer Payments does not apply to the vast majority of PERD’s funding agreements, since they do not include Vote 10 (Grants and Contributions) funding. The audit also observed that the documentation of the decisions on file was not always clear and that information necessary for internal management decision making was not always retrievable in a timely manner.

Performance Measurement

Effective performance measurement is also a key component of program delivery that provides the necessary information to enable informed decision making by senior management and assists in maintaining sound stewardship of program funds and activities. It is currently a Department-wide challenge to strengthen this area with targeted performance measurement tools. The audit sought to determine whether performance measurement processes are in place to support PERD. The audit observed that PERD has a draft Performance Management Framework and that performance information was being collected in project status reports. The draft Framework includes program-level and project-level activities, outputs, and outcomes, but presents a Performance Measurement Strategy that applies to the aggregate suite of projects, not to individual projects. Program Plans includes project-level indicators; however, these were not well-referenced in the overall Framework. In addition, based on a review of Project Status Reports and Project Completion Reports, the project milestones set in proponents’ proposals were not always specific or measurable. The audit also observed that the overall Program outputs presented in the Framework did not include the expected results from the Program as a whole.

Service Standards

The audit expected PERD to have established clear internal service standards, such as specific timeframes for when federal government proponents could expect to receive funds transfers, at the beginning of a fiscal year (i.e. starting on April 1^st of each year). The audit noted; however, that the Program has not established service standards for payments to OGDs and federal laboratories. Nine out of the 12 projects tested in the audit scope period received their funding more than two months after April first. In order to avoid potential project delays, the Program should strive to transfer funds within a reasonable timeframe, following the receipt of funds following the beginning of the fiscal year.

The audit also observed that during the audit scope period, NRCan subsequently received approximately $1.6M in returns of funds from proponents who were unable to use the transferred amounts for their projects within the budgeted timeframes. Out of the $1.6M in funds returned late in the year, $886k was from NRCan Sectors. Under the federal budget management process, funding amounts that are committed for a specific purpose will lapse (i.e. expire) if they are unused and not transferred to another activity within the same Fiscal Year; therefore, lapsed amounts cannot be used in a future Fiscal Year towards other PERD projects. It is worth noting that, in 2014-15, the Program succeeded in reallocating 100% of the returned funds to additional activities within IETS, prior to the year end. It is also worth noting that the Program made an improvement to this process in 2014-2015, by proactively sending call letters to proponents requesting information on spending plans, in order to better plan any year end reallocation process. Nevertheless, the Program should review the year end returns and reallocation process, to minimize the risk of lapsing funds.

RISK AND IMPACT

If the varying risks related to funding recipients are not considered by PERD program management, processes will not be effectively simplified or tailored to address risks. By not examining ways in which to strengthen information management, performance measurement, and service standards, this may impact management decision making and the ability of the Program to achieve its intended outcomes.

RECOMMENDATIONS

1. It is recommended that the Director General Office of Energy Research and Development (OERD) review PERD’s management framework to ensure that the processes currently in place reflect the relatively low risk levels related to transferring funds within the department and through interdepartmental transfers. In order to do so, the DG OERD should consider simplifying processes and reducing the reporting burden.
2. It is recommended that the DG OERD review processes supporting the administration of the Program of Energy Research and Development (PERD) projects to strengthen and simplify program administration and performance measurement.

MANAGEMENT RESPONSE AND ACTION PLAN

Management agrees.

In response to recommendation 1, the DG of OERD will review PERD’s management framework to ensure that processes reflect the program risk level. Building on current efforts, we will work collaboratively with stakeholders to review and continue to streamline:

• The proposal solicitation, application, assessment, and approval processes; MOU and departmental transfer agreements; and
• Reporting processes, while ensuring that program and project outcomes, and departmental reporting requirements are met.

Timing: The above activities are planned to be completed by March 31, 2017

Management agrees.

In response to recommendation 2, the DG OERD will review the processes supporting the administration of PERD projects, including:

• Using GCDOCS to consistently manage all program information;
• Establishing a service standard for the timing of funds transfers to recipients;
• Finalizing the updated PERD Performance Management Framework, and requiring proponents to write specific, measurable, achievable, reliable, and timely (SMART) performance indicators; and
• Documenting and implementing mechanisms to reallocate returned project funding, in an efficient and effective manner.

Timing: The above activities are planned to be completed by March 31, 2017

Funding Structure

Summary Finding

Under the current PERD funding model, a significant portion of the overall funding for the CanmetENERGY laboratories is dependent on the results of competitive calls for proposals from PERD. Over half of annual PERD funding is allocated to these laboratories. Opportunities exist to review the current PERD funding model to assist in alleviating CanmetENERGY laboratories’ funding uncertainties.

Supporting Observations

The audit sought to determine whether the PERD funding model is appropriate for the achievement of the Program objectives and priorities. In order to do so, the audit team reviewed the PERD governance structure; conducted interviews with selected PERD proponents including OGDs; and reviewed the historical allocation of PERD funds over the past 10 years.

Governance and Funding Allocations

PERD is an interdepartmental program with an interdepartmental competitive process in place for the selection of funded projects. PERD funding decisions involve four different governance committee levels. For each of the Program’s three portfolios (Fossil Fuels, Renewable and Clean Electricity, and End Use), there are related Technology Area Working Groups (TAWGs), chaired at the Director-level, that review project proposals, advise on the selection of projects, and report to the Portfolio Committees. The three Portfolio Committees, chaired by the Directors’ General (DGs) of the CanmetENERGY laboratories, with membership from departments involved in the R&D in each technology area and external stakeholders,  develop their portfolio strategic plans, recommend for approval the project proposals submitted by the TAWGs, and provide advice to the Interdepartmental DG Committee. The Interdepartmental DG Committee, overseeing the three Portfolio Committees, is chaired by the DG, OERD and includes DG representation from the three CanmetENERGY laboratories, NRCan’s other Sectors, and Program-participating OGDs as members. This Committee approves a suite of project proposals based on recommendations by the Portfolio Committees and advises the ADM Panel. The DG Interdepartmental Committee also advises the ADM Panel on the portfolio strategic plans. Finally, the ADM Panel, chaired by the ADM of IETS, and with ADMs from NRC, ECCC, ISED, and AAFC as members, provides high-level strategic advice and final approval of portfolio strategic plans.

The audit noted that the ADM Panel is chaired by the ADM responsible for the CanmetENERGY laboratories. The audit also noted that the CanmetENERGY laboratories’ DGs chair the Portfolio Committees that influence ultimate project funding decisions. Based on interviews with selected OGDs that receive PERD funding, this committee structure could lead to a perceived conflict of interest (COI) by funding proponents; however, since all proponents receiving PERD funding have an opportunity to participate in project governance and funding decisions, the audit concludes that the COI risk is inherently contained and the impact is limited. 

According to PERD program management, a significant amount of the federal government’s capacity for carrying out PERD-related projects lies within the CanmetENERGY laboratories, which aligns with the historical allocation of PERD funding to the laboratories, following a competitive process. On average, 70% or more of PERD funds were allocated to NRCan projects from FY 2004-2005 to FY 2013-2014, and 51% of PERD funds were allocated to the CanmetENERGY laboratories.

The audit also reviewed an assessment of the sources of funding for the CanmetENERGY laboratories salaries for FY 2013-2014. This analysis demonstrated that the CanmetENERGY laboratories rely heavily on sources such as PERD funding, a source of funding for which they must compete and on which they cannot rely on for long-term sustainability, to pay most of their salaries. According to an analysis recently conducted by program management, PERD funding currently represents, on average, 33% of the overall laboratories’ funding; other competitive funding represents 22% of their funding; and A-base funding represents less than one quarter of total laboratories’ funding. It is worth noting that there are other NRCan laboratories that are primarily A-base funded, and this funding ensures stability for laboratories by annually allocating them the funds required to pay employee salaries and fixed laboratory costs. The suitability and relevance of their projects may still be reviewed by senior management and potentially be subjected to a competitive funding process, but their ability to operate is sustained with a permanent funding source.

RISK AND IMPACT

Adjusting the PERD funding model could help alleviate CanmetENERGY laboratories’ funding uncertainties, which could strengthen their capacity to carry out PERD-funded projects and reduce program delivery risk.

RECOMMENDATION

3. It is recommended that the Assistant Deputy Minister Innovation and Energy Technology Sector (ADM IETS) review the funding model for the Program of Energy Research and Development (PERD) in order to assist in addressing funding uncertainties for CanmetENERGY laboratories.

MANAGEMENT RESPONSE AND ACTION PLAN

Management agrees.

In response to recommendation 3, the ADM IETS will review the PERD funding model, to help address funding uncertainties for CanmetENERGY laboratories, within the context of program renewal.

Timing: The above activities are planned to be completed by March 31, 2017

APPENDIX A – AUDIT CRITERIA

The audit criteria were derived from widely recognized control models (e.g. Management Accountability Framework, CICA Criteria of Control – CoCo) and relevant policies, acts, and legislation. Actual performance was assessed against the audit criteria resulting in either a positive finding or the identification of an area of improvement.

The objective of the audit was to provide reasonable assurance on the overall adequacy of the management controls supporting PERD operations, including compliance with the Program’s authorities.

Audit Sub-Objectives	Audit Criteria
Audit Sub-Objective 1: To determine whether adequate governance processes are established for the PERD.	1.1 Management exercises oversight to enable the achievement of Program objectives and priorities.
	1.2 Roles, responsibilities, and accountabilities are clearly defined, documented and communicated to the Program proponents.
Audit Sub-Objective 2: To determine whether the design and delivery of the PERD is appropriate, efficient, in accordance with the Program’s authorities, and in compliance with the Policy on Transfer Payments and the Financial Administration Act (FAA).	2.1 The PERD funding model is an appropriate model for the achievement of program objectives and priorities.
	2.2 Control processes are in place to ensure consistent solicitation, assessment, and approval of project applications in an efficient manner to meet program objectives and priorities.
	2.3 Funding agreements are consistently developed through a process tailored to address risks, recipient requirements, and to meet TB and departmental guidelines and requirements.
	2.4 Agreements are administered in accordance with approved Program authorities, and in compliance with the requirements of the TB Policy on Transfer Payments and the Financial Administration Act (FAA) related to entering into agreements, including amendments, and payments to recipients.
Audit Sub-Objective 3: To determine whether program planning, monitoring, and reporting activities are established and operating as intended to inform senior management decision making.	3.1 Planning, monitoring, and reporting processes are in place and are performed proportionately to their risk levels.
	3.2 Information necessary for internal decision making and reporting is available or provided to NRCan management in a timely manner.
	3.3 Performance measurement processes are in place to support program review and the departmental performance measurement strategy.
	3.4 Service standards have been implemented by Program Management.