Evaluation Report: Evaluation of the Office of Energy Efficiency

Table of Contents

• Executive Summary
• Recommendations and Management Response
• 1.0 Introduction and Background
  • 1.1 Introduction
  • 1.2 Overview of the Energy Efficiency Sub-Program
  • 1.3 Program Resources
• 2.0 Evaluation Objectives and Methodology
  • 2.1 Evaluation Objectives
  • 2.2 Evaluation Methodology and Approach
  • 2.3 Evaluation Limitations and Mitigation Strategies
• 3.0 Key Relevance Findings
  • 3.1 Ongoing Need for Energy Efficiency Programs in Canada
  • 3.2 Appropriateness of NRCan Role
  • 3.3 Alignment with NRCan and Federal Priorities
• 4.0 Key Results Findings
  • 4.1 Program Theory and Logic Model Outcomes
  • 4.2 Training and Information (Awareness and Capacity Outcomes):
  • 4.3 Labelling Programs (Awareness and Capacity Outcomes):
  • 4.4 Codes, Regulations & Standards (Adoption of Regulations Outcome)
  • 4.5 Energy Management, Monitoring, Rating Tools (Awareness & Capacity Outcomes)
  • 4.6 Final Outcome: Energy savings and associated reduction in GHG emissions and economic benefits
  • 4.7 Economy and Efficiency
• 5.0 Conclusions and Recommendations
• Appendix A: Description of Evaluation Case Studies
• Appendix B: Survey Populations and Response Rates

Acknowledgements

The Evaluation Team would like to thank those who contributed to the OEE Energy Efficiency Evaluation, particularly members of the Evaluation Advisory Committee, Energy Sector, as well as others who provided insights and comments crucial to this evaluation.

The evaluation project was managed by Nicholas Kowbel, with evaluation support from Carol Gudz, Odilia Maessen, and Zelda Yule of the Strategic Evaluation Division. Jennifer Hollington, Head of Evaluation at NRCan, and Gavin Lemieux, Director, provided senior management oversight. Science-Metrix also provided evaluation services for the project.

Executive Summary

Introduction:

This report presents the findings of the evaluation of Natural Resources Canada’s (NRCan’s) Energy Efficiency Programs (Sub-Program 2.1.4 in NRCan’s 2013-14 Program Alignment Architecture). The programs are administered by NRCan’s Office of Energy Efficiency (OEE), part of the Energy Sector. These programs were last evaluated in 2009-10. The evaluation covers program activities from 2009-10 to 2013-14, comprising $1.1 billion in NRCan funding.^{Footnote 1}

The objective of the Sub-Program is to increase energy efficiency, resulting in energy savings and reduced greenhouse gas emissions across targeted sectors of the Canadian economy (housing, buildings, vehicles, equipment, and industry). To do this, the Sub-Program has activities that target the energy using behaviour of consumers and businesses.

Methodology:

Data collection was from April to August 2014, and included 15 case studies, 100 interviews with stakeholders and program participants, a document review, and 8 surveys administered by OEE with guidance from Strategic Evaluation Division. Given that the OEE has a strong performance measurement system, and generally positive past evaluation results from 2009-10, the evaluation targeted efforts on newer activities. The evaluation took a theory-based approach, studying whether or not key program activities had an impact on achieving energy savings. While this methodology had several limitations (discussed in the report), a multiple-lines-of-evidence approach allowed evaluators to mitigate this by triangulating findings.

Relevance Findings:

There is an ongoing need to improve energy efficiency in Canada for environmental and economic competitiveness reasons. According to the International Energy Agency (2013) Canada continues to have high energy consumption relative to the size of its gross domestic product due to a high number of energy-intensive industry, a cold climate, vast geography and a high standard of living amongst the general population. In each of the five Canadian economic sectors targeted by OEE programming, energy consumption is forecasted to increase through 2020. This makes the public and businesses vulnerable to energy price spikes, jeopardizing competitiveness and producing negative environmental consequences (e.g., increased greenhouse gas emissions associated with energy use).

Canadians and Canadian businesses are concerned about energy use but experience persistent barriers to taking action such as weak price signals and capacity deficits of knowledge, human and financial resources. OEE supports the development of housing and building codes and regulations of energy-using products, as well as voluntary labelling initiatives to counteract weak prices signals. OEE also deals with capacity issues by providing training and information on energy management products and practices.

NRCan is uniquely placed to provide national level leadership and to coordinate efforts within Canada. This includes alignment of codes and standards, as well as administering several U.S. Environmental Protection Agency (EPA) programs in Canada to improve energy efficiency. As the only federal department with a mandate for responsible energy use, NRCan also has the content expertise to provide a high degree of credibility to improve energy efficiency information and tools. Interviewees across sectors noted that no other group could develop these tools as effectively.

Facilitating improvements to energy efficiency is consistent with NRCan strategic outcomes and federal government commitments and policies related to environmental responsibility and economic competitiveness. This includes supporting commitments made under the Copenhagen Accord, the Canada-US Clean Energy Dialogue, and NRCan’s responsibilities for addressing energy-using products under the Energy Efficiency Act.

Performance Findings:

The OEE Energy Efficiency Sub-Program has made good progress towards its objectives of increasing the adoption of energy efficient products and practices by raising the awareness and capacity of target groups. Overall, evaluation evidence corroborates the expected link between OEE activities and energy savings. The Energy Efficiency Sub-Program has facilitated energy savings by linking target groups (e.g., home builders, consumers and homeowners, building and industrial facilities managers, fleet drivers and managers) to energy efficient products and practices. Target groups in all sectors report adoption of energy efficient products and practices due in part to OEE activities. Therefore, the Energy Efficiency Sub-Program is likely to meet its 2015-16 targets for energy savings (36 to 44 Petajoules), and associated reductions in greenhouse gas emissions (3.6 to 4.4 Megatonnes) and economic benefits (estimated by OEE to be approximately $1 billion in direct energy savings^{Footnote 2}) realized from improved energy efficiency.

Impacts of activities on adoption of energy efficient products and practices

OEE training and information products are generally reaching target groups across sectors. The evaluation found strong evidence of training having influenced energy using behaviours, including implementation of energy savings practices and learned techniques attributable to NRCan. For example, fleets that used Smart Driver for Highway Trucking (SDHT) reported 5-6% annual fuel savings in the case study, Dollars to $ense energy management workshop participants reported implementing energy savings practices, and industrial facilities involved in the Canadian Industry Program for Energy Conservation (CIPEC) reported significant energy savings attributable to NRCan. As well, OEE content provided online and in training sessions was seen to be of high quality. However, challenges experienced with the time it took to publish material and information online, and usability issues identified with NRCan websites suggest the reach of some content is likely limited. Significant technical challenges were also identified that affected managing online content (notably with Smart Driver for Highway Trucking fleet training). It was also widely suggested that Canadian Industrial Program for Energy Conservation (CIPEC) should do more to reach out beyond those currently taking advantage of the program.

Labelling programs have been effective at influencing adoption of energy efficient products and practices for consumers, and manufacturers of energy using products, vehicles and homes. Beyond this, one of the key values of NRCan’s labelling programs such as ENERGY STAR® for consumer products and the EnerGuide Rating System (ERS) for homes is their usefulness in supporting credible and unified infrastructure, which provides stakeholders the ability to deliver consistent programming and also provides a simple tool to Canadian consumers to choose more efficient products. For example, ENERGY STAR supports consumer rebate and information programs from 47 Canadian utilities, and 88% of Canadians think ENERGY STAR is helpful when making energy efficiency choices. In addition, as of February 2014, there were more than 39 provincial, territorial and municipal, utility and industry home labelling programs across Canada supported by the ERS infrastructure. These activities have helped to voluntarily move product manufacturers and home builders towards providing higher levels of efficiency in their offerings to consumers, providing sustained energy savings. Builders that participate in the premium home labelling programs (i.e., ENERGY STAR for New Homes, R-2000) report implementing new energy efficient building techniques learned from premium label programs across many of the homes they build. However, public understanding and motivation for reacting to labels appears to be an ongoing challenge, particularly for premium labels like the new ENERGY STAR Most Efficient designation, which is seen positively by ENERGY STAR participants, but has experienced low uptake so far. Builders, provinces and utilities identify a need to improve consumer demand for new home labels such as Energy Star for New Homes (ESNH).

Codes, regulations and standards have been produced in housing, buildings, and equipment sectors, with early signs of adoption by jurisdictional authorities. NRCan supported the National Research Council with funding, technical expertise, and policy consultations to develop and implement the 2011 National Energy Code of Canada for Buildings (NECB), and updates to the National Building Code, which have set new levels of minimum energy efficiency requirements for buildings and homes in Canada. Energy savings have also been realized through previously implemented equipment regulations developed under the Energy Efficiency Act (Minimum Energy Performance Standards - MEPS). Once implemented, codes and regulations provide a cost-effective means of encouraging energy savings, removing the less energy efficient options from the market. However, challenges were noted with timeliness and communication. For example, Canada has had difficulty meeting its forward regulatory agenda for equipment regulations, which placed Canada out of alignment with the United States. Industry representatives noted the importance of keeping pace with the U.S. regulatory agenda. This has also raised concerns about the level of communication between NRCan and affected manufacturers. As well, the production of supporting materials for the NECB is significantly behind schedule.

Energy Management, Monitoring and Rating Tools have successfully been enabled by OEE for use in Canada including ENERGY STAR Portfolio Manager (2013 launch), the SmartWay Transport Partnership Program (2012 launch) and support for the use of Energy Management Standards including ISO 50001. Early reported impacts of these tools are that they provide organizations with a means to think about and communicate energy use to senior managers and owners (e.g., the building benchmarking score provided by Portfolio Manager). There are some early signs of energy savings from companies that adopted ISO 50001 energy management standard (e.g., companies reported as high as 19.2% drop in energy use), and SmartWay (72% of registered trucking companies reported a change they attributed in part to the program). For longer standing programs such as the Federal Buildings Initiative (FBI) there are observed annual energy savings of $45.5 million. The measurement of energy use through these initiatives has helped organizations to communicate savings value to senior managers and to implement projects. That said, concerns were identified with NRCan’s ability to commit resources to continue to provide the benchmarking ratings for the Portfolio Manager tool; that barriers may be affecting some partners’ motivation or ability to join SmartWay (such as a lack of capacity to mine the data needed to populate the tools, uncertainty over the value proposition, and low registration rate of shippers); and with securing commitment to implement retrofits under the Federal Buildings Initiative.

Factors Affecting Results:

Program results were influenced by a range of factors. Across program activities, three factors appeared to be key to program successes. First, the degree to which other supporting interventions were available to reinforce desired energy efficient behaviours influenced program success. These supports either came from within OEE (e.g., the combination of regulations and labelling) or were part of a wider target group plan to improve energy efficiency (e.g., trucking companies that provided incentives to drivers). Second, OEE relationships and strong collaborations have been key aspects of many of their successes in bringing together stakeholders, and providing tools that they can use. Third, adoption of energy efficiency in private and public organizations has been heavily influenced by the availability of human and financial resources to implement energy efficient products and practices and senior management buy-in. In addition, for consumer product purchases, consumers appear influenced by considerations of cost and cost savings, while homebuyers appear to take a more holistic view, favouring home aesthetics and resale value (potentially due to the fast rising prices of the Canadian housing market).

Efficiency and Economy:

The OEE Energy Efficiency Sub-Program is an efficient means of achieving savings in energy use by enabling the work of others with tools, systems, and information and by leveraging partners’ access to target audience groups. Examples of program outputs that are used by others in their own initiatives, extending the reach of OEE, include the use of ENERGY STAR and the EnerGuide Rating System for homes by utilities, provinces and territories, municipalities, and manufacturers; Portfolio Manager by provinces, cities and industry associations; and SmartDriver for Highway Trucking (SDHT) and Auto$mart training material, which is being leveraged by companies and by provinces in their licensing processes.

OEE has also levered considerable resources from others to deliver initiatives in Canada by sharing costs, infrastructure and expertise. OEE shares the cost of delivering energy efficiency training workshops (D2$), developing and maintaining standardized energy efficiency test procedures for regulatory purposes, and maintaining the National Energy Use Database (NEUD). Additionally, OEE has levered significant infrastructure, tools and expertise from the US Environmental Protection Agency and Department of Energy to deliver their ENERGY STAR, Portfolio Manager and SmartWay Transport Partnership Programs in Canada.

OEE has also adapted its programming to the new fiscal constraints by curtailing travel and increasing use of communication technologies and leveraging relationships with industry associations. While this has had efficiency benefits, stakeholders across sectors expressed concerns over the degree to which NRCan remains visible to target sector audiences at committee meetings, and sector promotional events such as trade shows. There is a risk that NRCan could lose control of its messages, and that the reduced visibility could result in lost influence with partners and target groups, if outreach operations are not carefully managed.

Recommendations:

The evaluation also identified risks to continued success and opportunities to improve outcomes from OEE activities. These, and the associated recommendations, are outlined below:

1. Explain or enhance the value of some offerings: Energy Sector should address challenges identified with uptake and use of some OEE products and tools by improving awareness of and better explaining their value to target audiences. In some cases, Energy Sector should also enhance the value of certain tools and products.
   • The Evaluation has found that OEE is not effectively communicating the value of some of its offerings to target groups. This challenge is being observed through limited consumer demand for and understanding of home labelling programs such as ENERGY STAR for New Homes, and the recently introduced ENERGY STAR Most Efficient product label; perceived uncertainty of the value to join SmartWay for some carriers and shippers; the difficulty in securing federal commitments to undergo building retrofits; and the identified opportunities to extend the reach of the Canadian Industry Program for Energy Conservation.
2. Address timeliness issues: Energy Sector should respond to timeliness issues associated with regulations and code development raised by OEE external target audience groups to ensure full energy savings potential from regulatory activities is met.
   • The evaluation has identified challenges with timeliness and communication of timelines to stakeholders. This includes the challenges OEE has experienced with meeting its forward regulatory agenda for equipment regulations, which has placed Canada out of alignment with the United States; the time taken to finalize agreement on vehicle labelling in Canada; and concerns expressed by stakeholders over the lack of communication from NRCan on these delays. There have also been challenges with delivering collateral tools to support implementation of the National Energy Code for Buildings (NECB). Across sectors, there was a sense that materials used to communicate to stakeholders were frequently delayed.
3. Strengthen coordination: Energy Sector should strengthen coordination of OEE efforts within and across sectors by offering program target audiences access to a client focused system of energy efficiency tools, training, information, and partner programs wherever possible.
   • OEE has contributed best to behaviour change where activities were part of a system of complementary interventions, of which there are many examples. However, opportunities were identified across sectors to further strengthen impacts by more systematically connecting OEE activities across and within sectors, and with complementary provincial programs as well. For example, SmartWay trucking companies expressed interest in learning about NRCan’s FleetSmart driver training program, Portfolio Manager could be more explicitly linked to various OEE components that provide energy management solutions, and CIPEC participants suggested being connected through OEE to complementary provincial programs.
4. Understand and address IT risks: Energy Sector should work with the Corporate Management and Services Sector (CMSS) and Shared Services Canada (SSC) to more effectively use existing technology resources, identify technology risks for OEE and develop a strategy to address these risks to delivering programs online.
   • While OEE has developed a new e-learning web-based approach to its Smart Driver for Highway Trucking training program, ongoing technical issues associated with the e-learning application (SABA - IBM platform) and limited IT capacity within NRCan and SSC—including expertise, human resources, and bandwidth—have placed this initiative in jeopardy. As OEE investigates making greater use of online tools, this challenge has the potential to affect other areas of OEE activity. For example, CIPEC has been delivering online webinars, and D2$ interviewees suggested online workshop delivery for remote locations.

Recommendations and Management Response

Recommendations	Management Response	Responsible Official /Sector (Target Date)
1. Energy Sector should address challenges identified with uptake and use of some OEE products and tools by improving awareness of and better explaining their value to target audiences. In some cases, Energy Sector should also enhance the value of certain tools and products.	ACCEPTED The Energy Sector recognizes the benefits of improving uptake through greater awareness of the benefits of OEE’s energy efficiency programs. In order to do so, OEE is reframing its policy outcomes around the following 5 key areas. This frame will better reflect the value of products and tools. Help Canadian consumers and businesses save money; Build public confidence in energy development by improving environmental performance and reducing wasteful energy use; Improve industry’s productivity and competitiveness;  Lower energy costs in communities, including northern, remote and aboriginal; and, Support energy market diversification with the promotion of products, services and technologies.  Specifically, OEE will endeavour to develop a comprehensive stakeholder engagement strategy and communications plan. The plan will also leverage new technologies and collaboration tools, such as social media, working through business partners to pursue greater uptake of products, services, and technology, and coordinating with the provinces and territories. Examples include: Engage directly with major provincial utilities to increase uptake of most efficient ENERGY STAR products. Use targeted webinars tailored to industry, freight carriers and private and federal building owners on tools such as SmartWay and the Canadian Industry Program for Energy Conservation. Streamline and create an e-version of the home rating system homeowner package for new and existing homes and provide an enhanced package of information for homeowners using the tool. Emphasise the importance of saving consumers money and improving the competitiveness and productivity of industry, e.g.: The Energy and Mines Ministers’ Conference in August 2014, referenced a joint federal-provincial-territorial document illustrating private and public sector economic benefits of energy efficiency across Canada. Offer consumers easy access to information by using recognized labels and standards; new media (e.g., mobile apps), social media, new collaborations (e.g., with reward programs like Air Miles) to better explain the benefits of energy efficiency to target audiences. Increase the reach of the programs by strengthening existing partnerships (e.g., Canadian Energy Efficiency Alliance and provincial bodies such as Energy One in Nova Scotia) and creating new ones (e.g., the National Hockey League). Accelerate the adoption of products, technologies, and services that improve productivity through collaborations with energy intensive sectors (e.g., oil and gas, mining and forestry) to collect and disseminate information and best practices.	ES, OEE (March 2016)
2. Energy Sector should respond to timeliness issues associated with regulation and code development raised by OEE external target audience groups to ensure full energy savings potential from regulatory activities is met.	ACCEPTED The OEE concurs with this finding, and is undertaking measures to address this issue. Specifically, with respect to regulations, the OEE has worked to align with the U.S. through the Regulatory Cooperation Council (RCC) on energy performance standards for energy using products (e.g., appliances, furnaces, and home electronics).  This initiative was announced August 2014; an industry workshop was held in Oct. 2014; and a shared Canada - U.S. Workplan will be signed in 2015. By aligning regulatory amendments via the RCC, there is a need to follow a critical path. Secondly, the RCC increases the profile of energy efficiency standards and regulations for decision makers in Canada, helping them understand the benefits of alignment. With respect to code development, the Sector will communicate scheduled publication dates for supporting documents on the program webpage creating clear expectations for an update of the 2011 National Energy Code for Buildings.	ES, OEE (March 2016)
3. Energy Sector should strengthen coordination of OEE efforts within and across sectors by offering program target audiences access to a client-focused system of energy efficiency tools, training, information and partner programs wherever possible.	ACCEPTED The OEE concurs with this finding, and to address this issue, plans to undertake a comprehensive review of its organizational design model and processes, with a view to increase efficiency, effectiveness and coordination of program delivery with program partners as well as bolster strategic policy and communications capacity. The program will work to strengthen the coordination of efforts within and across sectors and improve the customer experience with its tools, e.g.: Industry, transport, building and equipment programs are combining efforts to seamlessly address more energy use in client organizations. Increasing access to partner programs, e.g., targeted roll out of the improved EnerGuide Rating System with partners such as the City of Edmonton, the City of Whitehorse, and Nova Scotia. Marketing training tools to a wide network of local utilities to maximize outreach.	ES, OEE (March 2016)
4. Energy Sector should work with the Corporate Management and Services Sector (CMSS) and Shared Services Canada (SSC) to more effectively use existing technology resources, identify technology risks for OEE, and develop a strategy to address these risks to delivering programs online.	ACCEPTED In order to ensure strong governance and clarity of roles, the OEE will undertake a review of the way in which it uses Information Technology (IT). The Energy Sector will identify an EX Champion in the OEE by March 2015, who will meet on a regular basis with the Corporate Management and Services Sector (CMSS) to develop and implement a business plan that: Identifies risks associated with the use of IT technologies. Outlines a strategy to address these risks to delivering programs online. Ensures the OEE effectively works with Shared Services Canada to develop and implement modern, responsive IT products.	ES, OEE CMSS (Identify EX Champion in the OEE by March 2015  with implementation of business plan ongoing)

1.0 Introduction and Background

1.1 Introduction

This report presents the findings of the evaluation of Natural Resources Canada’s (NRCan’s) Energy Efficiency Programs (Sub-Program 2.1.4 in NRCan’s 2013-14 Program Alignment Architecture). The programs are administered by NRCan’s Office of Energy Efficiency (OEE), part of the Energy Sector. These programs were last evaluated in 2009-10 as part of three separate evaluations.^{Footnote 3}

This evaluation covers program activities provided with NRCan’s direct program spending of $1.1 billion over the period from 2009-10 to 2013-14. The evaluation data collection was from April to August 2014, and included 15 case studies, 100 interviews with stakeholders and program participants, a document review, and 8 target audience surveys administered by OEE with guidance from the Strategic Evaluation Division (SED).

1.2 Overview of the Energy Efficiency Sub-Program

Within NRCan’s 2013-14 Program Alignment Architecture, this Energy Efficiency Sub-Program supports Departmental Strategic Outcome #2: Environmental Responsibility. The overall goal of the Energy Efficiency Sub-Program is to increase energy efficiency, resulting in energy savings and associated reductions in GHG emissions across targeted sectors of the Canadian economy.

This includes five program areas organized along five Canadian economic sectors and two discontinued programs dealing with retrofit incentives:

Housing ($78.8 million / 5 years):
This program encourages the construction and retrofit of low-rise residential housing, making the stock more energy efficient. They do this by developing and maintaining infrastructure for home energy labelling and standards (e.g., the EnerGuide Rating System), licensing service organizations, disseminating information to industry and home owners, and collaborating with provinces, territories and utilities to develop and implement codes and standards (e.g., supporting inclusion of energy efficiency in the National Building Code). In 2011, the program began the process of establishing the next generation of premium voluntary labelling standards for ENERGY STAR® for New Homes (ESNH) and R-2000.

Buildings ($47.4 million / 5 years):
This program provides information and benchmarking tools to building managers to improve energy performance of new and existing commercial and institutional buildings in Canada. It does this by supporting the National Research Council (NRC) in development and implementation of the National Energy Code of Canada for Buildings 2011 and 2015/2016; since 2011 adapting and starting in 2013 delivering the US Environmental Protection Agency (EPA) ENERGY STAR benchmarking tool in Canada; training (e.g., Dollars to $ense workshops); tools and information materials for use by stakeholders and partners including industry associations, utilities and building managers; and facilitating retrofit projects in buildings owned by the Federal government (Federal Buildings Initiative).

Vehicles ($49.1 million / 5 years):
This program provides both individual Canadians and Canada's commercial fleet sector with decision-making tools for buying and operating their vehicles to reduce fuel consumption. It also promotes vehicle efficiency by introducing improved vehicle fuel consumption labels, rating guides and a light-duty tire information system (timing to align with U.S. regulations). It does this through funding driver information delivery partners, working with provinces and territories to encourage fuel efficient training in licensing requirements, and working on vehicle labelling. New in 2011, the program is implementing the Canadian version of the U.S. EPA SmartWay Transport Partnership Program (connecting energy efficient shippers and carriers).

Equipment Standards and Labelling ($45.1 million / 5 years):
This program works to eliminate the worst energy performers and accelerate the introduction of more energy-efficient products in Canada’s equipment stock. It does this by working with federal and provincial partners to introduce and raise energy efficiency standards for a wide range of products through amendments to the minimum energy performance standards (MEPS) under the Energy Efficiency Act and promote energy-efficient products through the voluntary ENERGY STAR® initiative in Canada. New in 2011, the program implemented the U.S. EPA’s ENERGY STAR “Most Efficient Products” initiative in Canada.

Industry ($34.3 million / 5 years):
This program works to advance the energy efficiency of Canada’s industrial sector by accelerating energy-saving investments and the exchange of best practices information within Canada's industrial sector. The program does this by developing energy management training (e.g., Dollars to $ense workshops) and information through industry partnerships, and maintaining the Canadian Industry Program for Energy Conservation (CIPEC), an industrial network for sharing energy efficiency information and practices. New in 2011, NRCan is directly supporting the implementation of the ISO 50001 Energy Management Systems Standard in Canada, and has expanded CIPEC offerings to include webinars on energy efficiency practices.

Retrofit Incentive Programs (received minimal attention in this evaluation):
The following programs were evaluated extensively in the 2010 evaluation and, given that they have been discontinued, they were not examined in-depth during this evaluation:

Homes Retrofit Program ($821 million/3 years):
The ecoENERGY Retrofit - Homes program existed from 2007-08 to 2011-12 and was delivered by the Housing Division. This program provided incentives to eligible homeowners for implementing energy savings measures based on a home energy audit.

Retrofit for Small and Medium Organizations (Industrial and Buildings sectors) ($17.8 million/2 years):
The ecoENERGY Retrofit for Small and Medium Organizations Program existed from 2007-08 to 2010-11 and was delivered by Industry and Buildings Divisions. Expenditures during the evaluation period were $17.8 million for contribution agreements to cost-share energy retrofit projects, such as upgrades to building envelopes or equipment, based on the estimated savings of the organization’s energy use.

1.3 Program Resources

Part of the Clean Air Agenda (2011), the current ecoENERGY Efficiency funding is $195 million from 2011 to 2016, when the program expires. The rest of the funding, received for the period 2009-10 to 2011-12, was also provided through the earlier Clean Air Agenda.

Exhibit 1 presents the expenditures by type – Operating and Maintenance (O&M), Salary, and Grants and Contributions (G&C) – for the Energy Efficiency Sub-Program by sector during the evaluation period (2009-10 to 2013-14). This table does not include the in-kind or leveraged resources.

Exhibit 1: Energy Efficiency Sub-Program Financial Expenditures* by type and Program Area, 2009-10 to 2013-14 ($)

Expenditure type	Equipment	Housing	Buildings	Industry	Vehicles	TOTAL
O&M	16,344,322	34,709,794	19,021,040	12,834,156	19,571,302	102,480,615
Salary	23,226,961	35,368,822	26,077,768	18,750,563	21,720,941	125,145,055
G&C	5,543,493	829,689,782	11,830,466	10,885,005	7,765,261	865,714,007
Total	45,114,776	899,768,399	56,929,274	42,469,724	49,057,504	1,093,339,676

2.0 Evaluation Objectives and Methodology

2.1 Evaluation Objectives

The specific objectives of the evaluation were to:

• assess or confirm the relevance of the OEE Energy Efficiency Sub-Program in terms of addressing an actual need, in relation to federal government priorities, and in relation to the role of the federal government;
• assess the performance of the OEE Energy Efficiency Sub-Program in terms of the degree to which it has achieved expected results, and whether there have been any unintended results; and
• assess the economy and efficiency of OEE Energy Efficiency Sub-Program

This evaluation study was designed using a calibrated risk-based approach as per Treasury Board Guidelines “to prioritize units of evaluation based on risk”. The assessment considered the needs of the program and the knowledge gained from the previous round of evaluations in 2009-10.^{Footnote 4} The retrofit incentives programs only received a cursory review given that they were discontinued in 2011-12 and were evaluated extensively in 2010.

2.2 Evaluation Methodology and Approach

A theory-based approach was used to conduct the evaluation based on the logic model developed for the overall Sub-Program as part of its renewal in 2011. Fundamentally, OEE outcome indicators rely on calculating foregone energy use and associated GHG emission reductions as a result of program interventions. When the evaluation methodology was being developed, the program’s performance measurement system – Better Energy Efficiency Reporting System (BEERS) – reported that most outputs and activities were on track and the program was on track to achieve its targeted energy savings of 36 to 44 PJ and the associated 3.6 - 4.4 Mt of GHG emissions savings from reduced energy consumption by 2015-16.^{Footnote 5} These outcomes are based on assumptions about the level of uptake and impact of its energy efficiency programming derived from previous evaluations and literature. The evaluation, therefore assessed the key mechanisms used by the programs to realize energy savings and reduced GHG emissions - namely, 1) training and information; 2) labelling; 3) codes, regulations and standards; and 4) energy management, monitoring and rating tools.

The evaluation methodology was targeted to challenge/corroborate the performance data that OEE collects, and explore opportunities for enhancing program effectiveness. The evaluation team developed the methodology and the data collection instruments/tools in consultation with OEE, using an advisory committee with representation across program sectors for this evaluation.

The evaluation employed the following multiple lines of evidence:

Document and Performance Measurement Data Review
Evaluators reviewed OEE’s Performance Measurement data, key program documents and completed impact studies and surveys that covered the evaluation period. The findings from this exercise assisted SED in calibrating the data collection instruments so as to focus primary data collection on specific evaluation questions, program activities, and issues where there were information gaps or where findings needed to be further explained. This information was used to inform the final data collection tools, interview and case study selections and to corroborate evaluation results. As additional documents were collected through interviews, they were incorporated into the document review to ensure they were considered in the final evaluation analysis.

Interviews
Key informant interviews were conducted to respond to the majority of the evaluation questions and to complement the other methods. A total of 100 semi-structured interviews were conducted across the five program sectors to provide in-depth qualitative information on activities, context, relevance, progress towards outcomes, and economy and efficiency including delivery alternatives. This included 20 internal program interviewees, 72 external interviews with partners, users, and non-users of OEE programs, and 8 interviews with other organizations who are not directly involved in the programs but could speak to energy efficiency needs and views of their membership vis-à-vis OEE Energy Efficiency programs. All interviews were semi-structured, lasting approximately 1 hour.

Case Studies (Details in Appendix A)
Fifteen case studies were conducted, across the five OEE sectors. The case studies provided an in-depth review of individual initiatives to more fully understand what occurs from program delivery to achieving results for particular instances. Case studies were selected to assess key causal mechanisms in the program theory and to confirm theories of implementation and change (e.g., the effectiveness of regulation in a particular instance, or the effectiveness of training).

• Ongoing activities: Ten standard-effort case studies (i.e., 5 interviews (mostly external), review of documents and data provided by OEE and interviewees) focused on ongoing activities that are updates to key activities that are part of the program’s theory of change: providing training, information development, and regulations, codes and standards; and
• New Activities: Given the need to provide information on activities that have not been evaluated before (i.e., part of the 2011 mandate), five case studies were selected to have a higher level of effort (i.e., 10 interviews (mostly external), review of documents and data provided by OEE and interviewees) to assess the roll-out and implementation of new program activities identified in the 2011 mandate for the ecoENERGY Efficiency programs

Surveys (Table in Appendix B)
Stakeholder and client surveys provided quantitative data on the views and experiences of target audiences that have used OEE products and services, including impacts of programming on their behaviour. SED worked closely with OEE to augment their already planned studies to capture data for the evaluation. To preserve neutrality, SED had final approval on specific evaluation questions added for the evaluation and advised on data collection. Target groups included building and industrial facilities managers, home builders, homeowners, ENERGY STAR program participants, and trucking companies (carriers).

2.3 Evaluation Limitations and Mitigation Strategies

Although the evaluation has been designed to target multiple lines of evidence at key questions to enhance the reliability of results, there are several limitations and challenges to the evaluation methodology.

1. Public Opinion Research limitations: OEE has been unable to do a public opinion research survey of Canadians to determine the strength of awareness of the ENERGY STAR brand and its impact on decision-making since 2010. In lieu of direct data on consumers’ perceptions of ENERGY STAR, the evaluation approximated the impact on decision-making through perceptions of ENERGY STAR participants (though this is inherently less accurate). As well, results from a nationally representative 2013 Canadian Energy Efficiency Alliance (CEEA) survey of 1584 Canadians were used to provide a more direct consumer perspective where applicable.^{Footnote 6}
2. Interview sample identified by OEE: At SED’s request, OEE compiled and provided contact information for interviewees for the evaluation. While OEE is best placed to develop the list of interviewees, the potential for a selection bias exists when the evaluation subject is responsible for identifying contacts. To mitigate, interviews were the decision of the evaluation team (subject to willingness and availability of interviewees), and additional interviewees were added to the list as they were identified during fieldwork.
3. Attribution challenges: The multiple offerings related to energy efficiency information, incentives and training programs from provinces, utilities and industry associations in all sectors as well as the complex nature of economic factors and behaviour change makes it difficult to determine with precision the extent of attribution of federal inputs to expected outcomes. Similarly, low response rates to the eight OEE surveys (2% to 23%) reduced the generalizability of the data. In anticipation of this possibility, SED tailored general and case study interview guides to ensure such information was captured from key informants and corroborated with survey results to improve confidence.

3.0 Key Relevance Findings

This section assesses the extent to which there is an ongoing need for the programs, the appropriateness of NRCan’s role in filling that need, and if the programs respond to department and federal priorities.

3.1 Ongoing Need for Energy Efficiency Programs in Canada

Finding: Net energy use in Canada continues to rise due in part to the characteristics of the Canadian economy. As well, in each of the economic sectors targeted by OEE programming energy use is forecasted to increase. This puts Canadians and Canadian businesses at increased risk of price shocks, decreased competitiveness, and contributes to the accelerated production of greenhouse gas emissions

High energy consumption in Canada: A review of background information, recent program documentation, and past evaluations suggests that since the last evaluations in 2009-10 there has not been a significant change in the overall need for the Energy Efficiency programming. That is, there remains an acute need for this kind of programming in Canada. According to a 2013 report from the International Energy Agency, Canada’s energy consumption is high relative to the size of its gross domestic product compared to other countries. Factors driving this are significant amounts of energy-intensive industry, a cold climate, vast geography, a high standard of living and minimal constraints on space occupation.^{Footnote 7}

Energy consumption continues to trend upwards: Evidence indicates that energy use and GHG emissions are significant for all Canadian economic sectors targeted by the OEE programs and are forecasted to increase. Housing, building, industrial (each of which includes energy using equipment) and transportation sectors are major energy users in Canada and upward pressure on energy demand is expected to continue. For all sectors energy use has increased over the twenty year period,1990 to 2010, with the largest increases occurring in the industrial and transportation sectors (particularly the freight transportation sector).^{Footnote 8} The number of households in Canada is also expected to increase by 20% from 2005 to 2020, increasing future demand for energy in the residential sector.^{Footnote 9} Upward pressure on energy demand is also expected in the commercial/institutional sector as commercial floor space is forecasted to increase 39% between 2005 and 2020.^{Footnote 10} Several external and internal interviewees also explained that in the last five years, the number of appliances and consumer electronics plugged in to the average home has increased exponentially, thus raising demand despite increased energy efficiency.

Energy efficiency has mitigated the rate of energy consumption growth: Improvements in energy efficiency have however been shown to reduce the impact of increased energy consumption spurred by economic growth. A Report by the Energy Mines and Ministers’ Conference indicates that between 1990 and 2009, even while Canada’s population increased 22%, and Canada’s gross domestic product increased 57%, energy use went up just 23%. In other words, Canada is using much less energy per unit of GDP compared to two decades ago, due in part to gains in energy efficiency. Without improvements in energy efficiency, energy use in Canada would have increased by 46% between 1990 and 2009.^{Footnote 11}

Unchecked energy consumption contributes to undesirable economic and environmental outcomes: The IEA reports an overall trend of increasing global energy prices,^{Footnote 12} while in Canada prices for oil and gas are especially volatile.^{Footnote 13} In the event of rising energy prices, consumers bear the costs of rising utility prices, and the competitiveness of energy-intensive economies such as Canada’s can be jeopardized.^{Footnote 14} Increased energy use also has negative environmental consequences due to the direct relationship to increased GHG emissions. In Canada, 80 % of total national greenhouse gases are predominantly associated with the production or consumption of fossil fuels for energy purposes.^{Footnote 15}

Finding: Canadians and Canadian businesses are concerned about their energy use, but experience barriers to taking action. OEE programming responds to those barriers which include weak price signals, lack of capacity, and split incentives (where the benefits of improvements in energy efficiency are not passed on to those who invest in the improvements).

Canadians are concerned and want to do more: Recent survey data continues to show that Canadians and Canadian businesses are concerned about energy use, and believe that they can do more to address it. According to a 2013 Canadian Energy Efficiency Alliance (CEEA) survey of 1584 Canadians, 75% say conserving energy is important, while only 58% are doing some things to conserve.^{Footnote 16} Similarly, a CEEA March 2014 survey of 661 Canadian companies found that 67% of Canadian businesses see energy as a high concern for their business (with a further 19% saying a moderate concern), and 61% observed energy cost increases in the last year. However, only 27% believe they are doing all that they can to save energy, and just 38% have set targets for energy efficiency and cost savings.^{Footnote 17}

Current low energy prices and weak price signals: While the price forecasts for energy to 2018 remain relatively low^{Footnote 18}, there are incidences of price spikes with the last major price spike occurring in 2008. Improving energy efficiency can mitigate the economic impact of price spikes or longer-term increases in energy costs.^{Footnote 19} To address this, OEE supports the development of voluntary labels, codes and regulations on energy using products, buildings, and homes. McKinsey (2010) identifies this as a mechanism to expedite energy savings when there is a reluctance to invest in energy efficiency due to a perceived unacceptably long payback period, a situation particularly relevant to home owners.^{Footnote 20} Similarly, internal and external interviewees consulted for case studies on building codes, housing codes, and equipment regulations reported that because the market fails to sufficiently value energy performance, these activities level the playing field and raise the energy performance for all. By raising the efficiency of available products and houses on the market, the opportunity to purchase inefficient products and houses is reduced.

Lack of capacity (human and financial resources): Most external interviewees in the buildings, transportation, and industrial sectors indicated that they were aware of the need to address issues like climate change and GHG emission reductions through the implementation of energy efficient practices. However, interview and case study evidence indicates that Canadian businesses continue to lack capacity to systematically implement energy efficiency objectives, given the scarcity of both human and financial resources to prioritize energy management. In other words, the need for energy management far outweighs the capacity to do it. This is also true of federal government buildings, where interviewees from other departments and documentary evidence^{Footnote 21} indicate that property and facility managers of federal real estate require training, but also tools and resources to effectively operate the buildings and to conduct energy efficiency upgrades.

Lack of capacity (training and information): A multi-country analysis conducted by an international working group concluded that training sessions are a common and effective professional development model to raise awareness or disseminate best practices on a large scale in a cost-effective manner. That analysis specifically identified Dollars to $ense as one such example, as well as training offered in Korea and the US.^{Footnote 22} Many of the major knowledge and skills areas for professionals involved in energy management recommended in this review are covered by the six Dollars to $ense standard workshops, including “energy management planning”, “identifying potential opportunities”, “measurement and verification”, and “obtaining resource commitments”. Similarly, internal and external interviewees reported that truck companies and drivers need to save fuel as part of their business model, but note that there are many myths, particularly among drivers with the most experience (20 years of experience), about how to drive fuel efficiently. Interviewed industry associations, driver trainers, and carriers reported that companies are now offering incentives to their drivers to reduce fuel consumption, and larger companies are beginning to ask for driver training, including NRCan Smart Driver for Highway Trucking.    

3.2 Appropriateness of NRCan Role

Finding: The OEE programs are consistent with responsibilities set out in federal legislation for the Department of Natural Resources

The Energy Efficiency Sub-Program activities are consistent with the responsibilities set out within two key pieces of legislation for the department: 1) the Department of Natural Resources Act (1994), and 2) the Energy Efficiency Act (1992,2009). The Department of Natural Resources Act gives the Minister the mandate to “seek to enhance the responsible development and use of Canada’s natural resources…”.^{Footnote 23} The Energy Efficiency Act gives the federal government the authority to make and enforce regulations that prescribe standards and labelling requirements for energy-using products and products that affect energy use that are imported to Canada or shipped across provincial borders for lease or sale, and to promote energy efficiency and to collect data on energy use.^{Footnote 24}

Finding: National level leadership and ability to align efforts within Canada widely seen as a key role for OEE that is not effectively undertaken by others. This includes alignment of codes and standards, as well as administering several U.S. EPA programs in use in Canada to improve energy efficiency.

NRCan addresses uneven program capacity across Canada: Across all sectors, interviewees and case studies demonstrated that NRCan plays a unique and valuable leadership role in Canada with respect to energy efficiency programming. There are large differences in spending on energy conservation and energy efficiency programs across Canada in terms of absolute and per capita dollars. For example per capita spending ranges from $.10 per capita in Alberta to $36.04 in British Columbia.^{Footnote 25} However, an OECD/IEA report notes a high degree of synergy between the different policies and regulations in Canada and that there is cooperation between the different levels of government (federal, provincial and municipal levels).^{Footnote 26} NRCan-OEE was identified by most stakeholders as having facilitated this high degree of coordination across Canada by introducing standardized tools and systems for energy efficiency programming that can easily be used in developing local or regional programs. For example, according to provincial government interviewees, provinces that do not have energy efficiency legislation, or a central body responsible for energy efficiency (but have energy efficiency priorities and programs) rely even more heavily on OEE expertise and information to build and maintain their own capacity.

NRCan’s position in the federal government has uniquely facilitated national platforms for energy efficient products and practices: According to external interviewees, NRCan is best positioned to play a leadership role on harmonizing efforts given its energy expertise and its level of reach including strong connections with the U.S. and international domains. There was a strong sense across interviewees that the OEE programs could not be delivered by other organizations in a comprehensive or cost-effective manner given the prohibitive cost and capacity that would be required for these organizations to develop the same tools. For example, most partners in the housing and equipment sectors indicated that the loss of NRCan’s programs (e.g., EnerGuide Rating System and ENERGY STAR) would open the door to duplication and “greenwashing”, or would result in a decrease in the size and/or quality of programs delivered by others. NRCan is also the single partner with U.S. Department of Energy (DOE) and EPA for all of Canada facilitating the use of ENERGY STAR energy efficiency brand, which interviewed utilities reported as “the backbone” of the their in-market energy efficiency programs. In the buildings and transportation sectors, NRCan has also recently brought EPA Portfolio Manager energy benchmarking tool and SmartWay Transport Partnership Program to Canada. Interviews with industry associations and EPA and DOE officials confirm that only NRCan could engage in such agreements, as they have an established trusted relationship and it would be prohibitively inefficient for DOE and EPA to deal with each interested province or industry associations separately.

NRCan plays a key role in harmonization of equipment standards and building codes: NRCan also plays a key alignment role with respect to equipment standards within Canada and with the United States. External stakeholders noted that NRCan has displayed good leadership through administration of Minimum Energy Performance Standards (MEPS) in Canada. NRCan was said to often show leadership by pursuing standards that align with similar standards in the United States where it makes sense for Canadians, and consulting broadly enough with provinces and utilities so that federal standards can be easily referenced (in fact, some provinces such as New Brunswick, rely exclusively on NRCan for their regulations). NRCan has also provided a high degree of national coordination across jurisdictions in order to harmonize requirements for national model energy efficiency codes in buildings (the National Energy Code of Canada for Buildings 2011) and houses (including energy efficiency in the National Building Code of Canada). All interviewees consulted noted that NRCan played an essential role in the success of both code initiatives because of its ability to bring together various stakeholders into a national discussion on energy efficiency in buildings and houses.

NRCan uniquely enables networking and sharing of best practices about energy efficiency within the industry sector: In the industry sector, evidence from interviews and document review demonstrates that the Canadian Industry Program for Energy Conservation (CIPEC) plays a key leadership role in building and expanding the knowledge base for improving energy efficiency by providing a suite of tools and networking platforms through sector task forces to develop widespread industry support for implementing energy management solutions. All interviewees agreed that NRCan is uniquely placed to lead CIPEC because of its credible and neutral position, pulling together all energy sectors and partners that is not limited to the needs of any one province, institution, or industrial sector.

Finding: NRCan has unique content expertise that is valued by stakeholders and brings a high degree of neutrality to the tools and information it develops.

Interviews and case studies of training initiatives show NRCan content to be of high quality and valued by stakeholders: NRCan administers a number of training initiatives across sectors to address information and knowledge gaps related to energy efficiency. Findings from case studies of the Dollars to $ense, Smart Driver for Highway Trucking (SDHT), and Auto$mart each highlight the unique content expertise that NRCan provides. In Dollars to $ense, interviewees reported that the NRCan developed content was an effective “off the shelf” solution to fill their training needs, without having to develop their own content. Large corporate users of SDHT also reported that it would have taken them years to develop training programs at the same level of quality without NRCan.^{Footnote 27} Similarly, the Auto$mart personal driver training curriculum used by driving instructors was seen as high quality and uniquely available from NRCan.

NRCan labelling and rating programs seen as highly credible because of the verified nature of the labels and ratings: Many external interviewees (e.g., partners working at provincial/territorial and municipal levels of government, automotive manufacturers) indicated that the federal government, as represented by NRCan is, importantly, a neutral third party whose presence at a national level provides credibility to energy efficiency programs and policies. Examples include the EnerGuide Rating System for homes (ERS), ENERGY STAR, and the Fuel Consumption Guide. All interviewees consulted for the ENERGY STAR case study reported that the 3^rd party verification requirement of the program was a key element of the brand. It was said to provide a rigour that consumers can trust, but most importantly it minimizes the opportunity for people to abuse the brand and dilute its value by affixing it to inferior products. (Note that while third party verification is clearly important to users of NRCan programs, surveys of ENERGY STAR participants consistently identify concerns among Canadian ENERGY STAR participants, mainly from fenestration manufacturers, in the area of costs for product verification.^{Footnote 28}) The ERS used by provinces and utilities in administering their home incentive programs is valued because it is standardized, providing a way to independently verify energy performance. Finally, the Fuel Consumption Guide for personal vehicles, was reported by automotive manufacturer interviewees and consumer associations as being highly credible with consumers because it is issued by the federal government.

3.3 Alignment with NRCan and Federal Priorities

Finding: The OEE Energy Efficiency programs are consistent with NRCan strategic outcomes and federal government priorities related to environmental responsibility and economic competitiveness.

Responds to NRCan strategic outcomes: The Sub-program is formally linked to NRCan’s Strategic Outcome 2: Environmental Responsibility – Natural Resource Sectors and Consumers are Environmentally Responsible as indicated in Reports on Plans and Priorities. This positioning is well understood by interviewed external stakeholders, though most note a strong relationship between energy efficient programming and economic competitiveness. Economic Competitiveness is NRCan’s Strategic Outcome #1 according to the 2013-14 Report on Plans and Priorities. Almost all interviewees reported that reduced energy use has competitiveness benefits for Canadian companies, as does alignment of codes and standards within Canada and of testing standards related to energy using products in North America.

Responds to Government of Canada strategic outcomes: Two of the Government of Canada’s strategic outcomes in the Economic Affairs area are Strong Economic Growth and A Clean and Healthy Environment,^{Footnote 29} which energy efficiency programming contributes directly to. Many interviewees across sectors also pointed to the consistency of OEE programming with respect to federal priorities in creating jobs, contributing to national and global economic growth and prosperity, competitiveness, and demonstrating environmentally responsible natural resources sectors while having a positive impact on consumers. In fact, several interviewed industry associations noted that energy efficiency programs have a strong relationship to the government’s more recent focus on defending Canadian consumers. Indeed, external interviewees in both the equipment and housing sectors noted significant benefits to the consumer from more efficient energy using products and homes in terms of the impact of total energy use on their utility bills.  

Finding: The Energy Efficiency Sub-Program is consistent with federal commitments and policy instruments referring to environmental and economic objectives for Canada (Canadian consumers, homeowners, businesses, and federal departments)

OEE’s energy efficiency Sub-program objectives are aligned with the federal government’s energy use, environmental and economic priorities as identified in Speeches from the Throne and Budgets:

• Speeches from the Throne and Budgets expressed commitment to address climate change through the Copenhagen Accord^{Footnote 30} and by recognizing integrated economic links between the United States and Canada, to reduce emissions (i.e. Canada-U.S. Clean Energy Dialogue)^{Footnote 31}; and investment in the development and deployment of clean energy technologies;^{Footnote 32} and a review of energy-efficiency and emissions-reductions programs to ensure they are effective;^{Footnote 33}
• The government committed to address environmental and economic needs through alignment with the United States. For example, through the Canada-U.S. Clean Energy Dialogue and a “sector-by-sector regulatory approach to align with the United States to achieve GHG emission reductions” ^{Footnote 34} The 2011 budget allocated $86M to support regulatory actions, focusing on energy efficiency.
• As part of the economic stimulus package, under “Action to Stimulate Housing Construction” the government provided $7.8 billion to build quality housing, stimulate construction, encourage home ownership and enhance energy efficiency.^{Footnote 35}
• A website for Canada’s Economic Action Plan notes that energy efficiency is a proven strategy to reduce energy use and costs as well as lower emissions. Furthermore, energy-efficient technologies and practices offer the opportunity to reduce costs and improve operating performance while contributing to a cleaner environment.^{Footnote 36}
• In Budget 2011, the federal government renewed funding for the Clean Air Agenda, which supports Government of Canada efforts to reduce GHG and air pollutant emissions while continuing to focus on regulatory development.
• In 2010, the federal government required 28 departments and agencies to develop and report on the Federal Sustainable Development Strategy (FSDS), which sets out goals, targets, and implementation strategies for reducing the federal government’s environmental footprint.^{Footnote 37}

4.0 Key Results Findings

This section reports on the effectiveness of the key groups of interventions the OEE undertakes to achieve outcomes, and assesses their contributions to reported results at the intermediate outcome level (i.e., adoption of energy efficiency products, practices, codes and regulations).

4.1 Program Theory and Logic Model Outcomes

The theory of change for the Energy Efficiency Sub-Program is based on an identified need to introduce a link between proven energy efficiency technologies and practices and adoption by industry and consumers, so that these practices and technologies can lead to more efficient energy use. Energy efficiency is also expected to provide corollary benefits such as reduced GHG emissions, and cost savings for consumers and businesses.

The Energy Efficiency programs provide the precursors to behaviour changes that result in energy efficiency by increasing target group awareness and capacity to adopt energy efficient products and practices. For the purposes of this evaluation OEE activities have been classified into four groups. 1) Training and Information; 2) Labelling; 3) Codes, Regulations, and Standards; and 4) Support for Energy Management, Monitoring, and Rating.

Table 2 provides the logic model and intended outcomes for the Energy Efficiency Sub-Program.

Table 2:  Overall Energy Efficiency Sub-Program Logic Model

Outcome Level	Outcome Description
Immediate Outcomes	Awareness of energy efficiency products and practices Capacity to adopt energy efficient products and practices Housing and building codes and equipment standards ready for adoption by authorities having jurisdiction
Intermediate Outcomes	Adoption of energy efficiency products and practices by the target population Housing and building codes and standards adopted by authorities having jurisdiction
Final Outcome	Increased energy efficiency in target sectors

4.2. Training and Information (Awareness and Capacity Outcomes):

Awareness/Capacity Building:

Finding: Target groups are for the most part being reached by OEE training and information; potential gaps identified with reaching industrial facilities not involved in CIPEC

OEE reaches a wide range of energy users, influencers or manufacturers of energy using products with its training programs, all of which are either on track to meet or have exceeded program registration targets during the evaluation. The following table provides an overview of number of professionals trained against targets for OEE. For example:

• FleetSmart SmartDriver Training: provides information and introduces fuel-efficient operating practices and smart-driving techniques to fleet drivers through workshops and online learning. The program is on track to meet its 2015/16 training targets, having so far trained a total of 16, 234 fleet drivers (for all Smart Driver streams) from 2011-12 to 2013/14, which is on track to meet 2015-16 target of 24,800.
• Auto$mart for personal drivers is a driver training program that aims to improve awareness and demonstrate importance of fuel efficient driving among new drivers. OEE develops the content, which is administered by driver training instructors. OEE estimates that approximately 210,000 students per year are exposed to Auto$mart training material.^{Footnote 38}
• Dollars to $ense energy management training workshops are provided by OEE to facilities managers and energy service companies in the buildings and industry sectors (e.g., industrial, commercial, and institutional organizations). These workshops are in high demand, surpassing targets for number of building and facility managers and energy consultants trained. Between 2011-12 and 2013-14, about 5,750 participants in the buildings sector (commercial or institutional) were trained, exceeding the cumulative targets of 1950 participants trained in this sector by 2015-16. Similarly, for the same period, 2,250 industry sector representatives participated in Dollars to $ense workshops, exceeding the cumulative target of 2,166 participants by 2015-16.^{Footnote 39}
• Training in support of home labelling programs (EnerGuide Rating System, ENERGY STAR for New Homes, and R-2000), is developed by OEE for industry professionals to support these construction standards. According to OEE Performance Measurement, training energy advisors and industry professionals have consistently exceeded targets from 2011-12 to 2013-14. Indeed, results from 2013-14 show that there were 784 active certified energy advisors (exceeding the Program’s target of 500) and over 3000 builders were trained on new energy efficient building techniques (energy professionals, builders, trades), exceeding the target of 2500.
• CIPEC information and training: The purpose of CIPEC is to promote greater energy efficiency through voluntary action that reduces industrial energy use through providing industry with training and information and networking opportunities. CIPEC’s reach extends to more than 2,400 registered companies in CIPEC’s Leaders network, covering 98% of Canadian industry through links with industry associations and a total of 5,800 participant facilities in the program.

Evidence suggests that CIPEC is highly valuable to those who participate and that the program could benefit from expanding or strengthening its reach: Interviews and the case studies of CIPEC suite of activities and support for Energy Management Systems demonstrated that the activities contribute to decision-making regarding energy efficiency, and that CIPEC is playing a crucial leadership role building awareness of energy management nationally. However, there was a unanimous perception across general and case study interviewees that opportunities exist to fill awareness gaps amongst both industry and the public (because energy management impacts the general public as well) and to promote partnerships across federal and provincial jurisdictions. Interviewed companies involved in the program reported that because CIPEC is such an effective platform, more should be done to raise its profile among those companies who are not members. CIPEC Leaders stand to benefit most from the program offerings because they are the most engaged, and will likely continue to take advantage of the information and support offered having seen value in it. While the number of workshops offered and participants reached in the building sector has increased over time, the number of participants in the industry sector has been dropping steadily, but remains above the annual target of 450 participants trained.

Finding: Strengths of OEE training programs include the format, content, delivery model. Challenges however have been identified with launching SDHT online.

CIPEC training and tools perceived to be useful by target groups. According to OEE performance measurement, there were 3,511 participants in industry sector training events (energy conferences, webinars, on-line joint training with partners), exceeding the target of 1800 training participants.^{Footnote 40} Delivering training through webinars was noted by all interviewees in the industrial sector to be a convenient, cost-effective, and efficient means to educate employees and build capacity. In the 2014 survey, 97% of webinar respondents (n = 197; response rate of 23%) found the information useful. In the 2014 survey of CIPEC-leaders and non-leaders, approximately six out of ten respondents rated energy management workshops (e.g., Dollars to $ense), webinars and tools, technical information and the Heads Up CIPEC newsletter as being either very or somewhat useful, and about half of each group rated the CIPEC tools (i.e., boiler efficiency, energy intensity and RETScreen^{Footnote 41}) as useful. Uptake of outputs seems to be marginally related to level of engagement with CIPEC (i.e., CIPEC Leaders were more likely to find each of these useful, but by a relatively modest margin).

Quality of content of Dollars to $ense seen as strong feature of the workshops by participants: Evidence from the Dollars to $ense and CIPEC suite of tools case studies as well as more general interviews shows that workshop participants consistently rated the quality of content and teaching highly. Other features highly appreciated by clients and participants include the interactive nature of the sessions, the intensive/one-day format, and the high level of customization offered to specific audiences (also enabled by the facilitators). Some interviewees also pointed to a growing need for even greater customization as awareness increases among their community.

Recent changes to SDHT and Auto$mart were well received by users: In both Auto$mart and SDHT cases it is important to note that recent updates (abridged classroom version in 2011 and e-learning version in 2010 for SDHT and version 3 of Auto$mart in 2010) to the content and format have been well received by users, many of whom noted that these updates were long overdue and necessary to maintain the credibility and suitability of the programs to their need. Interviewed instructors report that the latest version of Auto$mart provides them with easy to use, high quality, factually accurate materials that facilitate student interest in energy efficient driving which they can easily integrate into their safety messaging, which is by all accounts the focus of driver education programs. The case study showed this alignment with safety to be very important as instructors are under greater pressure from provinces to focus more on safety messaging. They were pleased with the latest version of the curriculum that focused on 5 key messages (accelerate gently, maintain a steady speed, anticipate traffic, coast to decelerate, and avoid high speeds). SDHT interviewees noted that the previous classroom version was too long (8 hours), repetitive and outdated (older truck models) which limited interest in uptake. Creating a 2 hour, customizable classroom version and adding an e-learning format was well-received and increased uptake.

Significant IT challenges identified with online delivery of SDHT: The program has recently further addressed user feedback and soon will pilot a redeveloped SDHT curriculum that supports a blended-learning approach (i.e., e-learning, classroom option and on-road practicum) for facilitated classroom sessions, self-directed learning (on and off-line), remedial self-evaluation and on-road applications. However, ongoing technical issues associated with the SDHT e-learning application (SABA -IBM platform^{Footnote 42}), and limited IT capacity within NRCan and Shared Services Canada (ie., SABA expertise, human resources and band-width^{Footnote 43}), have negatively affected the administration and management of the 2010 SDHT e-learning and have delayed the development and piloting of the newly-developed web-based SDHT curriculum. These issues appear to have placed the e-learning effort in jeopardy.

Finding: OEE produces high quality information products and tools. The uptake of information products by target audiences is mixed, though there are indications that products are used by target audience members. Evidence suggests opportunities, particularly for online information, to improve reach of these products

OEE has delivered information and tools to target groups, and these are seen to be high quality and valuable: Each sector provides a variety of information products meant to engage and inform target audiences on sector specific energy efficiency issues. Interviewees across sectors and initiatives generally indicated that OEE information products are of high quality, and well researched. According to OEE performance measurement data, the programs are largely on track or have exceeded their targets for producing this material. Within the evaluation period, each sector has updated numerous portions of its web content, held information sessions with stakeholders (e.g., buildings division held 27 benchmarking information sessions for specific stakeholders/NGOs against a target of 8), and developed information packages and publications. Some examples in this area include:

• Heads Up newsletters:  the document review and some interviewees mentioned the Buildings and Industry websites and the electronic Heads Up newsletters as useful resources. Note that the Heads Up: Building Energy Efficiency Newsletter is disseminated monthly to over 12,000 subscribers.^{Footnote 44}. In a 2010 study of industrial sector needs, the Heads Up CIPEC newsletter rated highest among CIPEC tools and was the main source of information used by some industry groups.^{Footnote 45} According to OEE, there has been a 30% increase in the number of Heads Up CIPEC subscribers since April 1^st, 2011, exceeding the 10% net increase over 5 years.
• Solid State Street Lighting Qualified Products List: Evidence from the Solid State lighting case study showed that based on 7 pilot projects funded in part by NRCan, information tools were developed to help municipal city managers to convert lighting schemes to solid state lighting. This included a DLC Qualified products list. Information products developed from this initiative were said to in part address external interviewees concerns about the complexity of LED street lighting systems, the rapid changes that are occurring in the technology, and the difficulty these factors represent in making procurement decisions.

Evidence suggests OEE has experienced challenges with publishing some outreach materials, that stakeholders would like to see more of. It is worth noting that the information activities that are off track tended to be related to publications, which may be due  in part to approval processes within NRCan. This includes, ENERGY STAR annual reports and awards, industry newsletters, and case studies of departments that have completed energy performance contracts under the Federal Buildings Initiative.  Interviews with target audiences in industry, buildings, transportation, and equipment sectors noted the importance of case studies as a means for convincing others to participate in the various energy efficiency initiatives. Many external interviewees suggested providing more case studies of companies, facilities and even consumers that have implemented OEE promoted energy efficient products or practices as they are among the most powerful means of influencing decision-makers. Internal interviewees across several sectors noted challenges to varying degrees with publishing outreach materials more generally.

OEE has redeveloped its WebPages, and tracking of page views suggests they are being accessed, though interviewed user groups suggest challenges: OEE performance measurement indicates that the number of page views expected for OEE online content is on track to meet targets for 2014 in all/most sectors.^{Footnote 46} However, evaluation evidence suggests that there are opportunities to improve accessibility. For example:

• Interview evidence indicates that there is a wealth of information available to fleet managers through the FleetSmart web-portal, and to those who access it is highly valuable. However, interviewees report that it is cumbersome to log onto FleetSmart to access SmartDriver which may be a deterrent. As an illustration of lack of use or accessibility, some interviewees suggested OEE develop information products that already existed but they were unaware of – such as a guide to driver incentive programs.
• NRCan provides online tools to help with understanding and using ENERGY STAR. Surveys of ENERGY STAR participants (2011 and 2014) suggest that this information has little uptake. Interviewed utilities, retailers, and manufacturers remain uncomfortable with the NRCan website, reporting that they hear from their customers that the information is very hard to follow (i.e., too technical, too disorganized). However, it should be noted that this research was conducted shortly after the launch of an updated NRCan ENERGY STAR website, and respondents were likely referring to the previous version. Interviews with utilities suggested that most are not specifically aware of the information available on NRCan’s website to help consumers and that they were primarily interested in the product lists they use to design their consumer rebate programs. A September 2014 study by Consumers Council of Canada on energy labelling found that the majority of consumers did not go to government websites to find information about energy efficient products. That study suggested a need for government authorities to support consumer information providers to facilitate education about energy efficiency in products. ^{Footnote 47}

Adoption of Energy Efficient Products and Practices:

Finding: Evidence from case studies, interviews, and surveys of target audiences together suggest strongly that training and information activities contributed to adopting energy efficient products and practices, which lead to energy savings.

Surveys of target audiences suggest that OEE training has contributed to behaviour changes, though the generalizability of these results is weak as response rates for all surveys were low (typically between 1% and 23%). However, the survey results in all training programs, align with the results of evaluation interviews and case studies, suggesting that the training has in fact lead to behaviour changes in target populations.

Training commercial drivers through SmartDriver has contributed to driving behaviour changes: Results of four, small,^{Footnote 48} 12-18 month post-training follow-up surveys with participants from all streams of SmartDriver training taken between 2009 and 2013 found that between 68% and 100% of respondents learned new skills from the training, implemented at least three new driving techniques, and between 50% and 84% noticed improved fuel economy since taking the course.^{Footnote 49} Evidence from the FleetSmart SDHT case study and more general interviews on SDHT indicate that both the SDHT e-learning and classroom-based fuel efficient driver training effectively increased driver knowledge and contributed to changes in how they drive. Interview evidence from the case study indicates that successful training of SDHT drivers was aided by several strategies that reinforce the learned fuel efficient driving behaviours as drivers can revert to non-fuel efficient driving habits shortly after training - . The experiences of the two companies explored in the case study (one using the workshop format and another using the online format) showed that discussing driving performance with drivers based on data from on-board driving monitoring devices, administering refresher training, and incentives are important complements to driver training for sustained behaviour change which result in fuel savings.

Impacts of AutoSmart are likely, though evidence is unclear: The available evidence on the behavioural impacts of the curriculum is unclear, based mainly on small sample size surveys with students shortly after participation. For example, in 2011 NRCan conducted pre and post surveys to test knowledge gains and use of energy efficient driving techniques of students enrolled in a beginner driver course that included Auto$mart as part of the curriculum.^{Footnote 50} On average, responding participants (66 responses) increased their knowledge of energy efficient driving by 10.4%. The follow-up survey indicated that overall, the 37 student survey respondents practiced fuel efficient driving practices.^{Footnote 51} However, it is not known the extent to which Auto$mart itself contributed to these behaviors. Eighty-five percent (85%) of the 37 respondents in the follow-up survey did report that the driver training course had given them the information they needed to conserve fuel while driving. Evidence from the case study suggests short term behaviour impacts are plausible from this messaging. Note that young drivers were seen by instructors as particularly responsive to the messages due to their interest in environmental issues. External interviewees and document evidence caution that there are many factors which will influence a novice driver’s behaviour in addition to training. Indeed the literature on the myriad of influences on driving in addition to training shows the complicated nature of this behaviour.^{Footnote 52}

Dollars to $ense contributed to enhanced capacity to implement energy management projects: Evidence from evaluation case studies, interviews and follow-up surveys conducted by OEE suggest that the Dollars to $ense workshops are contributing to decisions to implement energy efficient products and practices. Both the Dollars to $ense and CIPEC suite case studies and more general interviews showed that the increased knowledge gained from Dollars to $ense training translated directly into participants being able to go back to their organizations and implement what they have learned in their organizations. For example, one workshop participant identified an opportunity to avoid simultaneous operation of heating and cooling systems in an office area. Another learned about a project to improve energy efficiency in ice hockey arenas in community facilities, which was then shared across several municipalities. Small municipalities in particular have been implementing projects after having staff attend the Dollars to $ense workshops. Interviewees noted that the two main types of benefits from Dollars to $ense workshops are energy savings and avoided maintenance costs related to decreased energy use. These results corroborate previous OEE survey results. Over 90% of surveyed industry workshop attendees in 2012-2013 ( n= 1,271) reported increased awareness of good energy management practices,^{Footnote 53} with the most commonly reported practices for all attendees  being “identifying immediate savings opportunities” (buildings 59%; industrial 67%; and services sector 70%). One third of building sector participants indicated they implemented a retrofit or energy management plan. Close to half of industrial respondents conducted an energy audit, developed an energy management plan, or prepared a financial business case for an energy management initiative. As well, a 2014 survey of Dollars to $ense trained energy service company representatives (n=19; response rate of 6%) reported that the training helped them to, among others, establish or improve their clients’ energy management planning, energy management teams, identify immediate savings opportunities, and prepare a financial business case for an energy management initiative.

CIPEC tools and information contributed to behaviour change, with information being shared within organizations, likely broadening the impacts. Surveyed CIPEC leaders and non-leaders in 2014 assigned on average a rating of four out of ten to the influence of CIPEC (tools, training, information and resources) on their decisions to implement energy management strategies over the previous 12 months.^{Footnote 54} In fact, about half reported that their involvement in CIPEC influenced their decision to invest in new equipment and processes for energy management. Interview evidence led to similar findings with regard to the broader suite of CIPEC tools and initiatives examined through the case study, with most interviewees highlighting that being a CIPEC Leader^{Footnote 55} improved program awareness and uptake through the consistent sharing of best practices. As well, sharing webinar information with colleagues (55%) and exploring implementation ideas (49%) were the two most frequently cited follow-up actions among webinar participants in 2014. This is consistent with interview data, and evidence from CIPEC and Dollars to $ense case studies suggesting information from both CIPEC webinars and the available CIPEC suite of tools appears to be shared with colleagues within attending organizations. In a 2014 survey of CIPEC webinar participants (n = 197, response rate of 23%), 91% reported improved knowledge of the energy subject matter presented, 70% reported taking action immediately following the webinar (identifying immediate energy savings and conducting an energy audit were the two measures reported most frequently), and 65% expected to achieve or had already achieved energy saving from what they had implemented.^{Footnote 56}

Housing labelling programs have influenced the techniques used in the sector beyond building to label specifications. In total 80% of surveyed builders in 2014 reported using the building techniques they had learned when constructing non-labelled homes. Builder and service organization interviewees similarly reported that training and courses also caused a general overhaul of their processes to incorporate the new knowledge into their wider operations. As one builder explained, once the course material was absorbed, it would have been difficult to return to previous practices now that that knowledge was available within their organization.

Finding: OEE training and information programs are contributing to energy savings in target groups. Case studies surveyed and interviewed users consistently report energy savings.

Across each of the training activities examined above, where the evaluation found evidence that target groups have adopted energy efficient products and practices promoted by OEE, they did realize energy savings. For example,

• Both SDHT case study companies report sustained fuel saving of around 5-6% when training is combined with their broader corporate practices to reinforce the message, though all noted that training was a key feature of this effort. These savings, according to interviewees, are easily tracked due to the increasingly widespread use of onboard monitoring devices in transport trucks.
• Between the inception of the Dollars to $ense program (1997) and January 2014, OEE reports that over 26,000 participants have been trained, leading to $160 million in annual energy savings and 1.1 MT of avoided emissions.^{Footnote 57} The evaluation can corroborate this link. The large majority of participants surveyed between 2011 and 2013 reported improved energy performance of their building, company or clients’ facilities: buildings (99%); industrial (88%); and services sector (i.e., utilities and consultants, 88%). These savings appeared to be in the range of 10-15% reduction in their energy consumption.^{Footnote 58} Similarly, the 2014 survey of energy service company representatives (2014, n=19 caution 6% response rate) also reported that, using the information they obtained from the training, they expected to help their clients save energy, in the order of approximately 20% on average. Most companies interviewed for the Dollars to $ense case study noted that while they could not measure the effects of training on their organizations, they have observed a discernible difference in behaviours that have impacted the bottom line positively. Several anecdotal examples were identified through the documents and interviews of energy savings identified and/or practices implemented ( e.g., implementing a retrofit, conducting an energy audit, developing an energy management plan, prepared a financial business case for an energy management initiative).
• The 2014 builder survey (n = 154, response rate of 6%) found that responding trained builders are building significant portions of their homes to these standards. Approximately four out of ten reported building all of their homes to a higher performance level on the EnerGuide scale or to either ENERGY STAR, R-2000 standards over the past three years, while two out of ten reported building between 80 and 95% of their homes to one of these standards. About 84% of responding builders expected to continue building the same or more homes to EnerGuide, ENERGY STAR, R-2000 over the next five years. Energy savings in this regard are measurable based on modelling using the EnerGuide Rating System.
• In a 2014 survey of CIPEC leaders and non-leaders, about 40% of each group reported the reduction of energy costs as being one of the greatest benefits of participation in CIPEC. In 2012 alone, according to the CIPEC Annual report, CIPEC Leaders recorded total annual energy savings of 0.6 PJ and GHG savings of 74 kt.^{Footnote 59} Evidence of overall improvements in energy efficiency from a combination of CIPEC tools and training for the 9 companies examined in the CIPEC suite case study includes combined reported annual energy savings of more than $15.8 million, reductions in annual energy demands for various energy sources (electricity, natural gas, etc.) for these companies (i.e., not total company energy use) ranging from 10% to 92%, implementation of physical improvement in facilities such as LED lighting, motion lighting switches, retrofits to equipment, and use of monitoring instruments that are checked regularly.

Factors Influencing Adoption:

Finding: A key factor facilitating adoption of energy efficient products and practices appears to be access to a system of interventions, some of which have been provided by OEE and others provided by partners. The availability of reinforcing mechanisms and additional support has likely had a compounding effect on implemented energy saving best practices.

OEE tools reinforced and supported implementation of concepts learned in energy management workshops: Evidence from surveys, interviews and case studies suggests that having a system of OEE tools to reinforce training participants’ newfound capacity has been valuable and that those who received training have sought out additional OEE tools. For example, about half of industrial Dollars to $ense participants surveyed from 2011 to 2013 have used or consulted the online tools provided at the Dollars to $ense training, and nearly 70% of services sector participants have done so. Similarly, the majorities of surveyed energy service companies in 2014 that attended Dollars to $ense workshops (n = 19) reported seeking out all of the NRCan on-line tools for building energy efficiency.^{Footnote 60} In all groups, the CIPEC RETScreen and the Energy Savings Toolbox were the most popular (at least 50%), followed by case studies and the energy management information system. Surveyed CIPEC webinar participants in 2014 (n = 197) also reported accessing a range of CIPEC tools. After sharing webinar information with colleagues (57%) and exploring implementation ideas (49%), accessing the CIPEC website was the next-most popularly-reported action (41%), while between one-third and one-quarter accessed other materials available through CIPEC. It is also noteworthy that the most frequently-attended CIPEC webinars among respondents to the 2014 survey were ‘ISO 50001’, ‘Motor Management’ and ‘Energy Management Information Systems (EMIS)’ – both ISO 50001 and Energy Management Information Systems are also supported by broader CIPEC activities.

Combinations of CIPEC tools had a compounding effect on companies in CIPEC: The nine companies examined in the CIPEC suite case study also identified having changed their behaviours by implementing energy efficiency plans and changes to operations due to a combination of the training, use of CIPEC tools, and access to a network of companies sharing best practices in energy management. Indeed, interviewed companies for the CIPEC suite, Dollars to $ense, and Energy Management Systems Standards (EMSS) case studies all suggest that the strongest adoption of energy efficient practices were generally associated with those most engaged with OEE programming (i.e., taking advantage of a wide range of OEE offerings, mainly through CIPEC).

As well, interviewees and case study participants indicated that industry networking opportunities were essential aspects of energy management and these interviewees encouraged greater focus of CIPEC on such opportunities. In the 2014 survey, about half of CIPEC Leaders and non-Leaders reported that the greatest benefits to participation were ongoing discovery of opportunities to better manage energy use, followed by four out of ten that reported access to CIPEC tools, and ability to save on energy use.

Use of other OEE tools and services were also attributed to raising capacity of the home construction industry. Almost all interviewees reported that tools and support (e.g., software modelling, blower door testing, building research) given to both builders and other industry groups beyond the training had directly increased builder capacity. In addition, the builders appreciated the complementary training offered to improve their practices and use of these tools. For example, they cited ease of testing new homes through blower-door test and Hot 2000 modelling software.

Dollars to $ense participants suggested further opportunities to improve complementarity of OEE tools: The importance of awareness and training as only one part of the behaviour change process was highlighted by some interviewees consulted for the Dollars to $ense case study. They suggested the workshops should be even more aggressively coupled with other tools or measures provided by OEE (e.g., benchmarking, EMIS implementation, ISO 50001 certification) that could work together synergistically, to further increase capacity and adoption.

Importance of reinforcement for driver behaviour: Both interview and document evidence indicates that driver training effectiveness can be enhanced by follow-up reminders and refreshers.^{Footnote 61} Note that while SDHT messaging tends to be reinforced to truck drivers by large trucking companies that value the training and are interested in fuel efficiency, similar reminders do not necessarily exist for novice personal drivers trained under Auto$mart, which adds to the complexity of sustained behavior impacts from this program.

4.3 Labelling Programs (Awareness and Capacity Outcomes):

Awareness and Capacity

Finding: The product and home labelling infrastructure that OEE maintains is used by utilities provinces and other partners as the basis for their programs to reach Canadian consumers

OEE’s EnerGuide Rating System (ERS) is used by provinces, utilities and private organizations to administer incentive and home labelling programs, and for code development and maintenance in some jurisdictions. Document evidence indicates that as of March 2014 there were over 50 programs and regulations across Canada using the ERS.^{Footnote 62} There are 39 programs based on ERS including eight ongoing provincial incentive programs (New Brunswick, Nova Scotia, Quebec, Ontario, Saskatchewan, Alberta, British Columbia, and Yukon), and private labels such as Built Green, LEED Canada for Homes, Novoclimat and Greenhouse Certified Construction^{Footnote 63} There are also 14 jurisdictions using ERS to develop or implement energy performance requirements in building codes and regulations. External interviewees, including builders and utilities, explained that ERS  provides a needed level of uniformity for the various energy demand management and incentive programs running across the country, by having a single rating system, which has avoided having a variety of competing rating systems across jurisdictions.  Users of the ERS for their programs also noted that the infrastructure provided by NRCan in terms of the HOT2000 software, the administrative records management for homeowners, training of energy service companies, all offered high value to them as program deliverers.

ENERGY STAR labelling is used by utilities and provinces to administer incentive programs, and is promoted to consumers by participants. Several provinces and utilities use ENERGY STAR as a qualifying criterion for various incentives programs such as rebates and sales tax exemptions.^{Footnote 64} For example, NRCan’s ENERGY STAR website indicates that 47 utilities are currently registered as ENERGY STAR participants.^{Footnote 65} Interviewed utilities and documents reviewed indicate that utilities’ spending on various incentive programs for the purchase of ENERGY STAR equipment range from $3-4 million annually to $20 million annually. There are currently 1000 registered ENERGY STAR participants, including manufacturers that develop qualifying products, and all partners promote those products through the use of the ENERGY STAR trademark. Manufacturers consulted for the ENERGY STAR case study leveraged the ENERGY STAR brand by using its trademark in their flyers and inserts, and in turn spending anywhere from a few tens of thousands of dollars to a high of several million dollars each on promotional materials that include ENERGY STAR. The 2014 survey^{Footnote 66} of ENERGY STAR participants found that 82% of respondents (179 of 218 respondents) promoted ENERGY STAR in 2013, at an approximate average investment of $29,237 (based on the responses of 126). 91 % of respondent participants use at least one of the ENERGY STAR symbols and graphics. ^{Footnote 67} Thus OEE leverages the promotional efforts of ENERGY STAR participants, which as indicated above, is significant.

Finding: Labels are a valuable informational tool for consumers. Their impact could be strengthened by complementing the ENERGY STAR endorsement label with improved comparative labels. By providing energy or dollar cost savings information for EnerGuide for equipment and vehicle labels, and improving efforts to communicate value of housing labels, these tools would offer even greater value to consumers.

Product and vehicle labelling at point of sale seen as valuable aids to consumer decision-making, though consumers are seeking energy efficiency labels that help them understand cost to operate products and vehicles: Interviews with retailers of ENERGY STAR products and external stakeholders in the automotive sector each noted that the product and vehicle labels are important aids to consumer decision making. The 2013 CEAA survey found that 88% of Canadians considered ENERGY STAR product labelling at least somewhat helpful (including 58% very helpful) at helping Canadians to be more efficient.^{Footnote 68}  For vehicles it was said that physical labels are of decreasing importance in the longer term due to online and advertising sources of fuel information for consumers. However, even in this case, labels were noted as the final opportunity to influence consumer choice. These labels continue to provide important information to consumers, including estimated annual fuel use. External interviewees noted however that these labels could be more effective if they included or were complimented by an easy means to compare cost to operate. For example:

• Retailer and utility interviewees for the ENERGY STAR case study all reported that consumers want to know how much money they will save more than anything else, and are already thinking about money when they are in a store. Most case study interviewees as well as the 2014 survey of ENERGY STAR participants^{Footnote 69} reported that consumers at minimum see ENERGY STAR as a mark of energy efficient consumer goods, though there is room to improve consumer understanding to ensure they link saving energy with saving money. For example, a retailer pointed out that consumers don’t always understand that ENERGY STAR means energy savings to them over non-ENERGY STAR products.
• Similarly, OEE commissioned research with CrossChasm shows that Canadian car buyers are concerned about fuel efficiency, but it is not easy for them to purchase a fuel efficient vehicle without simple presentation of estimated driver fuel use. However, augmenting the Fuel Consumption Guide website to provide a calculation based on driver information would enhance capacity to make decisions. Automotive manufacturers also suggested improvements to the Fuel Consumption Guide to address deficiencies in the ability to make reasonable comparisons between hybrid electric vehicles, plug in vehicles, and traditional fuel vehicles. This will become a larger problem each year, as manufacturers release more and more of these non-traditional vehicles.
• Concerns were also raised by auto manufacturers with respect to OEE finalizing the agreement on voluntary labelling for personal vehicles. According to OEE performance measurement, as of March 2014, negotiations with auto manufacturers and dealers to implement a memorandum of understanding for a new voluntary vehicle label were still ongoing.  The new voluntary approach, which replaced the regulatory approach, would target the 2016  model year (which represents a two year delay from the original model year targeted by the regulation).  Interviewed manufacturers expressed concern that they would experience significant logistical challenges in providing labels for the 2016 model year vehicles, as was their understanding of the planned effective date, given the delays in getting agreement on the label.^{Footnote 70}

The new “Most Efficient” label seen as a positive ENERGY STAR step, but is hindered by low consumer and participant awareness in Canada: In 2011, ENERGY STAR introduced as a pilot the “Most Efficient” initiative, labelling the top performers in a handful of product categories in each calendar year, only becoming a standard ENERGY STAR offering in 2013 (i.e., this is a sub brand to the larger ENERGY STAR label). The ENERGY STAR case study found that utilities, retailers, and manufacturers see “Most efficient” as a positive step and an expedient means for differentiating the top performers in product categories where technology has made ENERGY STAR performance easily reachable (fenestration) or in categories that evolve very quickly (consumer electronics). A recent study of energy efficiency labelling by the Consumers Council of Canada (CCC) found that in a focus group, the “Most Efficient” label was viewed more favourably than the core ENERGY STAR brand as it enabled consumers to identify the most energy efficient products quickly. ^{Footnote 71} However, there appears to have been little promotion of the “Most Efficient” designation. As well, in Canada nearly all case study interviewees suggested that better communication from NRCan about this part of the brand would be helpful in making it relevant to them and Canadian consumers. For example, regardless of NRCan’s emails to each participant organization, interviewed manufacturers did not learn of the “Most Efficient” designation through their industry associations, trade events, or directly from NRCan and some noted that while pleased that their products qualified, they were unsure how it had occurred, and therefore were not able to immediately capitalize on that designation in their marketing material. Interviewees also suggested that NRCan needed to promote the brand “Most Efficient” to manufactures that could be candidates, and facilitate consumer awareness of what the brand means in order for utilities to use it as part of their in-market promotions. It should be noted that promotion of ENERGY STAR “Most Efficient” to manufacturers is done by the US EPA, with the exception of fenestration manufacturers – the only category managed by NRCan in Canada. The 2014 ENERGY STAR participants survey also found that those who were aware, were unlikely to have used the distinction, but most thought it was a positive step.^{Footnote 72}

Finding: Home builders continue to participate in NRCan premium home labelling initiatives, and are actively marketing energy efficiency to homeowners. However, consumer demand for premium labelled new homes is a persistent challenge.

Home builders are committed to the labelling programs they participate in and are marketing energy efficient homes to consumers, using NRCan materials. Interviewed new home builders and energy advisors remain committed to the home labelling programs. Indeed, 80% of surveyed builders in 2014 plan to recertify and about half reported that participation in the energy efficiency programs (ERS, ESNH, R-2000) plays at least somewhat of a role in the success of their business. Builders are marketing energy efficiency to consumers, and those interviewed are generally satisfied with the marketing material produced by the program, which they use with consumers. Eight out of ten surveyed builders reported including energy efficient features of their homes in sales and marketing materials, and four out of ten went beyond using the certification logo and label to promote their respective program (EnerGuide, ENERGY STAR, R-2000), mostly with print and web advertising.^{Footnote 73} Half of surveyed builders in the 2014 survey reported building homes that qualify for certification without getting them certified or labelled. The primary reason given was that their clients want energy efficiency but are not interested in receiving a rating (reported by almost three-quarters of respondents).

Consumer interest in and potential confusion over premium home labels is a challenge for the programs. Builders report weak consumer demand for premium home labelling. Interviewed builders continue to identify weaknesses regarding consumer awareness of and demand for energy efficient products and practices for housing. This is supported by two-thirds of surveyed builders who indicated that at least a moderate effort was required to sell houses built to an energy efficient standard (EnerGuide, ENERGY STAR, R-2000), with ENERGY STAR builders being significantly less likely than others to report an easy sell (5% vs 33% ERS and 65% R-2000 (n=5)). Surveyed builders also reported generally low consumer understanding of housing labels (15% gave a rating of good/very good, 40% moderate, and 45% poor/very poor). Many external interviewees and users of OEE programming suggested that future promotion and marketing should be directed to the consumer and public to increase awareness of the benefits and therefore demand for premium labelling. Indeed the ESNH case study suggests that the weakest aspect of this label is that it is not yet sufficiently driven by customer demand to provide a market advantage. This is not to say that Canadians do not see benefits in labelling the energy efficiency of homes in the resale market. According to a 2013 survey of Canadians, 86% of those who were considering buying a home (n =184) thought it would  be beneficial to have a home energy rating or label when considering resale homes to buy.^{Footnote 74} Interviewees (including builders) suggest more publicity of the benefits of energy efficiency to consumers, and training for realtors as potential new avenues that might bring more strength to the brand.

Most builders and service organizations also noted that the number of best-in-class labels administered by non-government organizations (most of which use NRCan ERS) available in Canada are creating some consumer confusion, diluting the brand recognition of ESNH label to Canadians. There were reportedly 17 such labels in 2013, which has prompted others to come to the same conclusion.^{Footnote 75} That said, 61% of Canadians surveyed in 2013 considered the ERS for homes to be at least somewhat helpful at helping Canadians to be more energy efficient.^{Footnote 76}

Adoption of Energy Efficiency:

Finding: OEE infrastructure behind labelling energy performance has helped to voluntarily move the market towards higher levels of energy efficiency by giving consumers greater choice - strong brand performance of ENERGY STAR for equipment on consumer purchasing, and the use of home labelling programs by builders to build better homes.

The infrastructure behind the various home labelling programs has helped builders to construct and market energy efficient homes, and NRCan’s ongoing support of this infrastructure is widely seen as important. Evidence from case studies and interviews shows that voluntary labels for new homes such as ENERGY STAR for New Homes (ESNH), R-2000and the R-2000 Net-Zero Energy Pilot provide a range of options to achieve energy efficiencies at least 20% and 50% better than what is regulated in the building codes. Most interviewees stated that NRCan has contributed to improvements in energy efficiency in the new housing stock. The ESNH labelling and promotion initiative contributes to increased energy efficiency in the new housing stock, mainly through its supporting tools and information that facilitate the ongoing development and introduction of new more efficient building technologies and practices. For example, OEE Performance Measurement data shows that ESNH accounts for over 50% of NRCan`s housing labels issued for new homes from 2010-11 to 2013-14. The full suite of NRCan labelling for new homes was perceived by interviewees as providing a familiar and easy to understand means of explaining and promoting energy efficient homes. Furthermore, interviewees also indicated while the R-2000 Standard made significant inroads for building science, ESNH made those innovations widely accessible for the average home buyer.

Available ENERGY STAR for New homes (ESNH) building practices provide consumers with energy efficiency options for new home construction, which has helped builders to promote such investments. Interviewees noted that ESNH was positively impacting consumer choice by facilitating the builders’ ability to market energy efficiency in new home construction. In particular, Builder Option Packages outlined in the ESNH standard were seen by all interviewees as essential to this effort because they provide clear options for consumers. While larger builders appear better placed to adopt ESNH labelling due to the scale of their operations, the proliferation of the label (strongest in Ontario and Saskatchewan) has led to wider adoption of more energy efficient building practices by smaller scale builders, and the associated trades (e.g., plumbers, framers, electricians, etc.). That is, the building practices advocated by these programs are improving efficiency in the new home construction market. Without NRCan, it was said that the level of expertise in the sector would be significantly reduced, as well as the progression towards higher efficiency.

ENERGY STAR has been used by utilities, retailers, and manufacturers to improve the performance of energy using equipment in Canada. Evidence from surveys, documents, case study and interviews all show that the ENERGY STAR core brand has contributed to energy savings for Canadians. Documents, interviews, and case study evidence all suggest that market penetration of ENERGY STAR is increasing and public awareness of the brand is high in Canada. The 2013 CEEA survey of Canadians shows that significant majorities (62-82%) have switched to energy efficient appliances in categories with ENERGY STAR labelled products.^{Footnote 77} While recent survey data on the influence of ENERGY STAR on consumer behavior are not available for Canada, results of a 2012 U.S., survey show that the label at least somewhat influenced the decision for 73% of the U.S. households that recognized the ENERGY STAR label and knowingly purchased an ENERGY STAR-labeled product. ENERGY STAR participants continue to realize energy savings for consumers through their rebate, in store promotional programs, and partnerships with manufacturers. Interviewed ENERGY STAR participants (including utilities, retailers, manufactures) estimate that they reach several thousand customers annually. Surveyed ENERGY STAR manufacturers and retailers in 2014 reported shipments of ENERGY STAR qualified products were for the most part the same or increased a little in 2013 compared to 2012.^{Footnote 78} Interviewees also report that sales of ENERGY STAR qualified products have increased year over year in most product categories.

Finding: EnerGuide Rating System for homes (ERS) is valued by partners, and is linked to energy efficient upgrades for existing homes, though value for labelling of existing homes is unclear for homeowners.

There is considerable evidence to indicate that the availability of the ERS system has supported energy efficient upgrades to existing homes. Almost all interviewees (utilities, provinces, energy service companies) indicated that the ERS enabled new and existing homeowners to improve their home’s energy efficiency by completing pre- and post- retrofit (renovation or maintenance) measurements. The 2014 survey of homeowners that received an ERS home energy assessment within the last 12 months found that on average, energy advisors recommended between 3 and 5 retrofits to each homeowner, and homeowners reported implementing between 3 and 4 retrofits. Energy advisors also recommended retrofits not previously considered for 27% of pre-retrofit respondents and 41% of post-retrofit respondents, most of whom implemented one or more retrofits they had not previously considered. Fifty-five percent of the pre-retrofit respondents who had not yet implemented retrofits said that they planned to do so over the next five years, while 31% of those who had implemented retrofits and received a post retrofit assessment said they planned to implement more retrofits over the next five years. After implementing retrofits, the top benefit cited by respondent homeowners was increased home comfort (82%), followed by reduced energy costs (67%), and improved indoor air quality (45%).

Homeowner decisions to implement retrofits appear to have been influenced by an energy advisor in their home: Many housing sector interviewees indicated that the home energy assessment report combined with a meeting with an energy advisor were central components of the Retrofit program that facilitated adoption of these products and practices. The ability to ask questions directly to somebody with expertise was explained as beneficial, and was followed by the receipt of a report that showed a graph illustrating the ‘before and after’ energy use if they were to implement the suggested retrofits. The homeowner could then base their decision to pursue the retrofits (or not) by comparing the cost of the work to the potential energy savings. The 2014 survey of homeowners that received an energy audit in the last 12 months corroborates this. For example, 69% of pre-retrofit respondents learned something new from their energy advisor and report, and 76% said it helped them decide which retrofits to implement. Similarly, these were 84% and 89% for those who had implemented retrofits and received a post-retrofit assessment.

The influence of utility and provincial incentive programs on homeowner decision to undertake an ERS home energy assessment to support energy efficient upgrades to existing homes is unclear. While there was a clear consensus that the federal ecoENERGY Retrofit program (discontinued in 2011) promoted a high uptake of energy efficient products and practices, since its end, labelling of existing housing through ERS has been on the decline (OEE Performance Measurement 2013-14 reports labelling for existing homes was off target and that cumulative targets for energy saved and GHG emissions reduced would be affected). The 2014 survey asked homeowners about the impacts of incentives on their decision to have their home assessed. For 60-70% of surveyed homeowners who received provincial or utility incentives towards either the cost of the home energy assessment, the retrofits themselves, or access to financing, they would have gone ahead with the home energy assessment regardless. That said, access to more grants was the most frequent suggestion for improvement among those who gave one (29-35% pre and post retrofit). Of the few interviewees who discussed the longer term plans for utility and provincial incentive programs, at least some of these programs appear to be winding down, or moving away from using the ERS label since it is no longer required as part of the Retrofit program. It is therefore unclear whether or not labelling of existing homes using ERS will continue to be sought without complementary retrofit incentive programs from the federal government, provinces, territories and utilities.

Factor influencing adoption:

Finding: Close alignment with U.S. EPA on ENERGY STAR specifications widely seen as a positive step, and allows NRCan more resources for brand management, an area where opportunities to improve were noted

NRCan has recently fully aligned ENERGY STAR product specifications in all but two product categories with the US. (fenestration products and electric heat recovery ventilators have unique Canadian specifications due to climatic differences). This was seen as an important accomplishment by external interviewees who are involved in the program in Canada as they expect it will decrease lag time for new product listing (a challenge identified by interviewed retailers and utilities), and free up resources for NRCan to focus more on managing the brand in Canada. In fact, while nearly all interviewed ENERGY STAR participants said that NRCan has done a good job at managing the brand so far, focusing on brand management was suggested by most interviewees as where NRCan should place its efforts going forward (i.e., greater engagement with participants and partners, supporting key messages at a national level, coordinating activities). Most interviewed ENERGY STAR participants reported that the tools that could be most helpful to them were those that helped to demonstrate to consumers (including companies) that the end use savings realized from ENERGY STAR products are substantial. These interviewees referred to availability of case studies, cost calculators (both available from NRCan, but interviewees not always aware of them) or aspects of labeling or other promotional materials that would help show cost savings.

Finding: ENERGY STAR and EnerGuide for energy using equipment have also facilitated the reach of new and existing housing label programs.

A central facilitating factor for the EnerGuide Rating System (ERS) and ENERGY STAR for New Homes (ESNH) home labelling systems identified by many interviewees was that ENERGY STAR and EnerGuide are already easily recognized brands in Canada for energy using products. Interviewees indicated that the EnerGuide Rating System was an effective metric to show enhanced energy savings via ESNH (as well as for R-2000 and R-2000 Net-Zero Energy Pilot). Some external interviewees also hoped that the anticipated update to the ERS would address concerns over consumer understanding of the rating.

Finding: Canadians are interested in energy efficiency labelling for new homes, but housing market characteristics impact demand, such that the longer term payback argument for energy efficiency is not a strong message.

Canadians appear broadly supportive of home labelling, but current market characteristics may be hindering interest. Consumers are aware of energy efficiency issues in their homes. The 2013 CEEA Gandalf survey (n = 1584) found that on home labelling rating, 61% felt that the ERS for houses label helps Canadians to be more efficient, that 89% think all new houses should be energy efficiency rated. However, 90% support tax rebates to do this, and interest to invest in energy savings in homes increased substantially if payback period could be reduced to 2 years.^{Footnote 79} Almost all interviewees indicated that consumers did not respond to a ‘dollars returned on investment’ reasoning. This may be related to contextual changes in the Canadian housing market. A few interviewees explained that many first-time home buyers are entering a market of quickly rising home prices. As such, buyers are less likely to remain in a home for more than 3 to 5 years, planning to sell and upgrade to a better location or bigger home. Interviewees suggested that this short-term occupancy coupled with persistent comparatively low energy costs in Canada, has likely reduced the perceived benefits of energy efficiency investments favouring more aesthetic upgrades or investments to improve resale value.

Dollar savings do not appear to be a sufficient driver on its own for improving home energy efficiency. Several studies of mandatory energy performance labelling in Germany found that despite high awareness, only a third of residents used this information when purchasing a home and that overall energy efficiency was not a criterion for home purchase decision-making.^{Footnote 80} Moreover, research conducted in US cities suggest that ENERGY STAR certification has no significant impact on sales prices, ^{Footnote 81} with the notable exception of California, a jurisdiction that is often ahead of the curve in terms of energy efficiency.^{Footnote 82} That said, Canadians are at least somewhat supportive of having all new homes (89%) and all existing homes (71%) rated for energy efficiency so that potential buyers would have that information; and, as already noted, most Canadians that were at least somewhat likely to buy a home (86%) feel that they would benefit from a label being included as part of the home buying process.^{Footnote 83}

Almost all housing interviewees contextualized the benefits of energy efficiency as broader than economic returns or energy savings alone, and suggested a need for promotion that does this as well, This sentiment is consistent with emerging international consensus, that the multiple benefits (health, well-being, reduced GHG emissions, etc. rather than only energy savings) that are interconnected with energy efficiency be part of the analysis.^{Footnote 84} Evidence from the 2014 survey of home owners also suggests that, while important, cost savings are not on their own a driver for improving energy efficiency of homes. The top reasons for participating in the program given by respondents (multiple responses accepted) were a mix of cost and non-cost motivations. This included 93% to learn how to reduce energy waste, or 90% to reduce costs, and 86% to improve home comfort or quality of life. These were notably more frequently mentioned than to get access to grants towards the cost of upgrades/retrofits (76%), or to get an EnerGuide rating to help increase the value of their house (44%).^{Footnote 85} Note that among those who had a post-retrofit assessment, improved quality of life/enhanced home comfort  was just as frequent as information on how to reduce energy costs (65-66%), and more frequent than reducing energy waste (57%), and gaining access to financing for upgrades/retrofits. In both populations, approximately half were motivated by doing their part for the environment (51-55%).

4.4 Codes, Regulations & Standards (Adoption of Regulations Outcome)

Adoption of Energy Efficient Codes and Standards:

Finding: Codes regulations and standards have a clear and sustainable impact on energy use once adopted and implemented

During the evaluation period, OEE undertook significant efforts in code and regulation development across the buildings, housing, and equipment sectors. For this evaluation, case studies were completed of each of these efforts: update to National Energy Code of Canada for Buildings (NECB) 2011, inclusion of energy performance in Part 9 of the National Building Code, and establishing or upgrading Minimum Energy Performance Standards for energy using equipment (MEPS) under the Energy Efficiency Act. Overall, case studies, interviews, and documents reviewed support the conclusion that these activities contribute significantly to energy savings and associated GHGemission reductions. Both building code efforts have shown strong early signs of adoption by authorities having jurisdiction, and several amendments to the Energy Efficiency Regulations have been produced resulting in energy savings, though there have been delays with publishing recent amendments to the Regulations. Interviewees across each of these initiatives concurred that codes and regulations are effective at saving energy because, once implemented, they remove the less efficient products from the market.

Finding: NECB 2011 and Part 9 National Building Code Updates successfully implemented, providing new minimum requirements for energy efficiency in new commercial, institutional and residential buildings signs of adoption in several jurisdictions and commitments to adopt in others

NECB updated and is being adopted by jurisdictional authorities: With the financial and technical support of NRCan, the National Research Council of Canada (NRC) developed the NECB 2011 to replace the 1997 Model National Energy Code for Buildings. The NECB 2011 provides minimum requirements for the design and construction of energy-efficient buildings and can serve to harmonize these elements among the provinces that adopt.^{Footnote 86} The first updated NECB was published in November 2011, and a second update scheduled for 2015-16 for which work is underway. The code is expected to cover at least 59% of new buildings in Canada, and according to OEE Performance Measurement 2013-14 Manitoba, Nova Scotia, BC, Ontario, and the city of Vancouver have adopted or adapted the code for their jurisdictions. Beyond that, evidence from interviews suggests that wider adoption is likely as all authorities having jurisdiction have made a commitment to adapt or adopt the code (except for the North West Territories where equivalent regulations are already in effect).

Part 9 of the National Building Code (NBC) includes energy efficiency requirements, and is being adopted by jurisdictional authorities: Evidence from the document review, interviews and the case study on the development of Energy Efficiency Requirements in Part 9 of the NBC indicate that the new section of the NBC containing the energy efficiency requirements for housing and small buildings was published in December 2012^{Footnote 87} and the code is ready for adoption by jurisdictional authorities. At this time, four provinces/territories have adopted the new energy efficiency requirements (BC, Nova Scotia, Newfoundland, Yukon), while most of those that have not adopted are going through a consultation process. Interviewees considered that awareness of energy efficient building will continue to increase with the adoption of these requirements by all jurisdictions. Evidence from interviews and the case study indicated that inclusion of energy efficiency in the NBC will foster broader changes by making the housing sector more aware of energy efficiency building practices. Some interviewees explained that these new requirements help provinces to set a minimum point of energy efficiency in their building codes, which has already fostered a set of changes in the field.

Interviewees agreed that the changes to the NECB and to the NBC would result in long-term energy savings across the country.

Finding: NRCan has historically been successful at setting Minimum Energy Performance Standards (MEPS) based on consultations and market analysis; implemented MEPS have been effective at saving energy in consumer products

NRCan has improved energy efficiency of equipment stock in Canada through the introduction of Minimum Energy Performance Standards (MEPS). During the evaluation period, OEE worked on four regulatory amendments to the Energy Efficiency Act (amendments 11, 12, 12b, and 13) and successfully implemented amendments 11, 12 and 12b to the Energy Efficiency Regulations, which resulted in new minimum energy performance requirements for a range of products. Interviewees consulted for the case study and more generally agreed that MEPS are a cost effective means of reducing energy use. They noted that removing the worst performing products from the market improves energy efficiency in equipment. It was said that they level the playing field for manufacturers so that nobody can significantly undercut on price by using the least efficient technologies. It is estimated that Canada’s energy performance standards from the first 12 regulatory amendments have resulted in a reduction of 26.03 Mt in aggregate GHG emissions in 2010 and energy savings of 185 petajoules.^{Footnote 88} Shipment data for new refrigerators, dish washers, and clothes washers reviewed for the case study show that with each new minimum energy performance standard regulation, there was an improvement in the average annual energy consumption of these appliances. As well, the International Energy Agencies’ product mapping analyses, comparing energy use in products across Canada reports that regularly revised MEPS appear to have been the most effective method of reducing consumption for washing machines and lighting, for example.^{Footnote 89}

Factors Influencing Adoption:

Finding: Codes, regulations and standards work well and are supported by OEE’s labelling programs that help signal to the market a move towards higher levels of energy efficiency.

ENERGY STAR has worked in conjunction with setting of Minimum Energy Performance Standards (MEPS) to improve energy efficiency of products sold in Canada. As of 2014, there are more than 65 product categories covered by ENERGY STAR, with 43 that are also regulated through MEPS. Some of these MEPS have been adjusted three or more times, and most with ENERGY STAR standards in their third generation (some as high as 6^th  – e.g., Televisions).^{Footnote 90} Evidence from case study interviews suggests that as MEPS get more stringent, ENERGY STAR is no longer needed, because the difference between the energy performances of these products is negligible assuming technology does not continue to improve. According to interviews with OEE and utilities, ENERGY STAR helps get the market ready for regulations or more stringent regulations where they already exist. It gives utilities offering incentives an exit strategy when the market has transformed, allowing them to focus on other product categories. . Interviewees explained that manufacturers are not interested in barely meeting a standard that they know will evolve in a few years, so, the combination of ENERGY STAR and MEPS can actually drive technology development as well.

Updates to the National Building Code and commensurate higher efficiency requirements for premium home labels also work together. Both the ERS and ESNH housing labelling case studies showed that updates to these standards are seen to be highly important to users in the provinces, utilities, as well as builders and energy advisors. The recent update to ESNH and the planned update to ERS that is expected to launch in 2015 were both seen as necessary given the new energy efficiency requirements in the National Building Code (also a product of NRCan collaboration). These labelling standards reportedly sensitize the market to more efficient products and practices, which then allows for increased stringency of the regulations presented in the National Building Code in terms of better energy performance. Moreover, the interplay of regulations and labels can drive technology development and building practice improvements (e.g. R-2000 must always be 50% superior to code) as there are manufacturers and builders that do not wish to be barely meeting regulation requirements given their potential to change.

Finding: NRCan efforts to consult have facilitated effective development of building codes and equipment regulations.

A key success factor identified across codes and regulation case studies was good consultation with those who must adopt, adapt, or work within the codes and regulations. External interviewees involved in the NECB and Part 9 NBC updates indicated that the high level of national consultation uniquely facilitated by NRCan was integral to the successful development of the codes.

• NRCan consolidated policy objectives for NECB: Efforts made by OEE to analyze and consolidate the policy objectives of the federal government and those of the jurisdictional authorities in a concise and integrated manner have guided the work of the Canadian Commission on Building and Fire Codes (CCBFC) throughout the development phase of the NECB 2011 and 2015/2016. This has ensured a good alignment between the NECB contents and provincial policy requirements on energy efficiency, which a House of Commons Committee deemed an important direction.^{Footnote 91}  Interviewees reported that this reduced duplication and make it easier for authorities having jurisdiction to adopt the code. This was contrasted with the less inclusive process that lead to the 1997 Model National Energy Code for Buildings, which was referenced by only two jurisdictions and became irrelevant in 2006 due to technology advances.
• NRCan educated stakeholders on energy efficiency to support buy in for the update to the NBC. As with the NECB effort, interviewees praised NRCan for its efforts in educating various stakeholders on the importance of energy efficiency when developing the energy efficiency requirements for the Part 9 NBC. In addition, most interviewees reported that the expertise (including OEE and Canmet ENERGY) and broad national outlook provided by NRCan contributed to developing the energy efficiency requirements in a more consistent and seamless way. Indeed, NRCan’s participation in the various code committees was seen as a key aspect of success.
• Consultations on MEPS adequately informed regulatory assessments, though some suggested a need to consider broader perspectives on regulations. The case study and broader interviews found that OEE analysis and consultation ahead of the implemented regulations had been sufficient to inform the Regulatory Impact Assessment Statement (RIAS), and considered appropriate information when developing equipment regulations. However, some external stakeholders (provincial, utility, and industry associations) beyond those consulted for the case study indicated that consultation with industry during the collaborative regulatory process could be improved with, for example, more attention to identifying possible market barriers and potentially broader consultation beyond what is available through the Canadian Standards Association (CSA) process.

Finding: Collaborative efforts and engagement of stakeholders by NRCan facilitated successful development of building codes

Most interviewees also praised the dissemination work performed by NRCan to raise awareness of the NECB and encourage adoption in the provinces and territories. The continuous support provided to jurisdictional authorities was appreciated as well as the seminars, webinars and meetings organised by NRCan. According to interviewees, tools and training provided by NRCan has contributed to target populations’ capacity to offer services that further promote the adoption of energy efficient practices in the building sector. An example includes the training given by the CAN-QUEST team to external consultants in the field to adapt the complementary CAN-QUEST^{Footnote 92} software according to their needs.

NRCan funding for codes and regulations seen as critical to successful implementation: For both the NECB and Part 9 NBC updates, NRCan provided the largest share of funding to facilitate the processes needed to develop the new codes. For NECB, NRCan provided and is providing the largest share of the funding for developing the NECB. NRCan also provided significant financial support (80%) to fund a portion of committee meetings, staff time and consultant reports for the Part 9 NBC update.

Flexible compliance paths built into the NECB update, which are aligned with international best practices, facilitated adoption: Another key factor in successful update to the NECB was the flexibility built into the code. One of the challenges with the 1997 MNECB, was its lack of flexibility. The decision to allow both prescriptive route which dictates minimum characteristics for building elements (e.g., wall insulation, thermal characteristics, etc.), the code includes “Trade-off Paths” and a “Performance Path” to ensure that compliance is respected, even if this is done through alternate solutions. This kind of flexibility is consistent with code development practices that have successfully been employed internationally.^{Footnote 93} A similar compliance route has been built into the Housing ESNH update, that includes a performance path and prescriptive path for builders (called Builder Option Packages). In both case studies, partners saw this as an important accomplishment and also having good potential to foster builder innovation in energy efficient construction.

Finding: Code adoption was facilitated by having complementary support tools available, though challenges identified with timeliness of NECB collateral materials

Collateral tools to support implementing the NECB were seen as a good practice, though most interviewees raised concerns about the timeliness of these tools relative to the publishing of the code. NRCan developed tools to support implementing the NECB, which was considered by all to be a good practice for facilitating compliance. Although the code has to be complete and officially published by the CCBFC before collateral tools can be ready (it would be costly to redevelop tools for any changes made prior to publication), tools for the 2011 NECB are still being produced. The NECB was published in November 2011, the guidelines were not until Dec 2012, software in Nov 2013, and the user guide was ready for publication in June 2014. The NECB is due for a planned update in 2015-16. The chief concern raised by almost all interviewed collaborators and users in the NECB case study was the delay in issuing the collateral material. The long lead time for the code’s supporting materials, namely the CAN-QUEST software and the User’s Guide, was said to have hindered implementation of the code. Similarly, interviewees pointed out the release of a Beta version of CAN-QUEST building energy modeling tool as a best practice helped provincial/territorial authorities to implement the code. External interviewees wished to see more timely delivery of the supporting materials for the 2015 NECB update.

OEE partners involved in the Part 9 NBC update developed tools to support implementation: The knowledge developed through the new requirements for Part 9 of NBC enabled industry association and groups (e.g., Canada Wood Group) to develop tools to help builders determine how to construct in compliance with these energy efficiency requirements. In addition, manufacturers have begun to include tools developed through ESNH and used in the Code in their products (e.g., creation of a work sheet with effective R-values), thus further enabling users to better implement, more energy efficient practices into their work. In fact, it was often noted that NRCan leveraged its existing relationships with builders, energy advisors, utility and provincial regulators from its work on voluntary R-2000 and ESNH labels, to ensure the energy efficiency requirements responded sufficiently to the needs of those who would have to implement it.

Finding: Coordination challenges identified by partners due to concurrent development efforts on NECB and Part 9 NBC

Note that the simultaneous updating of the Part 9 NBC to integrate energy requirements for houses and small building with the update exercise for the NECB was seen by interviewees in both case studies as a challenge. Interviewees reported that because many of the same people had to be involved in both processes  - each of which required considerable time investments in terms of meetings (as many as 75 meetings for Part 9 NBC) and drafting related to the codes -, several members of the task groups  in both code development processes were pulled away from work to participate in the other exercises. It was felt that this made the updating process inefficient, and some participants suggested a more coordinated approach could have helped.  

Finding: Challenges identified with NRCan’s ability to align MEPS with regulations in the United States for similar products, which has posed challenges for manufacturers and has impacted energy efficiency targets.

Alignment of MEPS in North America seen as an important aspect of effective regulations, while last minute changes to the regulatory schedule have resulted in delayed ability to align standards: During the timeframe covered by the evaluation, Canada had an informal policy of alignment on regulations with those of other jurisdictions. It appears to be widely understood that such alignment is important for achieving energy efficiency, and for minimizing negative impacts on Canadian manufacturers. In fact, in 2009, the Energy Efficiency Act was amended to require NRCan to report on the stringency of the federal standards relative to other jurisdictions, like the U.S. and Canadian provinces. . In August 2014, under the Regulatory Cooperation Council, Canada announced a formal policy of alignment with the United States.^{Footnote 94}  Interviewed provinces, industry associations and manufacturers expressed concerns with the recent delays in NRCan’s regulatory schedule after Amendment 11. Amendment 12 and 12b were not part of the regulatory agenda and were introduced to delay by two years the efficiency standard (in Amendment 10) for general service light bulbs^{Footnote 95},^{Footnote 96} in response to public and industry concerns over the availability of the technologies to meet the standard. Most interviewees concurred that this was a necessary adjustment, and to NRCan’s credit, was quickly developed. However this amendment has, according to interviewees and the Regulatory Impact Assessment (RIAS) for Amendment 11, put Canadian regulations that were to align with the U.S. on 16 product categories in Amendment 13 out of alignment with those in the U.S for two years.^{Footnote 97} Indeed, Amendment 13 has yet to be implemented. According to OEE performance measurement, OEE is off target on associated achieved energy savings and GHG emission reductions because of this delay.

Interviewed manufacturing associations noted that the reasons for the delays or the timing under which they can expect the regulations to go forward have not been communicated. This has logistical implications for testing their products, and use of the mandatory EnerGuide label, all of which have to be coordinated to comply with the Energy Efficiency Act. As well, there are short term competitiveness issues experienced by those manufacturers who begin selling equipment that meets the coming standard too far ahead of when an announcement is made. It was also explained that this leaves Canadian marketplace vulnerable to stockpiling less efficient products that have been phased out in more stringent markets.

The current fast pace of equipment regulations in the U.S. is likely to continue, with risks that NRCan will fall further out of alignment. There is a risk that the regulatory delays and resulting misalignment over the last three years will be compounded in the near term, as delays in publishing Amendment 13, have now pushed proposed Amendment 14 further out of alignment. Evidence from external interviews suggest that the regulatory and labelling program in the United States will continue at a fast pace at least in the next few years, potentially widening this gap. OEE has used flexibility within the CSA process to change a standard that is referenced in the Energy Efficiency Regulations. This allowed OEE to align quickly on two critical product categories as a stop gap measure. However, industry and CSA representatives stressed that this was a needed but temporary solution, that does not achieve alignment as directly as regulatory amendments would. Both the ENERGY STAR and Equipment Regulations case studies highlighted the importance of alignment in these areas. In the ENERGY STAR case, harmonization was a key feature of program changes to program delivery over the last three years. Alignment of regulations across jurisdictions is broadly seen by interviewees as a positive direction for OEE as it creates economies of scale for manufacturers and consumers by reducing the testing burden, and providing provinces with strong federal standards to use as reference in their local energy efficiency standards. It also limits the stockpiling of the least efficient products ahead of coming regulations.

4.5 Energy Management, Monitoring, Rating Tools (Awareness & Capacity Outcomes)

Awareness/Capacity:

Finding: OEE has successfully brought US EPA Portfolio Manager and SmartWay tools to Canada, with early observed uptake among target groups

As part of efforts to align with the United States, NRCan worked with US EPA to bring two US EPA energy management initiatives to Canada for use by Canadian companies: 1) The Portfolio Manager energy benchmarking tool, which commercial and institutional buildings managers can use to track and benchmark their buildings’ energy performance; and 2) The SmartWay Transport Partnership Program which creates a market incentive and access to information to encourage the uptake of fuel efficient practices and technologies in the freight sector by connecting shippers with an interest in reducing their environmental impact with carriers in Canada. NRCan began a process in 2011 to adapt these tools for use in Canada, with official Canadian launches in 2013 and 2012 for Portfolio Manager and SmartWay, respectively. Both Portfolio Manager and SmartWay were active in the United States for over a decade with many Canadian users (over 4000 Canadian buildings entered into Portfolio Manager and over 285 SmartWay partners prior to launch in Canada).

SmartWay is meeting targets for carriers (trucks) registered to the Canadian tool, though few shippers have registered: As of March 31, 2014, 18 months into its launch in Canada, there were 12, 205  registered trucks in SmartWay delivered by NRCan, some of which are associated with new partner registrants.^{Footnote 98}  However, only 2 shippers were registered in Canada at that time.^{Footnote 99} Carriers registered in SmartWay delivered by NRCan have access to both Canadian and US SmartWay registered shippers (255 registered). However, evidence suggests that shippers are an important driver of the program. Case study interviews suggest that Canadian shipper representation is important to motivate participation of both carriers and other shippers in Canada. Evidence from interviews does suggest that shippers not registered to the program are using it to varying degrees in their procurement of carriers. For example, case study interviewees reported that some non-registered shippers are asking for SmartWay registered carriers in their RFPs. Fifty-four percent of surveyed SmartWay carriers in 2014 (n = 99) noted that they joined the program because shippers they wanted to deal with required it.

Increasing uptake of the Portfolio Manager tool has been observed since its launch, though 1-100 benchmarking scores are not yet available for all building types, but are on track. Buildings are being registered to Portfolio Manager across all provinces and territories, with most of the activity concentrated in Ontario (most populated province, i.e., most buildings) and British Columbia (strong provincial emphasis and stringent building regulations for energy efficiency). In fact, according to OEE Performance Measurement the program has greatly exceed its 2016 target of 20-27 million m² of floor space benchmarked, with approximately 114 million m² of floor space across 7300 buildings benchmarked as of June 2014 due to higher demand in the market for this type of information than anticipated. However, a significant portion of this floorspace (25 million m² ) was only benchmarked against itself year over year, controlling for weather, as the program has not yet developed a “1-100” rating for all building types, although it is on track to do so. So far 1-100 scores are available for three building types, and work on a further three is underway (on track to meet target of six for 2016). This is consistent with the approach taken by US EPA to develop additional scores as the tool becomes more established in the marketplace. It is not known whether or not benchmarking in this way results in differing impacts than benchmarking against the statistically generalizable data.

NRCan worked collaboratively with EPA and engaged users of the tools to ensure a smooth transition to Canada. Evidence from case studies and interviews demonstrates that OEE worked closely with US EPA to bring these tools to Canada and that that collaboration has been very good. NRCan also worked to minimize the impact of the transition on affected users (i.e., Canadian companies registered to the US versions of the tools). The strategies used by OEE to manage the transition were considered efficient and effective by all involved. For example, the Portfolio Manager team did a sample transition of users and shared the results with them before developing situation specific messages to deal with possible benchmarking score changes. In March 2012, OEE sought input from the freight industry on how to effectively introduce and sustain SmartWay in Canada and held 4 workshops with carriers (3 with US EPA SmartWay members and one with non-members) and 2 with Shippers unfamiliar with SmartWay and conducted interviews with shippers that were U.S. EPA SmartWay members. Interviewed SmartWay users report that OEE Partner Account Managers have been very helpful in responding to questions and concerns and following up about using the Canadian registration tools.

Adoption of Energy Efficient Practices

Finding: Early signs of success for newly implemented energy management activities include users’ ability to communicate energy use to senior managers, and to think about options for reducing energy use.

Early benefits of Portfolio Manager scores are the facilitation of senior manager understanding of energy performance and ability to identify opportunities to reduce energy use, though impact on implementing energy efficient practices is unclear at this time given the recent launch in 2013: The Portfolio Manager case study observed some early benefits of the tool such as increased buy-in by stakeholders, and increased ability to identify opportunities for energy improvements among users. However, only one example was given where the use of Portfolio Manager concretely led an organization-wide implementation of an energy efficient practice (i.e., a lighting upgrade). Interviewed users reported that one of the key early benefits of this tool is that it gives a quick and simple number (a 1-100 score) that is very helpful in talking to their senior managers to think about energy efficiency (compared to traditional measures such as building performance index, kilowatt hours, etc…). In addition, the ability to track and compare information at the micro level (e.g., building by building across a large campus or factory) helps pinpoint exactly where improvements can be made. As the tool launched less than a year ago in 2013, detailed energy use analyses have not yet been conducted by OEE but are planned. It is early to expect to see many implemented energy improvements from the use of Portfolio Manager. However, U.S. EPA results show that between 2008 and 2011, users of Portfolio Manager averaged total savings of 7%.^{Footnote 100} Evidence from the case study suggests that the influence of benchmarking on the decision to adopt energy efficient practices is not fully understood. Case study interviewees noted that while Portfolio Manager is useful, various factors related to regulations, availability of incentives, and the cost benefit and ability to realize energy savings in the short term play a role as well.

There is evidence of early success from SmartWay in Canada at encouraging improved fuel efficiency, mainly through changes to driver behaviour. Users representing SmartWay registered shippers and carriers reported learning about the factors that affect fuel use, their company’s annual fuel consumption by model years and their emissions measures since registering. This information coupled with the resources available on the SmartWay website have contributed to an increased capacity for some interviewed users to understand the  fuel use in their operations and think about options to improve performance. Most changes identified by interviewees had to do with behaviour of drivers and route planning rather than implementing new technologies.

Analysis of carrier registration data indicates that some adoption has occurred which has been attributed to partners’ participation in SmartWay delivered in Canada. OEE Performance Measurement reports that these changes resulted in participant fuel consumption decreasing by 1.34% as of March 2014,^{Footnote 101} resulting in a cumulative 3 year total of 0.24 PJ of energy saved.^{Footnote 102} The survey of carriers suggests SmartWay influenced these savings with 72% of surveyed carriers (n = 99) in 2014 reporting that their membership in the program influenced to some extent their decision to implement fuel saving practices or products in their fleets. The majority (55%) also noted that the benchmarking report provided by OEE was at least somewhat useful in improving their energy performance, with 44% using it to compare their year over year performance. Overall, 69% of registered carriers were satisfied to very satisfied with their NRCan SmartWay experience, and 90% plan to re-enroll their company next year.   

The support for Energy Management Systems Standards (i.e., USO 50001) in Canada has also shown good progress. OEE provides support to implement Energy Management Standards in Canada, in particular to support the ISO 50001 energy management systems standard since 2011,^{Footnote 103} which establishes an energy management framework for all types of organizations. NRCan supports EMIS implementation within industry through financial support (up to 50% or $40,000 to implement energy management pilot projects or undertake process integration studies), capacity building (training and research support), and networking and collaboration. At the time of the evaluation there were 11 ISO 50001 certified plants in Canada, most of whom pursued certification with NRCan assistance.

NRCan support for Energy Management Systems has helped participating companies to communicate and implement energy savings practices with realized energy savings. All companies interviewed for the Energy Management System Standards case study (EMSS) are benefiting from the data management tools that NRCan offers (e.g., energy management calculators such as RETscreen and Energy Management Information System planning manual and tools) as a basis for both making the case for the inception of energy management projects and their ongoing monitoring within these organizations. Indeed, interviewees underscored the value of being able to measure energy savings with tools (e.g., energy management calculators) for both rationalizing energy management projects (e.g., benchmarking against self or others) and continuing to monitor results, which is presented to senior management. Interviewees characterized the cost shared assistance provided by NRCan to implement ISO 50001 as catalyzing, particularly for smaller and medium sized enterprises. While the funding was not seen to be overly high, it was in many cases sufficient to interest organizations in pursuing the standard, which interviewees note has changed how they think about energy management and enhanced capacity to adopt the standard particularly for small and medium enterprises (SME).

Interview data suggest a strong association between the increased capacity resulting from ISO 50001 and associated EMSS supports and significant savings in energy costs and reductions in energy usage across all seven companies examined through the EMSS case study. For example, two companies stand out as having reported savings of between $150,000 and $500,000 annually (representing a 19.2% drop in energy use), with one company reaching savings of $1.5M annually. These savings are anticipated to increase annually with sustained investments in energy management.

Federal Buildings Initiative (FBI) has realized energy savings for federal departments, though challenges experienced with implementation. NRCan also provides energy management support for federal buildings. This voluntary program developed and administered by the OEE helps facilitate energy efficiency retrofit projects in buildings owned or managed by the Government of Canada. It enables federal organizations to implement these projects through third-party energy performance contracts (EPCs) without necessarily using their own capital funds.^{Footnote 104} In response to renewed interest in FBI as a result of the FSDS in 2010, the FBI launched a Community of Practice in 2010 that aims to raise awareness of best practices in energy efficiency amongst federal property managers, environmental managers and operators of federal buildings.^{Footnote 105},^{Footnote 106} Interviewees taking part in these community of practice sessions stated that participation allowed them to see what other departments were doing in terms of energy efficiency and raised their awareness overall. This is consistent with the approach now being taken in the United States, to share best practices among federal agencies that undertake or plan to undertake retrofits. Federal Buildings Initiative is on track for initiating departmental projects, however, the actual adoption of retrofits is off track. According to OEE Performance Measurement data, the Federal Buildings Initiative is on track for the initiation of departmental projects (i.e., opportunity assessments, energy performance contracts, energy master plans) with 15 projects underway as of May 2013.^{Footnote 107} However, despite this increased awareness in FBI it appears that a number of contextual restraints have made it difficult for this program to meet its energy retrofit project targets, with just five energy performance contracts signed out of target of 20 by March 2016.

The general consensus from interviewees in other federal departments was that when energy performance contracts had been implemented, they resulted in significant energy savings. OEE Performance Measurement calculations suggest that the 82 projects implemented since 1994 have resulted in $329 million in investments, saving $45.5 million a year in operating costs. ^{Footnote 108}

Factors Influencing Adoption:

Finding: Evidence from case studies and interviews identified opportunities to improve the longer term delivery of energy management and monitoring tools, such as user capacity and federal commitment to maintain Portfolio Manager, and ability to connect carriers and shippers through SmartWay

Data entry challenges with new Portfolio Manager and SmartWay tools: The case study suggested that Portfolio Manager in Canada is facing some challenges regarding user capacity. The main barrier for users was the learning curve in understanding the technicalities of data entry (e.g., entering thousands of buildings at once, dealing with buildings in a campus situation, use of historical or template data, manual data manipulations). OEE has helped to shorten this learning curve by offering webinars and workshop on Portfolio Manager. As users become more familiar with the data entry process user capacity is expected to increase. Some users and industry representatives interviewed for the SmartWay case study believe strongly that for some potential participants, the tools may require significant effort/ resources to mine the data required to complete it which may negatively affect their motivation or ability to participate. This may be especially true for some very large shipping firms that have complex freight movements and/or do not have compatible data management capabilities, and for medium and small companies that may not have the capacity to collect the data to populate the tool.

Concerns were also raised by users over federal commitment to Portfolio Manager beyond 2016, and suggested that this was a deterrent to using it. Users noted that while Portfolio Manager is a valuable and cost effective tool for them to use, the value of the investment in the tool will decrease if there are no resources to regularly monitor and update it. Similarly, the availability of 1-100 scores for further building types was a concern to some users. However, so far OEE has met its targets for types of building with 1-100 ratings. Interviewees expressed strongly that some form of commitment to the tool is needed.

Finding: Uptake of energy management tools, influenced to varying degrees by perceived value to those other than the target group such as requirements in other jurisdictions (Portfolio Manager), and importance of the tool to clients (SmartWay).

Some provinces and cities in Canada are considering mandatory Portfolio Manager benchmarking to support adoption of benchmarking. A number of interviewees knowledgeable of Portfolio Manager noted that because energy benchmarking is voluntary, those organizations that do it are the same organizations with the voluntary will to implement efficient practices. To encourage adoption of benchmarking, some provincial governments are moving towards policy that ties funding to minimum energy use requirements. Similarly, LEED certification for existing buildings has always required an energy score from Portfolio Manager, and now uses the Canadian scores. Interestingly in the US, the cities of Minneapolis, New York, San Francisco, Seattle, Boston and Washington DC have mandated energy benchmarking and reporting (typically via Portfolio Manager). The cities of Toronto and Vancouver are considering mandatory energy benchmarking for city buildings using Portfolio Manager, thus are keeping an eye on what happens in the US.^{Footnote 109} 

Benchmarking is important to the supply chain industry and users report the SmartWay tool is valuable because it can quantify emissions by model year, track performance year over year, help to understand factors that affect fuel use, and is non-proprietary. That said, there is interview evidence across groups to suggest that the value proposition to join SmartWay may not be evident and/or clear for some shippers and carriers, particularly for 1-2 truck owner-operators. Many external and internal interviewees felt strongly  that more promotion to engage both shippers and carriers is needed. Some industry interviewees pointed out that while only launched 18 months ago, awareness of both the Program and of its benefits, particularly among shippers, is low or could be improved, based on the fact that they themselves inform shippers and carriers of its existence. Evidence suggests that shipper engagement is an important driver of the program in order to motivate participation of both Canadian carriers and other shippers. Currently Canadian-based shipper registration is low and there was general consensus among internal and external interviewees that the Program needs more. According to interviewees and documents, registering shippers and carriers to SmartWay is also challenged by the current driver shortage and 5 year high cost of fuel which makes SmartWay membership less important in shippers’ decision making about carriers than availability of trucks, use of multi modal, and cost. A similar finding was reported in the Green Shipper Survey carried out by the SCMA for NRCan in Feb 2014^{Footnote 110}.

Finding: Availability of resources and degree to which energy efficiency or energy management is a priority for senior managers were noted as key factors influencing the adoption of energy efficient practices across OEE energy management, monitoring and rating initiatives.

One of the early benefits cited for SmartWay, Energy Management Systems Standards, and Portfolio Manager was the ability to better communicate environmental impact, energy use and potential for energy savings to senior managers and investors. Nearly all surveys of target audience groups and interviews with industry and federal departments conducted for this evaluation suggested that management priority and cost were the two main challenges for implementing energy efficient products and practices, and are ongoing barriers across sectors. For example, some industrial and buildings interviewees explained that energy efficiency can be hard sell because of the up-front costs and short term horizons of senior managers.

• Surveyed SmartWay carriers (n = 99) reported main barriers to adopting energy efficiency are budget challenges (58%), uncertainty over return on investment (38%), and uncertainty over benefits (32%).
• Companies interviewed as part of the Portfolio Manager, CIPEC and EMSS case studies generally reported that energy management investments pay themselves off, with most citing a payoff of 2-3 years or more. Senior management buy in is important, especially for investments that will take longer to pay off. EMSS and CIPEC case studies have highlighted that in companies where energy management issues are discussed by senior managers, there is greater implementation of energy efficient practices (e.g., among CIPEC leaders).
• This challenge is also present for the Federal Buildings Initiative, where the energy performance contract (EPC) process is not easily accepted by target departments. The process is perceived to be too costly, and returns may not accrue to the department paying to implement the retrofits depending on who manages the building.^{Footnote 111} Together this makes it difficult for secure senior manager commitments to implement EPCs. A review of information on the US program to improve federal building energy performance also suggests that the complexities of working with EPCs are difficult for federal departments to manage with available resources. Two key differences between the US program and the FBI are the funding committed to federal building retrofits under the Presidential Directive,^{Footnote 112} and a requirement that agencies undergoing retrofits hire a qualified project facilitator. Interviewees also suggested that NRCan could “set an example” by undertaking more retrofits or energy management strategies of its own buildings and then actively promoting those projects with other federal departments (of eight large projects implemented between 2009 and 2014, only one is within NRCan.^{Footnote 113})

Finding: Opportunities identified to improve the combination of activities that support energy management and monitoring tools once implemented

Implementations of Energy Management Systems Standards are well supported through CIPEC suite of tools and information, though gaps noted in support available for facilities managers. The EMSS case study demonstrated that the combination of supports available through CIPEC and industry sector programming more generally to support implementation of EMSS have worked well. For example, while the ISO certification itself provided a framework for managing an organization’s energy projects, the associated Energy Management Systems (EnMS) tools provide by NRCan offer a valuable means to continue implementing the ISO standard and related energy management projects. However, an ongoing gap was noted in the coaching and consultative support for facilities managers in Canada needed to sustain the ISO standard’s implementation. In some cases, energy managers from the provincial utilities serve a support role in assisting to move projects forward by helping with equipment efficiency and conducting audits. It was recommended by some interviewees that NRCan consider a similar model or how to integrate that with NRCan offerings.

Evidence from evaluation research on SmartWay also suggests opportunities to improve uptake of energy efficient products and practices by integrating additional supports to SmartWay. Interview evidence from representatives of industry associations and some carriers indicate that companies are not confident in which technologies are energy efficient, reporting that this is either because of a lack of testing (companies are constantly marketed with tools to reduce fuel consumption) or a perceived lack of transparency in the testing methods used for the products listed in the U.S. EPA SmartWay Technology Verification Program.^{Footnote 114} External interviewees would like to see the NRCan SmartWay website endorse and rate validated/certified technologies. Evidence from the carriers survey (n = 99) and interviews suggests that there are likely opportunities to better connect SmartWay carriers with other existing OEE activities as well. For example, given descriptions of NRCan FleetSmart and Fuel Management 101, 60% of carriers expressed interest in more information on both of these followed by on-board recording devices (42%), and other technologies. Among those who already had electronic on-board recording devices (n=53), almost half indicated using these to monitor driver performance, give corrective feedback, or administer driver incentives.  FleetSmart SDHT training was found to work well with combinations such as this. Interviews with SmartWay carriers confirm that when thinking about fuel management and SmartWay, driver behaviour is one of the key factors they are considering. Indeed, most survey respondents that provided a suggestion for improving the program suggested tools/ information to help get drivers engaged and on how driving behaviour affects fuel economy.

Evidence from case studies and interviews suggests that Portfolio Manager, Federal Buildings Initiative, and ENERGY STAR suggest opportunities to further coordinate. Evidence from case studies and interviews on Portfolio Manager, the Federal Buildings Initiative, and ENERGY STAR products, suggests that there are opportunities to enhance impacts through collaboration between OEE sectors when reaching out to public and private organizations to discuss energy management opportunities. For example, both Equipment (manages ENERGY STAR) and Buildings divisions (manages Portfolio Manager and FBI) deal with Public Works and Government Services Canada Office of Greening Government Operations on energy management issues in federal buildings. Equipment division has also experienced challenges incorporating ENERGY STAR into federal public sector National Master Standing Offers. While no specific overlap has been identified, it is worth noting that both groups as well as PWGSC representatives have had difficulty securing commitments form public sector organizations on energy management.

Documentary and interview evidence suggests that Portfolio Manager may require the support of other tools and elements in order to bring about adoption of energy efficient practices. The Buildings division provides much information on energy management in buildings through the NRCan website and Dollars to $ense training. However, several interviewed users of Portfolio Manager as part of the case study saw a need for additional information on next steps after benchmarking, which suggests that they may not be aware that OEE has this support available for them. This may be because the tool does not explicitly link registrants to information on how to improve their building’s energy performance, although various OEE initiatives provide different components of the potential solutions. In the case of Portfolio Manager in the United States and Energy Performance Certificates in the UK and Germany, registrants are provided with information on how to improve their ratings, a factor that likely improves adoption of energy efficiency. The EPA’s Annual Portfolio Manager Report for 2011 suggests that implementing comprehensive retrofits and ensuring high performance among rated building requires both a senior management commitment and the development of energy management strategies as crucial elements. This suggests that while the rating is a very important first step, ensuring the availability of information on how to improve energy performance may be required to bring about behaviour change.

4.6 Final Outcome: Energy savings and associated reduction in GHG emissions and economic benefits

Finding: Evaluation evidence corroborates the expected link between OEE activities and adoption of energy efficient products and practices.

Target groups across sectors report energy savings attributable to varying degrees to OEE activities. Given the observed progress on adoption of energy efficiency products and practices at the intermediate outcome level, it is reasonable to expect that OEE is meeting its energy savings targets as described below in the program’s performance measurement data.

Table 3: OEE Performance Measurement Energy Savings and GHG Reductions 2011-12 to 2013-14

	Cumulative results* (as of March 31, 2014)	OEE Target * (as of March 31, 2016)	Reported Status * (as of March 31, 2014)
Energy Saved (all OEE programming)	24.24 PJ	36 to 44 PJ	On track
GHG Emissions Reduced (all OEE programming)	2.33 Mt	3.6 - 4.4  Mt	On track

Finding: Energy savings bring two corollary benefits – reduced GHG emissions and economic benefits. The evaluation confirmed that both relationships are real and well supported, both by the impacts identified in this evaluation research and by other studies and assessments of the impacts of energy efficiency.

A growing body of research confirms the relationship between energy efficiency and the economy. A 2014 report to the Energy and Mines Ministers’ Conference (EMMC) analyzing research on energy efficiency identifies many benefits arising from energy efficiency in Canada. The research suggests that improving energy efficiency in many cases provides a variety of economic and societal benefits, ranging from job creation to improved competitiveness. Given the strength of the evidence from this and other corroborating sources, these benefits should be taken into account to avoid understating the wide-ranging impacts of energy efficiency.

Savings to Consumers and Rate Payers: The report to the EMMC states that energy efficiency leads to lower energy bills, reducing the cost of living for consumers and businesses. The report refers to NRCan’s Energy Efficiency Trends data indicating that, in 2011, Canadian consumers saved more than $20 billion and business saved more than $14 billion in energy costs from all energy efficiency improvements made since 1990.^{Footnote 115} NRCan programming that contributes directly to consumer savings includes energy ratings tools for houses and equipment labels and regulations. While in many cases, non-behavioral energy efficiency measures involve upfront costs, evidence suggests that they result in a substantially greater reduction in the subsequent costs of purchasing energy. In particular, a 2011 analysis by the McKinsey Institute of the opportunities for energy efficiency estimates that, using the example of the United States, a $520 billion investment in energy efficiency measures (excluding program costs) would yield gross energy savings worth more the $1.2 trillion.^{Footnote 116}

Improvements to GDP and Employment: The report to the EMMC cites a recent study finding that, between 2002 and 2012, energy efficiency improvements in Canada increased GDP by about 1%, or $16 billion per year and added approximately 2.5% to overall employment.^{Footnote 117} This is consistent with a 2013 study that found that, within the OECD, energy efficiency improvements contribute positively to economic growth,^{Footnote 118} and with a 2012 review of studies of employment impacts of energy efficiency interventions that found significant job creation potential resulted from these activities within IEA-member countries.^{Footnote 119} Witness evidence presented to a parliamentary committee on the Future of Federal Buildings also confirmed that energy efficiency projects in the buildings sector stimulate substantial job creation, due to the large labour component of these projects.^{Footnote 120}

Benefits to Utilities and Governments: The report to the EMMC describes benefits of energy efficiency to utilities and governments, it reduces the need to build new energy generation capacity and related infrastructure, which is costly and faces social license challenges.^{Footnote 121} Interviewed utilities for this evaluation also noted that these were the principle reasons for running energy efficiency programs. This is corroborated by a 2007 U.S. National Energy Efficiency Action Plan study of impacts of energy efficiency investments on electric and gas utilities. The overall finding was that these investments consistently lower costs over time for both utilities and customers.^{Footnote 122} Evidence from an analysis of OECD countries also suggests that government budgets can be improved through lower expenditures on energy even for energy exporters, as reduced domestic demand makes more energy available for export.^{Footnote 123} NRCan experience with the Federal Buildings Initiative for example has shown that 82 energy efficiency retrofits, paid for through energy service contracts, have saved the federal government $45.5 million in annual operating costs since their implementation.^{Footnote 124} Similarly, the Government of British Columbia’s investment in 2008 of $75 million in capital funding to retrofit existing provincial public-sector buildings is estimated to be generating approximately $12.5 million in energy savings annually.^{Footnote 125} As well, a 2011 study of Federal government procurement policies using ENERGY STAR (based on government procurement data for 2007-08 to 2009-10), estimated annual savings of $5 million on energy costs.^{Footnote 126}

Improvements in Productivity and Competitiveness: Improved productivity and competitiveness is another benefit identified in the report to the EMMC. Energy Efficiency Trends data showing that, between 1990 and 2011, there has been a 23% improvement in the amount of goods and services produced per unit of energy used in the production process in Canada.^{Footnote 127} McKinsey (2011) analysis of an anthology of energy efficiency articles, found that the type, quantity and cost of energy used are key to the competitive advantage of 40% of businesses globally, particularly energy intensive industries, mechanical and plant engineering, information technology and the energy industry.^{Footnote 128} These are all targeted by OEE through CIPEC and its support for EMSS.

4.7 Economy and Efficiency

This section assesses the degree to which the program is an efficient means of achieving intended outcomes.

Finding: OEE effectively extends its reach by maintaining investments in tools that are levered by partners to administer their energy efficiency related programs using common national platforms

Reach of ENERGY STAR, EnerGuide Rating System is effectively extended: Given the reach that utilities, provinces, and manufactures have with Canadian consumers, providing tools to help them administer energy efficiency programs is a cost effective way for the federal government to achieve this goal. The leveraging by these groups of ENERGY STAR and EnerGuide Rating Systems is noted earlier in the report.

Portfolio Manager is used by provinces and industry associations for their own programs. Portfolio Manager is being leveraged by several very large and/or umbrella organizations in the buildings sector, such as the Canadian Green Building Council (CaGBC), BOMA, Civic Action’s Race to Reduce, and the provinces of New Brunswick and Manitoba. For example,  CaGBC’s core programs for existing buildings -  LEED Canada for Existing Building: Operations and Maintenance (EB:O&M) - uses Portfolio Manager^{Footnote 129} and the BOMA-BESt program has decided to align their  benchmarking program with  Portfolio Manager now that it has been implemented in Canada.^{Footnote 130}

SmartDriver for Highway Trucking (SDHT) and Auto$mart material is being leveraged by companies and provinces in licensing. Case study and documentary evidence reveals that companies with fleets leverage the SDHT curriculum and initial training they receive through OEE by delivering SDHT in-house themselves using OEE training materials. In addition, 8 provinces/territories have integrated a fuel efficiency chapter based on SDHT into their commercial (Class 1/A) drivers’ handbook and 4 provinces and 2 territories have included fuel efficiency questions on licensing exams. The integration of the fuel efficiency topic into the Class 1/A (commercial vehicle) licensing processes creates the opportunity to educate all entry-level professional operators with SmartDriver training messaging. Similarly, OEE has worked with provinces to include Auto$mart’s five key messages in new driver licensing tests  in Alberta, B.C., Manitoba, Ontario, Quebec, and Saskatchewan. Both private and public use of SDHT materials increases the reach of the SDHT messaging at minimal cost to NRCan.

Finding: OEE has levered cost sharing arrangements with partners to deliver on key activities.

Dollars to $ense workshops are delivered through “hosts” and 3^rd party bulk buying (e.g., Ontario Power Authority). NRCan interviewees indicated that cost-sharing has become increasingly important in the delivery of Dollars to $ense, such that the delivery cost for NRCan is now about 25% of the total cost, compared to 50% in 2010-11, with no reduction in registration. This cost reduction was the result of using alternative delivery models, including “hosted” workshops - where NRCan provides the curriculum and the trainers and identifies and engages potential “hosts” located across the country who are responsible for the logistics of the workshop, and a “third party bulk buying” model where third-party organizations buy a number of Dollars to $ense workshops and through a range of hosts, deliver it to others. Note however that case study evidence indicates that this “efficiency” gain has also further concentrated Dollars to $ense workshops in certain regions, particularly Ontario (i.e., from about 50% historically to about 75% in the most recent years), thus creating challenges for the OEE to ensure that there are sufficient trainers available in other regions in Canada. To support broader regional coverage, in December 2013, OEE identified 2 additional service delivery partners (which provide the trainers) to augment the existing three. Interviewee suggestions for helping to overcome resources challenges included exploring the option for Dollars to $ense online delivery models (e.g. webinar, e-Learning) to improve access and cost-efficiency, especially for remote locations.

NRCan cost shared the development and maintenance of standardized energy efficiency test procedures. Evidence from documents and interviews shows that test procedures, which were done by CSA in 2010-11^{Footnote 131} and 2011-12^{Footnote 132} and which are referenced by the Energy Efficiency Regulations^{Footnote 133} were cost shared largely with provincial energy authorities (Manitoba Hydro, Hydro Quebec, BC Hydro, SaskPower, Ontario Power Authority, Ontario Ministry of Energy). In 2010-11 NRCan, contributed about 41% of the total cost, and in 2011-12 the contribution agreement indicated that NRCan’s contribution was to be 50% of the total costs for the work.^{Footnote 134} The process was said to bring a broad based group of stakeholders and in kind contributions to the table to consider standards that can ultimately be referenced in regulations with board support.

The National Energy Use Database (NEUD) is cost shared across federal departments. NEUD is hosted and administered by OEE and contains data generated and analyzed by OEE. According to the program, there are cost sharing agreements between OEE, Statistics Canada and Environment Canada for costs associated with data collection of certain data sets and reporting such that in 2010-11, OEE provided approximately 50% of the data costs. This is a unique data set in Canada and is used regularly by all levels of Government to measure energy use and energy efficiency. Some interviewees suggested a need for technical improvements to the online database to make it faster and easier for users to query and extract data.

NRCan shared the Canada Border Service Agency (CBSA) tool to reduce the administrative burden of importing regulated energy-using products. NRCan (OEE) and CBSA collaborated in a pilot study from June 1, 2012 to 2013/14 whereby NRCan used CBSA’s existing Custom Self-Assessment (CSA) program as a mechanism for low-risk importers of energy-using products regulated under the Energy Efficiency Act,  to fulfill import reporting requirements. In doing so, the project reduced the compliance cost to low-risk importers of regulated energy-using products into Canada (e.g., reduced costs and delays associated with reporting at the border since the CSA Program allows for self-reporting subsequent to the crossing the border; and being able to use the same system to report for two purposes – customs and under the Energy Efficiency Act). In addition, NRCan efficiently built upon an existing compliance mechanism for low-risk importers reducing their costs, while at the same time reducing CBSA’s workload freeing up time to focus on high-risk import activities. The pilot project was so well received that it has been extended indefinitely.

Users see opportunities to improve CIPEC through enhanced coordination with provincial and utility programs: Interviewees also suggested that NRCan could do more to encourage the promotion of CIPEC, particularly in coordinating with provincial governments to improve complementarity of programming. These interviewees noted that given the capacity challenges facing industrial energy managers, coordinating efforts to use federal, provincial and utility programming is difficult. NRCan could provide a “one-stop-shop” for energy management nationally, which interviewees noted would be a considerable value add. To best leverage that role, it was recommended that NRCan proactively establish a leadership and collaborative model to engage both the utilities and provincial partners as part of supporting a national dialogue around energy efficiency. This could include linking federal programs to bolster provincial programs (for example, Nova Scotia is providing strong support for EMIS).

Finding: OEE has levered significant infrastructure and tools from US EPA and DOE to deliver their programs in Canada

NRCan benefited from US EPA’s Portfolio Manager experience (in U.S. since 2001) and access to systems, when adapting Portfolio Manager for delivery in Canada. According to case study evidence, the US EPA provided technical advice, expertise and guidance to NRCan, especially on software development and methodologies for calculating energy scores (i.e., the 1-100 rating). This support resulted in significant cost savings for NRCan. For example, early in the project the U.S. EPA was planning to upgrade the Portfolio Manager database, web services, and user interface of the U.S. version of the tool. By “piggy-backing” on this process, the developmental costs to NRCan were significantly reduced, and Program representatives estimate that it would have cost 3-4 times more if NRCan had developed the tool on its own. As well, by working together and using the EPA’s tool, it harmonizes the two countries’ approaches and provides convenience to companies with cross-border portfolios and avoids confusion in the market place (i.e., avoiding competing Canadian and U.S. tools).  

NRCan benefited from U.S. EPA’s SmartWay infrastructure and experience (in U.S. since 2004), when adapting the SmartWay Transport Partnership Program for delivery in Canada. Interview evidence from both U.S. and NRCan SmartWay groups indicates that NRCan took a leveraged and coordinated approach to implementing SmartWay in Canada which maximized opportunities to find efficiencies. For example, OEE uses U.S. EPA SmartWay’s program design and administration (e.g., registration tools and database, brand recognition, outreach documents, EPA staff for advice, and EPA’s SmartWay Technology Program information to recommend technologies to partners) and cost-shared the use of a consultant, to update the tools, to launch the program in Canada quickly and efficiently.

NRCan benefits from approximately $20 million in annual costs paid by EPA to administer ENERGY STAR (in U.S. since 1992) which frees up OEE resources for brand management and recruitment. NRCan leveraged U.S. EPA funding for core ENERGY STAR program architecture and product listing. Over the last two years, rather than develop uniquely Canadian specifications, NRCan has moved to referencing the US specifications for most product categories, which are an essential part of the program and have until recently been updated and developed for both markets at varying intervals (exceptions fenestration and Heat Recovery Ventilators). Case study evidence suggests that this improved alignment with the US will make it easier for partners to use the program, and decrease the effort required by NRCan to administer Canadian versions of the standards. Interviewed partners almost unanimously saw this as a positive direction, and suggested that NRCan could focus more on brand management in Canada. However many interviewees have noted that OEE should continue to monitor U.S. ENERGY STAR standards to ensure that only those that make sense in Canada are adopted.    

Finding: OEE has adapted to new fiscal constraints by leveraging relationships with industry associations to extend its reach to target groups. However, stakeholders across sectors expressed concerns on the degree to which NRCan is visible to target audiences and partners.

Communication and travel policies in current fiscal circumstances have impacted program delivery in some cases: Interview evidence indicates that the process for using traditional outreach mechanisms such as presenting and networking at sector conferences, events and tradeshows, and producing hard and soft publications, require significant resources to administer as well as lead time to meet the approval schedules. Those challenges, coupled with pressure to keep costs down, have hindered OEE ability to promote programs through traditional means of attending meetings and other functions to liaise with and engage stakeholders.

Several OEE programs work with industry associations to help promote programs and/or deliver key messaging. For example, the SmartWay Program has agreements with organizations^{Footnote 135} with an interest in greening the freight transportation supply chain, to encourage their members or customers to participate in SmartWay on behalf of NRCan. CIPEC has agreements with industry associations to present CIPEC information to their membership and promote CIPEC program offerings. The Canadian Autombile Association promotes Auto$mart’s five key messages on their website. The use and endorsement of Portfolio Manager by several large industry associations and provinces, and reference to it on their websites (e.g., CaGBA,^{Footnote 136} Efficiency New Brunswick) is a mechanism to reach those constituents. These types of promotion are typically through emails, newsletters, presentations at conferences, and via their web sites.

Interview evidence from external stakeholders suggests that NRCan can make more use of industry associations to reach and engage target groups. It was clear that NRCan has good relationships with industry associations across sectors because of long-standing collaborations, and there are advantages to partnering with these groups. Industry associations have reach and credibility among their membership, and they are able to extend NRCan’s reach effectively. However, the trade-offs of using this mechanism must be well understood and managed by the Programs. Several interviewed industry associations cited concerns that they have limited resources in terms of personnel, expertise, and funding for larger scale promotional efforts, and some interviewees suggested that NRCan must be careful to not lose control of its messages.

Evidence from interviews and several case studies suggest that external stakeholders appreciated the efficiencies of alternative communication mechanisms but are uneasy with what they perceive to be less engagement from NRCan in face to face meetings. Interviewees in the ENERGY STAR, Portfolio Manager and SmartWay case studies also stressed that for effective promotion, OEE staff have to be visible and present in order to engage and promote the programs, especially for newer initiatives where NRCan involvement may be part of the reason for uptake. A few external interviewees in the Portfolio manager case study noted a lack of presence of NRCan at industry events and forums or that they only knew of NRCan’s existence because of contact in previous projects. These concerns also presented themselves in OEE activities related to regulation and standards development. Some industry association, utility, and other government interviewees involved in standards setting noted that NRCan has been less visible in committees than in the past, attending fewer meetings face-to-face. Teleconferencing is perceived by stakeholders as a useful measure when used strategically, but for larger meetings on standards it was seen as less effective because of insufficient time to deal with issues and inadequate opportunity to raise issues or consult in side meetings. All interviewees agreed that while the communication through teleconferences, videoconferences and other tools were efficient and of great help all along the code development processes for NECB and Part 9 NBC, there is still a need for a minimum of face-to-face meetings and it is important to maintain that balance.

Finding: The addition of Portfolio Manager and SmartWay to OEE now means that there are a wide variety of benchmarking-type activities in use with various target groups, each with different cost effectiveness implications.

All OEE sectors use benchmarking-type activities, or the tracking of a performance indicator (such as energy consumption) for comparison between similar entities as a means of encouraging energy efficiency among target groups. There is considerable variability, however, in the approaches used to benchmark among the sectors. For example, some tools use nationally-representative data to arrive at benchmarking ratings (e.g., Portfolio Manager) while others use users’ data only (e.g., SmartWay); some rating are mandated (e.g., personal vehicles and some specific types of energy-using equipment as per the Energy Efficiency Regulations), some are not (e.g., Portfolio Manager, SmartWay, EnerGuide Ratings for houses, and benchmarking in the industrial sector); some benchmark ratings are published (e.g., for SmartWay carriers, personal vehicles, and energy-using equipment), some are not (e.g., for buildings, houses and industry);  some benchmarking activities are cost-shared with the users (sector benchmarking studies with industry) while NRCan covers the cost of the benchmarking tools in other sectors; some benchmarking activities offer tools and information on how to improve ratings (e.g., SmartWay and EnerGuide label for homes); and some are associated with provincial and utility incentive programs (e.g., retrofit programs for existing homes).

Evidence provided in this evaluation suggests that energy efficiency performance benchmarking is associated with improved energy performance. That is, measuring and comparing energy use has a relationship to behaviour change. As well, in particular for the US experiences with Portfolio Manager (7.1% annual energy savings in US buildings) and SmartWay ($16.8 billion in reduced fuel costs over 10 years) suggest the impacts can be significant. The approaches to benchmarking respond to the information needs of the different sectors. However, given the variation in the approaches (mandated versus non mandated, differing levels of quality of ratings, publication or non-publication of ratings, inclusion of informational tools in some cases to help improve ratings) and impacts of benchmarking observed in this evaluation, the mechanisms by which benchmarking actually facilitates improved energy efficiency is not clear. This suggests that there are opportunities for OEE to consider the reasons why and the circumstances under which  the various approaches to benchmarking that are used across OEE sectors are cost-effective approaches to improving energy efficiency.

5.0 Conclusions and Recommendations

The OEE Energy Efficiency Sub-Program activities are relevant. They respond to an ongoing need to improve energy efficiency in Canada in order to mitigate negative economic (i.e., competitiveness of Canadian businesses, and value for consumers) and environmental (i.e., GHG emissions) consequences associated with inefficient energy use. The Sub-Program clearly supports two of NRCan’s three strategic outcomes – #1 economic competiveness, and #2 environmental responsibility. Both of these are consistent with current Government of Canada objectives, agreements and policy drivers. NRCan is also uniquely able to deliver national platforms for improving energy efficiency that can be used by other jurisdictions, coordinating efforts and aligning with the United States where practical, and bringing neutrality and unique content expertise to information and training products.

Overall, the Sub-Program has shown strong performance during the 2009-10 to 2013-14 period under review. The evaluation finds that OEE has delivered most of its activities on track and as planned. Moreover, the evaluation found strong evidence that these have contributed to adoption of energy efficient products and practices, leading to improved energy efficiency (i.e., confirmed the theory of change). Target audiences across each sector reported energy savings due to OEE interventions.

Examples of key accomplishments include:

• Training and Information: Dollars to $ense and Smart Driver for Highway Trucking training participants report behaviour changes and implementing energy savings projects. CIPEC tools heavily levered by OEE target groups, realizing energy savings.
• Successfully supported use of U.S. EPA Portfolio Manager and SmartWay Transport Partnership, and implementation of ISO 50001 in Canada. Early signs of success include increased registration and reported energy use savings. Development (with NRC and CCBFC) of the updates to the National Building Code and National Energy Code of Canada for Buildings, with jurisdictional authorities adopting codes or committing to adopt.
• EnerGuide Rating System and ENERGY STAR program infrastructure continues to be heavily leveraged by partners to deliver energy efficiency programs and realize energy savings.

OEE programming is overall an efficient means for achieving savings in energy use by enabling the work of others and by leveraging resources of partners to participate in delivering key activities and promoting NRCan energy efficient programming. OEE has also levered considerable expertise and resources from others (e.g., U.S. EPA infrastructure for Portfolio Manager, ENERGY STAR and SmartWay).

Notwithstanding the strengths of OEE programming, the evaluation also identified risks to continued success and opportunities to improve outcomes from OEE activities. These, and the associated recommendations are outlined below.

Recommendation 1: Energy Sector should address challenges identified with uptake and use of some OEE products and tools by improving awareness and better explaining their value to target audiences. In some cases, Energy Sector should also enhance the value of certain tools and products.

---

Housing and Product Labels

The evaluation found that there is considerable value embedded in the infrastructure that supports each of NRCan’s labelling initiatives. That infrastructure is well established and highly valuable to OEE partners (especially utilities and provinces) in terms of providing consistency of approach in Canada, improving skills, and an easy means of establishing in-market incentive programs using a common platform in Canada. This is further supported by the use of regulations for equipment and housing that set new minimum standards in energy efficiency. Despite the high value of these programs at transforming the market there remains opportunities to improve consumer demand for several of the labels themselves.

There is weak consumer demand for housing labels in Canada for both existing and premium new homes. This is likely a function of characteristics of the Canadian housing market, where consumers privilege resale value and home aesthetic ahead of energy efficiency. Builders, provinces, and utilities identify a need to improve communication of the value of labels to Canadians, and to explore ways to create a market advantage for premium home labels such as ESNH.

As well, while the core ENERGY STAR brand continues to have high levels of market penetration and consumer awareness in Canada, the recently launched ENERGY STAR “Most Efficient’ premium labelling for equipment (a smaller sub-brand to ENERGY STAR) has had weak results to date, due largely to a lack of uptake among ENERGY STAR participants and low consumer awareness. However, recent consumer research suggests this label has strong potential in Canada.

SmartWay Transport Partnership:

While there has been good uptake of SmartWay, since its Canadian launch in 2012, and indications of realized fuel efficiency among registered carriers, evaluation evidence suggest that the value of joining the program is not always well understood (or present) for carriers and shippers. This is likely due to three factors.

• First, few shippers are publicly registered to the Canadian program (only two shippers at the time of the evaluation), which interviewees report weakens the motivation for carriers to join. The exact reasons why Canadian shippers are not registering is not known (though companies headquartered in the US register there and can be accessed by those registered in Canada), however the time and effort required to collect the data to register; low awareness of the program, the criteria for joining and its benefits in Canada; and the current shortage of carriers in North America, were each identified as possibilities.
• Second, carriers may be seeking more from the program beyond the energy benchmarking and tracking that it provides, although that service is clearly valued by registered carriers. Carriers report interest in more information on energy efficient technologies, and a need for NRCan to endorse/validate efficiency claims made by manufacturers of those technologies.
• Third, it appears to be difficult to communicate, in both qualitative and quantitative terms, the value of SmartWay to shippers who may not be self-driven for social responsibility or green branding reasons to join SmartWay.

Federal Buildings Initiative (FBI):

The FBI has experience renewed interest in the face of the FSDS, and has taken steps to engage with departments on pursuing energy management. However, implementation of energy performance contracts (EPCs) remains low due to difficulties securing commitments from departments. This challenge is widened by a lack of committed human and financial resources in federal departments to manage EPCs. These are similar to issues experienced in the United States Federal Government, where there is a similar initiative.

Canadian Industry Program for Energy Conservation (CIPEC):

The evaluation finds that CIPEC is playing a crucial leadership role in energy management, and that companies involved in CIPEC directly as CIPEC Leaders or because they have accessed CIPEC tools and training have experienced energy savings. According to the program, CIPEC’s reach extends to more than 2,400 registered companies in CIPEC’s Leaders network, covering 98% of Canadian industry through links with industry associations and a total of 5,800 participant facilities in the program. Nonetheless, there is a nearly unanimous perception among external interviewees (including in the case studies) that CIPEC should do more to reach out beyond those currently taking advantage of program offerings. These companies will continue to take advantage of CIPEC because they see value in energy management, however many saw a role for CIPEC to play in reaching out to those who have so far not been involved. Note that a key barrier to participation identified consistently in interviews and surveys across program areas was financial resources and senior management buy in. Programs like CIPEC help to give company managers the tools to make business cases for better energy management. As well, interviewed SMEs in the EMSS case study described the funding for pursuing ISO 50001 as a much needed catalyst to get the organization started down that path.  

Recommendation 2: Energy Sector should respond to timeliness issues associated with regulations and code development raised by OEE external target audience groups to ensure full energy savings potential from regulatory activities is met.

---

The evaluation found several areas where OEE could likely improve its impacts on facilitating improvements in energy efficiency through managing challenges related to timeliness.

MEPS and Personal Vehicle

OEE has had difficulty meeting its forward regulatory agenda for implementing amendments to the Energy Efficiency Act, which has implications for stockpiling of less efficient products in Canada, and the logistics associated with having different standards for Canadian manufacturers to comply with in North America. The delays began with the unplanned introduction of Amendments 12 and 12b in order to respond to public concerns over the availability of enough technologies to meet the lighting standard identified in Amendment 10. This has resulted in a two year delay of Amendment 13, which was to align with the U.S. for 16 products. Consideration should be given to adjust internal processes to avoid situations where only one package of regulations can go forward at a time, and to communicate these delays to affected target groups so that they may plan accordingly.

Concerns were also raised by automotive manufacturers with respect to OEE finalizing the agreement on voluntary labelling for personal vehicles. Interviewed, manufacturers expressed concern that they would experience significant logistical challenges in providing labels for the 2016 model year vehicles, as was their understanding of the planned effective date, given the delays in getting agreement on the label.

In both of these cases – MEPS and vehicle labels – stakeholders noted that there has not been sufficient communication from NRCan on what to expect given the delays. In future, stakeholders impacted by regulatory and labelling agreement delays could benefit from improved communication from OEE to help manage expectations and associated logistics.

NECB Collateral Materials:

NRCan developed tools to support implementing the 2011 NECB, which was considered by all to be a good practice for facilitating adoption by jurisdictional authorities. That said, the chief concern raised by almost all interviewed collaborators and users in the NECB case study was the delay in issuing the collateral material. Although the code has to be complete and officially published by the Canadian Commission on Building and Fire Codes (CCBFC) before collateral tools can be ready (as there is a significant cost to developing tools that could be erroneous), tools for the 2011 NECB are still being produced. The NECB was published in November 2011, the guidelines were not until Dec 2012, software in Nov 2013, and the user guide was ready for publication in June 2014. The NECB is due for a planned update in 2015-16. These delays may have had an impact on ability to comply with the code once implemented.

Publishing outreach materials:

Finally, OEE information materials were described as high quality by almost all who use them. However, across sectors there is a sense that materials used to communicate to stakeholders about the value of the programs are frequently delayed. Some external stakeholders noted, as an example, that publishing of case studies would be helpful in convincing other organizations to undertake energy efficient products and practices advocated by OEE. However, evidence from internal interviewees suggested that there have been significant delays in publishing outreach materials, with case studies often given as examples.

Recommendation 3: Energy Sector should strengthen coordination of OEE efforts within and across sectors by offering program target audiences access to a client focused system of energy efficiency tools, training, information, and partner programs wherever possible.

---

The evaluation found that OEE has contributed best to behaviour change in target audiences when a systems approach was used, either by OEE design or through the initiative of target audiences. In any event, the combinations of program tools were clearly an enabling factor. There a many examples of this within OEE sectors, where various activities are mutually reinforcing. For example, the “market push and pull” relationship for ENERGY STAR and Minimum Energy Performance Standards, and in housing the same relationship exists for refinements to the National Building Code and home labelling in terms of the influence on builder techniques. AutoSmart and Smart Driver curriculum messages are now being taught in workshops, but are also reinforced with incorporation into provincial licensing exams. CIPEC information and tools are often sought out by those who have attended energy management webinars or Dollars to $ense training and used to reinforce or implement what was learned. Companies that pursued ISO 50001 with NRCan assistance did so by using a number of CIPEC tools and training as well. In fact, documentation on energy efficiency initiatives in other countries suggests this is a good practice.

Evidence across sectors identified opportunities to strengthen impacts of OEE activities on encouraging adoption of energy efficient products and practices by more systematically connecting OEE initiatives across and within sectors. Consider the following observations:

• Evidence from the SmartWay carriers’ survey and interviews suggests that there are opportunities to better connect SmartWay carriers with other OEE activities, as 60% expressed interest in receiving more information about  fuel efficient driver training (e.g. NRCan’s FleetSmart) and fuel management  planning (e.g., NRCan’s Fuel Management 101), followed by on-board recording devices (42%), and other technologies.
• Portfolio Manager could be marketed as part of an OEE package. While benchmarking is associated with improvements in energy performance, the exact mechanism is unclear. Experience in the United States and similar programs in the EU suggest that combining ratings with information on how to improve energy performance improves the effectiveness of these initiatives. While Buildings division provides much information on energy management through the NRCan website and Dollars to $ense workshops, there are opportunities to better connect Portfolio Manager registrants to the information provided by Buildings division and  various OEE initiatives that provide different components of potential solutions.
• CIPEC interviewees noted opportunities to further the influence of the program by connecting it more to provincial and utility programs that offer incentives and support that could be combined with CIPEC. Industrial energy managers, particularly SMEs, noted that managing access to various federal and provincial programs to support energy management was a tremendous effort for them and could be made easier and encourage greater uptake through a “one-stop-shop” model led by NRCan.
• Evidence from case studies and interviews on Portfolio Manager, the Federal Buildings Initiative, and ENERGY STAR products, suggests that there are opportunities to enhance impacts through collaboration between OEE sectors when reaching out to public and private organizations to discuss energy management opportunities. For example, both Equipment (who manages ENERGY STAR) and Buildings divisions (manages Portfolio Manager and FBI) deal with PWGSC on Office of Greening Government Operations on energy management issues in federal buildings. While no specific overlap has been identified, it is worth noting that both groups as well as PWGSC representatives have had difficulty collaborating on energy management. To mitigate this concern a MOU Interdepartmental Collaboration on a Low Carbon Initiative was signed by PWGSC and NRCan in April 2012 to maximize the impact of their collective strengths in energy efficient building design, construction and operations resulting in improved energy efficiency, reduced greenhouse gas emissions, and lower operating costs within the federal building portfolio, with opportunities for replication elsewhere in the economy. OEE has had difficulty promoting ENERGY STAR in Federal National Master Standing Offers.

Recommendation 4: Energy Sector should work with Corporate Management Services Sector and Shared Services Canada (SSC) to more effectively use existing technology resources, identify technology risks for OEE and develop a strategy to address these risks to delivering programs online

---

While OEE has developed a new blended-learning web-based approach to Smart Driver for Highway Trucking based on user feedback, ongoing technical issues associated with the e-learning application (SABA -IBM platform ), and limited IT capacity within NRCan and Shared Services Canada (SSC) (in terms of expertise in SABA, human resources, and band-width), have place this e-learning initiative in jeopardy. These issues had negatively affected the administration and management of the 2010 SDHT e-learning and have now delayed the piloting of the newly-developed web-based SDHT curriculum. As OEE investigates making greater use of online tools, this challenge potentially affects other areas of OEE activity. For example, some interviewees suggested a need for technical improvements to the online National Energy Use Database to make it faster and easier for users to query and extract data, CIPEC has been delivering online webinars with plans to continue, and Dollars to $ense interviewees suggested further overcoming resource challenges by exploring online delivery models (e.g. webinar, e-Learning) to improve access and cost-efficiency for remote locations.

Risk to manage (no recommendation): In the context of fiscal restraints on travel and the current Government of Canada travel policies, there is a risk that NRCan could lose influence with partners and target audience groups, if operations within this context are not carefully managed.

---

OEE has responded to the travel and communication policy challenges by curtailing travel and making greater use of online communication tools and industry associations. However, there remains a concern across sectors that by doing this, NRCan runs a risk of losing control of the message and a reduced profile/visibility of NRCan, which could result in reduced influence with target audience groups, ultimately decreasing uptake of programs. Interviewed external stakeholders, including industry association representatives, have noticed a reduced visibility and presence of NRCan at promotional events and standards committee meetings for example, and that this may be negatively affecting stakeholders’ perception of NRCan’s commitment to energy efficiency, and may negatively impact on the quality and credibility/branding of outputs.

Appendix A: Description of Evaluation Case Studies

The table below presents an overview of evaluation case studies completed for this evaluation. Case studies with an * were new activities and had up to 10 interviews per case.

Sector	Case Study Description	NRCan investment $
Buildings Sector
Implementation of the US EPA ENERGY STAR Portfolio Manager Building Energy Benchmarking Tool*	Portfolio Manager is a tool developed by the US Environmental Protection Agency (EPA) to benchmark energy performance for commercial and institutional buildings. A process began in 2011 to adapt the tool for use in Canada, with an official Canadian launch in 2013. Tool refinement is currently ongoing.	$8.7M (estimated 5-year expenditure as of March 2014 for the period 2011-12 – 2015-16)
Updates to National Energy Code of Canada for Buildings (NECB)*	With the financial and technical support of NRCan, the National Research Council of Canada (NRC) worked on the development and implementation of a NECB that replaces the 1997’s Model National Energy Code for Buildings (MNECB). The first updated NECB was published in Nov. 2011. Ongoing activities seek to encourage the adoption or adaptation by jurisdictional authorities, the development of a   User’s Guide and to the development of a second update.	NRCan provided $3.8M over 4 years to the NRC for the update of the 1997 MNECB (as per 2007 MOU), and another $3.4M and for the activities undertaken following the development of the NECB 2011 (as per 2011 MOU).
Housing Sector
ENERGY STAR labelling and promotion for New Homes (ESNH)*	The ESNH labelling and promotion initiative encourages the construction of residential housing that is 20% more energy efficient than a house built to code. The initiative encompasses developing, maintaining and updating the standard; managing the infrastructure for home energy labelling and standards; licensing service organizations; disseminating information to industry and home owners; and collaborating with partners to develop/ implement codes and standards	Total not available; $840,000 in estimated costs related to project activities (2010-11 to 2013-14) and $48,000 for ESNH publications (2009-10 to 2013-14)
Provincial / Utility / Municipality  Partnerships using EnerGuide Rating System (ERS)	The ERS is a national system, developed in 1998, to measure a home’s energy performance and encompasses NRCan’s tools, technical expertise and administrative support. Partners can leverage this infrastructure to deliver their own energy efficiency programming and incentives.	Total value not available.
OEE support for development of  energy efficiency requirements in Part 9 of the National Building Code of Canada (NBC)	OEE provided financial, technical and policy support to the Canadian Commission on Building and Fire Codes (CCBFC) and the National Research Council of Canada (NRC) to develop and include energy efficiency requirements for houses and small buildings in Part 9 of the NBC, including the provision of tools and expertise in committee meetings and to the NRC’s Canadian Codes Centre (CCC) staff.	Approximately $1.2M (2010-11 to 2012-13)
Equipment Sector
ENERGY STAR labelling and promotion programs*	OEE has administered the US EPA ENERGY STAR® initiative in Canada since 2001, including recruitment and support of formal Participants. In 2011, ENERGY STAR introduced as a pilot the “Most Efficient” initiative, labelling the top performers in a handful of product categories in each calendar year. The initiative became a standard ENERGY STAR offering in Canada in 2013.	$12.5M (2009-10 to 2013-14)
Implementation of Equipment Energy Performance Regulations	As part of its 2011 mandate, OEE is developing and implementing up to 18 new or upgraded regulated minimum energy efficiency standards.  This includes the development and implementation of Amendments 11, 12, 12b and 13 to the Energy Efficiency Regulations on energy using equipment.	Total value not available. $15.1M (from 2011-12 to 2013-14 only).
Impact of commercialization efforts  for solid state street lighting	OEE provides technology market barrier and stakeholder assessments for new and innovative energy efficient products in order to help companies address market barriers. This case study focused on efforts to move the streetlight market to high efficiency semiconductor light emitting diodes (LEDs).	$1.4M  (2009-10 to 2013-14)
Administrative cost reduction efforts for border monitoring of imports of prescribed energy-using products	Dealers of products subject to the Energy Efficiency Regulations are required to provide information upon import through the Canadian Border Services Agency (CBSA). Various measures have been implemented to reduce the administrative cost to dealers of meeting these obligations.	Not available; no additional resources were required for the CSA pilot program - all activities were undertaken using existing resources.
Transportation Sector
Implementation of US EPA SmartWay Partnership Program in Canada*	The SmartWay Transport Partnership Program, first initiated in the U.S. by the EPA in 2004, is a voluntary, public/private partnership with a shared goal of saving fuel and reducing GHG emissions and air pollution from the freight transportation supply chain. SmartWay provides benchmarking and information to encourage the uptake of fuel efficient practices and technologies in the freight sector. A process began in 2011 to adapt the tool for use in Canada, with an official Canadian launch in the fall of 2012.	Approximately $3M (2011/12 – 2013/14)
Energy Efficient Fleet Driver Training – SmartDriver for Highway Trucking (SDHT)	OEE provides curricula and delivery tools to the Smart Driver suite of training activities for fleet drivers. SDHT provides free training and course materials including the development and introduction of e-learning activities for fleet trainers, truck drivers and professional driving schools to help companies and drivers save fuel.	Approximately $5,245,000M (2009/10 to 2013/14)
Energy Efficient Personal Driver Training –AutoSmart	Auto$mart Student Driving program, launched in 1997, is a driver-training program that aims to improve awareness and demonstrate the importance of fuel efficient driving practices among new drivers.  OEE provides curricula and delivery tools for AutoSmart; the current version of the program (Version 3) was launched in 2010.	$977,000 (2009-10 to 2013-14).
Industry Sector
Support for Energy Management Systems Standards (EMSS) in Canada*	OEE provides support (financial, training, research, and networking and collaboration) to implement EMSS in Canada, in particular to support the ISO 50001 energy management systems standard, which establishes an energy management framework for all types of organizations.	ISO 50001: $1.74M (2011-12 to 2013/14)
CIPEC suite of tools	CIPEC connects industry with a suite of tools that promote effective voluntary actions to help companies reduce their energy consumption and carbon footprint. These tools include: Leadership and networking opportunities; Employee training assistance (e.g., Webinars, Dollars to $ense workshops); Energy management standards implementation (including cost-shared assistance to implement the ISO 50001 – Energy Management Systems Standard); Technical support.	$6.141M  (2011 to 2016)
Cross-cutting
National Energy Use Database (NEUD)	NEUD is a free online portal where energy efficiency related data and publications collected and produced by NRCan and other federal departments are made available to all Canadians. NEUD data are mostly used to produce reports on energy efficiency, and as background information for multiple purposes (e.g., research and education, monitoring, inform media, program and policy development).	Total value not available. In 2010-11, OEE provided approximately $2.4M.
(Buildings and Industry Sectors) Dollars to $ense training for industrial, commercial and institutional organizations*	OEE provides Dollars to $ense energy management training workshops to target groups in the buildings and industry sectors. These workshops provide knowledge and information on how to better plan, monitor and reduce a building’s or company’s energy consumption. Energy consultants, building managers and facilities managers attend these workshops. NRCan offers both standards workshops (six different workshops as of 2013-14) and customized workshops adapted to specific audiences.	Buildings: Approx. $230,082/year Industry: Approx. $1.1-1.5M/year (2010-11 to 2013-14),

Appendix B: Survey Populations and Response Rates

Survey Population	Number of Responses	Response Rate
Online survey of Dollars to $ense Workshop Participants (energy service company representatives)	N = 19	6%
Online survey of Home Builder survey (participants of home labelling programs; EnerGuide for New Homes, ESNH, R2000)	N = 154	6%
Telephone survey of Homeowner Survey (participated in either pre and/or post retrofit assessments)	N = 844	Response rate not collected
Online survey of ENERGY STAR Participant Survey (manufacturers, retailers, utilities)	N = 264	27%
Online survey of SmartWay Carriers Survey (trucks)	N = 99	42%
Online survey of CIPEC Webinar Participants (industry or commercial and institutional buildings)	N = 197	23%
Online survey of CIPEC Participants (industrial facilities)	CIPEC leaders n = 101 CIPEC non-leaders n= 75	10% for CIPEC leaders (individuals) 5% for CIPEC non-leaders (individuals)