Evaluation of the Indigenous Partnerships Office West (IPO-West)

Audit and Evaluation Branch
Natural Resources Canada

October 2018

Table of Contents

• Acronyms
• Executive Summary
• Introduction
  • Program Information
  • Evaluation Objective and Methodology
• Findings
  • Relevance
  • Performance - Efficiency and Economy
  • Performance - Effectiveness
• Conclusion
• Evaluation Team
• Appendix: Evaluation Methodology
• Appendix 2: Logic Model
• Appendix 3: Program Information
• Appendix 4: IPO-Weat Financial Information
• Appendix 5: Examples of Issues Raised and Discussed

Acronyms and Abbreviations

Executive Summary

About the Evaluation

1. This report presents the findings from the evaluation of NRCan’s Indigenous Partnerships Office-West (IPO-West), one of the four components of the West Coast Energy Infrastructure Initiative (WCEI).

2. NRCan’s Indigenous Partnerships Office-West (IPO-West) was created to facilitate a coordinated federal response to Indigenous concerns related to major energy projects on the West Coast and ensure that federal engagement with Indigenous communities is translated into concrete actions and investments to address Indigenous priorities.

3. IPO-West roles and responsibilities did not include project-specific consultations (e.g. liquefied natural gas projects or pipeline expansion projects), with that role being under the responsibility of other federal organisations, such as the Major Project Management Office (MPMO).

4. The objective of the evaluation was to assess the relevance, efficiency and effectiveness of IPO-West. 

5. The evaluation covers the program spending over the period of 2014-15 to 2017-18. Data collection was completed in the fall and winter of 2017-2018. The evaluation had been identified in the Departmental Evaluation Plan, 2017-18 to 2021-22.

What the Evaluation Found

6. There was an initial need for IPO-West in light of the economic opportunities associated with the major oil and gas infrastructure projects proposed in the Western provinces, combined with a need to respond to recommendations made by the Special Federal Representative on West Coast Energy Infrastructure, Douglas Eyford (in the report Forging Partnerships, Building Relationships: Aboriginal Canadians and Energy Development). 

7. There is a continued need for IPO-W, and for sustained engagement with Indigenous groups. While IPO-West plays a meaningful role within WCEI, initially there was some confusion as to the role of IPO-West and that of other departments in the initiative. Roles and responsibilities were clarified over time, but a few interviewees from other federal departments still feel that the roles of IPO-West remain unclear.

8. Although Although IPO-West has been under spending up until last fiscal year, mostly due to staffing delays and the cancellation of the Tripartite Energy Forum, approximately 105 of 119 targeted communities in British Columbia and an additional 22 communities in Alberta were engaged through meetings, project funding and workshops, including engagement for the co-development of the Indigenous Advisory and Monitoring Committees (IAMCS).

9. Many non-government respondents underlined the importance of having grants and contributions available to support a diversity of capacity-building undertakings in communities. The Strategic Partnerships Initiative – West Coast Energy (SPI-WCE), administered by Indigenous Services Canada (ISC) and co-led by ISC and IPO-West, was key and provided $61M over five years in contributions (G&C) funding for projects led by Indigenous communities.

10. SPI-West projects supported by IPO-West had impacts on participation in processes around energy projects, and other projects influenced other stakeholders, including the National Energy Board (NEB) and industry. Although some of the major infrastructure projects have not yet been undertaken, SPI-WCE projects helped Indigenous communities and stakeholders prepare and build their capacity in anticipation of projects being implemented.

11. Interviewees indicated that some projects would not have been identified or funded at all without the work of IPO-West, who has contributed to building trust and good relationships with Indigenous communities, even some who are opposed to energy development projects. However, some stakeholders expressed concerns about the timeliness of funding decisions within the SPI-WCE Directors General Investment Committee (DGIC).

Recommendations and Management Action Plan

12. Recommendations in the following table have been made to ensure roles and responsibilities of federal partners be clarified and that the timeliness of decision-making for G&C’s be improved.  Considering the significant engagement work initiated by IPO-West, and that additional work is being initiated by NRCan in this area, it is also recommended that a lessons-learned and best practices document be developed to support NRCan and other federal departments who may be taking on similar work.  This also supports federal reconciliation efforts and the need for departments, including NRCan, to implement action.

INTRODUCTION

Program Information

13. This report presents the results of the evaluation of NRCan’s Indigenous Partnerships Office-West (IPO-West), which is one of the four components of the West Coast Energy Infrastructure Initiative (WCEI). The Major Projects Management Office – West (MPMO-West) was renamed in October 2017 to Indigenous Partnerships Office-West (IPO-West). The original mandate of IPO-West was to provide an on-the-ground federal presence in British Columbia and serve as a single window to engage with Indigenous communities on energy infrastructure development. The office was renamed IPO-West to better reflect its mandate of building partnerships with Indigenous peoples in the West.

14. The creation of IPO-West responds to the recommendation in Mr. Eyford’s report that Canada should establish a sustained presence of senior officials on the ground in British Columbia, with capacity to enable Crown engagement with Indigenous communities on key issues related to the development of energy infrastructures. Delivered by NRCan, IPO-West expenditures was $8.2 million for 2014-2019. It is located in Vancouver and has staff presence in Ottawa.

15. IPO-West roles and responsibilities did not include project-specific consultations (e.g. liquefied natural gas projects or pipeline expansion projects), with that role being under the responsibility of other federal organisations, such as the Major Project Management Office (MPMO).

16. As one component of the West Coast Energy Infrastructure Initiative (WCEI), IPO-West engages with Indigenous communities to develop relationships, identify priorities and issues, and seek to respond to and address these priorities and issues.  Through IPO-West, NRCan provides a single window to build relationships with Indigenous communities outside of the formal Crown consultation process to facilitate Indigenous engagement and participation towards a range of priorities expressed by communities related to energy development (e.g. land use planning, environmental remediation, training and employment) and is not linked to any specific major project consultation or accommodation undertaking..

17. IPO-West assists the Minister of NRCan to engage Special Ministerial Representatives (SMRs) to work with communities and was expected to be the secretariat for the Tripartite Energy Forum. The Forum did not go forward as planned but was intended to provide a venue for senior federal and British Columbia (BC) officials and Indigenous leaders to share information in order to enable a collaborative approach to solutions that benefit communities while also advancing development. 

18. In the course of its implementation, IPO-West was selected to house the Secretariat for the Indigenous Advisory and Monitoring Committees (IAMCs) and lead the Economic Pathways Partnership (EPP) pilot project. While the IAMCs and EPP are separate initiatives from the WCEI, the co-development of the IAMCs with Indigenous communities was under the existing mandate of the IPO-West.

19. To deliver on its commitments, IPO-West works in collaboration with federal departments to deliver the other components of the WCEI: Strategic Partnerships Initiative – West Coast Energy, administered by Indigenous Services Canada (ISC) and co-led by ISC and IPO-West; Skills and Partnership Fund - West Coast Energy, led by Employment and Social Development Canada; and Cumulative Effects Monitoring Initiative (CEMI) pilot, led by Environment and Climate Change Canada.

20. In addition, NRCan’s IPO-West works with other federal partners (e.g., Innovation, Science and Economic Development Canada (ISED), Canadian Environmental Assessment Agency (CEAA), Canadian Coast Guard (CCG)) through committees and processes to enhance the collaborative and single-window functioning of the WCEI. 

21. Appendix 1 provides further details on financial information and program description of NRCan’s Indigenous Partnerships Office-West.

Evaluation Objective and Methodology

22. The objective of the evaluation was to assess the relevance, efficiency and effectiveness of Indigenous Partnerships Office-West, in accordance with the Treasury Board (TB) Policy on Results. It also responds to the WCEI TB submission requirement for a separate evaluation of IPO-West.

23. The evaluation covers the program spending of $7.8 million over the period of 2014-15 to 2017-18. Data collection was completed in the fall and winter of 2017-2018. The evaluation had been identified in the Departmental Evaluation Plan, 2017-18 to 2021-22.

24. This evaluation of IPO-West was conducted using three lines of evidence (Key Informants interview, document review and files review) in order to collect and triangulate evidence from multiple sources.  The evaluation focused on the aspects of the WCEI logic model that referred to IPO-West’s expected outputs and outcomes attributable  (see Appendix 2)

25. Activities related to the co-development of the IAMCs were also included within the scope of this evaluation, as they were carried out under the existing mandate of the IPO-West with existing resources.

26. The evaluation approach had a number of limitations, including the limited number of interview respondents from Indigenous communities, and in some cases difficulty distinguishing IPO-West responsibility activities from IAMC activities. As a mitigation measure, the evaluation team was prudent in the interpretation of the findings, focussing on factual information to the extent possible.

27. More information on methodology, scope and limitations is contained in Appendix 3 –Evaluation Methodology.

FINDINGS

Relevance

28. The evaluation found that there remains a need for IPO-West to build and maintain good relationships with communities through coordinated engagement. The evaluation found also that IPO-West’s activities are aligned with current federal priorities related to relationship building and improved collaboration with Indigenous communities and are coherent and aligned with the federal government role in engaging with Indigenous communities. 

Is there a continued need for IPO-West?

29. All lines of evidence confirm that the needs that led to the creation of IPO-West are still current and continue to exist. IPO-West was created to strengthen on-the-ground federal presence on the West Coast, and to provide an overarching governance structure to facilitate a whole-of-government approach for Indigenous engagement and participation in West Coast energy infrastructure projects. This mandate includes oversight and coordination functions related to the implementation and administration of the West Coast Energy Infrastructure Initiative and liaison with other departments, and an engagement role to ensure that the interests and concerns of Indigenous communities are known, understood and addressed through concerted federal action.

30. According to documentation, the creation of IPO-West was driven by a need to improve relationships and establish partnerships with Indigenous communities with regards to energy development, in light of important economic opportunities associated with major oil and gas infrastructure projects in the West. In 2013, it was estimated that more than 600 major resource projects, worth approximately $650 billion, were planned for Canada over the following decade. Oil and gas in British Columbia in particular offered important opportunities. Infrastructure limitations such as limited access to tidewater and limited pipeline capacity constrained further developments. Major infrastructure projects including Trans Mountain Expansion, Enbridge’s Northern Gateway, and multiple Liquid Natural Gas (LNG) projects were initiated in this context, and in turn generated a need to engage with Indigenous communities affected by them. Upon its creation, IPO-West supported the federal government’s strategy for Responsible Resource Development, particularly with respect to enhancing Indigenous participation. It should be mentioned that upfront, the resources allocated to IPO-West sunset in 2019 and that this time-limited funding aligned with a critical period of proposed energy infrastructure development.

31. Most importantly, IPO-West was created as a response to the Eyford Report (Forging Partnerships, Building Relationships: Aboriginal Canadians and Energy Development, 2013) regarding the need for the federal government to improve its relations with First Nations. In 2012, the federal government appointed Douglas Eyford as Special Federal Representative on West Coast Energy Infrastructure to identify approaches to meet Canada’s goals of expanding energy markets and increasing Indigenous participation in the economy.

32. The Eyford reportdescribes a need for concrete action to increase trust and improve relationships between the federal government and Indigenous communities. His findings called for early and sustained engagement going beyond the government’s Duty to consult. Subsequent reviews and government guidelines on engagement also highlight a need for the government to support broader engagement with communities, and ensure they have the capacity to participate. The Eyford Report also recommended increased coordination between federal departments and agencies in their engagement efforts, and the creation of a single-window access allowing Indigenous communities to interact with the federal government. 

33. Responding to Eyford’s conclusions entailed developing respectful and collaborative rapports with communities, as well as supporting capacity-building to facilitate First Nations’ participation. As expressed by Indigenous representatives and as highlighted in the Eyford report, First Nations in British Columbia have a desire to be meaningfully included and involved in discussions and processes related to energy development. For instance, communities want to be engaged on socio-economic considerations such as possible benefits of projects for communities, business and community development, opportunities for skill development, training and employment. Environmental issues are also fundamental for many Indigenous communities who want to be involved in environmental monitoring and review processes, as well as mechanisms for marine safety, incident prevention and sustainable development, as they relate to energy projects.

34. According to program documentation and in the opinion of most key informants across various categories, IPO-West offers a way for NRCan to build relationships with Indigenous communities, and work with other federal departments to address Indigenous concerns and priorities in a coordinated manner.

35. All NRCan informants and representatives of other federal departments felt that there is a continued need for engagement and relationship building with Indigenous communities regarding energy infrastructure development in British Columbia. According to interviewees, the need for sustained federal engagement is now more clearly articulated around the federal government’s objective of building nation-to-nation relationships and strong partnerships with First Nations. Some NRCan respondents mentioned that relationship-building is a long process that requires time, resources and an in situ presence that IPO-West was equipped to provide.

36. However, a few government interviewees pondered whether contextual changes that have occurred since the creation of IPO-West now warrant a re-examination of its mandate to ensure the office’s activities are coherent with current dynamics and adequately address the priorities identified by Indigenous communities with respect to energy development. Those interviewees mentioned that various developments have shifted the landscape in this sector including the cancellation or confirmation of some major projects; new government approaches and policies (e.g., structural changes at INAC); a new responsibility for departments to interact with Indigenous groups; and new energy market realities (e.g., increased interest in clean energy).

37. In the context of those developments, some interviewees felt there may be a need to review the office’s mandate and activities. However, other interviewees rather felt that the now government-wide priority for Indigenous engagement may have made the office more relevant: since all departments are tasked with engaging Indigenous communities, there is an increased need for horizontal coordination. In this regard, IPO-West plays an important role, and can also share its expertise and best practices with other government players.

38. It should be noted here that the original impetus behind the activities of IPO-West was to provide an on-the-ground federal presence in BC and engage with Indigenous communities on energy infrastructure development. IPO-West interviewees indicated that the office has been making headway in building relationships, engaging stakeholders and supporting Indigenous participation broadly around energy infrastructure development. The office has become further oriented towards building partnerships with Indigenous communities, and in 2017 led the co-development of the IAMCs for the TMX and Line 3 projects. The rebranding of the office from MPMO-West to IPO-West was done to better reflect the office’s activities.

Does the program align with current federal government priorities?

39. Based on government documentation, and in the opinion of most government interviewees, the mandate and activities of IPO-West are aligned with current federal priorities of building renewed, collaborative, respectful relationships with Indigenous communities, and increasing Indigenous participation in a variety of sectors, including energy development.

40. The December 2015 Speech from the Throne stated the government’s commitment to provide a greater role for Indigenous Peoples in reviewing and monitoring major resource development projects, while also helping mitigate potential impacts on Indigenous rights. The 2016 Budget holds a renewed commitment to engaging with Indigenous People across Canada, and the 2017 Budget also highlights a priority to develop partnerships and collaborate with Indigenous groups in various domains, including the energy sector. The mandate of IPO-West thus reflects current federal priorities with regards to engagement of Indigenous communities. According to most federal informants, the office’s mandate is aligned with the government’s focus on Reconciliation, and building nation-to-nation relationships, which is also coherent with the recent government commitment towards a Recognition and Implementation of Rights Framework.

41. The objectives of IPO-West are also aligned with the priorities of the Ministry of Natural Resources. The initial economic impetus for the creation of the office related to NRCan’s mandate of developing and diversifying the energy sector in Canada. The Minister of Natural Resources’ current Mandate Letter states that “it is time for a renewed, nation-to-nation relationship with Indigenous Peoples, based on recognition of rights, respect, co-operation, and partnership”.

42. NRCan leads the whole-of-government effort on energy market diversification, and IPO-West is located under the NRCan Energy Market Access and Diversification sub-program. Looking at departmental priorities and reporting on results, IPO-West relates to 1) NRCan’s strategic outcome of making Canada’s natural resources sectors globally competitive, 2) the department’s priority to “expand market and global partnerships” and 3) is part of mitigating risks associated with market access and investment climate through working with Indigenous communities. 

43. NRCan interviewees also noted that the objectives of IPO-West are consistent with the department’s priorities of developing and diversifying the energy sector, which entails the development of energy infrastructures and consultation with Indigenous communities.

Is there a legitimate and necessary role for the federal government in the area?

44. IPO-West’s mandate is to provide a governance structure that facilitates a whole-of-government approach for Indigenous engagement and participation in West Coast energy infrastructure projects. IPO-West coordinates WCEI components, liaises with other departments, and engages Indigenous communities so their concerns are known, understood and acted upon in a concerted manner. All government interviewees saw a legitimate and necessary role for the federal government in this area.

45. The federal government has legal responsibilities vis-à-vis the constitutionally protected rights of Indigenous communities and its responsibilities regarding consultation and engagement with Indigenous Communities are entrenched in law. Canadian courts have established that the government has a constitutional duty to consult and, where appropriate, accommodate “when the Crown contemplates conduct that might adversely impact potential or established Aboriginal or Treaty Rights” (Engagement and Accommodation Guidelines). Court decisions as well as results of consultations highlight that early and continuous engagement must also take place beyond the government’s Duty to consult, and outside of project-specific negotiations, on the basis of respectful relationships.

46. IPO-West’s activities support this broad responsibility of the government to effectively engage Indigenous groups, and support their capacity to participate, for instance through information sharing and training on energy literacy. An IPO-West interviewee specified that the office works to identify and coordinate responses to issues that are within federal jurisdiction and responsibility.

47. Furthermore, the Government of Canada has made a commitment towards the United Nations Declaration on the Rights of Indigenous People. The declaration speaks to the inclusion of Indigenous peoples in decisions that affect their rights and interests.  IPO-West’s objectives are thus consistent   with Canada’s recognition of the right of Indigenous peoples to participate in decision-making on matters that affect their rights, and the need to consult and cooperate in good faith. The engagement efforts deployed through IPO-West also fall under the legal framework which delineates the role of NRCan. The Minister’s general duties listed in the Department of Natural Resources Act include “seeking to enhance the responsible development and use of Canada’s natural resources and the competitiveness of Canada’s natural resources products.” The act also states that the Minister shall “promote cooperation with the governments of the provinces and with non-governmental organizations in Canada”.

48. This being said, a few interviewees from other federal departments saw possible tension between NRCan’s economic mandate towards energy sector development, and IPO-West’s objectives of addressing Indigenous concerns.    

Recommendations

49. No recommendations were addressed for this section.

Performance – Efficiency and Economy

50. Interviewees confirmed that IPO-West plays a meaningful role within WCEI. According to most informants, the roles and responsibilities were clarified over time, but a few interviewees from other federal departments felt that the roles of IPO-West remain unclear. The fact that the participating departments have different mandates and different criteria to evaluate projects has created barriers to timely decisions within the SPI-WCE DGIC. On the other hand, it was said that without the work of IPO-West, some projects would not have been identified or funded, including environmental projects.  Many of the department and community representatives are involved in multiple committees or meetings directly or indirectly related to energy development, which are held by various departments or governments. Respondents found that these committees are very time-consuming for them and are concerned about potential overlap and duplication.

51. IPO-West has been in an under-budget situation (under spending) up until the last fiscal year, mostly due to the cancellation of the Tripartite Energy Forum did not go forward as planned. Based on documentation, IPO-West has met 88% of its primary output target, which was to engage with all targeted communities at least once. There were few recommendations made with respect to alternative delivery models. Some interviewees said that IPO-West managing its own grants and contributions could improve the timeliness of funding decisions.  Based on these results, it is concluded that IPO-West was efficient from an operational perspective.

To what extent the governance structure improves the delivery or effectiveness of the individual initiatives?  Of the entire initiative? 

52. The WCEI and IPO-West governance structure includes many components and processes. The WCEI is guided by the existing interdepartmental MPMO Deputy Ministers’ Committee, and uses the existing MPMO governance structure as the forum to share information and resolve issues.  IPO-West itself is overseen by a Senior Executive Director who reports directly to the Deputy Minister of NRCan. It was expected that IPO-West be responsible for coordinating and conducting ongoing engagements with Indigenous communities throughout BC; liaising with partnering departments in the development of program and policy responses to individual community requests; providing secretariat functions to the Tripartite Energy Forum (which was cancelled); and briefing, reporting, and providing strategic advice to senior officials, Ministers and ministerial staff and other stakeholders.

53. Although the Office does not manage grants and contributions directly, it does provide oversight for investments in West Coast energy infrastructure through a two-key approach, which requires that all spending proposals be subject to the approval of NRCan senior management and the responsible Minister (or delegated departmental authority). This applies to the Strategic Partnership Initiative – West Coast Energy (SPI, at DISC) and the Skills and Partnership Fund – West Coast Energy (ESDC). ESDC and ISC maintain ultimate authority and oversight for the activities and programs delivered by their respective departments. The Director General Investment Committee (DGIC) plays a key role in the SPI-WCE grants and contributions funding decisions. DGIC validates and prioritizes project funding opportunities and is co-chaired by ISC and NRCan (IPO-West). Both must agree to the use of project funds. The Technical Working Group (TWG) for SPI-WCE is comprised of working-level staff of each of the partner departments, with the objective of working collaboratively to ensure a robust approach to identifying and assessing proposals prior to consideration by DGIC.

54. The evaluation assessed the extent to which the roles and responsibilities of the various departments involved were clear. For many interviewees, including representatives from ISC and ECCC, the roles were not totally clear at the outset of the program. There was some initial confusion as to whether IPO-West’s role was at all related to the formal Crown consultation processes. Program officials explained that some stakeholders may have had difficulties distinguishing MPMO-W from MPMO (which conducts Crown consultations, contrary to MPMO-W and IPO-W subsequently). There is also some confusion about the exact roles and responsibilities of the departments involved in WCEI. These questions were later clarified according to most federal interview respondents, but some maintain that there is still no clear definition of roles of IPO-West with respect to the other WCEI partner departments, as well as the roles of other partners departments who participate in the DGIC (i.e., Department of Fishery and Ocean-DFO and Transport Canada-TC).

55. In particular, even though the process for reviewing and approving SPI-WCE proposals by DGIC is available, the roles of IPO-West with respect to the project selection and approval processes were unclear for some respondents.

56. The evaluation assessed the extent to which IPO-West improves the delivery and effectiveness of the entire WCEI, including the other components (SPI-WCE and SPF-WCE). There is evidence (from key informant interviews, including government and community respondents) that the G&Cs project approval process for SPI-WCE proposals takes more time than expected. The fact that the participating departments have different mandates and different approaches to applying program criteria to evaluate projects may have created a barrier to timely decisions within the DGIC. The DGIC aims to coordinate government investment decisions and make decisions that are supported by all partner departments. It was said that there can be a lot of back-and-forth between departments that can take several weeks, which is due to the time required to address various departmental concerns or questions about proposals. This may require additional interdepartmental meetings to understand the concerns and may require the proposal be re-scoped or changed to address comments.

57. Although the coordination process can be lengthy, some NRCan respondents disagreed that decisions are not timely and said that funding decisions are generally faster than other comparable programs at ISC and ESDC which consider proposals on an annual basis, as opposed to SPI-WCE’s continual intake process. In fact, SPI-WCE has funded proposals that would be eligible for funding through INAC programs, however they missed the application deadline and the group would have missed out on economic opportunities if they had to wait for the next intake period.

58. A few respondents were of the opinion that IPO-West plays a positive role in ensuring that relevant projects obtain funding from SPI-WCE. It was explained that the office plays a role in helping communities formulate their project and develop proposals. A few key informants noted that without the work of IPO-West, some projects would not have been identified or funded. One key informant noted that this was particularly the case for those projects related to environmental monitoring and safety.

59. As part of its coordination and oversight function within the WCEI, IPO-West partakes in frequent bilateral and multilateral meetings. IPO-West manages or co-manages the following meetings and committees for the WCEI: the DGIC, the SPI-WCE TWG, and IPO-West bi-weekly teleconferences (Director-level). The DGIC and TWG are dedicated to decision-making on SPI-WCE, while the bi-weekly teleconference is for federal coordination and information-sharing; participation in the teleconferences is optional, with generally strong turnout.

60. IPO-West is also involved in a number of committees and meetings outside of the scope of the WCEI. Since 2017, IPO-West has held monthly IAMC meetings for the TMX and Line 3 projects (with federal and Indigenous representatives); regular meetings related to the IAMCs (e.g., subcommittees and federal coordination); and ad hoc meetings for the Economic Pathways Partnership (EPP) initiative (with federal and Indigenous representatives, and others). IPO-West representatives also participate in various fora and bodies managed externally by other departments or governments, many of which were launched in 2016 or later, that address issues such as reconciliation, environmental stewardship, transportation and economic development (e.g., Oceans Protection Plan (OPP) calls, BC Federal Council’s Indigenous Issues and Interest Committee (IIIC), Reconciliation tables, Premier’s First Nations Advisory Council (FNAC), etc.).

61. Some NRCan staff mentioned that the co-development work for the IAMC was time-consuming and detracted staff from their other tasks within IPO-West. In addition, the interviewees did not necessarily distinguish between the IPO-West regular activities– which was funded through IPO-West’s budget and within scope of this evaluation – and subsequent activities of the committees.

62. Respondents from ISC and the communities mentioned that in many cases, the same community and departmental representatives are involved in multiple committees, which can lead to duplication and an inefficient use of their time. While committees were generally perceived as a good mechanism to share information and making decisions, especially the DGIC, some respondents from ISC and the communities noted there are many committees and sub-committees somewhat associated with the WCEI. Although there are only two committees under the WCEI mandate (DGIC and TWG), there are a growing number of committees and initiatives with specific objectives that necessarily overlap in theme or membership (e.g., marine response/preparedness and energy development).

63. One community representative noted that the working groups he is involved in are not action-oriented (“we have meetings about meetings”) and can be a drain on already limited capacity of Indigenous communities. Others, including government, Indigenous and non-government respondents, added that committees and sub-committees can slow down the process rather than improve the responsiveness. The feedback from respondents indicates that there is meeting fatigue due to the number of co-existing initiatives (particularly in recent years), and concern about potential overlap and duplication.

64. A few respondents mentioned that by design, the grants and contributions associated with the overall initiative are managed by other departments, namely ISC and ESDC. However, it was noted that the success of IPO-West’s engagement activities is very much tied to these funding sources. Project funding and resources for capacity-building are key factors in engagement and in developing long-term partnerships.

Are there alternative delivery models that can achieve similar outcomes more efficiently?

65. The evaluation assessed the extent to which the program could achieve similar outcomes more efficiently, that is, with less resources spent. This was assessed based on key informant interview findings. Few suggestions were made. Some respondents outside IPO-West said that G&Cs decisions take a long time and that the situation could be improved if IPO-West administered its own G&Cs. However, it was not mentioned if/how the other departments would provide input into these decisions.  A few also called for more flexibility in terms of what is eligible for funding (e.g., projects related to other natural resources).

To what extent have resources been used as planned to produce intended program outputs?

66. The evaluation assessed the extent to which resources were expended as planned to produce the program’s outputs. The planned budget for IPO-West (2014-2018) was $11,598 million. Figure 1 provides an overview of the actual expenditures against budget, indicating that 67% of the planned budget was spent.

Figure 1: IPO-West Budget against Actual Expenditures

67. According to documentation, under-spending occurred because of the following reasons:

• Delayed staffing for IPO-West in the first year;
• Cancellation of Tripartite Energy Forum (representing a yearly planned amount of $500,000);
• Less resources used for training and conferences for IPO-West staff;
• Travel budget not fully utilized; and
• Engagement activities delayed for various reasons (including national elections).

68. Documentation also confirms that significant resources were used for co-developing the IAMCs. In terms of resources used to achieve the intended outputs, program documentation indicates that there was only one quantified output target for IPO-West, which was to engage with each targeted community at least once. As detailed under the Performance Section, IPO-West engaged with most (105 or 88%) of the 115 targeted communities between March 2014 and March 2018.

69. A few respondents mentioned that IPO-West has proven to be efficient (in delivering its outputs) given its level of performance and the fact that it has been understaffed during many months.

70. Some respondents confirmed that the co-development of the IAMCs involved a significant amount of time from IPO-West’s staff in 2017 (see appendix 4 about the budget utilization). Before the IAMCs obtained its own funding envelope, IPO-West conducted a number of workshops and meetings with Indigenous communities to identify their needs and to establish the parameters of the IAMCs. Many analysts who were assigned to specific geographical areas for the purposes of IPO-West were assigned to these tasks leading to the IAMCs. As a result, although the work of IAMCs is in line with IPO-West’s mandate, some staff of IPO-West had to reprioritize their work to respond to more time-sensitive pressures and federal commitments, including previously dedicated analyst time to specific areas/communities.

Recommendations

71. Recommendation 1: Given confusion expressed by some key government stakeholders about roles of IPO-West in a changing policy and major projects landscape,  IPO-West roles and responsibilities should be well defined and communicated, with special attention to the role played by other federal departments and agencies that are working with west coast Indigenous communities on socio-economic issues.

72. Recommendation 2: Considering IPO-West’s new responsibilities related to Indigenous Advisory and Monitoring Committees and Economic Pathways Partnership, IPO-West roles and responsibilities should be clarified and that sound accountabilities and reporting be put in place.

73. Recommendation 3: Given the findings related to the lengthiness of Gs&Cs decision-making processes, IPO-West should initiate discussion with other departmental partners on potential options to improve the process including better communication with Indigenous communities.

MANAGEMENT RESPONSE AND ACTION PLAN

74. Agreed with Recommendation 1. IPO-West will engage with NRCan and partner departments through existing mechanisms (DGIC, Technical Working Group and WCE Coordination fora) to reaffirm IPO-West roles and responsibilities as defined and articulated in the mid-term report to the President of the Treasury Board.
Responsible Manager, Sector: ADM, Indigenous Secretariat and DG, IPO-West
Target Date: January 31, 2019

75. Agreed with Recommendation 2. IPO-West will clarify its roles, responsibilities and accountabilities for senior management and report on activities associated with the IAMC co-development phase as against the mandates provided through the WCEI, IAMC and Economic Pathways Partnership (EPP) initiatives.  
Responsible Manager, Sector: ADM, Indigenous Secretariat and DG, IPO-West
Target Date: January 31, 2019

76. Agreed with Recommendation 3. IPO-West will engage within NRCan and with partner departments to discuss the G&C process.  Discussions will include feedback from Indigenous communities to help in the design of future programming related to Indigenous capacity building, including the potential renewal of IPO-West.
Responsible Manager, Sector:  ADM, Indigenous Secretariat and DG, IPO-West
Target Date: February 28, 2019

Performance – Effectiveness

To what extent is the program achieving the intended immediate, intermediate and final intended outcomes?

77. The major evaluation findings related to the achievements of IPO-Wests are organised around the expected outcomes.

• Immediate Outcomes: the IPO-West effectiveness in engaging with targeted Indigenous Communities to gain understanding of their needs, priorities and concerns as they relate to energy development.
• Intermediate Outcomes: the extent to which IPO-West was successful in engaging federal and other partners, sharing information and contributing to coordinated and collaborative government response to address identified community issues, concerns and priorities.
• Long Term Outcomes: the impact of various IPO-West activities related to helping communities make informed decisions and fully participate in the energy projects. 

Improved Engagement with Indigenous Communities

78. Since its inception in 2014, IPO-West has engaged with a large number of Indigenous communities in British Columbia (see Appendix 5), and more recently in Alberta. Program documentation shows that the program engaged approximately 105 of 119 “target” communities in British Columbia (or 88% of the full objective), through meetings, project funding, workshops and other events; approximately 22 communities in Alberta were engaged as well, including engagement for the co-development of the IAMCs. Some communities have been involved in multiple engagement activities, while others have been involved in only one event/interaction. In addition, IPO-West has engaged with numerous Indigenous organizations and collectives that represent multiple communities.

79. According to the document and file review, specifically the Meeting Tracker, representatives from at least 56 different communities participated in nearly 350 meetings and events over three years (July 2014 to March 2018).  Nearly half of the meetings (47%) were conducted with the communities located in the North Coast region and in Lower Mainland. 

80. In 2017, IPO-West led the co-development of the IAMCs for the TMX and Line 3 projects. This work was within the mandate of the office to engage with Indigenous communities on their needs and priorities related to energy infrastructure development; however, the geographic scope of IPO-West’s engagement activities expanded to Manitoba, Saskatchewan and Alberta (although IPO-West had previously engaged with some communities in Alberta). IPO-West’s existing relationships with communities in BC enabled the office to move quickly to begin the co-development process for the TMX Committee, while additional time was required to build relationships in Alberta, Manitoba and Saskatchewan and begin the co-development of the Line 3 Committee.

81. IPO-West is perceived by some (particularly at the federal government level) to have established an innovative and successful model for engagement with the Indigenous communities, which can be replicated and emulated by other government departments. Having a dedicated office with the specific mandate to engage with Indigenous communities as well as the involvement of senior government officials were perceived as contributing to the effectiveness of the approach.

82. IPO-West implemented various strategies to identify community representatives and engage with them.  This included reaching out to communities and conducting workshops (this was largely done in the first year), direct contacts, and collaborating with provincial government to use their networks and structures to link with the communities.  According to interviewees from the federal government, IPO-West introduced an innovative approach with the co-development of the TORs for the IAMCs. Rather than developing TORs and subsequently seeking approval from the communities, the office built on existing relationships to work directly with Indigenous communities along the project routes to develop the mandate and TORs for the Committees together.

83. IPO-West outreach and engagement activities with Indigenous communities were perceived positively by most community representatives interviewed.  The type of engagement activities community representatives participated in included:

• Workshops (e.g., on marine safety, energy literacy);
• Environmental monitoring forums and discussions on environmental impacts of projects (e.g., impacts on fish and fish habitat restoration, moose population);
• Meetings about SPI-funded projects;
• Technical discussions on infrastructure projects;
• Meetings to discuss opportunities and initiatives related to energy infrastructure projects; and
• Other informal discussions.

84. The community representatives recognized that successfully engaging Indigenous communities is not an easy task given the number of communities, varying level of capacity, different means and ideas (e.g. some are supportive of energy development and some are strongly against it), etc. One community representative noted that prior to IPO-West, the federal government presence in the region and engagement on the major energy projects were limited or non-existent. It was also noted that IPO-West staff ‘went above and beyond’ to ensure that all communities are actively participating in the discussions, workshops, meetings and committees or focus groups. This was particularly important for smaller communities who do not have resources or networking capital with federal government to have their voices heard or access funding for projects that would address their needs and priorities.

85. The ability of IPO-West to have meaningful, community-centric engagement with communities resulted in increased trust and development of positive relationships.  The strength of the relationships and trust that has been developed is perhaps best illustrated in the continued, respectful and successful relationships that the Office has with the communities that are strongly opposed to the major projects.

86. Communities emphasized that they see IPO-West role in building relationships and engaging with the communities and not having an agenda with respect to approval or regulatory requirements for the major projects.  Other government representatives added that the reason IPO-West has been successful is because they are not a regulator and are generally perceived as neutral with a mandate specific to resource development.   

Understanding of Indigenous Needs, Priorities and Concerns

87. The ongoing engagement activities have helped identify a wide range of needs, issues and priorities of Indigenous communities that are impacted by major energy development projects. The document review (the Meeting Tracker) identified 37 types of issues that have been discussed with Indigenous communities.  Most commonly discussed issues included general update and SPI proposals (each discussed in nearly 15% of meetings), relationship building (10% of meetings), employment, skills and training (8% of the meetings), engagement, marine safety, workshops and fish habitat (each discussed in about 5-6% of the meetings).

88. Appendix 6 illustrates the issues raised and discusses in the IPO-West supported/SPI funded events that took place in 2016 and 2017. For example, 84 participants from Nisga’a citizens (as well as representatives of Transport Canada, Pacific Pilotage Authority, Coast Guard, the province, etc.) attended the NEST 360o event. This event allowed members of the Nisga’a community to learn more, and share ideas about marine safety, pipeline safety, and employment and training opportunities related to LNG.

89. Most Indigenous respondents were satisfied with the efforts made by IPO-West to work with the communities to help identify potential projects related to energy development, and access funding through SPI. Some examples of the issues and priorities identified by Indigenous representatives include: developing community capacity to respond to emergencies; environmental monitoring; water quality and water management; fish restoration and other critical restoration activities; technology; and socio-economic opportunities related to the projects.

90. One Indigenous representative noted that with IPO-West help, they were able to identify and secure funding for a project that focused on traditional ecological knowledge. The project consisted of gathering an inventory of knowledge among elders about the ecology of the territory. Another example provided was IPO-West assistance in helping train Indigenous responders on oil spill clean-up methods and addressing environmental stressors that impact shell-fish harvesting.

91. Others noted that the representatives of IPO-West were receptive, open and had genuine interest in hearing from communities and helping them address their needs and concerns. The accessibility, effort and commitment of IPO-West staff were highlighted by many respondents. 

Collaborative and Coordinated Government Response and Investment

92. IPO-West’s governance structure (regular DG meetings, ADM presence) - as well as its involvement in various committees and government networks - allowed for effective sharing of information and coordination of activities. Most key informants reported that IPO-West staff have excellent knowledge of relevant departmental programs (e.g. good point of referral for provincial partners and communities) and have good access to federal and provincial departmental networks and personnel.

93. Provincial representatives generally agreed that having a local federal office that is aware of the regional context and Indigenous concerns has improved communication and access to the federal government.  Both federal and provincial interviewees explained that the meetings and ongoing communication are effective in helping them learn about concerns raised by the communities, efforts made by other departments to address those concerns, specific projects implemented or planned in the communities. One respondent noted that IPO-West has helped develop stronger partnership between DFO and the communities (e.g. for the first time, DFO is sharing its real time and historical data with the communities).  

94. Some key informants noted that IPO-West activities resulted in a more collaborative and coordinated government response not only by increasing cross-government awareness of the issues, but also by helping communities have a unified voice about their priorities and potential actions and solutions. For example, it was noted that there are a number of communities around the ports that may have similar concerns about transportation safety and monitoring but different perspectives about potential course of actions. This made it very difficult for government to respond to individual requests.

95. IPO-West has helped bring communities together, identify opportunities for joint action, formulate project and develop proposals.  The proposals are then reviewed in coordination with the other departments including ISC, ESDC and WD and others to ensure coordinated investments. Since 2014, of the projects funded through SPI-WCE, IPO-West has been the lead department on about 70 projects, representing about $10M in funding.     

96. The coordinated approach is reflected in the diversity of the projects identified and supported through SPI-WCE (e.g., fisheries, marine safety, monitoring, training and skills development, economic opportunities). One key informant noted that without IPO-West’s work, some projects, particularly those related to environmental monitoring and safety may not have been identified and funded. Overall, interviewees highlighted the importance of adequate funding available to support communities in developing and undertaking projects in line with their priorities and concerns.

97. As illustrated in the following chart, nearly half of identified issues and concerns have been addressed and close to 15% are in progress. Identified issues have been categorized into over 30 themes in the IPO-West meeting tracker. Issues deemed addressed include the delivery of workshops, and actions and projects that support a range of employment, skills and training and pipeline safety priorities identified by communities. Other issues related to concerns around marine safety, long-standing capacity challenges, and emergency response priorities are ongoing issues in the sense that they cannot be easily addressed, and may require extensive concerted action or legislative changes for instance.

98. The interviewees provided numerous examples of ways in which IPO-West contributed to a more collaborative and coordinated government response to Indigenous concerns and priorities. For example, some of the WCEI engagement activities have allowed industry to recognize its obligations to accommodate and to discuss issues with the FNs. For example, according to an industry respondent, meetings with communities (supported through SPI-WCE) allowed industry to gain knowledge of concerns and in some situations, this resulted in some pipelines being re-routed. One company working in a community changed its design from a water-cooling system to air-cooling system after hearing concerns from the communities at an event supported through SPI-WCE.

99. A few provincial representatives highlighted the important roles that IPO-West played in coordinating investments between federal and provincial partners including achieving the maximum impact of provincial investments, engaging industry (LNG, sub-contractors), and communicating across federal departments to help shape the overarching approach to investment.  According to these interviewees, IPO-West provided information about community priorities and investments made by the federal government, which supported the provincial government develop its approach to skills and development investments. Their input also allowed the provincial government to move faster on certain projects because of the networks and relationships established by IPO-West (e.g., finding the right people to talk to about skills and training opportunities). A few examples of co-funded projects and issues that involved close collaboration between federal and provincial government were provided (e.g. projects related to Pacific Trail pipeline and LNG corridors).

Increasing Capacity of Indigenous Communities to Participate in Energy Projects

100. The NRCan interviewees mentioned that many of the initial meetings with Indigenous communities focussed on the development of energy literacy among communities impacted by energy development.  Many community members had very little knowledge about the energy sector and relevant government processes, legalities, regulations, risks and opportunities, and this was noted as a significant barrier to meaningful participation in activities related to energy infrastructure development. One representative explained that some SPI-WCE funded projects have specifically been focussed on these aspects.

101. Workshops supported or facilitated by IPO-West were in many cases held to discuss the potential impacts of oil and LNG infrastructure development on Indigenous communities, and how government investments could support increased community capacity to better participate in energy projects, including through business and employment opportunities, or capacity to better engage community members on related decisions. Key informants provided examples of the ways in which IPO-West increased the capacity of Indigenous communities through SPI-WCE projects.  

102. A number of communities sought SPI-WCE funding to help better engage community members in the decision-making process related to energy infrastructure proposals in their territories, including the consideration of whether to enter into mutual benefit agreements (MBAs) with industry, and if yes, what these MBAs should look like. In one example, a community was offered an MBA by industry, which was rejected by the membership via a referendum. Only 3% of community members had voted though, calling into question the validity of the result.   The SPI-WCE funding supported the establishment of an up-to-date community membership database, a community website and Facebook page to better share information, and resources to undertake door-to-door communications. The subsequent vote saw the participation rate increase to approximately 75% of community members.

103. The support provided by IPO-West helped equip a community with technical expertise and knowledge so that the community could conduct an environmental assessment related to LNG development. The community representatives felt that although ultimately the LNG project did not go ahead, the learning and capacity that was built was important in identifying significant and complex issues and has created a collaborative and community driven process to addressing those issues.

104. IPO-West helped put together a six-month pollution response internship, in partnership with Canadian Coast Guard, for candidates from coastal First Nations with the objective to build capacity to respond to pollution incidents near their communities.

105. The Marine Safety Workshop titled “Lessons from the Simushir” was highlighted by key informants as another example of building community capacity to respond to emergency situations. The workshop brought together a number of coastal First Nations including representatives from Washington State, technical experts, industry representatives, federal and provincial government representatives and other interested parties. A number of recommendations and concrete action items related to safer marine shipping were generated. As a result of that workshop the community was able to develop improved relationships with TC and DFO.

106. The access to a relatively flexible contribution program like SPI-WCE has been viewed by most key informants as critical in supporting a wide range and type of projects that build capacity. IPO-West has supported communities in devising, planning and executing these capacity-building projects. NRCan and other government representatives also noted that projects funded through SPI-WCE dealt with a wide variety of issues related to building capacity, which included skills/knowledge development and training programs related to fish habitat restoration, environmental safety, and technical capabilities, etc.

107. IPO-West’s activities also aimed at supporting the economic participation of Indigenous communities in energy infrastructure projects, whether through business or employment opportunities. Although some of the major infrastructure projects have not been undertaken on the ground yet, SPI-WCE projects helped Indigenous communities and stakeholders prepare and build their capacity ahead of projects being implemented.

108.Some concerns were raised by community members with respect to IPO-West’s ability to consider and address the diverse perspectives and opinions of Indigenous communities, including positive and negative views about the energy projects and views that vary from one community to another. Another interviewee warned that while IPO-West has helped to build the capacity in many communities, the capacity is not the end goal - investments must be sustained in order to achieve the ultimate objective of addressing the concerns and issues of Indigenous communities.

To what extent have external factors influenced (positively or negatively) the achievement of program outcomes?

109. When asked about external factors that positively or negatively influenced the achievement of outcomes, key informants described factors that are out of IPO-West’s direct control. These elements were also noted in the program’s performance documentation as possible sources of risks associated with the office’s activities.

110. Interviewees noted that these various factors impact IPO-West’s interlocutors’ positions and expectations, and can present a challenge when trying to establish or maintain relationships and collaboration. Since the activities of IPO-West are generally related to infrastructure projects, factors that have an impact on those developments are likely to change important parameters on which projects or conversations with Indigenous groups may be based. Although these external factors created challenges for IPO-West and sometimes required shifts in the office’s activities, they were not found to have a major detrimental impact on IPO-West’s outcomes overall. Most commonly identified factors affecting the achievement of IPO-West outcomes included:

111. Current political and regulatory environment related to major oil and gas projects: A few interviewees noted that project activities can be impacted by regulatory and fiscal uncertainty, and complexity of major project approval processes. Given changing public and government priorities on environmental protection and the cancellation of a few major projects (e.g. Northern Gateway, LNG), some respondents raised concerns regarding IPO-West’s ability to manage communities’ expectations regarding the economic benefits of major projects.  

112. Diversity within and across Indigenous communities: There is a wide range of opinions and perspectives regarding energy development projects within and across Indigenous communities. Some community members are strongly opposed to any development and perceive IPO-West as an advocate for the major energy projects. In addition, changes within the communities (e.g., change in governance/elected leadership, community priorities) can quickly impact the established relationships and networks.  

113. Historical lack of trust in federal government: Although IPO-West has been successful in building strong relationships with many communities, rebuilding trust takes a long time, and can only work if communities perceive government efforts to be genuine (as opposed to the perception that government only comes ‘knocking when they need something’).

114. The economic imperatives and market dynamics of the oil and gas. For example, a drop in oil prices can impact investments and development of energy projects and affect future investments and economic opportunities in the communities.

115. Involvement of partners: The readiness of other government departments and stakeholders to work with IPO-West influences the conduct of activities and the outcomes of multilateral engagement. The horizontal nature of the WCEI and the complexity of the issues addressed often require the participation and collaboration of multiple partners to generate concrete results. Policy and governance changes within departments are likely to impact activities. Activities are impacted when new governments change the priorities and mandates of individual departments.  

To what extent have there been unintended outcomes (positive or negative) resulting from the program?

116. One of the positive unintended outcomes identified by key informants was related to the positive relationships developed not only between IPO-West and communities but also with other stakeholders participating in various activities (e.g. industry leaders, experts, etc.).  Through its activities and supports, IPO-West provided opportunities for various Indigenous communities to come together, collaborate and reach mutual understanding on common issues.

117. Another unintended outcome noted is the high expectations set by the IPO-West office with respect to the level of engagement and responsiveness on a broad range of issues.  Indigenous communities now expect that the federal government will continue to engage at the same level and with the same intensity (something individual departments may not be able to do given their limited resources and specific mandate).

118. A few representatives raised concerns about potential overlap of some of IPO-West’s activities related to issues or projects that are a priority for other partners and stakeholders (e.g. DFO in the area of marine safety, ISC in economic development, etc.). The current government priority of renewing its relationship with Indigenous Peoples has resulted in an increase in the number of government agencies seeking input from, and consulting with Indigenous communities on a variety of initiatives and programs. This increased demand for Indigenous engagement risks placing a further burden on Indigenous communities, which are already experiencing meeting fatigue.

119. Industry representatives suggested that they also engage with communities regularly and have provided a significant amount of funding (about $13M) for projects that could be eligible for funding by IPO-West.  They signed Mutual Benefit Agreements which among other clauses included provisions related to environmental monitoring, and capacity building (e.g. building technical expertise).

120. Finally, some communities do not have the capacity to respond to requests for proposals and may feel overwhelmed. Furthermore, lack of capacity within the communities to take on some of the projects may limit the economic benefits they receive (e.g. most of the project funding may go to outside experts and consultants).

Recommendation

121. Recommendation 4: Considering the significant engagement work initiated by IPO-West, and that additional work is being initiated by NRCan in this area, a lessons-learned document should be developed and used to support NRCan and other federal departments who may take on similar work, including the best practices developed from Indigenous capacity building and engagement activities.

MANAGEMENT RESPONSE AND ACTION PLAN

122. Agreed with Recommendation 4. IPO-West will develop a lessons learned document prior to the end of IPO-West’s initial WCEI mandate in 2018-19. This will include best practices developed from Indigenous capacity building and engagement activities; Share these lessons learned, within NRCan via the Policy Science Integration Committee (PSIC) and with other partner departments via existing horizontal mechanisms such as MPMO and Federal Councils, so that these lessons can be used to inform the design and consideration of future Indigenous partnership initiatives

Responsible Manager, Sector: ADM, Indigenous Secretariat and DG, IPO-West
Target Date: March 31, 2019

Conclusion

123. The evaluation of IPO-W demonstrated a continued need for IPO-West, and for sustained engagement with Indigenous groups, beyond project-specific consultations, e.g. consultations for liquefied natural gas projects or pipeline expansion projects which are under other federal organisations, such as the Major Project Management Office (MPMO).

124. While IPO-West plays a meaningful role within WCEI, initially there was some confusion as to the role of IPO-West and that of other departments in the initiative. Roles and responsibilities were clarified over time, but a few interviewees from other federal departments still feel that the roles of IPO-West remain unclear.

125. The Strategic Partnerships Initiative – West Coast Energy (SPI-WCE), administered by the Department of Indigenous Services Canada (DISC) and co-led by ISC and IPO-West, was key and provided $61M over five years in contributions (G&C) funding for projects led by Indigenous communities. Some stakeholders expressed concerns about the timeliness of funding decisions within the SPI-WCE DGIC. However, interviewees also indicated that some projects would not have been identified or funded at all without the work of IPO-West, who has contributed to building trust and good relationships with Indigenous communities, even some who are opposed to energy development projects.

Evaluation Team

Christian Asselin, Chief Audit and Evaluation Executive (CAEE)
Bill Blois, Evaluation Director
David Ash, Senior Advisor to the CAEE
Olive Kamanyana, Program Evaluation Manager
Edmund Wolfe, Analyst
Goss Gilroy Inc.

Appendix 1      Evaluation Methodology

This evaluation of IPO-West was conducted using three lines of evidence in order to collect and triangulate evidence from multiple sources. The references used for the results assessment included the program’s targets as well as the WCEI logic model (see Appendix 2). The evaluation focused on the aspects of the logic model that referred to IPO-West (e.g., outputs and outcomes attributable in whole or in part to IPO-West).  This was validated by the program representatives.

A short description of the three methods is provided below. The full methodological details for each line of evidence are presented below 

• Document Review: the team reviewed 40 documents including design documentation, program documentation, NRCan documentation (e.g., Departmental reports), and other sources (e.g., academic sources, government reports) to answer the evaluation questions listed below.
• 37 key informant interviews were conducted with representatives of Indigenous communities and Indigenous organizations (n=11), NRCan informants (including IPO-West representatives) (n=6), interviewees from other federal departments (n=12), interviewees from the province (n=4) and representatives of industry and other (n=4).
• A file review was conducted using a variety of program documentation, a sample of workshop/activity final reports, and a sample of communications documents.  The file review analysis also included a review of the program’s financial information.

This evaluation intends to answer the following questions:
Relevance:

1. Is there a continued need for IPO-West?
2. Does the program align with current federal government priorities?
3. Is there a legitimate and necessary role for the federal government in this area?

Efficiency and Economy

1. To what extent does the governance structure improve the delivery or effectiveness of the individual initiatives?  Of the entire initiative? 
2. Are there alternative delivery models that can achieve similar outcomes more efficiently?
3. To what extent have resources been used as planned to produce intended program outputs?

Effectiveness

1. To what extent is the program achieving the intended immediate, intermediate and final intended outcomes?
2. To what extent have external factors influenced (positively or negatively) the achievement of program outcomes?
3. To what extent have there been unintended outcomes (positive or negative) resulting from the program?

Key Informant Interviews

Key informant interviews were conducted to provide qualitative information on activities and current practice, context, outcome achievement, alternative delivery approaches and areas for improvement. Interviews were conducted in person (n=5) or by phone (n=30). Five individuals declined. Interviewees included the following:

• NRCAN:
  • IPO-West managers and staff (n=3);
  • Other NRCan representatives (n=2) ;
• Representatives from other departments and agencies (n=9);
• Representatives of Indigenous communities (n=6)  and Indigenous organizations (n=3);
• Representatives from the provincial government (n=4); and
• Representatives of Industry (n=3).

Some respondents declined or were not available for the interviews (including 2 federal government representatives; 2 community representatives; and 1 provincial government representative). Interview notes and summaries were typed into an electronic spreadsheet for analytical purposes.

Limitations

The main limitation associated with this line of evidence is the limited number of respondents interviewed. There are hundreds of Aboriginal leaders and thousands of community members that are potentially covered by the activities in which IPO-West is involved. There are also many stakeholders from the private sector and the province. The interviews only covered a partial list of potential interviewees. The interviewees were selected with the assistance of IPO-West and included individuals who were knowledgeable about IPO-West and/or its activities.

File and Administrative Data Review

A file/administrative data review was conducted for the Indigenous Partnerships Office-West (IPO-West) evaluation. IPO-West financial information, Meeting and SPI-Trackers, along with a sample of workshop final reports, meeting notes and communications between the office and Indigenous groups was provided by NRCan. 

The files and documentation were examined to answer Evaluation Question 6 pertaining to the efficient use of resources, and Evaluation Question 7 on the extent to which the program has been achieving its intended outcomes.

The main quantitative analysis was conducted using the information from the Meeting Tracker, which is used mainly to record instances of communications with Indigenous communities. The analysis was conducted on a version dated October 2017 and covers the period since the beginning of IPO-West’s activities in 2014. During conference calls on the availability and usability of files and program data, IPO-West staff mentioned that not every meeting is logged in the Tracker due to lack of time or attention. However, the content of the Tracker was described by Program staff as representative of IPO-West activities and interactions with Indigenous communities. Program leadership also specified that the Tracker is not used to track inter-departmental meetings and meetings with other government stakeholders. Minor discrepancies were found between the summary tables found in Meeting Tracker and those derived from our own analysis of the Tracker data. These differences are deemed minor (less than 10%) and have no incidences on the interpretation of the results.

The Tracker’s data was transferred into SPSS for easier manipulation and analyses were run to provide descriptive information, and namely to try and establish which types of issues were the most or the least successfully addressed through the listed meetings.

Limitations

This file review yields some indication on the extent to which IPO-West is producing expected outputs and is achieving immediate outcomes. There is no direct evidence from the files about the extent to which the program achieves its expected intermediate and final outcomes. At this point, the following gaps have been identified in program information: up-to-date information on the Tripartite Dialogues (post November 2015); information of the Federal-Provincial Funder’s Task Group; as well as documentation on the agreement between Employment and Social Development Canada, Service Canada-BC and IPO-West regarding Skills and Partnership Fund.

Document Review

This report presents the findings of the document review conducted for the Indigenous Partnerships Office-West (IPO-West) evaluation. For the purpose of answering evaluation questions, documents (provided by NRCan and found through external research) were examined. Relevant information was filed into a document matrix. Important citations and document summaries compiled in the matrix were then used to draw conclusions under each indicator and under each evaluation question regarding the relevance, performance and efficiency of IPO-West.  It should be noted that not all evaluation questions and indicators listed in the evaluation matrix were covered in the document review – those remaining questions will be answered using other lines of evidence.

In total, over 36 documents were analyzed (a bibliography is appended). Below is a list of types of documents used in the review:

• Design documentation (e.g., Treasury Board submission) detailing the purpose, mandate and of the Office upon its launch;
• Program-specific documentation (e.g., Report to Treasury Board, minutes, meeting notes, etc.), including IPO-West’s performance documents (e.g., Business Plan, Performance Information Profile);
• Government documents (e.g., Mandate Letters, Speeches from the Thrones, policy documents, government reports);
• Departmental documentation (e.g., NRCan Reports on Plans and Priorities and Departmental Plans);
• Non-governmental documentation (e.g., academic publications, third-party reports, articles, summaries of court decisions, etc.).

Not all documents listed proved useful in answering evaluation questions regarding IPO-West specifically– some will be mobilized for the horizontal evaluation of the West Coast Energy Infrastructure Initiative (WCEI) more generally. Furthermore, some of the documentation did not directly answer evaluation questions, but provided important contextual information which is also summarized in the findings below.

Limitations

Most of the documentation which provides evidence on the extent to which IPO-West has achieved its intended outcomes does not capture the most recent activities of the program. For instance, the Report to Treasury Board (2016) does not provide information on 2016-2017 results. Other lines of evidence will complement the finding of the document review to establish clearly to what extent the program is currently meeting its objectives. Also, there are few documents that provide evidence that directly demonstrate the achievement of intermediate and ultimate outcomes of IPO-West.

Appendix 3:     Program Information

The West Coast Energy Infrastructure Initiative (WCEI) was launched in 2014 to facilitate a coordinated federal response to Indigenous concerns related to major energy project on the West Coast. The objective of the WCEI is to create both the environment and the mechanisms necessary to effectively engage with key Indigenous leaders to address priorities, issues and concerns identified by Indigenous communities. Ultimately, WCEI is intended to ensure that federal engagement with Indigenous communities is translated into concrete actions and investments to address Indigenous priorities.

To this end, federal spending was targeted to facilitate Indigenous participation in West Coast energy infrastructure development in areas such as business development, skills and employment, early and ongoing engagement, and environmental action.

WCEI is a horizontal federal initiative comprised of different components delivered by four departments:

• Natural Resource Canada’s (NRCan) Major Projects Management Office – West (MPMO-W), rebranded in October 2017 to Indigenous Partnerships Office-West (IPO-W);
• Environment and Climate Change Canada’s (ECCC) Cumulative Effects Monitoring Initiative (CEMI) pilot;
• ISC’s  Strategic Partnerships Initiative – West Coast Energy (SPI-WCE); and
• Employment and Social Development Canada’s (ESDC) Skills and Partnership Fund - West Coast Energy (SPF-WCE).

Other federal departments are also involved and are member of the Directors General Investment Committee (DGIC):

• Western Economic Diversification Canada (WD) – member of SPI- WCE DGIC, including on relevant project development.
• Fisheries and Oceans Canada (DFO) – member of SPI-WCE DGIC; lead on fish habitat restoration initiatives.
• Transport Canada (TC) –member of SPI-WCE DGIC, lead on World Class Tanker Safety

Through the WCEI, NRCan provides a single window to build relationships with Indigenous communities outside of the formal Crown consultation process, and to work across multiple federal departments to facilitate Indigenous engagement and participation in West Coast energy infrastructure development. This engagement is oriented towards a range of priorities expressed by communities (e.g. land use planning, environmental remediation, training and employment) and is not linked to any specific major project consultation or accommodation undertaking.

Components

The Indigenous Partnerships Office – West (IPO-W)

The creation of IPO-West responds to the recommendation in Mr. Eyford’s report that Canada should build on the model it has developed with the Major Projects Management Office, and establish a sustained presence of senior officials on the ground in British Columbia, with capacity to enable Crown engagement and conduct consultations with Indigenous communities on key issues related to the development of energy infrastructure. Delivered by NRCan, IPO-West was provided with a budget of $16.4M (O&M) for 2014-2019. It is located in Vancouver and has staff presence in Ottawa.

IPO-West engages with Indigenous communities to develop relationships, identify priorities and issues related to West Coast energy infrastructure development, and seek to respond to and address these priorities and issues.  IPO-West assists the Minister of NRCan to engage Special Ministerial Representatives (SMRs) to work with communities and was expected to be the secretariat for the Tripartite Energy Forum. The Forum did not go forward as planned but was intended to provide a venue for senior federal and BC officials and Indigenous leaders to share information in order to enable a collaborative approach to solutions that benefit communities while also advancing development. 

The Strategic Partnerships Initiative – West Coast Energy (SPI-WCE)

In response to the Eyford Report recommendation to help prepare Indigenous communities for economic opportunities, and address Indigenous priorities and issues associated with energy infrastructure development,  $61M in G&Cs funding was provided over five years to the Strategic Partnerships Initiative -West Coast Energy. Activities are targeted to four priority areas: early and ongoing engagement; creating jobs and growth; environmental action; and fish habitat restoration. Many of the Indigenous communities impacted by proposed energy projects are small and remote, and have limited human and financial resources to engage with industry proponents.

SPI-WCE funding is intended to enable these communities to effectively participate in a dialogue with industry and government. It is also recognized that some local Indigenous groups lack the capacity to undertake work that would allow them to assess the environmental, socio-cultural and economic impacts of multiple proposed projects through their traditional territories. SPI-WCE funding dedicated to environmental action, engagement, jobs and growth, and fish habitat restoration supports Indigenous communities to participate in and undertake projects to preserve, protect, and restore areas affected by energy infrastructure development. Targeted areas for SPI investment in regions impacted by West Coast energy development are identified through IPO-West engagement with both Indigenous groups and the provincial government, and by on-the-ground intelligence garnered from INAC and SPI partner departments. Regular (average once every six to eight weeks) meetings of the interdepartmental Directors General Investment Committee (DGIC) which reviews and recommends for approval or rejection all projects under consideration for financial support under SPI-WCE. 

Skills and Partnership Fund - WCE (SPF-WCE)

Improving the economic prospects of Indigenous individuals and communities was identified in the Eyford Report as a condition for Indigenous support for major energy infrastructure projects. Employment and Social Development Canada (ESDC) extended its Indigenous Employment Opportunities- Skills and Partnership Fund (SPF) program to fund projects that support West Coast energy infrastructure development in BC, and to employ Indigenous people in BC’s oil and gas sector between 2014-15 and 2016-17. To be eligible for funding, all projects must have met the selection criteria of supporting training-to-employment for Indigenous peoples in BC, while also benefitting West Coast energy development. 

SPF is a proposal-based program that encourages stakeholders, such as training institutions, community organizations, local businesses and industry, to partner with Indigenous organizations to support skills development for Indigenous people. These partnerships are intended to address a broad range of Indigenous socio-economic issues, while also better meeting labour market demand.  Only pre-existing budgets were to be used by SPF for WCEI (no additional funds were provided by the initiative).

Consultation and updates are provided weekly to management (ESDC-SC) on progress of negotiations and existing project. ESDC participates in the Funders Task Group to align work and funding efforts every quarter. As well, information is provided at the interdepartmental Directors General Investment Committee (DGIC), which reviews and recommends for approval or rejection all projects under consideration for financial support under SPI-WCE

Cumulative Effects Monitoring Initiative (CEMI)
For the purposes of the program, “cumulative environmental effects” is defined as a change in the environment caused by multiple interactions between human activities and natural processes that accumulate across space and time. Following the Eyford recommendation to address concerns regarding the cumulative effects of major energy projects in BC, and in light of the limited cumulative effects work in BC at the time, ECCC was tasked to develop a pilot Cumulative Effects Monitoring Initiative, in collaboration with the Government of BC and Indigenous communities. Under CEMI, ECCC was to engage relevant parties to scope the location for two pilot projects and support Indigenous participation in scoping and designing a monitoring system. These included a project with First Nations in Prince Rupert involving monitoring of marine-based species, and one with Burrard Inlet/ Howe Sound First Nations. As energy infrastructure spans a number of BC’s ecosystems and impacts close to 100 Indigenous communities, the choice of location of the monitoring pilots required sensitive negotiations involving multiple considerations around scientific, policy and Indigenous issues.

For this initial funding cycle, the focus was to make a final decision on the location of the two areas for the pilots, and on the development of the monitoring pilots themselves, including scoping and design issues.  Opportunities for Indigenous engagement in implementation of monitoring on an ongoing basis would also be identified. CEMI was provided with a budget of $662K in O&M and $2.7M in G&Cs to deliver this component of WCEI (for 2014-2019). When this report was written, it was learned that the pilots would not be supported by CEMI, and that other ECCC programs could support these monitoring activities.

Appendix 4: IPO-West Financial Information

Notes: ¹Additional funds were allocated for the Indigenous Advisory and Monitoring Committees beginning in 2017-18. These funds are not included in the table as the implementation of the Committees is outside the scope of the evaluation. Source: NRCan Financial Information

APPENDIX 5: IPO-WEST ENGAGEMENT IN BRITISH COLUMBIA

Source: IPO-West spreadsheet; as of March 2018)

APPENDIX 6: EXAMPLES OF ISSUES RAISED AND DISCUSSED

Source: Final Event Reports, based on a sample of six events