Evaluation of the Geo-Mapping for Energy and Minerals (GEM-2) Program

Mineral and energy exploration and production are primary economic drivers in Canada’s North. They provide opportunities for sustainable employment and skills development, and the development and expansion of infrastructure to support wider economic growth.^{Footnote 1} To undertake exploration activities in the North, companies rely on pre-competitive and foundational geoscience maps and information to help them identify prospective areas of resource potential. Interview and documentary evidence indicated that companies would typically not invest in the production of such basic geoscience research,^{Footnote 2} and even less so in the northern context where the costs of conducting exploration are very high.^{Footnote 3}

The federal government’s production of “low-cost, high-resolution, and high-quality geological information in northern Canada’s vast territory and remote environment [was thus] fundamental to reducing the investment risks and uncertainties associated with resource exploration and development in the north.^{Footnote 4} Survey data confirmed this need. When asked to rate the need for geoscience to help identify areas of mineral or energy potential, 38 responses from external respondents were received, and the majority of them agreed. Of the 11 industry responses, 82% (n=9) rated the need for geoscience information to help identify areas of mineral or energy potential, and the need for geoscience information to reduce the cost of minerals or energy exploration activities, as “very” or “extremely” relevant to them or their organization.

Documentary and interview evidence indicated that a significant portion of the North (i.e., above 600 north or the area north of the limit of discontinuous permafrost) had not been mapped in 2008 (when GEM-1 was launched). Following the completion of GEM-1 in 2012, an area equivalent to the total area of Quebec, Ontario and Manitoba remained to be mapped to a level appropriate to assess resource potential. Phase 2 of the GEM program was expected to fill this identified gap by 2020.^{Footnote 5} As well, interviewees noted that geological knowledge was a rapidly evolving area since modern geological concepts, methods and technologies (e.g., geochemistry, age-dating techniques and tectonics) created a need for new and updated maps.

Survey and interview evidence suggested that the needs of the GEM-2 program would continue beyond 2020. Some internal and external interviewees and a few internal survey respondents believed that the program would not cover all of the strategic areas of Canada’s North by 2020. Additionally, a few interviewees and a few survey respondents stated that the need for more detailed/lower-scale maps and updated versions of earlier rounds of geological maps (including GEM-1 maps) would remain after 2020. The need for more detailed maps was expressed primarily by representatives of the three territories, and by an industry key informant interviewee representing the views of several mining companies, who stated that the “…north should be mapped completely at 1:100 000 and after that [GEM 2 staff should] pick out interesting spots and map at 1:50 000.” Given the national focus of the GEM 2 program, a few internal stakeholders questioned whether the need for smaller-scale maps was consistent with the role of the Government of Canada and NRCan.

Documentary and interview evidence showed that updated geoscience products were necessary to inform land-use planning and management by public agencies and Indigenous stewardship organizations in order to facilitate responsible resource development. Some interviewees explained that the three northern territories had limited financial and scientific human resource capacities to produce the geoscience maps that were needed to inform decision-making in these areas. In Nunavut for example, the Canada–Nunavut Geoscience Office supported government geoscience activities. In this context, some external interviewees saw GEM-2 as an opportunity to build capacity in the North in the form of partnerships between the Geological Survey of Canada (GSC) and territorial surveys, as well as scientific training and job opportunities for Northerners.

Survey and interview evidence pointed to a growing need for geoscience information to help address climate change impacts. Sixty-three percent of the external survey respondents rated the need for geoscience information to inform Indigenous authorities’ and governments’ decision-making regarding environmental planning and protection as “very” or “extremely” relevant. Industry respondents were more likely than non-industry respondents to point to the need at 82% (n=9) and 56% (n=15) respectively. Some interviewees stated that GEM-2 geoscience products could provide highly useful baseline knowledge to inform infrastructure planning in the North (e.g., industry platforms; local buildings, roads and ports) and to study the effects of climate change on terrain stability (e.g., effect of melting permafrost).

The use of geoscience products by regional stewardship organizations for land-use and environmental protection purposes would also require the development of capacity aimed at combining geoscience data with traditional knowledge. The SRRB case study found that the support from the GEM-2 program facilitated initial capacity development in combining geoscience and traditional knowledge.

While the GEM-2 program was generally perceived as contributing to meeting the capacity gap in the North, interview and survey data suggested that more could be done. Survey data indicated there were further opportunities for developing tools and methods to facilitate the use of GEM-2 products by community and regional organizations. While 44% (n=12) of the non-industry respondents reported that such tools were “very” or “extremely” important to them or their organization, only 8% (n=2) reported that the program had addressed the need for such tools to a “large extent or entirely”. A further 29% reported the program had not addressed this need at all or had addressed it only to a small extent (n=7). The program life cycle may be a factor in the lack of tools and methods to facilitate the use of GEM-2 products by community and regional organizations, since the evaluation covered the first four years of a seven-year program. However, the program will need to consider this data as it continues its community engagement efforts.

3.2 Consistency with government priorities and NRCan strategic objectives

Finding #2:     GEM-2 program objectives were consistent with Government of Canada (GC) priorities regarding support for economic and social development of the North, and were also aligned with NRCan’s Strategic Objective 1: Canada’s Natural Resource Sectors are Globally Competitive.

Alignment with federal government priorities

Initial funding for the GEM program (GEM-1) was announced by the federal government in its 2008 Budget, at which time it noted that “publicly available geoscience maps help companies target new exploration activity to the areas of highest mineral potential.^{Footnote 6} ; A year later, in Canada’s Northern Strategy, the GEM program was identified as a component of the action to be taken to support economic and social development in the North, one of four priorities in the strategy.^{Footnote 7} The policy statement noted that the program’s results would “highlight areas of mineral and petroleum potential, [and] lead to more effective private sector exploration investment and create employment opportunities in the North.^{Footnote 8} A subsequent publication, Achievements under Canada’s Northern Strategy, 2007–2011, saw the value of geo-mapping data as not only contributing to private sector exploration activities, but also for informing land use decision-making, such as the creation of protected areas and parks.^{Footnote 9}

In its 2012 report, Resource Development in Northern Canada, the House of Common’s Standing Committee on Natural Resources recommended that:

. . . the Government of Canada continue to support geoscience programs in order to improve the development prospects of northern energy and mineral resources, and specifically renew the geo-mapping for energy and minerals (GEM) program to another productive five-year term.^{Footnote 10}

In response to the Standing Committee’s report, the GC noted that “renewed investment in GEM will be considered through the process of priority setting and fiscal planning of the Government.” In Budget 2014, the federal government committed a further $100 million over seven years (2013–2020) for GEM as part of a package of initiatives to support the economic and social development of the North by investing in public infrastructure.^{Footnote 11}

Alignment with NRCan’s strategic objectives

The Prime Minister’s mandate letters to ministers translate the government’s overall vision into sets of specific policy objectives and challenges to be addressed as a matter of priority by each minister. In 2015, the Minister of Natural Resources’ letter identified that “getting natural resources to market” was a “core responsibility of the federal government” but had to be achieved by “addressing environmental, Indigenous Peoples’, and local concerns.” In doing so, the “overarching goal is to ensure that the resource sector remains a source of jobs, prosperity, and opportunity within the context of a world that increasingly values sustainable practices and low carbon processes.”

During the period of the evaluation (2013–14 to 2016–17), the GEM-2 program was one of the four programs expected to contribute to the achievement of Strategic Outcome #1: Canada’s Natural Resource Sectors are Globally Competitive in the NRCan Program Alignment Architecture (PAA).^{Footnote 12} More specifically, the GEM-2 program was a stand-alone sub-program (1.3.3) in Program 1.3, Investment in Natural Resource Sectors^{Footnote 13} and was expected to advance Strategic Outcome 1 by contributing to the following result: “Natural resource sectors have increased investment.^{Footnote 14} The intent was to make the GEM 2 products available to industry investors, land use planners, governments and community agencies to inform resource exploration and development which would help increase exploration investment in Canada’s north.

Besides the investment through direct program spending to increase geoscience knowledge, investment in public geoscience programs such as GEM-2, which help to identify areas for minerals and energy potential, is deemed to spur private sector investment in exploration.^{Footnote 15} A 2010 study conducted by the Prospectors and Developers Association of Canada (PDAC), indicated that for every $1 million invested in public geoscience, $5 million in exploration expenditure would be generated.^{Footnote 16} GEM-2 could therefore be expected to generate more than $500 million in economic activity within 10 to 15 years, through private sector exploration for new energy and mineral resources.^{Footnote 17}

3.3 Alignment with federal roles and responsibilities

Finding #3:     Generating geoscience data and knowledge in the North is a legitimate and appropriate role for the federal government. The Department of Natural Resources Act and the Resources and Technical Surveys Act provide the legal authority for the Geological Survey of Canada (GSC) and, by extension, GEM-2. As per the Intergovernmental Geoscience Accord (IGA), the federal government is authorized to deliver research programs that are thematic and national in scope and that can collaborate with the provinces and territories on programs that have common objectives. The federal government complements rather than duplicates the work of the provinces/territories, academia and industry, and helps to fill their capacity gaps. It has done so by providing substantial funding, appropriate expertise, and by covering different areas, considering different scales and timelines and producing cross-border geoscience knowledge.

Both the federal and provincial/territorial jurisdictions have a role to play in the provision of geological survey and geoscience services in Canada. The Department of Natural Resources Act^{Footnote 18} and the Resources and Technical Surveys Act^{Footnote 19} provide the legal authority for the Geological Survey of Canada (GSC) and, by extension, GEM-2. The respective roles and responsibilities of the F/P/T geological surveys are defined in the Intergovernmental Geoscience Accord (IGA).^{Footnote 20}

According to the IGA, the GSC role is “to carry out geoscience programs that are typically thematically based and national in scope and significance,” whereas the P/T surveys would carry out programs “at scales appropriate to addressing provincial or territorial geoscience priorities.” Furthermore, the P/T geological surveys are the “principal stewards of and resident authorities for public geoscience in their jurisdiction” and “contribute to a systematic description of the geology of the province or territory which may include its mineral, energy and groundwater endowment.” As a result, geoscience activities in Canada, including the production of GEM-2 geoscience products, are undertaken on a partnership basis through work which is coordinated among and between the F/P/T geological surveys. With respect to the GEM-2 program, the scope and activities of the respective parties are defined in task-shared agreements (TSAs), with each party funding and staffing their activities.

The majority of external and internal interviewees confirmed that the federal government worked in concert with the provinces and territories, with no major overlap or duplication. They said that the GSC brought different expertise, covered different areas and considered different scales and timelines. It also produced cross-border geoscience knowledge, including looking at frontier areas, which provided provinces/territories with useful cross-jurisdiction information/learning.

Some interviewees noted that the F/P/T roles in the production of geoscience information might continue to evolve as the devolution of federal powers and authorities progress with regard to land and resource management to territorial governments. They believed that the federal government would remain an essential player because of the scientific, human resource and financial capacity limitations of the territories as well as the provinces. For instance, an internal interviewee noted that work in the Arctic was too complex and expensive for any other jurisdiction or organization to invest in without the substantial involvement of the federal government.

Studies^{Footnote 21} have also shown that, from an economic perspective, the generation and dissemination of geoscience data and knowledge was a legitimate, appropriate and necessary role for governments. Pre-competitive government geological information typically addressed market limitations, because the production of geological maps and data would not be undertaken to the same degree by the private sector in the absence of government geological research.^{Footnote 22},^{Footnote 23} In the context of the GEM-2 program, this meant that there would be less industry interest and investment in the North, which would in turn affect the northern economy as it is largely driven by mineral and energy exploration and development, as previously discussed.

4 Findings: Performance

4.1 Achievement of expected outcomes

4.1.1 Outputs

Finding #4: Overall, outputs related to collaboration agreements and northern geoscience products were produced as planned. However, outputs related to knowledge transfer and dissemination, community-level stakeholder engagement, and tools and methods to facilitate the use of geoscience by northerners were more limited in terms of availability and effectiveness. Collaboration agreements with academic researchers and P/T geological survey partners Documentary, interview and case study evidence indicated that GEM-2 established and implemented collaborative agreements aligned to GEM-2 priority research themes and project areas as planned. Interviewees and case study participants involved in these agreements expressed positive views on the process that led to the signing of the agreements, whether with provincial and territorial surveys or with academia. A few interviewees suggested that the Task-Shared Agreements (TSAs) be streamlined to reflect their purpose. As well, one key informant suggested that the TSA be regularly reviewed and updated to reflect changes in priorities, and human and financial resources. Knowledge transfer to promote and disseminate northern geoscience products   All lines of evidence indicated that knowledge transfer consisted primarily of presentations at industry and geoscience fora and symposia, as well as the placement of knowledge products on GEOSCAN and the publication of articles in academic/industry journals. Survey evidence indicated that industry particularly valued scientists’ presentations at conferences and geoscience fora because they could discuss the implications and possibilities of the research directly. The absence of scientists responsible for the research being presented was noted as a limitation on the effectiveness of GEM research dissemination. As well, challenges accessing information on GEOSCAN were identified in the interview evidence with the suggestion that better organization of information would facilitate accessibility. Community-level stakeholder engagement Evidence from the various lines of evidence showed mixed viewpoints on the success of community engagement activities. On the one hand, project staff were largely successful in obtaining the social license to operate as a result of community engagement activities conducted at the beginning of research activities. Efforts made to hire local people contributed to this success. Interview and survey evidence on the other hand, showed a marked difference between internal and external viewpoints with respect to the effectiveness of the community engagement activities. Internal respondents tended to view these activities as being more effective than external sources. Interview and survey data also pointed to a need for improved engagement activities aimed at increasing awareness of GEM-2 outputs and at developing tools and facilitating their use by northerners and their institutions in the longer term. Production and release of northern geoscience products Principal geoscience outputs from the GEM-2 program were for the most part produced or undertaken as planned. There were some delays experienced in the fieldwork activities and the production of GEM-2 products due to the availability of scientific resources, weather and logistical factors, and the ensuing limited time to prepare final deliverables. While some activities and deliverables experienced delays, mid-year and year-end project reports indicated that these factors did not significantly impact overall project timelines. Tools and methods to facilitate the use of GEM-2 geoscience products by Northerners Nine multidisciplinary projects were funded to develop tools to facilitate the use of geoscience information by Northerners. All evidence pointed to the value of the multidisciplinary grants, and the utility of the projects themselves.  However, tight timelines, northern capacity gaps and insufficient program assistance limited the ability of proponents to undertake planned work and maximize participation.

Collaboration agreements with academic researchers and P/T geological survey partners
Geoscience research activities carried out in collaboration with P/T geological surveys and university-based researchers had to comply with the terms of the IGA and were formalized through the establishment of a Task Shared Agreement (TSA) with participating stakeholder organizations. Each party to the TSA agreed to independently fund and perform the identified activities. The resulting research findings were to be shared between the participating organizations and jointly published. Research findings of the joint projects were expected to be presented at the Geological Open Houses of the P/T geological surveys, as per commitments made in the IGA. Also, the knowledge products were made publically available at no charge through the publications websites of the GSC and its partners.

Interview and case study evidence indicated that GEM-2 established and implemented collaborative TSAs aligned to GEM-2 priority research themes and project areas as planned. Interviewees and case study participants involved in these agreements expressed overall positive views on the process that led to the signing of the agreements.

There were no major issues raised regarding the timeliness, complexity or fairness of the process. In the context of the NTGS case study, the TSA was generally considered to reflect the discussions between the GSC and the NTGS to set the parameters of the projects. When prompted on the matter, interviewees felt the whole process was extremely effective and simply formalized what was already occurring.

The TSA process was the same for collaborations with other jurisdictions as it was for collaborations with academics. A few internal interviewees suggested streamlining the TSA process to include only those paragraphs that were relevant to the other party to the agreement. Several issues that applied to academia for example, did not apply to other jurisdictions. Further, an NTGS case study informant suggested that TSAs with P/T geological surveys should be regularly reviewed and updated to reflect changes in budgets, priorities, and/or the availability of staff with the required skills.

Knowledge transfer activities to promote and disseminate northern geoscience products

According to interview and case study evidence, the primary means of promoting GEM-2 geoscience products to potential users in industry and, to a lesser extent, government ministries, Northerners and their institutions, was via presentations by GEM-2 staff at P/T geological survey geoscience fora, industry and academic research conferences.  The NTGS case study data suggested that geoscience conferences/fora were highly regarded by industry representatives because they were able to obtain advance information on GSC activities and pending outputs. They particularly valued the opportunities for direct interaction with the researchers at these events. However, documentary evidence,^{Footnote 25} as well as some interviewees and case study informants, indicated that the absence of the scientists who conducted the research reduced the opportunities for interaction and thereby the effectiveness of GEM-2 presentations at these fora. The evidence indicated that travel restrictions/caps imposed by NRCan were the reason for the absences.

In some cases, only a single or sometimes no GEM-2 scientist, was present to deliver research results of a project. For instance, external case study interviewees noted the absence from the Yellowknife Geoscience Forum of GSC scientists involved in collaborative work with the NTGS, stating that this impeded the dissemination of results. In the case of the Hudson-Ungava project, mid-year and year-end reports noted that a travel cap and ensuing limitation on domestic travel had a negative impact on the dissemination of results. In that regard, NRCan appears to have applied a restrictive interpretation of the Treasury Board Directive on Travel, Hospitality, Conference and Event Expenditures, which stipulates that departments are responsible for “limiting the number of public servants attending a conference from a department to the minimum necessary to achieve the departmental objective.^{Footnote 26}

All GEM-2 open file and data products were available and downloadable from the online GEOSCAN database of publications. A key informant representing the views of several exploration companies mentioned that this search engine provided too many results for a given search enquiry. This made it difficult for industry users to identify relevant information and therefore affected companies’ ability to access GEM-2 information. The informant suggested that search results should be distinguished using different categories (e.g., geological studies, thematic geological maps, large-scale geological maps, geochemical studies, environmental studies, ground water studies). The same key interviewee also suggested building on Ontario’s “highly regarded” survey tool, which exported data to Google Earth.^{Footnote 27}

Community-level stakeholder engagement activities

A 2012 evaluation of the first phase of GEM recommended that “NRCan should clarify the program objective with respect to the role of community engagement and developing appropriate strategies and approaches.^{Footnote 28} As a result, the GSC developed a northern engagement strategy which established the framework through which GEM-2 would operate in relation to communities. In accordance with the GEM-2 Engagement Strategy, community engagement activities would occur at the beginning, throughout, and at the end of the projects being undertaken. 

Based on documentary evidence to this point in the program cycle, community-level engagement heavily focused on explaining and getting approval for planned fieldwork, presenting the results of prior season fieldwork, and resolving issues affecting support and participation by communities and residents.^{Footnote 29} There was some limited evidence of engagement activities aimed at increasing and sustaining awareness of GEM-2 outputs and at facilitating their use by community or regional organizations in the longer-term. All lines of evidence indicated that the GEM program successfully created awareness of the field research activities resulting in the social licence to operate. However, in other areas related to community engagement, there was a marked difference in perception about the effectiveness of the community engagement activities. Sources internal to the program viewed them as largely effective while external sources did not.

Survey respondents who participated in community engagement activities were asked to rate the effectiveness of these events in achieving elements of community engagement. Eleven non-industry and 17 internal responses were received. Results are presented below in Table 4. While the activities were rated as effective for the most part, marked differences in viewpoints emerged between the internal and non-industry stakeholders (no industry respondents indicated participation). 

Overall, the groups rated the activities related to employing community members and/or companies to assist with the conduct of GEM field work as effective, at around 43%. However, 29% of the internal respondents rated the practices as “very effective”, whereas only 9.1 % of non-industry respondents did so. In considering the effectiveness of identifying and responding to community concerns and interests regarding proposed GEM activities, 65% of the internal group rated it as effective compared to 46% of non-industry respondents. Again, internal respondents were much more likely than non-industry respondents to rate it as “very” effective at 24% and 9% respectively. Both groups’ views coalesced in only one case: that activities in keeping local communities and other stakeholders aware of the progress and results of GEM activities were effective (68% overall, 64% non-industry, and 71% internal).  

Community engagement activities aimed at providing guidance and assistance to northern community members and organizations to understand and use the GEM geoscience products were more likely to be rated as effective by the non-industry respondents at 46%, versus 18% for internal respondents who were much more likely to respond “don’t know”. Of note in this instance however, is that all of the non-industry respondents who responded very ineffective were from stakeholders with the strongest ties to the community.

Table 4:  Effectiveness of Community Engagement Activities : Survey Results
		% Response
Q 12	# of responses	Very ineffective	Ineffective	Effective	Very effective	Don't know /NA	Total
Identifying and responding to community concerns and interests regarding proposed GEM activities	28	0.0	14.3	57.1	17.9	10.7	100.0
Non-industry	11	0.0	18.2	45.5	9.1	27.3	100.0
Internal	17	0.0	11.8	64.7	23.5	0.0	100.0
Employing community members and/or companies to assist with the conduct of GEM field work	28	0.0	14.3	42.9	21.4	21.4	100.0
Non-industry	11	0.0	18.2	45.5	9.1	27.3	100.0
Internal	17	0.0	11.8	41.2	29.4	17.6	100.0
Keeping local communities and other stakeholders aware of the progress and results of GEM activities	28	3.6	10.7	57.1	10.7	17.9	100.0
Non-industry	11	0.0	9.1	54.5	9.1	27.3	100.0
Internal	17	5.9	11.8	58.8	11.8	11.8	100.0
Providing guidance and assistance to northern community members and organizations to understand and use the northern geoscience products generated by the GEM program	28	7.1	21.4	28.6	3.6	39.3	100.0
Non-industry	11	0.0	27.3	45.5	0.0	27.3	100.0
Internal	17	11.8	17.6	17.6	5.9	47.1	100.0

Source: GEM-2 survey data compiled by NRCan Evaluation Division, March 20, 2018

Documentary evidence and some interviewees and case study participants also noted that efforts had been made by the program to ensure that communities were consulted and were aware of, and participated in, fieldwork activities. The following examples were identified:

• The Advisory Group of Northerners (AGN) continued as an interface between the program and the communities.
• Efforts were made by program management and scientific staff to introduce the program, discuss the research projects, and present results during community visits.
• Some GEM-2 researchers conducted more extensive engagement activities, such as handing out reports and a non-scientific photo book to locals, providing data in GIS format to be uploaded on a community web platform, or arranging for a community to visit the research camp.
• Local businesses and workers were hired whenever possible, although a few interviewees and project managers^{Footnote 30} noted that it was difficult to find local workers with the appropriate level of qualification, and local businesses were often expensive.

A few internal and external interviewees and NTGS case study participants noted that the program operated in a context where there was sometimes a lack of interest in, receptivity to, and understanding of GEM-2 geoscience work and the long-term benefits that might result from the research. Local communities could be difficult to reach on the phone and by email, and sometimes very few people attended local events organized by GEM-2 to present the program and projects.
Interview evidence supported the case study findings. Some internal and external interviewees felt that substantial improvements to the GEM-2 engagement strategy were needed to build stronger and more sustainable relationships with northern communities. The list below reflects suggestions provided by interviewees and survey respondents to improve engagement efforts.  

• While program documentary evidence^{Footnote 31} suggested that AGN’s recommendations were incorporated into GEM-2’s communication plan, a few interviewees felt that further efforts could be made to build on AGN’s advice to improve communications with communities and to hire local people.
• Engagement officers should have expertise in how to engage with Indigenous communities specifically, not just engagement expertise in general (it was suggested by internal and external interviewees that GEM-2 might consider hiring Northerners to fill that role).
• More efforts could be made toward capacity building by providing more training and job opportunities to local Indigenous people, so they are better equipped to analyze and use the geological knowledge produced by GEM-2 for land use planning and management purposes. This need was also expressed by a few external survey respondents.
• There should be more frequent and widespread communication of the projects’ progress at the regional and local level (e.g., expand the letter distribution list to include regional associations; interact with regional and community research-oriented institutions). A few external survey respondents also identified this need.
• A few interviewees expressed the need for GEM-2 to clearly articulate and communicate the “day-to-day” benefits for Northerners that might result from GEM-2 work. They suggested this should be done early in a project’s lifecycle to ensure community buy-in. A few external interviewees and survey respondents also advocated the production of plain language summaries of project activities that would be regularly updated.

Production and release of northern geoscience products

Documentary evidence indicated that data collection and analysis, and the production of related geoscience products (e.g., summary of activities’ reports, research papers, maps), were mostly on track throughout the period from 2014–15 to 2016–17.^{Footnote 32} Table 5 presents the release of GEM-2 products by year.

Table 5: Release of GEM-2 Geoscience Products, 2014-15 to 2016-17

	2014–15	2015–16	2016–17
Number of new GEM-2 products made available for public release on GEOSCAN	120	217	137
Cumulative number of GEM-2 products made available for public release on GEOSCAN	139*	356	493

* The cumulative number of GEM-2 products in 2014–15 includes 19 GEM-2 products published by third-party organizations that were listed on GEOSCAN prior to 2014–15.

Source: GEM-2 data on product releases and downloads provided to Science-Metrix, July 2017

Documentary, interview and case study evidence pointed to some delays in the fieldwork activities and the production of GEM-2 products. Delays in fieldwork activities were typically caused by challenges in the logistical arrangements and timing (for example, arrangements for fuel positioning, equipment supply), poor weather during planned fieldwork periods, availability of key scientific resources-skills, as well as difficulties in hiring students and local field support. GEM-2 project managers reported that measures were taken in subsequent years to mitigate delays in order to ensure all planned deliverables would be produced by 2020. Challenges encountered were overcome, in some instances, by the willingness of P/T geological survey partners to modify their work programs, logistical support arrangements and/or deployment of staff.^{Footnote 33}

According to some interviewees and case study participants, delays in the preparation of the final deliverables were very frequent, because GEM-2 scientists had to share their time between the preparation of final deliverables and competing demands from other projects. While such delays were deemed inevitable by some, a few suggested that better planning and better allocation of resources could help accelerate the release of GEM-2 products.

Tools and methods to facilitate the use of GEM-2 geoscience products by Northerners

In 2015–16 and 2016–17, GEM-2 funded nine multidisciplinary grants to develop innovative approaches and tools to facilitate the integration and use of GEM-2 knowledge into decision-making regarding land-use planning and resource development.^{Footnote 34}. In relation to the overall GEM grant budget, these funds represented about $400K (or about 25% of the total grant budget over the two-year period referenced). The grant recipients were mostly Indigenous community and regional organizations, but also included the Nunavut Arctic College and the University of Western Ontario. Projects typically sought to achieve combinations of the following objectives:

 

• Provide training and skills development in the use of GIS tools, software and data
• Develop tools such as databases and web-based database interfaces (apps)
• Introduce participants to GEM-2 geoscience products
• Compile and/or integrate traditional Indigenous knowledge with GEM-2 geoscience knowledge
• Increase understanding of geoscience data and its application to resource and land-use planning and management

End-of-project reports submitted by the grant recipients indicated that projects aligned to the objectives listed above. For instance, the SRRB projects led to the production of interactive maps integrating GEM-2 geological, archaeological and industry activity information and traditional Indigenous knowledge about the Sahtú region. The project was designed to increase communities’ awareness of their land and contribute to more informed land use discussions and decision-making.^{Footnote 35} To develop such maps, the SRRB identified, compiled, and analyzed existing traditional knowledge and integrated it with the geoscience knowledge.^{Footnote 36} Workshops, training sessions and open houses were held to familiarize users with the geoscience information and maps that were produced.^{Footnote 37} Members of the Tulit’a community, one of the five Sahtú communities, also actively participated in the project planning and methodology, and in the preparation of the traditional knowledge narratives of the different Sahtú locations represented on the maps.

Despite such positive project results, a majority of interviewees were unclear as to whether and what tools were being developed. A few internal and external interviewees expressed the need to have more tools developed and to increase their visibility/accessibility. Interviewees involved in the AGN noted that a communication plan, including suggestions on tools to facilitate the use of geoscience data, was submitted to GEM-2 by the AGN, but they could not tell if this plan was implemented. A few interviewees also mentioned that, whenever possible, GEM-2 maps should be available in GIS format, which was a standard format that could be more easily accessed and used by the communities and their institutions. A few internal interviewees explained that GEM-2 needed to increase efforts in the planning and development of tools. They suggested that such projects be planned annually and prepared at the same time as the technical geoscience products.

Documentary and interview evidence indicated that there was a short lead time to develop proposals for the Multidisciplinary Grants program, and impacts of funded projects could have been increased if the program’s two calls for proposals had been undertaken earlier in the fiscal year.^{Footnote 38} Proposals for both rounds of projects, in 2015–16 and 2016–17, were due late in their respective calendar years (e.g., December 9, 2016 for the 2016–17 projects). Furthermore, projects had to be completed in the same fiscal year that the Call was issued, i.e., no later than the end of March in both 2016 and 2017.^{Footnote 39}

Participation in projects was reported to be lower than anticipated due to the short notice for submitting proposals, as well as competing demands on the capacity of prospective participants or weather-related constraints on travel to carry out the projects. Some of the internal interviewees suggested that many communities did not submit proposals because they could not meet the short timelines for proposing and completing projects. This evidence was supported by the documentary evidence found in proponent project reports as well as the case study.

Two of the four 2016–17 recipients also noted that the program did not have an assigned contact available to provide assistance when needed. In that regard, the SRRB case study illustrated further how the proponent experienced two different levels of support from the GEM-2 program between 2015–16 and their second project of 2016-17.

In 2015–16, the SRRB project benefited from administrative and technical guidance of a GEM-2 officer during the application and data collection process of the project. This enabled the proponent to complete the project within the grant timelines. The officer answered proponent questions and explained the grant terms, helping to get the project started and provide meaningful reports so as to comply with the grant terms and conditions. The officer also provided relevant information about GEM-2 research conducted in the Sahtú area, including research reports, data sets, lists of publications and the contacts of GEM-2 researchers to the SRRB proponent. However, this type of support was not provided during the second SRRB project of 2016–17.  

A key challenge arising from the lack of support provided in 2016-17, was timely access to information about studies being conducted in the Sahtú area in 2016–17, particularly studies that were not yet available publicly. According to information from the program, the grants were administered at arms-length in order to avoid any perception of conflict of interest. Subsequent clarification of the conflict of interest guidelines with respect to multidisciplinary grant recipients was expected to alleviate this circumstance with respect to the administrative and technical support available to future multidisciplinary grant recipients.

4.1.2 Immediate outcome: Industry and Northerners are aware of and have access to new regional geoscience data and knowledge generated by GEM

Finding #5:     Overall, industry and Northerners were aware of and had access to GEM-2 products to a moderate extent. While GEM-2 publications and open data file were posted online immediately as available in a downloadable format, interactive engagements were noted as being more effective at increasing awareness and access. Presentations to geoscience conferences and community engagement activities were the primary means of actively disseminating and sharing new GEM-2 products, and were of particular importance to industry. Three improvements to knowledge transfer to community and to industry were suggested by interviewees and survey respondents as follows: 1) ensuring GEM-2 lead scientists’ presence during geoscience conferences generally and related to northern geoscience specifically, 2) improving the dialogue with communities and 3) increasing the availability or visibility of tools to facilitate uptake.

All lines of evidence showed that the level of access and awareness varied by audience and level of engagement with the program, as well as method of dissemination. As noted in section 4.1.1 however, there were limits on the effectiveness of the knowledge transfer activities in the form of absence of scientists from geoscience fora, organization of knowledge products on GEOSCAN, and the need to emphasize the “day-to-day” utility of the science in a layperson’s format.

Most interviewees believed that industry was aware of and had access to GEM-2 products primarily through presentations on project activities and emerging results . Documentary and survey evidence also supported this view.

The majority of external interviewees stated that Northerners and their institutions were unfamiliar with GEM-2 products, and were not exposed to them. They cited reasons such as the engagement-related issues (discussed previously): a possible lack of interest in the topic; limited understanding of the relevance of geoscience information to their local or regional interests: Internet access barriers; and the limited availability or visibility of facilitating tools as contributing factors.

The analysis of GEOSCAN data on product downloads provided a proxy measure of accessibility. GEOSCAN data showed that the program generated approximately 500 geoscience products to the end of 2016–17. Two-thirds of these products were accessed and downloaded at least once during 2016–17 (this figure was 42% in 2014–15), and the total volume of downloads increased from 2,370 in 2014–15 to 12,084 in 2016–17.

Information based on users accessing and downloading the GEM-2 products is shown in Table 6. While imprecise as to user, the data for GEM-2 is available for 2015, 2016, and 2017 and show totals for each category. The data suggest that both the number of users (particularly the target audience of northeners) accessing the documents and the number of downloads increased over time among users external to the public sector.

Table 6: GEOSCAN Data of Organizations and Downloads of GEM-2 Products by Category and Year, 2015 to 2017

GESCAN Data for Geo-Mapping for Energy and Minerals (GEM-2) Products
User Category	2015		2016		2017
	# Organizations	# Downloads	# Organizations	# Downloads	# Organizations	# Downloads
Public Sector	17	573	22	1,681	20	304
Private Sector	37	99	48	154	51	429
Academic Sector	48	134	57	277	74	1,862
Generic / General Public	125	864	127	1,495	146	2,415
Other*	20	734	31	1,637	41	7,387

Source: As provided by GEM-2 program, February 5, 2018.
* Includes the GEOSCAN tracking categories: Other Sectors, No Category for Now, and No ID possible. For the purposes of this discussion, they are grouped in the Other category.

Figure 2 shows that the targets for the number of unique GEM-2 products downloaded were exceeded each year. The patterns of growth in the numbers of new products released, numbers accessed and volumes of downloads for each project area were consistent with the program-wide patterns.

Figure 2: GEM Products Downloaded and Released, 2014 to 2017

Note: *19 GEM-2 products were listed on GEOSCAN prior to 2014–15. These documents were published by third-party organizations and thus GEOSCAN was unable to collect data on the download volumes.
Sources: Numbers of product releases and downloads are from data on GEM-2 product releases and downloads provided to Science-Metrix in July 2017; targets for GEM-2 product downloads are from the NRCan Reports on Plans and Priorities for 2014–15, 2015–16 and 2016–17

4.1.3 Immediate outcome: Northerners and their institutions have access to tools that facilitate use of new regional geoscience data and knowledge generated by GEM

Finding #6:     Funding support to community and regional organizations through multi-disciplinary grants was the principal means by which the program sought to provide northerners and their institutions with access to tools that facilitate use of new regional geoscience data and knowledge generated by GEM. While there is some evidence that the multidisciplinary grants were helpful in creating tools to facilitate the use of the GEM knowledge, the extent to which the proponents successfully applied the tools developed could not be determined because grant recipients were not required to report on the deployment of their tools. Consequently, the GEM-2 program did not track their use. Bearing in mind that it was still early to observe meaningful impacts on access to facilitating tools, document review and case study results suggested that this outcome had been achieved to a moderate extent.

As noted earlier, GEM-2 aimed to increase access to tools by providing multidisciplinary grants to community and regional organizations in the North, including organizations with stewardship authority for various lands and regions in the North. The grant recipients would in turn develop the tools that would facilitate use of the geoscience data generated by GEM. Information about the multidisciplinary grants indicated that the projects were successfully completed in spite of challenges. However, limited evidence was found of the extent to which these organizations had successfully implemented these tools for several reasons.

First, grant recipients were not required to report on the deployment of their tools.^{Footnote 40} Therefore, the GEM-2 program could not track their use. In this regard, GEM should apply provisions in the Policy on Transfer Payments and the Treasury Board Directive on Grants and Contributions that permit the collection of performance measurement and results information from grant recipients.^{Footnote 41}

Second, the majority of interviewees reported moderate access to such tools by Northerners and their institutions because few tools had been produced or were visible. Survey evidence corroborated this finding. Among external survey respondents (n=35 responses), 37% (n=13) reported that Northerners and their institutions have access to tools that facilitate use of new regional geoscience data and knowledge to a moderate or large extent. However, 23% (n=8) of them reported that this immediate outcome had been achieved to a small extent or not at all, while an additional 40% (n=14) of them responded “did not know/not applicable.” When combined, the evidence suggested a low level of access to facilitating tools.

Third, documentary and SRRB case study evidence also suggested that tools created during the period under evaluation were conceived to reach a limited targeted audience (i.e., one or a few communities) and would not be widely available. As such, multidisciplinary grant projects alone could not be expected to generate widespread access to facilitating tools beyond the communities they served.

Fourth, because of the challenges described previously (e.g., short timeframes), they were used mainly by those directly involved in the development of the method or tool.^{Footnote 42} In the case of the SRRB projects, for instance, awareness and access to GEM data increased primarily among the participants in workshops and training sessions and among students exposed to classroom curricula that incorporated GEM data—all members of the same Sahtú community. At the time this evaluation was conducted, the SRRB had yet to expand their tool to the other four communities in the region, as intended.^{Footnote 43}

4.1.4 Intermediate outcome: Northerners, their institutions and exploration industry use modern regional geoscience knowledge to inform their decision-making

Long-term outcome: A strong northern economy through stable, long-term investment in responsible resource development

Finding #7:     Opportunities for northerners, their institutions and the exploration industry to use GEM-2 products in decision-making have been limited given that GEM 2 research was on-going and the release of such products only commenced in 2014-15. The long lead times from initial prospecting to the development of new mines meant that it was premature to expect the program to have made extensive contributions to the intended intermediate and long-term outcomes. However, stakeholders were reasonably confident that GEM-2 would generate meaningful impacts on exploration and resource planning/management decision-making and on the northern economy in the longer term.

Documentary evidence indicated that the timelines from prospecting and exploration for new mineral resources to the development and construction of new mines and supporting infrastructure varied from at least 10 years to over 20 years in Canada’s North, with initial prospecting and exploration taking approximately three to five years.^{Footnote 44} The generation of GEM-2 outputs only effectively commenced in 2014–15, and production was planned to continue through to the program’s end in 2020. Therefore, opportunities for the use of GEM -2 geoscience products to inform exploration planning were relatively limited as of 2017. Hence, it was premature to expect extensive evidence of the contribution by the program to the intended intermediate and long-term outcomes. Views expressed by interviewees and the NTGS case study participants confirmed these findings.

The majority of interviewees noted that the use of GEM-2 products by northern communities and institutions was limited due to the fact that research was ongoing with final deliverables to be produced. Other factors impacting the use of GEM products by northerners and their institutions included: of low awareness of the GEM-2 program and products, limited access to GEM-2 data, limited skills in applying geoscience data and the low demand for such products by these stakeholders. The impact of facilitating tools on the use of GEM-2 data was also minimal during the period covered by this evaluation because the multidisciplinary grants program started in 2015–16, and few projects had been funded as of mid-fiscal year 2017–18. In the case of the SRRB case study, interviewees reported that most impacts on decision-making would likely be observed in the longer term, when the technical challenges experienced during the project were solved and their tool was extended to cover all Sahtú communities.

Nevertheless, documentary, interview and case study evidence identified early examples of use of GEM-2 data by industry and provincial, territorial and local governments. Three research projects carried out in collaboration with the NTGS resulted in interest in or use of the projects’ results by exploration companies and an organization serving the exploration industry. In the context of the South Rae project, several exploration companies, including Areva and Rio Tinto, requested access to the results before they were released and expressed a clear interest in following up. Following the completion of the Banks Island project, the NTGS had informal discussions with the Inuvialuit Development Corporation, which expressed interest in the Banks Island work being pursued to assess the potential for Kimberlite Indicator Minerals on the island. One mineral exploration company also expressed strong interest in a geophysical survey conducted in the western part of the Brock Inlier map area and asked to access the data before they were published in an open file.

Some interviewees indicated that provincial, territorial and local governments used GEM-2 information to complete or refine their own maps for land-use planning purposes, including future exploration and infrastructure development. The Northwest Territories Department of Transport used GEM-2 “published Surficial Geology maps/data … to assess all-season road alternatives to reduce the reliance on winter roads,” and the municipalities of Yellowknife and Dettah both used GEM-2 products to “map permafrost and terrain hazards related to community infrastructure and development.”

^{Footnote 45}

Almost all of the internal and external interviewees were confident that GEM-2 would generate meaningful economic impacts in the medium to longer term. Some thought that this might require extending the program beyond 2020. Among the survey respondents overall, 56% (n=35) believed that the GEM-2 program would make a large or very large contribution to the use of geoscience knowledge by Northerners, their institutions and the exploration industry to inform their decision-making. The breakdown among sub-groups mirrored the overall finding, with 50% of non-industry respondents (n=12), 55% of industry respondents (n=6), and 61% of internal respondents (n=17) believing the GEM-2 program would make a large or very large contribution. 

Of the total survey respondents, 40% (n=25) expected the longer-term contribution of the GEM program to a stronger northern economy to be large or very large. Among the sub-groups, industry respondents were more likely to expect the contribution to be large or very large at 45.5%, whereas 42% of non-industry and 36% of internal respondents held that view.

4.2 Factors impacting the achievement of intended outcomes

Finding #8:  Three key factors were identified as facilitating progress on the program’s intended outcomes. They are: 1) engagement with local communities prior to data collection to obtain the social licence to operate; 2) geoscience collaborations with provincial/territorial geological surveys, academic researchers, and exploration companies, and 3) logistical and administrative support from the GEM Coordination Office and the Polar Continental Shelf Program.   On the other hand, impediments to achieving intended outcomes can be grouped in four key areas: 1) technical barriers in electronic access to information; 2) lack of a clearly articulated dissemination and knowledge transfer strategy; 3) capacity challenges; and 4) exploration for minerals and energy was at a low point due to depressed demand for energy and minerals and uncertainty over opportunities for energy exploration in the North. As a result, the uptake of GEM geoscience information was somewhat diminished, as least in the near term.

Facilitating Factors

Documentary, interview and case study evidence pointed to three key factors that contributed to achieving GEM-2 projects’ objectives. They are as follows:

1. Well-established and effective relationships and working arrangements with P/T geological surveys that facilitated collaborative work on project activities.

   GEM-2 project managers indicated in their mid-year and year-end reports that collaborations with P/T geological surveys and academic researchers enabled the program to benefit from the complementary scientific skills of collaborators and to address gaps in the program’s own expertise in some project areas.^{Footnote 46} In particular, collaboration between the Quebec Geological Survey and GSC was viewed by external key informants as complementary and generally effective. Other external interviewees mentioned that the collaboration between GSC, the Yukon Geological Survey and the German Federal Institute for Geosciences and Natural Resources in the Arctic was very effective and complementary: “all those people bring different perspectives, focus on different scales and there is no competition.^{Footnote 47}

   NTGS case study participants also mentioned that the NTGS brought a first-hand and in-depth understanding of the local geology, while the GSC contributed a regional and cross-border perspective. NTGS participants involved in South Rae, Banks Island and Brock Inlier projects emphasized the positive influence a good relationship with NTGS and their active involvement in project planning and execution had on the success of the project. For example, the involvement of the NTGS facilitated contacts with the local communities.  As well, collaborative arrangements enabled the deployment of skilled personnel that helped to mitigate skilled capacity gaps within projects.

2. Effective engagement with local communities to obtain consent for work and to identify and respond to concerns regarding proposed data collection activities (i.e., social licence to operate).

   Interviewees and case study participants stressed the importance of frequent and adequate engagement activities with local communities to identify and address their needs. The South Rae project illustrated this type of engagement. In this project, the GSC and NTGS researchers visited the communities closest to their research field several times to present the project and its benefits for the communities and discuss collaborative opportunities (e.g., hiring of local workers).

   According to case study participants, these engagement activities exceeded expectations as previously members of the same communities were reluctant to communicate with federal government representatives. In response to the interest and support for the GEM-2 project communications efforts from one of the communities, the project team arranged to fly community members to the camp to observe the field work, and subsequently sent them project-related reports. The community then invited the GEM-2 team to present the project at their annual general meeting.

   In the context of the Brock Inlier project, a magnetotelluric survey that required helicopter flights was planned for summer 2014. However, according to mid-year and year-end project reports, this survey was postponed “due to perceived disturbance to caribou in the proposed transect area.” Additional engagement activities were carried out and the Inuvialuit Land Administration agreed to provide access to the area by helicopter for the 2015 season. According to the data from internal interviews, because this challenge and the possibility of conducting a two-year rather than a one-year project had been anticipated at project onset, it did not meaningfully affect its overall timeline or scope.

3. Logistical and administrative support from the GEM Coordination Office and the Polar Continental Shelf Program.

   All project field season, mid-year, and year-end reports cited the importance of administrative and logistical support in enabling data collection to be carried out. The GCO role in arranging travel and accommodations, as well as in handling issues related to travel expenses, was a factor that enabled the scientists and field work personnel to focus on data collection. The logistical support provided by the Polar Continental Shelf Program (PCSP) was also mentioned in most of the reports as a factor facilitating achievement of objectives. Arrangements for on-site logistical support and services for activities such as wildlife monitoring, boat operation, fuel and equipment handling enabled the scientists to maximise short field seasons. In one project for example, the arrangement for a turn-key camp operation provided an extra seven days for data collection.

In the annual mid-year and year end project reports, project managers also identified other supporting factors that facilitated the conduct of the GEM-2 projects. They included the quality of support from transport providers and opportunities to showcase GEM-2 results through participation in industry, science and other events which had high levels of interest from attendees.

Hindering Factors

Achievement of program objectives was impeded by the following factors:

1. Technical barriers
   • Low bandwidth
     Four of the nine multidisciplinary grant project reports noted that technical barriers created inefficiencies in accessing GEM information. Within GEM, the open data was placed on GEOSCAN, but northerners’ access to it was limited by the low band-width and inefficiencies in downloading the information. For example, one of the projects reported that it could not show live simulations of the data that was available, reducing student opportunities to gain familiarity and understanding of the online placement. Low bandwidth and charges for exceeding usage would also make it challenging to communicate by email and by videoconferencing, also limiting access.
   
   
   • Inconsistent data formats
     In addition to the low bandwidth challenges, another impediment was the inconsistent formats of the datasets produced by GEM. This created challenges for integrating GEM data the tools that were being developed in two of the multidisciplinary projects. One of the proponents suggested that all data be in GIS format.


2. Lack of a clearly articulated dissemination and knowledge transfer strategy;
   • Poor organization of information on GEOSCAN
     GEOSCAN is the online repository for all GEM knowledge products. However, as noted earlier, the database was described as being poorly organized and its search function limited.
   • Absence of researchers from northern mining fora and geological conferences
     Industry respondents in the survey identified Geoscience Fora and geological conference as the key means of acquiring the geoscience information related to the north, and particularly appreciated the face-to-face interaction. Both internal and external sources identified a mixture of travel restrictions (the application of “event” limits on number of NRCan attendees) and travel caps (expenditures exceeding planned amounts for a sector) as contributing to the absence of research scientists at such events. The issue was specifically noted by external informants in the case study evidence. In that case, GSC scientists were absent from the presentation on their research at the Yellowknife Geoscience Forum.
   • Research results insufficiently translated to community interests
     Some internal and external interviewees felt that there was a disconnect between GEM-2 and the communities. The nature of the projects and geoscience information produced was not always appropriately communicated to and easily understood by the communities, which limited the communities’ uptake of information.


3. Capacity Challenges
   • Capacity challenges with respect to the development of tools that use GEM geoscience

     Within the program, the main capacity gap was the support available to the multi-disciplinary grants proponents, and the limitations of the multidisciplinary grants program as discussed in section 4.1.1 and 4.1.3 earlier. One interviewee suggested that a person with community engagement experience in the north would positively address the capacity issue. In addition, some interviewees and case study participants explained that the high staff turnover and staff shortfall, including project activity leaders, impeded GEM-2 production and support to stakeholders.

   • Science resource gaps
   • Documentary evidence also pointed to difficulties in addressing scientific resource gaps in certain fields due to retirements and workload demands on staff with specialized skills (e.g., palynology in the Cordillera, conodont biostratigraphy in the Hudson-Ungava), although the strength of the collaborations mitigated the overall impact on the project timelines.
     Permitting Issues
     Project managers reported that the complexity and timeframes associated with obtaining permits for fieldwork activities impeded the conduct of projects. An interviewee suggested that project officers dedicated to obtaining permits and licences be located in the regional offices rather than in Ottawa. The interviewee believed that it would be more effective, in that it would ensure permitting was not performed by project managers, which was time-consuming and not part of their responsibilities.
4. Low point in the exploration cycle
   • Depressed world prices for mineral commodities limited the level of mineral and energy exploration that would otherwise rely on GEM geoscience products.^{Footnote 48} A few interviewees, including a key informant representing several exploration companies, mentioned that the mining industry experienced a very strong downturn, resulting from a decrease in the demand for mineral commodities. According to the informant, market capitalization in junior companies was reduced by 90%, resulting in their inability to raise money to do prospecting work, and therefore reducing the demand for GEM-2 geoscience. Data published by NRCan on trends in exploration and development spending and outlooks for future spending confirmed the significance of the downturn, albeit with some expectation that spending would increase in 2017.^{Footnote 49}
   • Another external factor impacting the program was that the energy industry investments had stalled following the December 2016 moratorium on Arctic oil and gas exploration. The joint announcement by the United States and Canada made Arctic Canadian waters indefinitely off limits to future offshore Arctic oil and gas licensing, subject to a five-yearly “climate and marine science-based life-cycle assessment.^{Footnote 50} In addition, the moratorium on fracking in the Yukon also decreased demand for new geoscience products in the area.^{Footnote 51}

Other challenges included weather and transport availability issues during fieldwork in some project areas, higher costs of shipping than budgeted/expected, and problems with proper payments to students and temporary help due to the Phoenix pay system.^{Footnote 52}

4.3 Unintended outcomes

Finding #9:     The evaluation found no evidence of any major positive or negative unintended outcomes.

4.4 Demonstration of efficiency and economy

4.4.1 Program design

Finding #10:   The program design provided an appropriate foundation for conducting geoscience research outputs and outcomes. However, as designed and implemented, the program provided insufficient support to facilitating use and access to GEM-2 knowledge, particularly for northerners and their institutions. Some case study participants and interviewees suggested that the short timelines and the lack of program assistance for multidisciplinary projects limited the effectiveness of the funded projects. Evidence also suggested that program planning and delivery could leverage AGN’s strategic advice more effectively, and ongoing consultation with external stakeholders to inform project planning, design and coordination could be strengthened. The program’s performance measurement strategy did not clearly depict the intended program outcomes.

As noted previously, evidence from the GEM-2 project managers’ year-end reports, interviews and case studies indicated that research project activities were generally undertaken as planned and that the generation of program outputs was largely on track. Additionally, documentary evidence indicated that .  The program’s approach to data compilation and analysis also built on well-established working arrangements with P/T geological surveys enabling the program to obtain additional leverage on its resources and funds. These findings suggest that the program design provided an appropriate foundation for achieving the geoscience objectives as presented in the program rationale.

However, interviewees and case study participants identified possible opportunities to improve the GEM-2 program design, as follows:

1. Consultation with external stakeholders: While the process for evaluating research activities was standard, survey, interview and case study evidence suggested that earlier and sustained collaborative program and project planning and fieldwork coordination with GEM-2 stakeholders (i.e., industry, provinces/territories, communities) would make it possible to identify and deliver projects that generate the most relevant outcomes per input for all stakeholders.


2. Improvements to the Multidisciplinary Grants Program - Timeliness of the call for proposals and program assistance for multidisciplinary grants: The multidisciplinary grants program was welcomed by northerners and their institutions who viewed it as a positive initiative overall. However, interviewees and grant proponents recommended that GEM-2 issue the call for proposals for multidisciplinary grants earlier in the fiscal year and provide more lead time for grantees to submit their application (the decision to increase the length of 2017–18 projects to two years was a step in the right direction). The planning and delivery of multidisciplinary grant projects would also benefit from personalized and sustained assistance from program staff.


3. Use of AGN’s advice: The Advisory Group of Northerners (AGN) was established to provide advice to GEM-2 management to maximize the benefits for the Northerners resulting from the GEM activities and knowledge. This consultative body is chaired by the LMS ADM, with the membership composed of representatives from territorial governments, the private sector and Indigenous socio-economic development organizations. In particular, the AGN identified the following five themes as the basis for its activities and provision of advice:
   1. community engagement and stakeholder identification
   2. capacity and skills development
   3. integration of knowledge and data into other decision-making processes
   4. messaging and communication with northerners
   5. environmental impact^{Footnote 53}

However, evidence provided mainly thorough the external interviewees indicated that the full value of the AGN was not being realised. External interviewees suggested that GEM-2 consider revisiting AGN’s mandate so its advice is better integrated in areas such as program priority setting, the selection of projects, community engagement in relation to knowledge dissemination, and how capacity could be developed in the territories. Interviewees also felt that the program could have greater transparency in how advice provided during AGN meetings was translated into decisions or actions.

4.4.2 Measures taken to improve the efficiency of program delivery

Finding #11:   Lessons learned during GEM-1 have been used to improve program administration and support. Major actions taken included establishing the GEM Coordination Office, standardizing the approach to project-related community engagement, and developing a project management database.

Documentary evidence indicated that lessons learned during GEM-1 were applied to the design of GEM-2 to improve both the effectiveness and efficiency of key activities. Little evidence was found of the impact of these measures on the costs and efficiency of GEM-2 operations during the evaluation period.

Of particular note was the creation of the GEM Coordination Office (GCO) to manage administrative activities that were common to, or supported, all project areas. The mandate of the GCO was to facilitate the implementation of GEM program activities and enable GEM staff involved in project activities to focus on operations. The GCO was responsible for coordinating the logistics for fieldwork; providing a consistent basis for, and support to, planning and budgeting processes; standardizing tracking and reporting; and providing a consistent framework and support processes for community and stakeholder engagement.^{Footnote 54} Internal interviewees stated that the GEM-2 coordination structure contributed to reduced operational costs through increased administrative support, monitoring and cross-project coordination. 

Progress in obtaining the social license to operate was notable in GEM 2 due to steps taken to improve engagement activities, as illustrated in the following examples:

• In the wake of legal challenges to planned seismic work in Lancaster Sound (under GEM-1), changes were made to community engagement processes to improve awareness of and support to GEM activities, better train program staff to engage with community representatives and increase the program’s internal capacity to undertake engagement activities. Engagement was recognized as an integral element in the planning and conduct of project activities.^{Footnote 55}
• According to program documentation, advice from the GEM-1 AGN informed the development and provision of awareness training to project leaders, ensuring that the content and language of materials were relevant to the communities with which the program interacted; maps and other resources were provided to communities to facilitate their understanding of planned activities and utilization of GEM products; and close-out engagement was conducted, including explaining the geoscience information produced.^{Footnote 56}
• Internal interview data indicated that at the time the evaluation was conducted, the GCO was leading the development and deployment of a project database designed to consolidate and track the progress of GEM-2 project activities and products and the assignment of human resources to project activities.

4.4.3 Opportunities to achieve more cost-effective program delivery

Finding #12:   Further improvements to administration, community engagement, support and coordination across projects, as well as increased collaboration across projects were the most commonly suggested means of achieving more cost-effective program delivery.

A few interviewees and case study participants suggested a number of possible improvements to GEM-2 activities that could result in efficiency and cost-effectiveness gains. The three most common  suggestions are listed below.

• Inter- and intra-project collaborations: Survey and case study evidence indicated that increased collaborations between multidisciplinary grants and GEM-2 research projects could contribute to more efficient and cost-effective project delivery through sharing of information and resources.  Survey data indicated researchers had an interest in improving intra-project collaboration, including more face-to-face meeting opportunities for project participants, e.g., workshops, field trips, mini-symposia. Sahtú case study participants suggested that GEM-2 support the creation of a network of all GEM-2 multidisciplinary grant projects to enable project participants to learn from each other’s experiences (i.e., what worked well, what did not).
• Increased program planning and fieldwork coordination with external GEM-2 stakeholders: Such collaboration and cooperation could generate efficiencies in the field through the sharing of resources and use of common tools (e.g., dissemination platforms and activities). This would require sustained communication and consultation over the life of the projects.
• Streamlining and further developing the coordination structure: All lines of evidence indicated that the GCO structure enabled more efficient program delivery of phase 2 of GEM. However, room for further improvement was noted in documentary evidence and by some survey respondents and interviewees with respect to timeliness and communication of project decisions, both internally and with external project collaborators and partners.  Interview and NTGS case study evidence suggested that effectiveness and efficiency would be improved if the GSC scientists received more support from GEM-2 officers for administrative (including permitting) and engagement activities. They would then be able to focus on the scientific work and the production of geoscience deliverables, which is their area of expertise.

4.4.4. Alignment between planned and actual spending

Finding #13:   Annual program expenditures have been closely aligned with budgets each year since the start of GEM-2. Some project areas have had significant variances in some years as a result of changes to project activities.

Table 7 shows the breakdown of approved funding for the GEM-2 program, by Vote and year, covering the program’s seven-year time frame. These data indicated that GEM-2 activities were anticipated to progressively ramp up from $6.43 million in 2013-14 to a peak of $17.86 million in 2016–17 and 2017–18 and tapering to $12.14 million in 2019–20. This pattern of planned spending was implemented to allow a phased approach to data collection and analysis activities in each project area, and accommodate differing rates of data synthesis and publication.

Table 7 Approved GEM funding, 2013–14 to 2019–20

	2013–14	2014–15	2015–16	2016–17	2017–18	2018–19	2019–20	TOTAL
Vote 1
Personnel	$1,072,400	$1,534,100	$1,557,100	$1,580,500	$1,604,200	$1,628,200	$1,577,400	$10,553,900
O&M	$5,001,720	$10,216,240	$14,474,280	$15,160,500	$15,132,060	$12,960,440	$9,810,080	$82,755,320
EBP (@20%)	$214,480	$306,820	$311,420	$316,100	$320,840	$325,640	$315,480	$2,110,780
Sub-Total	$6,288,600	$12,057,160	$16,342,800	$17,057,100	$17,057,100	$14,914,280	$11,702,960	$95,420,000
Vote 10
Gs&Cs	$140,000	$800,000	$800,000	$800,000	$800,000	$800,000	$440,000	$4,580,000
Total New Funding	$6,428,600	$12,857,160	$17,142,800	$17,857,100	$17,857,100	$15,714,280	$12,142,960	$100,000,000

Note:                  O&M amounts include provisions for $6.03 million in minor capital.

Table 8 below shows the breakdown of budgeted and actual GEM expenditures for the period examined by the evaluation. These data show that actual program expenditures have been closely aligned with annual budgeted amounts since the launch of GEM-2. The variance ranged between 2.7% and 3.4% in each of 2014–15, 2015–16 and 2016–17. Variances were more significant in some years for individual project areas as a result of reallocations between project areas to accommodate planned and unplanned changes in activities, including delays in activities due to factors such as equipment availability (for example, a planned aeromagnetic survey did not go ahead in 2016–17 due to late approval).^{Footnote 57}

Table 8 Budgeted and actual GEM-2 expenditures, 2013–14 to 2016–17

	2013–14		2014–15		2015–16		2016–17
	Budget	Actual	Budget	Actual	Budget	Actual	Budget	Actual
O&M
GCO	$3,336,706	$2,177,423	$1,376,954	$1,265,794	$1,516,600	$1,484,809	$2,459,258	$1,515,833
Baffin			$1,220,747	$1,166,186	$1,176,538	$1,161,877	$284,032	$263,957
Hudson-Ungava			$773,574	$787,741	$1,525,065	$1,504,612	$1,126,146	$1,115,878
Rae		$1,154,187	$1,806,945	$1,793,909	$2,197,663	$2,155,877	$2,982,417	$3,231,972
Western Arctic			$582,900	$526,967	$1,444,621	$1,423,588	$1,203,784	$1,102,570
Mackenzie			$611,913	$630,011	$1,081,092	$1,040,900	$829,866	$891,700
Cordillera			$456,239	$380,364	$926,040	$915,472	$538,461	$881,673
Sub-Total	$3 336 706	$3 331 610	$6 829 272	$6 550 972	$9 867 619	$9 687 136	$9 423 964	$9 003 583
Personnel	$1,302,940	$1,306,294	$1,445,264	$1,451,596	$1,994,842	$1,783,910	$3,149,344	$3,109,000
Capital	$0	$0	$1,072,837	$1,072,232	$1,028,128	$1,010,354	$824,950	$807,234
Grants & Contributions	$129,350	$86,400	$800,000	$799,993	$800,000	$789,652	$800,000	$800,000
TOTAL	$4,768,995	$4,724,304	$10,147,373	$9,874,793	$13,690,589	$13,271,051	$14,198,258	$13,719,817
Variance		0.94%		2.69%		3.06%		3.37%

Note:                   Data excludes 20% EBP provision for Vote 1 Personnel amounts.
Source:                GEM2 year-end reports; GEM2 program dashboards

5 Conclusions and recommendations

Relevance

Based on the evidence, GEM-2 is generating basic geoscience research that would be unavailable otherwise. In a context where mineral and energy exploration and production remain key drivers of Canada’s northern economy, the products generated help identify areas of resource potential and help industry identify targets for exploration.  Increasingly, they are used by public agencies and Indigenous stewardship organizations for land-use planning and management. The evidence also points to a growing need for geoscience information that can be used by territorial and Indigenous authorities to address climate change impacts and inform decision-making regarding environmental planning and protection, and infrastructure planning. Key informants and survey respondents anticipated that significant gaps would remain beyond 2020.

The impact of changing technologies and methods for collecting and interpreting geoscience data, and the varying capacity of the northern regions to deliver full geoscience research programs also support the need for the program. There is evidence to suggest that emerging needs (e.g., related to climate change, environmental protection) may require different products (e.g., smaller-scale mapping) than are currently being produced.

The GEM-2 program is consistent with federal government priorities regarding support for economic and social development of the North, as set out in the 2014 Budget Plan. It is a legitimate and appropriate role for the federal government to deliver the GEM-2 program, and the role is clearly delineated in the Intergovernmental Geoscience Accord (IGA). In all lines of evidence, program stakeholders confirmed that the intervention of the federal government is needed to provide the provinces/territories, academia and industry with essential and complementary scientific skills and resources. In addition, the role is consistent with the role of NRCan as per the authority provided to the Minister in the Department of Natural Resources Act and the Resources and Technical Surveys Act.

Performance – Effectiveness

Overall, the program made moderate progress toward the achievement of its immediate, intermediate and long-term outcomes. Evidence from all sources indicated that, at the mid-point of its program cycle, GEM was on track to add significantly to the geoscience knowledge base of northern Canada.

Knowledge transfer activities mainly took the form of including GEM information on GEOSCAN, the online NRCan repository, and presentations by GEM-2 researchers at geoscience conferences and community engagement activities. GEOSCAN downloads by external sources increased each year, suggesting an increase in awareness over time. However, interview evidence suggested that the GEOSCAN site was difficult to search and information was not well-organized, thereby limiting accessibility of the information. In addition, the low bandwidth in the north made it challenging for northerners and their institutions to access the geoscience information stored online and acquire data for community-use purposes. Interview and survey evidence indicated that industry particularly valued the presence of GEM-2 researchers at conferences and fora because they provided direct and preliminary access to GEM-2 research results. Documentary evidence as well as some interviewees and case study participants indicated that limitations on scientists’ travel to these events reduced the effectiveness of GEM-2 presentations at these fora.

Applying learnings from GEM-1, the program expanded its community engagement activities in recognition of the need to engage northerners and their institutions to achieve GEM-2 objectives. All lines of evidence indicated that community engagement efforts were particularly effective in obtaining the social licence to operate and ensuring ongoing support and participation for the projects. Interviewees and case study informants noted that these efforts were undertaken in conditions where there was limited receptivity and/or understanding of GEM-2 research work by local communities. In other areas related to community engagement, there was a marked difference in perception about the effectiveness. Sources internal to the program viewed them as largely effective. In contrast, external Interviewees and survey respondents pointed to a need for improved engagement activities aimed at increasing awareness of GEM-2 outputs and at developing tools and facilitating their use by northerners and their institutions in the longer term.

All evidence pointed to the value of the multidisciplinary grants, and the utility of the projects themselves. Documentary evidence suggested that projects funded through the multidisciplinary grants achieved their intended project-level objectives. For example, the SRRB project successfully integrated GEM-2 and traditional Indigenous knowledge into their Atlas of the Sahtú region. However, limited data were available to assess the impact of these projects in terms of GEM-2 outcomes because the grant agreements did not ask recipients to report on the deployment of their facilitating tool.

Given that the lead time from initial prospecting through to the construction of new mines is 10 to 20 years in Canada’s North, and that GEM-2 products only began to be released in 2014–15, it was too early in the program lifecycle to observe any meaningful impacts of the program on GEM-2 products’ use and on the northern economy overall. However, early signs of the potential impact exist. For example, several exploration companies expressed interest in research data from the South Rae project and expressed a clear interest in following up. Also, some interviewees indicated that provincial, territorial and local governments used GEM-2 information to complete or refine their own maps for land use planning purposes, including future exploration and infrastructure development. In the long-term, internal and external interviewees, as well as survey respondents, were reasonably confident that GEM-2 would generate meaningful economic impacts. In fact, 40% (n=25) of the survey respondents expected the longer-term contribution of the GEM program to a stronger northern economy to be large or very large

Performance – Economy and Efficiency

The program design provided an appropriate foundation for conducting geoscience research and reaching knowledge production outcomes. However, as designed and implemented, the program provided insufficient support to facilitating use and access to GEM-2 knowledge, particularly for northerners and their institutions. Some case study participants and interviewees suggested that the short timelines and the lack of program assistance for multidisciplinary projects limited the effectiveness of the funded projects. Evidence also suggested that program planning and delivery could leverage AGN’s strategic advice more effectively, and ongoing consultation with external stakeholders to inform project planning, design and coordination could be strengthened. Challenges in assessing the program design arose because the program’s performance measurement strategy omitted outcomes for the core geoscience research business of the program.

Lessons learned during GEM-1 were used to improve program administration and support. Major actions taken included establishing the GEM Coordination Office and standardizing the approach to project-related community engagement. In addition, the GEM-2 program was developing a project management database, which, as conceived, could provide opportunities for cross-project sharing of information and improved sharing of resources. Further improvements to administration, particularly in obtaining permits for fieldwork, community engagement, support and coordination across projects, as well as increased collaboration across projects were the most commonly suggested means of achieving more cost-effective program delivery.

Recommendations

1. In light of identified and emerging needs for geoscience data and knowledge beyond 2020, it is recommended that NRCan conduct a needs assessment with a full range of external stakeholders to inform future programming in northern Canada.


2. To build on the success of its community engagement efforts in obtaining the social license to operate in northern Canada, it is recommended that the GEM program conduct a review of its northern engagement strategy.


3. Given the challenges with respect to accessing GEM knowledge, the development, distribution and use of tools, and the role of research scientists in the dissemination of research and knowledge transfer, it is recommended that the GEM program develop and implement a strategic plan to enhance knowledge transfer/uptake by industry and the community.


4. Given the challenges to operating in the north it is recommended that NRCan identify a plan to address the barriers (i.e. travel, low bandwidth) to successful program delivery in northern communities


5. Given the need for clarity in identifying and representing the intended outcomes of geoscience research programs, it is recommended that subsequent programs develop a performance measurement strategy that reflects the program design and rationale more accurately.

Appendix A: Program logic model

Appendix B: Evaluation matrix

Evaluation Issues	Evaluation Sub-Issues	Indicators	Lines of Enquiry	Data Sources
Relevance
1.  Is there an ongoing need for the GEM-2 program?	1.1  Who are the principal stakeholders and what are their needs? 1.2  Are there new or emerging needs?       1.3  What factors are driving demand?	Principal stakeholder groups   Current needs for geoscience data, maps, tools and knowledge (“geoscience information”) among stakeholders Gaps in program coverage of needs New or emerging needs   Factors driving demand	Document review	Policy and program documents presenting the rationale for GEM-2, identifying principal stakeholders and evidence of their needs
		Key informant interviews	GEM program management Stakeholders: -  Indigenous organizations with land use oversight roles-       Community organizations in the north P/T geological surveys - Academic researchers -  Industry
		Survey of program staff and stakeholders	GEM program management Stakeholders: Indigenous organizations with land use oversight roles Community organizations in the north P/T geological surveys P/T government ministries with northern land use responsibilities Academic researchers Industry
		Case studies (topics to be decided with SED and EAC)	Depending on case study topics: Engagement planning and conduct document/file reviews Project/activity-specific documents/files Interviews with program and stakeholder participants
2.  Is the GEM-2 program consistent with government priorities and NRCan Strategic Objectives?	2.1 To what degree are the GEM-2 intended outcomes aligned with federal government priorities? 2.2 To what degree are the GEM-2 intended outcomes aligned with NRCan’s strategic objectives?	Alignment/consistency of GEM-2 activities, outputs and outcomes with: Federal priorities Applicable NRCan Strategic Outcome(s)	Document review	Policy documents – Speeches from the Throne, Budget Plans, policy statements, Ministers of Mines and Energy statements/papers, PAA, DPRs, RPPs/Departmental Plan
3.  Is there a legitimate, appropriate and necessary role for the federal government in the Sub- Program? Is NRCan’s role appropriate in the context of the role of others?	3.1 Is the generation of geoscience data and knowledge in the North a legitimate and appropriate role for the federal government? 3.2 Are there other organizations that produce geoscience products similar to those produced by the GEM program? If so, is NRCan’s role appropriate in the context of the role of others?	Program mandate aligned with federal government jurisdiction   Degree of overlap or complementarity of roles of other federal departments, other levels of government and/or industry Alignment of GEM-2 activities with NRCan’s core capabilities and functions	Document review	Policy/program documents – Inter-governmental Geoscience Accord (IAC), Natural Resources Act and the Resources and Technical Surveys Act, program descriptions – other similar/ equivalent programs
			Key informant interviews	GEM program management Stakeholders: Indigenous organizations with land use oversight roles Community organizations in the north P/T geological surveys P/T government ministries with northern land use responsibilities Academic researchers Industry
Performance - Effectiveness, Efficiency and Economy
4.  To what extent have intended outcomes been achieved as a result of the GEM-2 program?	4.1 To what extent have the program’s principal outputs been produced or undertaken as planned?	Collaboration agreements with academic researchers and P/T geological survey partners (TSAs and grants) Production and release of northern geoscience products versus plan Knowledge transfer activities to promote and disseminate northern geoscience products versus plan Community-level stakeholder engagement activities versus plan Tools and methods to facilitate use of GEM geoscience products by northerners versus plan	Document review	Program documents and data relating to planned and actual production of geoscience products, rates of product downloads, peer reviewed publications and citation rates, and planned and actual community engagement activities.
		Key informant interviews	GEM program management
		Survey of program staff and stakeholders	GEM program management
		Case studies (topics to be decided with SED and EAC)	Documents/files relating to production and dissemination of selected GEM-2 products and tools, and delivery of selected engagement activities Interviews with applicable program and/or stakeholder participants
	4.2 To what extent is the program achieving intended outcomes:
	Immediate outcomes: - Industry and Northerners are aware of and have access to new regional geoscience data and knowledge generated by GEM - Northerners and their institutions have access to tools that facilitate use of new regional geoscience data and knowledge generated by GEM Intermediate outcome: Northerners, their institutions and exploration industry use modern regional geoscience knowledge to inform their decision-making   Long-term outcome: A strong northern economy through stable, long term investment in responsible resource development	Rates of awareness of GEM-2 data and knowledge products among primary beneficiaries Satisfaction with GEM-2 data and knowledge products   Evidence of usage of new tools for accessing regional geoscience data Satisfaction levels with new tools   Estimates/evidence of usage and/or anticipated future usage of GEM products by stakeholders Satisfaction levels with usefulness of GEM-2 data and knowledge products     Qualitative assessments or examples or examples of economic and/or social benefits of future resource developments in the North	Document review	Program documents relating to the results of awareness building, product uptake, provision on training and support for usage of tools, application of GEM-2 products and tools by beneficiaries
		Key informant interviews	GEM program management Stakeholders: -  Indigenous organizations with land use oversight roles -  Community organizations in the north -  P/T geological surveys -  Academic researchers -  Industry associations/ companies
		Survey of program staff and stakeholders	GEM program management Stakeholders: Indigenous organizations with land use oversight roles Community organizations in the north P/T geological surveys P/T government ministries with northern land use responsibilities Academic researchers Industry
		Case studies (topics to be decided with SED and EAC)	Documents/files relating to uptake and use of GEM-2 products and tools Data on download volumes, participation in training and product use/ promotion events Interviews with applicable program and/or stakeholder participants
5.  What factors – internal and/or external to the program – have enabled or limited the achievement of the intended outcomes?	5.1 What factors have enabled or limited the achievement of the a) intended program outputs b) intended GEM-2 outcomes?	Impacts on performance of the enabling and/or limiting factors	Document review	Program documents – program/ project/activity reports, mid-year and year-end review reports
		Key informant interviews	GEM program management Stakeholders: -  Indigenous organizations with land use oversight roles -  Community organizations in the north -  P/T geological surveys -  Academic researchers -  Industry
		Survey of program staff and stakeholders	GEM program management Stakeholders: Indigenous organizations with land use oversight roles Community organizations in the north P/T geological surveys P/T government ministries with northern land use responsibilities Academic researchers Industry
		Case studies (topics to be decided with SED and EAC)	Applicable project/activity documents and performance data Interviews with applicable program and/or stakeholder participants
6.  Have there been any unintended (positive or negative) outcomes?	6.1 Have there been unintended outcomes that can reasonably be attributed to GEM-2 activities? 6.2 What actions have been taken in response to any unintended outcomes?	Unintended outcomes partially or fully attributable to program activities   Actions taken to address unintended outcomes (e.g., integrate positive outcomes into the program’s design or mitigate impact of unintended negative outcomes)	Document review	Program documents – program/ project/activity reports, mid-year and year-end review reports
		Key informant interviews	GEM program management Stakeholders: -  Indigenous organizations with land use oversight roles -  Community organizations in the north -  P/T geological surveys -  Academic researchers -  Industry
		Survey of program staff and stakeholders	GEM program management Stakeholders: Indigenous organizations with land use oversight roles Community organizations in the north P/T geological surveys P/T government ministries with northern land use responsibilities Academic researchers Industry
		Case studies (topics to be decided with SED and EAC)	Applicable project/activity documents and performance data Interviews with applicable program and/or stakeholder participants
7.  Is the GEM-2 program the most economic and efficient means of making progress toward the intended outcomes?	7.1 Is the program’s design appropriate for achieving the intended outcomes? 7.2 What measures have been taken to improve the efficiency of program delivery in recent years? How significant were the efficiency benefits or cost savings? 7.3 Are there opportunities to achieve more cost-effective delivery of program activities and production of outputs? 7.4 Has actual spending been in line with planned spending?	Program design issues encountered in program delivery Levels of satisfaction with the way the program was designed and is being delivered Efficiency and/or cost saving initiatives implemented and their impacts   Potential opportunities to improve program efficiency and/or economy   Comparison of actual versus planned spending Explanations for variances and/or budget adjustments	Document review	Program documents – program/ project/activity reports, mid-year and year-end review reports
			Key informant interviews	GEM program management Stakeholders: -  Indigenous organizations with land use oversight roles -  Community organizations in the north -  P/T geological surveys -  Academic researchers -  Industry
			Survey of program staff and stakeholders	GEM program management Stakeholders: Indigenous organizations with land use oversight roles Community organizations in the north P/T geological surveys P/T government ministries with northern land use responsibilities Academic researchers Industry

Appendix C: Case study approach

Case study of the collaboration between the Geological Survey of Canada (GSC) and the Northwest Territories Geological Survey (NTGS)

Background

Provincial and territorial (P/T) geological survey organizations and university geoscience researchers collaborate with the program on the planning and performance of geoscience research activities, and the production and dissemination of the resulting geoscience products. The scope and activities of the respective parties are defined in Task Share Agreements (TSAs), with each party funding and staffing their respective activities. Geoscience products generated may be published jointly, published in scientific journals or made available for free public dissemination via publication websites. The resulting research findings – new geoscience knowledge and data – are shared between the participating organizations and jointly published. Joint results of collaborative work with P/T geological surveys are presented at P/T “geological open houses”, as per commitments made in the IGA as well as other outreach events, such as industry and university-sponsored conferences and workshops.  

The Northwest Territories Geological Survey (NTGS) is a division of the Department of Industry, Tourism and Investment, Government of Northwest Territories. The mission of the NTGS is to:

(P)rovide geoscience information about the Northwest Territories (NWT) to inform decisions by governments, industry, and the public concerning the responsible development of mineral and energy resources, use of the land, and environmental stewardship. The NTGS also plays an important role in delivering geoscience education and outreach services. Other than strategic advice provided to the Minister, geoscience information provided by the NTGS is freely available to all.^{Footnote 58}

The GSC and NTGS currently had two TSAs in place at the time of this evaluation and two that expired on March 31, 2017. The applicable GEM-2 project areas and objectives for each of these are listed below. 

Table 1 GEM-2 Task Shared Agreements between the GSC and NTGS

Project Areas	Activity Areas	Term	Objectives
Rae	South Rae	June 2016 – March 2018	South Rae Geoscience Bedrock and Quaternary mapping activity: Update the geoscience knowledge of southeastern Northwest Territories to a modern level at regional scale
Western Arctic	Banks Island	June 2016 – March 2017	Undertake kimberlite indicator mineral (KIM) sampling and analyses in the Nelson head – Rufus River are, southeastern Banks Island
Mackenzie	Mackenzie-Selwyn Geotransect	June 2017 – March 2020	To place Proterozoic to Cenozoic strata of the Mackenzie Platform, Selwyn Basin, and adjacent regions into a modern tectono-stratigraphic and metallogenic framework.
Mackenzie	Shield to Selwyn Geotransect – Brock Inlier/ Darnley Bay	June 2015 – March 2017	Improve geological maps of the area by increasing the density of geological field observations in NTS 97A and 97D. Existing geology maps of the area are out of date and of low resolution. Data relevant to the (GEM-2 Mackenzie Project) will be obtained from detailed documentation of outcrop exposures of lower Paleozoic and Proterozoic rocks along Hornaday River Canyon. A side benefit of the project will be to provide more current interpretive information for the area within the adjacent Tuktut Nogait National Park.

Source:               TSAs for collaboration between the GSC and NTGS in the Rae, Western Arctic and Mackenzie project areas.
Planned deliverables under these TSAs are combinations of:

• scientific / Yellowknife Geoscience Forum presentations;
• current research summaries and external journal papers;
• open File reports and data (Geoscan and/or NTGS); and
• geological maps

Objective of the NTGS case study

The objective of this case study was to obtain more detailed insights into the effectiveness and efficiency of the GEM-2 program’s collaborative arrangements with the NTGS. Conduct of the case study provided information that contributed to the assessment of the following evaluation issues and questions (see also the logic model in Appendix A and evaluation matrix in Appendix B for details).

Table 2            Case study coverage of the evaluation issues and questions

Evaluation Issues & Questions	Sub-Questions	Case Study Focus
1. Is there an ongoing need for the GEM-2 program?	1.1 Who are the principal stakeholders and what are their needs?	Nature of the NTGS and GSC’s expectations of their collaborative arrangements
	1.2 Are there new or emerging needs?	Reasons for the selection of the agreed activities and deliverables
	1.3 What factors are driving demand?
3. Is there a legitimate, appropriate and necessary role for the federal government in the GEM-2 program? Is NRCan’s role appropriate in the context of the role of others?	3.2 Are there other organizations that produce geoscience products similar to those produced by the GEM program? If so, is NRCan’s role appropriate in the context of the role of others?	Rationale for the selection of the respective activities and deliverables of the GSC and NTGS Potential for other organizations to undertake these activities and generate the associated deliverables
4. To what extent have intended outcomes been achieved as a result of the GEM-2 program?	4.1 To what extent have the program’s principal outputs been produced or undertaken as planned?	Extent to which agreed activities and deliverables have been realized or are on track for realization
	4.2 To what extent is the program achieving intended outcomes: Immediate outcomes: -   Industry and Northerners are aware of and have access to new regional geoscience data and knowledge generated by GEM -   Northerners and their institutions have access to tools that facilitate use of new regional geoscience data and knowledge generated by GEM	Contribution (or expected contribution) of the deliverables to the achievement of: GEM-2 immediate outcomes NTGS goals and strategies
5. What factors – internal and/or external to the program – have enabled or limited the achievement of the intended outcomes?	5.1 What factors have enabled or limited the achievement of the: a) intended program outputs? b) intended GEM-2 outcomes?	Extent to which processes involved in establishing TSAs, undertaking planned activities, and generating agreed deliverables met or exceeded expectations, or could have been improved.
6. Have there been any unintended (positive or negative) outcomes?	6.1 Have there been unintended outcomes that can reasonably be attributed to GEM-2 activities?	Nature of any unintended or unexpected results Responses to these results
	6.2 What actions have been taken in response to any unintended outcomes?
7. Is the GEM-2 program the most economic and efficient means of making progress toward the intended outcomes?	7.1 Is the program’s design appropriate for achieving the intended outcomes?	Lessons learned and/or best practices that could be applied to future collaborations with P/T geological surveys
	7.3 Are there opportunities to achieve more cost-effective delivery of program activities and production of outputs?	Opportunities to improve the efficiency (cost-effectiveness and/or timeliness) of activities and generation of deliverables
	7.4 Has actual spending been in line with planned spending?	Planned and actual data on respective levels of effort and expenditures on agreed activities.

Case study of the Sahtú Renewable Resources Board (SRRB) Multidisciplinary Grants

Background

Two of the nine grants were awarded to the Ɂehdzo Got’ı̨nę Gots’ę́ Nákedı / Sahtú Renewable Resources Board (SRRB). The SRRB was created by the Sahtú Dene and Métis Comprehensive Land Claim Agreement to be the main instrument of wildlife management within the Sahtú Settlement Area. Its Missions is to:

• Protect, conserve, and manage, in a cooperative spirit, all renewable resources within the Sahtu Settlement Area in a sustainable manner to meet or exceed the needs of the public today and in the future for generations to come, and
• Assist communities with the management of wildlife and habitat for the benefit of the people of the Sahtu Region.^{Footnote 59}

In carrying out its mandate, the SRRB works closely with Renewable Resources Councils (Ɂehdzo Got’ı̨nę) in the five communities of the Sahtú Region. The role of the Renewable Resources Councils is to “encourage and promote local involvement in conservation, harvesting studies, research and wildlife management in the community”.^{Footnote 60}

The SRRB’s 2015/16 project:

enabled the research, acquisition, cataloguing, preparation, and integration of a sampling of available GEM GIS spatial and documentary datasets into the online Atlas: a series of data conversions (kml, shapefile, pdfs) of the GEM data was conducted to make the GEM data technically compatible with existing traditional knowledge (TK) GIS datasets and attribute information. […] The resulting product now enables users to access TK concurrently with existing GEM studies and independently research energy and mine data within the context of their own cultural knowledge and environment. Further, decision-makers, educators, and students are now able to consider cross-cultural interpretations of geoscience and TK and consider mechanisms to tailor future Atlas versions with geoscience knowledge for their specific needs.^{Footnote 61}

The “Atlas” referred to in the above statement is the Sahtú Atlas, a compilation of maps and supporting descriptions of the region’s heritage, natural history (climate, ecology, wildlife and resource distribution), and state of resources and development.  

In carrying out the project, the proponents conducted over 20 workshop presentation/training sessions and community open houses that demonstrated the geoscience datasets and trained participants to use the geoscience component of the Atlas. Participants in these sessions spanned “primary and high school students and teachers, adult learners and adult learning instructors/program heads, Renewable Resource Councils (RRCs), co-management boards, elders, and community members”.^{Footnote 62}

A second project in 2016–17 was used to:

explore and showcase Dene and Métis relationships with their traditional territory via the interplay between indigenous/community knowledge and western science (geology), using material culture (archaeology) as a bridge between the two. Current resource interest and exploration activity datasets (fossil fuels, minerals) are also used to present contemporary land interest and use, as are classroom curriculum aids designed to assist teachers and educators promote student knowledge and reflection.^{Footnote 63}

GIS data sets were acquired and incorporated into the Sahtú Atlas. Landscape and community knowledge was then compiled using research sessions with Tulit’a community knowledge holders and high school students (Tulit’a (formerly Fort Norman) is a hamlet in the Sahtú region). Locations of stories obtained through this line of enquiry were integrated into the digital atlas, with text in both English and Slavey. Special attention was paid to capturing “information related to geological and resource activity considerations, as well as areas of key importance to the Tulıt́ ’a people.”^{Footnote 64}

The end-product from the project formed the basis for the preparation of materials for use in the local high school’s curricula, consisting of “explanatory documents, reference material, and discussion questions aimed for both independent study and group discussion”. The project proponents also indicated in their end-of-grant report that they intend to showcase the project results more widely within the Sahtú region and potentially seek to develop similar integrated content for other communities in the region (Délınę̨, Fort Good Hope, Colville Lake, and Norman Wells).^{Footnote 65}

Objective of the SRRB case study

The objective of the present case study was to obtain more-detailed insights into the SRRB projects’ relevance and key achievements and the effectiveness of the GEM-2 program’s support.

Table 3 provides more details on the evaluation issues and questions that were addressed by this case study (see also the logic model in Appendix A and evaluation matrix in Appendix B for details).

Table 3 Case study coverage of the evaluation issues and questions

Evaluation Issues & Questions	Sub-Questions	Case Study Focus
1. Is there an ongoing need for the GEM-2 program?	1.1 Who are the principal stakeholders and what are their needs?	Nature of the SRRB’s and GSC’s expectations of the support for tool development and deployment
	1.2 Are there new or emerging needs?	Extent to which GEM geoscience products can be utilized Types of decisions where GEM geoscience products can make a difference
	1.3 What factors are driving demand?
3. Is there a legitimate, appropriate and necessary role for the federal government in the GEM-2 program? Is NRCan’s role appropriate in the context of the role of others?	3.2 Are there other organizations that produce geoscience products similar to those produced by the GEM program? If so, is NRCan’s role appropriate in the context of the role of others?	Extent to which other organizations provide support to integrate and use geoscience products with other GIS-based data Degree of complementarity or duplication
4. To what extent have intended outcomes been achieved as a result of the GEM-2 program?	4.1 To what extent have the program’s principal outputs been produced or undertaken as planned?	Extent to which activities and commitments by the SRRB and GEM program have been realized or are on track for realization
	4.2 To what extent is the program achieving intended outcomes: Immediate outcomes: -   Northerners and their institutions have access to tools that facilitate use of new regional geoscience data and knowledge generated by GEM Intermediate outcome: Northerners, their institutions and exploration industry use modern regional geoscience knowledge to inform their decision-making	Contribution (or expected contribution) of GEM-supported tools to the achievement of: GEM-2 immediate and intermediate outcomes SRRB’s goals
5. What factors – internal and/or external to the program – have enabled or limited the achievement of the intended outcomes?	5.1 What factors have enabled or limited the achievement of the: a) intended program outputs? b) intended GEM-2 outcomes?	Extent to which the GEM program’s approach to supporting tool development and deployment has aided, or limited, the achievement of the GEM programs and the SRRB’s expected outputs and outcomes
7. Is the GEM-2 program the most economic and efficient means of making progress toward the intended outcomes?	7.1 Is the program’s design appropriate for achieving the intended outcomes?	Lessons learned and/or best practices that could be applied to improve the effectiveness of GEM support for tool development and deployment

---