Evaluation Report: GeoConnections Program Phase III

Table of Contents

• Executive Summary
• Recommendations and Management Response
• 1.0 Introduction and Background
  • 1.1 Introduction
  • 1.2 Overview of the GeoConnections Program
  • 1.3 Program Resources
• 2.0 Evaluation Methodology
  • 2.1 Evaluation Methodology and Approach
  • 2.3 Evaluation Limitations and Mitigation Strategies
• 3.0 Evaluation Findings: Contribution of GeoConnections Program to Expected Results
  • 3.1 Expected Result: NRCan determines a need (lack of standards and interoperability) and defines GeoConnections objectives
  • 3.2 Expected Result: Appropriate governance bodies formed for building consensus among key stakeholders
  • 3.3 Expected Result: Shared understanding of geospatial priorities among key stakeholders
  • 3.4 Expected Result: Common geospatial standards and policies are developed
  • 3.5 Expected Result: Common geospatial standards and policies are widely adopted and implemented
  • 3.6 Expected Result: Geospatial data is interoperable and available online for Canadians to access enabled by technology/applications
  • 3.7 Expected Result: Geospatial data is accessed by decision-makers (or their staff) and used to make better decisions on social, economic and environmental priorities
  • 3.8 Expected Result: Benefits to Canadians
• 4.0 Evaluation Findings: Efficiency and Economy
• 5.0 Evaluation Conclusions and Recommendations
• Appendix A: List of Terms and Definitions
• Appendix B: Stakeholder Survey Respondent Characteristics

Acknowledgements

The Evaluation Team would like to thank those who contributed to the GeoConnections evaluation, particularly members of the Evaluation Advisory Committee, as well as others who provided insights and comments crucial to this evaluation.

The evaluation project was managed by Nicholas Kowbel, with evaluation support from Carol Gudz of the Strategic Evaluation Division. Jennifer Hollington, Head of Evaluation at NRCan, and Gavin Lemieux, Director, provided senior management oversight. Phoenix Strategic Perspectives Inc. also provided evaluation services for the project.

Executive Summary

Introduction:

This report presents the findings of the evaluation of Natural Resources Canada’s (NRCan’s) GeoConnections Program Phase III (within the Geospatial Innovation Sub-Program 1.2.3 in NRCan’s 2014-15 Program Alignment Architecture). GeoConnections is administered by NRCan’s Canada Centre for Mapping and Earth Observation (CCMEO), which is part of the Earth Sciences Sector. Phase II of this program was last evaluated in 2010. This evaluation covers program activities from 2010-11 to 2014-15, comprising $25.1 million in NRCan funding. 

Program Design and Objectives

GeoConnections Phase III provides federal leadership to connect geospatial data sources, and optimize the use of geospatial data in decision-making. The program also provides federal leadership for geomatics policy development. 

The program achieves this by supporting the development and implementation of the Canadian Geospatial Data Infrastructure (CGDI) through work on common standards, operational policies, and governance. 

Phase III of the program was designed based on recommendations from the 2009 Evaluation of Phase 2 which highlighted the need for the Earth Sciences Sector to:

• Examine mechanisms to ensure local expertise developed in Phase II of the program could be sustained and transferred to a broader federal or provincial level;
• Improve usability of the Canadian Geospatial Data Portal, through pilot testing;
• And, developing a broad sustainability strategy to ensure that the CGDI brand is clearly understood and that the Canadian Geospatial Data Infrastructure is maintained.

Methodology:

Evaluation data collection was from June to September 2015, and included 31 key informant interviews (with industry, federal, provincial and territorial governments, academics and external experts), a document review (including program studies and international comparisons to Europe and the US), and a stakeholder survey (n = 79; 38% response rate). The evaluation took a theory-based approach, studying whether or not and under what circumstances program activities contributed to expected results (i.e., assessing the validity of program theory). To do this, a results chain was developed in consultation with the program. The findings are presented by intended results, identified in bold below. While this methodology had several limitations (discussed in the report), multiple-lines-of-evidence were used by the evaluators to mitigate this by triangulating findings.

Evaluation Findings: Contribution of GeoConnections Program to Expected Results

Understanding the need and defining the program: GeoConnections is aligned with federal and NRCan priorities, and there is a continuing need for activities it undertakes. The importance of geographic data has increased enormously beyond the traditional geomatics sector involving more players and approaches to location based data than ever before. At the same time, the geospatial community, which is increasingly broadly defined due to the rise in interconnectedness and technologies that use geospatial data, continues to be highly fragmented. GeoConnections Phase III responds to public and private sector stakeholder needs for data and technical standards linking regional and thematic spatial data infrastructures. This facilitates the efficient use and re-use of interoperable geospatial data. The program also focused on governance to respond to the need for leadership and a coordinated approach to implementing CGDI. Program stakeholders articulate a need for improved communication from NRCan and others about CGDI and greater user focused technology development support. Having said that, stakeholders also recognize that the program is doing a lot with few resources in a continuously changing context.

GeoConnections has had to adapt to the changing technology and policy context in geospatial data over its three phases. In Phase 1 (1999-2004), the program responded to a need articulated by the Canada Council on Geomatics (CCOG) to link and make interoperable federal provincial and territorial geospatial data sets over the internet. The program began development of the CGDI and contributed to the creation of GeoBase which makes available many federal, provincial and territorial publicly owned geospatial data sets with public and private sector users. GeoConnections Phase II (2005 to 2009) focused on supporting community use of geospatial data and policy development that contributed, with provincial and territorial support, to NRCan’s 2007 implementation of an Open Data policy. This also supported Canada’s Open Government (Data) initiative by providing Canadians with access to a large number of geospatial datasets. GeoConnections Phase III (2010 to 2015) focused on data and technical standards that are essential to the functioning of CGDI. The program also sought to establish coordination between traditional geomatics and growing number of geospatial data users and producers in non-traditional sectors (e.g., health care, insurance, etc).

Appropriate governance bodies formed: GeoConnections has put in place appropriate governance bodies supporting CGDI development. These include the Federal Committee on Geomatics an Earth Observations (FCGEO), the Canadian Council on Geomatics (CCOG) and the Canadian Geomatics Community Round Table (CGCRT) – now GeoAlliance Canada. The CGCRT has been cited as an example of Open Government by establishing a collaborative, formal relationship between state and non-state actors enabling the identification of shared interests and goals.^{Footnote 1} Stakeholders are concerned , however, with the readiness of the geospatial sector to transition from the NRCan-led CGCRT to the sector-led GeoAlliance. This is due in part to challenges securing resourcing, difficulty obtaining buy-in  for an organization where NRCan is not a visible lead, and the fragmented nature of the geomatics and geospatial data community. Note that GeoAlliance is not taking over a role or tasks once done by NRCan. It seeks broader membership than CGCRT encompassing the whole geospatial data community.

In general, interviewee evidence suggests that most of the appropriate organizations are participating in the CGDl governance organizations. Having said that, there is also a perceived need to further engage broader user communities such as municipalities, oil and gas and forestry, and non-traditional groups such as health care providers, information technology companies and the insurance industry. These groups were also seen as important geospatial data users. Results of the stakeholder survey indicate that GeoConnections’ efforts on improving coordination in the sector have been important (73% at least somewhat, including 44% highly important). However, the geospatial community continues to see progress by geospatial stakeholders on sector governance as weak, though improved from five years ago due to GeoConnections’ efforts. This ongoing challenge is likely due to the highly fragmented geospatial community in Canada, and the lack of dedicated human and financial resources supporting involvement in CGDI development.

Shared understanding of priorities: There is high level agreement on geospatial priorities within the community, as evidenced by the Pan-Canadian Geomatics Strategy, the Canadian Geomatics Accord and the commencement of work on the Federal Geospatial Platform. GeoConnections has contributed to these achievements  through inclusive processes implemented by the CGCRT and CCOG governance bodies to develop these agreements as well as the work of the FCGEO to organize the implementation of the Federal Geospatial Platform. However, interviewee evidence suggests there is some disagreement over how to implement these agreements (what should be done first, who should act and who should pay). This is partially due to the reported difficulties with sectoral coordination and obtaining buy-in to the GeoAlliance noted earlier. Program tools and guidance materials promoting the benefits of CGDI were found to be persuasive, valued and used by the community (86-88% found these tools at least somewhat helpful, including 52-62% highly helpful). At the same time these tools are not likely to be reaching non-traditional users.

Common CGDI standards and policies developed: There is clear evidence that common technical geospatial standards and policies have been developed for CGDI. Regarding standards, this work has been aided by program research on standards and effective program participation in international standards fora. However, some interviewees suggested that consultation within Canada could be more systematic. Operational policies have also been developed based on identified community needs, though interview and document evidence suggests that further guidance may be needed on intellectual property issues and data stewardship. Speaking to its leadership in the area of spatial data infrastructure standards and policies, GeoConnections led the research and development work for the Spatial Data Infrastructure Manual for the Americas, which provides guidance on the development of spatial data infrastructures for an international audience. ^{Footnote 2} Stakeholders see GeoConnections’ work on developing standards (75%) and operational policies (61%) that meet user needs as important priorities. However, stakeholders provide weak overall assessments of the program’s contribution to progress in these areas (27-33% rated progress as strong). Evidence from interviews shows  that in some cases this is likely based on a low awareness by program stakeholders of NRCan activities. Interviewees also indicated a need to renew focus on metadata standards to bring them up to date and make them easier to implement.

Common CGDI standards and policies widely adopted and implemented: Multiple lines of evidence show progress on implementing CGDI standards and policies, including numerous examples of implementation. That said, survey and interview evidence suggests progress towards full implementation is likely modest. For example, 70% of surveyed stakeholders had at least somewhat implemented CGDI standards, with 43% having mostly implemented them. It is difficult to assess how widely adopted the standards are as there is no systematic monitoring in place (highlighted in the 2012 and CGDI assessment exercises^{Footnote 3}). Work on metadata standards supporting data discovery is seen by stakeholders as an ongoing and important effort, but concerns have been raised over data discovery in general and implementation of meta data standards specifically. Metadata is also an ongoing challenge for Spatial Data Infrastructure (SDI) efforts in the EU and US where application of spatial data infrastructure standards in public sector is mandatory unlike in Canada where it is not mandatory. Note however that progress appears similar in both mandatory and non-mandatory regimes.

The evaluation finds that application of CGDI standards and policies is negatively impacted by external factors within the geospatial data community including availability of organizational resources, competing operational priorities, understanding of CGDI, and available geo professionals to implement. That said, implementation progress continues. This is aided by GeoConnections’ pursuit of international standards, geomatics industry consolidation, increased use of off-the-shelf software, and demonstrations of applied CGDI standards through GeoConnections G&C projects that help NRCan to advocate for practical standards for Canada.

Geospatial data are interoperable, available online using technologies: There has been good progress on making geospatial data available online in Canada, based on CGDI standards and policies, and accessible through technology/applications. In total, 84% of surveyed stakeholders said that the availability of interoperable web-based geospatial data from their organization has increased a little (24%) or a lot (60%) in the last five years. Documents and interviews also confirm many examples of online geospatial data portals, platforms and web-services making public data available in Canada (e.g., GeoGratis, GeoDiscovery Portal, regional spatial data portals). GeoConnections has enabled public and private investment in geospatial technologies through support for projects with its G&C program and work on standards that makes technology/application development easier. However, interviewees suggested the impact of this work could be improved with communication of G&C project results (either by NRCan or others) to the wider community so that they may leverage these in their own work.

Even though it is clear that the availability of interoperable web-based geospatial data in Canada has increased in the last five years, community attribution to GeoConnections is weak. Just 49% of surveyed stakeholders thought GeoConnections/CGDI was at least somewhat important to this result. Survey evidence suggests that this is most likely due to stakeholders’ lack of awareness of NRCan’s actual role  and its activities in the development of standards and data interoperability, given that much of this occurs in the background. Interviewees widely agreed that while progress on data availability has been good, online interoperability of geospatial data is really only at the beginning. The evaluation identifies a number of challenges to overcome within the geospatial data community to increase the availability of interoperable online data. These include competing organizational priorities, restrictive data licensing, governance of shared data systems, insufficient  bandwidth and treatment of legacy datasets.

Geospatial data accessed and used by decision-makers: The evaluation has corroborated the expected link between GeoConnections’ activities and greater access to and use of geospatial data. Evidence from the stakeholder survey and interviews shows that demand for and use of geospatial data have risen considerably in the last five years (87-89% of surveyed data producers and users cited increased demand). There are also many examples of use enabled by CGDI (e.g., tracked data downloads, interprovincial coordination on forest fire responses, directed infrastructure investments, ). That said, systematic measurement of data use is not done by the program or anyone else. The evaluation also found that many factors impede the use of geospatial data. These include the degree to which senior decision-makers are aware of the benefits of data use, the need to confirm quality accuracy and currency of available data, the need to break down decision-making silos that ignore the ability to combine data, and the need for technology that allows for easy integration of data into workflows for non-experts in geomatics.

Benefits to Canadians: A 2015 economic modelling study commissioned by GeoConnections showed that the use of geospatial outputs contributed $20.7 billion to Canadian Gross Domestic Product (GDP) in 2013. .Based on the evaluation evidence, it is reasonable to say that GeoConnections makes a significant contribution to enabling this geospatial data use through its work on governance, standards and interoperability. There are many examples of benefits, such as improved competitive advantage and improved efficiency that have resulted from using geospatial outputs (information, technology, applications) for decision-making. Geospatial outputs are providing benefits to Canadians in areas that are somewhat more measurable in economic terms (e.g. indicated by its contribution to GDP), as well as in areas that are less tangible and less measurable (e.g. environmental protection, health and social benefits).

NRCan products and services have also benefited from the CGDI standards and operational policies or the base data made available through GeoBase and GeoGratis. Some examples include the NRCan led Federal Geosapatial Platform, the Canada Land Cover Data project, Groundwater Information Network, National Forest Inventory System, and the Canadian Wildland Fire Information System. 

Evaluation Findings: Efficiency and Economy

The GeoConnections Program design includes many examples of efficient practices. GeoConnections is seen to be well managed by internal and external stakeholders. It includes focus on national CGDI that can be levered by provincial/territorial governments through governance bodies established by GeoConnections which limits duplication. It now focuses on international consensus based standards to increase adoption of common standards, and has consolidated geospatial data portals under GeoGratis. Note that both governance and GeoGratis continue to be challenges, although CCMEO expects the Federal Geospatial Platform to continue to address these issues.

Finally, the G&C component of GeoConnections has achieved a strong leveraging ratio of 3.9 dollars in cash and in-kind contributions to projects for every dollar invested by GeoConnections through its G&C program. Interviewees expressed concern that the planned decrease in G&C funding would slow program progress.

Conclusions and Recommendations:

GeoConnections has responded to a well-understood need for activities that support the CGDI, enabling access to public data that would otherwise be difficult to find or use.

Evaluation evidence corroborates the program’s contribution to each of its intended results, suggesting that the program theory is correct. It also finds the program to be an efficient means of achieving program objectives.

Many of the challenges identified in this evaluation are consistent with the experiences of the US and EU in implementing their respective spatial data infrastructures. These include issues with data discovery and meta-data, implementing standards and policies across jurisdictions, and coordination of public and private stakeholders. 

A number of opportunities were also found to improve results for the next phase of the program. These are discussed in the context of the recommendations made below.

Recommendation 1: GeoConnections should strengthen its national convening role and work with CCOG, FCGEO, and GeoAlliance to communicate widely the value of CGDI and the importance of its uptake.

• The evaluation finds that there is a generally low awareness of GeoConnections’ pan-Canadian and international role in developing the spatial data infrastructures that comprise the CGDI and in understanding the CGDI itself. The CGDI focuses on sharing open, high value geospatial data. The leadership, experience and knowledge generated through the GeoConnections program could be better communicated to the emerging open data community as they move towards open data environments (i.e., greater use of open data).
• Evidence from surveys and interviews shows that while stakeholders see value in the GeoConnections program goals, there is likely a low understanding of what NRCan does with respect to standards, operational policies, and technology work on CGDI. These components are key to making open and interoperable forms of geospatial and non-geospatial data highly valuable. To leverage greater participation in implementing and supporting CGDI, as well as to reach  new and emerging user communities, a coordinated communication effort is needed. Note that this recommendation is consistent with the recommendation of the 2015 CGDI Assessment.

Recommendation 2: GeoConnections should‎ reinforce its leadership on open data by supporting more efficient collection and sharing of data by addressing concerns raised over a) the discovery of data and b) coordination on updating of foundational geospatial datasets.

• While the provision of interoperable and integrated geospatial data are higher-order goals pursued by GeoConnections, the ability to discover the data, in any form, needs to be assured in order for it to be used and to avoid duplication. The evaluation found that more work is needed on discovery of public datasets across Canada. This challenge appears tied to development and application of metadata standards, search functionality on public portals such as GeoGratis, and data sharing agreements. 
• The evaluation evidence suggests a need for a coordinated approach to framework data maintenance that would avoid duplication across jurisdictions. Framework data is an essential part of the CGDI as it provides the base layer onto which other location-based data can be fixed. There are considerable gaps emerging in Canada with respect to the completeness and currency of framework data. This is in part due to underinvestment in framework data due to fiscal constraints over the past ten years, but also due to a lack of coordination that could ease those constraints. Many industry and public sector interviewees suggested that national strategies would be very important for finding cost effective ways to update and maintain framework data. NRCan, most note, would be the natural lead on such an effort.

Recommendation 3: GeoConnections should provide targeted support to integrate geospatial data into the decision-making processes of non-experts

• Use of geospatial data depends on the availability of and access to data. However, even when the maximum amount of data is available and accessible; if it cannot be translated into easily consumable analyses it will not inform decision-makers. Evidence suggests there are considerable opportunities to improve use of data in decision-making through developing applications that make the data accessible and usable to non-experts (i.e., analysts and decision-makers who are not geomatics professionals). Many interviewees suggested that this was the next most important step in enhancing geospatial data use. As well, surveyed stakeholders gave the lowest assessments of progress and capability on CGDI issues of use of geospatial data and development of user friendly applications/technologies.

Recommendation 4: GeoConnections should continue to work with the geospatial community to encourage the transformation from a narrow focus on geomatics experts to a broad focus on non-traditional user groups. This should include supporting the GeoAlliance, whose objectives of implementing the Pan Canadian Geomatics Strategy are complementary to the GeoConnections program.

• The evaluation found that sector coordination, while better than it has been, is still weak overall. There is a need to attract non-traditional user groups to the community so that they may provide input to and derive greater benefit from the CGDI. The evaluation also found concerns with sector readiness for the evolution from the NRCan-led CGCRT to the sector-led GeoAlliance Canada (whose mandate is to have a broad membership). This is due in part to limited geospatial data sector  resourcing, challenges obtaining buy-in to an initiative where NRCan is not the lead, and the fragmented nature of the geomatics and geospatial data community from which broad participation is being sought.

Recommendations and Management Response

Recommendations	Management Response	Responsible Official/Sector (Target Date)
1: GeoConnections should strengthen its national convening role and work with CCOG, FCGEO, and GeoAlliance to communicate widely the value of CGDI and the importance of its uptake.	Management concurs with this recommendation. Action: The Canada Centre for Mapping and Earth Observation (CCMEO) will work with: Federal/Provincial/Territories and, where possible, municipal and local partners to work with the Open Data Community to communicate the pivotal role of geospatial standards, data, and the CGDI to the open data and open government agenda.  The Open Geospatial Consortium (OGC) to further the Arctic Spatial Data Infrastructure with Canadian and International Arctic stakeholders through rigorous OGC piloting and testbed activities in conjunction with OGC’s 515 member base, FCGEO, CCOG and GeoAlliance. The Canada Centre for Mapping and Earth Observation will work with new and emerging communities (e.g., open science, CIO communities) to communicate the GeoConnections role on geospatial data standards, operational policies, and interoperability. This will include: working with NRCan Communications to develop a coordinated national communications and outreach strategy.  Internationally, CCMEO will: coordinate with member countries of the Arctic Spatial Data Infrastructure initiative to identify data of mutual interest and establish approaches to collect and share data seamlessly. support the Arctic Council (Global Affairs Canada) and the Arctic Spatial Data Infrastructure project by creating a promotional video on the importance of spatial data infrastructures, interoperability and geospatial data standards in the global effort to provide evidence that can be used to combat climate change. Promotional materials and engagement within federal departments, provincials, territories, and other actors will be developed to showcase practical applications, and the importance of geospatial standards to data sharing and interoperability. These will be shared at conferences, and through various web-based media including group discussions, and YouTube. Lessons learned will be developed from a joint international project with the United States Geological Survey, and the Open Geospatial Consortium to develop geospatial standards to build interoperability with new data forms and sets, such as earth observation or satellite imagery.  Results of the interoperability project will be shared on the NRCan website in March 2017, at technical working group, scientific and open data conferences. Joint international sharing of project results will be done in collaboration with the USGS, and possibly the United Nations, and the Open Geospatial Consortium. Results will also be shared with GeoAlliance Canada, and specific conferences with the Canadian private sector to provide technology and standards transfer throughout the course of 2017 and 2018. Pending positive results on testbed activity, the USGS and CCMEO will pursue international standards adoption for use globally.	ADM, ESS (March 31, 2017)
2: GeoConnections should‎ reinforce its leadership on open data by supporting more efficient collection and sharing of data by addressing concerns raised over a) the discovery of data and b) coordination on updating of foundational geospatial datasets.	Management concurs with this recommendation. The Canada Centre for Mapping and Earth Observation will develop a comprehensive action plan to promote its leadership role on geospatial standards; more effective data collection and efforts at data discovery.  The Canada Centre for Mapping and Earth Observation will: Deliver on the Federal Geospatial Platform initiative which is designed to enhance discoverability, access, and interoperability of federal data. This initiative will make accessible and interoperable over 300 new data sets and layers, which were previously unavailable and non-interoperable. The federal geospatial platform will be built as part of an enterprise wide approach to ensure those voluntarily contributing do not duplicate data acquisition or technology implementation, using the principle of build once, use many times, for maximum value. Next generation data access, sharing and collection capability will be developed through the implementation of new technology solutions in collaboration with the Government of Canada and NRCan Chief Information Officers.  With respect to findings on the coordination of approaches to update foundational data sets, management agrees that there are considerable gaps in Canada’s foundational geospatial data. This issue will be covered in the upcoming evaluation of PAA activity 3.2.1, and management will develop actions in that context.	ADM, ESS (December 2017)
3: GeoConnections should provide targeted support to integrate geospatial data into the decision-making processes of non-experts.	Management concurs with this recommendation. The Canada Centre for Mapping and Earth Observation will:  Work with the private sector, universities, and non-governmental organizations to either pilot or develop new analytical tools and approaches to ensure better use of spatial data in a variety of applications requiring better location data. Work with other federal departments and agencies to ensure big and open data initiatives leverage CCMEO’s expertise and funding. An on-going biannual review process will be implemented by CCMEO to assess progress made on the use of data, by December 2017.	ADM, ESS December 2017 implementation of a biannual review process on progress made. These actions will be on-going and part of operating processes.
4: GeoConnections should continue to work with the geospatial community to encourage the transformation from a narrow focus on geomatics experts to a broad focus on non-traditional user groups. This should include supporting the GeoAlliance, whose objectives of implementing the Pan Canadian Geomatics Strategy are complementary to the GeoConnections program.	Management concurs with this recommendation. The Canada Centre for Mapping and Earth Observation will:  Work with Geo-Alliance, in its convening capacity, to bring together various non-traditional users of geospatial data and tools. Support Geo-Alliance financially and provide regular oversight to the project, as required	ADM, ESS (March 2017)

1.0 Introduction and Background

1.1 Introduction

This report presents the findings of the evaluation of Natural Resources Canada’s (NRCan’s) GeoConnections Program (within Sub-Program 1.2.3 in 2014-15 Program Alignment Architecture). The GeoConnections Program is administered by NRCan’s Canada Centre for Mapping and Earth Observation (CCMEO), which is part of the Earth Sciences Sector. This program was last evaluated in 2009. On April 1^st 2015, GeoConnections transitioned to an A-base program, with ongoing Departmental funding of $5 million/year.

The evaluation covered the five-year period from 2010-11 to 2014-15 and expenditures of about $25.5 million. The evaluation did not cover the Federal Geospatial Platform, which will be evaluated separately in 2018-19.

1.2 Overview of the GeoConnections Program

The overall goal of the GeoConnections Program is to facilitate access to and use of public geospatial information in Canada that would otherwise not be accessible or easy to use.,

 To achieve this goal, the program supports the development, and use of the Canadian Geospatial Data Infrastructure (CGDI).^{Footnote 4}

The CGDI is the policies, standards, technologies and framework data necessary to harmonize all of Canada’s location-based information on the Internet.^{Footnote 5} The CGDI reduces barriers to using geospatial information so that Canadians can discover, access, visualize, integrate, apply and share quality location-based information and make effective decisions.

History:

GeoConnections has led an iterative process to develop and implement the CGDI, starting with establishing governance bodies overseeing the development of data and technology standards, key geospatial data, and operational policy guidance. Once developed, the next stage was implementation, characterized by the ongoing development of standardized geospatial data and standardized technology tools enabling the efficient access, sharing and use of the geospatial data. These activities were intended to support the achievement of the ultimate goal of the program, namely the use of geospatial data by decision-makers for improved decision-making on social, economic and environmental priorities.

Since 1999, the Canadian federal government has invested in coordinating a national effort to build the CGDI supported by the GeoConnections program. During its first five years (Phase 1, 1999-2005), GeoConnections partners and stakeholders laid the foundation for the CGDI, focusing on the use of the internet as a platform for sharing geospatial data, and creating a distributed network where each data provider could retain control of and update their own information.^{Footnote 6} Phase 1 also focused on developing strong partnerships, an inclusive governance structure, and promoting development and adoption of standards that would enable interoperability – the ability to share and compare different data sets.^{Footnote 7}

Phase 2 (2005-2010) focused on (a) expanding partnerships to facilitate and improve access to Canadian geospatial data and (b) enhancing user community (e.g., municipalities, resource-based communities) capacity to harness the CGDI to support integrated decision making in areas of federal priority: public health, public safety and security, sustainable development and the environment, and matters of importance to Aboriginal peoples.^{Footnote 8} 

The goal of Phase 3 (2010-2015) was to provide federal leadership to optimize the use of geospatial data for decision-making and lead strategic geomatics policy development.^{Footnote 9} 

Program Structure:

The Components of CGDI:

The CGDI is a web-based knowledge infrastructure that is intended to improve the efficient access, sharing and use of geospatial data, or data that is associated with geographic locations in Canada.^{Footnote 10} The CGDI consists of: 

• Geospatial ( location-based) data;
• Technology tools to enable the search, discovery, viewing, downloading and transformation of geospatial data;
• Standards for the geospatial data and for the technology tools to ensure that all elements function seamlessly and efficiently with each other; and
• Operational policies, covering areas such as intellectual property, data sharing and privacy, governing the use and re-use of the geospatial data

Governance of CGDI

GeoConnections’ GeoSecretariat fosters collaboration and partnerships between federal, provincial, territorial and regional governments; the private sector; and academia to guide the development of all parts of the CGDI and ensure their interoperability. The GeoSectretariat does this by convening, providing coordination, and support for the following committees:  

• Federal Committee on Geomatics and Earth Observations (FCGEO): Created in January 2012 and chaired by ADM ESS, has 20 federal department and agency members that produce and/or consume geospatial data, or have an interest in geomatics-related issues. The FCGEO coordinates federal geomatics efforts, including development of the Federal Geospatial Platform (FGP).
• The Canadian Council on Geomatics (CCOG): is a federal-provincial-territorial forum for exchanging information on and addressing common operational issues, and developing and promoting national geomatics standards. Coordination through CCOG has produced the Canadian Geomatics Accord, and GeoBase, a portal that provides access to a base of quality geospatial data for Canada at no cost and with unrestricted use.
• Canadian Geomatics Community Round Table: The CGCRT was a multi-stakeholder forum, created in 2009, to enable open dialogue on issues in geomatics across industry, government, and academia. CGCRT led development of the Pan-Canadian Geomatics Strategy, a roadmap for enhancing the geospatial sector and use of geospatial data. On April 20th 2015, CGCRT was replaced by GeoAlliance Canada.

GeoAlliance Canada evolved from the CGCRT to implement the Pan-Canadian Geomatics Strategy, by pooling resources from all of its members. It plans to become an umbrella structure that includes the whole “geospatial data community” encompassing both the “geomatics sector” (as with CGCRT) and the users of spatial data and information products, technologies, services and expertise. While the CGCRT was coordinated and co-chaired by NRCan through GeoConnections, NRCan does not coordinate GeoAlliance. Rather, it is a member organization.  

1.3 Program Resources

Table 1 presents planned and actual expenditures by type – Operating and Maintenance (O&M), Salary, and Grants and Contributions (G&C) – for the GeoConnections Program from 2010-11 to 2014-15 (the evaluation period). This table does not include in-kind or leveraged resources. Note that in 2014-15, the program employed 15 FTEs.

Table 1: GeoConnections Program Phase III Financial Expenditures, 2010-11 to 2014-15 ($000)

	2010-11		2011-2012		2012-13		2013-14		2014-15		Total all Fiscal Years
	Planned	Actual	Planned	Actual	Planned	Actual	Planned	Actual	Planned	Actual	Planned	Actual
NRCan O&M*	4,500	4,324	5,500	4,027	5,500	4,917	5,500	4,841	5,500	3,971	26,500	22,080
NRCan G&C	500	499	750	197	750	890	750	733	750	739	3,500	3,059
Total	5,000	4,823	6,250	4,225	6,250	5,807	6,250	5,574	6,250	4,710	30,000	25,139

*incl. Salary, EBP, operating

2.0 Evaluation Methodology

2.1 Evaluation Methodology and Approach

Contribution Analysis was used in this evaluation. This is consistent with Treasury Board Secretariat Guidance for evaluating complex programs such as GeoConnections.^{Footnote 11} Contribution Analysis does not attempt to prove that one factor – a policy or program – ‘caused’ the desired outcome. Instead, this approach explores the contribution made by a policy or program to the observed results. To do this, a results chain was developed, based on input from GeoConnections staff,that articulated the ‘theory of change’ showing the links between program activities, outcomes and contexts. Evidence was collected to test the validity of the results chain and support the construction of a credible ‘performance story’ to demonstrate the extent to which the program was an important influencing factor in driving the observed change.

The evaluation effort was calibrated to take into account the recent assessments that had been conducted by the GeoConnections Program.^{Footnote 12} Data collection activities built on existing activities that the GeoConnections program had commissioned, including an updated assessment of the CGDI and case studies exploring the activities and accomplishments of the program in the areas of (i) supporting the work of Canadian Council on Geomatics and (ii) supporting standards development work.

The evaluation employed the following lines of evidence:

• Document and literature review: SED reviewed key program documents including the 2012 and 2015 CGDI assessments (desktop exercises), program case studies, and international literature on the development of spatial development infrastructure in the United States and the European Union.
• Survey: A private contractor conducted a census survey of key stakeholder groups (members of the CCOG, FCGEO, and CGCRT/GeoAlliance) during August and September 2015. In total, 79 respondents completed the online survey, representing a strong response rate of 38%. Appendix A contains an analysis of survey respondent characteristics.
• Interviews: A total of 31 interviews (6 internal and 25 external) were conducted from July to September 2015. Interviewees included representatives from NRCan senior and program management, federal, provincial, academic and industry members of the GeoConnections program stakeholder committees, recipients of grants and contributions, and external experts in spatial data infrastructure.

Below is the results chain developed to evaluate the GeoConnections program. On the left are the series of results that the program expects and on the right are the assumptions and factors thought to impact these results. The evaluation tested these to assess contribution of the program to observed results.

Table 2: Results Chain for GeoConnections Program Phase III

Results Expectation	Assumptions & factors
Program Design and Management
NRCan determines a need (lack of standards and interoperability) and defines GeoConnections objectives	Appropriate information, understanding and analysis of problems convert into appropriate program design, investment
Appropriate governance bodies formed for building consensus of key FPT, industry, and academic geospatial data actors (CGCRT, FCGEO, CCOG)	Stakeholders have will and ability (expertise) to engage and participate The right stakeholders are involved
Shared understanding of geospatial data priorities across FPT, industry, academe (Cdn Geomatics Accord, Pan-Canadian Geomatics Strategy, National Mapping Strategy)	Agreement across sectors and groups on how to proceed Equitable competition for ideas across sectors and groups. Target groups have been exposed to and persuaded by program white papers, webinars, etc
Program Outcomes
Common geospatial standards and policies are developed	Standards and policies respond to the needs of the community (data holders, infrastructure)
Common geospatial standards and policies are widely adopted and implemented	Public and private sector data producers have will and available resources (cost, HR are factors) to implement
Geospatial data is interoperable and available online for Canadians using technology tools	Public and private organizations launch services/platforms that combine geospatial information from various sources Public and private sector investment climate is conducive to developing innovative geospatial data products Data collection organizations have been persuaded of the value of collecting data using common standards
Broader Outcomes
Geospatial data is accessed by decision-makers (or their staff) to make better decisions on social, economic and environmental priorities.	Decision makers understand the value of and are attracted to using geospatial data to make better decisions Geospatial information reaches Canadian policy-makers, industry, and Canadians.
Net benefits to Canadians (e.g., improved competitiveness of Canadian firms, safety, infrastructure planning, service delivery and development)	The most optimal choices are made with the available information Improved decisions result in positive outcomes

2.3 Evaluation Limitations and Mitigation Strategies

Although the evaluation has been designed to target multiple lines of evidence at key questions to enhance the reliability of results, there are several limitations and challenges to the evaluation methodology.

1. Program Complexity: Geospatial data infrastructures in general involve diverse interdependent agents interacting in an environment that is continuously evolving. This creates challenges to define what is to be measured by the evaluation. To mitigate, early in the evaluation definitions of key concepts were developed by GeoConnections staff in collaboration with SED.
2. Interview sample identified by the GeoConnections Program: At SED’s request, GeoConnections program staff compiled and provided contact information for interviewees for the evaluation. While the program is best placed to develop the list of interviewees, the potential for a selection bias exists when the evaluation subject is responsible for identifying contacts. To mitigate, interviews were the decision of the evaluation team (subject to willingness and availability of interviewees), and additional interviewees were added to the list as they were identified during fieldwork.
3. Attribution challenges due to nature of program interventions: Program interventions, primarily consisting of setting up governance structures and working on standards and policies, by their nature took place behind the scenes in the overall CGDI development and implementation process. This means that they were far removed from the ultimate outcome of data usage. In addition, data use for decision-making was clearly ubiquitous. The nature of the interventions made it difficult to determine the extent of attribution of program work to expected outcomes. To mitigate, SED employed a Contribution Analysis approach and multiple lines of evidence to improve confidence in findings of the Program’s contribution to observed results.

3.0 Evaluation Findings: Contribution of GeoConnections Program to Expected Results

3.1 Expected Result: NRCan determines a need (lack of standards and interoperability) and defines GeoConnections objectives

Results Expectation	Assumptions & factors tested	Summary of Findings
3.1 NRCan determines a need (lack of standards and interoperability) and defines GeoConnections objectives	Appropriate information, understanding and analysis of problems convert into appropriate program design, investment	GeoConnections Program goals aligned with federal and NRCan priorities GeoConnections responds to ongoing needs for a coordinated, standards-based approach to developing and implementing the CGDI . The need likely exceeds program scope

Finding: GeoConnections is aligned with federal and NRCan priorities

• The GeoConnections Program’s focus on data accessibility is consistent with Canada’s 2011 Open Government initiative, while its focus on supporting cost-effective geospatial information solutions is aligned with federal priorities indicated in Canada’s 2012 Economic Action Plan (which prioritized investment in innovation, education and training as well as deficit reduction).
• The Program’s goals are also consistent with goals of the federal government articulated in 2015 to accelerate and expand open data initiatives so that Canadians may easily access and use that data.^{Footnote 13}
• GeoConnections’ involvement in the Arctic Spatial Data Infrastructure is consistent with Canada’s 2009 Northern Strategy. 

The GeoConnections Program is formally linked to NRCan’s Strategic Outcome 1 (Canada’s natural resource sectors are globally competitive, but is also consistent with Strategic Outcomes 2 and 3, since the data made available for use through the program supports decisions on economic development, environmental protection, and safety and security decisions.

Finding: The importance of geographic data has increased enormously beyond the traditional geomatics sector involving more players and approaches to location based data than ever before. GeoConnections provides geospatial community leadership that responds to needs for a centralized, standards-based approach to developing and implementing the CGDI

There is ample evidence that location-based data use is becoming ubiquitous, involving more players than in the past. As one example, Google Earth, a platform for hosting and sharing geographically-referenced information, was downloaded one billion times between 2005 and the end of 2011.^{Footnote 14} Technological improvements to computer hardware and software, particularly related to geographic information systems, have enabled greater numbers of non-experts to use spatially-referenced information.^{Footnote 15} In addition, geographic information systems software has expanded to include hundreds of tools to integrate different kinds of information.^{Footnote 16} Given the proliferation of use of geospatial data and enabling technologies, evidence suggests that there is a need for standardization for efficient production and use. Stakeholder survey respondents confirmed standards development as a priority (75%) Without NRCan coordination at a national level, internal and external interviewees expected that there would be competing standards and less cooperation within the sector, leading to data that is less accessible, less interoperable, and ultimately less useful for informing decisions. Internationally, the Open Geospatial Consortium (OGC), which GeoConnections contributes to, develops standards for the global geospatial community, through the work of government, commercial organizations non-governmental organizations, academics and research organizations.^{Footnote 17} This speaks to the international recognition of the need for a centralized, standards-based approach to geospatial data and technologies. Within Canada, the federal government and all but two provinces and territories recently signed the 2014 Geomatics Accord, signalling their respective interest, will and commitment to cooperate in geomatics initiatives of mutual interest.^{Footnote 18} This suggests a broad national recognition of the need for a standardized approach to geomatics in Canada.

Finding: Evidence suggests that the geomatics and geospatial community is fragmented. For this reason, GeoConnections’ Phase III focus on governance responds to the need for leadership and a coordinated approach to developing and implementing the CGDI.

Several provincial/territorial and industry interviewees expressed concern over the level of fragmentation within the geomatics and geospatial data community. As well, 58% of surveyed stakeholders rated as weak the coordination across government, academe, and industry on geomatics and geospatial data issues. Interview and survey evidence suggests that GeoConnections plays an important and ongoing leadership role in supporting stakeholder involvement in governance to enable development of data and technical standards, given the recognized sectoral fragmentation. About 70%of surveyed stakeholders identified the development and maintenance of governance mechanisms to build consensus on standards, policies and technologies as a high priority. Interviewees confirmed that the governance bodies (FCGEO, CCOG, and CGCRT) established by Geoconnections provided common space for various groups representing the geospatial community to discuss spatial data infrastructure issues, and that this has been critical to improving coordination. Internal and external interviewees agreed that GeoConnections fills a need for leadership, that provinces and industry will follow if GeoConnections leads, and that Canada was at risk of falling behind other nations without this program.  

GeoConnections’ focus on improving sector coordination and governance are consistent with efforts in other jurisdictions.  The United States and the European Union have put in place leadership bodies to coordinate spatial data initiatives. In the United States, the Federal Geographic Data Committee was established as an interagency coordinating body to promote the coordinated use, sharing and dissemination of geospatial data^{Footnote 19} while, in the European Union, it was the European Parliament and the Council of the European Union that led the development of a European spatial data infrastructure.^{Footnote 20} Similarly, Canada’s Treasury Board gave NRCan’s GeoConnections Program the mandate to lead the development of the CGDI, by coordinating efforts by governments at all levels, as well as the private sector, non-governmental organizations and academia.^{Footnote 21}

Finding: Overall, evidence suggests that the program has a strong understanding of the issues of the sector. Emerging identified needs include communication with broader geospatial user communities, and technological support for geospatial data use by non-experts. It is recognized that program resources are stretched

Industry and provincial/territorial interviewees were generally impressed with the level of understanding the program staff have of geospatial issues and with their role in  consultation and engagement in the development of GeoConnections throughout all phases. Nearly all interviewees identified an emerging need to communicate the benefits of geospatial data use to the broader user communities, such as oil and gas and forestry, and non-traditional groups such as health care providers, information technology companies and the insurance industry.  Several identified a possible role for NRCan to serve as a rallying point for the sector, given its reputation and strong mapping tradition. Interviewees also identified an emerging need for developing applications that facilitate the use of geospatial data by non-experts. However, a possible challenge to an expanded mandate may be resourcing: both interviewees and surveyed stakeholders were mostly of the opinion that GeoConnections resources are stretched, especially given the continuously changing technology and policy context in geospatial data.

GeoConnections has had to adapt to the changing technology and policy context in geospatial data over its three phases. In Phase 1 (1999-2004), the program responded to a need articulated by the Canada Council on Geomatics (CCOG) to link and make interoperable federal provincial and territorial geospatial data sets over the internet. The program began development of the CGDI and contributed to the creation of GeoBase which makes available many federal, provincial and territorial publicly owned geospatial data sets with public and private sector users. GeoConnections Phase II (2005 to 2009) focused on supporting community use of geospatial data and policy development that contributed, with provincial and territorial support, to NRCan’s 2007 implementation of an Open Data policy. This also supported Canada’s Open Government (Data) initiative by providing Canadians with access to a large number of geospatial datasets. GeoConnections Phase III (2010 to 2015) focused on data and technical standards that are essential to the functioning of CGDI. The program also sought to establish coordination between traditional geomatics and growing number of geospatial data users and producers in non-traditional sectors (e.g., health care, insurance, etc).

3.2 Expected Result: Appropriate governance bodies formed for building consensus among key stakeholders

Results Expectation	Assumptions & factors tested	Summary of Findings
3.2 Appropriate governance bodies formed for building consensus of key F/P/T, industry, and academic geospatial data actors (CGCRT/GeoAllaince, FCGEO, CCOG)	Stakeholders have will and ability to engage and participate The right stakeholders are involved	Program has put in place and supported appropriate governance bodies Good participation in CCOG, FCGEO, CGCRT, while governance continues to be seen as weak though improved from five years ago. Perceived challenges to engaging major geospatial data users in CGCRT and the traditional geomatics sector in GeoAlliance. GeoAlliance seen as positive step, though concerns raised over short term sustainability Key challenges are l fragmentation within the geospatial data community and availability of representatives to participate

Finding: Governance bodies have been formed, including the recently-launched GeoAlliance, and useful support has been provided in this process by the GeoConnections Secretariat. However, there are concerns that the transition from the NRCan-led CGCRT to the sector-led GeoAlliance may be premature

The overall role of the GeoSecretariat has been to foster cooperation among Canadian governments, industry, academia, professional associations and non-government organizations in the area of geomatics and earth observation.^{Footnote 22} The 2012 CGDI Assessment reported that stakeholders were successfully engaged through structured and formalized networks, put in place by GeoConnections, that form the governance bodies for spatial data infrastructure development. Up to 2015, the governance bodies were CCOG, FCGEO and the CGCRT, all of which were coordinated and supported by the GeoConnections’ GeoSecretariat. The recently-launched GeoAlliance Canada, replacing the CGCRT, reportedly evolved after recognizing a need for an over-arching governance structure, encompassing the entire geomatics and geospatial data community, to take ownership of the Pan Canadian Geomatics Strategy.^{Footnote 23}

Evidence suggests satisfaction with leadership provided by GeoConnections in establishing the governance bodies, although there are concerns about sector readiness for the transition to GeoAlliance Canada. Most interviewees familiar with the CGCRT said that it would not have been successfully convened without GeoConnections. The CGCRT has been cited as an example of Open Government by establishing a collaborative, formal relationship between state and non-state actors enabling the identification of shared interests and goals.^{Footnote 24} Interviewees who participated in FCGEO and CCOG viewed those bodies positively as well. All provincial and territorial interviewees indicated that CCOG supports efficient implementation of CGDI by facilitating sharing of information and discussing problems related to spatial data infrastructure. Some interviewees however, suggested that there could be better information sharing from CCOG to working level members and those outside the committee.

The transition from GeoConnections-led CGCRT to the sector-led GeoAlliance Canada is described in the literature as “second generation” governance.^{Footnote 25} This is in line with the European Union where there is a broad umbrella organization including private and public sector representation of geospatial information organizations.^{Footnote 26} However, several interviewees suggest that more time is needed to build the sectoral community before passing off leadership to it via GeoAlliance Canada.  In addition, several provincial interviewees expressed concern that the move to GeoAlliance Canada, which is not coordinated by NRCan, could be incorrectly perceived as  signaling a lack of federal interest, and that this increased the challenge to obtaining buy-in to by some members of the geospatial community. National coordination of spatial data infrastructure remains the responsibility of a federal-level committee in the United States.^{Footnote 27}

Finding: In general, the appropriate organizations are participating in CGDI governance, although there are challenges getting buy-in to GeoAlliance from the traditional geomatics sector organizations, and organizations that are significant users of geospatial outputs..

Most interviewees noted that, based on their involvement with CCOG, FCGEO and CGCRT, most of the appropriate organizations were participating in the CGDI governance fora. An area noted as lacking representation was organizations that are significant users of geospatial data and other outputs (e.g. oil and gas sectors, forestry, municipalities) but are not part of the traditional geomatics sector that is involved in the production of geospatial outputs. According to program files and interviewees, these groups have at various times been invited to participate, but declined. Experience of the European Union was similar in the challenge to obtaining local stakeholder involvement in coordinating structures, and this was associated with a lower overall level of satisfaction for this group with coordination efforts.^{Footnote 28}

Almost all interviewees expressed concerns about the more established parts of the geomatics sector community, which is responsible for producing geospatial data and inputs, buying in to GeoAlliance Canada. These geomatics organizations were thought to be less able to see the value in participating in an additional sectoral association beyond their own. Many interviewees suggested that federal seed funding was the key to keeping GeoAlliance alive in the near future. They explained that this would allow GeoAlliance to demonstrate value to the community in order to encourage investment by industry associations and /or their memberships in this body. Interviewees also noted that when federal government is seen as the lead, industry associations are more interested in buying in. Under the GeoAlliance model, NRCan would no longer be the convenor for the group, rather a partner at the table with a far less visible leadership role.

Finding: GeoConnections governance activities to improve sector coordination have had a relatively positive impact. However, sector coordination remains a significant challenge, due in part to the high level of fragmentation and lack of dedicated resources supporting  CGDI development.  

About three-quarters (73%) of surveyed stakeholders respondents indicated that GeoConnections’ contribution to improving coordination on sectoral issues over the last five years was at least somewhat important, including 44% for whom it was highly important. Note that CCOG members were much more likely to say GeoConnections was highly important than either members of CGCRT/GeoAlliance Canada or the FCGEO (60% vs. 23-33%).  That said, sector coordination remains a significant challenge given  that the majority (58%) provided a negative assessment of the overall level of coordination in the sector today.  Half of surveyed stakeholders observed that the level of coordination has improved across the sector in the last five years, though also noting there is considerable room for improvement when it comes to coordination.

Two key challenges to CGDI governance were identified. First, the community is highly fragmented crossing many traditional industry sectors, meaning that the amount of effort required to coordinate is immense (which was part of the rational for implementing GeoAlliance).^{Footnote 29} Concern over the level of fragmentation was also reflected in several interviews across provinces/territories and industry. ., Second, nearly all interviewees explained that participation in collective efforts for CGDI development are undertaken in addition to regular duties rather than through dedicated staff, and that the community is experiencing attrition of its leaders due to retirement. This is a significant risk, especially for GeoAlliance.

3.3 Expected Result: Shared understanding of geospatial priorities among key stakeholders

Results Expectation	Assumptions & factors tested	Summary of Findings
3.3 Shared understanding of geospatial data priorities across FPT, industry, academe (Cdn Geomatics Accord, Pan-Canadian Geomatics Strategy, National Mapping Strategy)	Agreement across sectors and groups on how to proceed Equitable competition for ideas across sectors and groups. Target groups have been exposed to and persuaded by program white papers, webinars, etc.	Agreement on overall geospatial priorities in Pan-Canadian Geomatics Strategy and Geomatics Accord, although there is a lack of agreement on activities to implement the strategy. Program webinars and guidance documents supporting shared understanding were  found to be persuasive, though mainly accessed by traditional geospatial groups

Finding: Stakeholders have demonstrated shared understanding of geospatial priorities through agreement with Pan-Canadian Geomatics Strategy and Geomatics Accord. At the federal level, shared understanding has been demonstrated by the commencement of work on the Federal Geospatial Platform. GeoConnections contributed to this through facilitating discussions among stakeholder governance bodies

There is high level agreement on geospatial priorities within the community, as evidenced by the Pan-Canadian Geomatics Strategy, the Canadian Geomatics Accord and the commencement of work on the Federal Geospatial Platform.

GeoConnections has supported a shared understanding of geospatial priorities by facilitating discussions through CGCRT and CCOG informing the 2014 Pan-Canadian Geomatics Strategy and Action and Implementation Plan and the Canadian Geomatics Accord. The Pan-Canadian Geomatics Strategy was intended to align the geomatics community as a whole around a shared vision,^{Footnote 30} of a Canadian geomatics sector that is engaged in providing reliable geospatial data and information products, technology, services and expertise. ^{Footnote 31} The CGCRT led the development of the strategy, identifying shared interests and involving a very broad and open consultation process that aimed to include a wide range of stakeholders from across the geomatics community.^{Footnote 32}  Interviewees familiar with this process held positive views about its development and felt that it reflects the right priorities, though these are changing quickly.

At the federal level, in 2014 the FCGEO initiated work on the Federal Geospatial Platform, a project which involves agreement between 21 federal departments and agencies.^{Footnote 33} The platform is intended to facilitate a coordinated approach to federal geospatial data management.^{Footnote 34} Interviewed FCGEO members held a common view that this was an important initiative.

The Canadian Geomatics Accord, which has been signed by all provinces and territories except Newfoundland and Labrador and Nunavut (due to resourcing constraints), indicates a willingness to adhere to principles for developing and promoting best practices, standards and policies and the pursuit of efficiencies through common strategies and shared geomatics infrastructure.^{Footnote 35} Interviewees mostly commended GeoConnections on the support provided through its secretariat role and its facilitation of discussion of key geospatial issues.  

Finding: Survey results suggest some differences in focus between data producers and data users

While data users and producers generally agreed on what the priorities should be, stakeholder survey responses also highlight some differences in emphasis. For example, key priorities more likely to be reported by data producers include data accuracy, timeliness and accessibility, whereas the top priority reported by data users was increasing the use of geospatial data to improve productivity.

Finding: Interviewee evidence suggests some lack of agreement over which activities to be undertaken first, who should act and who should pay. 

While there is high level agreement on overall geospatial priorities, interviewees indicated that there is some lack of agreement regarding particular activities to undertake to further these priorities, who should act and who should be investing in the activities.  The Pan-Canadian Geomatics Strategy states that implementation activities supporting the strategy require broad sectoral engagement.^{Footnote 36} However, the reported difficulties with sectoral coordination and obtaining buy-in to the GeoAlliance suggest that this requirement is not being met. 

Finding: Program tools and guidance materials were found to be persuasive, valued and used

GeoConnections prepared a suite of informational tools which, based on a content review, appears to support the development of a shared understanding of priorities through capacity building and promoting the benefits of the CGDI. Between 2011 and 2014, GeoConnections held 32 webinars promoting and communicating the importance of developing, adopting and implementing geospatial standards and operational policies. Webinars covered areas such as open source software, handling volunteered geographic information, data archiving and cloud computing.^{Footnote 37} Most survey respondents reported using at least one GeoConnections resource, with 54% reporting accessing webinars, 52% reporting accessing papers and studies and 34% reporting using best practices guides. Most also found each of these to be at least somewhat helpful.  The output most frequently mentioned by interviewees was the 2015 Economic Values Study. Interviewees saw this study as providing a compelling case that more user groups can benefit from geospatial data, though it was noted that those groups are not likely the ones reading about this or being exposed to other GeoConnections resources. Some interviewees also noted that program policy papers, such as the SDI Guide for the Americas, provide valuable information on what constitutes acceptable geospatial standards and operational policies (the guide was also reportedly used to inform SDI development in other national jurisdictions).

3.4 Expected Result: Common geospatial standards and policies are developed

Results Expectation	Assumptions & factors tested	Summary of Findings
3.4 Common geospatial standards and policies are developed	Standards and policies respond to the needs of the community (data holders, infrastructure)	Program has developed common standards and policies based on geospatial community input Standards and policies work seen as important priorities, although a need is identified for greater focus on metadata standards. Weak stakeholder assessment of program contribution suggests challenges, but also likely based in some cases on low awareness of program efforts

Finding: Common geospatial technical and data standards have been developed, aided by research and effective program participation in international standards fora. Some interviewees sought more systematic consultation on standards  

GeoConnections has supported development of standards and policies by engaging with the Open Geospatial Consortium, providing support for testing of geospatial. GeoConnections contributed to key OGC standards through participation in OGC working groups covering areas such as metadata standards, web map service standards, and catalogue service for the web.^{Footnote 38} The program also contributed to several projects supporting development of test environments for a standard on open web services.^{Footnote 39}. Speaking to its leadership in the area of spatial data infrastructure standards and policies development, GeoConnections led the research and development work for the Spatial Data Infrastructure Manual for the Americas, which provides guidance on the development of spatial data infrastructures for an international audience.^{Footnote 40}  Both NRCan and external interviewees pointed out that work on standardization will be needed for as long as there are new types of data and more technology that incorporates it. According to interviews and documents, the fact that G&C funding for demonstrations and implementations of new technical applications required compliance with CGDI standards^{Footnote 41} provided opportunities to test the appropriateness of standards in real environments as they were being developed.

GeoConnections  makes a strong contribution to standards development work of the  OGC and the United Nations initiative on Global Geospatial Information Management and this is important for Canadian industry . NRCan participates in OGC and other standards committees and has been reported by those involved at the OGC processes to be a strong contributor with impactful influence at OGC. Note that as of 2015 NRCan is a Planning Committee level member to OGC, an elevated status from its positon in 2010, which speaks to the respect for GeoConnections staff by OGC members. Interviewees involved in this effort noted that  Canada’s involvement in standards setting helps to ensure standards take into account the needs of Canadian governments and businesses within a global context. Interviewees involved in standards noted that GeoConnections’ involvement in testing these standards, through G&C projects, gave NRCan credibility in its efforts to advocate for certain features of standards based on what would work in Canada.  

Geospatial standards development work was based on consultation with stakeholders, although some stakeholders saw room to improve consultations. The 2012 and 2015 CGDI assessments both concluded that Canada has a set of geospatial standards, developed in consultation with CGDI stakeholders, many of which are based on the work of the Open Geospatial Consortium, the International Standards Organization and the U.S. Federal Geographic Data Committee.^{Footnote 42} According to the 2012 CGDI Assessment, the CGDI describes and identifies standards related to (i) geospatial data structure (metadata and data syntax), data content (framework and thematic data) and (ii) services/applications which allow for data and service interoperability (i.e., diverse data sources, services, applications and systems work with each other).^{Footnote 43} Provincial/territorial and industry Interviewees with knowledge of standards and GeoConnections held mainly positive views of the level of consultation in developing these. NRCan interviewees explained that  concerns of industry related to standards development are communicated informally to NRCan officials who can then bring those concerns to the OGC fora.  Similarly, perspectives communicated from CCOG have influenced NRCan positions at OGC, ISO and UN-GGIM.  Some external interviewees felt that the program could be more systematically consultative on standards. 

Finding: Operational policies have been developed based on identified community needs. Evidence suggests further guidance may be needed on intellectual property issues and data stewardship  

Operational policies have been developed based on need, although there are some identified gaps. GeoConnections has undertaken several research projects to obtain feedback on user needs to support operational policy development: CGDI Operational Policies: 2010 Preliminary Needs Assessment Report (GeoConnections), 2011 Report on CGDI Operational Policy Needs Analysis^{Footnote 44} , and 2012 Geospatial Operational Policy Roadmap Research 2012-2015.^{Footnote 45} A review  of the GeoConnections website, as well as the 2012 and 2015 CGDI assessments, shows that the program has developed operational policy guidance on a wide range (at least 15 in the last five years) of pertinent spatial data issues. However, according to the 2012 and 2015 CGDI Assessment exercises, more policy guidance is needed in the areas of data stewardship and data integration. Similarly, interviewees frequently suggested guidance is needed on intellectual property (e.g., licensing, privacy), and treatment of legacy datasets in terms of digitizing and making them discoverable. 

Finding: GeoConnections work on developing standards and operational policies that meet user needs seen as important priorities, although interviewees indicated a need for renewed focus on metadata standards.  A weak overall assessment of program contribution to progress in some cases is likely based on  low awareness of NRCan activities

Significant majorities of surveyed stakeholders considered developing common technical and data standards (75%) as well as operational policies and guidelines (61%) to be important priorities for GeoConnections. In fact, work on technical standards was seen as a priority more often than any other issue. Despite assigning a high priority to these issues, surveyed stakeholders had relatively weak perceptions of GeoConnections contribution to progress on standards and operational policies development, with just 33% rating progress as strong for standards and 27% for operational policies and guidelines. Some interviewees also noted that while metadata was a strength of GeoConnections in earlier phases, there is a need to renew the focus on metadata standards to bring them up to date and make them easier to implement.

While the survey figures do suggest that stakeholders perceive a need for improvement in the development  of  standards and policies that meet user needs, evidence from interviews suggests that the weak assessment may in some cases be explained by : low awareness of program efforts. This is due to the fact that GeoConnections’ contribution to standards development occurs in the background. For example the program may work on committees at OGC to influence a standard that is then adopted by industry. NRCan is not visible outside of that process.

3.5 Expected Result: Common geospatial standards and policies are widely adopted and implemented

Results Expectation	Assumptions & factors tested	Summary of Findings
3.5 Common geospatial standards and policies are widely adopted and implemented	Public and private sector data producers have will and available resources (cost, HR are factors) to implement	Numerous examples of implemented CGDI standards and policies, though full implementation is a challenge. Also, implementation is not systematically monitored. Issues identified with data discovery and implementation of standards, similar to challenges in the US and EU Adoption and implementation impeded by organizational resources, operational priorities, understanding of CGDI, and available geo professionals to implement. Adoption and implementation aided by international standards, geomatics industry consolidation, off the shelf software, and G&C demonstrations

Finding:  Canada’s progress on implementing standards and policies is similar to that of the US and EU where implementation is mandatory (unlike in Canada). There are numerous examples of implemented standards and policies in Canada, though full implementation is a challenge. Survey and interview evidence suggests implementation tends to be modest due to insufficient resources and competing organizational priorities.

Some interviewees suggested that making CGDI implementation mandatory is what is needed for Canada to truly achieve national implementation of CGDI standards and policies. However, a review of assessments of the state of spatial data infrastructure in both the United States and the European Union suggests they are experiencing similar implementation challenges to those experienced in Canada. In particular, a 2015 report by the Government Accountability Office found that, twenty years after guidance was issued to establish the American spatial data infrastructure, the U.S. federal government had not yet fully implemented it.^{Footnote 46}  The 2014 Mid-term Evaluation of the European Union’s spatial data infrastructure found inconsistent implementation across the EU and the adoption national laws requiring implementation of the spatial data infrastructure in each country had not been completed several years after the deadline in seven countries.^{Footnote 47}

Within the Canadian context, 70% of surveyed stakeholders indicated that their respective organizations had at least somewhat implemented CGDI technical and data standards, including 43% mostly implemented, while 54% had at least somewhat implemented operational policies, including 26% mostly implemented.^{Footnote 48} This is consistent with interview data where no provincial and territorial interviewees claimed that their SDIs were fully CGDI compliant. Many said that they were as compliant as their resources and current operational contexts would allow, with some explaining that lack of full compliance was due to complexity (but not quality) of the standard. Interviews from provincial governments, federal government and industry, all gave examples where CGDI compliant standards and operational policies have been implemented in their organizations.

Consider the following examples of implemented CGDI standards and operational policies:

• GeoBase: A federal, provincial and territorial government initiative that is overseen by CCOG. It is undertaken to ensure the provision of, and unrestricted access to, a common, up-to-date and maintained base of quality framework or key geospatial data for Canada. According to documents and interview data, GeoBase is enabled by applying the standards and policies suggested by CGDI, and are considered a significant achievement in open data.
• Provincial Government SDIs: the 2015 CGDI Assessment found that, in addition to GeoBase, the provinces and territories each have their own SDI’s, which evaluation interviewees say are largely based on CGDI (e.g., GeoBC, GeoAlberta, GeoNT, GeoYK) and consider that to be a major achievement of GeoConnections. In fact, all province and territory interviewees involved in their jurisdictions’ SDI efforts attributed those developments to GeoConnections’ work on CGDI.
• NRCan data holdings: The 2015 CGDI Assessment and evaluation interviewees pointed to NRCan data holdings that are compliant with CGDI standards and policies as well. It was noted that NRCan data has the single largest holding of federal geospatial data that it makes available to the public, and at least some of this is CGDI compliant.
• Federal Geospatial Platform: The upcoming launch of the FGP was noted as a major example of adopted CGDI standards and policies by several internal and external interviewees. A similar conclusion was reached in the 2015 CGDI Assessment, noting that the FGP leveraged existing CGDI guidance materials covering policy classification and identification and inventory processes to enable the FGP to understand its policy landscape, classify and inventory data holdings.
• Arctic SDI: The 2015 CGDI Assessment notes that GeoConnections’ operational policy guidance has been incorporated into the Arctic SDI framework document. According to external evaluation interviews, GeoConnections’ staff, operational policy guidance and CGDI standards were highly important in the development of Arctic SDI. Representatives from the Arctic Council's Conservation of Arctic Flora and Fauna reported NRCan providing outstanding technical advice on how to structure data using standards to make it available to users.

Finding:  While there are positive indications from interviews and the CGDI assessment exercises, it is difficult to know the scale of implementation as it is not systematically monitored

As noted above, there are many examples of implementation of CGDI standards and policies. However, both the 2012 and 2015 CGDI assessments found that there are no mechanisms for systematically monitoring technical and data standards implementation.^{Footnote 48 Footnote 49} The reported exception is the Treasury Board standard, against which federal departments are assessed for compliance.^{Footnote 50 Footnote 51} GeoConnections has put in place the CGDI Operational Policy Activities Project Plan 2012-2015 that proposes the following indicators of adoption of operational policies in particular: ^{Footnote 52}

• The inclusion of common operational policies  in the renewed Geomatics Accord;
• The adoption of a core suite of geospatial operational policies by the FCGEO; and
• The adoption and publication of geospatial operational policy guidance by TBS.

However, the ability to track progress using these indicators is mixed.  In particular, the 2014 Geomatics Accord only commits signatories to promote and contribute to the development of standards and policies to support geospatial information sharing and integration, with no indication of reporting on progress on their implementation..^{Footnote 53}

Finding: Metadata standards which support data discovery are seen as an ongoing and important effort, but  some level of dissatisfaction has been expressed regarding data discovery. It is not clear whether the issue is with the standards themselves or their implementation. A range of metadata challenges reportedly exist in the US and EU as well

The importance of metadata is underscored by interviewee evidence suggesting that the ability to have federated discovery of data across Canada through metadata is the most successful piece of CGDI. At the same time, the availability of metadata to help organizations discover who has what data holdings has been described as a key issue for provincial/territorial government and private sector interviewees.

Interviewees raised concerns about the discoverability of data; however, it is not clear whether the cause was the metadata standards to support data discoverability or a lack of their implementation. User feedback suggests some dissatisfaction with data discoverability on GeoGratis, which is a web platform that provides access to a wide collection of Canadian geospatial data as well as metadata.^{Footnote 54} A 2014 report summarizing ad-hoc user feedback on GeoGratis, indicates that the site contains insufficient information about the geospatial datasets (i.e., there is a lack of metadata, indexing, and a mechanism to browse the datasets). Users frequently indicated that they preferred the previous iteration of metadata offerings through the Discovery Portal.^{Footnote 55} Interviewees also noted difficulties finding data through GeoGratis.

Metadata challenges are not unique to Canada. The experiences of both United States and the European Union indicate that there are many challenges for metadata availability, completeness and usability as well as concerns about the quality of the standards themselves in terms of being outdated or excessively complex. According to a 2015 report by the US Government Accountability Office, only a quarter of American federal departments had made their metadata available on the United States geospatial platform,^{Footnote 56} often due to technical difficulties when loading metadata onto the platform.^{Footnote 57} The report also cited concerns over outdated metadata standards, resulting in difficulties searching and discovering geospatial data.^{Footnote 58} In the European Union, only one-third of countries had created metadata for more than 90% of their key holdings by the deadline for 2014 INSPIRE SDI Assessment,^{Footnote 59} and independent verification found inconsistency in naming conventions where metadata were provided,^{Footnote 60} resulting in difficulties discovering geospatial data. Public sector data providers reportedly criticized the complexity of metadata standards and the lack of tools and time to create it.^{Footnote 61}

Finding: Challenges with adopting and implementing common CGDI standards and policies include cost, understanding of CGDI, availability of geo professionals, and organizational context (especially for policies)

A number of factors have been found to impede the adoption and implementation of standards and operational policies:

Understanding of CGDI:  While knowing that the CGDI exists, some interviewees representing industry or the provinces demonstrated varied levels of understanding of the various parts of the CGDI and the process involved in its ongoing development.

Communication about G&C projects and the CGDI in general: Most interviewees were aware of the support that GeoConnections provides to developing testbeds for standards, but some expressed concerns that more could be done to keep the community informed of those project results and their application to their organizations. They sought more analysis on how this information can help them in their own SDI efforts. Several suggested making these projects an agenda item that NRCan could bring to  venues such as CCOG and GeoAlliance.  Some interviewees representing industry and the provinces said that adoption and implementation of the CGDI could be improved through more aggressive communications and marketing activities related to the CGDI generally and its importance. Other interviewees suggested that communications activities could be part of the GeoAlliance mandate.

Resources and operational context: Survey respondents who provided explanations for not having fully implemented CGDI standards and operational policies tended to focus on their organizational context, noting either that they did not have the resources to make full implementation a priority or that the standards and polices were not all applicable to their operations. Interviewees who had not fully adopted CGDI explained that they had done so on purpose, selecting standards and policies that work best for them (the increasing reliance on OGC based standards seen as very positive). However, interviewees are working on efforts to implement common standards closest to data source, but they reported first needing to work on organizational cultures that do not easily consider external use of their data.

Availability of Geo professionals: Interviewees across provinces/territories, industry and academe noted that attracting professionals to work in the geospatial fields was a challenge. This has implications for ability of organizations to implement and update standards and policies as they change.

Finding: Adoption and implementation of common CGDI standards and policies aided by pursuing international standards for CGDI, leveraging G&C funded demonstrations, geomatics industry consolidation, and increased use of off the shelf software

Evidence from documents and interviews suggest that adoption and implementation of CGDI standards has been greatly aided by four factors, of which the first two are directly within control of the program.

• First, the decision to pursue international standards in CGDI rather than exclusively Canadian standards was seen by most interviewees as an important decision. This means that Canadian public and private sectors will have an interest in adopting standards that make their products more easily used outside Canada – a key efficiency for both business development and data sharing with other jurisdictions. According to many public and private sector interviewees, most commercial Canadian companies are using OGC global standards in which NRCan has had significant influence. This is consistent with the approaches taken in the US and EU.
• Second, NRCan has been able to leverage the results of demonstrations and testbed projects it has enabled through G&C. For example, NRCan has supported demonstration projects with key Canadian Geomatics companies all of whom have senior representation at OGC and are seen as leading influential experts in those committees. Several interviewees across groups noted that these projects give NRCan credible basis upon which to negotiate standards with the international community that will work in a Canadian context. Documentation also confirms that compliance with CGDI standards is a requirement to receive G&C funding.^{Footnote 62}
• Third, interviewees reported that the Canadian geomatics sector is increasingly (since the 2008 recession) characterized by consolidation. The resulting smaller number of larger firms reportedly increased the use of common standards. The smaller firms, using common standards, are working at the forefront of new technologies which is what makes them attractive to potential buyers. For example, there are a number of small Unmanned aerial vehicle companies in Canada right now, which interviewees expect will either grow or be bought by larger firms. A few interviewees explained that as data in Canada becomes more and more open, you can expect to see a greater growth in the industry producing value added geospatial data products (e.g., Google, ESRI etc) t and growth in the number of  smaller geospatial businesses taking advantage of the opportunities presented to innovate.
• Fourth, related to the third, is the increased use of off the shelf software. Interviewees reported that more and more organizations, are turning to proprietary off the shelf software solutions (e.g., ESRI, AutoCad) rather than developing their own in house geomatics solutions. Industry and provincial/territorial interviewees report that these were normally compatible with CGDI. This, combined with the consolidation of the geomatics sectors in Canada, has led to a de-facto widespread adoption of common geospatial standards. NRCan and expert interviewees see the widespread use of proprietary off the shelf software solutions as consistent with CGDI because the major firms in Canada are all using OGC standards that CGDI supports.

3.6 Expected Result: Geospatial data is interoperable and available online for Canadians to access enabled by technology/applications

Results Expectation	Assumptions & factors tested	Summary Observations
3.6 Geospatial data is interoperable and available online for Canadians using technological innovations	Public and private organizations launch services/platforms that combine geospatial information from various sources Public and private sector investment climate is conducive to developing innovative geospatial data products Data collection organizations have been persuaded of the value of collecting data using common standards	Good progress making geospatial data available online. Many examples of portals and platforms identified Program has enabled investment in technology/application development through G&C projects and work on standards. Impact of G&C could be improved with communication Progress on data interoperability is in early stages. Challenges to overcome include competing with organizational priorities, limits imposed by data licensing, governance of shared data insufficient bandwidth and treatment of legacy datasets.

Finding: Documents, interviews confirm many examples of online geospatial data portals, platforms and web-services making public data available in Canada using CGDI standards, policies, and technologies

Technology tools, such as portals, platforms and web services, are essential to the provision of geospatial data over the internet. Documents and interviews confirmed many examples of geospatial data that is interoperable and made available online using technologies. Both the 2012 and 2015 CGDI assessments found many publicly-supported portals covering various federal and provincial geospatial data holdings, made accessible through the GeoConnections Discovery Portal (GDP) which, up until July 2015, has been a prime mechanism for accessing geospatial data and services within the CGDI. Interviewees confirmed that these examples are all enabled through CGDI. Consider the following:

• Regional SDI: Several provincial/territorial government interviewees reported having centrally coordinated, web-enabled data platforms based on CGDI. They also noted that a number of municipal governments are moving in that direction as well.
• GeoBase: Nearly all interviewees noted GeoBase as a major accomplishment of CCOG efforts and a flagship of publicly available framework/key data that is interoperable with other organizations in Canada. Several industry and province/territory interviewees explained that they use GeoBase data to varying degrees and that they rely on it. GeoBase is seen as an ongoing and important effort by all interviewed members of CCOG, and some members of industry.
• GeoGratis: A web platform that consolidates free, unrestricted access to a wide collection of CGDI-compliant Canadian geospatial data, maps, Images and publications from across the Canadian public sector. The platform also provides support for application developers. 
• GeoDiscovery Portal: The Portal has provided a metadata catalogue and constitutes the prime data discovery and access component to CGDI. Until July 2015, it was the main point of entry to Canadian geospatial data and related services through links to national, regional, thematic and international portals. Built using Compusult’s Web Enterprise Suite based on open standards, the Portal enables discovery, access, and downloading of over 10,000 databases.^{Footnote 63}
• Canada Land Cover Data project: A comprehensive, integrated, interoperable land cover dataset was developed from data provided by various provincial and territorial governments, Agriculture and Agri-Food Canada, NRCan’s Canadian Forest Service, and Canada Centre for Remote Sensing CCRS using CGDI standards and principles. The resulting national land cover data product is also interoperable with other national geospatial data that use CGDI standards including the National Road Network and the National Hydro Network.

Geospatial web services are software designed to support interchange of geographic data and services between machines/organizations over the internet. NRCan’s Atlas of Canada, National Forest Information System portal, and the Groundwater Information Network have each been supported by GeoConnections and use open web services standards conforming to CGDI.  

Finding: GeoConnections has enabled public and private investment in geospatial technologies/applications through G&C support of technology-related projects and work on standards. However, interviewees suggested the impact of this work could be improved with communication of project results

In addition to the above examples, it is important to note that documents show, confirmed by interviewees, that the G&C contributions of the GeoConnections program has enabled public and private investment in geospatial technologies. For example:

• Multi-Agency Situational Awareness System (MASAS): GeoConnections provided $200,000 of the $419,882 in total project costs to build MASAS (which is available for uptake by any province/territory) based on CGDI standards and operational policies. MASAS aggregates incident-relevant information from diverse sources into a consolidated view that is available to partners via a centralized hub (i.e., it is published once for multiple uses). ^{Footnote 64} This project has contributed to increased availability of open geospatial information (e.g., on flooding, weather, earthquake locations, wildfire hotspots) where it has been adopted (e.g., New Brunswick). 
• The Geofoundation Exchange Prototype Project : The GFX project demonstrated the implementation of international geospatial technical and data standards (upon which CGDI standards are based), implementing a prototype for an open data exchange infrastructure intended to support a government-centred national base map.^{Footnote 65} The $1.8 million project, which received $500K from GeoConnections, was led by ESRI Canada Limited. 
• Canada Land Cover Data project: A comprehensive, integrated, interoperable land cover dataset was developed from data provided by AAFC, CFS, and CCRS using CGDI standards and principles.
• Water and Environment Hub: GeoConnections provided $140K in support (62% of total project cost) for the expansion and enhancement of the capabilities of the Water and Environmental Hub, which is a platform that aggregates global water and environmental data and then distributes it support the development of applications. ^{Footnote 66} The enhancements consisted of enabling more types of data to be accessed and shared in more ways and increasing the supported formats, facilitating application development.^{Footnote 67} 

In fact, 46% of surveyed stakeholders reported at least partially implementing CGDI technology and tools (including 17% mostly implemented).^{Footnote 68} Despite the successes of these projects, interviewees were critical of the lack of sharing or applicability of the results. Unless specifically involved in a project, many interviewees indicated that they were unaware of specific technology and applications that NRCan- supported projects generated.  A few explained that they did not leverage the work of GeoConnections on technology demonstration because the results were not promoted to the community.

Evidence from documents as well as interviewees demonstrated that the increased standardization in geospatial that has been occurring over the last 5 to 10 years, in part due to program efforts, has made it easier for technology and application development. Interviewees explained that the pursuit of increased standardization means that developers can know with some certainty that they can develop innovative data products that can be used in multiple markets across Canada and in other parts of the world. 

Finding: Availability of interoperable web-based geospatial data increased in the last five years, but community attribution to GeoConnections is weak (likely due to confusion over NRCan role)

Both interviewees and survey respondents reported that the availability of interoperable geospatial data online has increased immensely over the last five years. In total, 84% of surveyed stakeholders said that the availability of interoperable web-based geospatial data from their organization has increased a little (24%) or a lot (60%) in the last five years (no-one said there has been a decrease). Having said that, just 49% thought that GeoConnections and the CGDI has been at least somewhat important to their organization’s ability to provide interoperable web-based data (including 20% who said definitely important).  This weak assessment of the program’s contribution is likely due to some confusion over NRCan’s role with respect to development of geospatial standards and data interoperability, given that much of this occurs in the background. Those who said the program was unimportant (42%) gave a variety of explanations, most of which were either unrelated to interoperability or demonstrated a lack of awareness of program activities on this issue.  

Finding: Organizations are increasingly contributing to shared data, available online, while interoperability is still in early stages. However, evidence suggests challenges to overcome including competing organizations priorities, limits imposed by data licensing, governance of shared data, insufficient bandwidth and treatment of legacy datasets.

Nearly all internal and external interviewees explained that while progress on data availability has been good, interoperability is really only at the beginning right now and that there are still many obstacles to overcome. In fact, several internal and external interviewees suggested that true interoperability is likely too high a standard to meet, and that a hybrid model where data is available online through web services and other data is at least accessible though not fully interoperable is the most likely end state.

The following challenges with making data available online and  interoperable have been identified:

Competing with organizational priorities: According to interviewees, data sharing within jurisdictions has improved considerably, but existing inter-jurisdictional data sharing remains a challenge, reducing wide data availability. Some interviewees noted that, despite all of the work on standards, organizations still prioritize their political or jurisdictional needs ahead of collaborative enterprises like CGDI, which makes widespread availability of data difficult to achieve.

Data Licensing: Several industry interviewees noted that most municipalities and some provinces do not have open data policies, or do but with restrictive licensing. Both of these factors impede access to data which is a key outcome of GeoConnections. An additional result of these practices is, according to industry interviewees, the reproduction of the same data by multiple organizations that require the same data but cannot share it.  Such legal and policy impediments to data sharing may in some cases be instituted for the protection of personal privacy or they may be the unintended consequences of unrelated government policies.

Governance of shared data and systems: Several program case studies illustrated concerns over how to govern data. For example, in the MASAS project, an otherwise successful initiative experienced significant issues with maintaining data sharing agreements and commitment to long term maintenance of the system. Interviewed province and territory and several experts noted that data governance was an area where more progress was needed. They noted that the needed technology solutions will not matter if the institutional arrangements necessary for data sharing and system maintenances are not in place. Several province/territory interviewees noted that they would like to see guidance from the federal government on how best to do this.

Challenges implementing semantic standardization for data interoperability: Several program case studies highlighted challenges with demonstration projects that related to translating data from various sources. For example the Canada Land Cover Data Project reportedly encountered some challenges in processing data for consolidation, and the GFX project had challenges with reconciling the varied names of data points. Not surprisingly, interviewees described true interoperability as requiring standardization at the semantic level – something all acknowledged is difficult to achieve because of the coordination required. For this reason, much of the standardization that has occurred so far has been one level above this at the point of translating data points into common terms, as opposed to standardizing the points themselves. Some interviewees suggested that standardization at the semantic level might be too high a standard to meet.

Insufficient bandwidth and data processing over the web: Evidence from interviews and case studies of several GeoConnections G&C projects showed that availability of bandwidth is a key challenge for achieving web-based interoperability of geospatial data. For example, in GFX, MASAS and Hydrologix project reports each showed that the ability to exchange data over the internet through these platforms was constrained by bandwidth, and related processing and memory requirements. Industry interviewees noted that as bandwidth increases and costs come down, we can expect to see greater use of data available through web-service, which is difficult to achieve right now.

Deficiencies in framework Data Quality:  Internal and external interviewees explained that there has been underinvestment in key geospatial data, or framework data, for many years in Canada. While there is much data being made available through GeoBase and GeoGratis, there are growing concerns with its currency.  In some cases, mapping data is said to be out of date by 40 years. This is also a problem in the US where challenges with framework data, including its condition for use and currency, have been noted in a report card on framework data.^{Footnote 69} Many interviewees in both industry and government suggested that both a national imaging strategy and a national mapping strategy (some said for base layers and elevation levels) would be very important for finding cost effective ways to update and maintain framework data. NRCan, most noted, would be the natural lead on such an effort. Other federal and provincial/territorial interviewees noted that such a strategy would also limit the ongoing duplication of data collection due to proprietary or not as yet shared public data sets, and duplication of data collection between federal and provincial governments, provincial and municipal governments.

Challenges with treatment of legacy datasets. The 2015 CGDI assessment found that, in general, there is a gap regarding the uncertainty of updating existing data to align with future technological requirements.^{Footnote 70} . Academic, provincial and federal interviewees noted concerns about this issue, explaining that while treatment of and availability of data collected in the present is greatly improved over five years ago, historical data must still be translated into CGDI compatible formats and would come at great financial and human effort costs. Several suggested a need to prioritize what kinds of data this should be done for.  

3.7 Expected Result:  Geospatial data is accessed by decision-makers (or their staff) and used to make better decisions on social, economic and environmental priorities  

Results Expectation	Assumptions & factors tested	Summary of Findings
3.7 Geospatial data is accessed by decision-makers (or their staff), to make better decisions on social, economic and environmental priorities.	Decision makers understand the value of and are attracted to using geospatial data to make better decisions Geospatial information reaches Canadian policy-makers, industry, and Canadians.	Evidence that demand for geospatial data has increased considerably, though measuring use is a challenge Corroborated link between program activities and data use. Numerous examples of CGDI facilitated data use Many factors found to impede use of data including the degree to which senior decision-makers value it, its quality, accuracy and currency, silo-cultures, and lack of technology/tools for using data.

Finding: There is evidence that demand for geospatial data has risen considerably in the last five years, although measuring actual use of data once retrieved is a challenge

Eighty-nine percent of surveyed stakeholders whose organizations produce geospatial data reported an increase in demand for the data they produce. Those whose organizations use such data also reported a similar increase (87%). Moreover, majorities described this increase as a lot (67% and 55% respectively). For the most part, interviewees reported that geospatial data are being used to inform decisions. Many explained that this has been a trend for more than a decade. While few interviewees could specifically attribute this increase to the work of GeoConnections, most felt that it was likely that GeoConnections had contributed to this increase because of its work behind the scenes on CGDI standards that are being taken up within industry and public sector, and enabling initiatives such as GeoBase, and GeoGratis. That said, evidence from interviews and documents shows there is no systematic measurement of use of geospatial data in Canada. The 2012 CGDI Assessment concluded that there is no composite Canadian picture and analysis of the usage of geospatial data across the various portals that are available.^{Footnote 71} 

Finding: Interview and document evidence show numerous examples of publicly available data based on CGDI standards and operational policies being used by decision-makers, though there is widespread agreement that this is only a small portion of what can be achieved

Evidence from evaluation interviews, program case studies, and both CGDI assessment exercises in 2012 and 2015 found numerous examples of available data, enabled by CGDI, being accessed and used to inform decisions. Some, of many examples, are:

• The GeoConnections Discovery Portal, which provided access to geospatial infrastructure tools and services including a GSC Geoscience Repository and the National Land and Water Information Service had 15,775 visits in 2012-13, although tracking only began in the second quarter of the reference period. (2015 CGDI, p. 56). The portal closed down in July 2015, and was integrated into GeoGratis.
• GeoGratis: In April 2013, NRCan launched a revised consolidated GeoGratis platform with a single point of access for all CCMEO maps, geospatial data, remotely sensed data and research publications.^{Footnote 72} In addition, it provides tools for viewing and downloading data and providing support for developers in the form of application programming interfaces.^{Footnote 73} Overall downloads of data from GeoGratis averaged 3.9 million from 2011-12 to 2014-15.^{Footnote 74}  Note that GeoConnections is not responsible for managing GeoGratis.
• Provincial/territorial data portals: Regional government interviewees all noted that their SDI related data holdings have become increasingly centralized over the last five years, with much of the work starting about ten years ago. These centralized data holdings are made available to the whole of provincial and territorial government. For example, DataBC hosts over 400 applications accessing their data to use in analyses for line departments, and the Ontario Geospatial Data Exchange (OGDE) has over 600 public sector organizations using this centralized data source typically for land registration, property tax and emergency response. Interviewee evidence shows that while there are still issues to overcome, government analysts now have access to all of the data in the repository integrated into a common base map.
• Coordinating fire response: Interviewees reported that governments of BC and Alberta in 2014 used web-enabled geospatial data to coordinate their responses to forest fires along their borders. Managed by NRCan-CFS through the CIFFC, The Canadian Wildland Fire Information System (CWFIS) uses OGC standards consistent with CGDI for data sharing and interoperability to enable data to be easily collected from distributed systems and shared. This combination of data is manipulated on the web into dynamic mapping products that aid decisions on fire management (a variety of users can track and forecast fire activity, fire danger rating, determine forest values at risk across jurisdictions in Canada). Note that the base map data used for this application comes from CGDI enabled data available through GeoGratis.^{Footnote 75}
• Directing infrastructure investment: Alberta officials used location enabled data through the Alberta SDI to prioritize and direct infrastructure rebuilding efforts during the 2013 floods based on strategic need based on population flows. This reportedly allowed for much faster and more efficient deployment of rebuilding efforts. In another example, Alberta’s Land Use Framework for planning oil sands development uses CGDI enabled data to facilitate discussions on how much development is good for Alberta.
• According to the North American Environmental Atlas case study describing the project to integrate North American environmental data into a has reportedly produced data which is being used by government senior decision-makers.
• Companies using data in operational decisions: Interviewees described major North American companies using data made available through CGDI, combined with their own data, to inform decisions on aspects of their business such as shipping routes, when and where to build facilities.

Findings: GeoConnections can be said to have meaningfully contributed to observed increases in use of geospatial data by decision-makers.

In the preceding stages of the results chain analysis, Geoconnections has been shown to be uniquely facilitating use of geospatial data in Canada through its development of the standards, policies, and partnership arrangements of the CGDI. Evidence at each stage of the results chain has noted many challenges, though there is no shortage of indications that the work of the program is having a positive impact. Just over half (51%) of surveyed stakeholders agreed that the GeoConnections program is helping to improve the use of geospatial data by decision-makers, while just 14% disagreed with this statement.

Findings: Many factors found to impede use of geospatial data including the degree to which senior decision-makers value it, its quality, accuracy and currency, silo-cultures, and lack of facilitating technologies supporting data search, discovery and use.

Evidence from interviews and documents clearly indicate that much more opportunity exists to increase use of geospatial data in decision-making, and leverage the associated benefits. Just 20% of surveyed stakeholders who thought increasing use of geospatial data should be a priority of the geospatial community believed that they are making good progress on this priority. Additionally, 88% of respondents identified barriers to use of geospatial data within their own organizations.

There is a lack of awareness of benefits of geospatial data: The degree to which decision-makers are aware of and understand the benefits of geospatial information for their decisions is a key factor that influences use. In total, 69% of surveyed stakeholders thought that increasing awareness of the benefits of using geospatial data/tools should be a high priority. However, just 30% perceived program performance on this to be strong. Many internal and external interviewees across groups explained that while the use of geospatial in decisions is getting better, most decision makers do not know what geomatics is or understand the decision-making power it has for their organizations. Interviewees explained that if that could be changed, it would not only translate into greater use of geospatial information but would also translate into investment in data infrastructure projects like CGDI and on data development/updating. Some looked to NRCan to play a role in fostering this understanding, others saw this as something GeoAlliance could be responsible for.

There is a need for more technology/applications that integrate data into workflows for non-experts: Web applications are software accessed through a user interface that integrate and combine geospatial data with other data and services to create new value-added data and services. Many interviewees (internal and external) noted that the greatest gains in the use of geospatial data will be made once technology/applications are used to integrate the available data into the workflows of analysts that are not experts in GIS and geomatics. CGDI has been described by interviewees as laying the foundational work that will enable that transformation. All provincial/territorial government and some industry interviewees expressed concerns over the use of easily available Google geospatial data in decisions. They noted that it is so easy and user friendly, that decision-makers are not either aware of superior data available from the provinces own data or unwilling to wait for it. Some suggested that public sector data services must continue to modernize to meet that need. Similarly, a review of program case study documents and other analyses showed that technology/applications required to easily integrate multiple data flows into decision-making processes are needed to maximize the potential for use. 

The GeoConnections program has provided resources for application development. These include the GeoGratis program’ application programming interface for developers (updated in 2013), GDP documents supporting applications developers (i.e., API Guide, Portlet Development, Iframes, Application Sharing). The 2015 CGDI assessment noted the replacement of “A Developer’s Guide to the CGDI: Developing and Publishing Geographic Information, Data and Associated Services” with the current suite of information and notes some uncertainty as to the extent to which this change has left a gap in this area.^{Footnote 76} Similarly, the 2012 CGDI assessment noted that for data use to improve, related technology/applications need to be easier to use. According to a program case study, Parks Canada has been using and sharing data using CGDI standards. The case study reported that among the enabling factors was engaging the organization in the integration of the geo-technology to demonstrate its usefulness thereby increasing its acceptance.^{Footnote 77}

The GeoConnections program has supported new technology/application development intended to put data more readily in the hands of analysts. Review of GeoConnections project reports for the Water and Environmental Hub^{Footnote 78} and the GeoFoundation Exchange Prototype (GFX)^{Footnote 79} show that these applications encountered the following challenges: limited public data availability via web services, difficulties integrating data when input formats or file names changed, variation in the naming of data points, operational security issues, and higher than expected use of the hub by those seeking data discovery as opposed to the more advanced functions.

There is a need to confirm the accuracy and currency of data: Several interviewees noted that a key role for government should be vetting and attesting to the authoritativeness of data and saw this as increasingly important in a world where (virtually) anyone can generate geospatial data. In fact,  the top barriers to data use mentioned by 23% of surveyed stakeholders were data inaccuracy and outdatedness. Project level case studies also noted challenges with the accuracy and currency of available data.

There is a need to break down decision-making silos - Interviewees and documents show that because decisions in public sector organizations are made along departmental lines, the organizational structures do not easily support a more integrated look at the available data. Interviewees explained that further thought to decision-governance models was needed. This can be seen as well in the funding sustainability challenges described in the Multi-Agency Situational Awareness System), Water and Environmental Hub and GeoFoundation Exchange case studies where governance for maintaining these services was shown to be an issue.

Data search ability and discovery: While many interviewees applauded the efforts at making federal and provincial data available through GeoBase and GeoGratis, they also emphasize challenges with data discovery. They explained that finding data through these portals was difficult, complicated and at times unwieldy, especially for non-geo professionals. Along similar lines, a 2013 survey of GeoGratis users notes that while 74% found at least some of what they were looking for, just 45% were satisfied with their search experience.^{Footnote 80}

3.8 Expected Result: Benefits to Canadians

Results Expectation	Assumptions & factors tested	Summary of Findings
3.8 Net benefits to Canadians (e.g., improved competitiveness of Canadian firms, safety, infrastructure planning, service delivery and development)	The most optimal choices are made with the available information Improved decisions result in positive outcomes	Economic Value Study, case studies and interviews all show examples of benefits arising from geospatial data use in decisions Use of geospatial outputs provided economic benefits, as well as less-measurable non-economic benefits, in addition to a positive return on investment.

Finding: Interview and case study evidence provides many examples of benefits resulting from the use of geospatial outputs in decision-making

Highlights of just a few examples of benefits include the following:

• The Yukon Government provides location-based data, through an online coordinated information network enabling simultaneous consideration of multiple factors and potential resource project impacts, to support decisions on water licensing.^{Footnote 81} Project reporting and evidence from interviewees indicated that this initiative has moved the Yukon into a stronger and more informed position regarding resource management.
• RSA Canada, a property and casualty insurance company, bases its coverage decisions on geospatial information. According to reporting, this information enabled the company to improve its understanding of its exposure and reduce risk of excessive claim costs. In addition, the study noted that, for RSA Canada, geospatial information represents a source of competitive advantage.^{Footnote 82}
• Golder Associates, a resource extraction consulting company, indicated that its use of geospatial information and technologies enabled it to offer more enhanced services to clients and also to offer services a lot more efficiently than could be done previously.^{Footnote 83}
• Provincial interviewees mentioned that geospatial data were used in Alberta to prioritize and direct infrastructure rebuilding efforts following the 2013 floods, and this enhanced the deployment of rebuilding efforts by focusing them in areas of greatest need.

Finding: Studies commissioned by GeoConnections report that the use of geospatial outputs contributed $20 billion to Canada’s GDP in 2013, and that the use of geospatial outputs in various applications provided economic benefits, as well as less-measurable non-economic benefits, in addition to a positive return on investment

While it is challenging to systematically measure the use of geospatial data in decision-making and, by extension, the benefits arising from this use, research has been conducted in various countries to develop measurement methodologies. GeoConnections sponsored the Canadian Geomatics Environmental Scan and Value Study which modelled the economic benefits of the use of geospatial outputs in decision-making across 60 sectors of the Canadian economy. Information from 14 case studies, 137 consultations with industry leaders and analysis of comparable studies in other jurisdictions provided the basis for determining the overall level of applicability of geospatial information to each sector, while standardized adoption rates were applied across all sectors. Based on the model, the report estimates that the use of geospatial outputs across Canada in 2013 contributed $20.7 billion^{Footnote 84} to Canada’s GDP, representing 1.1% of the total Canadian GDP for that year.^{Footnote 85} The report also estimated that two-thirds of the contribution to GDP attributable to the use of geospatial outputs was related to more effective use of labour and capital.^{Footnote 86} The largest impacts of geospatial sector outputs were reportedly in the mining and oil and gas extraction sector.^{Footnote 87}

GeoConnections also sponsored studies demonstrating a positive return on investment resulting from using geospatial information and technologies for decision-making. For example, Hectares BC, a web-accessible geospatial analytical tool for the natural resource sector, was found to provide government agencies with an overall return on investment of 108%.^{Footnote 88} Similarly, BCeMap, which provides a single resource to aggregate relevant incident data for emergency management and public safety personnel in British Columbia and across Canada, was found to have an annualized return on investment of 60% for related agencies.^{Footnote 89} Finally, implementation of the multi-agency situational awareness system (MASAS) in New Brunswick was found to provide an annualized return on investment of 16.4% for organizations involved in emergency operations, police, regional health offices and the provincial transportation department.^{Footnote 90} While the results of these studies are encouraging, an audit conducted of the U.S. spatial data infrastructure^{Footnote 91} identified the understatement of the cost of geospatial data acquisition as an issue which, if also found to be the case in Canada, would overstate return on investment.

In addition to estimating the economic benefits of geospatial data use for decision-making, the Canadian Geomatics Environmental Scan and Value Study identified less-tangible benefits of using geospatial sector outputs, including:^{Footnote 92}

• Environmental benefits (improved environmental protection, better compliance with regulatory requirements, better management of resources and reduced impacts of natural resources)
• Health benefits (improved allocation of resources to manage disease outbreaks and emergency situations, better patient care)
• Social benefits (more effective communication among governments, business and citizens, increased confidence in services, improved community engagement
• Knowledge benefits (improved presentation and understanding of complex information, better focus on areas of risk, improved ability to plan, better analyses, better evidence-based decision-making and improved citizen literacy).

4.0 Evaluation Findings: Efficiency and Economy

Summary of Findings

• The GeoConnections Program design includes many efficient practices: good management practices, a national CGDI leveraged by regional governance partners, use of international consensus based standards to increase adoption, and consolidating portals under GeoGratis. Challenges remain in the areas of governance and efficient usability of GeoGratis.
• Through G&C, GeoConnections has strong leveraging ratio of 1:3.9 including cash and in-kind contributions to projects. Concerns that planned decrease in G&C funding would slow program progress.

Program seen as well-managed: All interviewees familiar with GeoConnections program operations said that it was well managed. External interviewees generally described program personnel as highly motivated and committed to the program. Most that had attended sessions or worked with GeoConnections staff described them as very collegial, and having a good knowledge of the sector and its challenges.

Public sector partners leveraging work of CGDI: GeoConnections’ national approach to developing the CGDI and associated governance mechanisms has provided a uniform approach to SDI that each province and territory has leveraged for their regional SDI initiatives. Interviewees report that this has saved time because the regional jurisdictions are not starting from scratch, and saved resources because they are able to discuss their efforts and collectively troubleshoot problems they encounter. That said, as noted earlier coordination of efforts between jurisdictions is an ongoing challenge.

Incorporation of international consensus base standards is beneficial: Interviewees reported that using international standards within the CGDI increases the likelihood that standards will be adopted in Canada. This makes it easier for Canadian companies to develop applications that can be marketed elsewhere or that incorporate data from other jurisdictions. 

Consolidating Portals: Evidence from documents and interviews shows that over the last two years, GeoConnections (and NRCan’s Essential Geographic Information and Support Program) has been consolidating the number of portals through which geospatial information is available. As of 2015, GeoBase, GeoDiscovery Portal and GeoGratis are now combined in a revised GeoGratis architecture.^{Footnote 93} However, while this has reduced the number of portals, many interviewees noted that the resulting portal of many portals is not necessarily a more efficient means of finding geospatial data as it still requires a lot of work to locate the datasets one is looking for.

Leveraging from G&C is high: According to data provided by the program, the G&C component has achieved a strong leveraging of 1:3.9 in recipient resources (this includes  1:1.2 in cash contributions to projects and 1:2.7 in in-kind contributions of personnel and equipment). Interviewees familiar with this work explained that there is lots of interest in working with GeoConnections to develop projects because of the program’s expertise and ability to secure partnerships. That said, it was often noted that the available G&C funding of approximately $500,000 per year (down from a five year average of $611,768 per year over Phase III) may be too small to achieve change at a faster pace and stay at the leading edge of technology and standards development work.

Table 3: GeoConnections Grants and Contributions Program Leveraging 2010-11 to 2014-15

	Total GeoConnections G&C Funding		O&M Expenditures to administer G&C	Total (G&C and O&M)	Total Leveraged $ (Cash) from Recipients	Total Leveraged (In-kind) from Recipients	Total Leveraged $ (Cash) and In-kind from Recipients
	Planned	Actual	Estimated		Actual	Actual	Actual
	$3,500,000	$3,058,842	$400,000	$3,458,842	$4,067,917	$9,469,611	$13,537,528
Leveraging ratio (program G&C and O&M expenditures : recipient contributions)					1:1.7	1:2.7	1:3.9

5.0 Evaluation Conclusions and Recommendations

GeoConnections, well aligned with NRCan priorities, has responded to a well-understood need for activities that support implementation of a national Canadian Geospatial Data Infrastructure (CGDI) that enables access to public data that would otherwise be difficult to find or use.

GeoConnections has convened appropriate governance bodies and facilitated a common understanding of geospatial issues across public, private and academic sector groups. The program has contributed to the development of and supported implementation of common CGDI standards and operational policies, and facilitated technology/tool development. Collectively, this has contributed to making geospatial data available online, and to some degree interoperable. Evaluation evidence confirms the expected links between GeoConnections activities and contribution to the expected result of increased use of geospatial data.

Overall, GeoConnections is an efficient means of achieving the objectives of implementing the CGDI and thereby increasing use of geospatial data in decision-making.

While evidence was found to support contribution to all expected results in the chain, suggesting that the program theory is correct, a number of opportunities were found to mitigate risks posed by intervening factors and to improve results for the next phase of the program. 

Issue 1: Improved Communication About CGDI

The evaluation finds that there is a generally low awareness of GeoConnections’ role in CGDI and understanding of CGDI itself. Evidence from survey and interviews shows that while stakeholders see value in the GeoConnections program goals, there is likely a low understanding of what NRCan does with respect to standards, operational policies, and technology work on CGDI. This is due in part because much of this occurs in the background (i.e., may not be aware of NRCan role in OGC standard setting). This is seen as an important issue to address, according to interviewees, as an increased awareness of GeoConnections’ and other efforts in implementing the CGDI could help with standards and policies uptake as well as increased leveraging of cooperation and investment in CGDI. Similarly, interviewees suggested that the impacts of the projects GeoConnections supports through G&C could be improved with communication of project results to help the wider community generalize from the work that is already being done.

Note that this recommendation is consistent with the recommendation of the 2015 CGDI Assessment to work with FCGEO and GeoAlliance to define lead responsibility for determining priority communication activities and implementation around all components of CGDI including framework data, standards, operational policies, and technology.

Recommendation 1: GeoConnections should strengthen its national convening role and work with CCOG, FCGEO, and GeoAlliance to communicate widely the value of CGDI and the importance of its uptake.

Issue 2: Reinforce Open Data Leadership

While the provision of interoperable and integrated geospatial data are important higher-order goals pursued by GeoConnections, the ability to discover the data, in any form, is necessary for it to be used and to avoid duplication. The evaluation found that more work is needed on discovery of public datasets across Canada. This challenge appears tied to development and application of metadata standards, search functionality on public portals such as GeoGratis, and data sharing agreements. 

• Support for implementing meta-data standards: There is widespread agreement from interviewees that the program should renew efforts on meta-data standards to enable discovery of datasets across Canada. The US and EU are experiencing similar challenges with data discovery. The program should investigate if the difficulties around meta-data are due to the standards themselves or their implementation. 
• Data search capability: Evidence from interviews and documents shows that there continue to be challenges finding data through GeoBase and GeoGratis.
• Data sharing agreements: Evidence from interviews and documents shows that data sharing agreements impact availability of data. For example, according to interviewees most municipalities and some provinces do not have open data policies, or do but with restrictive licensing that limits access.

The evaluation evidence suggests a need for a coordinated approach to framework data maintenance that would avoid duplication across jurisdictions. Framework data is an essential part of the CGDI as it provides the base layer onto which other location-based data can be fixed. Evidence from interviews and documents suggests that there are considerable gaps emerging in Canada with respect to the completeness and currency of framework data. This is in part a result of underinvestment in framework data due to fiscal constraints over the past ten years, but also a lack of coordination that could ease those constraints by avoiding duplication. Many interviewees in both industry and government suggested that both a national imaging strategy and a national mapping strategy would be very important for finding cost effective ways to update and maintain framework data. NRCan, most note, would be the natural lead on such an effort. This strategy should include verifying the existence of available geospatial data and should support collaborative initiatives to obtain and develop geospatial data.

Recommendation 2: GeoConnections should‎ reinforce its leadership on open data by supporting more efficient collection and sharing of data by addressing concerns raised over a) the discovery of data and b) coordination on updating of foundational geospatial datasets.

Issue 3: Support for Integrating Geospatial Data into Decision Making

Use of geospatial data depends on the availability of and access to data. However, even when the maximum amount of data is available and accessible, it will not inform decision-makers if it cannot be translated into easily consumable analyses. GeoConnections has enabled public and private investment in geospatial technologies by supporting technology projects through its G&C program. However, evidence from surveys and interviews suggests that there are considerable opportunities to improve use of data in decision-making through developing applications that make the data accessible and usable to non-experts. Many interviewees suggested that this was the next most important step in enhancing data use. As well, surveyed stakeholders gave the lowest assessments of progress and capability on issues of geospatial data use and developing user friendly applications/technologies.

Recommendation 3: GeoConnections should provide targeted support to integrate geospatial data into the decision-making processes of non-experts

Issue 4: Support for Geospatial Community Transformation

It was also widely suggested that there is great potential to increase geospatial data use in Canada by non-traditional sector groups (i.e., not only geomatics and mapping). However, to benefit they must be reached by the traditional community and included in discussions about CGDI and data use as part of a much more inclusive geospatial sector. While the Program has provided a compelling case for CGDI adoption through studies and papers, it was noted that the non-traditional user groups now targeted are not likely to have been reached by these analyses. 

The evaluation found that the creation of GeoAlliance in April 2015 is a necessary and positive achievement for addressing geospatial community fragmentation by creating an organization that would bring all of the disparate players together and leverage their combined resources. There are some concerns, however, with sector readiness for the transition from the NRCan-led CGCRT to the sector-led GeoAlliance Canada. This is due in part to resourcing, obtaining buy-in, and the fragmented nature of the geomatics and geospatial data community.

Resourcing: there is a lack of dedicated human and financial resources to supporting the CGDI. Nearly all interviewees explained that participation in collective efforts for CGDI development are undertaken on the corners of desks rather than through dedicated staff. . Many suggested a need for federal seed funding to support GeoAlliance in its early days.

Obtaining buy-in: Under CGCRT, the community came together for an initiative that was chaired and funded by NRCan and did not have membership dues requirements to sustain it. As GeoAlliance will be an association, as opposed to a government initiative, concerns were raised over what role NRCan would play and that a reduced level of visibility would incorrectly be interpreted as a signal a lack of interest to the community. To attract membership, GeoAlliance will have to demonstrate clearly its value to potential members.

Fragmented community: Survey and interview evidence shows that the level of fragmentation in the geospatial community, while improving, continues to be a challenge. This is significant given that GeoAlliance seeks broad participation beyond CGCRT membership to include the non-traditional user community.

Recommendation 4: GeoConnections should continue to work with the geospatial community to encourage the transformation from a narrow focus on geomatics experts to a broad focus on non-traditional user groups. This should include supporting the GeoAlliance, whose objectives of implementing the Pan Canadian Geomatics Strategy are complementary to the GeoConnections program.

Appendix A: List of Terms and Definitions

Geospatial Data: any type of data that is associated with particular geographic locations. Examples include community-level demographic data, pollution data for rivers and the location data for buildings in cities.

Framework Data is one type of geospatial data that provides context and reference to other data, enabling it to be associated with particular geographic locations. For example, framework data related to roadways provides a frame of reference for the buildings located on the roadway, while framework data related to federal electoral boundaries provides a frame of reference for information about households located within those boundaries. There are three types of framework data:

• Alignment data, which comprises the geometric controls which are critical to accurately associating all types of data to the appropriate physical locations.
• Land Feature data, which comprises the locations of readily observable natural or human-made physical features such as roads, rivers and elevation.
• Conceptual data, which comprises the locations of human-made features that were developed for governance, such as municipal and provincial boundaries, federal electoral districts and ecological areas.

The development of framework data is largely the responsibility of the public sector and these data function as the underpinning for many geospatial information applications.

Geospatial Metadata is reference information describing the basic characteristics of geospatial data and services (sometimes referred to as data about data).

Technical and Data Standards allow diverse data sources, services, applications and systems to operate compatibly with each other.

Operational Policies are a range of instruments, such as guidelines, best practices, directives, procedures and manuals, that address legal and administrative requirements of the lifecycle of geospatial data. Policies cover topics such as privacy, intellectual property and licensing as well as technological trends such as cloud computing.

Geomatics sector refers to organizations, from industry, government, and academia, that are involved in the provision of geospatial data and related information products, geospatial technologies, services and expertise. This sector is also responsible for the development and delivery of Canada’s network of spatial data infrastructures, including framework data and core data. ^{Footnote 94}

Geospatial data community includes the geomatics sector plus the organizations and individuals that use the geospatial data, technologies, and services provided by the geomatics sector.^{Footnote 95}

Geospatial Technologies consist of web-based data portals and platforms that provide a gateway to distributed internet-based geospatial data, as well as tools and web services that enable the discovery, viewing, downloading, analysis, sharing and transformation of geospatial data.

Web applications are software, accessed through a user interface and a web browser, that provide value-added data and services through the integration of geospatial data with other data and services. Web applications may be enabled by web services.

Open Geospatial Consortium (OGC) is an international voluntary organization that develops geospatial standards. More than 500 commercial, governmental, non-profit and research organizations worldwide collaborate in a consensus-based process to develop and implement open standards for geospatial data and services, data processing and sharing.^{Footnote 96}

Appendix B: Stakeholder Survey Respondent Characteristics

Table 4 compares the distribution of stakeholder population to the distribution of survey respondents. Members of the CGCRT/GeoAlliance Canada were over-represented among survey respondents while members of FCGEO were under-represented. However, given the broad membership of the CGCRT (including government, industry, academia), the greater representation of this group among survey respondents enhances the overall usefulness of the survey results for providing insights into the perspective of the geomatics community as a whole.

Table 4: Comparison of Stakeholder Population to Survey Respondents

Stakeholder Group	Population of GeoConnections Stakeholders % of total	GeoConnections Stakeholders Survey Respondents % of total
CCOG	28	26
FCGEO	38	17
CGCRT/GeoAlliance Canada	34	58

In terms of regional breakdown, Ontario was the most frequently identified location among survey respondents (51%) and the stakeholder population as a whole (44%). Ontario was also the most frequently identified location for each group of survey respondents, reported by 30% of CCOG respondents, 77% of FCGEO respondents, and 51% of CGCRT/GeoAlliance Canada respondents.

Table 5 shows the self-assessed familiarity of surveyed stakeholders with geospatial issues within their organizations for each stakeholder group. Majorities in all groups (87% total) reported high levels of familiarity (scores of 4-5 on a 5 point scale) with geospatial data issues within their respective organizations. These most knowledgeable respondents also tended to be more critical, providing lower scores when rating achievement/success of the GeoConnections program. An exception to this pattern is CCOG members’ high ratings of achievement in the area of putting in place governance mechanisms through the Canadian GeoSecretariat and the CGDI.

Table 5: Stakeholder Group Familiarity with Geospatial Issues

Stakeholder Group	High familiarity (scores of 4-5) with geospatial data issues in their organizations % of stakeholder group
CCOG	95
FCGEO	83
CGCRT/GeoAlliance Canada	83
Total (all stakeholder groups)	87