AUDIT OF THE FEDERAL GEOSPATIAL PLATFORM

Presented to the Departmental Audit Committee (DAC)
September 26, 2019

Table of Contents

• Executive Summary
  • Introduction
  • Strengths
  • Areas for Improvement
  • Internal Audit Conclusion and Opinion
  • Statement of Conformance
  • Acknowledgements
• Introduction
  • Audit Purpose and Objectives
  • Audit Considerations
  • Scope
  • Approach and Methodology
  • Criteria
• Findings and Recommendations
  • Governance and Project Management
  • Partner and Stakeholder Management
  • Information Technology Management
• APPENDIX A – Audit Criteria

Executive Summary

Introduction

The Government of Canada collects and utilizes geospatial data^{Footnote 1} to support objectives such as economic growth, environmental management and social well-being. The Federal Geospatial Platform (FGP) initiative was envisioned by the Federal Committee on Geomatics and Earth Observations (FCGEO) to develop capacities and utilize the growing collection of geospatial data to support these objectives. This Committee, created in January 2012, is comprised of senior executives from 21 various departments and agencies that produce and/or consume geospatial data. The objectives of the Committee include developing frameworks, policies, and directives to strengthen federal capacity in geospatial and Earth observation information as well as capacities in people, technology, service, and information. In May 2014, Treasury Board (TB) approved the creation of the FGP with a budget of $37.4 million (M) to be contributed by the partner departments.

The objective of the FGP is to facilitate access to government geospatial data and reduce barriers in sharing data. The FGP integrates large repositories of economic, social and environmental geospatial data with technology from multiple departments and agencies to better support location-based decision making on various complex issues. Topics may include energy infrastructure, Indigenous communities, marine species at risk, caribou habitats, reported pollution and water quality.

While the FGP is governed horizontally by partner departments, leadership of the governance structure is the responsibility of Natural Resources Canada (NRCan) as the Executive Sponsor of the platform. Within this role, NRCan has taken the lead in the development and implementation of the platform. The FGP division created within NRCan to build and manage the platform, falls within the Canada Centre for Mapping and Earth Observation (CCMEO) in the Strategic Policy and Results Sector (SPRS). CCMEO is the Government of Canada’s centre of excellence for geomatics, mapping, and earth observations.

In addition to the FGP partner departments, users of the platform may include other federal departments who use geospatial data; as well as non-governmental organizations; municipal, provincial and territorial governments; academia; aboriginal organizations; private sector; public volunteers; and citizens.

The FGP went into its post-project operational stage on March 31, 2017, and continues to be a collaborative effort among the platform partners. NRCan continues to lead the governance of the FGP in the operational stage of the platform’s life-cycle; and the FGP Division within CCMEO continues to serve as the project management office for the platform.

This audit was included in the 2018-2021 Risk-Based Audit Plan, approved by the Deputy Minister on
April 12, 2018.

Strengths

Overall, the FGP’s management team within the CCMEO has demonstrated geospatial- and
project-management expertise as evidenced by established processes in project planning, risk management, and stakeholder awareness and outreach. Their efforts to bring together a large number of partner departments and stakeholders to develop a shared platform has made significant progress towards supporting numerous Government of Canada priorities.

Areas for Improvement

Opportunities were identified to strengthen governance mechanisms, risk management, and monitoring and reporting activities. The implementation of the required Information Technology (IT) security controls for the platform is an area for improvement and opportunities also exist to improve outreach and feedback mechanisms to increase stakeholder awareness and platform usability.

Internal Audit Conclusion and Opinion

In my opinion, CCMEO has taken significant steps in recent years to establish effective project-management and information-management processes, and stakeholder outreach activities for the FGP. Opportunities exist to clarify and improve management processes regarding effective monitoring and IT security, and to continue to build stakeholder awareness.

Statement of Conformance

In my professional judgement as Chief Audit Executive, the audit conforms with the Institute of Internal Auditors’ International Standards for the Professional Practice of Internal Auditing and the Government of Canada’s Policy on Internal Audit, as supported by the results of the Quality Assurance and Improvement Program.

Christian Asselin, CPA, CA, CMA, CFE
Chief Audit and Evaluation Executive
September 26, 2019

Acknowledgements

The audit team would like to thank those individuals who contributed to this project and, particularly employees who provided insights and comments as part of this audit.

Introduction

The Government of Canada collects and utilizes geospatial data^{Footnote 2} to support objectives such as economic growth, environmental management and social well-being. Given that data has become a new global currency in the age of information, the ability to turn such data into meaningful information is increasingly important.

Since 2007, there has been a significant rise of dynamic mapping with the influx of new technologies and increasing computing capabilities to capture in near real-time geospatial data for use such as flood information, earthquakes monitoring, soil saturation level, topography, etc. Before the creation of FGP, many departments and agencies were collecting geospatial data for their own needs using different types of technology and standards. As a result, most of the government geospatial data was not released openly for Canadians to use and could not be easily shared or joined between departments and agencies.

In May 2014, Treasury Board approved the creation of the FGP for an estimated cost of $37.4M with the objective to facilitate access to government geospatial data and reduce barriers in sharing data. The FGP aims to be a collaborative, on-line environment consisting of authoritative geospatial data, services and applications to enable the federal government’s most relevant geospatial data to be found easily and viewed on maps to enhance decision making, foster innovation and ensure better service to Canadians. The FGP’s objective is to integrate large repositories of economic, social and environmental geospatial data with technology from multiple departments and agencies to support location-based decision making on various complex issues. Topics may include energy infrastructure, Indigenous communities, marine species at risk, caribou habitats, reported pollution, and water quality.

Similarly developed nations including the United Kingdom, Australia, Singapore, and the United States have all made significant investments in geospatial platforms over the last decade and are at varying stages of development. It is clear that these nations are also making open and accessible sharing of geospatial data a priority to reduce duplicative efforts and increase economic activity.

The FGP was designed to align with and support Government of Canada priorities, such as Open Government, Economic Competitiveness, Safety and Security, Environment, and Public Service Renewal. The FGP is comprised of numerous repositories of datasets brought together through interdepartmental agreements and IT components, including a web application to access, display, and visualize data held in these repositories. As of July 2019, the platform had 1,145 geospatial datasets, which are shared with platform users through two interfaces: an internal site that can be found on the internal government network; and a more limited selection of datasets available on a public site entitled Open Maps on the Open Government Portal for which the TB Secretariat is accountable.

The FGP initiative is led by the FCGEO. This Committee, created in January 2012, is comprised of senior executives from 21 various departments and agencies (see Exhibit 1 below) that produce and/or consume geospatial data. This Committee serves as the collective voice of federal geomatics across the public service. The objectives of the Committee include developing frameworks, policies and directives to strengthen federal capacity in geospatial and Earth observation information, as well as capacities in people, technology, service and information.

Exhibit 1 : Federal Committee on Geomatics and Earth Observation (FCGEO) Membership

While the FGP is governed horizontally by partner departments who voluntarily contribute resources to the platform, the leadership of the governance structure is the responsibility of NRCan as the Executive Sponsor. As the Executive Sponsor, NRCan has taken a lead role in the development and implementation of the platform. The FGP Division created within NRCan to build the platform, falls within the CCMEO. CCMEO is the Government of Canada’s centre of excellence for geomatics, mapping, and earth observations.

In addition to the partner departments, other FGP platform users include other federal departments who use geospatial data; as well as non-governmental organizations; municipal, provincial and territorial governments; academia; aboriginal organizations; private sector; public volunteers and citizens.

The FGP went into its post-project operational stage on March 31, 2017, and continues to be a collaborative effort among the platform partners. NRCan continues to lead the governance of the FGP in the operational stage of the platform’s life-cycle and the FGP division within CCMEO continues to serve as the project management office for the platform.

This audit was included in the 2018-2021 Risk-Based Audit Plan, approved by the Deputy Minister on
April 12, 2018.

Audit Purpose and Objectives

The objective of the audit was to assess the overall adequacy and effectiveness of key management processes supporting the implementation and continuing operations of the FGP.

Specifically, the audit assessed whether:

• NRCan has established adequate and effective project management processes for the implementation of the FGP and to inform future horizontally-led projects;
• NRCan has established adequate and effective processes to manage partnerships and address stakeholder and user needs; and
• NRCan has effectively managed IT to support the FGP. 

Audit Considerations

A risk-based approach was used in establishing the objectives, scope, and approach for this audit engagement. The following areas were identified as having significance in the achievement of the Department’s objectives, and were therefore assessed as increased areas of risk for this audit:

• Effective project management and governance mechanisms ensure efficiency, reduce duplication of efforts across numerous partner departments and increase the likelihood of achieving project objectives.
• Clear assignment and monitoring of partner roles and responsibilities for information management supports effective data management throughout the project lifecycle.
• Effective stakeholder management via outreach, collaborative activities and training ensure that users across the federal government (e.g., policy analysts, senior managers, and non-technical users) can ultimately use federal geospatial data to inform decision making, reporting, operations and research.
• Effective management of IT is critical to achieving the objective of a common “platform” for information sharing.

Scope

The audit focused on the FGP activities within the Department, including an examination of the roles and activities of FGP as the Office of Primary Interest. The audit also examined the work of relevant geospatial governance committees (e.g., FCGEO and the FGP Board of Directors) and how they contributed to the FGP.

The scope of this audit included relevant departmental and Sector processes, procedures, and tools used to plan, conduct, monitor, and report on FGP activities. The audit focused primarily on the platform’s operational phase period from March 31, 2017, to March 31, 2019, in order to examine recent operational platform activities and processes. The audit also considered the progression of its project phase from the project’s approval in 2014 until March 31, 2017, when the project closed and moved to the operational phase, as relevant to the criteria. Consideration was also given to the results of previous relevant advisory, audit and evaluation related topics.

NRCan interfaces with other federal departments and agencies in the geospatial domain. As such, the interactions between NRCan and other governmental departments was examined during the Planning and Conduct phases of the audit to understand FGP governance and communication activities from a horizontal perspective; however, it is not within the mandate of the Audit and Evaluation Branch to audit the activities of these departments.

Approach and Methodology

The approach and methodology followed the Institute of Internal Auditors' International Standards for the Professional Practice of Internal Auditing and the Government of Canada’s Policy on Internal Audit. These standards require that the audit be planned and performed in such a way as to obtain reasonable assurance that audit objectives are achieved. The audit included tests considered necessary to provide such assurance. Internal auditors performed the audit with independence and objectivity as defined by the International Standards for the Professional Practice of Internal Auditing.

The audit approach included the following key tasks:

• Interviews with key personnel, stakeholders, platform users, and subject matter experts;
• Review of selected key documents, business process, and communication materials;
• Review of information and documentation pertaining to project planning, oversight, outreach, feedback and information management activities; and
• Various file testing and analyses pertaining to the specified criteria.

The conduct phase of this audit was substantially completed in May 2019.

Criteria

Please refer to Appendix A for the detailed audit criteria. The criteria guided the audit fieldwork and formed the basis for the overall audit conclusion.

Findings and Recommendations

Governance and Project Management

Summary Finding

Overall, CCMEO has established adequate and effective project management processes for the implementation of the FGP and identified lessons learned to inform future horizontally led projects. CCMEO followed and met the standards, directives, and guidelines set out by the TB Secretariat during the project phase of the FGP. Opportunities were identified to improve monitoring activities, as well as communication of monitoring and risk management activities to NRCan senior management and FGP governance committees, during the operational phase of the platform.

Supporting Observations

Effective planning and management of projects including implementation of governance and reporting mechanisms are integral to the success of all NRCan projects. The audit team expected that adequate mechanisms were designed and utilized by CCMEO to support the implementation of the platform. It was also expected, given the scope of the platform and the involvement of a large number of partner departments that lessons learned during the course of the project would be documented to inform future horizontal projects. In order to effectively manage the project into the operational phase, the audit team expected that monitoring of the platform’s performance and risk management activities would continue to be performed and communicated to internal and external stakeholders.

Project Planning

The FGP project was required to complete the Government of Canada Project Complexity and Risk Assessment (PCRA) tool since this is mandatory for projects with a budget greater than 1 Million dollars. The FGP PCRA was completed and scored a Level 3 based on its risk and project characteristics. Projects that score Level 3 or 4 on their PCRA require a TB Submission to be completed and approved. This requirement is also in accordance with the NRCan Project Management Framework, which states that the Deputy Minister is responsible for ensuring that TB approval be sought for major projects with a Level 3 or 4 PCRA rating.

CCMEO established a project plan in April 2014, along with the supporting documentation required by the NRCan Project Management Framework and TB Secretariat Policy on the Management of Projects. The documentation included the design of the governance and oversight mechanisms put in place to effectively govern, manage, monitor, and report on the FGP. The documentation also included mechanisms to ensure that any major changes to the roles, responsibilities, and governance and oversight mechanisms were appropriately captured through updates to the appropriate documentation. The audit team also noted that the project plan and supporting documentation and subsequent updates to documentation were communicated in a timely manner to the various project stakeholders.

The audit team found that after the project phase was completed on March 31, 2017, the governance committees continued to meet during the ensuing operational phase of the project. CCMEO, along with oversight from the FGP Board of Directors, continued to plan the strategic path forward through the development of the FGP Strategic Plan that was finalized on June 26, 2018.

Monitoring and Reporting

The audit examined whether structures and processes were in place to enable monitoring and reporting on project related performance. During the project phase (May 2014 - March 2017), performance metrics for the project were clearly defined and communicated to senior management. After the project phase ended and the operational stage began in April 2017, limited performance monitoring and reporting were found to be conducted, with the focus being primarily on meeting corporate level reporting requirements i.e. Departmental Results Framework.

Project phase

Providing senior management with regular performance reporting metrics is a key element supporting effective and timely decision making and resource allocation. During the project phase of the platform (May 2014 – March 2017) various activities including project key performance indicators (KPIs) and project-related risks and issues were monitored and reported to NRCan senior management at the Assistant Deputy Minister (ADM) level. Several channels were used to communicate the results of the monitoring and the status of the project and they included quarterly dashboards, governance committee meetings, annual reports and workshops. In addition, as part of the project plan, governance structures and committees were established to provide oversight, guidance, and decision-making on project related activities. During the project phase, these committees met monthly and received the results of monitoring activities from CCMEO.

The quarterly dashboards provided information to stakeholders regarding the financial health of the platform, including a financial summary, the project schedule, risks and issues facing the project, and requests for change. The audit team expected that some of these reported items and monitoring activities including the KPI’s established in the project phase would continue to be used to report on the performance of the platform throughout the operational phase.

Operational Phase

When the project phase was closed out and transitioned to the operational phase at the end of fiscal year (FY) 2016-17, the TB requirements for reporting were no longer required as the platform was no longer under the purview of the TB Secretariat. During the operational phase, CCMEO planned to continue to report on the status of the platform to NRCan senior management and the various governance committees; however, the KPIs that were established and reported on during the project phase of the platform were not used to measure the performance of the platform during the operational phase.

CCMEO indicated that their focus was on developing the strategic plan that would guide the FGP. They also indicated their focus was on developing new KPIs to measure the platform’s performance and align with the goals identified in the strategic plan. Some examples of the KPIs in the strategic plan include: the number of datasets published; the percent of Other Government Departments using the FGP; the number of new individuals using the FGP; and the change in the number of datasets accessed. The audit team noted that the FGP Strategic Plan was completed in June 2018; however, the new strategic plan KPIs have not been used to measure and report on the performance of the FGP to either senior management or other senior governance committees as of the completion of the conduct phase of this audit.

Risk Management

A critical component to the achievement of FGP objectives includes the design and implementation of a risk management plan that continuously assesses the environment in which the FGP operates to identify risks and actions to address these risks. It was expected that CCMEO had established a risk management plan that addresses the critical components for the FGP. In addition to these risk-management processes, it is expected that risks that could result in major changes or the potential for failure is communicated to oversight committees and NRCan senior management.

The audit found that CCMEO designed a risk management plan as part of the planning activities during the project phase of the platform that is consistent with the TBS Framework for the Management of Risk. This plan identifies the FGP Project Management Office (PMO) as the primary group responsible for overseeing the planned activities. These activities include internal and external scans of the environment to identify risks. These risks were then assessed for their likelihood and impact and recorded in risk registers, as required by the NRCan PMO Framework. The risk registers also included the documentation of the decisions to address the identified risk as well as the implementation plan and assigned an individual who is responsible for completing the implementation plan. The plan states that the FGP PMO would ensure that risks to the project and outcomes receive the necessary oversight and engagement and act as a liaison to the various FGP governance committees.

The audit found that during the project phase, the FGP was supported with clearly defined processes for the management of risks. The audit team evidenced this through the identification, analysis, and treatment of risks in the Risk Register, which was updated on a regular basis at FGP PMO meetings. Reports on these risks were provided at regular intervals through the quarterly Executive Dashboards and FGP Board of Directors (BoD) meetings. Furthermore, processes and mechanisms necessary for escalating risks were established during the project phase. The audit found that during the project phase, risks were communicated and discussed with governance committees and NRCan PMO if the risk would result in major changes or have potential to cause project failure. During the operational phase, CCMEO determined that risks identified did not require communication to the FGP BoD.

The audit team conducted sample testing of the risk registers maintained throughout the project and operational phases. The audit found that for the sample of project phase risks tested, all risks identified were tracked, and monitored using the risk register. All risks have been closed and it was determined that the specified risk treatments were completed by the responsible parties.

For the sample of risks selected from the operational phase, the audit found that all identified risks were being tracked, and monitored via the risk register and that descriptions of the risk responses being implemented were documented. One exception was identified by the audit related to the risk that potential FGP users across the government may not be informed of the benefits of the FGP. Despite being deemed a ‘Very High’ threat level, the FGP team suspended the planned implementation plan to mitigate the risk through communication and outreach activities, accepting the potential impacts of the risk going unaddressed. The decision to suspend the implementation activities was not formally documented or communicated.

Lessons Learned Documentation

The documentation of lessons learned is relevant at both the project and corporate level as the documentation of issues faced can contribute toward improving efficiency and effectiveness of project management across an organization. It enables improved capabilities, decision-making, and efficiencies when used to inform future horizontally led projects.

The audit team found that CCMEO documented and communicated the information gathered in lessons learned activities during the project phase in a timely manner. This information was communicated to the Treasury Board Secretariat through annual reports and the Project Closeout Report. Additionally, the lessons learned throughout the completion of the project were communicated internally to the NRCan PMO with the intention that it be shared and used to improve future horizontally led projects.

Overall, the documentation of lessons learned during the project phase of the platform by CCMEO and their communication internally and externally to inform future horizontally led projects has been completed in a timely manner.

Risk and Impact

There is a risk that groups charged with oversight, including governance committees and NRCan senior management may not be receiving adequate and timely reports on the platform’s performance and risk management activities and therefore may not have all relevant information to properly conduct management and oversight of the platform.

Recommendation

Recommendation 1: It is recommended that the ADM, SPRS, ensure that NRCan senior management and FGP governance committees receive adequate and appropriate information to support oversight of the platform through a review of the following areas:

• Requirements and timelines for performance reporting on FGP activities; and
• Requirements for the reporting of risks.

Management Response and Action Plan

Management agrees with Recommendation 1.

In response to Recommendation 1, the ADM SPRS will ensure that NRCan senior management and FGP governance committees receive adequate and appropriate information to support oversight of the platform on an annual basis. In order to accomplish this, the ADM SPRS will conduct a review of the current performance monitoring and risk assessment activities of the FGP in order to develop a framework that will support annual reporting and risk management. The ADM SPRS will report back annually to PSIC and EXCOM. The ADM SPRS will also assess the current FGP governance structure to ensure the FGP is positioned to strengthen decision making, both at NRCan and across government, especially in the context of the Platform’s role supporting the Government of Canada’s Data Strategy Roadmap.

Position responsible: Director of FGP

Timing:  ADM approval of FGP performance and risk reporting requirements – January 31, 2020

Partner and Stakeholder Management

Summary Finding

Overall, CCMEO has established adequate mechanisms to engage in outreach activities to promote the platform to new users, collect and utilize feedback, communicate effectively via formal lines of communication, and effectively manage data through assignment and monitoring of roles and responsibilities pertaining to information management. CCMEO has defined, assigned and operated numerous activities pertaining to training outreach, feedback, communication, and information management. Opportunities were identified to strengthen and improve some FGP processes and procedures related to the provision of training to new users, proactive measures to collect feedback, and clarifying roles and responsibilities for addressing the information management issues identified in the audit.

Supporting Observations

Ensuring appropriate lines of communication have been established between CCMEO, FGP users and FGP partners is critical to the success of the platform, given that the FGP relies on these parties in various ways, including information management, funding, feedback from users, and as part of the governance and oversight mechanisms.

Communication channels between NRCan and FGP partners and stakeholders are important to enable the FGP team to conduct outreach activities to promote onboarding of new users, as well as to allow existing and potential users to share feedback to drive continuous improvement of the platform. The audit team expected that adequate lines of communication have been established between stakeholders and NRCan and that these lines of communication are being used by the FGP team to conduct adequate outreach activities to promote the platform. It is also expected that mechanisms have been established to collect feedback from users to improve the platform. In addition, the audit team expected that information management roles and responsibilities have been assigned and are being monitored to ensure that data is effectively managed throughout the lifecycle of the platform.

Lines of Communication

The establishment of adequate lines of communication are a critical component to accomplish the platform’s objectives. Effective and defined lines of communication between CCMEO and FGP Partners may be utilized to conduct governance and oversight functions, to communicate changes and change management activities and to report on various aspects of the platform including the costs and risk management. 

The audit found that CCMEO has established adequate lines of communication between themselves and FGP Partners. During the project phase the formal agreement to these lines of communication were contained in the Project Charter documentation. When the FGP project phase closed, Memorandum of Understanding (MOUs) were signed with FGP Partner departments to establish the lines of communication for the operational phase. The audit found that formal lines of communication were appropriately established and agreed to by the FGP and partner departments and that the terms aimed to ensure effective collaboration to achieve the platform’s objectives. The primary method of communication between these groups was the FGP BoD governance committee. The committee is chaired by the Executive Director of the FGP, and each department that agreed to contribute resources to the FGP was granted a seat on the Board. The audit noted that the FGP BoD has held meetings since the beginning of the project phase, which began in FY 2014-15.

Outreach and Onboarding

In order to promote awareness and to encourage new departments and users to use the FGP, the FGP Policy and Communications team designed and implemented a communications plan. The Policy and Communications team developed communications tools designed to reach stakeholders within NRCan, FGP Partner departments, and the geospatial community at large. Communication activities educate current and potential users internal and external to the federal government about the capabilities of the platform and to promote onboarding of new users. Between October 2015 and February 2019, the Policy and Communications team conducted over 200 demonstrations of the platform to the various stakeholders of the FGP. In addition to the in-person demonstrations, the Policy and Communications team utilized departmental intranet sites, social media, press releases from offices of Ministers and Deputy Ministers to promote the platform. FGP partner departments, as part of their agreements with NRCan, established in the project charter for the project phase and in MOUs for the operational phase, were tasked with undertaking communications activities to increase awareness and use of the FGP within their respective departments.

Interviews were conducted with various NRCan employees regarding the outreach and feedback processes of the FGP. The interviewees were selected based on their roles, which included both current users of geospatial data and persons occupying positions identified by the FGP as potential users of the platform. The audit noted that these individuals were made aware of the FGP through the communication and outreach activities of the FGP Policy and Communications Team; however, numerous interviewees, including policy analysts, felt that beyond the demonstration received, additional training would be required for them to utilize the platform in their day-to-day work. The audit team was informed that during the operational phase between March 2018 and November 2018, CCMEO made the decision, due to resource constraints, to suspend interdepartmental promotional campaigns and outreach activities. These suspended activities were used to target potential FGP users across the Government of Canada.

During the Project and Operational phases of the platform, the FGP Policy and Communications team and CCMEO performed outreach activities with representatives from a wide variety of departments. The audit also found that the FGP team effectively communicated by keeping these partners, as well as other stakeholders, apprised of platform updates and changes through the established governance committees.

Stakeholder Feedback

The FGP Client Services team has designed numerous mechanisms to collect stakeholder feedback throughout the life of the platform. During the project phase, the FGP team conducted analysis of various types of potential future users of the platform and consulted employees within these roles to design the platform to suit the requirements of these users. During the operational phase of the FGP, the primary mechanism used to collect feedback is the Client Services Team mailbox which is accessible through the “Help” section of the FGP website. Feedback and requests received through this mailbox are triaged by a centralized FGP Client Services team.

During the operational phase the primary mechanisms used by the Client Services team to collect feedback did not include proactively seeking feedback from users or potential users. The audit team interviewed various NRCan employees some of which use geospatial data on a daily basis and others whose position as Policy Analyst has been identified by the FGP as a potential user of the platform. All of the individuals interviewed were aware of the FGP and its services but were not frequent users of the platform. Reasons for not using the platform included: not having enough training or expertise, having access to alternate commercially available mapping software, and not having a need for the platform as part of their role. These interviewees did share with the audit team feedback and suggestions they felt would improve the FGP’s usefulness; however, none of the interviewees had shared their feedback with the FGP team. A client survey to proactively reach out to users of the FGP to solicit their feedback on the platform is being planned for release by the FGP team in FY 2019-20; however, the survey has not been completed as of the conclusion of the conduct phase of this audit.

The Client Services team has established effective and efficient processes to track the feedback and requests collected and to deliver an annual report which includes statistical information regarding numerous criteria including: the types of requests, who the requests are from, and the improvements that are desired. The collection and reporting processes examined by the audit team were effectively designed and implemented and were used to provide CCMEO and governance committees with accurate and timely information regarding the feedback obtained.

While the audit found that the processes used to collect and track user provided feedback and requests were effective and adequate, the audit team also conducted testing of feedback and requests sent to the Client Services Team to determine whether they were responded to within the timeframe established by the service standards of the Client Services team. The Client Services Team triages incoming feedback and requests and forwards them to the appropriate party, which could be internal to NRCan or a group external to NRCan if for example the feedback pertains to a dataset contributed by a partner department. Audit testing indicated that while the processes employed by the Client Services Team are adequate to collect and communicate feedback to the appropriate parties, the responsibility to action feedback and requests is decentralized and remains with dataset owners. The audit team did note through testing conducted on a sample of feedback submitted, that the Client Services Team quickly triaged the requests and issues they received to the appropriate parties. Their service standard specifies that the Client Services Team’s first response to a user submitted request or issue should take place within 48 hours of the request being received. The Client Services Team achieved this service standard for 96% of requests tested.

Information Management

Given that the FGP is a repository to organize and share geospatial data, the creation and implementation of a robust information management plan, as well as the assignment and communication of the roles and responsibilities for completing the plan are an important aspect to effectively manage data within the platform.

The audit found that during the project phase of the platform, CCMEO had established a Data Asset Management plan and assigned the roles and responsibilities related to the implementation of this plan. The responsibilities have been clearly communicated to the various groups identified as having a role in carrying out the plan. These responsibilities include that departmental data owners ensure compliance with the appropriate data and web service standards chosen for the platform by the FCGEO and the Data Maturity Model created for the FGP. The FGP Data Team at NRCan is responsible for providing assistance to data providers who wish to share data on the FGP, and ensuring that dataset use is monitored and tracked. To complete the aforementioned monitoring of dataset use, the FGP Data Team is responsible for the creation and communication of an Annual Data Asset Report. The audit found that this report has not yet been completed and the inaugural report is planned to be communicated to FGP governance committees in March 2020.

The formal agreements between the FGP and Partner Departments nominate a representative to act as a data champion and sit on the Data Work Committee (DWC) established as part of the FGP governance structure. The audit found the committee includes a wide variety of stakeholders and that the roles and responsibilities of the governance committee and the data champions are adequately assigned in the DWC Terms of Reference, formal agreements, and the Data Asset Management plan.

The Data Strategy document created by the FGP team is used to outline activities to be undertaken regarding data management and who is responsible for these activities. The strategy identifies that CCMEO is responsible for conducting monitoring of the quality of the datasets within the FGP. The audit found that Data Quality Assessments (DQAs) were conducted for each of the datasets in the FGP by an independent party during the summer of 2018. A sample of datasets found that for each a DQA was completed and issues were identified and communicated to dataset owners to be corrected to improve data quality. The audit also found that while the issues noted during the assessments were communicated to the parties responsible for correction of the datasets, actions to correct the identified issues were not always taken by dataset owners. The audit noted the responsibility for each departments’ data contributed to the platform remains with each respective departmental Data Owner, which is a role held by the department’s Chief Information Officer who, according to the TB Policy on Management of Information Technology, is responsible for ensuring that departmental data and applications are secure, reliable, and trusted.

Overall, the audit found that the roles, responsibilities, and accountabilities of both the FGP and the various stakeholders have been clearly defined, documented, and communicated to the relevant stakeholders. While, there is an opportunity for improvement relating to assignment of actions to correct deficiencies identified during monitoring, the FGP team is commended for their completion of the Data Quality Assessment process with the resources available and their commitment to improving and maintaining the quality of data available on the FGP.

Risk and Impact

There is a risk that suspension of communication activities will have a significant impact on the ability of the platform to continue to operate and promote outreach activities in order to attract new users to the platform and to train users who are aware of the platform but not currently using the platform.

There is a risk that feedback is not proactively sought and that the FGP is not receiving important feedback from potential users that have suggestions to improve the platform functionality to a broader user base. There is also a risk that some feedback on datasets and issues noted with the platform may not be actioned since the FGP team does not have the ability to enforce the responsible contributing department to correct reported issues.

There is a risk that the governance committees’ ability to conduct oversight of the platform and their understanding of the data currently held in the FGP will be affected by some monitoring and tracking activities outlined in the Data Asset Management plan not being completed. There is also a risk that issues identified by the monitoring activities conducted by the FGP could remain uncorrected if dataset owners chose not to take action. Datasets remaining uncorrected could lead to a loss of trust by users if the level of data quality does not meet the expectations of users.

Recommendations

Recommendation 2: It is recommended that the ADM, SPRS, review the communications and engagement strategies to clarify objectives related to onboarding new users in order to maximize participation in the platform and ensure alignment to the strategic plan.

Recommendation 3: It is recommended that the ADM, SPRS, review the established stakeholder feedback mechanisms to determine the degree to which feedback is obtained from both current and potential users of the FGP to drive continuous improvement.

Recommendation 4: It is recommended that the ADM, SPRS, ensure the Data Asset Management plan is reviewed to ensure the timely reporting of these monitoring and tracking activities to the appropriate stakeholders.

Management Response and Action Plan

Management agrees with Recommendation 2. 

In response to recommendation 2, the ADM SPRS will ensure that the FGP communications and engagement strategies for new users are aligned with its strategic plan and that communications activities maximize participation in the Platform. The ADM SPRS will analyze lessons learned from onboarding new users to identify the key elements which successfully supported user engagement and identify training requirements. The findings of this analysis will be used to adjust, as appropriate, the FGP communications strategy.

Position responsible: Director of FGP

Timing:  ADM approval of FGP communications and engagement strategy for new users – January 31, 2020

Management agrees with Recommendation 3. 

In response to recommendation 3, the ADM SPRS will review the established stakeholder feedback mechanisms to ensure it is obtained from both current and potential users, and establish clear procedures to ensure that feedback is regularly analyzed and included in its performance and risk framework to inform continuous improvement of the Platform.

Position responsible: Director of FGP

Timing:  ADM approval of FGP stakeholder feedback mechanism – January 31, 2020

Management agrees with Recommendation 4. 

In response to recommendation 4, the ADM SPRS will ensure the Data Asset Management Plan provides timely reporting of the monitoring and tracking activities to the appropriate stakeholders through the Annual Data Asset Report. The plan and annual report will be aligned with the Platform’s role supporting the GoC Data Strategy Roadmap. The ADM SPRS will report on the Data Asset Report annually.

Position responsible: Director of FGP

Timing:  ADM approval of FGP Data Asset Management plan – January 31, 2020
ADM approval of Annual Data Asset Report to FGP Board of Directors – March 27, 2020

Information Technology Management

Summary Finding

The FGP was created with the objective of facilitating open access to Government of Canada geospatial data and to reduce barriers in sharing of data. Before the current Shared Services Canada (SSC) hosted platform (also referred to as the on-premise FGP solution) was implemented in 2017, NRCan determined that it would not efficiently meet the need for infrastructure that can be continually expandable and adaptable to demands as well as to accommodate new and improved business models and innovative pursuits.  Given the considerable time required for procurement and the need for agility and flexibility, NRCan has decided to move the complete FGP solution (software, infrastructure and platform) to the Cloud. The audit also identified opportunities to improve the required security controls within the existing FGP platform.

Supporting Observations

Effective management of IT is critical to achieving the objective of a common “platform” for information sharing. The audit team expected that NRCan has effectively developed and implemented plans to manage information technology to support and achieve the objectives of the FGP.

Current State

The FGP platform went into production in April 2017. In the simplest terms, the FGP is an open data platform that makes accessible, in standard interoperable format, geospatial data and technology from numerous departments and agencies. The FGP infrastructure hosted by SSC, is responsible for providing the supporting infrastructure (e.g., data centres, networks, servers). The platform solution consists of data sets from over 20 contributing partners, the best available software and technologies, and includes a data catalogue, web services and applications, and analytical tools.

The original Security Assessment & Authorization Report (SAAR) for the FGP was prepared on
February 25, 2016. At that time, the security residual risk was rated as High, and the FGP was provided with Interim Authority to Operate (IAO) expiring on March 1, 2017, as there was not enough evidence to allow for the required security controls to be properly traced to design and operational procedures. The SAAR was reviewed in September 2017 and the IAO was extended to March 30, 2019, with the overall risks being rated Medium to High.

The SAAR, requires that 48 security controls be in place for systems accommodating Unclassified Data, with Low Availability and Low integrity (UCLL). The SAAR noted that for the FGP, 35 were assessed at partially met, 8 not met and, 5 met.  However, the audit team was not able to assess the accuracy of this information since we were not able to obtain an updated listing of controls by category that are met, not met, and partially met. At the time of the audit, the audit team was informed that due to resource constraints, staff were re-allocated to address the security controls for the future FGP cloud solution and little work was being performed on the on-premise platform. Consequently, the on-premise FGP will continue to operate with medium to high security risk until it is migrated to the cloud; and the cloud platform under development has received an Interim Authority to Operate, to be reviewed by March 31, 2020.

Future state

In 2016, before the implementation of the FGP, the Government of Canada issued its 2016-2021
IT Strategic Plan indicating that cloud computing will become the primary IT infrastructure option for the federal government moving forward. At about the same time and before the on-premise FGP solution went into production, NRCan prepared a business case indicating that the current on-premise FGP solution would not efficiently meet the FGP need for infrastructure to be continually expandable and adaptable to demand, as well as to accommodate new and improved business models and innovative pursuits. Given the considerable time required for procurement and need for agility and flexibility, NRCan has decided to move the complete FGP solution (software, infrastructure and platform) to the Cloud.

The FGP cloud solution is aligned with the current Government of Canada’s IT Strategic Plan regarding the use of enterprise and cloud-based IT services to deliver services faster, for less money, all within a climate of continued expansion and change. The Cloud Service Provider (CSP) for the FGP cloud solution is a CSP approved by SSC. At the time of the audit, the FGP contained mostly unclassified information, and no personally identifiable information.

The cloud platform is being built to meet a security assessment profile to handle unclassified information with Low integrity and Low Availability (UCLL) with consideration of the potential impact associated with the inclusion of information at the Protected B Medium integrity and Medium availability (PBMM) level. There is a large amount of GoC geospatial data being rated at the Protected B level that could be put on the platform in the future. However, currently, SSC and the TB Secretariat have not approved any CSP to handle Protected B information. The effort required to meet the PBMM security requirements vs UCLL is significant. For example, there are 48 security controls that are required for an unclassified system, compared to approximately 500 controls to meet the requirements of a Protected B system.

Risk and Impact

There is a risk that operating the FGP on-premise solution without a complete assessment of its IT controls could compromise integrity and availability of data in the system.  This may lead to an erosion of trust by the partner organizations, and impact any future cloud-based FGP solution in particular to those stakeholders who may wish to store confidential geospatial data.

Recommendation

Recommendation 5: It is recommended that the ADM, SPRS, ensure that a review of the critical security controls for the existing on-premise FGP solution is completed.

Management Response and Action Plan

Management agrees with Recommendation 5. 

In response to recommendation 5, the ADM SPRS, and NRCan CIOSB, will ensure that a review of the critical security controls for the existing on-premise FGP solution is completed.

Position responsible: Director of FGP

Timing:  ADM approval of completed review of the critical security controls – December 31, 2019

APPENDIX A – Audit Criteria

The audit objectives and criteria were developed based on the key controls set out in the TB’s Core Management Controls. The criteria guided the fieldwork and formed the basis for the overall audit conclusion.

The objective of the audit was to assess the adequacy and effectiveness of the governance and management processes supporting the administration of the FGP.

Audit Sub-Objectives	Audit Criteria
Audit Sub-Objective 1: To determine whether NRCan has established adequate and effective project management processes for the implementation of the FGP and to inform future horizontally led projects.	1.1 It is expected that a clearly defined project plan was implemented for the FGP project.
	1.2 It is expected that project performance is monitored and reported to senior management to support effective decision making concerning the FGP.
	1.3 It is expected that risks that may preclude the achievement of objectives are identified and formally addressed by management.
	1.4 It is expected that the FGP project team has documented lessons learned to inform future horizontally led projects.
Audit Sub-Objective 2: To determine whether NRCan has established adequate and effective processes to manage partnerships and address stakeholder and user needs.	2.1 It is expected that the FGP project team engages in outreach activities to promote onboarding of new users.
	2.2 It is expected that feedback from partners, users, and stakeholders identify changes to requirements and drive continuous improvement of the platform.
	2.3 It is expected that adequate lines of communication have been established with FGP partners to ensure effective collaboration in achieving program objectives.
	2.4 It is expected that roles and responsibilities for information management are assigned and monitored to support effective management of data throughout the project lifecycle.
Audit Sub-Objective 3: To determine whether NRCan has effectively managed information technology to support the FGP.	3.1 It is expected that plans to effectively manage information technology have been developed and implemented with a view of achieving the objectives of the FGP.