Audit of the Implementation of NRCan's Open Government Strategy

Presented to the Departmental Audit Committee (DAC)
April 8, 2021

TABLE OF CONTENTS

• EXECUTIVE SUMMARY
• Introduction
• Strengths
• Areas for Improvement
• Internal Audit Conclusion and Opinion
• Statement of Conformance
• INTRODUCTION
• Audit Purpose and Objectives
• Audit Considerations
• Scope
• Approach and Methodology
• Criteria
• FINDINGS AND RECOMMENDATIONS
• APPENDIX A – AUDIT CRITERIA
• APPENDIX B – OPEN GOVERNMENT GOVERANCE STRUCTURE

Executive Summary

Introduction

Canada’s 2018-2020 National Action Plan on Open Government defines Open Government as a governing culture that is aimed to foster greater openness and accountability, enhance citizen participation in policy making and service design, and create a more efficient and responsive government. The expected objective is to ensure that the Canadian public has the right level of access to documents and proceedings, and make the government more accessible to everyone.

The Treasury Board Secretariat (TBS) has established a Directive on Open Government to promote information management practices that enable the proactive and ongoing release of government information. The implementation of the Directive is intended to ensure that Canadians are able to find and use Government of Canada information and data to support accountability, to facilitate value-added analysis, to drive socio-economic benefits through data reuse, and to support meaningful public engagement with government.

Natural Resources Canada (NRCan) has specific accountabilities in contributing to the TBS Directive and Canada’s National Action Plan on Open Government. NRCan must maximize the release of open science, data, and information on the Open Government Portal, under an open and unrestrictive licence and in an accessible format. The Department has various initiatives underway to support Open Government, including: the Federal Geospatial Platform (FGP) to facilitate access to government geospatial data and reduce barriers in sharing data, as well as its partnership to the Federal Science Library (FSL), where departmental publications, reports, data sets and other content are freely available for anyone to access or download.

The Directive on Open Government took effect on October 9, 2014 and the Federal Government’s central agency TBS has continued to renew its approach to open government in subsequent years via updates to the Canada’s National Action Plan on Open Government. This continued renewal by TBS highlights the need for NRCan to continuously renew and adapt its approach to ensure alignment with Federal objectives.

According to the Open Government Partnership, the application of a transparent, accessible and open public service can meaningfully contribute in a wide variety of departments’ activities in times of disaster response and relief including the worldwide COVID-19 pandemic. The COVID-19 pandemic has further highlighted the importance of transparency so that Canadians are able to trust and understand the decisions made by their government as well as the science supporting these decisions. On April 17, 2020, the Prime Minister announced the Government’s commitment to making COVID-19 information and data proactively available on the Open Government Portal.

The objective of the audit was to assess the overall adequacy of governance, management processes and controls supporting the implementation of a department-wide plan to fulfill the Department’s Open Government obligations and further facilitate the delivery of its broader mandate.

The audit was included in the 2019-2024 Integrated Audit and Evaluation Plan, approved by the Deputy Minister on May 14, 2019.

Strengths

Overall, the Department has been an early adopter of Open Government initiatives and a key contributor to the Open Government Portal. CMSS has demonstrated information management leadership through established management processes over planning, publishing restrictions, and the planning of IT solutions to enable the Department in its goal of achieving open government objectives.

Areas for Improvement

Opportunities were identified to strengthen governance, roles and responsibilities, central coordination, and communication activities to strategically direct the Department’s Open Government initiatives. Opportunities were also identified to review existing processes and IT solutions to ensure that adequate controls and guidance have been implemented to support the implementation of the Department’s strategy; and, that sectors have adequate IT solutions to achieve open government objectives.

Internal Audit Conclusion and Opinion

In my opinion, CMSS has taken significant steps in recent years to establish governance and management processes to direct the Department’s open government initiatives, as well as implement processes and controls to support the implementation of the Department’s open government strategy. Opportunities exist to clarify and improve management processes and departmental controls and strategy to achieve the Departments’ open government objectives regarding: effective prioritization of information for release, monitoring, and IT process guidance.

Statement of Conformance

In my professional judgement as Chief Audit Executive, the audit conforms with the Institute of Internal Auditors' International Standards for the Professional Practice of Internal Auditing and the Government of Canada’s Policy on Internal Audit, as supported by the results of the Quality Assurance and Improvement Program.

Michel Gould, MBA, CPA, CIA
Chief Audit and Evaluation Executive
April 8, 2021

ACKNOWLEDGEMENTS

The audit team would like to thank those individuals who contributed to this project and, particularly employees who provided insights and comments as part of this audit.

Introduction

Canada’s 2018-2020 National Action Plan on Open Government defines Open Government as a governing culture that is aimed to foster greater openness and accountability, enhance citizen participation in policy making and service design, and create a more efficient and responsive government. The expected objective is to ensure that the Canadian public has the right level of access to documents and proceedings, and make the government more accessible to everyone.

Within Open Government, there are various facets such as open science, data, and information. Open science is ensuring scientists and science contributors in federal science-based departments and agencies make their work more accessible to the public. Open data is structured data that is machine readable, freely shared, used and built on without restrictions. Open information is unstructured information that is freely shared without restrictions.

The federal government has committed to the concept of Open Government, developing Canada’s National Action Plan on Open Government through a Multi-stakeholder Forum with six guiding principles: inclusion, gender equity, accessibility, user-centric thinking, reconciliation, and collaboration. The Treasury Board Secretariat (TBS) has also established a Directive on Open Government to promote information management practices that enable the proactive and ongoing release of government information. The implementation of the Directive is intended to ensure that Canadians are able to find and use Government of Canada information and data to support accountability, to facilitate value-added analysis, to drive socio-economic benefits through data reuse, and to support meaningful public engagement with government.

Natural Resources Canada (NRCan) has specific accountabilities in contributing to the TBS Directive and Canada’s National Action Plan on Open Government. NRCan must maximize the release of open science, data, and information under an open and unrestrictive licence and in an accessible format. The Department has various initiatives underway to support Open Government, including: the Federal Geospatial Platform (FGP) to facilitate access to government geospatial data and reduce barriers in sharing data, as well as its partnership to the Federal Science Library (FSL), where departmental publications, reports, data sets and other content are freely available for anyone to access or download. The Department must also establish and maintain a comprehensive inventory of data and information resources of business value and develop a plan for their effective release, in addition to annually updating a departmental Open Government Implementation Plan (OGIP).

In 2015, the Department created the Open NRCan initiative, which sought to develop a department-wide integrated approach to open data, science, and information. The intended outcomes were to increase the availability of NRCan data and information to Canadians, to provide opportunities to further exploit the Department’s contributions to science and technology and the federal government’s horizontal initiatives, and to prioritize the release of NRCan’s data assets based on their value to Canadians. Portal analytics show that NRCan has released the most datasets of all departments; and, for the month of April 2020, five of the top ten most viewed datasets on the portal were contributed by NRCan.

The Chief Information Officer and Security Branch (CIOSB), within the Corporate Management and Services Sector (CMSS), is responsible for coordinating the Department’s Open Government initiatives and for the collaboration with key stakeholders to oversee the creation, prioritization, publication, and implementation of the OGIP. The Office of the Chief Scientist (OCS) is responsible to ensure the alignment between open science and the Department’s open government initiatives, through collaboration and engagement across the Department.

According to the Open Government Partnership, the application of a transparent, accessible and open public service can meaningfully contribute in a wide variety of departments’ activities in times of disaster response and relief including the worldwide COVID-19 pandemic. The COVID-19 pandemic has further highlighted the importance of transparency so that Canadians are able to trust and understand the decisions made by their government as well as the science supporting these decisions. On April 17, 2020, the Prime Minister announced the Government’s commitment to making COVID-19 information and data proactively available on the Open Government Portal.

The audit was included in the 2019-2024 Integrated Audit and Evaluation Plan, approved by the Deputy Minister on May 14, 2019.

Audit Purpose and Objectives

The objective of the audit was to assess the overall adequacy of governance, management processes and controls supporting the implementation of a department-wide plan to fulfill the Department’s Open Government obligations and further facilitate the delivery of its broader mandate.

Specifically, the audit assessed whether:

• Adequate governance and management processes have been established to strategically direct the Department’s Open Government initiatives;
• Adequate processes and controls have been established to support the implementation of the Department’s Open Government strategy; and,
• IT solutions are adequately planned and in place to support the implementation of the Department’s Open Government strategy.

Audit Considerations

A risk-based approach was used in establishing the objectives, scope, and approach for this audit engagement. A summary of the key underlying potential risks that could impact the effective implementation of NRCan’s Open Government initiatives include:

• Adequate governance structures that enable and define the strategic direction, organizational transformation, and change management for the Department’s Open Government initiatives, to accomplish the objectives set out in the plan within the timelines prescribed by TBS;
• Effective planning processes to enhance two-way communication and feedback with public users as well as departmental sector management, to properly inform and drive decisions for allocation of resources to support Open Government initiatives;
• Effective processes and controls to ensure appropriate prioritization for the release of information in accordance with applicable restrictions associated with privacy, confidentiality, and security, as well as adequate monitoring and reporting of Open Government initiatives; and,
• Effective planning of Information Technology (IT) solutions to support the implementation of the Department’s Open Government initiatives.

Scope

The audit focused on the current and planned Open Government related activities and initiatives within the Department, including a limited examination of the NRCan Data Strategy as it relates to activities within the context of Open Government. The audit examined only the activities under the responsibility of NRCan, including CIOSB, OCS and the sectors involved in Open Government initiatives. All sectors were considered for examination to assess governance, planning, roles and responsibilities, feedback mechanisms, and IT solutions used to achieve NRCan’s Open Government objectives.

The audit timeline covered the period commencing with the effective date of the TB Directive on Open Government, October 9, 2014, through to March 31, 2020. However, the audit focused primarily on the period from March 31, 2018 to March 31, 2020, in order to examine recent departmental activities and processes. As such, the scope does not include any changes in processes that may have occurred in response to the COVID-19 Pandemic.

Given that FGP, intellectual property, publishing and enterprise architecture have been the subject of internal audits by the Audit and Evaluation Branch within the last five years, the scope of the audit did not specifically include steps to test these areas; however, where relevant the audit team verified progress made to address recommendations stemming from these audits as they related to Open Government. The results of previous advisory, audit, and evaluation projects on related topics were also considered by the audit team as deemed relevant.

Processes pertaining to proactive disclosure were excluded from the scope of the audit as recent and ongoing continuous audits have tested this area extensively.

Approach and Methodology

The approach and methodology followed the Institute of Internal Auditors' International Standards for the Professional Practice of Internal Auditing and the Government of Canada’s Policy on Internal Audit. These standards require that the audit be planned and performed in such a way as to obtain reasonable assurance that audit objectives are achieved. The audit included tests considered necessary to provide such assurance. Internal auditors performed the audit with independence and objectivity as defined by the International Standards for the Professional Practice of Internal Auditing.

The audit approach included the following key tasks:

• Interviews with key personnel, committee representatives, and central agency representatives;
• Review of the Department’s documentation and business processes with regards to Open Government; and,
• Various file testing and analyses pertaining to the specified criteria.

The conduct phase of this audit was substantially completed in April 2020.

Criteria

Please refer to Appendix A for the detailed audit criteria. The criteria guided the audit fieldwork and formed the basis for the overall audit conclusion.

Findings and Recommendations

Governance and Management Processes

Summary Finding

Overall, NRCan has established adequate governance and management processes to strategically direct open government initiatives, including clearly defined roles and responsibilities. CMSS and OCS have established governance structures internally and participated in intra-departmental committees to advance the Department towards the accomplishment of Federal Open Government objectives. Opportunities were identified to: improve the Department’s Open Government plans to enable the achievement of the desired outcomes, clearly define roles and responsibilities, and to improve communication and feedback between CMSS and sectors who have been delegated responsibility for various open government activities. An opportunity to improve internal monitoring processes was identified, which could provide sectors better understanding of the prioritization of the Department’s open government activities.

Supporting Observations

The development and implementation of adequate governance and management oversight processes are necessary to support the effective delivery of departmental initiatives. The audit expected that the Department would develop and implement an Open Government Implementation Plan (OGIP) per the Directive on Open Government, and that this OGIP would outline adequate governance structures and management oversight processes to support implementation of the Department’s open government strategy. In order to adequately support NRCan’s OGIP and drive continuous improvement, the audit team expected: roles, responsibilities, and accountabilities to be clearly defined and communicated; adequate communication and feedback mechanisms for sectors to be established; and effective processes established to monitor and report on the achievement of Open Government Initiatives within the Department.

Open Government Implementation Plan

The TBS Directive on Open Government requires departments to produce an OGIP annually; the OGIP is intended to outline the activities that will be undertaken to meet the requirements of the Directive on Open Government and the objectives in the National Action Plans on Open Government. OGIPs serve as a key source document for detailing each departments’ planning efforts, with integrated tracking, monitoring and reporting of open government activities. NRCan developed an OGIP for the years 2015 and 2016, with a focus on satisfying stated policy objectives and establishing the Open NRCan initiative. The most recent iteration of the OGIP (2016) was found to: be aligned with the TBS Directive on Open Government and the National Action Plan on Open Government; contain relevant information from a planning perspective; and, identify the activities and resources that contribute toward to the open government initiative.

Despite the requirement for all departments to complete and publish annually an OGIP, the audit found that NRCan had not prepared an annual OGIP since 2016; and, the OGIPs that were prepared by NRCan have not been published on TBS’s website in accordance with requirements of the Directive on Open Government at the time of the audit. The audit noted that NRCan was not the only Federal Department that did not publish annual updates to their OGIPs to the Open Government portal for the past several years. In 2018, TBS released a Guidebook for Releasing Open Government Resources, which identified that an annual iteration of an OGIP would no longer be required; however, the Directive on Open Government has not been updated to reflect this change. The audit learned through interviews with TBS personnel that TBS is working to update the Directive on Open Government and the related guidance.

When the implementation of open government activities began, the OGIP was intended to complement the Information Management (IM) Plan. However, as updates to the OGIP were no longer being maintained, it was noted the NRCan IM Plan for FY2019-20 became the primary tool used to track the Department’s contribution to its open government commitments. The 2019-20 IM Plan consolidated information on activities required per the TBS Directive on Open Government (i.e. updating the annual data inventory) as well as commitments per the National Action Plan on Open Government. While the IM Plan was found to provide a common place to track, monitor and report on open government commitments and activities, the audit learned that CMSS does not plan to develop an IM Plan for FY 2020-21 and instead will produce a plan for each service delivery, inline with the TBS change from the Policy on Management of Information Technology and Policy of Information Management. Therefore, it is not clear how the Department will plan, monitor, and track activities to support open government implementation in future years. Furthermore, it was also noted that the IM Plan contained limited information from a strategic lens regarding the enablement and advancement of open government throughout the Department. In the absence of a strategic perspective, momentum may be hindered in the efforts to facilitate an integrated and coordinated approach to open government.

Governance Structures

The 2016 OGIP identified a governance structure consisting of various IM committees with mandates designed to enable NRCan to achieve the Open Government requirements set out in the Directive on Open Government. In December 2018, CMSS renewed the IM governance structure, including the creation of the Information Management Data Committee (IMDC). IM governance committees with open government mandates include: the Information Management and Technology Committee (IMTC), the Information Management Working Group (IMWG), and the Information Management Data Committee (IMDC). (See Appendix B for the current structure.)

The IMDC, a Director-level committee, is responsible for articulating the future state of information and data governance across the Department in order to ensure an enterprise-wide approach that aligns to the Government of Canada’s data strategy. The IMDC’s mandate is to provide a central governance body to act as a forum for collaboration and decision-making across the Department; the IMDC reports directly to the IMTC. It was noted that some parallels exist in terms of membership and topics discussed between the IMWG and the IMDC, and there may be an opportunity to delineate the roles and responsibilities of these committees. IMTC includes representation at the executive senior management level including the Department’s Chief Information Officer (DG, CIOSB) who serves as the co-Chair of the committee; however, the audit found that topics and discussions related to open government at IMTC were limited in nature.

Overall, the audit found that NRCan’s sectors were well represented on these committees and actively participated in the discussions of operational issues related to open government topics; however, the audit found that there was a lack of discussion regarding strategy to implement open government initiatives throughout the Department. All sectors were also well represented at Open Science governance committees, which include: the Open Science Committee and the Director General Science and Technology Committee (DG-STC). The audit found that the overall governance structure appeared to be well designed to meet the requirements established in the Directive on Open Government; however, existing governance committees could benefit from updated mandates and terms of reference to steer the Department’s open government strategy.

The audit noted that NRCan representatives participated in interdepartmental Open Science and Open Government committees. NRCan is represented on the interdepartmental Open Science DG Council, chaired by Environment and Climate Change Canada, which is responsible for monitoring and reporting compliance with Open Science related commitments identified in the National Action Plan on Open Government. Inter-departmental committees for Open Government are chaired by TBS; however, TBS-led interdepartmental working-group level meetings last took place August 2019 and TBS representatives noted that these would be re-established along with a revitalized Open Government Directive and guidance documentation.

Roles and Responsibilities

The audit team found that overall roles and responsibilities within the Department have been adequately defined and communicated through: the 2016 OGIP, the Directive on Open Government, the Directive on Service and Digital, and the draft NRCan Data Strategy. CMSS has used a decentralized model and delegated control over many open government activities to NRCan sectors including: identification of datasets, prioritization of datasets, procurement of storage and IT hosting solutions and data conversion to appropriate formats. Some additional delegation of activities to sectors, including determination of what data is of business value, may be beneficial given that sectors are the subject matter experts on their own data.

The draft NRCan Data Strategy outlines that as part of the Data Strategy Governance a new position of Chief Data Officer will be created within the Strategic Policy and Innovation Sector (SPI) to work with the Chief Information Officer and the Office of the Chief Scientist to integrate infrastructure, policy and science considerations with a common governance structure. The document was still draft at the time of the audit; however, may have an impact on NRCan’s data management principles going forward particularly as the new TBS Policy on Service and Digital comes into effect April 1, 2020.

The audit found through review of departmental documentation and interviews with open government representatives from various sectors, that while the roles and responsibilities are generally understood, there remains an opportunity for the Chief Information Officer, as NRCan’s Information Management Senior Official, to provide more centralized guidance and support in some areas. Multiple sector representatives indicated that it is not clear where they should be focussing their efforts, as a result of the Department’s decentralized approach, coupled with a lack of: current plans outlining how the Department intends to achieve its desired open government objectives, internal monitoring/reporting, and central coordination. Furthermore, this lack of central coordination within the Department has led to disparity between sectors’ maturity with regard to open government plans, processes and procedures.

Monitoring and Reporting

The audit examined the Department’s roles and requirements related to monitoring and reporting of Open Government activities. Providing senior management with regular performance reporting metrics is a key element supporting effective and timely decision-making and resource allocation. Several committees were identified as being responsible for monitoring and reporting on open government activities at NRCan.

The Open Science Committee developed an Open and Digital Science Tracker, implemented in June 2019, that is intended to track the status of NRCan’s Open Science Initiatives; including commitments referenced in the National Action Plan on Open Government. The Open and Digital Science tracker is led by the Office of the Chief Scientist (OCS), and was found to be regularly discussed and updated through the Open Science Committee. The interdepartmental Open Science DG Council produces an annual report using metrics developed by the council and reported on by several science based departments and agencies (SBDAs) including NRCan.

Overall, the audit team noted that the Department has continued to take action toward its commitments outlined in recent National Action Plan on Open Government, reporting on its progress through the 2019 Departmental IM Plan as well as the NRCan Open and Digital Science Tracker in addition to regular external reporting under the Management Accountability Framework (MAF). However, while the Department has demonstrated that they meet the requirements of the MAF and other identified external reporting requirements, the audit found that there was a lack of adequate internal monitoring and reporting within the Department. Numerous sector representatives interviewed noted that with many of the open government processes being decentralized and responsibilities being delegated to sectors there is a lack of centralized reporting that could serve to guide sectors to understand the Department’s open government priorities.

Communication and Feedback from Sectors

Given the decentralized model used for Open Government and the delegation of activities to sector management, the audit expected to see that CMSS and sector management were communicating regularly and that sectors would share feedback (e.g. best practices, challenges faced) on open government activities that might benefit the Department. The audit noted that the Department has taken initial steps to establish processes to allow for communication and feedback. When NRCan developed its OGIP in 2016, the audit found that it was communicated at various levels throughout the organization. At the working group level, activities are shared and communicated through the IMWG, which plays a role in synthesizing initiatives and ensuring that the perspectives of all sectors are taken into consideration. The audit found through both document review and inquiry that discussions at the IMWG regarding open government primarily refer to communications regarding the annual update to the data inventory. CIOSB has also established a Departmental Open Government mailbox to address communications related to processes on “as needed” basis.

Given the renewed IM governance structure, there may be an opportunity for CMSS to leverage these existing channels to obtain feedback from sector management and to communicate the future direction and advancement of Open Government at NRCan.

RISK AND IMPACT

In the absence of a centrally coordinated approach and clearly defined roles and responsibilities to plan and direct open government activities, there is a risk open government activities may not be prioritized. While NRCan has mitigated this risk to some extent through incorporating requirements of the OGIP in its most recent IM Plan, it is not clear whether this approach will be continued in future years. There also exists a risk that sectors may duplicate efforts, and best practices may not be shared.

Without appropriate planning and steering of NRCan’s open government strategy taking place at senior management level committees such as IMTC, there is a risk that NRCan might face challenges to accomplish current and future Federal objectives laid out in the National Action Plan on Open Government.

Recommendations

Recommendation 1: It is recommended that:

1. The Assistant Deputy Minister CMSS, in collaboration with the Assistant Deputy Minister SPI and the Chief Scientist OCS clearly define and communicate roles and responsibilities for the management of open government in alignment with the evolving state of digital data and governance in NRCan.
2. the Assistant Deputy Minister CMSS, in collaboration with NRCan Sectors, develop and communicate a plan and approach to direct and guide the advancement of open government activities to achieve the Department’s open government objectives.
3. the Assistant Deputy Minister CMSS, in collaboration with NRCan Sectors, develop and maintain a comprehensive planning, monitoring and reporting tool to manage and track the Sectors’ efforts in advancing open government both across the Department, and towards overall Government of Canada objectives.
4. the Assistant Deputy Minister CMSS, review the existing departmental committees responsible for open government, to ensure that their mandates include adequate strategic focus on open government priorities aligned with Federal National Action Plan objectives.

Management Response And Action Plan

Management agrees with Recommendation #1

1. The Chief Data Officer (CDO), SPI; the Chief Scientist (CS), OCS; and the Chief Information and Security Officer (CISO), CMSS; will define roles and responsibilities in the NRCan Data Strategy, this document will be stewarded through existing governance tables for approval and will be refreshed at a minimum of once every three years but ideally annually.
   
   Position responsible:CISO, CMSS; in collaboration with CDO, SPI; and CS, OCS.
   
   Timing: CDO/CS/CISO sign-off by December 2021. Stewarded through governance tables by March 31, 2022.
   
   
2. and c.
   Open Government activities are tracked through NRCan’s Information Management Plan. This plan is created on an annual basis and is circulated to the CISO, ADM CMSS and approved by the Deputy Minister. As a result of significant changeover in staff at the senior management level and working levels within the Digital Services and Libraries Division (DSLD), no IM Plan was completed for the 2020-2021 fiscal year. DSLD is currently working on the 2021-2022 version of the IM Plan, which lends considerable attention to NRCan’s commitments to Open Government and more specifically, Open Science.
   
   The IM Plan indicates the resources responsible for delivering on each Open Government activity or initiative and provides estimated timeframes for completion.
   
   Following approval of the IM Plan, all activities identified in the plan will be tracked in a progress report, which will identify dependencies, progress and what activities will continue into upcoming years.
   
   Position responsible: CISO, CMSS.
   
   Timing: DG/CISO sign-off by March 31, 2021.
   Stewarded through governance tables by December 31, 2021.

4. The Open Government team will review any changes or updates to the objectives of the Federal National Acton Plan to ensure the mandate of existing NRCan committees and working groups reflect this change and adequately promote Open Government priorities.
   
   Position responsible: Open Government team under Digital Services and Libraries.
   
   Timing: December 31, 2022.

Processes and Controls

Summary Finding

Overall, CMSS has established processes and controls to support the implementation of the Department’s open government strategy and to report externally on the Department’s progress. Opportunities were identified to strengthen and improve processes and controls relating to processes including compilation of a data inventory, prioritization of the release of that data, as well as controls to ensure that data is not published in contravention to any requirements on privacy, confidentiality, or security.

Supporting Observations

Adequate processes and controls are necessary to ensure that the Department is able to implement the Open Government strategy. Given that the Department must know what information it has before it can plan and prioritize release, the audit expected that the Department had established and maintains an accurate and complete inventory of data and information resources within the Department to effectively plan for their release. The audit also expected that adequate practices had been developed and implemented to prioritize the release of data and information within the Department and also to consider end-user needs in that prioritization. The audit team also expected that adequate and effective processes had been designed to prevent the release of information that should not be released.

Inventory of Data and Information

The TBS Directive on Open Government identifies the role of the Information Management Senior Official (IMSO) to be responsible for the development of the data inventory. At NRCan, the IMSO role is held by the Chief Information Officer (CIO) and therefore the process of identifying data and information for inclusion to the data inventory falls within CMSS; however, some data inventory activities have been delegated to the Sectors for support in meeting requirements of the Directive.

The audit team noted the Department primarily leveraged templates, tools and guidance made available by TBS to drive the annual updates to the NRCan Data Inventory. In the absence of Departmental guidance, the process for the identification and inclusion of datasets was not entirely clear to sector representatives. CMSS is responsible for the annual process to update the data inventory and works with sector representatives from the IMWG committee. These sector representatives responsible for providing their sectors’ data inventory to CMSS noted that additional guidance from CMSS is needed and could include standard definitions (e.g., what constitutes a dataset?). Sector representatives noted that CIOSB provided guidance on an ‘as needed’ basis and questions would be answered frequently through references to the terminology provided in the TBS Directives and Guidelines. Through interviews with Corporate and Sector representatives, it was recognized that room for improvement exists in terms of the level of guidance provided to the Sectors.

The audit found that the CIO has limited insight into each Sector’s entire data holdings given a decentralized model, and lack of centralized guidance, standard definitions, and corporate data management system. CMSS representatives acknowledged that the NRCan Data Inventory is not a complete listing of NRCan datasets; however, it was found that it serves as a good starting point in identifying datasets held within the Department. However, per TBS guidance, a complete departmental inventory should include a list of all datasets even if they are identified as not eligible for release. The NRCan Data Inventory was found to have been updated on an annual basis and published to the Open Government Portal as required by the Directive on Open Government. However, the audit found that for FY 2018-19, CMSS did not circulate an annual call-out to Sectors requesting an update to their listing of datasets due to competing priorities and resource limitations. While the data inventory was updated, it reflected information that was proactively provided by one Sector, raising concerns over the accuracy and completeness of NRCan’s data inventory that year. An annual call-out was completed the next year FY 2019-20.

While it was acknowledged that the Department’s Data Inventory is not complete, partially due to it being driven by manual processes, the audit found that the Department plans to replace these manual processes with the Enterprise Data Catalogue (EDC) which is currently in development. The EDC is expected to provide a corporate tool as a single source of cataloguing NRCan’s datasets. This proposal was presented at the IMWG in September 2019 and at the time of the audit, CMSS was in the early phases of exploring vendor options.

Prioritization of Information and Data

The audit expected that CMSS would have a documented process for releasing data on the Open Government portal. It was expected, per TBS guidance, that this release schedule would be based on the data inventory prioritization process that would identify what data should be released first and where resources should be directed. At the onset of the implementation of the Open Government Directive, and as outlined in the Open Government Implementation Plan (OGIP) for NRCan, the Department was found to have developed a plan to prioritize the Department’s datasets following their identification in the annual update to the data inventory. This plan included the development of a prioritization matrix, which describes how datasets are to be ranked to determine the order in which datasets are determined to be high-value based on established criteria and therefore should be published on the open government portal.

Effective prioritization for release of datasets relies on a complete and accurate listing of data and information provided by Sectors as part their submission to the annual departmental data inventory exercise. During the implementation of the open government initiative, and the subsequent year after which NRCan had created a Departmental Data Inventory, the Department made efforts to apply the prioritization matrix to identified datasets in the inventory in order to score and determine which datasets are released to the public. This resulted in the creation of a release schedule in FY2017-18 to guide the release process in a timely manner.

It was expected that public interest would be a consideration when determining what information should be prioritized for release. The NRCan Prioritization Matrix that was developed assigned a score based on the weighted ranking of the following three categories: Value (30%), Readiness (30%), and Effort (40%). Public interest was found to be included as part of ‘Value’ and accounts for 10% of the total used to prioritize datasets, and as part of this area, it considered end-user needs as well as feedback for requests from the public.

In subsequent years, despite documentation developed to guide the process of prioritizing datasets and planning for release, datasets were not prioritized using these processes. The prioritization activity was perceived by Sectors to be a constraint on resources and to have limited value. While formal application of the prioritization matrix was no longer being performed, Sectors were found to have employed their own processes in determining which datasets are of business value to the end-users and the public as well as how these datasets are prioritized. These processes were found to be functioning in the current environment; however, in the absence of documented and consistent processes in place and established guidance to produce a complete and accurate inventory, it was indicated by sector representatives that there are limited benefits to be obtained through the prioritization of an incomplete inventory.

In review of the Open Canada portal, the audit team found analytical information was available for datasets available through the portal. This information included monthly statistics on the number of datasets visited, downloads, released, and the average user rating covering the period June 2013 to May 2020. In speaking with corporate representatives within CMSS, it was stated Sectors are responsible for how they use this information in determining end-user needs. The audit noted limited information was available to determine the extent to which this analytical information is used to drive informed decision-making. Furthermore, the audit found that the Department does not have planning, monitoring or reporting mechanisms in place to assess whether Sectors are achieving their objectives with regard to planned targets of datasets released vs actual numbers of dataset released on the Open Government portal.

Privacy, Confidentiality, and Security

The Access to Information and Privacy Secretariat (ATIP) completed a revitalization of the proactive disclosure processes as part of the legislative changes that came into effect as part of Bill C-58. NRCan’s ATIP secretariat worked closely with CIOSB to implement these changes to meet the legislative changes. ATIP also organized training delivered in tandem with IM training to educate NRCan employees on what is confidential or personal information. An ATIP mailbox was set-up to allow employees to inquire about various requirements for personal, confidential or secure information. However, given that NRCan is a science-based department, it holds significantly less personal information than other departments, which may reduce the risk of private information being released. The audit did not examine the proactive disclosure processes in detail given that these have been audited through recent and regular continuous audits.

Through review of existing central agency guidance including the Policy on Service and Digital which took effect April 1, 2020, the audit noted that there has been clear guidance issued to departments that outlines how they must manage many areas of service delivery, including information and data management. This includes the maximization of the release of departmental information and data through the Open Government Portal while respecting information security, privacy, and legal concerns.

Once data is identified for release, it is the responsibility of the Business Owner to inform the CIO (referred to as the IMSO per the TBS Directive) of their intention to release data. The audit found that the Department utilized the TBS recommended Release Checklist process for Open Government which involves the submission of a mandatory “Release Checklist” to the NRCan Information Management Security Official (IMSO) by Business Owners who want to send their data to the TBS Open Government Portal. The Release Checklist is a mandatory tool designed to support decision-making when releasing datasets to the public and includes mandatory release criteria, which evaluate whether the data is eligible for release by verifying that it meets applicable restrictions associated with privacy, confidentiality, security, and format. Once the Release Checklist has been signed by the CIO or a delegate, the Business Owner is notified of their approval to post their dataset on the Open Government portal. Through review of the process and through interviews with CMSS, the audit noted that this checklist process is not gated and therefore while the checklist should be completed prior to uploading data to the TBS Open Data Portal, there is no mechanism in place to prevent NRCan employees from uploading data without completing the checklist first.

The audit also noted that no formal monitoring process exists for data and information that has been published to ensure that datasets published have received approval, or whether or not they are in fact eligible for release due to privacy, confidentiality, or security concerns. CMSS noted that Sectors are ultimately responsible for their datasets and the data they publish. The audit also noted that procedures for removing non-compliant data or information had not been developed by the department. Numerous interviewees noted that no data had been published by the Department that was identified as non-compliant; however, without adequate monitoring the Department may not be able to accurately determine this fact.

RISK AND IMPACT

In the absence of corporate level guidance and tools to guide and direct Sectors in identifying data for inclusion in the data inventory, current processes may not enable an accurate and complete landscape of NRCan’s federal data holdings. The prioritization process relies on the accuracy and completeness of the departmental data inventory and is not viewed as a value added activity, which increases the risk that the Department may not be maximizing the release for publication of its data holdings.

There is a risk that NRCan employees could knowingly or unknowingly publish NRCan data to TBS websites that may be non-compliant with existing privacy, legal, or security laws and regulations or that should not be published for other reasons.

Recommendations

Recommendation 2: It is recommended that the Assistant Deputy Minister CMSS in collaboration with the Assistant Deputy Minister SPI:

1. ensure centralized tools and guidance (including the roles of Sectors) are developed and disseminated across the Department to enable the identification of datasets, compiled according to a consistent methodology in support of a complete and accurate departmental data inventory.
2. review departmental processes for prioritization and release planning to ensure that departmental resources are focused on publishing high value datasets, and that these processes are communicated across the Department.
3. review the existing process/control activities in place to ensure the prevention of the publication of data or information that may not be published due to privacy, legal, and/or security requirements and ensure that decentralized activities receive oversight, which may include periodic monitoring.

Management Response and Action Plan

Management agrees with Recommendation #2

1. Currently, CMSS maintains the data inventory and its submission to TBS. Each Fall, the CMSS IM Planning and Policy team sends a callout to all Sectors through the Information Management and Data Committee (IMDC) membership to update the TBS Data Inventory. The team works with the various Sectors to complete the inventory and submits it to TBS.
   
   CMSS will work with OCS and SPI to clarify the roles and responsibilities in the NRCan Data Strategy and develop a plan to update tools and guidance for Sectors.
   
   Position Responsible: CISO in partnership with CS and CDO.
   
   Timing: October 31, 2022
   
   
2. NRCan releases data to the Open Government portal on a consistent basis. NRCan has an Open Government mailbox that is monitored on a daily basis for data that is ready to be released.
   
   In collaboration with SPI and OCS, CMSS will develop a communications plan, to be circulated at IMDC and approved through established governance tables, to encourage Sectors to continue to make open datasets available, focusing on the benefits for the authors, for the Department and Canadians.
   
   Position responsible: CISO with support from the CDO, SPI and the CS, OCS.
   
   Timing: December 31, 2022
   
   
3. Requirements for the checklist will be more widely circulated. Nothing prevents any employee from publishing directly to the Open Government portal without filling out the Open Government Approval Form.
   
   Position responsible: CISO, CMSS.
   
   Timing: December 31, 2022

IT Solutions

Summary Finding

Overall, the Department has planned and implemented Information Technology solutions to support its Open Government strategy. CMSS has worked together with sectors to implement various solutions, including the Federal Geospatial Platform, and continues to develop solutions to assist sectors in completing the Open Government activities for which they are responsible. Opportunities exist to improve communication of Open Government IT strategies to host the Department’s data, create comprehensive data inventories and improve process documentation for the conversion of data to acceptable and reusable formats for release.

Supporting Observations

The implementation of Open Government initiatives relies upon Information Technology (IT) and therefore the audit team expected to see that the Department has designed and planned appropriate IT solutions and infrastructure to support the established initiatives. It was also expected that the Department has established adequate and effective processes to ensure that data and information is released in accessible and reusable formats.

IT Solutions and Infrastructure

The audit examined NRCan’s planning documents to determine the extent to which the Department’s IM and IT plans are aligned with and adequately support the objectives set out in the National Action Plan on Open Government. The audit found that the Department’s IM plan was the main method used to communicate the IM/IT plans in support of Open Government objectives. Some projects on the IM plan have been developed and are operating such as the Federal Geospatial Platform, while others such as the NRCan Digital Accelerator are currently in development. The Open Science and Data Platform, which is used to support Open Science cumulative effects initiatives, launched in July 2020.

Departmental representatives noted that there are existing gaps in the available IT solutions to implement Open Government at NRCan. Furthermore, the responsibility for procuring hosting services in order to make their NRCan datasets public, has been delegated to sectors. This was identified as an issue for some sectors with limited capacity. At the time of the audit, CMSS noted that an enterprise IT solution that would allow sectors to host their data on NRCan's servers and publish their data effectively on the TBS Open Data Portal does not exist.

In 2017, the Audit and Evaluation Branch conducted an audit of the Department’s publishing activities which recommended the development and implementation of a department-wide approach to disseminating publications. The related management response and action plan for an enterprise tool which was planned to have been developed by March 2019 has not yet been completed at the time of this audit. An updated status was provided by CPS in August 2019 with a target date to have an enterprise tool in place by March 2020, this had not been implemented at the time of the audit.

Accessible and Reusable Formats

The Directive on Open Government states that departments’ Senior Information Management Officials are responsible for ensuring that open data and open information are released in accessible and reusable formats via Government of Canada websites designated by the TBS. The audit team conducted testing of NRCan’s 2019 data inventory using random sampling to test whether datasets were available and were in accessible, open, and reusable formats based on established TBS criteria. Of the 25 samples tested: 10 samples did not meet the established criteria, 5/10 of these datasets were “eligible for release” but were not available on Government of Canada websites; and, 5/10 of these datasets were listed as “not available for release” but adequate justification was not provided to preclude them from release requirements. Furthermore, 6/10 samples that did not meet the testing criteria were not held in reusable, open, or accessible formats. The 15 sampled datasets that met the testing criteria and that had been released by NRCan received an average openness^{Footnote 1} rating of 2.4 stars by the TBS openness rating scoring system. The average openness rating of the approximately 69,466 NRCan’s datasets as of June 1, 2020 was 1.01 stars compared to an average of 1.25 stars for all the Federal Governments’ datasets on the Open Data Portal. The low average openness rating for NRCan datasets is a result of a large number of the Department’s datasets being inadequately structured or having inadequate metadata and therefore receiving one star for meeting the minimum requirement of ‘being available under an open license’.

The audit found that the Department had planned to develop data conversion processes to assist sectors in reformatting data to appropriate and recommended formats as identified in the 2016 Departmental OGIP. CMSS planned to complete these processes by March 2017; however, they had not been developed at the time of the audit. Development of these procedures could help sectors increase the number of available datasets on the Open Government portal given that a large number are listed as “eligible for release” but are not currently in an open/reusable/accessible format. The audit learned through interviews with sector representatives that there were various reasons that sectors’ data holding are not released on TBS websites even if they are eligible for release. Reformatting data can require a significant amount of human resources; and prioritization processes could be used to ensure that these resources are spent reformatting unpublished datasets that will bring the maximum impact to Canadians.

RISK AND IMPACT

There is risk that departmental datasets that could be of interest to the public may remain unpublished if the Department does not provide sectors with appropriate IT solutions and centralized guidance to effectively publish their data and utilize best practices.

The Department may not be maximizing the release of its eligible data to the TBS open data portal if datasets that are eligible for release are not being converted to an open and accessible format.

Recommendations

Recommendation 3: It is recommended that the ADM CMSS, ensure that the Department’s Open Government IT strategy is communicated and that appropriate IT solutions to enable sectors to achieve the objective of maximizing the release of data, are planned, developed, and implemented.

Recommendation 4: It is recommended that the Assistant Deputy Minister, of each sector, regularly conduct a review of their data inventory to ensure that datasets that are eligible for release can be uploaded to TBS or Government of Canada websites in accordance with the Directive on Open Government after ensuring the all requirements for upload are met.

Management Response AND Action Plan

Management agrees with Recommendation #3

The CDO, the CS and the CISO will include an Open Government section in the NRCan Digital Strategy, this document will be stewarded through existing governance tables for approval and will be refreshed at a minimum of once every three years but ideally annually.

Through the Data and Analytics group, CIOSB is running the NRCan DataHub, a project that aims to put in place a unified platform for data storage, sharing, collaboration and more. The NRCan DataHub is working with a variety of clients from all Sectors to help them address their data management needs and provide a solution where the information can be continuously accessed, re-used and re-purposed for one or multiple groups.

Position responsible: CISO in partnership with the CDO and CSI.

Timing: First iteration of the Digital Strategy - CDO/CS/CISO sign-off by December 2021. Stewarded through governance tables by March 31, 2022. Ongoing

Management agrees with Recommendation #4

Currently, CMSS maintains the data inventory and its submission to TBS. There is currently no formalized process to engage senior management in this activity. This is a dependency on the creation of the CDO role.

The data inventory will be stewarded through existing governance tables for approval and will be refreshed annually. Once the inventory has been completed within the prescribed timelines, the results for each Sector will be sent for sign-off by their respective ADMs.

Position responsible: ADM, CMSS; in collaboration with all Sector ADMs.

Timing: Annually – begin Fall 2022.

APPENDIX A – Audit Criteria

The criteria were derived based on key controls set out in the Treasury Board of Canada’s (TB) Core Management Controls, in conjunction with the Treasury Board Secretariat’s (TBS) Directive on Open Government, and relevant associated policies, procedures, and directives. The criteria guided the fieldwork and formed the basis for the overall audit conclusion.

The following audit criteria were used to conduct the audit:

Audit Sub-Objectives	Audit Criteria
Sub-Objective 1: To determine whether adequate governance and management processes have been established to strategically direct the Department’s Open Government initiatives.	1.1 It is expected that the Department has established and implemented adequate governance structures to strategically direct its Open Government initiatives.
	1.2 It is expected that the Department has developed and implemented an Open Government Implementation Plan (OGIP) in support of achieving Federal Open Government initiatives and the National Action Plans on Open Government.
	1.3 It is expected that the roles, responsibilities, and accountabilities pertaining to the Department’s Open Government Strategy are clearly defined and communicated, and adequately support the OGIP.
	1.4 It is expected that the Department has established adequate mechanisms for communication and feedback from sectors to ensure effective collaboration and drive continuous improvements to the OGIP.
Sub-Objective 2: To determine whether adequate processes and controls have been established to support the implementation of the Department’s Open Government strategy.	2.1 It is expected that the Department has established adequate processes to enable sectors to effectively prioritize the publication of information and data, that considers end-user needs and supports intended OGIP objectives.
	2.2 It is expected that the Department has established adequate and effective processes to ensure that information published is managed in accordance with applicable restrictions associated with privacy, confidentiality, and security.
	2.3 It is expected that the Department has established effective processes to ensure adequate monitoring and reporting of its Open Government initiatives.
Sub-Objective 3: To determine whether IT solutions are adequately planned and in place to support the implementation of the Department’s Open Government strategy.	3.1 It is expected that the Department has designed and implemented appropriate IT solutions including infrastructure to support its Open Government initiatives.
	3.2 It is expected that the Department has established adequate and effective processes to ensure that open data and open information is released in accessible and reusable formats via the Government of Canada websites and services designated by TB.
	3.3 It is expected that the Department has established and maintains a comprehensive inventory of data and information resources of business value to determine their eligibility, and to plan for their effective release.

Appendix B – Open Government Goverance Structure

NRCan’s CMSS sector renewed the Department’s information management and technology governance structure in December 2018. The updated structure shown below contains several committees that have open government activities as part of their mandates: