Targeted Audit of NRCan’s Data Strategy
Presented to the Departmental Audit Committee (DAC)
October 24, 2023
Introduction
In 2018, the Clerk of the Privy Council (PCO) developed “A Data Strategy for the Federal Public Service” and required all Government of Canada (GoC) Departments to develop their own data strategy and treat data as a strategic asset for policy making, program design and service delivery. Natural Resources Canada (NRCan) developed its first Data Strategy by the September 2019 deadline. The NRCan Data Strategy seeks to develop the foundation for effective data management and use, to enable the Department to lead change within the Government and across the natural resources sector. In June 2022, Canada’s Chief Data Officer formally announced a renewal of the federal government’s data strategy, requiring all GoC Departments to update their strategies. This renewal builds upon the previous Data Strategy direction and is intended to support the improvement of departmental data management practices. During the conduct of the audit, the 2023-2026 Data Strategy for the Federal Public Service was released in May 2023, and it is anticipated that the associated implementation roadmap will be released in the Summer of 2023, after the completion of the audit conduct phase. This audit focused on examining current structure and processes in place to support the department in meeting objectives of the renewed GOC strategy.
About the Audit
Objective
The objective of this audit was to assess the adequacy and effectiveness of specific governance structures, processes, and activities supporting the development and implementation of NRCan’s Data Strategy, in anticipation of the GoC Data Strategy renewal.
Scope
The scope of this audit focused on activities, practices and procedures surrounding the Data Strategy, between January 2022 and July 2023. The audit examined governance and oversight mechanisms, communication, and monitoring processes in place. The audit also considered the processes and activities involved in the adoption of new Treasury Board Secretariat guidance and direction relating to the renewal of the Data Strategy.
In light of the GoC Data Strategy renewal initiative, the audit did not examine whether the objectives of the 2019 NRCan Data Strategy were met, but focused on current structures and processes in anticipation of the renewal. The audit did not explicitly examine the Departmental data that is created, collected, and managed, or aspects of the Open Government Initiative. In addition, the audit did not examine intellectual property, or the security of scientific data, due to planned future audit coverage in these areas.
Audit Considerations
A risk-based approach was used to establish the engagement’s objectives, scope, and approach. A
summary of key underlying risks identified during the planning phase include:
Risk Area 1 – Governance and Strategic Direction
- Adequate and effective governance and oversight mechanisms are necessary to provide strategic and operational direction to enable the achievement of outcomes.
- Adequate organizational structure and position of the Chief Data Officer and staff, to effectively enable the overall advancement of the Data Strategy and the attainment of its objectives.
Risk Area 2 – Performance Measurement, Monitoring and Reporting
- Adequate business processes for monitoring performance are necessary to support the implementation of the Data Strategy, and to measure, track, and report on progress in support of Data Strategy objectives and implementation.
Risk Area 3 – Communication, Collaboration, and Information Sharing
- Adequate and effective mechanisms are needed to enable and support communication, collaboration, and the sharing of relevant and timely key information for the purposes of effective decision-making.
Approach and Methodology
The approach and methodology followed the Institute of Internal Auditors’ International Standards for the Professional Practice of Internal Auditing and the Treasury Board Policy on Internal Audit. The audit included tests considered necessary to provide such assurance. Internal auditors performed the audit with independence and objectivity as defined by the International Standards for the Professional Practice of Internal Auditing.
The audit included the following key tasks:
- Interviews with key personnel: Chief Data Officer (CDO), Chief Information Officer (CIO), Chief Scientific Data Officer (CSDO) and sector representatives;
- Review of key documents and business processes; and
- Conduct of a comparative analysis of governance structures and implementation activities of other
federal government departments (OGDs). OGDs selected were primarily science-based and / or
data-driven departments of various sizes.
The conduct phase of this audit was substantially completed in July 2023.
Please refer to Appendix A for the detailed audit criteria.
Results of the Audit
Strengths
Overall, the audit found that the Department has taken actions to progress the Data Strategy and the 2022-2023 implementation plan, with advancements made to promote data literacy and skills, in addition to the development and pilot of a functional review component for data.
The CDO office has also raised awareness relating to the 2023-2026 Data Strategy for the Federal Public Service renewal initiative and the changes required, and is proactively engaging with sectors and key governance committees on its potential impacts.
Areas for Improvement
The audit found a need to clarify roles, responsibilities, and accountabilities for the data strategy, as well as to strengthen existing governance and oversight structures, to ensure they provide sufficient oversight, visibility, and direction. Opportunities exist to enhance existing communication and collaboration processes, mechanisms, and channels to enable and support integration between policy, IM/IT, scientific domains, and Sectors.
In light of the 2023-2026 Data Strategy for the Federal Public Service Renewal, there is a need to develop processes to track, monitor and report on the overall progress and implementation of data strategy activities, as well as supporting mechanisms to identify, collect and share information to inform and direct decision-making activities.
Conclusion
In my opinion, NRCan has some elements of governance structures and activities in place that support the development and implementation of its data strategy, with consideration given to the 2023-2026 renewal of the Data Strategy for the Federal Public Service. The audit identified areas of improvement to clarify and strengthen governance and communication mechanisms, and performance measurement activities.
Statement of Conformance
In my professional judgement as Chief Audit and Evaluation Executive (CAEE), the audit conforms with the Institute of Internal Auditors' International Standards for the Professional Practice of Internal Auditing and the Government of Canada’s Policy on Internal Audit, as supported by the results of the Quality Assurance and Improvement Program.
Michel Gould, MBA, CPA, CIA
Chief Audit and Evaluation Executive
October 24, 2023
Audit Findings and Recommendations
| Key Findings | Recommendations and Management Response | |
|---|---|---|
| Governance and Communication Mechanisms | Audit Criteria (CR) | |
Governance and Oversight Structures The audit examined whether governance and oversight structures were clearly defined and provide strategic and operational direction with respect to advancing departmental data strategy efforts. The 2019 NRCan Data Strategy (DS) identifies a triadic DS governance leadership model comprised of the Chief Data Officer (CDO), Chief Information Officer (CIO) and Chief Scientific Data Officer (CSDO) (formerly known as the Chief Science Advisor (CSA)) along with the respective governance committees they chair. These governance committees include the DG-level Policy Committee (DG-PC), Information Management and Technology Committee (DG-IMTC) and the Science and Technology Committee (DG-STC), all of which are identified as linking to an ADM level joint leadership committee. In reviewing their Terms of Reference (TORs), along with those of committees with key linkages, it was noted that their mandates lack explicit mention of the DS and its implementation. Through a detailed review of the terms of reference (TORs) of these governance committees and meeting materials, the audit found that CDO, CIO and CSDO were not collectively represented as core members among the governance structures identified for the DS, resulting in the absence of a dedicated forum that includes their combined participation. Additionally, it was noted that these main committees lacked expected linkages and interconnections to one another, highlighting a gap in the integration of the department’s data strategy governance. This underscores the lack of clarity in the stated governance structure and missed opportunities for synergies and alignment between policy, IT, and scientific domains on activities surrounding the DS. The review of meeting materials found that these governance structures were not effectively utilized as mechanisms to provide strategic and operational direction on the DS, as a result, limited input was provided to guide and direct activities and decision-making on this initiative. For example, relevant information related to the DS that was shared at senior-level committees was infrequent and ad hoc in nature, and discussions focused primarily on information sharing. Limited focus on the DS was noted by the audit team across all committees, limiting the overall effectiveness of existing governance structures. Interviews with sector representatives indicated that operational and working level committees were widely regarded as effective facilitators of meaningful exchanges on data-related matters, although topics of discussions did not specifically address the DS. The audit team engaged with select Other Government Departments (OGDs) to perform a comparative analysis that examined the governance structures and oversight mechanisms in place to support their respective data strategies. As noted above, data strategy discussions at NRCan occur at various governance bodies, with an absence of a lead or dedicated governance forum to provide oversight, strategic, and operational direction on the DS. OGDs indicated that they have established data governance to oversee and support the implementation and advancement of their respective data strategy activities. These structures encompass both vertical and horizontal flows of communication, coordination, and decision-making with engagement at all levels, and was recognized to be a foundational element contributing to their achievements and driving their successes. The audit noted the organizational position of the CDO is important for effective data strategy implementation. As the Department prepares for a review of its existing DS to align it with the GoC Data Strategy renewal initiative, there is an opportunity to confirm whether the organizational position of the CDO is aligned with the requirements of the DS. The audit team observed that most of the OGDs consulted had reviewed the organizational position of their CDO offices, and in some cases had moved this position to different areas of the organization. Risk and Impact The absence of formalized governance mechanisms to oversee, direct, and guide departmental data strategy activities may result in challenges to advance and sustain data-related initiatives and their overarching goals, objectives, and priorities. Without proper governance and oversight structures in place that provide strategic and operational direction, efforts may result in the adoption of fragmented or siloed approaches, leading to ineffective actions throughout the organization, impeding its ability to drive future successes and long-term sustainability. In addition, the absence of a departmental data governance structure may further result in potential gaps, overlaps and a lack of accountability, inherent through the triadic governance leadership approach, and impede potential synergies and alignment between Sectors, policy, IM/IT, and scientific domains. The organizational positioning of the CDO is important to ensure the effective implementation of the DS. |
CR 1.2 | Recommendation 1: It is recommended that ADM SPIS, in collaboration with the ADM CMSS and the Chief Scientific Data Officer OCS:
Management response: Management agrees. In response to Recommendation 1, SPI-SPB will ensure existing governance structures are reviewed and updated to provide clear strategic oversight and operational direction for the development and implementation of the renewed NRCan data strategy. This will be done in the context of the broader data environment at NRCan, where the Chief Data Officer, Chief Scientific Data Officer (Chief Scientist), and Chief Information Officer work together to advance an enterprise approach to data. To support this, in connection with work to renew the NRCan Data Strategy and develop the accompanying action plan, and in collaboration with the Chief Scientific Data Officer and the Chief Information Officer, SPI-SPB will:
Position responsible: SPIS ADM and DG SPI-SPB, in collaboration with ADM CMSS and Chief Scientific Data Officer. Timing:
|
Roles, Responsibilities and Accountabilities The audit examined whether roles, responsibilities, and accountabilities for the CDO, CIO, CSDO and sectors were clearly defined, documented, and communicated. Although the 2019 NRCan DS is the main document through which roles, responsibilities and accountabilities are documented, overall, the audit noted that roles, responsibilities and accountabilities were not clearly defined, adequately documented or communicated to relevant parties. The audit noted that there are no recent, finalized and agreed upon roles among these key parties. Interviews with sector representatives indicated that many were not aware whether the DS had been approved, potentially furthering confusion regarding the roles, responsibilities, and ultimate accountability for its implementation. Documentation reviewed by the audit team did not define the roles, responsibilities, and accountabilities beyond the CDO, CIO and CSDO. This is further corroborated through the offices of the CDO, CIO, OCS and select sectors, who noted a need to clarify the roles, responsibilities and accountabilities of all parties going forward. In addition, based on a review of relevant TORs and meeting minutes, the audit found that governance committees (detailed above) that were intended to provide oversight of the DS do not outline clear roles, responsibilities, and accountabilities for DS implementation. The audit team noted that this challenge is recognized within the 2023-2026 Data Strategy for the Federal Public Service renewal initiative, and that there will be efforts to clarify data leadership responsibilities across the Government of Canada (GoC). This includes establishing clear accountabilities for data in planning processes to ensure that individuals with data responsibilities are providing input to programs, policies, and services as they are being established and managed. In addition, Treasury Board Secretariat is set to develop guidance on the roles and responsibilities of executives with data accountabilities, review the policy suite, and clarify roles, responsibilities and interconnections between existing data bodies and activities. Risk and Impact Without clearly defined, documented, and communicated roles, responsibilities and accountabilities, the Department may experience duplication of and / or gaps in the roles and responsibilities of key DS players and senior management, with this lack of clarity extending to all Staff. This lack of clear accountability may result in inactions and impediments to advancing the Strategy and its implementation plan, impacting the Department’s success in the strategic management of data and position as a data leader within the GoC. |
CR 1.1 |
Recommendation 2: It is recommended that ADM SPIS, in collaboration with the ADM CMSS and the Chief Scientific Data Officer OCS:
Management response: Management agrees. In response to Recommendation 2, SPI-SPB will formalize the renewal of NRCan’s enterprise Data Strategy and the development of its action plan. It will clearly define, document and communicate roles, responsibilities and accountabilities. To support this, in collaboration with the Chief Scientific Data Officer and the Chief Information Officer, SPI-SPB will:
Position Responsible: SPIS ADM and DG SPI-SPB, in collaboration with ADM CMSS and Chief Scientific Data Officer. Timing:
|
| Governance and Communication Mechanisms | Audit Criteria (CR) | |
Communication and Collaboration Mechanisms The audit examined whether mechanisms were in place to enable clear and consistent communication and collaboration across relevant parties. Overall, the audit found that mechanisms in place in support of communication and collaboration may not consistently enable clear communication and collaboration across all relevant parties in relation to the DS. Through the review of the 2019 NRCan DS and relevant governance committee documentation, the audit noted that formal communication mechanisms include the DG-PC, the DG-IMTC, the DG-STC, the Information Management and Data Committee and occasionally the Planning and Reporting Committee. Informal mechanisms include regular bi-lateral meetings between the CDO, CIO, and the CSDO and working-level relationships among staff in these offices and sectors. The audit found that there are gaps within the mechanisms employed, including:
Documentation reviewed demonstrated efforts led by the CDO and their office to engage with Sectors and discuss the DS during the time that the 2022-2023 NRCan DS Implementation Plan was developed, and more recently with the 2023-2026 Data Strategy for the Federal Public Service renewal. The CDO office noted that they intend to create clearer lines of communication to ensure that all parties have regular updates through developing and implementing a distribution list and holding regular meetings with sectors to discuss the renewal. Risk and Impact There is a risk that inadequate communication and collaboration mechanisms may not support engagement with the Sectors on the data strategy and its implementation, potentially leading to missed opportunities to share and leverage work in other areas of the organization. In addition, the absence of a cohesive approach among policy, IM/IT and science domains may contribute towards the adoption of misaligned or siloed approaches, limit DS advancement, and ultimately undermine the importance and overall value of the DS to the department. |
CR 3.1 |
No formal recommendation issued. Through recommendations 1a and 2b, it is anticipated that communication and collaboration issues will be addressed. See management response to recommendations 1a and 2b. |
| Performance Measurement and Information for Decision-Making | Audit Criteria (CR) | |
Performance Measurement and Information for Decision Making The audit examined whether the Department has established adequate processes to track, monitor, and report on the progress of DS objectives. The audit found an absence of established and formalized processes to track, monitor and report on the progress of the DS or its implementation plan, with ad hoc tracking and monitoring occurring for the activities identified within the 2022-2023 DS Implementation Plan. The audit also examined whether relevant and timely information is shared with key parties, including senior leadership, for the purposes of informing effective decision making. The audit findings revealed that formalized processes to identify, collect, compile, and disseminate information are not in place, and that there are no formalized reports to distribute key information to relevant parties for the purposes of informing decision-making activities. Furthermore, there is an absence of clearly defined measurement criteria for how to assess progress, impacts and successes. This is a key factor required to enable supporting mechanisms and processes to collect, compile and disseminate information for decision-making purposes. The CDO office has provided presentations to key governance committees on the anticipated changes arising from the 2023-2026 Data Strategy for the Federal Public Service renewal and its implementation plan. As per the 2023-2036 Data Strategy for the Federal Public Service, these changes include the requirement to establish, monitor and report on key performance indicators and metrics to assess progress and report on overall outcomes, with a quarterly tracking and status updates and annual reporting on results to TB. In addition, the plan identifies requirements for departments to embed data related activities at the onset of initiatives, including TB Submissions, as well as to strength data literacy, with on-going progress and efforts observed within the Department. Risk and Impact In the absence of processes and tools to support and enable tracking, monitoring, and reporting on the DS performance, as well as its implementation, the Department may be unable to measure the progress and success of its activities. This may impede its ability to focus on areas in need of attention, take course corrective actions to achieve its intended outcomes, and address emerging risks, which may adversely affect the Departments ability to attain and sustain its related goals and priorities. Without a formalized approach to share key information and updates with relevant stakeholders on progress, the Department may also miss opportunities to leverage key information to informed decision-making, impacting the overall efficiency and effectiveness of its activities and implementation of the DS. This may also result in inconsistent or conflicting decision-making that take place at other levels within the organization, hindering the cohesive execution of the strategy. |
CR 2.1 |
Recommendation 3: It is recommended that ADM SPIS, in collaboration with the ADM CMSS and the Chief Scientific Data Officer OCS:
Management response: Management agrees. In response to Recommendation 3, SPI-SPB will establish a formalized process to track, monitor and report on progress related to the renewed NRCan enterprise Data Strategy, and any requirements linked to the 2023-2026 Federal Data Strategy. To support this, in collaboration with the Chief Scientific Data Officer and the Chief Information Officer, SPI-SPB will:
Position Responsible: SPIS ADM and DG SPI-SPB, in collaboration with ADM CMSS and Chief Scientific Data Officer. Timing:
|
Appendix A – Audit Criteria
The criteria were developed based on key controls set out in the Treasury Board of Canada’s Audit Criteria related to the Management Accountability Framework – A Tool for Internal Auditors, in conjunction with key risks identified by NRCan’s personnel who were consulted in the Planning Phase. The criteria guided the fieldwork and formed the basis for the overall audit conclusion.
| Audit Objective: | |
|---|---|
| Audit Sub-Objectives | Audit Criteria |
| Audit Sub-Objective 1: To determine whether the Department has adequate and effective governance mechanisms in place to support the sustainability of NRCan’s Data Strategy, its implementation, and its future renewal. |
1.1 It is expected that roles, responsibilities, and accountabilities of the CDO, CIO, CSDO and Sectors are clearly defined, documented, and communicated. |
| 1.2 It is expected that governance and oversight structures are clearly defined and provide strategic and operational direction with respect to advancing departmental data strategy efforts. | |
| Audit Sub-Objective 2: To determine whether the Department has adequate processes in place to track, monitor and report on progress towards the achievement of current and future Data Strategy objectives. |
2.1 It is expected that the Department has established adequate processes to track, monitor, and report on the progress of Data Strategy objectives. |
| Audit Sub-Objective 3: To determine whether the Department has adequate and effective mechanisms in place to enable communication, collaboration and information sharing, across relevant parties as it relates to the current and future Data Strategy. |
3.1 It is expected that mechanisms are in place to enable clear and consistent communication and collaboration across relevant parties. |
| 3.2 It is expected that relevant and timely information is shared with key parties, including senior leadership, for the purposes of informing effective decisions making activities. | |
Page details
- Date modified: