Language selection

Search


Targeted Audit of NRCan’s Data Strategy

Presented to the Departmental Audit Committee (DAC)
October 24, 2023

Introduction

In 2018, the Clerk of the Privy Council (PCO) developed “A Data Strategy for the Federal Public Service” and required all Government of Canada (GoC) Departments to develop their own data strategy and treat data as a strategic asset for policy making, program design and service delivery. Natural Resources Canada (NRCan) developed its first Data Strategy by the September 2019 deadline. The NRCan Data Strategy seeks to develop the foundation for effective data management and use, to enable the Department to lead change within the Government and across the natural resources sector. In June 2022, Canada’s Chief Data Officer formally announced a renewal of the federal government’s data strategy, requiring all GoC Departments to update their strategies. This renewal builds upon the previous Data Strategy direction and is intended to support the improvement of departmental data management practices. During the conduct of the audit, the 2023-2026 Data Strategy for the Federal Public Service was released in May 2023, and it is anticipated that the associated implementation roadmap will be released in the Summer of 2023, after the completion of the audit conduct phase. This audit focused on examining current structure and processes in place to support the department in meeting objectives of the renewed GOC strategy.

About the Audit

Objective

The objective of this audit was to assess the adequacy and effectiveness of specific governance structures, processes, and activities supporting the development and implementation of NRCan’s Data Strategy, in anticipation of the GoC Data Strategy renewal.

Scope

The scope of this audit focused on activities, practices and procedures surrounding the Data Strategy, between January 2022 and July 2023. The audit examined governance and oversight mechanisms, communication, and monitoring processes in place. The audit also considered the processes and activities involved in the adoption of new Treasury Board Secretariat guidance and direction relating to the renewal of the Data Strategy.

In light of the GoC Data Strategy renewal initiative, the audit did not examine whether the objectives of the 2019 NRCan Data Strategy were met, but focused on current structures and processes in anticipation of the renewal. The audit did not explicitly examine the Departmental data that is created, collected, and managed, or aspects of the Open Government Initiative. In addition, the audit did not examine intellectual property, or the security of scientific data, due to planned future audit coverage in these areas.

Audit Considerations

A risk-based approach was used to establish the engagement’s objectives, scope, and approach. A
summary of key underlying risks identified during the planning phase include:

Risk Area 1 – Governance and Strategic Direction

  • Adequate and effective governance and oversight mechanisms are necessary to provide strategic and operational direction to enable the achievement of outcomes.
  • Adequate organizational structure and position of the Chief Data Officer and staff, to effectively enable the overall advancement of the Data Strategy and the attainment of its objectives.

Risk Area 2 – Performance Measurement, Monitoring and Reporting

  • Adequate business processes for monitoring performance are necessary to support the implementation of the Data Strategy, and to measure, track, and report on progress in support of Data Strategy objectives and implementation.

Risk Area 3 – Communication, Collaboration, and Information Sharing

  • Adequate and effective mechanisms are needed to enable and support communication, collaboration, and the sharing of relevant and timely key information for the purposes of effective decision-making.

Approach and Methodology

The approach and methodology followed the Institute of Internal Auditors’ International Standards for the Professional Practice of Internal Auditing and the Treasury Board Policy on Internal Audit. The audit included tests considered necessary to provide such assurance. Internal auditors performed the audit with independence and objectivity as defined by the International Standards for the Professional Practice of Internal Auditing.

The audit included the following key tasks:

  • Interviews with key personnel: Chief Data Officer (CDO), Chief Information Officer (CIO), Chief Scientific Data Officer (CSDO) and sector representatives;
  • Review of key documents and business processes; and
  • Conduct of a comparative analysis of governance structures and implementation activities of other
    federal government departments (OGDs). OGDs selected were primarily science-based and / or
    data-driven departments of various sizes.

The conduct phase of this audit was substantially completed in July 2023.

Please refer to Appendix A for the detailed audit criteria.

Results of the Audit

Strengths

Overall, the audit found that the Department has taken actions to progress the Data Strategy and the 2022-2023 implementation plan, with advancements made to promote data literacy and skills, in addition to the development and pilot of a functional review component for data.

The CDO office has also raised awareness relating to the 2023-2026 Data Strategy for the Federal Public Service renewal initiative and the changes required, and is proactively engaging with sectors and key governance committees on its potential impacts.

Areas for Improvement

The audit found a need to clarify roles, responsibilities, and accountabilities for the data strategy, as well as to strengthen existing governance and oversight structures, to ensure they provide sufficient oversight, visibility, and direction. Opportunities exist to enhance existing communication and collaboration processes, mechanisms, and channels to enable and support integration between policy, IM/IT, scientific domains, and Sectors.

In light of the 2023-2026 Data Strategy for the Federal Public Service Renewal, there is a need to develop processes to track, monitor and report on the overall progress and implementation of data strategy activities, as well as supporting mechanisms to identify, collect and share information to inform and direct decision-making activities.

Conclusion

In my opinion, NRCan has some elements of governance structures and activities in place that support the development and implementation of its data strategy, with consideration given to the 2023-2026 renewal of the Data Strategy for the Federal Public Service. The audit identified areas of improvement to clarify and strengthen governance and communication mechanisms, and performance measurement activities.

Statement of Conformance

In my professional judgement as Chief Audit and Evaluation Executive (CAEE), the audit conforms with the Institute of Internal Auditors' International Standards for the Professional Practice of Internal Auditing and the Government of Canada’s Policy on Internal Audit, as supported by the results of the Quality Assurance and Improvement Program.

Michel Gould, MBA, CPA, CIA
Chief Audit and Evaluation Executive
October 24, 2023

Audit Findings and Recommendations

Key Findings Recommendations and Management Response
Governance and Communication Mechanisms Audit Criteria (CR)  

Governance and Oversight Structures

The audit examined whether governance and oversight structures were clearly defined and provide strategic and operational direction with respect to advancing departmental data strategy efforts.

The 2019 NRCan Data Strategy (DS) identifies a triadic DS governance leadership model comprised of the Chief Data Officer (CDO), Chief Information Officer (CIO) and Chief Scientific Data Officer (CSDO) (formerly known as the Chief Science Advisor (CSA)) along with the respective governance committees they chair. These governance committees include the DG-level Policy Committee (DG-PC), Information Management and Technology Committee (DG-IMTC) and the Science and Technology Committee (DG-STC), all of which are identified as linking to an ADM level joint leadership committee. In reviewing their Terms of Reference (TORs), along with those of committees with key linkages, it was noted that their mandates lack explicit mention of the DS and its implementation.

Through a detailed review of the terms of reference (TORs) of these governance committees and meeting materials, the audit found that CDO, CIO and CSDO were not collectively represented as core members among the governance structures identified for the DS, resulting in the absence of a dedicated forum that includes their combined participation. Additionally, it was noted that these main committees lacked expected linkages and interconnections to one another, highlighting a gap in the integration of the department’s data strategy governance. This underscores the lack of clarity in the stated governance structure and missed opportunities for synergies and alignment between policy, IT, and scientific domains on activities surrounding the DS.

The review of meeting materials found that these governance structures were not effectively utilized as mechanisms to provide strategic and operational direction on the DS, as a result, limited input was provided to guide and direct activities and decision-making on this initiative. For example, relevant information related to the DS that was shared at senior-level committees was infrequent and ad hoc in nature, and discussions focused primarily on information sharing. Limited focus on the DS was noted by the audit team across all committees, limiting the overall effectiveness of existing governance structures. Interviews with sector representatives indicated that operational and working level committees were widely regarded as effective facilitators of meaningful exchanges on data-related matters, although topics of discussions did not specifically address the DS.

The audit team engaged with select Other Government Departments (OGDs) to perform a comparative analysis that examined the governance structures and oversight mechanisms in place to support their respective data strategies. As noted above, data strategy discussions at NRCan occur at various governance bodies, with an absence of a lead or dedicated governance forum to provide oversight, strategic, and operational direction on the DS. OGDs indicated that they have established data governance to oversee and support the implementation and advancement of their respective data strategy activities. These structures encompass both vertical and horizontal flows of communication, coordination, and decision-making with engagement at all levels, and was recognized to be a foundational element contributing to their achievements and driving their successes.

The audit noted the organizational position of the CDO is important for effective data strategy implementation. As the Department prepares for a review of its existing DS to align it with the GoC Data Strategy renewal initiative, there is an opportunity to confirm whether the organizational position of the CDO is aligned with the requirements of the DS. The audit team observed that most of the OGDs consulted had reviewed the organizational position of their CDO offices, and in some cases had moved this position to different areas of the organization.

Risk and Impact

The absence of formalized governance mechanisms to oversee, direct, and guide departmental data strategy activities may result in challenges to advance and sustain data-related initiatives and their overarching goals, objectives, and priorities. Without proper governance and oversight structures in place that provide strategic and operational direction, efforts may result in the adoption of fragmented or siloed approaches, leading to ineffective actions throughout the organization, impeding its ability to drive future successes and long-term sustainability. In addition, the absence of a departmental data governance structure may further result in potential gaps, overlaps and a lack of accountability, inherent through the triadic governance leadership approach, and impede potential synergies and alignment between Sectors, policy, IM/IT, and scientific domains.

The organizational positioning of the CDO is important to ensure the effective implementation of the DS.

CR 1.2

Recommendation 1:

It is recommended that ADM SPIS, in collaboration with the ADM CMSS and the Chief Scientific Data Officer OCS:

  1. Review and update existing departmental data strategy governance structures to ensure their mandates and operations provide clear strategic oversight and operational direction, meet at regular intervals, and align and support the renewed data strategy and its upcoming implementation plan; and
  2. Determine if the organizational position of the CDO aligns and supports the renewed data strategy and its upcoming implementation plan.

Management response:

Management agrees.

In response to Recommendation 1, SPI-SPB will ensure existing governance structures are reviewed and updated to provide clear strategic oversight and operational direction for the development and implementation of the renewed NRCan data strategy. This will be done in the context of the broader data environment at NRCan, where the Chief Data Officer, Chief Scientific Data Officer (Chief Scientist), and Chief Information Officer work together to advance an enterprise approach to data.

To support this, in connection with work to renew the NRCan Data Strategy and develop the accompanying action plan, and in collaboration with the Chief Scientific Data Officer and the Chief Information Officer, SPI-SPB will:

  • Map key interconnections between data-related work at NRCan, including between the department’s enterprise data strategy renewal, other data-related strategies (e.g., scientific data, digital, clean tech, EDI), and against requirements of 2023-2026 Federal Data Strategy.
  • Review existing internal and external governance and oversight structures and develop strategy to update structures to address identified gaps, leveraging existing committees where appropriate.
    • Consider both governance needed to support the development of the renewed strategy and action plan, as well as governance required to support ongoing implementation.
    • Define roles, responsibilities and accountabilities of key internal partners (e.g., Chief Data officer, Chief Scientific Data Officer, Chief Information Officer), and internal and external contributors and decision-making bodies, in alignment with any TBS guidance on roles and responsibilities for executives with data accountabilities and the GC Policy Suite.
  • Develop options and recommendations related to the placement of the CDO within the NRCan organizational structure, including ongoing resources required for the implementation of the renewed Data Strategy.

Position responsible: SPIS ADM and DG SPI-SPB, in collaboration with ADM CMSS and Chief Scientific Data Officer.

Timing:

  • By the end of Q3 2023-2024
    • Map key requirements/ongoing NRCan data-related work
    • Establish governance approach to support renewal of enterprise Data Strategy and drafting of action plan
    • Develop proposal for revised governance and oversight framework to support implementation of action plan
  • By the end of Q4 2023-2024
    • Finalize ongoing governance structure as part of the renewed NRCan enterprise Data Strategy and Action Plan
    • Develop options and recommendations related to placement of the CDO within the NRCan organizational structure, and approach for ongoing resourcing related to the implementation of the renewed NRCan Data Strategy
  • Q1 2024-2025
    • Seek formal approval of the renewed NRCan enterprise Data Strategy and Action Plan, including governance structure, placement of CDO within NRCan organizational structure, and approach for ongoing resourcing related to the implementation of the renewed NRCan Data Strategy

Roles, Responsibilities and Accountabilities

The audit examined whether roles, responsibilities, and accountabilities for the CDO, CIO, CSDO and sectors were clearly defined, documented, and communicated. Although the 2019 NRCan DS is the main document through which roles, responsibilities and accountabilities are documented, overall, the audit noted that roles, responsibilities and accountabilities were not clearly defined, adequately documented or communicated to relevant parties. The audit noted that there are no recent, finalized and agreed upon roles among these key parties. Interviews with sector representatives indicated that many were not aware whether the DS had been approved, potentially furthering confusion regarding the roles, responsibilities, and ultimate accountability for its implementation. Documentation reviewed by the audit team did not define the roles, responsibilities, and accountabilities beyond the CDO, CIO and CSDO. This is further corroborated through the offices of the CDO, CIO, OCS and select sectors, who noted a need to clarify the roles, responsibilities and accountabilities of all parties going forward. In addition, based on a review of relevant TORs and meeting minutes, the audit found that governance committees (detailed above) that were intended to provide oversight of the DS do not outline clear roles, responsibilities, and accountabilities for DS implementation.

The audit team noted that this challenge is recognized within the 2023-2026 Data Strategy for the Federal Public Service renewal initiative, and that there will be efforts to clarify data leadership responsibilities across the Government of Canada (GoC). This includes establishing clear accountabilities for data in planning processes to ensure that individuals with data responsibilities are providing input to programs, policies, and services as they are being established and managed. In addition, Treasury Board Secretariat is set to develop guidance on the roles and responsibilities of executives with data accountabilities, review the policy suite, and clarify roles, responsibilities and interconnections between existing data bodies and activities.

Risk and Impact

Without clearly defined, documented, and communicated roles, responsibilities and accountabilities, the Department may experience duplication of and / or gaps in the roles and responsibilities of key DS players and senior management, with this lack of clarity extending to all Staff. This lack of clear accountability may result in inactions and impediments to advancing the Strategy and its implementation plan, impacting the Department’s success in the strategic management of data and position as a data leader within the GoC.

CR 1.1

Recommendation 2:

It is recommended that ADM SPIS, in collaboration with the ADM CMSS and the Chief Scientific Data Officer OCS:

  1. Formalize and communicate NRCan’s renewed data strategy and its implementation plan across the Department; and
  2. Clearly define, document, and communicate roles, responsibilities, and accountabilities of all parties, including the CDO, CIO, CSDO, executive leadership and working level staff, in alignment with the renewed Government of Canada data strategy and its upcoming implementation plan.

Management response:

Management agrees.

In response to Recommendation 2, SPI-SPB will formalize the renewal of NRCan’s enterprise Data Strategy and the development of its action plan. It will clearly define, document and communicate roles, responsibilities and accountabilities.

To support this, in collaboration with the Chief Scientific Data Officer and the Chief Information Officer, SPI-SPB will:

  • Review existing internal and external governance and oversight structures and develop plan to update structures to address identified gaps, including roles, responsibilities and accountabilities, leveraging existing corporate governance committees where appropriate. (see response to Recommendation 1, above).
  • Develop an engagement and communications plan to support the implementation of the renewed NRCan Data Strategy and Action Plan.

Position Responsible: SPIS ADM and DG SPI-SPB, in collaboration with ADM CMSS and Chief Scientific Data Officer.

Timing:

  • By the end of Q3 2023-2024
    • Establish governance approach to support renewal of enterprise Data Strategy and drafting of action plan.
    • Develop proposal for revised governance and oversight framework to support implementation of action plan.
  • By the end of Q4 2023-2024
    • Finalize ongoing governance structure as part of the renewed NRCan enterprise Data Strategy and Action Plan
  • Q1 2024-2025
    • Seek formal approval of the renewed NRCan enterprise Data Strategy and Action Plan, including governance structures.
  • By the end of Q2 2024-2025
    • Develop engagement and communication plan, with implementation to follow
Governance and Communication Mechanisms Audit Criteria (CR)  

Communication and Collaboration Mechanisms

The audit examined whether mechanisms were in place to enable clear and consistent communication and collaboration across relevant parties. Overall, the audit found that mechanisms in place in support of communication and collaboration may not consistently enable clear communication and collaboration across all relevant parties in relation to the DS. Through the review of the 2019 NRCan DS and relevant governance committee documentation, the audit noted that formal communication mechanisms include the DG-PC, the DG-IMTC, the DG-STC, the Information Management and Data Committee and occasionally the Planning and Reporting Committee. Informal mechanisms include regular bi-lateral meetings between the CDO, CIO, and the CSDO and working-level relationships among staff in these offices and sectors.

The audit found that there are gaps within the mechanisms employed, including:

  • Communication and collaboration activities through governance forums are infrequent, occurring on an as needed basis for information purposes.
  • Governance forums may not represent and facilitate communications across all relevant parties. For example, there is no CDO (or CDO office) representation on the DG-STC, and the there are no explicit linkages between the DG-IMTC, the DG-PC and the DG-STC; and
  • At the time of the audit, regular bi-lateral meetings were not occurring between the CDO and the CSDO due to recent turnover. It was noted that this was intended to change in the near future.

Documentation reviewed demonstrated efforts led by the CDO and their office to engage with Sectors and discuss the DS during the time that the 2022-2023 NRCan DS Implementation Plan was developed, and more recently with the 2023-2026 Data Strategy for the Federal Public Service renewal.

The CDO office noted that they intend to create clearer lines of communication to ensure that all parties have regular updates through developing and implementing a distribution list and holding regular meetings with sectors to discuss the renewal.

Risk and Impact

There is a risk that inadequate communication and collaboration mechanisms may not support engagement with the Sectors on the data strategy and its implementation, potentially leading to missed opportunities to share and leverage work in other areas of the organization. In addition, the absence of a cohesive approach among policy, IM/IT and science domains may contribute towards the adoption of misaligned or siloed approaches, limit DS advancement, and ultimately undermine the importance and overall value of the DS to the department.

CR 3.1

No formal recommendation issued. Through recommendations 1a and 2b, it is anticipated that communication and collaboration issues will be addressed.

See management response to recommendations 1a and 2b.

Performance Measurement and Information for Decision-Making Audit Criteria (CR)  

Performance Measurement and Information for Decision Making

The audit examined whether the Department has established adequate processes to track, monitor, and report on the progress of DS objectives. The audit found an absence of established and formalized processes to track, monitor and report on the progress of the DS or its implementation plan, with ad hoc tracking and monitoring occurring for the activities identified within the 2022-2023 DS Implementation Plan.
Progress to date includes the implementation of a data functional review framework, enabling the CDO to review and assess data investments through the Enterprise Investment Fund (EIF), as well as planned data needs identified through TB Submissions. Additional progress was noted in efforts to advance departmental data and digital literacy skills, through employee learning resources.

The audit also examined whether relevant and timely information is shared with key parties, including senior leadership, for the purposes of informing effective decision making. The audit findings revealed that formalized processes to identify, collect, compile, and disseminate information are not in place, and that there are no formalized reports to distribute key information to relevant parties for the purposes of informing decision-making activities.

Furthermore, there is an absence of clearly defined measurement criteria for how to assess progress, impacts and successes. This is a key factor required to enable supporting mechanisms and processes to collect, compile and disseminate information for decision-making purposes.

The CDO office has provided presentations to key governance committees on the anticipated changes arising from the 2023-2026 Data Strategy for the Federal Public Service renewal and its implementation plan. As per the 2023-2036 Data Strategy for the Federal Public Service, these changes include the requirement to establish, monitor and report on key performance indicators and metrics to assess progress and report on overall outcomes, with a quarterly tracking and status updates and annual reporting on results to TB. In addition, the plan identifies requirements for departments to embed data related activities at the onset of initiatives, including TB Submissions, as well as to strength data literacy, with on-going progress and efforts observed within the Department.

Risk and Impact

In the absence of processes and tools to support and enable tracking, monitoring, and reporting on the DS performance, as well as its implementation, the Department may be unable to measure the progress and success of its activities. This may impede its ability to focus on areas in need of attention, take course corrective actions to achieve its intended outcomes, and address emerging risks, which may adversely affect the Departments ability to attain and sustain its related goals and priorities.

Without a formalized approach to share key information and updates with relevant stakeholders on progress, the Department may also miss opportunities to leverage key information to informed decision-making, impacting the overall efficiency and effectiveness of its activities and implementation of the DS. This may also result in inconsistent or conflicting decision-making that take place at other levels within the organization, hindering the cohesive execution of the strategy.

CR 2.1
CR 3.2

Recommendation 3:

It is recommended that ADM SPIS, in collaboration with the ADM CMSS and the Chief Scientific Data Officer OCS:

  1. Establish formalized processes to track, monitor, and report on progress related to the renewed Government of Canada data strategy and its upcoming implementation plan.
  2. Implement mechanisms to share and distribute key information and updates at regular intervals to the appropriate parties to enable decision-making.

Management response:

Management agrees.

In response to Recommendation 3, SPI-SPB will establish a formalized process to track, monitor and report on progress related to the renewed NRCan enterprise Data Strategy, and any requirements linked to the 2023-2026 Federal Data Strategy.

To support this, in collaboration with the Chief Scientific Data Officer and the Chief Information Officer, SPI-SPB will:

  • Develop an approach to identify and define key indicators, timelines and a process for tracking progress on key activities.
  • Develop an engagement plan to support the development of a renewed NRCan enterprise Data Strategy and Action Plan (see response to Recommendation 2, above)

Position Responsible: SPIS ADM and DG SPI-SPB, in collaboration with ADM CMSS and Chief Scientific Data Officer.

Timing:

  • By the end of Q2 2024-2025
    • Develop engagement and communication plan with support from CPS, with implementation to follow.
    • Develop key indicators, timelines and process for tracking progress on key activities

Appendix A – Audit Criteria

The criteria were developed based on key controls set out in the Treasury Board of Canada’s Audit Criteria related to the Management Accountability Framework – A Tool for Internal Auditors, in conjunction with key risks identified by NRCan’s personnel who were consulted in the Planning Phase. The criteria guided the fieldwork and formed the basis for the overall audit conclusion.

Audit Objective:
Audit Sub-Objectives Audit Criteria
Audit Sub-Objective 1:
To determine whether the Department has adequate and effective governance mechanisms in place to support the sustainability of NRCan’s Data Strategy, its implementation, and its future renewal.
1.1 It is expected that roles, responsibilities, and accountabilities of the CDO, CIO, CSDO and Sectors are clearly defined, documented, and communicated.
1.2 It is expected that governance and oversight structures are clearly defined and provide strategic and operational direction with respect to advancing departmental data strategy efforts.
Audit Sub-Objective 2:
To determine whether the Department has adequate processes in place to track, monitor and report on progress towards the achievement of current and future Data Strategy objectives.
2.1 It is expected that the Department has established adequate processes to track, monitor, and report on the progress of Data Strategy objectives.
Audit Sub-Objective 3:
To determine whether the Department has adequate and effective mechanisms in place to enable communication, collaboration and information sharing, across relevant parties as it relates to the current and future Data Strategy.
3.1 It is expected that mechanisms are in place to enable clear and consistent communication and collaboration across relevant parties.
3.2 It is expected that relevant and timely information is shared with key parties, including senior leadership, for the purposes of informing effective decisions making activities.

Page details

Date modified: