Audit of Information Management (AU1603)

Audit Branch
Natural Resources Canada

Presented to the Departmental Audit Committee (DAC)
June 23, 2016

TABLE OF CONTENTS

EXECUTIVE SUMMARY

INTRODUCTION

The management of information within Natural Resources Canada (NRCan) represents a significant challenge as illustrated by the extensive and varied information sources and information formats present within the various sectors and regions of the Department. These include paper records, electronic documents, e-mail and other elements such as research or laboratory datasets or collections. Appropriate management of these information resources requires all departmental employees to consistently assess whether or not the information has business or enduring value and to apply consistent practices to the organization, use and dissemination, retention and disposition of this information.

Addressing these challenges within NRCan requires coordinated investment in people, processes and systems in an environment where the associated benefits, beyond policy compliance, while inherently understood, are not easy to comprehensively measure and assess in order to justify investment decisions.

NRCan’s Chief Information Officer (CIO) has accountability for overall departmental IM direction and performance, and oversees a centralized Information Management Division that delivers a portion of departmental IM services, with remaining IM services delivered by sectors. The current IM operational governance structure is decentralized, wherein priorities, plans and resource allocations are primarily determined at a sector level. Alignment of departmental and sector IM focus is supported by the Information Management and Technology Project and Architecture Review Board, a committee of sector representatives co-chaired by the CIO.

Within this context, the objective of the audit was to provide reasonable assurance that adequate governance and monitoring control frameworks, along with recordkeeping practices, relating to Information Management (IM) are in place.

STRENGTHS

The Department has established a governance structure to support information management within NRCan. The Department has also established and communicated IM policy guidance which defines roles and responsibilities expectations, and has developed a suite of strategic and operationally focused plans intended to guide IM activities and initiatives. In addition, in 2014, NRCan implemented GCDOCS to support management of electronic records by employees, and the Department has developed an IM performance measurement framework.

AREAS FOR IMPROVEMENT

Areas for improvement were identified during the audit relating to management:

• Leveraging the current IM governance structure to support IM priority setting, planning and investment decision making processes involving the CIO and NRCan sector leadership;
• Clarifying key roles in IM within NRCan sectors to support departmental consistency in IM practices;
• Re-examining departmental strategic and operational plans to ensure they reflect departmental IM priorities; define costs and associated benefits to support investment and resource allocation decisions; and provide appropriate guidance and a means of monitoring IM change initiatives and activities;
• Developing and implementing comprehensive business process guidance to ensure information resources of business and enduring value are managed in a consistent manner; and
• Expanding the focus of the IM performance measurement framework to support management decision making related to IM investment and resource allocations, and associated benefits and results.

AUDIT CONCLUSION AND OPINION

Since 2012, the department has implemented several changes to improve IM practices and respond to IM challenges including the introduction of revised IM guidance, the conduct of a broad IM awareness campaign, and the implementation of GCDOCS as the primary electronic records management system.

While the Department has implemented key elements of an IM governance framework, an IM monitoring control framework, and IM recordkeeping practices, challenges remain to ensure that information is managed in a complete and consistent manner across the Department. I encourage the department to continue to bring management attention in the areas of: leveraging the IM governance structure to support the definition of IM priorities and plans and to ensure alignment between departmental and sector IM approaches; clarifying key roles in IM  within NRCan sectors; developing IM business process guidance; and refining the IM performance measurement approach.

STATEMENT OF CONFORMANCE

In my professional judgement as Chief Audit Executive the audit conforms with the Internal Auditing Standards for the Government of Canada, as supported by the results of the internal Quality Assurance and Improvement Program.

Christian Asselin, CPA, CA, CMA, CFE
Chief Audit Executive
June 23, 2016

ACKNOWLEDGEMENTS

The audit team would like to thank those individuals who contributed to this project and, particularly employees who provided insights and comments as part of this audit.

INTRODUCTION

The management of information within Natural Resources Canada (NRCan) represents a significant challenge as illustrated by the extensive and varied information sources and information formats present within the various sectors and regions of the Department. These include paper records, electronic documents, e-mail and other elements such as research or laboratory datasets or collections. Appropriate management of these information resources requires all departmental employees to consistently assess whether or not the information has business or enduring value and to apply consistent practices to the organization, use and dissemination, retention and disposition of this information.

NRCan’s ability to make its data and information accessible and useful is fundamental to supporting the objective that “information resources of business and enduring value are managed in an efficient and effective way to ensure that they are findable, shareable and useful in the delivery of NRCan’s programs, policy and services.”^{Footnote 1}

Addressing these challenges and objectives within NRCan requires coordinated investment in people, processes and systems in an environment where the associated benefits, beyond policy compliance, while inherently understood, are not easy to comprehensively measure and assess in order to justify investment decisions.

In 2012, NRCan conducted an Audit of Information Management which identified opportunities for improvement related to policy compliance monitoring, electronic tools to support IM, training and awareness, and performance measurement. Since this audit was completed, the Department has introduced several changes impacting department-wide IM including the introduction of revised guidance (Directive on Information Management) in 2013, the conduct of a broad IM awareness campaign in 2013/14 and the implementation of GCDOCS as the primary electronic records management system in 2014.

Within this context, the Audit of Information Management was included in the Department’s Risk-Based Audit Plan, and approved by the Deputy Minister on March 12, 2015.

BACKGROUND – IM GOVERNANCE

Within NRCan, the Chief Information Officer (CIO) has been designated as the Information Management Senior Officer (IMSO) who is accountable via the Assistant Deputy Minister, Corporate Management and Services Sector (CMSS) to the Deputy Minister, for overall departmental information management (and technology) direction and performance.

Supporting these accountabilities is a governance structure that includes:

Executive Committee, chaired by the Deputy Minister, which is responsible for establishing directions, priorities, and policy direction as well as overseeing resource allocation.

The Executive Committee is supported by the Business Transformation Committee (BTC), a senior management committee responsible for the development of corporate asset strategies (including IMT) and the implementation of government-wide management initiatives with an impact on NRCan’s business processes and practices.

Information Management and Technology Project and Architecture Review Board (IMT- PARB) is the Department's senior Information Management and Technology (IMT) committee responsible for bringing a business focus to the IMT domain. It is co-chaired by the CIO and its membership includes Director General representatives from the Department’s sectors. IMT-PARB is intended to provide a departmental forum for discussion, analysis, and guidance of horizontal IMT-related issues, plans or requirements and either concurs with or develops recommendations for consideration by the Business Transformation Committee, as appropriate.

IM Working Group, a subcommittee of IMT-PARB, is chaired by the Director, Information Management.  Its membership comprises representatives from most of NRCan’s sectors. This working group is intended to provide advice and a sector specific perspective on IM related activities of the Department. It is intended to foster collaboration and coordination in the execution of IM initiatives.  

While the CIO is identified as having delegated accountability for overall department IM direction and performance, the current IM operational governance model within the Department is decentralized wherein priorities, plans and resource allocations are primarily determined at a sector level. Some sectors within NRCan maintain their own IMT related governance committees that provide a sector focus on IM.

AUDIT PURPOSE AND OBJECTIVES

The objective of the audit is to provide reasonable assurance that adequate governance and monitoring control frameworks, along with recordkeeping practices, relating to Information Management (IM) are in place.

Three lines of enquiry support the overall audit objective:

1. Departmental governance frameworks are in place over IM to support departmental objectives.
2. Departmental monitoring control frameworks are in place for the oversight of IM.
3. Recordkeeping practices at the departmental level meet the requirements for records retention and disposition and have been implemented with an appropriate focus on behavioural change management.

DEPARTMENTAL RISKS

A risk-based approach was used in establishing the objectives, scope and approach to this audit engagement. The following risks were identified as applicable to the NRCan's management of the IM function:

Governance and Strategic Direction

• IM strategies, policy frameworks, initiatives and tools may not be updated/developed or implemented in a timely manner to effectively address GC and NRCan priorities in a changing environment.
• Monitoring and reporting frameworks may not be established or implemented to support oversight and compliance with IM requirements.

Transformation and Change Management

• The IM foundation (e.g. recordkeeping requirements, training, communication) may not have been implemented to build on IM initiatives (e.g. GCDOCS implementation).

Communications

• IM strategies, objectives, roles and responsibilities may not be clearly communicated to ensure their effective and timely achievement.

Human Resources / Resource Management

• NRCan, may be unable to maintain (i.e. recruit, retain, train) IM professionals with the right mix of skills and experience to effectively deliver IM initiatives.

Recordkeeping Practices

• The Department may not have established recordkeeping practices in place to support the lifecycle management of information resources of business value.
• Information contained outside of official repositories (e.g. GCDOCS) may not be properly managed, thus impeding the Department’s ability to index, search, retrieve, retain, and dispose of information as per TBS recordkeeping and other (e.g. Access to Information) requirements.

SCOPE & METHODOLOGY

The scope of this audit included the IM governance and monitoring control frameworks, and recordkeeping practices, in place at NRCan from April 1, 2014 to December 31, 2015. Key elements of these frameworks were expected to align with the TB policy instruments including definitions of roles and responsibilities; accountability structures; policies, plans, tools, systems, and human resources planning; and monitoring and reporting.

The scope of the audit also included recordkeeping practices pertaining to legacy records and systems for the handling and disposition of records in both electronic and paper-based formats.

The scope excluded assessment of the application of document security classification (e.g. PROTECTED A, B or C and CONFIDENTIAL, SECRET, TOP SECRET classifications).

The audit methodology was based on Treasury Board (TB) Policy on Internal Audit and Government of Canada Internal Auditing Standards, and included:

• Interviews with key personnel with respect to the Department’s IM activities and related governance models;
• Review of key documents including IM governance committee terms of reference and meeting minutes, and relevant policies and directives; and
• Review of IM planning and results reporting documentation.

CRITERIA

The audit criteria were primarily developed using Treasury Board and NRCan policies and directives relating to Information Management. Please refer to Appendix A for the detailed audit criteria.

FINDINGS AND RECOMMENDATIONS

IM GOVERNANCE

Summary Finding

The Department has established an IM governance structure wherein the CIO is identified as having accountability for departmental IM direction and performance while IM priorities, plans, and resource allocations are primarily determined at a sector level. Alignment of departmental and sector IM focus is intended to be supported by IMT-PARB, a committee of sector representatives co-chaired by the CIO.

In relation to the current governance structure, the audit found opportunities to: leverage the current IM governance structure, including BTC, to support IM priority setting, planning and investment decision making processes involving the CIO and NRCan sector leadership; ensure an IM focus for the IMT-PARB, and clarify the mandate of the IMWG which supports the IMT-PARB.

While the Department IM policy guidance includes the general definition of responsibilities for some IM roles, the audit found opportunities to clarify processes between the CIO and sector leadership to ensure consistent application of information retention and disposition practices; clarify the role of IM Specialist within the Department; and, clarify accountability for monitoring compliance with mandatory IM training for employees.

Supporting Observations

IM Governance Structure

Although the CIO is identified as having delegated accountability for overall departmental IM direction and performance, the current IM operational governance model within the Department is decentralized, wherein priorities, plans, and resource allocations are primarily determined at a sector level. Alignment of departmental and sector IM focus is intended to be supported by IMT-PARB, a committee of sector representatives co-chaired by the CIO. 

IMT-PARB and the supporting IM Working Group currently provide a forum for the communication of IM related issues and initiatives. Committee members confirmed; however, that these committees are not effectively leveraged to establish IM priorities, nor for driving consistency in information management practices across the Department. This challenge has been recognized in the 2015-18 NRCan Information Technology Plan, which references the Department’s intent to develop a “strategic framework that prioritizes and defines standards, practices and policies aligned with Government of Canada requirements” for various areas including Information Management. 

While the audit found that the primary focus for IMT-PARB has been on Information Technology matters (e.g. infrastructure, cyber security) which reflect the criticality of IT issues facing the Department. From an IM perspective, the focus of IMT-PARB has been on the communication of updates on departmental initiatives (e.g. GCDOCS implementation), but not on addressing broader IM issues/challenges facing the Department such as addressing IM requirements and needs that cannot be fulfilled by GCDOCS (e.g. information management of laboratory datasets).

The mandate of the IMWG includes an expectation that the working group “provide strategic advice and direction and increase coordination of IM activities across the Department”. The audit found that the working group does not have a “strategic advice” focus. Instead, the focus of the group is limited to communication of information relating to IM initiatives and operational issues (e.g. GCDOCS sign-on requirements) vs. forward looking IM strategic challenges and potential responses.

IM Roles and Responsibilities

The NRCan Directive on Information Management (the Directive) defines roles and responsibilities of various stakeholders including NRCan employees, managers, IM Specialists and the CIO. The audit identified several areas where additional clarity is required in the definition of roles and responsibilities for Information Management across the Department.

The Directive identifies the CIO as being responsible for ensuring that all information resources of enduring value are transferred to Library and Archives Canada at the end of their retention periods and that the departmental Recordkeeping Implementation Plan is implemented. This, at a minimum, requires the Department to take into consideration the retention and disposition of both paper and electronic information within all sectors of NRCan. The Records Office (a unit within the Information Management Division) manages retention and disposition of paper corporate records and the paper records of a limited number of NRCan sectors, but does not manage the retention and disposition of paper records for all sectors. For those paper records not managed by the Records Office, there is a lack of clarity in how the CIO is able to gain assurance that these records are retained and disposed of in accordance with Department and Treasury Board Secretariat policy requirements.

For electronic records, the Information Management Division has identified over 300 electronic repositories maintained across NRCan sectors. There is a risk that information maintained in these repositories is not subject to consistent retention and disposition processes. While the Department is in the process of developing a disposition plan for electronic records, given the decentralized nature of information/records management, additional clarity of sector roles and responsibilities for the application of consistent retention and disposition practices is required.

The Directive defines roles and responsibilities for IM Specialists, including their role of “providing training, advice and best practices on the topic of information management”. Given the decentralized nature of information management within the Department and based on interviews with corporate and sector IM representatives, it is not clear who within NRCan’s sectors is expected to play this role, and whether, for example, there are any specific IM competency or reporting requirements for those expected to be IM Specialists.

The Directive includes a requirement for mandatory IM training for employees. This training includes e-learning modules accessed online through the Canada School of Public Service and participation was previously tracked by the GCDOCS implementation team. There is a lack of clarity as to who is currently responsible for the ongoing monitoring of performance against this mandatory training requirement.

RISK AND IMPACT

The current IM governance structure may not be appropriately leveraged to ensure alignment of departmental and sector IM priorities, plans, or investment and resource requirements and as such may not be appropriately supporting the implementation of effective, efficient and consistent information management practices throughout the Department.

Further, identified gaps in IM role and responsibility definition contribute to an increased risk that NRCan is not able to fully and consistently manage the complete life cycle of information of business/enduring value, as required by IM policy guidance.

RECOMMENDATION

1. The Assistant Deputy Minister Corporate Management and Services Sector (ADM CMSS) should leverage the Business Transformation Committee to increase the focus on Information Management (IM) to address areas including defining IM priorities; developing IM plans and related resource requirements; and supporting alignment between Natural Resources Canada’s (NRCan’s) departmental and sector IM focus.

2. The Chief Information Officer (CIO), in collaboration with Information Management and Technology Project and Architecture Review Board (IMT-PARB) members, should update current department IM policy and supporting guidance to ensure clarity in departmental roles and responsibilities for IM.

MANAGEMENT RESPONSE AND ACTION PLAN

Management agrees. In response to recommendation 1, IMT-PARB membership, roles and responsibilities will be reviewed to ensure that the IM function is. properly reflected. IMT-PARB chairs will add an IM Update as a standing item to the IMT-PARB Agenda. A review mandate of IMWG and ARC will also be completed to ensure alignment and improved integration with IMT-PARB:  September 30, 2016

A FY 2017-20 IMT Strategy will be developed and approved by BTC. This strategy will outline the departmental IM vision and strategy: October 31, 2016

BTC will approve annual Departmental IM Plans. Progress against this plan will be reported to BTC semi-annually. September 2016

Chief Information Officer and Security Branch (CIOSB) IM representative to be included on Sector IMT Governance Committees: June 30, 2016

Positions Responsible: ADM CMSS, Co-chairs IMT-PARB, Sectors ADM

Management agrees. In response to recommendation 2, CIOSB, in collaboration with IM Working Group, will review roles and responsibilities in the IM Policy to ensure clear accountabilities. April 30, 2016

CIOSB will create an awareness campaign, with an emphasis on IM roles and responsibilities to ensure that all employees understand their IM responsibilities: September 30, 2016

Position Responsible: CIO

IM PLANNING

Summary Finding

The Department has developed a number of IM related planning deliverables that range from those providing strategic guidance to those that provide operational guidance for specific IM related initiatives such as the implementation of the Directive on Recordkeeping. The audit found that these plans require re-examination in order to ensure that they reflect departmental IM priorities, define costs and associated benefits to support investment and resource allocation decisions; and provide appropriate guidance and a means of monitoring IM change initiatives and activities.

Supporting Observations

The Department has developed a number of IM related planning deliverables to provide strategic and operational guidance for specific IM related initiatives. Examples of these plans and their scope include:

Based on an examination of these plans and interviews with IM Division representatives, the audit found that several of the plans developed to support the management of IM initiatives have not been maintained nor monitored and are no longer considered relevant. The audit also found that the plans generally do not define the expected benefits, including financial, nor the human and financial resources required to execute the plans. Further, the scope of these planning documents does not fully address identified IM policy compliance gaps within the Department. Areas of note include:

• A disposition plan for electronic records (currently under development), including those records maintained in GCDOCS and those maintained in the many other electronic repositories maintained within the Department, has not yet been completed.
• The 2014-2016 paper records disposition plan has not yet been fully executed, and does not address paper records for all sectors and physical locations of the Department.
• Roles and responsibilities of the IM Division pertaining to NRCan sector practices employed in the retention and disposition of paper records have not been clarified. Currently, the IM Division has limited ability to influence retention and disposition practices of NRCan locations outside the National Capital Region.
• There is no recognized plan to address the migration of legacy records. The management of legacy records during GCDOCS implementation included a “day-forward” approach wherein documents for current and new projects are created in, or migrated to, GCDOCS. This approach has not yet addressed the records stored in NRCan legacy systems/repositories.

The expiry of the current IM Plan (in 2015-16) presents an opportunity for the Department to revisit and refresh the strategic and operational IM risks, issues and opportunities facing the Department and to establish departmental IM priorities, plans and associated resource requirements to guide the Department’s IM improvement efforts in the coming years.

RISK AND IMPACT

Lack of clarity in IM priorities and alignment of supporting operational plans increases the risk that the Department is not able to achieve its IM objectives. Further, lack of definition of the resource requirements and expected benefits related to IM initiatives and activities limits management’s ability to evaluate the business case related to IM investment and resource allocation decisions.  

RECOMMENDATION

3. The Assistant Deputy Minister Corporate Management and Services Sector (ADM CMSS), in collaboration with the Business Transformation Committee (BTC) members, should:
   1. Define IM strategic priorities and establish a process for developing and aligning departmental and sector IM plans; and
   2. Coordinate the completion of IM strategic and operational plans including the definition of resource requirements and expected benefits.

MANAGEMENT RESPONSE AND ACTION PLAN

Management agrees. In response to recommendation 3, a 2017-20 IMT Strategy will be developed, including a review of Government of Canada IM and IT strategies: October 31, 2016

A Departmental IM operational plan, which will incorporate the Departmental IM priorities and activities as well supporting resource requirements, will be developed and refreshed annually. This plan will be approved by BTC. September 30, 2016

Position Responsible: ADM CMSS. CIO will work with IMT-PARB to develop IM Operational Plan and IMT Strategy, including review and approval by Business Transformation Committee.

IM PROCESSES AND TOOLS

Summary Finding

The CIO is responsible for ensuring appropriate processes and tools are in place to manage departmental information. Efforts have been undertaken to improve the consistency and completeness of information management processes within the Department primarily through the development of IM policy, through communication and training initiatives, and through the implementation of GCDOCS. Despite these efforts, the Department still faces a significant IM challenge illustrated by inconsistent IM processes related to paper and electronic records across the Department. In addition, the audit identified factors that may be impacting the user uptake of GCDOCS as the departmental electronic records management solution.  

Supporting Observations

Under Treasury Board IM policy, the designated senior executive (CIO) is responsible for ensuring appropriate processes and tools are in place to manage departmental information.

Supporting this responsibility, the IM Division has undertaken various approaches and initiatives to promote IM and bring consistency to the processes and tools used to support IM within the Department including: 

• Development and dissemination of the Department’s Directive on IM – the Directive includes a definition of IM-related roles and responsibilities for employees, managers, and IM specialists, and also includes the requirement for a mandatory training course for employees (Information Management-Personal Awareness and Capacity Test (IM-PACT);
• Development and dissemination of a supporting Guide on IM via NRCan’s intranet site;
• Establishment of the IMWG with representatives expected to support the IM focus of their representative sectors;
• Implementation of an IM awareness campaign in 2013-2014 through which various training sessions, coaching clinics and broad-based IM communications were provided to NRCan employees to build a foundation for IM;
• GCDOCS, a tool that enables IM, was made available to departmental employees in 2014. This implementation was supported by a comprehensive department-wide communication effort;
• Ongoing communication/promotion of introductory IM training initiatives (e.g. IM-PACT) to employees; and
• The IM Division is currently in the process of developing an updated communication plan related to the Division’s IM and Library services supporting the Department.

Despite these efforts, the Department continues to face a significant challenge in implementing consistent processes and tools to support a complex information management environment which is characterized by extensive and varied information sources and information formats present within the sectors and regions.

Illustrating this challenge, efforts are currently ongoing to determine the current state of paper records and electronic repositories (beyond GCDOCS) within the Department. Over 33,000 linear metres of paper records and over 300 distinct electronic repositories have been identified to date, all of which should be managed via consistent storage, retention and disposition processes. Beyond the IM Directive, Guide, and associated training material, there is a lack of comprehensive process guidance (e.g. standard operating procedures) available to ensure that information resources of business and enduring value within and beyond these paper and electronic repositories are managed in a consistent and complete manner. 

A departmental IM focus over the past two years has been the 2014 implementation of GCDOCS, the enterprise document and records management (EDRM) solution of the Government of Canada. This departmental implementation was supported by a comprehensive communication and training effort. While GCDOCS management reporting (April 2015 – August 2015) indicates that system usage, in terms of the number of users regularly accessing GCDOCS, and the volume of files stored in GCDOCS, is increasing, the audit identified several factors that may be impacting the behavioural change required to support GCDOCS user uptake:

• System stability – Sector representatives interviewed indicated that system downtime was a concern, although IM Division reporting indicates that GCDOCS downtime has been reduced from 19 hours in March 2015 to under 2 hours in September 2015.
• System usability – Interviewees expressed concerns with a requirement for multiple logons (this issue has been addressed), and system responsiveness/speed in some NRCan regional offices.
• User choice – Users continue to have access to other formats beyond GCDOCS for retaining electronic information (e.g. shared drives, SharePoint).

The audit further identified that the Records Management module within GCDOCS has not yet been enabled. This module has the capacity to manage retention and disposition by applying a file classification structure (a folder structure with embedded retention periods based on the type of record) and initiating the disposition process for electronic records once they have reached their defined retention period. This module is expected to be implemented upon NRCan’s migration of servers to new data centres in 2016.

Given that NRCan’s Directive on IM aims to ensure “that information resources of business and enduring value are managed in an efficient and effective manner to ensure they are findable, shareable and useful in the delivery of NRCan’s programs, policies and services”, the audit confirms that there is a significant gap between this expectation and the current processes and tools in place to meet this expectation. 

RISK AND IMPACT

Lack of clarity and consistency in the processes and enabling tools used to support information management across NRCan’s sectors and regions increases the risk that the Department may not be able to capture, organize, disseminate, maintain and dispose of its information in an efficient, effective or consistent manner.

RECOMMENDATION

4. The Chief Information Officer (CIO), in collaboration with Information Management and Technology Project and Architecture Review Board (IMT-PARB) members, should define information management business process requirements to be consistently implemented within NRCan and define the manner and timing by which these business processes will be implemented.

MANAGEMENT RESPONSE AND ACTION PLAN

Management agrees. In response to recommendation 4, Chief Information Officer and Security Branch (CIOSB), in collaboration with IM Working Group, will create guidelines, procedures and processes to improve the management of information at NRCan: March 31, 2017

CIOSB, in collaboration with Communications, will create a communications plan so that any new guidelines, processes and procedures are well communicated: September 30, 2016

Position Responsible: CIO

IM PERFORMANCE MEASUREMENT

Summary Finding

The Department has developed and is in the process of implementing an IM performance measurement framework (PMF) to enable improved monitoring and reporting of select IM activities within the Department and to support IM policy compliance monitoring. The audit identified opportunities to broaden the Department’s IM performance measurement scope from its current “activity” focus (e.g. how many times is an IM related activity performed?) to a “results” and “value” focus to support management evaluation of IM investments and resource allocations, and associated benefits.

Supporting Observations

The Department has defined and is in the process of implementing an IM performance measurement framework to enable improved management and reporting of select IM activities within the Department and to support IM policy compliance monitoring. Full implementation and reporting against the performance indicators defined in the framework is targeted for March 31, 2016.

The proposed IM PMF identifies a series of performance indicators across four perspectives:

Performance Perspective	Description
IM Training, Awareness and Engagement	Nine indicators including performance measures related to various activities such as the number of IM presentations to staff, and the number of website visits to NRCan’s IM Guide.
GCDOCS	Eleven indicators including volumes and trends in GCDOCS files, total storage used in enterprise workspace vs. personal workspace, and system downtime information.
Recordkeeping	Seventeen indicators including volumes of paper and electronic records eligible for disposition, and volume of paper and electronic records disposed.
NRCan Library	Six indicators including volume of information requests, and volume of print items circulated.

Implementation of the framework will expand the current departmental IM performance measurement approach which has focused on CIOSB maintaining a monthly dashboard to measure and communicate GCDOCS activity levels (e.g. volume of GCDOCS files by sector, and growth of GCDOCS file volumes). CIOSB also currently coordinates and completes an annual assessment for IM/IT which includes the identification of policy compliance areas of focus for the Department (e.g. development of disposition processes and plans for paper and electronic records).

While the audit recognizes that the planned IM PMF is in its early stages, the audit notes that the resulting management information, which is focused on the measurement of IM activity levels, will be limited in its ability to support the measurement of the “results” of IM activities (i.e. are IM activities having an impact in support of NRCan objectives of efficient and effective management of information?)

In analyzing the PMF, the audit notes the following opportunities for future development consideration to provide insight into IM results:

• Broadening the metrics to support assessment and management decision making related to IM investments and resource allocations, and the associated benefits of improved IM practices (e.g. reduced electronic and physical storage costs);
• Broadening and integrating metrics to assess the impacts/results of IM improvement efforts (e.g. measurement of GCDOCS usage relative to other electronic repository usage (shared drives, SharePoint, etc.) to determine if information is being migrated to GCDOCS as planned or whether it is being merely replicated in GCDOCS); and
• Broadening the metrics to include other IM policy compliance areas of focus (e.g. IM training compliance, information resource risk profiling) and defining performance targets.

RISK AND IMPACT

There is a risk that performance measures, as defined in the proposed IM PMF, will not enable comprehensive management assessment of the results and impacts of departmental IM activities and investments.

RECOMMENDATION

5. The Chief Information Officer (CIO), in collaboration with Information Management and Technology Project and Architecture Review Board (IMT-PARB) members, should ensure the development and implementation of an expanded IM performance measurement framework to support assessment of departmental IM performance and results.

MANAGEMENT RESPONSE AND ACTION PLAN

Management agrees. In response to recommendation 5, CIOSB, in collaboration with IMT-PARB will expand the IM performance management framework (PMF) to further support the assessment of departmental IM performance and results: November 2016

Positions Responsible: IMT-PARB Co-Chairs

APPENDIX A – AUDIT CRITERIA

The audit criteria were primarily developed using Treasury Board and NRCan policies and directives relating to Information Management. Actual performance was assessed against the audit criteria resulting in either a positive finding or the identification of an area of improvement.

The overall objective of the audit was to provide reasonable assurance that adequate governance and monitoring control frameworks, along with recordkeeping practices, relating to Information Management are in place within NRCan.

The following audit criteria were used to conduct the audit:

Line of Enquiry	Criteria
1. Departmental Governance Frameworks Departmental governance frameworks are in place over IM to support departmental objectives.	1.1 Governance and accountability structures, including definitions of roles and responsibilities, are in place to support IM in departments.
	1.2 Departmental IM policies and tools (including training), aligned with the government’s IM policy framework, have been developed and communicated to support IM in departments.
	1.3 Departmental plans have been developed and communicated to implement government-wide and departmental IM initiatives.
	1.4 Departmental human resources plans supporting IM professionals have been established and are aligned with government-wide and departmental priorities.
	1.5 Implementation of departmental IM initiatives includes an appropriate focus on change management.
2. Monitoring Control Frameworks Departmental monitoring control frameworks are in place for the oversight of IM.	2.1 Departmental monitoring of and reporting on compliance with IM policy framework requirements have been implemented.
	2.2 Action plans have been developed and implemented to address gaps in the compliance with IM policies frameworks.
3. Recordkeeping Practices Recordkeeping practices at the departmental level meet the requirements for records retention and disposition.	3.1 Information resources of business value are identified, stored, and retained for appropriate periods and subject to regular disposition processes.
	3.2 Records disposition authorities pursuant to section 12 of the Library and Archives of Canada Act enable NRCan to carry out its records retention and disposition plans.
	3.3 Departmental programs and services integrate IM requirements into development, implementation, evaluation, and reporting activities.