Audit of IT Enterprise Architecture (AU1802)
Audit and Evaluation Branch
Natural Resources Canada
Presented to the Departmental Audit Committee (DAC)
December 14, 2017
TABLE OF CONTENTS
- Executive Summary
- Introduction
- Findings and Recommendations
- APPENDIX A – AUDIT CRITERIA
- APPENDIX B – ABBREVIATIONS
EXECUTIVE SUMMARY
INTRODUCTION
What is IT Enterprise Architecture?
IT Enterprise Architecture (commonly known as Enterprise Architecture) for the purpose of this audit is the blueprint or foundation that links the organization’s programs and business processes to Information Management and Technology (IMT) resources and is the basis on which IMT investment decisions are made. It provides these perspectives for both the “as is” or current state and the “to be” or future state. IMT resources include applications, information (data), and infrastructure (i.e., hardware, operating systems, networks, database management systems, and the environment that houses and supports them).
Importance of Enterprise Architecture
Organizations in both the public and private sectors have recognized that the understanding of Enterprise Architecture (EA) enables effective decision making about Information Technology (IT) investments, costs, and risks. EA serves many additional purposes. It allows organizations to optimize performance and deliver on priorities in the ever-changing digital landscape. A well-functioning EA is a guiding management tool for technology in the department and is essential to successfully enable the delivery of programs that rely on technology and to ensure a consistent approach to key issues, such as cyber threats. It is a foundational element of any IMT transformation initiative.
EA is a key component of the Government of Canada’s (GoC’s) IT Strategic Plan 2016-2020, established to support the enterprise transformation of IT architecture in implementing the enterprise approach envisioned in the GoC’s Blueprint 2020. The Treasury Board of Canada Secretariat (TBS) has actions underway to lead the development of an EA framework for the GoC.
Enterprise Architecture at Natural Resources Canada (NRCan)
The Chief Information Officer is responsible for ensuring that the NRCan Reference EA is developed and available to all NRCan Sectors. Sectors have a certain level of autonomy to manage their IT needs and prioritize and plan IT projects with the support of Sector-level IT teams.
The Architecture Review Committee (ARC) is responsible for building an EA for NRCan and for providing guidance and recommendations in building, prioritizing, executing, and managing the Information Management (IM) and Information Technology (IT) portfolios to reach the future IMT state, defined by TBS and NRCan senior management. ARC reports to the Information Management and Technology Committee (IMTC) and, depending on the scope and level of decision making required, the Business Transformation Committee (BTC) or the Executive Committee will be consulted.
At the GoC-level, TBS’ Chief Information Officer Branch (CIOB) is responsible for developing an EA framework for the GoC. Many IMT infrastructure decisions affecting NRCan are outside the control of the Department. The creation of Shared Services Canada (SSC), in August 2011, changed the landscape of managing IT, as some functions continue to be the responsibility of NRCan, while other functions have been transferred to SSC. As part of SSC’s mandate to consolidate, standardize, and streamline the delivery of email, data centres, and network services in the GoC, the responsibility for NRCan’s IT Infrastructure was transferred to SSC, along with that of 42 other departments. The move to GoC enterprise solutions, such as Shared Case Management Service, and the challenges of establishing interoperable, open information systems are also changing the GoC IMT landscape. As a result, NRCan has to constantly align and adapt to the new directions of the GoC.
The objective of the audit was to assess whether NRCan has well-defined and functioning Enterprise Architecture principles and processes in place that meet current and future business needs of the organization and that are aligned with the whole-of-government enterprise approach.
STRENGTHS
NRCan has taken initial steps to move towards a more enterprise-wide approach, after determining that the current approach to manage IMT in the Department is no longer viable. In May 2017, a major IMT transformation initiative was launched, including the development of a departmental EA, which will require significant collaboration and coordination between Sectors and the Chief Information Officer and Security Branch.
AREAS FOR IMPROVEMENT
- Most of the core elements that are required to develop the foundation of an EA, such as a Reference EA, have yet to be developed.
- IMT governance structures for EA exist; however, some are not well-defined and not working as intended.
- EA systems and practices have not been developed or need to be strengthened. These include a System Development Life Cycle methodology, policy instruments for EA, a process to manage EA risk, and performance measurement.
- Significant work remains to obtain an accurate estimate of the costs and benefits to develop and implement a departmental EA.
INTERNAL AUDIT CONCLUSION AND OPINION
The Audit Branch concludes that significant improvements are required for NRCan to have well-defined and functioning EA principles and processes to meet current and future business needs and to align with the whole-of-government enterprise approach.
The most important challenge faced by NRCan is the significant cultural change that will be necessary to move to a more enterprise-based approach to manage IMT, and to monitor and enforce the new EA standards going forward.
STATEMENT OF CONFORMANCE
In my professional judgement as Chief Audit and Evaluation Executive, the audit conforms with the Institute of Internal Auditors' International Standards for the Professional Practice of Internal Auditing and the Government of Canada’s Policy on Internal Audit, as supported by the results of the Quality Assurance and Improvement Program.
Christian Asselin, CPA, CA, CMA, CFE
Chief Audit and Evaluation Executive
December 14, 2017
ACKNOWLEDGMENTS
The audit team would like to thank those individuals who contributed to this project and particularly employees who provided insights and comments as part of this audit.
MANAGEMENT RESPONSE AND ACTION PLAN
Overall, Management agrees with the findings and recommendations as outlined in the Audit and has identified EA activities in the IMT Transformation Plan which was approved by the Business Transformation Committee on November 28, 2017. The Management Action Plan (MAP) activities to respond to Recommendations 1 through 6 align with this approved IMT Transformation Plan. As such, it is Management’s position that these MAP activities will establish the conditions and framework to enable the Department to approach the management of EA in a systematic way, with appropriate tracking mechanisms to ensure compliance. Timelines included in this MAP take into consideration related IMT Transformation activities such as defining the IMT service delivery / operating models, the technology roadmap and governance changes which are fundamental to delivering an EA.
INTRODUCTION
What is IT Enterprise Architecture?
IT Enterprise Architecture (commonly known as Enterprise Architecture) for the purpose of this audit is the blueprint or foundation that links the organization’s programs and business processes to Information Management and Technology (IMT) resources and is the basis on which IMT investment decisions are made. It provides these perspectives for both the “as is” or current state and the “to be” or future state. IMT resources include applications, information (data), and infrastructure (i.e., hardware, operating systems, networks, database management systems, and the environment that houses and supports them).
Importance of Enterprise Architecture
Organizations in both the public and private sectors have recognized that the understanding of Enterprise Architecture (EA) enables effective decision making about Information Technology (IT) investments, costs, and risks. EA serves many additional purposes. It allows organizations to optimize performance and deliver on priorities in the ever-changing digital landscape. A well-functioning EA is a guiding management tool for technology in the department and is essential to successfully enable the delivery of programs that rely on technology and to ensure a consistent approach to key issues, such as cyber threats. It is a foundational element of any IMT transformation initiative.
EA is a key component of the Government of Canada’s (GoC’s) IT Strategic Plan 2016-2020, established to support the enterprise transformation of IT architecture in implementing the enterprise approach envisioned in the GoC’s Blueprint 2020. The Treasury Board of Canada Secretariat (TBS) has actions underway to lead the development of an EA framework for the GoC. Departments and agencies, through their investment plans, are expected to detail how the whole-of-government enterprise approach will be implemented in their organizations.
Enterprise Architecture at Natural Resources Canada (NRCan)
The NRCan Chief Information Officer and Security Branch (CIOSB) is responsible for most corporate applications and is held accountable for compliance with the TBS’ information technology policy instruments. Sectors have a certain level of autonomy to manage their IT needs and prioritize and plan IT projects with the support of Sector-level IT teams. The Chief Information Officer is responsible for ensuring that the NRCan Reference EA is developed and available to all NRCan Sectors. A Reference EA is a document or set of documents to which an organization will refer for standards and best practices. In IT, a Reference Architecture will usually document such things as applications, information, and infrastructure standards (specifications and configurations) as well as business rules and business processes.
At the departmental-level, the NRCan Architecture Review Committee (ARC) is the main committee responsible for addressing EA systems and solutions. ARC reports to the Information Management and Technology Committee (IMTC) and, depending on the scope and level of decision making required, the Business Transformation Committee (BTC) or the Executive Committee will be consulted. ARC is responsible for building an EA for NRCan and to provide guidance and recommendations in building, prioritizing, executing, and managing the Information Management (IM) and Information Technology (IT) portfolios to reach the future IMT state, defined by TBS and NRCan senior management.
At the GoC-level, TBS’ Chief Information Officer Branch (CIOB) is responsible for developing an EA framework for the GoC. The Committee on Enterprise Priorities and Planning (CEPP) is the governance and oversight body for all government IT investments. Many IMT infrastructure decisions affecting NRCan are outside the control of the Department. The creation of Shared Services Canada (SSC), in August 2011, changed the landscape of managing IT, as some functions continue to be the responsibility of NRCan, while other functions have been transferred to SSC. As part of SSC’s mandate to consolidate, standardize, and streamline the delivery of email, data centres, and network services in the GoC, the responsibility for NRCan’s IT Infrastructure was transferred to SSC, along with that of 42 other departments. The move to GoC enterprise solutions and the increased demand of interoperable, open information systems are also changing the GoC IMT landscape. As a result, NRCan has to constantly align and adapt to the new directions of the GoC.
The Audit of Enterprise Architecture was included in the Department’s Risk-Based Audit Plan for 2017-2020, approved by the Deputy Minister on March 30, 2017.
AUDIT PURPOSE AND OBJECTIVES
The objective of the audit was to assess whether NRCan has well-defined and functioning Enterprise Architecture principles and processes in place that meet current and future business needs of the organization and that are aligned with the whole-of-government enterprise approach.
Specifically, the audit assessed whether:
- There is an adequate governance structure in place which supports transparent planning and decision making related to EA;
- There are effective EA development and implementation processes in place to ensure alignment between the GoC and NRCan; and
- There is effective oversight of EA through performance measurement and monitoring to ensure expected outcomes and key benefits are realized.
AUDIT CONSIDERATIONS
A risk-based approach was used in establishing the objectives, scope, and approach for this audit engagement. A summary of the key inherent risks that could impact the development and implementation of Enterprise Architecture include the following:
There is a risk that…
- NRCan IMT investments may not be optimized, resulting in a lack of strategic investment in areas of greatest priority;
- Cyber security risks may not be properly mitigated, resulting in inconsistent implementation of security measures;
- NRCan key stakeholders may have a limited understanding of EA principles, resulting in a lack of buy-in by key stakeholders;
- The EA governance structure may not be effective, and the roles and responsibilities for EA may be dispersed amongst several stakeholders at NRCan, resulting in an IMT environment that continues to be fragmented;
- The NRCan strategy for EA may not be clearly communicated, resulting in a lack of commitment and support from some key stakeholders;
- The GoC vision and strategies for EA may not be clearly understood by key stakeholders at NRCan, resulting in misalignment with the whole-government approach; and
- A centralized EA may not be able to respond to NRCan business needs and the evolving needs of science programs, resulting in business needs not being fully met.
SCOPE
The audit focused on current and planned activities at NRCan related to Enterprise Architecture and their alignment with the whole-of-government approach.
Specifically the audit examined:
- The governance structure in place to support transparent decision making related to EA and the evolving needs of science programs;
- The principles on which all IMT decisions regarding applications, information, and infrastructure are made at NRCan; and
- Oversight and monitoring processes in place to ensure expected outcomes and key benefits related to EA are defined and being realized.
The audit examined only the activities under the responsibility of CIOSB and the Sectors in managing the NRCan’s EA. The audit included liaison and communications between SSC, TBS’ Chief Information Officer Branch (CIOB), and NRCan related to EA, but excluded a direct review of SSC and TBS’ CIOB.
The scope did not include an assessment of the chosen solutions by the Department or GoC for EA or the technical decisions made. The audit took into consideration the audit work that was performed in the recent audits of IT Governance, Information Management, and Cyber Security, and made reference to these audits when deemed appropriate.
The audit focused primarily on relevant activities between April 1, 2016 and September 29, 2017, but also included a review of other pertinent documents prior to these dates.
APPROACH AND METHODOLOGY
The approach and methodology followed the Institute of Internal Auditors' International Standards for the Professional Practice of Internal Auditing and the Government of Canada’s Policy on Internal Audit. These standards require that the audit be planned and performed in such a way as to obtain reasonable assurance that audit objectives are achieved. The audit included tests considered necessary to provide such assurance. Internal auditors performed the audit with independence and objectivity as defined by the International Standards for the Professional Practice of Internal Auditing.
The conduct phase of this audit was substantially completed in September 2017.
CRITERIA
Please refer to Appendix A for the detailed audit criteria. The criteria guided the audit fieldwork and formed the basis for the overall audit conclusion.
FINDINGS AND RECOMMENDATIONS
ENTERPRISE ARCHITECTURE GOVERNANCE
Summary Finding
A number of governance structures are in place at NRCan to support the development and implementation of Enterprise Architecture (EA) in the Department. Opportunities to improve the current structure to support transparent planning and decision making related to EA were identified. The roles and responsibilities for directing, overseeing, and approving the Department’s EA have not been fully established and assigned to specific positions; the Architecture Review Committee (ARC) does not have all the tools, guidance, and expertize to fully deliver on its mandate; and NRCan has not yet developed policy instruments for EA (i.e., standards, procedures, and guidelines).
Supporting Observations
Roles, responsibilities, and accountabilities
In 2016, NRCan’s Audit Branch conducted an Audit of IT Governance. One of the key audit findings was that the Department lacked common standards and an enterprise-wide architecture. Specifically, the audit indicated that resources should be allocated in a manner that minimizes duplication and creates synergies; that reflect decisions that are aligned with common standards; and that result in an EA that will allow synergies within the departmental IT infrastructure to better protect the Department against cyber security threats. A second finding led to a recommendation that NRCan’s expectations of the Chief Information Officer and Security Branch (CIOSB) should be reviewed and the structure adjusted accordingly. If the expectations of CIOSB are not clearly established, there is a risk that EA may not be successfully implemented in the Department. The audit team followed-up on the status of these two recommendations and found that the recommendations have yet to be implemented.
This audit of Enterprise Architecture focused specifically on EA governance. The audit expected that a governance structure for the EA program would be in place, and that the roles and responsibilities for directing, overseeing, and approving EA functions would have been established and assumed in order to be able to ensure compliance and accountability with the EA.
A number of governance structures supporting the development and implementation of EA at NRCan currently exist. The structures supporting EA include the Business Transformation Committee (BTC), Information Management Technology Committee (IMTC) and Architecture Review Committee (ARC). ARC is the key player in EA. One of ARC’s role is to “champion and guide the development of an enterprise approach to NRCan’s architecture by providing a central point for review and recommendation in support of an effective NRCan EA”. Also, the first element of ARC’s mandate is to “Build an Enterprise Architecture for NRCan”. The associated committees (BTC and IMTC) have a broader mandate and are focused more on strategic initiatives and not specifically EA. ARC is presently not assuming the role for which it was created, mainly because there is limited EA expertize on the Committee, and the appropriate tools (such IMT architectural principles, policies, standards, and guidelines) have not yet been developed.
The persons, commonly known as enterprise architects, that should be responsible for the development and implementation of EA key governance practices have not been assigned. These practices include: the development of an enterprise architecture vision, the definition of a Reference Architecture, the selection of opportunities and solutions, the definition of architecture implementation, and the provision of EA services. In addition, these governance practices have not been performed, nor have resources been allocated to support these functions. The roles and responsibilities of a Chief Architect has also not been assigned. A RACI chart (Responsible, Accountable, Consulted, Informed), detailing key roles and responsibilities for EA, would be an adequate tool to assign these roles and responsibilities. However, the audit found that no such chart has been created.
At the time of the audit, a review was underway to assist the Department in defining key roles and functions for a new IMT governance process. The Terms of Reference for this review acknowledges that NRCan currently uses a decentralized IMT operating model. The Department itself is organized in separate business sectors that have limited relationships in terms of information management, IMT systems, or IMT architecture. The review is examining ways for the Department to clarify what IMT governance decisions need to be made, who has accountability for decision making, and what mechanisms support how IMT governance should work. The review includes the development of a RACI chart to illustrate the target state governance structure for IMT in the Department.
Policies, Directives and Procedures
The audit expected that complete, up-to-date, and approved policies, directives, and guidance would exist for EA development and maintenance.
At the time of the audit, NRCan had limited policy instruments to govern Enterprise Architecture. There was also limited available guidance in terms of written and approved organizational policies or procedures to assist NRCan in the understanding of EA, and to ensure and enforce compliance. As part of the IMT transformation initiative, NRCan plans to develop an EA policy for the Department. Another important component of the transformation initiative is the requirement to develop an approach to address legacy applications and systems.
TBS’ Chief Information Officer Branch (CIOB), as part of the GoC IT Strategic Plan 2016-2020, has actions underway to develop an EA framework to be used by all departments. There are currently no official GoC policy instruments or frameworks available to guide departments on how to implement EA. The EA Group within the TBS’ CIOB informed the audit team that they are in the process of developing guidance tools and material for EA implementation. At the time of audit reporting, there were no guidance tools posted to the TBS working group portal in GCpedia.
There is also limited internal guidance to support the Department in its EA efforts. For instance, there are no common architecture layers, key models, or practices. Few standards, guidelines, procedures, templates, and tools have been developed to contribute to the departmental vision of EA. Each Sector uses their own IMT solutions that meet their business needs, and there is presently limited exchange or collaboration across the Department. In addition, there is limited central control and direction to drive the Department-wide EA agenda. Programs have developed their own tools and practices independently, which has led to more agile applications development and faster implementation of business solutions. However, this has also resulted in a significant use of resources and divergent architectures that may not be interoperable with each other in the future or compliant with the end state requirements. The support of multiple IT platforms is costly and may not fully leverage possible synergies across the Department, and could increase the risks, for example, of cyber threats.
Some authoritative NRCan documents contain references relating to EA and are being used as a means to drive the NRCan IMT Transformation agenda. Examples of these documents include the IMT strategy 2017-2022, the NRCan IT Plan 2017-20, the NRCan IM Plan for 2017-18, the NRCan Directive on Security Assessment and Authorization (SA&A), and external documentation such as the ITSG-33 IT Security Risk Management documentation used by the GoC.
RISK AND IMPACT
Unclear roles, responsibilities, and accountabilities for NRCan to govern EA may limit the Department’s ability to implement an enterprise-wide EA approach, thus increasing the likelihood of duplication of efforts, lost opportunities, and increased costs. This could result in a continued fragmented NRCan IMT environment that is not being aligned with the enterprise approach of the NRCan IMT strategy and the GoC IT Strategic Plan. The lack of guidance in the form of policy instruments may also limit the ability of NRCan to adopt an enterprise-wide IMT approach and may make it very difficult to monitor and enforce EA standards across the Department.
RECOMMENDATION
- The Chief Information Officer (CIO), in consultation with all Sectors and the Treasury Board of Canada Secretariat (TBS), should lead the development and implementation of Enterprise Architecture (EA) policy instruments (i.e., standards, procedures, and guidelines); clarify roles and responsibilities of committees and individuals involved with EA; and develop procedures to ensure compliance with EA policy instruments going forward.
MANAGEMENT RESPONSE AND ACTION PLAN
Management agrees. In response to recommendation 1:
A policy will be developed, which outlines the roles and responsibilities to deliver EA in the Department (individuals and committees). The policy will outline procedures and governance to ensure compliance with EA related policy instruments going forward.
Positions Responsible: Chief Information Officer and Security Branch (CIOSB) – CIO
Timing: July 2019
DEVELOPMENT AND IMPLEMENTATION OF ENTERPRISE ARCHITECTURE
Summary Finding
NRCan has recognized the need to adopt Enterprise Architecture (EA) for several years and a significant transformation initiative was launched in May 2017 to accelerate the move towards a more enterprise-wide Information Management Technology (IMT) approach in the Department. One of the foundational elements of an Enterprise Architecture is the development and maintenance of a Reference Enterprise Architecture describing the current and future state for applications, information, and infrastructure; this is presently missing in the Department. The IMT transformation initiative includes the development of an EA for the Department; however, at the time of the audit, it was still in the planning phase. The most important challenge faced by NRCan is the significant cultural change that will be necessary to move to a more enterprise-based approach to manage IMT.
Supporting Observations
Foundation of Enterprise Architecture
The audit expected that a Reference Enterprise Architecture describing the current and future state for applications, information, and infrastructure would be in place. One of the foundational elements of EA is to develop and maintain a Reference Enterprise Architecture. The Reference EA defines the recommended technical infrastructure, applications, and data in technical terms as well as business terms such as business processes, business rules, governance, security requirements, information needs, performance, locations, and users. The Reference EA should provide these perspectives both for the organization’s current environment and for its future environment. The Reference EA is important in order for the Department to have clear guidance when developing new systems and applications. It should include an EA repository containing standards, reusable components, modelling artefacts, relationships, dependencies, and views to enable uniformity of architectural organization and maintenance. The Reference EA is essential in order to achieve a more enterprise-wide approach and should be aligned with the GoC, Shared Services Canada (SSC), Public Services and Procurement Canada (PSPC), and NRCan IMT strategy plans.
The need to develop and implement an EA at NRCan is not new. The 2013 NRCan IMT Strategic Plan included an initiative to develop the current state and future vision of the NRCan EA model in collaboration with SSC. The NRCan EA model was to be used as a tool to develop a cohesive and cost-efficient IT architectural solution for NRCan’s future IMT needs. The mandate of the Architecture Review Committee includes building an EA for NRCan. One of the responsibilities of CIOSB in the 2016 NRCan Directive on Security Assessment and Authorization is to ensure that the NRCan Reference EA is developed and available. At the time of the audit, these three activities had not been completed.
The NRCan IMT strategy 2017-2022 and the NRCan IT Plan 2017-2020 support the transformation efforts to an enterprise IMT infrastructure. The 2017-18 performance agreement for all NRCan executives also contains a commitment to support the transformation strategy of IMT by bringing a horizontal and enterprise approach to IMT.
A Departmental Reference Enterprise Architecture, as required by the NRCan Directive on Security Assessment and Authorization, has not yet been developed. CIOSB informed the audit team that it does not currently have the resources, skills, nor departmental awareness of all systems to develop a Reference EA. The audit team noted, however, that some elements that could be related to a Reference EA were found in various documents. For example, one of the guiding principles in the IMT strategy 2017-2022 and the IT Plan 2017-20 is directly related to EA: “Align IMT Strategy activities and investments with GoC direction and departmental mandate, needs and priorities – current and future”. Another example is the NRCan Directive on Security Assessment and Authorization (SA&A) requiring that the security program complies with the minimum requirements established by Communications Security Establishment (CSE) and the NRCan Reference EA.
During the audit, several employees interviewed indicated that the Federal Geospatial Platform (FGP) was a good example of a solution architecture for geospatial data that contains most of the core elements the audit expected to find in a high-level Reference EA. The solution architecture for the Federal Geospatial Platform (FGP) was reviewed during the audit because it contains the following core elements that the audit would have expected to find in an NRCan EA. Namely:
- Architecture vision
- Architecture principles and guidelines
- Baseline architecture (incl. business models, data models, application models, and technology models)
- Target architecture
- Gap analysis
- Transition planning
The FGP solution architecture was created because there were, within departments and agencies, dozen of vertical implementations of geospatial capability at various degrees of maturity and effectiveness. The FGP is in production, continues to onboard new clients and data sets, and regularly releases new capabilities. The FGP clearly describes, through a common platform of technical infrastructure, policies, standards, and governance on how geospatial data will be managed in the future to meet the requirements of government departments and agencies and other external users.
NRCan has taken initial steps to move towards a more enterprise-wide approach after determining that the current approach to manage IMT in the Department could not be sustained. The IMT transformation initiative includes the establishment of an overall EA program to better understand the current state and the future state of IMT in the Department, and the activities that need to be performed to achieve the future state. At the time of the audit, the Department was in the process of developing the three-year IMT Transformation Plan, covering applications, infrastructure, information, and governance. The Plan is scheduled to be presented for approval by the Executive Committee, upon the recommendation of the Business Transformation Committee, in March 2018. According to the documents provided, the Transformation Plan will include the development of an overall EA program for the Department and will include the following elements:
- Change management and communications strategy
- Current IMT state
- Desired IMT state
- Financial and human resources
- Risks
- Action plan roadmap
Change Management and Cultural Change
The audit expected that a strategy would be in place to manage the cultural change resulting from a more enterprise-based approach to IMT.
By far, the most important challenge faced by NRCan to move to a more enterprise-based approach to manage IMT is the significant cultural change that will result. The current IT governance model at NRCan is decentralized and is inherently more responsive to user needs, yet more prone to inconsistencies and less effective at leveraging synergies. The implementation of EA does not necessarily mean a centralized IMT delivery model; however, it will imply the development of common standards for applications, information, infrastructure, and systems, and practices to monitor and enforce the new EA standards going forward. A period of transition will be necessary to transform legacy applications, information, and technical infrastructure to the new standards. Exceptions to the standards will need to be approved by an appropriate governing body, such as the Business Transformation Committee (BTC), and supported by a plan on how Programs will comply with the EA in the future.
The recent IMT transformation workshops confirmed the need for more IMT enterprise standards, and opportunities for rationalization exist. The workshop participants also indicated that better coordination and collaboration across the Department is necessary to break down the current “silo” approach.
The recent Change Readiness Assessment survey conducted by the Department indicated that most respondents (87.4%) see the need to change the way they do things and felt flexible enough to adapt (91.8%). 85.5% of respondents felt a flexible culture is critical to NRCan’s successful delivery of business priorities.
At the time of audit reporting, the draft IMT transformation plan included a project called “Engagement and Culture Roadmap to support IMT transformation”.
System Development Life Cycle Methodology
The audit expected that NRCan would have adopted an Enterprise Architecture development and maintenance methodology or an integrated architecture repository to enable consistencies across the organization to support IMT development activities. A common EA methodology and architecture repository containing standards, reusable components, modeling artefacts, relationships, dependencies, and views to enable uniformity would result in appropriately integrated EA products. The audit also expected that architecture products would be approved by an appropriate governing body, such as the Architecture Review Committee, to ensure compliance with NRCan’s IMT strategy and policy instruments. A corporate EA development and maintenance methodology and an effective governance structure should prevent the Sectors from developing and maintaining their own methodologies; facilitate the exchange of good practices; and provide assurance that all systems are developed using the same standards.
Many Programs have adopted their own systems development approaches to meet their business needs, but there is no Department-wide development and maintenance methodology. NRCan’s research activities and processes are heavily reliant on the use of information and technology resources. Presently, different Programs develop or acquire various IT solutions to fulfill their specific business needs. The business requirements drive and influence the decisions and approach relating to the development and maintenance of different IT solutions acquired by individual Sectors. This approach has permitted NRCan to be agile and has enabled the implementation of new systems rapidly, but has created challenges in ensuring compliance and alignment with key GoC requirements and long term direction, as well as working with service providers.
As mentioned in the Governance section of this report, the absence of clear EA governance principles and a Reference EA has made it difficult for key stakeholders to be aware of enterprise-level requirements and align their activities to enterprise-wide standards. Furthermore, Sectors find it difficult to engage CIOSB at an early stage of a project because there is a perception that CIOSB lacks understanding of business requirements versus enterprise requirements.
RISK AND IMPACT
Without a high-level EA, including a Reference EA, a System Development Life Cycle methodology, and a strategy to manage the significant cultural change required to move to a more enterprise-based approach to manage IMT, it is difficult for ARC, CIOSB, and all other stakeholders to understand and perform their EA roles and responsibilities. This is especially important in a decentralized environment in order to provide consistent EA guidance and recommendations for IMT investments and design decisions to prevent the development of systems that are not aligned with the enterprise approach of the Department and the GoC.
RECOMMENDATIONS
2. The Chief Information Officer (CIO), in consultation with all Sectors, should develop and implement a high- level Enterprise Architecture, including a Reference Enterprise Architecture for NRCan, and a System Development Life Cycle methodology to ensure a consistent solution approach to Information Management & Technology (IMT).
3. The Chief Information Officer, in consultation with all Sectors, should develop and implement a strategy and communications plan to manage the cultural changes that will result from the implementation of a more enterprise-based approach to manage IMT.
MANAGEMENT RESPONSE AND ACTION PLAN
Management agrees. In response to recommendation 2:
As part of the EA Program, a high-level target reference architecture will be developed and approved at IMTC.
Positions Responsible: Chief Information Officer & Security Branch (CIOSB) - CIO
Timing: April 2019
A System Development Life Cycle methodology will be developed and approved at IMTC for use by the Sectors.
Positions Responsible: CIOSB - CIO
Timing: July 2019
Management agrees. In response to recommendation 3:
As part of IMT Transformation Plan, communications and change management plans to support Enterprise First delivery of IMT will be developed and approved at IMTC.
Positions Responsible: CIOSB - CIO
Timing: July 2018
RESOURCE REQUIREMENTS
Summary Finding
At the time of the audit, preliminary estimates indicated that a significant amount of resources will be required to achieve the desired future IMT state; however, a comprehensive business case, including elements such as business needs, options analysis, cost-benefit analysis, sources of funding, and risk analysis had not yet been developed.
Supporting Observations
The audit expected that EA plans and activities would be appropriately defined and adequately costed. Funding allocation decisions should be based on reliable program cost estimates and include expected benefits, such as improvements to organizational efficiency and alignment; better product and/or service delivery; and reduced investment and/or operating costs by avoiding duplication of efforts.
In 2015-16, NRCan spent a total of $50.2 Million on IT. The total cost includes the cost incurred by different Sectors, including the Corporate Management and Services Sector (CMSS), to manage their IT-related projects. The total cost also includes expenses in the categories of hardware, software, human resources, external services (e.g., SSC), office equipment and supplies, and IT services from other government departments.
A key element of EA is to understand the current state and the future state of the IMT environment, as well as having a transition plan to move from the current to the future state. NRCan’s 2017-2022 IT Plan includes references to engaging in a strategic partnership with SSC to help address key priorities for NRCan, to develop a target end state for NRCan’s network, and to establish a roadmap to the end-state. However, a significant amount of work remains to be done to have an accurate and complete view of the current state of IMT infrastructure (i.e., information, applications, and infrastructure) and the efforts in terms of resources (people, finance, and time) to reach the future state. For example, there are presently over 600 applications listed in the NRCan Application Portfolio Management (APM) system, a tool used to inventory all applications in the Department. There are still a significant number of applications that are not in the APM, making it difficult to estimate the transformation level of effort that will be required to reach the future state. Further, most applications are in poor health due to an aging or unsupported platform. There is also a level of uncertainty about the possible end-state for different applications, which further complicates the estimation process.
At the time of the audit, the IMT Transformation Committee had adopted a draft costing methodology that is founded on a server-based costing model to arrive at a Rough Order of Magnitude (Class D Estimate) of the costs to move from the current to the future state. The model covers the costs associated with the migration and transformation of applications from a legacy data center to the end state data center. It includes the costs of revising, refactoring, re-purposing, rebuilding, and consolidating the legacy applications to meet the requirements of the end state data center, but does not factor in the potential savings to be realized from implementation. The draft IMT Transformation Plan indicates that a significant amount of work will be necessary over the next three years for its implementation.
The draft Transformation Plan also includes the establishment of an overall EA Program for the Department. The class D estimate indicates that it would cost approximately $415K and will take approximately seven months to establish the foundation of an EA program for applications in the Department. The cost estimate includes the development of the EA governance framework (vision & mission, EA governance bodies and processes, architecture domains, EA policy, and architectural standards); performing a current state assessment; and developing a future state. The cost estimates do not include costs associated with the implementation of the EA.
In order to assess the resource requirements in terms of number and skill requirements, including enterprise architect skills, NRCan has started an initiative to document an IMT skills inventory; however, the initiative is limited to CIOSB. The initiative includes several IT skills domains, and under each domain, a particular resource is assessed on a scale of four competency levels. The IMT skills inventory information could be used as a tool to supplement the IT project planning process; however, the current service delivery model does not offer the flexibility to take advantage of this information. In addition, the IMT skills inventory at the Sector-level is not documented or shared with the rest of the Department.
RISK AND IMPACT
The risks associated with the lack of clearly defined resource requirements for EA, such as an inconsistent and costly IMT environment, may continue if a comprehensive business case is not developed, approved, and implemented.
RECOMMENDATION
4. The Chief Information Officer (CIO) should prepare a comprehensive business case for the implementation of the Enterprise Architecture program as part of the overall Information Management & Technology (IMT) Transformation Plan, including well-defined activities, schedules, budgets, required resources, and performance measures.
MANAGEMENT RESPONSE AND ACTION PLAN
Management agrees. In response to recommendation 4:
CIOSB will develop a business case for the establishment for an ongoing EA program, including defined activities, schedules, required resources and performance measures. This business case will be presented to the CFO.
Positions Responsible: Chief Information Officer & Security Branch (CIOSB) - CIO
Timing: April 2019
RISK MANAGEMENT, OVERSIGHT, AND MONITORING
Summary Finding
Many systems and practices related to the development, implementation, and monitoring of an EA have not been developed or need to be strengthened. These include a process to manage EA-related risks, oversight and monitoring mechanisms, and the development of key performance indicators.
Supporting Observations
Risk Management Process
The audit expected that Enterprise Architecture (EA)-related risks would be proactively identified, reported, and mitigated.
NRCan has an established a risk management approach to identify key risks, including EA risks to the Department. However, opportunities were identified to improve risk management processes within the Department in order to consider an integrated approach to the risks related to the development and implementation of an EA.
CIOSB’s role in IMT risk assessment is limited to its own Branch, and each Sector is responsible to identify their own IMT risks and coordinate mitigation strategies with the Strategic Policy and Results Sector (SPRS), but the accountabilities for mitigating these risks are not clear. The IMT risks are captured as part of the departmental IT and IM Plans. At NRCan, SPRS coordinates various processes supporting the identification and management of corporate risks and prepares the Department’s Corporate Risk Profile (CRP). The CRP outlines the Department’s main strategic, external, and operational risks, as well as mitigation strategies and key accountabilities. At the Sector-level, each Sector is responsible for preparing a Sector Risk Profile, which outlines its respective risks and mitigation strategies.
There is presently no specific risk related to the absence of an EA program in the NRCan CRP. At the time of the audit, a risk register for the IMT transformation initiative, including EA risks, had not been developed. However, several risks and challenges related to the lack of EA have been identified during the recent IMT transformation workshops and the change readiness assessment survey. Examples of EA risks include absence of alignment between IMT investments and business priorities, and performance measurement and governance issues related to IMT activities, services, and projects.
In addition, as part of the IMT transformation process, CIOSB has conducted a cultural engagement survey and reached out to IM/IT staff to obtain their perspectives on the IMT Transformation Plan. Inadequate IMT resources, unclear IMT roles and responsibilities, limitation of IT systems, and the decentralized IMT approach were identified as key risks or challenges to the successful execution of the IMT Transformation Plan.
Oversight, Monitoring, and Performance Measurement
The audit expected that an EA performance and accountability framework would be in place, and performance metrics/indicators would be established to assess whether the Reference EA standards are applied consistently in all IT investments.
NRCan has not established a formal EA program; therefore, no formal processes have been established to measure the EA progress. Isolated EA reporting and monitoring activities exist; however, these activities only provide a fragmented picture of the current state. In addition, there are no processes and procedures established to assess whether IT investments are aligned with the NRCan’s EA approach. Currently, NRCan does not have approved Reference EA standards; therefore, IT investments cannot be assessed for effectiveness and compliance.
Application Portfolio Management (APM) Snapshot Reports are presented at the Architecture Review Committee (ARC). These reports provide the application portfolio status, including the current total number of applications (retired, mission critical, essential, and Critical Business Services) per SSC’s approved process. Application Portfolio Health Indicator status, time status, business value status, and retirement status are also included with graphical representation. The statistics contained in the APM snapshots are not complete because APM does not contain all the applications that are currently running on the NRCan’s corporate network. For example, many desktop applications acquired and used by regional offices are not included in the APM.
Sector IT Leads indicated that the internal reporting of IT within Sectors differs considerably. In some Sectors, the internal quality control or reporting is limited to the tracking of IT project deliverables. On the other hand, some Sectors assess the progress of IT projects, Security Assessment and Authorization (SA&A) documentation, and operations issues, and a dashboard is maintained to keep Sector management abreast of the current status of IT issues.
The Department IMT Strategy for 2017-2022 identified that more key performance indicators (KPIs) need to be developed to assess the performance in the areas of infrastructure, applications, information, enterprise resources, and enterprise governance. Indicators such as Application Portfolio Health Indicator (APHI) are currently prepared with limited information; however, most of the KPIs are not in place, partially due to the absence of the required processes and procedures, accountabilities for their enforcement, and dependencies on partner organizations like Shared Services Canada (SSC) for architectural support. The KPI’s planned for different IMT domains provide appropriate information; however, there is a no overarching performance management scoreboard to align different performance statistics and provide a holistic view of NRCan’s EA. Examples of EA Key Performance Indicators (KPI) could include:
- Number of exceptions to the EA standards granted;
- Architecture customer feedback; and
- Benefits realized that can be tracked back to the implementation of EA.
The Department currently reports on its IMT performance using the Management Accountability Framework (MAF), which is limited to reporting on IM stewardship; IT Stewardship and Application Portfolio Health, IT Program/Service Enablement; Enterprise Priorities Alignment; Service Performance; IT Security; Canada Cybersecurity Strategy; and Federated Identity. However, these indicators alone do not provide a complete picture of the current view of the NRCan’s EA. Regular performance measurement and reporting on the performance of EA would support senior management in monitoring the value of IT projects enabling business operations and making informed decisions.
The audit noted that one area that has recently seen significant improvements in oversight and monitoring is Cyber Security. IMTC and BTC members are provided with periodic updates on the Cyber Security Action Plan (CSAP) and Departmental IT Plan. The Cyber Security updates include information on areas such as vulnerability assessments conducted and results and status of Security Assessments & Authorizations.
RISK AND IMPACT
A lack of an integrated approach to manage the risks and challenges related to IMT transformation and EA may impede management’s ability to provide timely input to the strategies to mitigate all relevant, significant risks. The lack of a formal performance and accountability framework, as well as a lack of performance metrics, may hinder the Department’s ability to generate complete, accurate, and meaningful reports to effectively provide oversight and monitor the effectiveness of EA, and to proactively recognise efficiencies in the use of EA.
RECOMMENDATION
5. The Chief Information Officer (CIO), in consultation with all Sectors, should develop, as part of the IMT transformation initiative, a risk management strategy and report on the status of the most significant risks to the Information Management and Technology Committee on a regular basis.
6. The Chief Information Officer, in consultation with all Sectors, should develop and implement key performance indicators and effective oversight and monitoring mechanisms to ensure Enterprise Architecture standards are consistently implemented throughout the organization.
MANAGEMENT RESPONSE AND ACTION PLAN
Management agrees. In response to recommendation 5:
As a result of the Information Management & Technology (IMT) transformation initiative, CIOSB will implement appropriate mechanisms to report IMT risks to IMTC on a regular basis which can be leveraged across the department.
Positions Responsible: Chief Information Officer & Security Branch (CIOSB) - CIO
Timing: April 2019
Management agrees. In response to recommendation 6:
As part of the business case to establish the EA program (MAP 4.1), key performance indicators and monitor mechanisms will be identified and brought to Architecture Review Committee for regular oversight.
Positions Responsible: CIOSB - CIO
Timing: April 2019
APPENDIX A – AUDIT CRITERIA
The objective of the audit is to assess whether NRCan has well-defined and functioning Enterprise Architecture principles and processes in place that meet current and future business needs of the organization and that are aligned with the whole-of-government enterprise approach.
The following audit criteria were used to conduct the audit:
| Audit Sub-Objectives | Audit Criteria |
|---|---|
|
Audit Sub-Objective 1: Governance: To determine whether there is an adequate governance structure in place that supports transparent planning and decision making related to IT Enterprise Architecture and the evolving needs of science programs. |
|
|
|
|
|
|
Audit Sub-Objective 2: Development and implementation: To determine whether there are effective IT Enterprise Architecture development and implementation processes in place to ensure alignment with the GoC, CIOSB and various Sectors. |
|
|
|
|
|
|
|
|
|
|
Audit Sub-Objective 3: Oversight and monitoring: To determine whether there is effective oversight of IT enterprise architecture through performance measurement and monitoring to ensure expected outcomes and key benefits are realized. |
|
|
Source: The audit objectives and criteria were developed based on NRCan IMT Strategy 2017-2022, NRCan ARC Terms of Reference, NRCan IT Plan 2017-20 and GoC IT Strategic Plan 2016-2020. COBIT5Footnote 1 section AP003 (Manage Enterprise Architecture) and TOGAF 9.1Footnote 2 were also used for additional guidance.
APPENDIX B – ABBREVIATIONS
| APHI | Application Portfolio Health Indicator |
| APM | Application Portfolio Management |
| ARC | Architecture Review Committee |
| BTC | Business Transformation Committee |
| CIOSB | Chief Information Officer & Security Branch |
| CSAP | Cyber Security Action Plan |
| COBIT | Control Objectives for Information Technology |
| DAC | Departmental Audit Committee |
| EA | Enterprise Architecture |
| GoC | Government of Canada |
| KPI | Key Performance Indicator |
| IM | Information Management |
| IMT | Information Management & Technology |
| IMTC | Information Management & Technology Committee |
| IT | Information Technology |
| ITSG | Information Technology Security Guidance |
| NRCan | Natural Resources Canada |
| RACI | Responsible, Accountable, Consulted, Informed |
| SA&A | Security Assessment & Authorization |
| SDLC | System Development Life Cycle |
| SPRS | Strategic Policy and Results Sector |
| SSC | Shared Services Canada |
| TBS | Treasury Board of Canada Secretariat |
| TOGAF | The Open Group for Enterprise Architecture Framework |
Page details
- Date modified: