Struggling to incorporate the COSO recommendations into your audit process?
Here’s one audit shop’s winning strategy.

By Dennis Applegate and Ted Wills
December 1999 issue of Internal Auditor
Published by The Institute of Internal Auditors 

In 1992, the committee of sponsoring organizations of the Treadway Commission (COSO) issued a landmark report on internal control. Internal Control—Integrated Framework, which is often referred to as "COSO" provides a sound basis for establishing internal control systems and determining their effectiveness.

Following the report’s publication, The Boeing Company adopted the COSO principles partly as the basis for its internal control policies and procedures. As a result, our internal audit department began to rate the quality of internal controls covered in each audit. We soon discovered that incorporating these standards into actual practice proved challenging. While informative, our ratings were mostly subjective, lacking the systematic analysis and documented support normally reflected in our reports. To achieve a higher quality result, we reengineered our existing audit methodology—from inception, through fieldwork, to final reporting—to fit the COSO framework.

Our effort was a success. No longer incidental to our processes, COSO now provides the foundation for all our audit work.

The approach
Our integration of COSO into the audit process is similar to one described in The IIA Research Foundation report, The Internal Auditor’s Role in Management Reporting on Internal Control. The report suggests that audit results be cataloged in terms of the COSO framework and that this information be utilized in top-level reports to management and the board of directors. Our approach builds on some of these concepts by incorporating COSO criteria into each stage of the audit process.

According to COSO, the three primary objectives of an internal control system are to ensure (1) efficient and effective operations, (2) accurate financial reporting, and (3) compliance with laws and regulations. The report also outlines five essential components of an effective internal control system:

° THE CONTROL ENVIRONMENT, which establishes the foundation for the internal control system by providing fundamental discipline and structure.

° RISK ASSESSMENT, which involves the identification and analysis by management—not the internal auditor—of relevant risks to achieving predetermined objectives.

° CONTROL ACTIVITIES, or the policies, procedures, and practices that ensure management objectives are achieved and risk mitigation strategies are carried out.

° INFORMATION AND COMMUNICATION, which support all other control components by communicating control responsibilities to employees and by providing information in a form and time frame that allows people to carry out their duties.

° MONITORING, which covers the external oversight of internal controls by management or other parties outside the process; or the application of independent methodologies, like customized procedures or standard checklists, by employees within a process.

We use these elements to define the control objective to be audited, assess the components of Boeing’s control system, and report the results to management. Integrating COSO in this manner adds structure to our audit process, ensures that appropriate criteria are considered in key phases of each audit, and provides a trail to support the conclusions reached.

Defining Objectives
A key aspect of our reengineered process is that we focus each audit on a single COSO objective, rather than on many audit objectives. Each auditor, in conjunction with management, determines the appropriate COSO objective—operations, financial reporting, or compliance. This determination is made during audit planning and formally documented in the working papers. Concentrating on one audit objective allows us to improve audit focus and efficiency. If another objective needs to be addressed, a separate audit can be initiated.

If, after conducting audit research or fieldwork, a deviation from the predefined objective becomes necessary, the proposed change must be reviewed and approved by audit management. The working papers also must be amended to describe the rationale for changing the objective. As a practical matter, audit projects are normally completed without varying from the initial COSO objective.

Most audit projects will have a readily apparent objective based on the function or process to be reviewed. For example, a "program shop scheduling" audit clearly falls into the operations category. The objective of other audits, however, may not be so obvious. A "receiving inspection" audit may have either an operations focus or a compliance focus, depending on the type of management controls to be examined. Similarly, an audit such as "cost collection on the shop floor" may focus on operations or financial reporting, again depending on the controls to be assessed. For those few projects where the COSO objective is not clearly ascertainable, it is the auditor’s responsibility to identify the controls on which the majority of the audit work will concentrate and to select the appropriate audit objective based on the following guidance.

___________________________________________________________

Effectiveness
Testing all five COSO control components provides a solid foundation for determining the degree of assurance provided by controls.

Efficiency
Focusing on one COSO objective category guards against costly "scope creep."

Comparability
Using a common audit framework and rating system enables the controls in different business segments to be contrasted.

Communication
Integrating COSO criteria in discussions with clients enhances their understanding of control concepts.

Audit Committee
Reporting in terms of the COSO framework helps to portray strengths and weaknesses of the internal control system.

___________________________________________________________

Operations An operations objective focuses on controls governing efficiency and effectiveness. Effectiveness concerns the quality of controls over the achievement of specific management objectives, while efficiency addresses the quality of controls yielding an optimum measure of resource inputs to productive outputs. An operations audit should determine whether the organization can be reasonably assured that no material inefficiencies or lack of effectiveness exist in the audited organization or process.

Because Boeing is in a highly regulated industry, it is tempting to regard every operations audit as a compliance audit. However, an overall evaluation cannot be provided unless the audit evaluates the entire system of controls for ensuring compliance with laws and regulations. Such a system includes the relationship with the regulatory agency, Boeing internal policies and procedures, the people specifically assigned to promote compliance, and the methods for monitoring compliance effectiveness. Only audits that address each aspect of the compliance program can render an overall opinion on how well the system of internal controls assures compliance with the laws and regulations in question.

At the same time, operations audits that incidentally identify noncompliance with internal procedures provide useful information that must be communicated to management. Auditors are expected to note such potentially illegal violations as incidental findings on the control evaluation form.

___________________________________________________________

RATING CRITERIA FOR COSO-BASED AUDITS

Control Component	CRITERIA FOR UNSATISFACTORY RATING
Control Environment	"Hard controls" are missing or inadequate. There are verified instances of breakdowns of "soft controls."
Risk Assessment	Management has not predefined relevant objectives. Such objectives are incompatible with broader objectives. Management has not identified relevant risks to achieving its objectives. Management does not have a basis for determining which risks are most critical. Management has not ensured mitigation of critical operating risks. Audit tests detect key risks not previously contemplated by management.
Control Activities	Key control activities are not functioning as intended. Management’s risk mitigation strategy is not adequately reflected within control activities.
Information & Communication	Key metrics are not identified, collected, and communicated. Employees do not understand their control responsibilities, and this is pervasive. Customer or supplier complaints and disputes are not resolved, or remedial action is not undertaken in a timely manner.
Monitoring	Management has not established a means of determining the quality of the internal control system over time, either through independent evaluations or ongoing, structured, and independent process checks.
Overall	The ratings of all components should be considered to determine whether controls provide reasonable assurance that management objectives will be achieved. A strength in the internal controls of one component may compensate for a control weakness in another.

Financial Reporting In audits where the objective is financial reporting, emphasis is placed on the adequacy and effectiveness of management controls governing the reliability of financial data used for external reporting purposes. An audit based on such controls should provide reasonable assurance that no material misstatements exist in the examined data. Tracing audit controls and financial data back to the financial statements is indicative of an audit with a financial reporting objective.

An audit that reviews the assumptions and methods used to estimate contract costs-at-completion typically has a financial reporting objective. Similarly, audits of accounting controls that govern the preparation of financial statements generally will have financial reporting as their audit objective. Though there are exceptions, many finance-function audits focus on financial reporting since their scope often reflects the audit objective.

It might be initially difficult to determine, for example, whether an audit objective for accounts payable belongs in the operations or the financial reporting arena. The answer would depend on the kind of management controls to be audited. Likewise, audits of "cost screening for government allowability" could potentially address all three objectives: process efficiency and effectiveness—operations; adherence to Federal Acquisition Regulations (FAR) requirements—compliance; and reliability of cost classification and transaction processing—financial reporting. Selecting the objective is a matter of auditor judgment and depends on the nature and preponderance of management controls to be examined.

Compliance Audits based on compliance focus on the adequacy and effectiveness of management controls governing adherence to external laws and regulations. Such audits are primarily concerned with the correlation between laws and company procedure and actual practice. We usually consult our in-house legal counsel extensively during audits of this nature—an excellent indication that the audit should have compliance as its objective.

An audit of company adherence to the provisions of the Foreign Corrupt Practices Act is illustrative of a typical compliance audit. Examples also include audits of controls governing compliance to FAR "cost disclosure," FAR "defective pricing" requirements, and the Federal Aviation Administration’s "unapproved parts" inspection requirements.

ASSESSING CONTROL COMPONENTS
According to COSO, each of the five control components must be assessed before an opinion can be rendered about the design and effectiveness of the overall internal control system. Therefore, our reengineered process requires that each COSO component be covered in all audits.

In defining each control component, COSO identifies several control factors. We use these factors, as criteria for rating the effectiveness of controls. Our auditors must consider each factor during audit program development, and they must design appropriate inquiries and tests when assessing control effectiveness. By requiring each control component and factor to be addressed, we are seeking to ensure greater consistency in audit performance and to maximize audit effectiveness.

To ease understanding and application of the criteria, control component ratings are restricted to satisfactory or unsatisfactory. We considered alternative rating structures, but concluded that this binary approach, along with supporting rationale, communicated the right information. In unusual cases where the auditor stumbles across a reportable condition outside the system of internal controls being audited, an "incidental—satisfactory or unsatisfactory" observation may be recorded.

The assigned ratings must comport with the predefined criteria, be predicated on reliable audit evidence, and be documented in the working papers. If controls provide reasonable assurance that management objectives will be achieved, a satisfactory rating is assigned. An unsatisfactory rating is used if controls do not provide such assurance. While auditor judgment plays a significant role in assigning ratings, the existence of corrective recommendations suggests an unsatisfactory condition. To support auditors in rating controls, we’ve provided guidance that addresses each control component.

___________________________________________________________

___________________________________________________________

Control Environment For ease of analysis, the control factors for this component have been divided into hard and soft controls. At Boeing, hard controls consist of organizational structure, assignment of authority and responsibility, and human resources policies and practices. All three are relatively traditional areas examined in most audits. Audit evidence for each should be readily available.

Soft controls include ethics, commitment to competence, and management operating style. Such controls have traditionally been overlooked in audits because documented evidence of the audit condition is difficult to obtain and test.

If any one of the hard controls isn’t functioning effectively in the area being audited, an unsatisfactory rating is warranted. On the other hand, proper behavior is assumed for soft controls. An unfavorable audit conclusion is reached only if improper behavior is observed. A satisfactory rating wouldn’t be ruled out if the auditor finds no direct evidence that the "soft controls" are in place. Only if instances of unethical, incompetent, or improper management behavior are discovered should the auditor consider an unsatisfactory rating. The level of assurance provided by the auditor for soft controls is, therefore, much less than normally rendered. As techniques for testing soft controls improve, rating criteria may be revised to render more positive assurance.

Risk Assessment According to COSO, effective risk assessment requires:

° Predefinition of objectives.
° Compatibility of objectives.
° Identification of risks to achieving objectives.
° Judgment of which risks are critical.
° Determination of actions to mitigate risks.

If any one of these factors is absent, an unsatisfactory rating is generally warranted. Furthermore, audit inquiries and tests should be designed to determine if there are key risks not contemplated by management. If such risks are identified and deemed critical, an unsatisfactory rating should be rendered on that basis alone, even if all the factors listed are present.

Control Activities Regardless of the type of audit or the nature of the control activities being examined, it is standard audit practice to document specific control activities and related control objectives in auditor working papers. Generic control activities by type of audit might include:

Financial—written procedures, authorizations, record-keeping, management reviews, and asset safeguards segregated to prevent fraudulent financial data and asset misappropriation.

Information System—general, hardware, and application controls designed to ensure the reliability of the operating system, the accuracy of the data outputs, and the protection of equipment and files.

Operational— directive, preventive, and detective controls focused on achieving efficient resource usage and effectiveness as measured by the extent to which specific control objectives are achieved.

If key control activities have not been implemented or are not achieving their prescribed purpose, then an unsatisfactory rating for this control component normally would be warranted. Stated differently, there must be reasonable assurance that key control activities are operating as intended, based on their control objectives. If the risk mitigation strategy of management is absent or not adequately reflected in control activities, then this condition alone would call for an unsatisfactory rating.

INFORMATION AND COMMUNICATION COSO mentions several control factors for information and communication. We expect our auditors to assess at minimum:

• Identification, collection, and communication of key metrics for evaluating performance of the area being audited.
• Employees’ understanding of their control responsibilities relative to the larger system.
• Mechanisms for addressing customer, supplier, or employee concerns, complaints, and disputes in a timely manner.

If one or more of these factors are not operating effectively, an unsatisfactory rating should be considered.

Monitoring A process of control assessment established by management, monitoring determines the quality of the internal control system over time. Some form of independence from the daily process is necessary to ensure that monitoring serves as an effective control. Therefore, we do not consider routine management reviews of performance within a process as part of the monitoring component. Instead, such reviews are considered control activities.

Monitoring involves external oversight of internal controls by management or other parties outside the process. It may also include the application of independent methodologies, such as customized procedures or standard checklists, by employees within a process. If management fails to establish a monitoring process for its internal control system, either in the form of independent evaluations or ongoing monitoring, then a satisfactory rating for this control component normally would be inappropriate.

COSO-BASED REPORTING
We communicate our audit assessments to management using a COSO-based control evaluation form that we developed. This form, which is shown on page 64, provides management with a snapshot of how the audited area stacks up against the COSO control requirements. The ratings for each control component are shown, as well as an overall summary rating, which determines whether there is reasonable assurance that management’s objectives will be achieved. We also note the rationale for any unsatisfactory ratings given.

The control evaluation form is completed prior to the audit exit conference and reviewed with other auditors and audit management. These interim reviews determine whether sufficient basis exists for the ratings assigned. Auditors are encouraged to discuss the ratings with the audit customer—either directly, using the control evaluation form, or indirectly during review and discussion of the audit findings. These discussions are most effective when we speak in terms of "our control framework" and avoid use of the "COSO" acronym.

The control evaluation form is finalized as the audit report is prepared, and a completed form is transmitted to the general auditor at the end of each project. Control ratings are summarized at group and company segment levels, forming a basis for reporting on the status of internal controls to senior management and the audit committee. Data is tracked to determine unsatisfactory control trends, as well as areas of audit risk to address in planning future audit projects. For example, if unsatisfactory ratings consistently appear in the monitoring control component, then additional audit emphasis and management visibility may be given to self-reviews, control self-assessment, and similar ongoing monitoring techniques. All control evaluations are summarized semi-annually and presented, with appropriate explanation, to the audit committee as part of the company’s internal control systems assessment.

THE ROAD TO SUCCESS
Although our COSO implementation has ultimately been successful, we did encounter a few hurdles. We implemented the process incrementally, starting with development and testing of the control evaluation form. This earlier version of the form combined COSO control components with internal control objectives from The IIA’s Standards for the Professional Practice of Internal Auditing. An immediate problem developed when auditors were confused by variations in terminology between COSO and the Standards.

In addition, there was limited understanding of the COSO framework among audit staff, and no criteria regarding control ratings were provided to ensure consistency. Compounding the problem, two significant mergers involving Boeing, Rockwell, and McDonnell Douglas consolidated three audit staffs with different perspectives on performing internal audits. Of even greater concern was the fact that many auditors failed to see any real benefit in rating internal controls—a perception not without merit given the lack of guidance and confusion over terminology.

These problems were overcome through a consensus-building process combining employee involvement, management direction, detailed written guidance, workshop training, and follow-up verification. Training workshops were held to educate audit staff in the nature, purpose, and use of COSO. Revisions to our audit methodology were presented at these workshops to ensure consistent application. COSO-based rating criteria also were provided to guide auditors in reporting internal control conditions. To ensure goal congruence, a new internal control policy and procedure were issued, standardizing COSO application throughout the company and emphasizing management’s responsibility for implementation.

In addition, senior management support was crucial. Without this backing, many typical barriers to change would have blocked COSO implementation. Such support included:

• Memoranda from our general auditor communicating the need to adopt the COSO framework as the basis for reporting and tracking the condition of company-wide internal controls.
• Senior management assistance and active participation at our COSO workshop training sessions, conveying support for the COSO model and the need for employee buy-in.
• Audit director use of control evaluation metrics to prepare assessments of the control environment.
• Project management support of the COSO guidance, including checks to ensure proper application of COSO criteria in the audit process.

One benefit of this effort has been improved reporting on the company’s internal control status to our audit committee. Since we’ve reengineered our audit process to reflect the COSO principles, company directors and senior executive management receive a snapshot of internal control effectiveness based on assessments derived from our COSO rating system.

Adapting the COSO framework to our standard audit process has been beneficial in other ways, as well. Most notably, we believe that concentrating on a single COSO objective during each audit will ultimately improve the efficiency and effectiveness of our projects. Our policy, which requires that all five COSO components be covered in all audits, has already improved the effectiveness of our audit work in cases where these components were not addressed in prior audits.

Another key benefit has been the provision of a universal framework for internal auditing at the new Boeing. Focusing on COSO helped develop a perspective of internal control concepts common to the auditors of the three recently combined companies. The reliability and comparability of our audit work product, however, depends on continued adherence to the prescribed methodology, as inconsistent implementation poses an ongoing risk. Peer reviews and ongoing monitoring should help to mitigate this possibility.

The path to COSO-based auditing is not easily navigated. Yet those shops willing to make the journey will be rewarded by more thorough audit coverage and improved management reporting.

Dennis Applegate, CPA, CMA, CFM,
is Audit Manager for The Boeing Company in Seattle. He can be reached via e-mail at dennis.b.applegate@boeing.com.

Ted Wills, CPA, is Senior Auditor for The Boeing Company. He can be reached via e-mail at theodore.w.wills@boeing.com.

Published in the December 1999 issue of Internal Auditor, The Institute of Internal Auditor

Copyright © 1985-2005 The Committee of Sponsoring Organizations of the Treadway Commission. All rights reserved. Reproduction in whole or in part in any form or medium without the express written permission of The Committee of Sponsoring Organizations of the Treadway Commission is strictly prohibited. All trademarks used or referred to on this Web site are the property of their respective owners.