AACE® International Professional Guidance Document No. 02
GUIDE TO QUANTITATIVE RISK ANALYSIS
TCM Framework: 7.6 - Risk Management

Rev. March 18, 2024

 

This document is copyrighted by AACE International and may not be reproduced without permission.

 

TABLE OF CONTENTS

 

Purpose
Introduction

QRA Fundamentals

QRA Principles and Method Types – RP 40R-08
QRA Principles and Method Types – Escalation and Currency
Hybrid or Combined Methods and Aggregation of Projects
Complexity, Systems, Behavior, and QRA
Key Terminology – RP 10S-90

Guide to Specific Method RPs (In PGD Map Numerical Order)

1. Understanding the Project System (Stage-Gate Process) and Scope
2. Understanding Estimate Classification (Level of Scope Definition)
3. Value Management: Capitalize on Opportunities
4. Understanding the Base Estimate and Schedules: Basis Documents and Reviews
5. Understanding Concepts and Principles of QRA
6. Quantify the Bias Using Validation/Benchmarking
7. Quantify Uncertainty and Risk in Support of Decision Making

Predetermined Guidelines
Parametric Method
Simulation (Monte-Carlo Simulation)
Hybrid Methods
Escalation

Currency Exchange

Long-Range/Life Cycle (Class 10)
Complexity/Non-Linearity Methods
Programs and Portfolios
Management Reserves

8. Application of QRA in Strategic Asset (Portfolio) Management
9. Project Control Planning and Contingency Allocation
10. Change Management and Contingency Management
11. Maintain Methods and Tools Considering Historical Data

Methods Application Guide
References
Contributors

 

 

PURPOSE

 

Professional guidance documents (PGDs) provide roadmaps to improve use and understanding for collections of related AACE International® (AACE) recommended practices (RPs). They provide clarification on their purpose, function, context, and inter-relationships so that users can locate and focus on those RPs that best meet their requirements. The PGDs also highlight in-progress and preliminarily planned RPs to inform and guide AACE RP development (contact the Decision and Risk Management (DRM) Technical Subcommittee if interested in RP development). A hyper-linked format is used instead of publishing a standard text (which would have hundreds of pages of content). In recognition of the fact that AACE RPs are living documents, always refer to the latest RP. RPs are available free to AACE members (JOIN AACE).

 

PGDs also support the Total Cost Management (TCM) Framework in which RPs relating to each chapter are listed but not described. The heart of most PGDs is a map showing the relationship of the RPs in the context of the subject TCM process (and other TCM processes). This PGD covers RPs for the quantitative risk analysis (QRA) steps in the TCM Chapter 7.6 Risk Management process map (re: TCM Figure 7.6-1) and also support the Chapter 3.2 Asset Planning process map (re: TCM Figure 3.2-1). Key uses of QRA in TCM are to support contingency and other risk funding determinations for projects and programs, to support investment alternative evaluation and selection and to support capital portfolio management. This covers both the project, and the asset life cycles. This PGD was developed because of strong interest in the QRA area of practice, particularly in contingency. Many have an established project risk management process but lack reliable risk quantification. Also, there is a robust set of RPs to choose from in this area, justifying PGD treatment.

 

While there are no RPs for analytics methods at this time (e.g., machine learning and artificial intelligence), there are commercial applilcations on the market. Therefore, this RP touches on the topic including paper reference(s). At this time, in respect to data-driven methods, RP 114R-20, Project Historical Database Development, sets the stage for analytics application, and RP 42R-08, Risk Analysis and Contingency Determination Using Parametric Estimating, with its use of multiple linear regression (MLR) provides an entry point for analytics. Also, the RP 122R-22, Quantitative Risk Analysis Maturity Model, establishes the use of analytics as a QRA stretch goal (i.e., level 5).

 

QRA covers a broad set of practices used primarily to quantify uncertainty and risks in a way that supports asset planning and investment decision analysis (TCM 3.2 and TCM 3.3), change management decisions (TCM 10.3), and cost and schedule contingency, reserves, escalation and other phased project funding and control basis determinations (TCM 8.1). However, the focus of this PGD is TCM 7.6 which summarizes the QRA step as: “For each risk selected by the team for quantitative risk analysis, a probabilistic estimate of its impact will be prepared. The chosen methodology will have been specified during the risk planning phase.”.

 

This PGD also covers fundamental QRA RPs such as for selecting probability density functions as well as highlighting key QRA terms as defined in AACE RP 10S-90, Cost Engineering Terminology. QRA practices intersect with (and may be part of) base cost estimating and scheduling practices upstream of the QRA (e.g., historical data analysis and estimate and schedule validation to quantify bias) as well as control of the risk funds and contingency durations downstream of the QRA (e.g., contingency drawdown assessment). These related RPs will be highlighted where applicable.

 

It should be noted that PGD-01, Guide to Cost Estimate Classification Systems, addresses the concept of accuracy; a shorthand expression of the uncertainty characteristic of estimates. The classification RPs recommend that accuracy be determined through the application of appropriate QRA methods (i.e., there are no standard accuracy ranges). This guide addresses the appropriateness of the various QRA methods for different project phases and types. Given that accuracy is a key concept and communication challenge, RP 104R-19, Communicating Expected Estimate Accuracy (see video) may be considered the start of the map and journey into QRA. Next, RP 40R-08, Contingency Estimating – General Principles, covers the principles of recommended contingency and related methods. From there, the QRA RPs cover the alternative methods in detail, and this PGD’s map shows the path to related RPs and the relationship to other TCM processes.

 

Given that a purpose of this PGD is to guide users to RPs that best meet their requirements, the RP descriptions in this document include comparative information. For example, the appropriate method may vary depending on the size of the project, the level of scope development, or the resources available to the organization. No single method addresses all situations. Each RP should address the method’s strengths and weaknesses; this PGD brings some of this information to a single reference. In the end, users must study the RPs themselves to make the best method selection. A key RP to review in respect to assessing QRA requirements is RP 122R-22, Quantitative Risk Analysis Maturity Model.

 

Where RPs are needed to address the TCM 7.6 or other processes, but not yet developed, they are identified. The intention is to update the PGD regularly as RPs and DRM Subcommittee plans are updated. Where RPs are proposed or in process, key references (primarily AACE Transaction papers and books by RP contributors) are noted with links provided in lieu of the pending RP.

 

 

INTRODUCTION

 

The AACE Decision and Risk Management Technical Subcommittee began planning an RP series in 2007 with the initiation of a project to develop the Decision and Risk Management Professional (DRMP) certification. An early challenge the team of subject matter experts (SMEs) faced was which of the existing QRA methods to recommend. QRA is an evolving field of practice and not settled science. The approach selected was to first establish the principles that any recommended practice should follow (re: RP 40R-08). Then each candidate method was compared to how well it met the principles. Only a limited set of methods met the criteria.

In all cases, the RPs are aligned with the TCM Framework process for risk management (TCM 7.6) and other TCM processes to which TCM 7.6 is linked (TCM 3.2 perhaps being most significant). TCM 7.6 is the only industry risk management process that includes explicit steps for QRA. The TCM risk management process map in Figure 1 depicts that when a phase-gate decision is to be made, the RM process loops back through risk assessment for QRA (the step labeled “Analyze Contingency”). The term contingency in the map is shorthand for any quantified cost or time allowance for risks. This would include quantification for minor decisions over the project life-cycle (such as those made as part of the change management process (per TCM 10.3)) or for major over-arching analyses and decisions over the asset life cycle (such as those made as part of asset planning (per TCM 3.2)).

 

Figure 1. Process Map for Risk Management (Figure 7.6-1 from the TCM Framework; 2015 edition)

 

The TCM asset planning process map in Figure 2 (TCM 3.2) depicts the use of risk management, including QRA, in asset alternative analysis which usually involves asset life cycle and portfolio management considerations. This is where unclassified/Class 10 estimates (re: PGD-01) with long-range time horizons and special QRA considerations come into play.

 

 

Figure 2. Process Map for Asset Planning (Figure 3.2-1 from the TCM Framework; 2015 edition

 

The index map in Figure 3 presents the above processes in a way that highlights the relationship of the many existing and proposed QRA RPs (in red font) in the context of the related TCM Framework processes. The Figure 3 map highlights general information flow linkages between QRA fundamentals and practices and the associated TCM processes or process groups starting with Risk Management (TCM 7.6) (note: QRA practice is not limited to these processes) and including:

Figure 3. QRA Index Map: QRA RPs in the TCM Framework Context

The following list summarizes the key TCM processes and their respective QRA RPs as shown in the Figure 3 QRA index map. They are listed in a suggested order of study for a practitioner new to QRA. The later “Guide to Specific Method RPs” section of this PGD is organized in this same order.

  1. Understanding the Project System (Stage-Gate Process) and Scope (TCM 4.1) (No RPs)

  2. Understanding Estimate Classification (Level of Scope Definition) (TCM 7.3) (PGD No. 01)

  3. Understanding Value Management (Opportunities) (TCM 11.5) (RP 30R-03 and RP 48R-06)

  4. Understanding the Base Estimate and Schedules (TCM 7.3 and 7.2) (RP 31R-03, RP 78R-13, RP 34R-05, RP 38R-06)

  5. Understanding Concepts and Principles of QRA (TCM 7.6) (RP 40R-08, RP 66R-11, RP 104R-19, RP 122R-22, and RP 132R-23)

  6. Quantify the Bias using Validation & Benchmarking (TCM 6.1, TCM 7.2, and TCM 7.3) (RP 110R-20 and -TBD-)

  7. Quantify Uncertainty and Risk in Support of Decision Making (TCM 7.6) (See Guide to Specific Method RPs later)

  8. Application of QRA in Strategic Asset (Portfolio) Management (TCM 3.2 and TCM 3.3) (RP 85R-14 and RP 133R-23)

  9. Project Control Planning and Contingency Allocation (TCM 8.1) (RP 67R-11)

  10. Change Management and Contingency Management (TCM 10.3) (No QRA-focused RPs)

  11. Maintain Methods and Tools considering Historical Data (TCM 10.4) (RP 114R-20)

 

QRA FUNDAMENTALS

 

Prior to the “Guide to Specific Method RPs” section, the next sections address some QRA-related fundamentals including:

 

QRA PRINCIPLES AND METHOD TYPES – RP 40R-08

 

RP 40R-08, Contingency Estimating – General Principles provides a summary and description of the general QRA principles. In summary, the principles are:

RP 40R-08 describes in detail what is considered with each principle. These principles are fundamental considerations in the QRA Maturity Model described in a later section (RP 122R-22). The principles are not all repeated here, but as an example of the principle perhaps most applicable to the purpose of this PGD, consider the fit-for-use principle description:

 

Fit-For-Use

In addition to considering the general requirements of the client and the process, the practitioner must also consider any other significant contextual characteristics that may or may not affect the estimating practices selected and how they are managed and/or performed. These include, but are not limited to the following:

As should be evident, no single QRA method can be best in addressing all these characteristics. This PGD focuses on the above fit-for-use characteristics for each method, while also highlighting how an RP addresses other principles as appropriate (see the "Methods Application Guide" section).

RP 40R-08 also provides a general categorization of common QRA method types, and also has a table of how each general type addresses the principles. This is a starting point for method selection. However, this PGD adds to it with the "Methods Application Guide". The RP 40R-08 general QRA method types are as follows (analytics will be a future addition):

 

QRA PRINCIPLES AND METHOD TYPES – ESCALATION AND CURRENCY – RP 68R-11

 

RP 68R-11, Escalation Estimating Using Indices and Monte Carlo Simulation adds further QRA principles specific to escalation and currency analysis as follows:

RP 68R-11 calls for the use of probabilistic methods for escalation. Applying a probabilistic method should be considered for all projects of long duration (e.g., mid-point of spending >18 months from the starting date) because escalation may be the most uncertain cost element on a project and it has significant impacts to net present value (NPV). The following communication points will help explain to business and finance stakeholders why a probabilistic method may be appropriate:

Currency or exchange rate risk is somewhat related to escalation as both are driven by economic conditions. AACE does not yet have a currency QRA RP. However, the principles and methods used for probabilistic escalation are essentially the same as for currency. Currency adds to the equation the need to forecast future spending not only by cost account, but by currency. Exchange rate indices are then used to assess the uncertainty resulting from varying valuation of currencies as-expended relative to the base estimate currency. Where it is a major risk, finance departments may indemnify the project for currency risk (e.g., via hedging) making it less of a project team issue.


It should be noted that escalation is particularly sensitive to project schedule risk; i.e., schedule slip pushes spending into later time periods with potentially significant escalation cost impact. Also, escalation is applied to contingency (which has the same date basis as the rest of the base cost estimate). Because both cost growth and schedule slip are quantified as part of contingency QRA, it should be obvious that contingency and escalation quantification are optimally integrated with escalation analysis following QRA for contingency.

 

QRA MATURITY MODEL (QRAMM) – RP 122r-22

 

RP 122R-22, Quantitative Risk Analysis Maturity Model defines a method for assessing the level of capability for quantifying the uncertainty and risks associated with projects, programs and portfolios within the risk management function of a capital investment or project management organization. The QRAMM is not prescriptive to specific QRA methods; it highlights that many options exist (as covered here in PGD-02 and aligned with the principles in RP 40R-08) and provides examples of how various methods may be considered at various maturity levels as appropriate.


The QRAMM is also not a maturity model for assessing the depth of capability for any particular QRA method selected by an organization (e.g., see paper on a model for CPM-based QRA [1]). Instead, the QRAMM offers a point of reference or road map to facilitate continuous improvement. Each QRAMM capability level (it defines five levels of maturity) can be applied with various degrees of rigor or maturity depending on the organization needs. The QRAMM assumes that to achieve a high level of QRA maturity, an organization must clearly understand it’s risk management needs and apply any particular QRA method to the degree that is fit-for-purpose for those needs. A basis of the QRAMM is that the highest level (i.e., Level 5) be set as an aspirational or stretch goal, with the current aspiration being to apply analytics in QRA.

 

 

QRA INPUTS: DATA, ELICITATION, AND BIAS – RP 62R-11

 

QRA methods require various information inputs about uncertainties and risks (review the section on "QRA Risk Taxonomy" in the "Key Terminology  ̶  RP 10S-90" section first). This includes identified risks and information about their probability of occurrence and potential impacts for project-specific risks. In addition, ratings of systemic risk attributes or parameters are needed for parametric/MLR-based models. Also, ranges of impact are needed for uncertainties (probability = 100%) of all types for non-parametric/MCS methods. The primary sources of information include:

RP 62R-11, Risk Assessment: Identification and Qualitative Analysis addresses methods of eliciting input on the subjective inputs. Elicitation methods such as brainstorming, interviewing, Delphi, structured elicitation, and subject matter calibration are addressed there. One reason these topics are critical to QRA is that subjective inputs are inherently biased; all QRA methods must address bias as one input in order to produce realistic outputs. RP 110R-20, Cost Estimate Validation documents a method to quantify bias in base cost estimates. RP 114R-20, Project Historical Database Development documents capturing project risk information (e.g., past occurred risks and impacts) as well as metrics to assess historical project cost and schedule outcomes as well as bias. See Section 6 on "Quantify the Bias Using Validation/Benchmarking".

In respect to data, analytics (e.g., machine learning and artificial intelligence) represents a key topic; however, there are no QRA RPs on analytics methods at this time. The RP 122R-22, Quantitative Risk Analysis Maturity Model includes the use of analytics as a stretch goal (i.e., Level 5); this applies to the AACE DRM Subcommittee as well.


 

HYBRID OR COMBINED METHODS AND AGGREGATION OF PROJECTS

 

An implication of following the QRA principles stated is that there is no single method that can be used to quantify all risk on a given project. While a single tool addressing all risk may be possible, such a tool would sacrifice one or more principles. Therefore, typical combinations of methods are addressed in the QRA RPs. These include RP 113R-21 (parametric + expected value), RP 117R-21 (parametric + CPM) and RP 123R-22 (estimate ranging + expected value). Each of these incorporate Monte Carlo simulation at some level. These two-part combinations address systemic versus project-specific risks (see "Key Terminology - RP 10S-90" definitions) with the exception of RP 123R-22 which applies estimate ranging for background variability in lieu of the parametric method when systemic risks are not significant.

In addition, there are special considerations for and variations of methods when applied in aggregations of projects in a program or portfolio. These topics will also be discussed; however, there are no program or portfolio level QRA RPs at this time.


As stated, QRA is an evolving area of practice, and combination and integration of methods, and application for aggregations of projects, are areas to pay particular attention to developments. Given that it is evolving, QRA is a great area of practice for papers, presentations and for volunteer contributions to RP development.


 

COMPLEXITY, SYSTEMS, BEHAVIOR, AND QRA

 

Other areas where QRA is evolving include the understanding and quantification of risk relative to:

The trend in industry appears to be towards projects of increasing size or complication (measures of the number of parts) and complexity (measures of the extent and nature of interactions of the parts). There is also greater appreciation of the systems nature of projects for which learnings from systems engineering provide insight. Finally, the impact of human behavior, particularly as evidenced in bias, presents a challenge to quantification.


While there are no specific RPs for these topics at this time (e.g., measuring complexity), some QRA methods address these attributes to some extent and this will be noted where applicable (e.g., parametric methods for systemic risks apply complexity and bias measures). AACE Transaction papers will be highlighted that focus on these issues (e.g., non-linearity of impacts).


 

KEY TERMINOLOGY – RP 10S-90

 

The terminology related to QRA is extensive. One QRA text has over 170 terms in its glossary [2]. AACE RP 10S-90, Cost Engineering Terminology is the recommended reference for QRA terminology. The AACE definitions evolved from its 60-plus year history and are believed to reflect “what works best” for its members in general cost engineering and project control application. Other industry definitions often contain what the DRM Subcommittee feels are ambiguities (often originating from a qualitative point of view) on one hand, or over-specificity on the other (to suit specific standards that may or may not apply to one’s situation).

 

There are a number of terms defining the objects of QRA (the control accounts or activities being determined), and QRA risk taxonomy worth highlighting in this PGD because they are fundamental to QRA methods. there are variations in industry definitions, and they are often misunderstood. These definitions are:

QRA Objects

QRA Risk Taxonomy

Always refer to the original source document for the latest definition (in this case RP 10S-90)

 

 

Allowances

10S-90 Definition: For estimating, resources included in estimates to cover the cost of known but undefined requirements for an individual activity, work item, account or sub‐account.

Discussion:
Allowances are items in the above-the-line base estimate, not contingency. A key phrase is known but undefined which means it is the cost for a specific (individual) item, not general uncertainties. For example, the requirements for a sump pump’s horsepower may be uncertain (e.g., in range from 10 to 20), but the need for the pump is known. Hence the estimate may include an item titled "sump pump allowance" with the term allowance flagging the need to better define the item during scope development. Allowances may also be specific adders to known items (e.g., concrete waste allowance or mechanical equipment design allowance).

General uncertainties (e.g., allowance for weather or scope growth allowance) should be quantified in contingency. A rule-of-thumb is that if a cost item cannot be physically progressed, it belongs in contingency.

 

Special Case - Design Development Allowance:
This account, typically intended to address inadequacy in scope definition, has evolved into common usage, but the DRM Subcommittee considers including this in base costs to be a poor practice. This is distinct from specific equipment design allowances. Design development is not specific (i.e., not progressable). It is believed to have evolved because management has not allowed adequate contingency to address poor scope definition; therefore, estimators allowed for additional contingency above-the-line. If AACE RPs for contingency are used, which include design development risk in contingency as a systemic risk, this allowance will result in over-estimation or padding.

 

 

Contingency

10S-90 Definition: An amount added to an estimate to allow for items, conditions, or events for which the state, occurrence, or effect is uncertain and that experience shows will likely result, in aggregate, in additional costs. Typically estimated using statistical analysis or judgment based on past asset or project experience. Contingency usually excludes: 1) Major scope changes such as changes in end product specification, capacities, building sizes, and location of the asset or project; 2) Extraordinary events such as major strikes and natural disasters; 3) Management reserves; and 4) Escalation and currency effects. Some of the items, conditions, or events for which the state, occurrence, and/or effect is uncertain include, but are not limited to, planning and estimating errors and omissions, minor price fluctuations (other than general escalation), design developments and changes within the scope, and variations in market and environmental conditions. Contingency is generally included in most estimates and is expected to be expended.

Discussion:
Key words and phrases in this definition include: in aggregate which means only meaningful at the bottom line (not readily amenable to allocation); and expected to be expended which literally means the mean of the QRA outcome distribution; and changes within the scope which puts an emphasis on the team to define what scope means (scope will differ between the owner and contractor). It states design developments are part of contingency. It also clearly delineates contingency from management reserves and escalation and currency.

Special Case - Earned Value Management Based Upon ANSI EIA 748:
An amount held outside the performance measurement baseline (PMB) for owner level cost reserve for the management of project uncertainties. This definition can create ambiguity (is it contingency or reserve?; is it expected to be expended?) and applies only when standardized EVM is applied (but it must be understood if one is working in that environment. See RP 75R-13 for EVMS considerations.)

Special Case - Unclassified/Class10 Estimates:
This classification is for long-range estimates; i.e., estimates for projects to be executed more than 10 years in the future. These estimates are usually as part of life cycle cost analysis, surety valuations (e.g., bonding) or similar practices. For these future investments, scope change (e.g., changes in technology, regulations, etc.) is generally expected to some extent. Therefore, estimators/analysts need to clearly define how risks, including scope change, have been accounted for in QRA (contingency? reserve allowance?, etc.). An RP for QRA for these estimates is in consideration [3].

 

 

Management Reserve

10S-90 Definition: An amount added to an estimate to allow for discretionary management purposes outside of the defined scope of the project, as otherwise estimated. May include amounts that are within the defined scope, but for which management does not want to fund as contingency or that cannot be effectively managed using contingency.

Discussion:
Key words and phrases in this definition include: Management which means authority lies above the project manager level. Also, discretionary and outside the defined scope allows wide latitude to that management. The phrase cannot be effectively managed using contingency explicitly addresses the common situation of low-probability, high-impact risk events for which partial funding (i.e., multiplying impact times probability) is illogical. It also clearly delineates management reserves from contingency and escalation. In practice, common management reserve types include "general" reserve based on risk tolerance/appetite (funding at higher confidence evels) and "specific" reserves intended for specific high impact/low probability (HILP) risk events or similar.

Special Case - Earned Value Management Based Upon ANSI EIA 748:
An amount held outside the performance measurement baseline (PMB) to handle unknown contingency at the total program level. Management reserve has no scope, is not identified to specific risks, and is not time‐phased. It is typically not estimated or negotiated and is created in the budget development process. Per RP 75R-13, management reserve is “used to address undefined project risks that are within the scope of the project. MR is part of a contractor’s strategy for managing overall project cost and schedule risk; however, it is distinct from the project risk register in so far that it has no specifically defined risks or scope”. This definition is unnecessarily limiting by restricting management discretion to fund (or not) specific risks not amenable to contingency management. It applies only when standardized EVM is applied (but it must be understood if one is working in that environment. See RP 75R-13 for EVMS considerations.)

 

 

Escalation

10S-90 Definition: A provision in costs or prices for uncertain changes in technical, economic, and market conditions over time. Inflation (or deflation) is a component of escalation.

Discussion:
Key words and phrases in this definition include over time which differentiates economic driven trends from discrete risk events; e.g., a gradually improving but more costly technology trend is escalation while a potential instantaneous step change in technology and price would be a risk event and in contingency or management reserves. The term conditions implies pervasive, over-arching background such as would affect all projects of a type or in a region. It also differentiates escalation from inflation.

 

The following risk taxonomy terms define risk types in respect to the most appropriate or fit-for-use method for their QRA. Therefore, understanding their definition is important to selecting and communicating about QRA methods. The following term-pairs (i.e., versus) are discussed with the pairings resulting from QRA method application distinctions:

Systemic versus Project-Specific Risks

10S-90 Definitions:

Systemic Risk: A risk taxonomy designation used to classify project risks for the purposes of selecting a quantification method (i.e., contingency determination). Systemic risks are uncertainties (threats or opportunities) that are an artifact of an industry, company or project system, culture, strategy, complexity, technology, or similar over‐arching characteristics.

 

Project-Specific Risk: A risk taxonomy designation used to classify project risks for the purposes of selecting a quantification method (i.e., contingency determination). Project‐specific risks are uncertainties (threats or opportunities) related to events, actions, and other conditions that are specific to the scope of a project. (e.g., weather, soil conditions, etc.). The impacts of project‐specific risks are more or less unique to a project.

Discussion:
These taxonomy terms are for purposes of selecting specific QRA methods. The implied linking of QRA to systems thinking is intentional; the impact of system attributes (systemic risks) by nature is best studied and quantified through empirical research of project systems for which there is considerable research available. Systemic risks, as “artifacts” of the system as it is, are uncertainties (probability = 100%) with impacts usually derived empirically (the QRA use of the term originated in alignment with the parametric method). Systemic risks are usually not clearly identified in risk registers. Project-specific risks on the other hand usually dominate risk register content being more readily identifiable by teams. They may be uncertainties such as certain variability of a site condition (probability = 100%) or risk events (probability < 100%) which can be quantified effectively with simulation methods; i.e., they are amenable to project team judgment and experience as well as specific data (e.g., weather variability) as to probability and impact. Some have called potential future changes in the project system a systemic risks (focusing on the term system), but for QRA, such a change would be identified and quantified as a project-specific risk event (probability < 100%).

 

 

Uncertainties versus Risks Events

There is a host of risk taxonomy terms in use for many purposes. For QRA methods that quantify the probability of occurrence (as typically used in probability x impact calculations) an important risk taxonomy distinction is between risks that have 100% probability of occurring and those that have less than 100% probability. The following term usage, which has been proposed for RP 10S-90 (practitioners should monitor the RP for risk terminology development which evolves along with methods), makes this probability distinction:

 

Uncertainty: A risk with probability of occurrence of 100%; i.e., a risk condition, not a risk event. Other terms found in RP 10S-90 for risks with 100% probability of occurrence are background variability, inherent variability, and inherent risk. Systemic risk, which is an artifact of the project system is also an uncertainty. Project-specific risks, which includes conditions, may be an uncertainty (e.g., 100% probability that site soil conditions or weather will vary from the project plan basis for better or worse).

 

Risk Event: In respect to risk quantification, an incident or occurrence whose nature or result could be a threat or opportunity to the outcome of the project, and that has a probability of occurrence of < 100%. Contingent risk is sometimes used as a synonym. Project-specific risks are most often risk events (condition uncertainty being an exception). A potential identified change in a project system is a risk event.

 

 

Critical Risks

A risk taxonomy designation used to classify project risks for the purposes of selecting a quantification method. A critical risk is an identified condition or event risk that in its own right may materially affect management cost, schedule, profitability or similar quantitative objectives. Specific criteria for rating a risk as critical are established with management concurrence. For quantitative risk analysis, the rating is more specific than qualitative risk analysis rankings (i.e., risk matrix).

Discussion:
The purpose of identifying critical risks is to focus limited quantitative risk analyses resources and team attention on the risks that matter. The concept is time-tested having originated with the first use of MCS for quantifying project cost risks (re: RP 41R-08) where it was observed that, given the MCS requirement to address risk correlation (and in addition to the limitations of team focus), the attempt to quantify every risk was detrimental to analysis quality (in particular, including all risks tends to result in too narrow outcome distributions as the lows and highs of a multitude of independent risks tend to balance in MCS iterations). Critical risk identification is a fundamental step for the expected value method applied to project-specific risks as covered in RP 44R-08 and RP 65R-11 (which are also used in several hybrid methods). Typical criteria for establishing criticality are documented in those RPs. The concept is applied in all hybrid methods employing the parametric method (RP 42R-08) where the impact of non-critical risks is addressed by the uncertainty quantified in MLR (i.e., in developing a parametric model, actual cost and schedule data is normalized to remove the impact of critical risks). In effect, non-critical risks are contributors to systemic risks and background uncertainty.

 

 

GUIDE TO SPECIFIC METHOD RPS (IN PGD MAP NUMERICAL ORDER)

 

The following table summarizes the various method RPs available for various QRA purposes. The summary QRA process steps outlined in the Introduction and Figure 1 index map is used to organize this listing as follows. Where there are no RPs yet, AACE Transaction papers or other sources are suggested. The "Methods Application Guide" near the end of the PGD provides comparative information about these methods.

 

1. Understanding the Project System (Stage-Gate Process) and Scope

TCM 4.1 - Project Implementation

No RPs

Discussion:
An owner’s capital project system (including integration with its suppliers) and its effectiveness (or lack thereof) is a major source of uncertainty in performance and cost and schedule outcomes. Understanding the system through which a project is being performed, and its strengths and weaknesses (systemic risk) is essential grounding for any QRA analysis. An essential element of that system is the stage or phase-gate process which guides business framing and project scope definition, including at the portfolio level if applicable.

Selected References: [4,5,6,7,8]

  • Stage-Gate: Dysert, Larry and T. Pickett, “Supporting Estimates with Effective Scope of Work Definition”, 2020 AACE International Transactions, EST-3424, AACE International, Morgantown, WV, 2020.
    MEMBER DOWNLOAD | WATCH VIDEO | JOIN AACE

  • Stage-Gate: Ahmad, Mir M. and Christopher Carson, “Capital Improvement Program Stage-Gate Planning & Scheduling”, 2019 AACE International Transactions, PS-3112, AACE International, Morgantown, WV, 2019.
    MEMBER DOWNLOAD | WATCH VIDEO | JOIN AACE

  • Phase-Gate & Control: Carson, Christopher W. and Leo Carson-Penalosa, "Implementation of an Integrated Phase-Gate Project Controls Process", 2023 AACE International Transactions, TCM-4048, AACE International, Morgantown, WV, 2023
    MEMBER DOWNLOAD | WATCH VIDEO | JOIN AACE

  • Project Systems: Barshop, Paul, “Capital Projects: What Every Executive Needs to Know to Avoid Costly Mistakes and Make Major Investments Pay Off”, John Wiley & Sons, Inc., Hoboken, New Jersey, 2016.
     PURCHASE ON AMAZON

  • Project Systems: Stephenson, L. and M. Pruneau, “Strategic Portfolio Management: Improving Capital Utilization and Competitive Advantage”, 2016 AACE International Transactions, TCM-2110, AACE International, Morgantown WV, 2016.
     MEMBER DOWNLOAD | WATCH VIDEO | JOIN AACE

 

 

2. Understanding Estimate Classification (Level of Scope Definition)

TCM 7.3 - Cost Estimating and Budgeting
Professional Guidance Document (PGD) No. 01: Guide to Estimate Classification Systems

Discussion:
PGD-01, provides a roadmap to the various classification RPs. Classification defines the scope definition deliverables required as the basis for cost estimates of each class. The classes align with prevailing industry stage-gate system phases. Research shows that the level of scope definition, an artifact of the project system (systemic risks), is a major source of uncertainty in cost and schedule outcomes and driver of accuracy ranges. Hence a scope definition rating such as classification is a necessary input for QRA analysis. AACE offers an evolving series of Classification RPs for various industry segments. A similar concept is addressed by the Construction Industry Institute’s (CII) Project Development Rating Indices (PDRI); however, the PDRI is a continuous measure while Classifications are threshold measures (i.e., must achieve the defined level of definition of each element to be rated a certain Class). Like AACE RPs, CII offers an evolving series of PDRI related products.

Selected References: [9,10,11]

  • Classification: Hollmann, J., (Presentation Only) “Cost Estimate Classification Overview”, 2020 AACE International Virtual Conference & Expo.
    WATCH VIDEO | JOIN AACE

  • PDRI and Classification: Zaheer, Syed H. and C. Fallows, “Document Project Readiness by Estimate Class Using PDRI”, 2011 AACE International Transactions, EST-604, AACE International, Morgantown WV, 2011.
    MEMBER DOWNLOAD | JOIN AACE

  • PDRI: CII Project Definition Rating Index (PDRI) Overview.
    CII

 

 

3. Value Management: Capitalize on Opportunities

TCM 11.5 - Value Management and Value Improving Practices (VIPs)
 RP 30R-03: Implementing Project Constructability
 RP 48R-06: Schedule Constructability Review

Discussion:
There are many practices that have a specific focus and/or significant effect on getting the most value from the project scope and plans. Some call these value improving practices or VIPs; they are applied as part of stage-gate systems. In risk management terms, VIPs are focused on capitalizing on opportunities. If risk identification finds a potential opportunity, it should be referred to a value function to apply VIPs as appropriate. Each VIP, be it value engineering/value analysis (TCM 7.5), constructability (RP 30R-03 and RP 48R-06) or others, includes making decisions as to recommended changes or improvements to the scope (changes may be channeled through the change management process if in effect). Each change resulting from a VIP also carries risk, which needs to be considered using QRA. Constructability is perhaps the most common VIP used in engineering, procurement and construction. SAVE International is the foremost association addressing value methodology (also known as value engineering, value analysis or value management).

Selected References: [12,13,14]

  • Value Engineering: Opfer, Neil, “Where & Why Value Engineering Goes Wrong with Capital Projects”, 2021 AACE International Transactions, TCM-3707, AACE International, Morgantown WV, 2021.
    MEMBER DOWNLOAD | WATCH VIDEO | JOIN AACE

  • Value Engineering: Dell’Isola, Michael D., “Better Use of Value Engineering in Project Delivery”, Cost Engineering, Vol. 56, No. 06, AACE International, Morgantown, WV, 2014.
     MEMBER DOWNLOAD | WATCH VIDEO | JOIN AACE

  • VIPs: Lozon, Jim and G. Jergeas, “The Use and Impact of Value Improving Practices and Best Practices”, Cost Engineering, Vol. 50, No. 06, AACE International, Morgantown, WV, 2008.
     MEMBER DOWNLOAD | JOIN AACE

 

 

4. Understanding the Base Estimate and Schedules: Basis Documents and Reviews

TCM 7.3 - Cost Estimating and Budgeting
 TCM 7.2 - Schedule Planning and Development
 RP 31R-03: Reviewing, Validating, and Documenting the Estimate
 RP 78R-13: Original Baseline Schedule Review - As Applied in Engineering, Procurement, and Construction
 RP 34R-05: Basis of Estimate
 RP 38R-06: Documenting the Schedule Basis

Discussion:
QRA starts with understanding the base estimate and schedule. To the extent the base is biased (i.e., aggressive or conservative) and to the extent it includes allowances for uncertainty and risk, explicit or not, the need for contingency and other risk funds and duration will vary. A well-done basis document will describe the planning strategy, assumptions, allowances, and other important information. Estimate and schedule reviews will further surface quality issues (systemic risks). Validation and benchmarking (covered later) will surface and quantify bias and competitiveness issues.

 

 

5. Understanding Concepts and Principles of QRA

TCM 7.6 - Risk Management
 RP 40R-08: Contingency Estimating - General Principles
 RP 66R-11: Selecting Probability Distribution Functions for use in Cost and Schedule Risk Simulation Models

RP 104R-18: Communicating Expected Estimate Accuracy
 RP 122R-22: Quantitative Risk Analysis Maturity Model (QRAMM)
 RP 75R-13: Schedule and Cost Reserves within the Framework of ANSI/EIA-748

RP 132R-23: Schedule Risk Analysis Maturity Model
RP -TBD-: Complexity Assessment

Discussion:
These RPs cover elements of QRA; their titles are self-explanatory. They are not QRA methods in themselves, but cover general principles (e.g., QRA maturity) and inputs (e.g., PDFs, complexity) and outputs (e.g., accuracy) that apply to all QRA methods to some extent.

Selected References: [15]

  • Complexity and Analytics: Zangeneh, Pouya, L. McMullan, M. Pearson, and B. McCabe, 2021 AACE International Transactions, TCMA-3733, AACE International, Morgantown, WV, 2021.
    MEMBER DOWNLOAD | WATCH VIDEO | JOIN AACE

 

 

6. Quantify the Bias using Validation/Benchmarking

TCM 6.1 - Asset Performance Assessment
 TCM 7.2 - Schedule Planning and Development
 TCM 7.3 - Cost Estimating and Budgeting
 RP 110R-20: Cost Estimate Validation
RP -TBD-: Benchmarking

Discussion:
Every estimate and schedule is biased in some way. Establishing a cost and schedule strategy is a way to plan bias to meet strategic objectives (see: Basis, Target). Whether planned or not, it is necessary to quantify the bias as the first quantitative step of QRA because aggressiveness (optimism) or conservativeness (pessimism) in the base adds or deducts from the needed risk funds and durations to some extent. Also, aggressiveness implies taking risks and can be a risk driver in its own right. Validation is a quantitative practice for measuring bias (and to some extent quality) of the estimate and schedule. Validation relies mainly on comparisons of project cost and schedule metrics to historical metrics. Benchmarking is similar but focuses on comparison to metrics for external projects or project systems with a focus on competitiveness of the business process and/or project system. Validation and benchmarking are typically closely linked to project historical database management as well (see #11) (e.g., validation metrics may be generated directly by the database system). Reference class forecasting (RCF) is a limited form of benchmarking that is used mainly in the transportation sector to identify potential bias.

Selected References: [16,17]

  • Benchmarking: Stephenson, H. Lance and P. Bredehoeft, “Benchmarking for Competitive Advantage”, 2020 AACE International Transactions, TCMA-3502, AACE International, Morgantown, WV, 2020.
    MEMBER DOWNLOAD | WATCH VIDEO | JOIN AACE

  • RCF: Devine, J. Michael and Rob Adkison, “Reference Class Forecasting: Theory and Practical Application”, 2019 AACE International Transactions, OWN-3056, AACE International, Morgantown, WV, 2019
    MEMBER DOWNLOAD | WATCH VIDEO | JOIN AACE

 

 

7. Quantify Uncertainty and Risk in Support of Decision Making

TCM 7.6 - Risk Management

Discussion:
Each QRA method is discussed in its own sub-section by general type of method. Also see the "Methods Application Guide" (Figure 4) later in this document.

 

Predetermined Guidelines

RP 119R-21: Cost Estimate Accuracy Range and Contingency Determination Using Tables Derived from Parametric Risk Models

Discussion:
This type of method may vary from simple contingency percentages (e.g., 20, 15, and 10 percent for Class 5, 4 and 3) to elaborate multi-dimensioned tables of percentages that vary by other parameters such as the level of project technology. Unfortunately, pre-determined percentages are often not realistic and may have little empirical basis; such values are generally not recommended. However, RP 119R-21 provides tables of p10/50/90 cost growth percentages for various levels of project scope definition, technology and complexity (i.e., the key systemic risk drivers). It is based on the RAND parametric cost model that is provided in RP 43R-08). It is not a separate method; just a way of reporting and applying the RAND model parametric results. The RP can be used as a guide of how to create similar tables from other parametric models.

 

Parametric Method

RP 42R-08: Risk Analysis and Contingency Determination Using Parametric Estimating

RP 43R-08: Example Models as Applied for the Process Industries

Discussion:
Per the RP text: ...the method is used to estimate contingency based on risk parameters (e.g. level of scope definition, process complexity, etc.). This RP includes practices for developing the parametric methods and models (generally empirically-based). The method covers parametric modeling of both cost and schedule outcomes, but it is focused on quantifying systemic risks which have fairly universal implications. Being based on multiple linear regression (MLR), the method is probabilistic, but unlike with MCS the output distributions represent plots based on actual data rather than simulation generated data. However, reliance on actual data is a constraint on what can be modeled when quality data is limited.

The parameters in a model are quantitative ratings of systemic risks (e.g., the estimate classification level number is an example rating of the level of scope definition). The method produces single output distributions for cost and schedule; the challenges this aggregation presents to contingency allocation and management are discussed in those sections. The RP also covers the method to calibrate an existing parametric model (e.g., the RAND model in RP 43R-08) to align with a company’s internal data.

RP 43R-08 is an adjunct to RP 42R-08 and includes working Excel® versions of the John Hackney and RAND Corporation parametric models. The Hackney model covers cost, while RAND provided models for both cost and construction schedule. (Note: open the RP using the original Adobe Acrobat Reader to see the links to the Excel attachment). The RAND cost model was used to develop the tables in RP 119R-21.

 

MLR and the parametric method, being data-driven, might be considered a first step toward machine learning and artificial intelligence (ML/AI) albeit limited to a somewhat static basis in historical data, rather than a dynamic basis in contemporaneous data. It is expected that ML/AI methods will eventually supersede the parametric method.

Hybrid Use:
The parametric method can be used alone at early phases (i.e., Class 10/5). In later phases, it is typically applied in combination with a simulation method to address identified critical project-specific risks. Hybrid applications include RP 113R-21 (parametric + EV) and RP 117R-21 (parametric + CPM-based).

 

 

Simulation (Monte-Carlo Simulation)

General Discussion:
Monte-Carlo simulation (MCS) is an inferential statistical method that generates a distribution of possible outcomes from a mathematical model. MCS output distributions represent plots based on simulation generated data rather than actual data as per MLR (parametric); however, MCS frees users from the constraint of data limitations. However, being unconstrained by actual data, it puts a premium on assuring the base QRA model is actually modeling risk behavior and the inputs are reasonable. MCS generates outputs by sampling from model variables for which fixed values have been replaced with distributions of possible values (see RP 66R-11 for distribution types) while considering dependencies between the model variables where they are not independent.


There are various RPs in the MCS group that are intended to meet different needs. See the "Methods Application Guide" (Figure 4) later in the PGD for more information. This group of RPs include methods using estimate ranging, CPM-based, and expected value models with MCS. They all address the RP 40R-08 principles when applied within the usage constraints defined by the RPs and discussed in the application guide (e.g., some apply only to certain risk types or certain estimate classes, etc.).

 

Note on Joint Confidence:
Any MCS model that analyzes both cost and schedule in an integrated way can be used to assess joint confidence; i.e., setting cost and schedule contingency together based on a single defined probability of underrunning both (i.e., the joint confidence level or JCL) [18, 19].

 

Note on Hybrid Use:
See the discussion on hybrid method application in respect to optional ways that systemic risks or background variability may be evaluated using parametric or estimate ranging methods respectively and then integrated into either a CPM or expected value based MCS simulation.

RP 57R-09: Integrated Cost and Schedule Risk Analysis Using Risk Drivers and Monte Carlo Simulation of a CPM Model

Plus: RP 64R-11: CPM Schedule Risk Modeling and Analysis: Special Considerations (Adjunct)

Plus: RP 132R-23: Schedule Risk Analysis Maturity Model

57R-09 Discussion:
Per the RP introduction: The methods are based on integrating the cost estimate with the project schedule by resource-loading and costing the schedule’s activities. The probability and impact of risks/uncertainties are specified and the risks/uncertainties are linked to the activities and costs that they affect. The method makes optimal use of the schedule model; the analysis of how a schedule behaves under risk results in an improved, risk-robust plan (i.e., the method can be considered a risk-aware schedule development and optimization practice in addition to just QRA).

64R-11 Discussion:
This is an adjunct to RP 57R-09; not a stand-alone method. Per the text: This RP discusses key procedural, analytical and interpretive considerations in preparation and application of a CPM model; considerations that were not covered in the broader methodological RPs (i.e., RP 57R-09). Topics include schedule quality, schedule model building, linking risks to activities, and interpreting results. The rigor and complexities of CPM based methods may lead one to use the less imposing EV method (RP 65R-11) for less strategic projects.

132R-23 Discussion:

This is a supplement to RP 57R-09 and RP 64R-11. This RP provides a rating scheme from levels 0 to 5 to measure the levels of maturity of an organization’s capability and use of the CPM-based quantitative risk analysis methods covered in RP 57R-09 and RP 64R-11. Note that the levels are not consistent with the quantitative risk analysis maturity model (QRAMM) in RP 122R-22 and the RP does not address the use of CPM-based methods in hybrid application per RP 117R-21.

Hybrid Use:
See RP 117R-21 for use of the RP 57R-09 CPM-based method in combination with the RP 42R-08 parametric method. However, RP 57R-09 addresses the modeling of systemic risks (via interviews) if the hybrid method is not used.

RP 44R-08: Risk Analysis and Contingency Determination Using Expected Value (EV)

44R-08 Discussion:
The method in this RP addresses cost risk only (see the "Methods Application Guide" in Figure 4 for limitations). With EV, the probability of occurrence of each critical risk is multiplied by a cost impact distribution (p x i) to calculate an expected value. The sum of all risk EV results is calculated and MCS is applied to the model. The RP covers various analytical tasks and challenges. RP 65R-11 extends this method to provide integrated cost and schedule risk analysis using EV and is recommended over using entry level RP 44R-08 alone.

Hybrid Use:
This RP is offered as a building block for hybrid methods that cover various risk types (within the noted limitations of each method). RP 44R-08 is used in hybrid RP 113R-21 (parametric + EV for cost) and RP 123R-22 (estimate ranging + EV for cost)

RP 65R-11: Integrated Cost and Schedule Risk Analysis and Contingency Determination Using Expected Value (EV)

65R-11 Discussion:
This RP extends the EV method in RP 44R-08 to add analysis of the schedule duration impact of identified critical risks. The cost and schedule impact of each risk is based on a conceptually planned risk response (what will be done if the risk occurs). As such it is an integrated cost and schedule method (i.e., it supports joint confidence level determination), albeit without direct use of a CPM model. It is a practical method for non-strategic projects and/or those without a quality CPM (schedule analysis is intuitive).

Hybrid Use:
This RP is offered as a building block for hybrid methods that cover various risk types (within the noted limitations of each method). See hybrid RP 113R-21 (parametric + EV for cost and schedule) for use in modeling systemic and project-specific risks.

RP 41R-08: Understanding Estimate Ranging (replaces the former Risk Analysis and Contingency Determination Using Range Estimating)
 RP 118R-21: Cost Risk Analysis and Contingency Determination Using Estimate Ranging for Inherent Risks with Monte Carlo Simulation

41R-08 Discussion:
Describes and provides considerations for a general class of estimate ranging QRA methods, but is not a QRA method RP in itself.

118R-21 Discussion:
See RP 41R-08 for general discussion of the estimate ranging class of methods. In general, estimate ranging with MCS replaces fixed values in a cost estimate with 3-point or uniform distributions based on team input. This method is only recommended for quantifying inherent cost uncertainty (see "uncertainties versus risk events" terminology discussion) when the project scope is well-defined (i.e., Class 3 or better) and when the project has no new technology and minimal complexity. In all cases, the method is not to be used alone when there are significant (critical) project-specific risks (e.g., contingent risks or events). With these limitations, its use is generally limited to smaller, lower risk projects, or as part of a hybrid method.

Hybrid Use:
See RP 123R-22 (estimate ranging + EV) for use of this method in combination with expected value for cost when there are no significant systemic risks, but significant or critical project-specific risks.

 

Hybrid Methods

General Discussion:
Hybrid methods are various combinations of the parametric and simulation methods described in the prior sections. Methods are combined because no single method is optimal for all risk types and situations. See the "Method Application Guide" section (Figure 4) for more information. The descriptions below are in the order of increasing coverage of risk types and situations.

RP 123R-22: Integrated Cost and Schedule Risk Analysis and Contingency Determination Using Estimate Ranging and Expected Value with Monte Carlo Simulation

Discussion:
This is a hybrid of RP 118R-21 and RP 65R-08 for projects with no significant systemic risks and not significantly benefiting from an analysis of intermediate schedule milestone behavior. It adds the ranging of overall schedule inherent risk to the RP 118R-21 treatment of cost to provide an integrated cost and schedule analysis.

RP 113R-21: Integrated Cost and Schedule Risk Analysis and Contingency Determination Using Combined Parametric and Expected Value

Discussion:
This is a hybrid of RP 42R-08 and RP 65R-08 for projects of most types, but not significantly benefiting from an analysis of intermediate schedule milestone behavior (see RP 117R-21 in that case).

RP 117R-21: Integrated Cost and Schedule Risk Analysis and Contingency Determination Using a Hybrid Parametric and CPM Method

This is a hybrid of RP 42R-08 and RP 57R-09 (without the RP 57R-09 use of interviews to quantify systemic risks) for projects of any type that have a quality CPM schedule (typically Class 3 or better).

 

Escalation

58R-10: Escalation Principles and Methods Using Indices
68R-11: Escalation Estimating Using Indices and Monte Carlo Simulation

Discussion:

From the RP text: Defines basic principles and methodological building blocks for estimating escalation using forecasted price or cost indices while also addressing uncertainty using Monte Carlo simulation. RP 58R-10 describes a deterministic or base estimating method while RP 68R-11 is an addendum adding probabilistic and scenario/sensitivity analysis to the base method. The RP starts with extending the RP 40R-08 principles to cover escalation.

 

A feature of RP 68R-11 is that it uses the distribution outputs from cost and schedule contingency models as inputs to the probabilistic escalation model. This integration produces outcome distributions of not only escalation cost, but total CAPEX including cost and schedule contingency; this universal CAPEX distribution can then be readily applied in probabilistic NPV models.

 

Currency Exchange

No RPs

Discussion:
Currency exchange risks are related to escalation risks in that they are all driven by the external economy. It addresses uncertainty in the relative value of money (exchange rate) between the estimate basis currency and the currencies paid out during the project. Forecasts of future exchange rates (i.e., indices) are usually provided by the same econometric sources as for prices. Currency risk analysis is similar to escalation in that the model starts with a cash flow by account by period, but in this case, the cash flow is expanded to also be by currency where there is non-base currency spending (which may or may not be known). The forecast exchange rates by period are then applied to each non-base cost either deterministically or probabilistically.

 

One difference between escalation and currency risk is that finance departments may indemnify the project for currency risk on the premise that through hedging or similar practices, it is their role to somewhat mitigate currency risk, not the project team’s. Also, project cost control may be by currency and risk methods may need to align with control practices. The use of such practices need to be considered for each project.

Selected References: [20,21,22]

  • Currency: Wenger, Natalia and Jochun W. Lai, “Project Cost Management in Multiple Currencies: A Case Study”, 2014 AACE International Transactions, CSC.1711, AACE International, Morgantown, WV, 2014.
    MEMBER DOWNLOAD | JOIN AACE

  • Exposure Management: Fritsche, Hans, “Transaction Exposure Management in International Construction”, 1994 AACE International Transactions, INT.8, AACE International, Morgantown, WV, 1994.
    MEMBER DOWNLOAD | JOIN AACE

  • Escalation and Currency: Hollmann, John, “Project Risk Quantification", Chapter 13: Probabilistic Escalation and Currency Exchange”, Probabilistic Publishing, Sugar Land TX, 2016.
    PURCHASE ON AMAZON

 

Long-Range/Life Cycle (Class 10)

RP -TBD-: QRA for Unclassified/Class 10 Estimates

Discussion:
Unclassified/Class 10 estimates are prepared 10 or more years in advance of the project occurring. These are usually associated with portfolio management (see Programs and Portfolios) and life cycle cost estimating, but also for evaluating surety for future requirements (e.g., bonding of mine closure costs). With such long time-horizons, it is typical that key estimate scope basis assumptions will change (e.g., technologies, regulations, etc.) which adds uncertainty. The base estimate methods are usually similar to Class 5 (i.e., conceptual, modeling, etc.) and so too are some of the possible QRA methods (e.g., parametric). However, scenario analysis (a common practice in portfolio management) aligns well with probabilistic decision tree approaches.

Selected References: [3]

Class 10 QRA Method: Hollmann, John, “Risk Analysis and Contingency Estimating for Class 10 Estimates”, AACE International Transactions, RISK-3908, AACE International, Morgantown, WV, 2022.
See section on "Application of QRA in Strategic Asset (Portfolio) Management"

 

Complexity/Non-Linearity Methods

No RPs

Discussion:

Empirical research has indicated that cost and schedule outcome distributions of actual/estimate metrics are sometimes not well modeled by traditional QRA simulation (e.g., actual distributions sometimes exhibit bimodality, fat/long overrun tails, etc.). It is hypothesized that as complexity and other systemic risks increase, the failure mode of projects can shift from orderly to disorderly or chaotic (e.g., blowouts). Methods have been published that apply methods of measuring or rating complexity, and interaction with other risks, and applying that to QRA models in a way that generates non-linear outcomes such as seen in some empirical research.

Selected References: [23,24,25]

  • Research: Ogilvie, Alexander (IPA, Inc.), “A Tale of Two Tails: Chaos in Estimating Predictability”, Cost Engineering, Vol. 59, No. 03, AACE International, Morgantown WV, 2017.
     MEMBER DOWNLOAD | WATCH VIDEO | JOIN AACE

  • SME Assessment: Raydugin, Yuri, “Non-linear Probabilistic (Monte-Carlo) Modeling of Systemic Risks”, 2018 AACE International Transactions, RISK-2808, AACE International, Morgantown WV, 2018.
     MEMBER DOWNLOAD | WATCH VIDEO | JOIN AACE

  • Hybrid: Hollmann, John, “Risk Analysis at the Edge of Chaos”, Cost Engineering, Vol. 57, No. 01, AACE International, Morgantown WV, 2015.
    MEMBER DOWNLOAD | WATCH VIDEO | JOIN AACE

 

Programs and Portfolios

No RPs

Discussion, Programs:

As stated in TCM 7.1: Sometimes a project is part of a program or set of related projects. A program can be viewed as the top level on the WBS with the next level including the projects within the program. The basic project control concepts apply to a program with the added complication of integrating its projects. This same complication arises with contingency; each project manager in a program requires a contingency budget, and so to should the program director to address integrative risks that manifest themselves at the program level. This could be in addition to management reserve. As with control, the same QRA concepts apply to a program as for projects, but with different or compounded risks.

 

Discussion, Portfolios:

Managing a portfolio of projects is usually part of an overall business asset or capital management process as opposed to program/project management per se. Portfolio risk analysis is more from a financial planning viewpoint of how much capital is needed and when considering changing business strategy and needs over time, and uncertain performance (i.e., risks) of programs and projects that make up the portfolio. Because portfolio management tracks projects over time, the data captured can be used to improve individual project QRA methods and models (e.g., if projects are over/under spending their contingency, calibrate the contingency method or p-value funded).

Selected References: [26,27,28]

  • Program/Portfolio Funding: Stephenson, H. Lance and Robert Gerber, “Strategic Portfolio Management: Funding and Finance Methodologies”, 2020 AACE International Transactions, TCM-3503, AACE International, Morgantown, WV, 2020.
    MEMBER DOWNLOAD | WATCH VIDEO | JOIN AACE

  • Program/Portfolio Funding: Alves, Ricardo Gonçalves, “Portfolio Management: Case Study of a Brazilian Mining Company”, 2019 AACE International Transactions, PM-3085, AACE International, Morgantown, WV 2019.

    MEMBER DOWNLOAD | WATCH VIDEO | JOIN AACE

  • Programs Using Parametric/EV Hybrid: Hollmann, John, “Project Risk Quantification", Chapter 7: Introduction to Risk Quantification Methods, Probabilistic Publishing, Sugar Land TX, 2016.
    PURCHASE ON AMAZON

 

Management Reserves

No RPs

Discussion:

The definition of management reserves (MR), and its distinction from contingency was discussed earlier. Given that MR determination is at management discretion, there are no specific RPs. However, there are generally two approaches to determination. The first is to address management’s general risk tolerance and is typically funded at some elevated probability of underrun value (e.g., p70 to 90) with MR being the difference between this value and contingency (usually set at p50 or the mean). The other approach is for funding low probability, high impact risks not amenable to contingency funding (e.g., funding a small value based on p x i is illogical). The Hollmann reference below suggests removing those risks from the contingency QRA and probabilistically analyzing each potential high impact/low probability (HILP), risk separately for decision making by the business. Both methods can be applied at management discretion. Management reserve can be established at a program level as well.

Selected References: [29,30]

  • Using CPM-Based Model: Vinueza C, Gustavo, “Optimizing Your Management Reserve”, AACE International Transactions, 2019 OWN-3236, AACE International, Morgantown WV, 2019.

    MEMBER DOWNLOAD | WATCH VIDEO | JOIN AACE

  • Using EV Hybrid Model: Hollmann, John, “Project Risk Quantification", Chapter 12: Project Specific Risks and the Expected Value Method, Probabilistic Publishing, Sugar Land TX, 2016.
     PURCHASE ON AMAZON

 

 

Analytics (Future)

No RPs

Discussion:

Discussion: Analytics is shorthand for QRA algorithms and methods developed using machine learning and artificial intelligence (ML/AI). These methods depend on having quality data. These methods will likely come to dominate the QRA world in the future. At this time, the parametric method covered in RP 42R-08, which is based on “manual” regression analysis of data, can be viewed as a baby-step toward machine learning based QRA algorithms and methods of the future. Eventually, AI applications will tie QRA ever closer to, or embedded in, decision making algorithms. The AACE “Data Science & Advanced Analytics” (DSAA) technical subcommittee and transactions should be followed because references in this field will quickly become dated.

Selected References: [31]

  • Machine Learning: Schedule Risk: Hovhannisyan, Vahan, Peter Zachares, Alan Mosca, Yael Grushka-Cockayne, and Carlos Ledezma, "Data-Driven Schedule Risk Forecasting for Construction Mega-Projects", 2023 AACE International Transactions, DSAA-4101, AACE International, Morgantown, WV, 2023
    MEMBER DOWNLOAD | WATCH VIDEO | JOIN AACE

 

 

 

8. Application of QRA in Strategic Asset (Portfolio) Management

TCM 3.2 - Asset Planning

TCM 3.3 - Investment Decision Making

RP 85R-14: Use of Decision Trees in Decision Making

TCM 3.2 – Asset Planning:

This process is focused on identifying potential alternative solutions (including projects but also other options) to asset portfolio needs and requirements. The analysis of alternatives often takes place as a business planning process (i.e., asset portfolio management) outside the stage-gate process (see #1). Unclassified/Class 10 estimates for projects with >10 years, with associated QRA considerations (see section on Class 10) are often applied in early asset life cycle cost analyses in this process. Scenario or what-if analysis is often applied in this process (No RPs).

TCM 3.3 - Investment Decision Making:

In general, decision analysis of alternatives in consideration of risk as covered by TCM 3.3 is a separate topic with the exception of decision trees (RP 85R-14) which can also be employed in general QRA practice. Decision analysis involves more inputs than the capital cost and schedule risks covered by the QRA methods in this RP (e.g., must consider revenue and OPEX as well as CAPEX in NPV analysis). However, there are three areas where alignment between business decision analysis and project risk analysis is worth highlighting. One is that the start of the revenue stream in NPV analysis is directly driven by the project completion date quantified in project QRA. Another is that CAPEX price escalation and revenue (sell price) are driven by economic conditions and hence their analysis should be based on common assumptions and scenarios. Finally, the nature of a business’s interaction with stakeholders and project teams and the efficacy of its capital management system are systemic risks to a project; i.e., the business owns most project systemic risks to some extent. As was noted, the probabilistic escalation method generates a universal capex cost distribution that can be readily used in NPV models.

Selected References: [32,26]

  • Stephenson, H. Lance, “Strategic Portfolio Management: Asset Management Model”, 2021 AACE International Transactions, TCM-3775, AACE International, Morgantown, WV, 2021.
    MEMBER DOWNLOAD | WATCH VIDEO | JOIN AACE

  • Stephenson, H. Lance and Robert Gerber, “Strategic Portfolio Management: Funding and Finance Methodologies”, 2020 AACE International Transactions, TCM-3503, AACE International, Morgantown, WV, 2020.
    MEMBER DOWNLOAD | WATCH VIDEO | JOIN AACE

 

 

9. Project Control Planning and Contingency Allocation

TCM 8.1 - Project Control Plan Implementation

General Discussion:
The RP 10S-90 definition of contingency expresses or implies two principles relative to the issue of allocation. The first principle is that contingency is only meaningful in aggregate (aggregation principle). The second principle is that contingency is an account under the spending authority of the project manager or high level WBS element manager (authority or control principle).


The aggregation principle derives from the fact that risk quantification and contingency determination are stochastic and hence cannot be reliably allocated to lower level control accounts (e.g., activities, items or disciplines). While it is technically possible to drive project-specific risk impact quantification calculations to increasingly finite levels of cost or activity impact detail, this increasingly implies deterministic behavior. Quantifying systemic risk impacts is even more stochastic and less reliable.


The authority principle means high-level WBS managers with wide authority may require contingency to deal with risks as they occur in the sub-project in their control. This justifies allocating contingency at an appropriate aggregation level. For this reason, the only RP for this topic is about contract risk allocation, not low-level cost account allocation which is generally not recommended.


For analysis by and allocation to sub-projects, QRA methods applicable to programs apply.

RP 67R-11: Contract Risk Allocation as Applied in Engineering, Procurement, and Construction

Discussion:
As stated in RP 67R-11: The burden of contingency funding and its management may fall to different entities. Whoever underwrites the risk of cost or schedule overrun must be capable of utilizing the appropriate contingency and, irrespective of the contractual arrangement, the principles of contingency derivation and management remain unchanged. This RP covers the principles of allocating risks, and associated funds and durations, to contractors; primarily a risk treatment practice. QRA practices must consider the nature and manner of this allocation.

 

 

10. Change Management and Contingency Management

TCM 8.1 - Project Control Plan Implementation
 TCM 10.3 - Change Management

No RPs

TCM 10.3 - Change Management:
The project change management process covered in TCM 10.3 includes a step for evaluating the risks of trends, deviations and proposed changes to support change management decisions. While scope changes may be elevated to business investment decision making, most change management decisions are tactical and limited in nature and involve less rigorous analysis. The analysis may include elements of formal QRA, but there is no RP specifically for QRA during execution. However, change management processes often call for full QRA to be conducted as part of periodic, milestone-based, or as needed assessment during execution. An element of change management is contingency and reserves management as per discussion below.

 

TCM 10.3.2.7 - Manage Contingency and Reserves:
Once a budget for contingency has been determined, there are quantitative steps for its time-phased budgeting and then for later contingency update. During execution, the use of contingency, as controlled by the change management process (see above) working with risk management, is plotted against a contingency time-phased spending curve (draw-down curve) for reporting and assessment. There are two general approaches to contingency management that reflect differences in viewpoint as to the nature of risk behavior and its manageability. The words stochastic and deterministic are used to distinguish them although both approaches employ probabilistic methods, and continually manage risks (e.g., maintain risk registers, etc.) during execution.


Stochastic Methods:
These assume the primacy of the aggregation principle (see Contingency Allocation); i.e., that QRA methods are highly uncertain approximations (e.g., a Class 5 estimate) that are not amenable to allocation either by account or risk. As such, just as contingency cannot be allocated by account, its consumption cannot be rigorously forecast against time. A common stochastic method is to plot drawdown in proportion to spending of the base accounts known to be particularly uncertain (e.g., proportional to labor and labor-related spending). During execution, if the identified risks change materially and/or contingency use is out-of-line with the plan curve, and at a minimum at pre-determined milestones, overall QRA is re-done (using methods in Section 7) for the scope of work remaining (including systemic risks which may degrade during execution). The approach is not dependent on the quality of the base estimates and schedules or rigor of control practices.

 

Deterministic Methods:
These typically assume that QRA calculations establish a meaningful basis for planning contingency consumption, risk-by-risk. That includes assuming systemic risks and their impacts can be parsed (i.e., it is not amenable to the parametric QRA method). Further, those using this method also tend to track contingency drawdown by using the risk register as a mechanistic control tool whereby when a risk is no longer active, it’s cost impact estimate is subtracted from remaining contingency. As risks evolve and as new risks are added, the plan is revised as determined by risk-by-risk quantification. Overall QRA may be re-assessed periodically or as needed using methods in Section 7. The approach is dependent on high quality base estimates and schedules and rigorous control practices.

Selected References: [33,34,35]

  • General: Stephenson, H, Lance, “Change Management for the Project Life Cycle”, 2022 AACE International Transactions, TCM-3934, AACE International, Morgantown WV, 2022.
     MEMBER DOWNLOAD | WATCH VIDEO | JOIN AACE

  • Stochastic: Hollmann, John, “Project Risk Quantification", Chapter 17: Budgeting for Risks and Account Control , Probabilistic Publishing, Sugar Land TX, 2016.
    PURCHASE ON AMAZON

  • Deterministic: Stroemich, Christopher, and M. Dissanayake, “Project Risk Drawdown - A Structured Approach to Contingency Management”, Cost Engineering, Vol. 61, No. 06, AACE International, Morgantown WV, 2019.
     MEMBER DOWNLOAD | WATCH VIDEO | JOIN AACE

 

 

11. Maintain Methods and Tools Considering Historical Data

TCM 10.4 - Project Historical Database Management

RP 114R-20: Project Historical Database Development

Discussion:

The use of historical data is a principle of QRA methods per RP 40R-08. This is particularly true for parametric modeling (re: RP 42R-08) which is directly based on historical data using MLR. However, other methods rely on data to some extent. For example, history should inform the factors in predetermined guidelines, provide lessons learned and typical risk behavior for simulation analysis. Also, weather history is often used to inform quantification of weather event risks. It is also essential for the development of comparison metrics for validation methods. The capture and management of data to support QRA is the same process as for base estimating and schedule development purpose. An area of current and future practice development in respect to data analysis is machine learning and AI (ML/AI). While ML and AI may use unstructured data from many sources, in the near term, the focus of database development to support AI/ML for practical QRA applications is on structured data Follow the AACE Data Science & Advanced Analytics (DSAA) Technical Subcommittee and transactions where database development to support ML/AI is a key focus area.

 

 

METHODS APPLICATION GUIDE 

 

Figure 4 provides a guide to which QRA RP methods (listed in Section 7 and shown in the Figure 1 index map) are likely to be most fit-for-use in various project scope and risk situations (e.g., project complexity, estimate class, etc.). A Venn diagram is used to highlight applicability to various risk types (i.e., systemic, inherent and contingent) that are a key method selection determinant. The colors in the Venn diagram represent the following:

The method selection factors summarized in Figure 4 include:

The final selection always calls for reviewing and understanding the RPs themselves. Per the QRA maturity model (RP 122R-22), an organization at Level 3 or above maturity will have a suite of methods available to best meet its various needs.

Figure 4. QRA Method Application Guide

 

 

REFERENCES

 

[1]

Hulett, David T., “Journey Map to a More Mature Schedule Risk Analysis (SRA) Process, Cost engineering Journal, AACE International, Morgantown, WV, March/April 2019.

MEMBER DOWNLOAD | WATCH VIDEO | JOIN AACE

[2]

Hollmann, John, “Project Risk Quantification, Glossary”, Probabilistic Publishing, Sugar Land TX, 2016.
PURCHASE ON AMAZON

[3]

Hollmann, John, “Risk Analysis and Contingency Estimating for Class 10 Estimates”, AACE International Transactions, RISK-3908, AACE International, Morgantown, WV, 2022.
MEMBER DOWNLOAD | WATCH VIDEO | JOIN AACE

[4]

Dysert, Larry and T. Pickett, “Supporting Estimates with Effective Scope of Work Definition”, 2020 AACE International Transactions, EST-3424, AACE International, Morgantown, WV, 2020.
MEMBER DOWNLOAD | WATCH VIDEO | JOIN AACE

[5]

Ahmad, Mir M. and Christopher Carson, “Capital Improvement Program Stage-Gate Planning and Scheduling”, 2019 AACE International Transactions, PS-3112, AACE International, Morgantown, WV, 2019.
MEMBER DOWNLOAD | WATCH VIDEO | JOIN AACE

[6]

Carson, Christopher W. and Leo Carson-Penalosa, "Implementation of an Integrated Phase-Gate Project Controls Process", 2023 AACE International Transactions, TCM-4048, AACE International, Morgantown, WV, 2023

MEMBER DOWNLOAD | WATCH VIDEO | JOIN AACE

[7]

Barshop, Paul, “Capital Projects: What Every Executive Needs to Know to Avoid Costly Mistakes and Make Major Investments Pay Off”, John Wiley & Sons, Inc., Hoboken, New Jersey, 2016.
PURCHASE ON AMAZON

[8]

Stephenson, L. and M. Pruneau, “Strategic Portfolio Management: Improving Capital Utilization and Competitive Advantage”, 2016 AACE International Transactions, TCM-2110, AACE International, Morgantown, WV, 2016.
MEMBER DOWNLOAD | WATCH VIDEO | JOIN AACE

[9]

Hollmann, J., (Presentation Only) “Cost Estimate Classification Overview”, EST-3480, 2020 AACE International Virtual Conference & Expo.
WATCH VIDEO | JOIN AACE

[10]

Zaheer, Syed H. and C. Fallows, “Document Project Readiness by Estimate Class Using PDRI”, 2011 AACE International Transactions, EST-604, AACE International, Morgantown WV, 2011.
MEMBER DOWNLOAD | JOIN AACE

[11]

CII Project Definition Rating Index (PDRI) Overview.
CII

[12]

Opfer, Neil, “Where & Why Value Engineering Goes Wrong with Capital Projects”, 2021 AACE International Transactions, TCM-3707, AACE International, Morgantown WV, 2021.

MEMBER DOWNLOAD | WATCH VIDEO | JOIN AACE

[13]

Dell’Isola, Michael D., “Better Use of Value Engineering in Project Delivery”, Cost Engineering, Vol. 56, No. 06, AACE International, Morgantown, WV, 2014.
MEMBER DOWNLOAD | WATCH VIDEO | JOIN AACE

[14]

Lozon, Jim and G. Jergeas, “The Use and Impact of Value Improving Practices and Best Practices”, Cost Engineering, Vol. 50, No. 06, AACE International, Morgantown, WV, 2008.
MEMBER DOWNLOAD | JOIN AACE

[15]

Zangeneh, Pouya, L. McMullan, M. Pearson, and B. McCabe, 2021 AACE International Transactions, TCMA-3733, AACE International, Morgantown, WV, 2021.
MEMBER DOWNLOAD | WATCH VIDEO | JOIN AACE

[16]

Stephenson, H. Lance and P. Bredehoeft, “Benchmarking for Competitive Advantage”, 2020 AACE International Transactions, TCMA-3502, AACE International, Morgantown, WV, 2020.
MEMBER DOWNLOAD | WATCH VIDEO | JOIN AACE

[17]

Devine, J. Michael and Rob Adkison, “Reference Class Forecasting: Theory and Practical Application”, 2019 AACE International Transactions, OWN-3056, AACE International, Morgantown, WV, 2019.
MEMBER DOWNLOAD | WATCH VIDEO | JOIN AACE

[18]

Steiman, Samuel and Dr. David T. Hulett, “Identifying the Most Probable Cost – Schedule Values from a Joint Confidence Level (JCL) Risk Analysis”, AACE International Transactions, RISK-3111, AACE International, Morgantown, WV, 2019.
MEMBER DOWNLOAD | WATCH VIDEO | JOIN AACE

[19]

Hollmann, John K., “Realistic and Practical Project Risk Quantification (Without CPM)”, Cost Engineering, Vol. 60, No. 04, AACE International, Morgantown, WV, 2018.
MEMBER DOWNLOAD | WATCH VIDEO | JOIN AACE

[20]

Wenger, Natalia and Jochun W. Lai, “Project Cost Management in Multiple Currencies: A Case Study”, 2014 AACE International Transactions, CSC.1711, AACE International, Morgantown, WV, 2014.
MEMBER DOWNLOAD | JOIN AACE

[21]

Fritsche, Hans, “Transaction Exposure Management in International Construction”, 1994 AACE International Transactions, INT.8, AACE International, Morgantown, WV, 1994.
MEMBER DOWNLOAD | JOIN AACE

[22]

Hollmann, John, “Project Risk Quantification", Chapter 13: Probabilistic Escalation and Currency Exchange”, Probabilistic Publishing, Sugar Land TX, 2016.
PURCHASE ON AMAZON

[23]

Ogilvie, Alexander, “A Tale of Two Tails: Chaos in Estimating Predictability”, Cost Engineering, Vol. 59, No. 03, AACE International, Morgantown, WV, 2017.
MEMBER DOWNLOAD | WATCH VIDEO | JOIN AACE

[24]

Raydugin, Yuri, “Non-Linear Probabilistic (Monte Carlo) Modeling of Systemic Risks”, 2018 AACE International Transactions, RISK-2808, AACE International, Morgantown, WV, 2018.
MEMBER DOWNLOAD | WATCH VIDEO | JOIN AACE

[25]

Hollmann, John, “Risk Analysis at the Edge of Chaos”, Cost Engineering, Vol. 57, No. 01, AACE International, Morgantown WV, 2015.
MEMBER DOWNLOAD | WATCH VIDEO | JOIN AACE

[26]

Stephenson, H. Lance and Robert Gerber, “Strategic Portfolio Management: Funding and Finance Methodologies”, 2020 AACE International Transactions, TCM-3503, AACE International, Morgantown, WV, 2020.
MEMBER DOWNLOAD | WATCH VIDEO | JOIN AACE

[27]

Alves, Ricardo Gonçalves, “Portfolio Management: Case Study of a Brazilian Mining Company”, 2019 AACE International Transactions, PM-3085, AACE International, Morgantown, WV 2019.
MEMBER DOWNLOAD | WATCH VIDEO | JOIN AACE

[28]

Hollmann, John, “Project Risk Quantification", Chapter 7: Introduction to Risk Quantification Methods, Probabilistic Publishing, Sugar Land TX, 2016.
PURCHASE ON AMAZON

[29]

Vinueza, J. Gustavo, “Optimizing Your Management Reserve”, 2019 AACE International Transactions, OWN-3236, AACE International, Morgantown, WV, 2019.
MEMBER DOWNLOAD | WATCH VIDEO | JOIN AACE

[30]

Hollmann, John, “Project Risk Quantification", Chapter 12: Project Specific Risks and the Expected Value Method, Probabilistic Publishing, Sugar Land TX, 2016.
PURCHASE ON AMAZON

[31]

Hovhannisyan, Vahan, Peter Zachares, Alan Mosca, Yael Grushka-Cockayne, and Carlos Ledezma, "Data-Driven Schedule Risk Forecasting for Construction Mega-Projects", 2023 AACE International Transactions, DSAA-4101, AACE International, Morgantown, WV, 2023

MEMBER DOWNLOAD | WATCH VIDEO | JOIN AACE

[32]

Stephenson, H. Lance, “Strategic Portfolio Management: Asset Management Model”, 2021 AACE International Transactions, TCM-3775, AACE International, Morgantown, WV, 2021.
MEMBER DOWNLOAD | WATCH VIDEO | JOIN AACE

[33]

Stephenson, H, Lance, “Change Management for the Project Life Cycle”, 2022 AACE International Transactions, TCM-3934, AACE International, Morgantown WV, 2022
MEMBER DOWNLOAD | WATCH VIDEO | JOIN AACE

[34]

Hollmann, John, “Project Risk Quantification", Chapter 17: Budgeting for Risks and Account Control , Probabilistic Publishing, Sugar Land TX, 2016.
PURCHASE ON AMAZON

[35]

Stroemich, Christopher, and M. Dissanayake, “Project Risk Drawdown – A Structured Approach to Contingency Management”, 2018 AACE International Transactions, RISK-2795, AACE International, Morgantown, WV, 2018.
MEMBER DOWNLOAD | WATCH VIDEO | JOIN AACE

 

  

Contributors

 

Disclaimer: The content provided by the contributors to this professional guidance document is their own and does not necessarily reflect that of their employers, unless otherwise stated.

 

March 18, 2024 Revision:

John K. Hollmann, PE CCP CEP DRMP FAACE Hon. Life (Primary Contributor)

Editorial, added links to recent RPs and several reference papers

 

August 23, 2022 Revision:

John K. Hollmann, PE CCP CEP DRMP FAACE Hon. Life (Primary Contributor)

Editorial, added links to 117R-21

 

January 31, 2022 Revision:

John K. Hollmann, PE CCP CEP DRMP FAACE Hon. Life (Primary Contributor)

 

December 20, 2019 Revision:

John K. Hollmann, PE CCP CEP DRMP FAACE Hon. Life (Primary Contributor)
Dr. Manjula Dissanayake, CCP

 

This document is copyrighted by AACE International and may not be reproduced without permission.