Examples of Technical and Management Reviews Systems Engineering

Technical Reviews and Audits

Purpose of Technical Reviews and Audits

For DoD weapon systems development, a properly tailored series of technical reviews and audits provide key points throughout the life cycle to evaluate pregnant achievements and assess technical maturity and risk. The Adaptive Acquisition Framework Document Identification (AAFDID) Tool presents the statutory, regulatory and milestone requirements for acquisition programs at these central program life cycle decision points. The Plan Manager (PM) and Systems Engineer work to properly align the technical reviews to support noesis-based milestone decisions that streamline the conquering life cycle and save precious taxpayer dollars. Technical reviews and audits allow the PM and Systems Engineer to jointly ascertain and control the plan's technical effort by establishing the success criteria for each review and audit. A well-defined programme facilitates effective monitoring and control through increasingly mature points.

Event-Driven Technical Reviews and Audits

Technical reviews of program progress should be event driven and conducted when the arrangement under development meets the review entrance criteria equally documented in the programme'south Systems Engineering science Programme (SEP). A cardinal associated activeness is to identify technical risks associated with achieving entrance criteria at each of these points (see the DoD Take chances, Issue and Opportunity Management Guide). Systems engineering (SE) is an event-driven process based on successful completion of key events as opposed to arbitrary calendar dates. As such, the SEP should clarify the timing of events in relation to other SE and program events. While the initial SEP and Integrated Master Schedule have the expected occurrence in the time of various milestones (such as overall system CDR), the plan should be updated to reverberate changes to the actual timing of SE activities, reviews and decisions. Figure 1 provides an example of the end-to-finish perspective and the integration of SE technical reviews and audits across the Major Adequacy Conquering (MCA) pathway life cycle.

Effigy 1: Weapon System Development Life Cycle (Major Capability Acquisition Pathway Case)

image image of weapons system development life cycle

Support of the Defence force Acquisition Organisation

Properly structured, technical reviews and audits back up the Defense Acquisition Organization by:

Providing a disciplined sequence of activities to ascertain, assess and command the maturity of the organization's design and technical baseline, reducing risk over time
Facilitating an accurate technical assessment of the system's ability to satisfy operational requirements established in capability requirements documents
Providing a framework for interaction with the Joint Capabilities Integration and Development Organization (JCIDS) and Planning, Programming, Budgeting and Execution (PPBE) processes
Providing a technical assessment and assurance that the cease product fulfills the design and process requirements

Knowledge-Based Approach

Successful development of a complex weapon arrangement requires a knowledge-based approach. Increasing levels of knowledge are a natural outcome of design maturation; yet, successful programs establish a deliberate acquisition approach whereby major investment decision points are supported by requisite levels of cognition. The Government Accountability Role's (GAO) written report on Assessments of Selected Weapons Programs (GAO-12-400SP) provides quantitative prove to affirm this best practice.

Timing and Tailoring of Technical Reviews and Audits

Effigy 1 illustrates the notional sequence of technical reviews and audits for the Major Adequacy Acquisition (MCA) Pathway. Information technology also provides typical timing associated with the MCA conquering phases. Regardless of acquisition pathway selected, technical reviews should occur when the requisite cognition is expected and required. OSD established the expected reviews and audits for each acquisition plan, unless waived by the SEP blessing potency, in DoDI 5000.88, Engineering for Defense force Systems, Section 3.v.a., and mandates technical reviews and audits be addressed in the program'due south SEP per the DoD SEP Outline v4.0, Section three.2.12. These policy and guidance documents provide a starting point for the PM and Systems Engineer to develop the program's unique fix of technical reviews and audits. Tailoring is expected to best adjust the program objectives. The program'south SEP captures the output of this tailoring and is reviewed and approved to solidify the plan program. Programs that tailor the timing and scope of these technical reviews and audits to satisfy program objectives increment the probability of successfully delivering required adequacy to the warfighter. Technical reviews provide the forum to frame important issues and assumptions. They define options necessary to rest take a chance in support of connected development.

Technical Baselines and Technical Reviews

The technical baseline (including the functional, allocated and production baselines) established at the conclusion of certain technical reviews inform all other programme activity. Accurate baselines and disciplined reviews serve to integrate and synchronize the organisation as it matures, which facilitates more effective milestone decisions and ultimately provides better warfighting capability for less money. The technical baseline provides an authentic and controlled basis for:

Managing change.
Cost estimates, which inform the Planning, Programming, Budgeting, and Execution (PPBE) process and the program'southward Acquisition Program Baseline (APB).
Program technical plans and schedules, which also inform the APB.
Contracting action.
Test and Evaluation efforts.
Run a risk assay and adventure balancing.
Reports to acquisition executives and Congress.

Organization Developers and Technical Reviews

The PM and the Systems Engineer need to keep in mind that technical reviews and audits provide visibility into the quality and completeness of the developer'south piece of work products. These requirements should be captured in the contract specifications or Statement of Work (SOW). The plan function should consider providing the SEP with the Asking for Proposal (RFP) and requiring the contractor deliver a SE Management Programme (SEMP) that is consistent with the SEP. As a best do, the SEMP should include entrance criteria and associated design data requirements for each technical review and inspect. The configuration and technical information management plans should clearly ascertain the audit requirements.

Contract incentives are oft tied to completion of technical reviews. Some stakeholders may have a strong incentive to telephone call the review consummate as soon as possible. The review chairperson and Systems Engineer should exercise best judgment in an objective, informed style to ensure the reviews are not prematurely declared complete.

Circuitous Systems and Incremental Reviews and Audits

For circuitous systems, reviews and audits may be conducted for 1 or more arrangement elements, depending on the interdependencies involved. These incremental organisation element-level reviews lead to an overall system-level review or inspect. After all incremental reviews are consummate, an overall summary review is conducted to provide an integrated system assay and capability assessment that could not exist conducted past a single incremental review. Each incremental review should complete a functional or physical area of blueprint. This completed expanse of design may need to exist reopened if other organisation elements bulldoze additional changes in this area. If the schedule is being preserved through parallel design and build decisions, whatever system deficiency that leads to reopening design may result in rework and possible material flake.

Roles and Responsibilities for Technical Reviews and Audits

For each technical review, a technical review chair is identified and is responsible for evaluating products and determining the review'due south entrance and exit criteria are met and that actions items are closed. The Service chooses the technical review chair, who could exist the PM, Systems Engineer or other discipline affair skillful selected according to the Service'south guidance. It is a best practise to choose someone independent of the program part to chair a program technical review. The Service'south technical review guidance may identify roles and responsibilities associated with technical reviews and audits. It as well may specify the types of blueprint artifacts required for various technical reviews. In the absence of additional guidance, each programme should develop and document its tailored design review plan in the program's SEP.

The following notional duties and responsibilities associated with the PM and Systems Engineer should exist considered in the absence of specific Service or lower level (e.g., System Control or Programme Executive Officer (PEO)) guidance:

Typical PM Responsibilities for Technical Reviews and Audits:

Co-developing with the Systems Engineer the technical objectives of the program that guide the technical reviews and audits
Co-developing with the Systems Engineer the earned value credit derived from the review
Approving, funding and staffing the planned technical reviews and audits; documenting this plan in the SEP and applicable contract documents
Ensuring the programme for each review includes participants with sufficient objectivity with respect to satisfying the pre-established review criteria
Ensuring the plan addresses the need for timely and sufficient data to satisfy the statutory and regulatory requirements of the program'due south chosen conquering pathway as reflected in the Adaptive Acquisition Framework Certificate Identification (AAFDID) Tool
Controlling the configuration of each baseline and convening configuration steering boards when user requirement changes are warranted. This can lead to an unscheduled gateway into the Functional Capabilities Lath (FCB) and JCIDS process, if applicable

Typical Systems Engineer Responsibilities for Technical Reviews and Audits:

Co-developing with the PM the technical objectives of the programme that guide the technical reviews and audits
Developing and documenting the technical review and inspect program in the SEP, carefully tailoring each event to satisfy program objectives and SEP outline guidance associated with the minimum technical reviews and audits
Ensuring the plan is effect-based with pre-established review criteria for each event, informed past the knowledge point objectives for the selected acquisition pathway
Identifying the resources required to support the plan; ensuring the activities leading upward to the official review and audit are integrated. Come across Figure 2.
Ensuring technical reviews and audits are incorporated into the Integrated Master Plan (IMP) and Integrated Master Schedule (IMS)
Analogous with Chief Development Tester to provide at each technical review: DT&E activities to-engagement, planned activities, assessments to-appointment and risk areas
Ensuring a condition of applicable design considerations are provided at each technical review
Establishing technical reviews and audits and their review criteria in the applicable contract documents (e.thousand., Argument of Work (SOW), IMP)
Monitoring and controlling execution of the established plans
Analogous with the appointed technical review chairperson on the technical review plans and supporting execution of the technical reviews
Assigning responsibilities for closure deportment and recommend to the chairperson and PM when a system technical review should exist considered consummate, see Figure two.

Figure 2: Notional Technical Review Procedure

Notional Technical Review Process

Fundamental Stakeholders for Technical Reviews and Audits

The PM and Systems Engineer should identify key stakeholders who take an interest or office in the review, which may include:

Technical review chairperson
Program Executive Function
Contracting Officer
Defense Contract Management Agency (DCMA) and Defense Contract Inspect Agency (DCAA)
Product Support Manager
Product Improvement Manager/Requirements Officer
End-user Community
Chief Developmental Tester
Interdependent Acquisition Programs
Business Financial Manager
Office of the OUSD DDR&Due east(Ac) Deputy Director for Engineering science (DD Engineering)
Service Technical Leadership such as main engineers
Discipline Affair Experts

Technical Review and Audit Criteria and Definitions

Entry and exit criteria should exist defined for each review and audit in the program's SEP. The technical review chair determines if these criteria have been achieved and all activeness items airtight before a technical review is considered complete. The Systems Engineer may refer to IEEE 15288.ii, Standard for Technical Reviews and Audits on Defence force Programs, as a resources for criteria. DoD military and civilian employees can access IEEE 15288.2 via the Acquisition Streamlining and Standardization Information System (ASSIST). To pattern for security, the plan protection planning and execution activities should be integrated into the systems engineering technical reviews and audits. See DAG CH 9–3.four.3. for system security engineering (SSE) criteria for each systems engineering technical review and audit. A brusk summary for typical DoD technical reviews and audits is provided below. DAG Chapter iii-3.3.1 through 3-three.iii.viii and 8-3.nine.one discuss these specific technical reviews in detail. The technical baseline established by completion of a certain technical review/inspect is annotated in that review/inspect clarification.

Culling Systems Review (ASR)

The Alternative Systems Review (ASR), typically held for the MCA pathway, is conducted to support a dialogue between the end user and conquering customs. It leads to a typhoon performance specification for the preferred materiel solution. The ASR typically occurs during the MCA Materiel Solution Analysis (MSA) phase, afterwards completion of the Analysis of Alternatives (AoA) and before Milestone A. It focuses technical efforts on requirements analysis.

Organisation Requirements Review (SRR)

The System Requirements Review (SRR) is a multi-disciplined technical review to ensure that the developer understands the organization requirements and is set up to proceed with the initial arrangement design. This review assesses whether the organization requirements are captured in the system performance specification (sometimes referred to equally the System Requirements Certificate (SRD)).

Organization Functional Review (SFR)

The Organization Functional Review (SFR) is held to evaluate whether the functional baseline satisfies the stop-user requirements and adequacy needs and whether functional requirements and verification methods support accomplishment of performance requirements. At completion of the SFR, the functional baseline is normally taken under configuration control past the authorities.

Preliminary Pattern Review (PDR)

The Preliminary Pattern Review (PDR) should provide sufficient confidence to proceed with detailed design. The PDR ensures the preliminary design and bones system architecture are complete, that at that place is technical confidence the capability need tin can be satisfied within cost and schedule goals and that risks have been identified and mitigation plans established. It likewise provides the acquisition community, stop user and other stakeholders with an opportunity to understand the trade studies conducted during the preliminary design, and thus ostend that design decisions are consistent with the user's performance and schedule needs and the validated JCIDS Capability Development Document (CDD), or CDD-like document for other pathways. The PDR establishes the allocated baseline.

Critical Design Review (CDR)

The Critical Design Review (CDR) confirms the system blueprint is stable and is expected to meet arrangement operation requirements, and confirms the system is on track to attain affordability and should-toll goals as evidenced by the detailed design documentation. The CDR establishes the initial product baseline.

System Verification Review (SVR) / Functional Configuration Audit (FCA)

The Organization Verification Review (SVR) is the technical assessment point at which the actual system performance is verified to run across the requirements in the system operation specification and is documented in the functional baseline. The Functional Configuration Audit (FCA) is the technical audit during which the actual performance of a system element is verified and documented to encounter the requirements in the system element performance specification in the allocated baseline. Further information on FCA tin can exist institute in MIL-HDBK-61, Configuration Direction Guidance. SVR and FCA are sometimes used synonymously when the FCA is at the system level.

Production Readiness Review (PRR)

The Production Readiness Review (PRR) for the organisation determines whether the system design is ready for product, and whether the developer has accomplished acceptable production planning for entering Low-Charge per unit Initial Production (LRIP) and Full-Charge per unit Production (FRP) for the MCA pathway. Production readiness increases over time with incremental assessments accomplished at diverse points in the life cycle of a program.

Concrete Configuration Inspect (PCA)

The Physical Configuration Audit (PCA) is a formal examination of the "equally-built" configuration of the system or a configuration item against its technical documentation to establish or verify its production baseline. The objective of the PCA is to resolve whatever discrepancies between the product-representative detail that has successfully passed Operational Test and Evaluation (OT&E) and the associated documentation currently under configuration control. A successful PCA provides the Milestone Decision Authority (MDA) with testify that the product design is stable, the adequacy meets stop-user needs and production risks are passably low. At the conclusion of the PCA, the final production baseline is established and all subsequent changes are processed by formal engineering change action. Farther data can be institute in MIL-HDBK-61, Configuration Management Guidance. SVR and FCA are sometimes used synonymously when the FCA is at the organization level.

Test Readiness Review (TRR)

The TRR is used to assess a contractor'due south readiness for testing configuration items, including hardware and software. It typically involves a review of earlier or lower-level test products and exam results from completed tests and a look forward to verify the test resources, examination cases, exam scenarios, test scripts, environs and test data have been prepared for the adjacent exam activeness. For programs using the MCA pathway, TRRs typically occur in the EMD and P&D phases. A TRR provides the formal blessing authority with a review showing that the system is ready to enter the test and that the funding and execution of a exam executes the examination and gathers the required information. TRRs appraise test objectives, test methods and procedures, test scope, condom, and whether test resources accept been properly identified and coordinated. TRRs are also intended to make up one's mind if any changes are required in planning, resource, training, equipment, or timing to successfully continue with the examination. If whatever of these items are not ready, senior leadership may decide to proceed with the test and accept the risk, or mitigate the risk in some manner.

Resources

DoDI 5000.88, Section 3.5, Plan Technical Reviews and Assessments

Defense force Acquisition Guidebook (DAG), iii-3.iii.1 – 3-iii.3.8 and 8-3.9.one

IEEE 15288.2, Standard for Technical Reviews and Audits on Defense force Programs

Grooming

CLE 003: Technical Reviews

ACQuipedia Articles

barrettelifere.blogspot.com

Source: https://www.dau.edu/tools/se-brainbook/Pages/Technical_Reviews_and_Audits.aspx