Application of proper specification language is an art in itself, for avoidance of ambiguity, redundancy, or impossible imposition (such as to a higher system which cannot be controlled from the lower level). A critical point:
If a "requirement" cannot be verified, it is merely a string of
text which may or may not contain useful information.
A fundamental System Engineering task is to determine what the set of valid
requirements actually is; to understand customer requirements and to make up
for the above listed deficiencies before proceeding with the system
development. The only "holes" remaining after developing the
essential requirements are those allowed as design decisions for implementation
of the selected architecture. These should be stated in lower level
requirements documentation, of course. Maintain traceability to the higher
level requirements through paragraph number and title equivalency or other
simple control means, so justification of each requirement is
simultaneously maintained.
Another approach to requirement verification is to determine necessity in the
first place. Have the author of each requirement document the reason for its
inclusion. Analysis of the rationale can reveal actual or near duplication,
design implementation, or erroneous assumptions. Reviews by novice people can
highlight ambiguities and other deficiencies. The ensuing deletion or rewriting
results in a more valid specification of the functional requirements and an
education for the authors. System Engineering interpretations of the
requirements become more defensible.
The four methods of requirements verification (in order of preference) are
Test, Demonstration, Inspection, and Analysis:
- Test indicates where instrumented (perhaps automated) measurement of a
device parameter indicates its level of requirement fulfillment.
- Demonstration indicates a test where the phenomenon being observed could
not occur without the requirement at issue being met.
- Inspection is direct physical observation of the requirement fulfillment
such as comparison with a drawing or evaluation of computer program source code.
- Analysis includes investigations and calculations to determine, where
other methods cannot be directly employed, the level of fulfillment of a
requirement by the item (to be) produced under the specification.
Callout of military, industrial, or company standards is to provide
minimum requirements for processes, procedures, and methods employed
in design and development of the system and subsystems specified. The System
Engineering responsibility here is ensuring that the procurement or other
specification requirements are all tailored, adequate, and traceable.
It is easy to grab the block of specifications and standards used on other
projects and assign them to new projects without tailoring to that minimum and
assigning order of precedence. Don't. Such overspecification is expensive,
requires conflict resolution, and limits design options to as few as no
feasible solutions that fulfill all requirements.
Synthesis of Conceptual Systems
Another major System Engineering process task is Synthesis of Conceptual
Systems. Within this recommended methodology, the summary list of the major
system requirements and desired features is set forth in accordance with a
structure definition showing the attribute tree. Having that list and the
preliminary system specifications of the system functional essence, design
engineering groups or appointed architects can postulate feasible system
physical concepts as well as generate the associated subsystem hardware/software
and support concepts. Design concept synthesis should be performed concurrently
with Functional Analysis, to assure responsive concepts with the minimal set of
interfaces. This top level design begins to provide hows for the
requirements specifications along with a preliminary list for project Technical
Performance Measurement.
The criteria for architectural concept designs are:
- Unified components, with duplication only for deliberate fault-tolerant
redundancy
- Structural and functional balance, supportive of development, integration,
and requirements verification
Engineering creativity in technology application as expressed in design studies
is balanced by the System Engineering process to generate an optimal concept
for fulfilling system requirements. Specialty designers provide technical
assistance to system engineers, whereas the system engineers review specialty
designs for requirements fulfillment. A high-tech or complex solution for a
simple manual process is not desirable:
Availability of a technology is insufficient justification for its use.
Research to identify candidate technical solutions to system design problems
is ongoing, so breakthroughs are possible even if unlikely. Usually, even when
subjected to Research & Development, the advances are small evolutionary
improvements because producibility of new technologies is unproven. System
Engineering at this stage is part of cost savings rather than an additional
cost.
Any attempt to schedule technology breakthroughs is risky, at best.
When system design must leave off-the-shelf adaptation and requires invention,
uncertainty becomes a major project driver and evaluation of the technical,
schedule, and cost risk is required. (See the Project Risk Assessment Tutorial.)
Understand, however, that all system designs are fallible in the sense of
not fully meeting all requirements (at least) to complete failure (in the worst
cases).
Evaluation and Decision
The next military System Engineering process is Evaluation and Decision. You
employ trade-off analysis here, with output in a Trade Study Report. For
selecting a best mix of system elements, the DSIDE™ process assists your
evaluation of candidate system approaches. After you establish the decision
bases, that process virtually causes the best final decision through the
ranking of candidates in accordance with their net utility scores. Don't forget
Risk Assessment for the project technical, cost and schedule issues, so you
can take risk abatement or mitigation actions.
A decision process takes place under Evaluation whenever you perform design
reviews at critical points during a system's concept development activity. You
design to match the anticipated system performance capability initial
requirements. You need to identify worst-case function input loading situations
to determine if an overload condition will exist. Functions which affect
Availability or reaction time are time-critical. Perhaps the throughput and
output capabilities must be increased. This is part of optimizing the system as
an alternative to establishing its best use.
You should conduct formal design reviews at critical program milestones to
check for technical adequacy of functional schematic block diagrams, interface
specifications, requirements specifications, and for complete traceability of
the system requirements to determine that the resulting baseline design
actually will fulfill its specifications if built. The customer will be
conducting a preliminary validation as well, to determine if the system concept
will support accomplishment of the proposed mission needs, confirming that the
right system was defined.
Documentation presenting results of each review is in accordance with the Data
Item Descriptions (DIDs) called out in the contract or supplier data
requirements list (CDRL or SDRL). Technical reviews should not be instruction
sessions, so only appropriate and knowledgeable personnel should attend --
after having adequate time to study the documentation that will be presented.
Description of System Elements
The concluding listed System Engineering process activity is Description of
System Elements. An approved set of system level specifications that was
produced to fulfill the tailored program's Supplier Data Requirements List
(SDRL) is the resulting primary output documentation from this activity.
A system specification is the design baseline descriptor for translating all
the generated paper into tangible hardware. It contains an explanation of what
the system is, what it has to do, maximums of weight and power required,
accuracy, capacity, and so forth. Any qualified manufacturer could develop the
system or interfacing subsystem from the applicable design specification, if
it sufficiently sets forth the overview as well as key details. Where combined
system attributes affect any other performance parameter, the specification
must address that system performance requirement in section 3 and its method of
verification in section 4. This is true for every requirement at every level of
system decomposition, and insistence upon it keeps requirements statements
verifiable.
The Interface Control Document (ICD) defines every interface from analog signal
level to digital control codes on a bus for electronic items but physical
interfaces are described as well. It can include facility descriptions for
exchange of models and define responsibility for verification of its content.
It does not duplicate the specification but describes actual implementation in
fulfillment thereof.
When different contractors make equipment that communicates with each other,
both must agree on the definition in all respects. Upon signoff and placement
under configuration control, an ICD has the status of a specification with
respect to electrical and mechanical definitions.
Configuration Management (CM)
Configuration Management, simply put, is being sure that you have what you
think you have for each Configuration Item. The baseline is recorded and all
changes are formally controlled with records maintained for audits. Each
equipment is identified for hardware configuration, such that the same complete
part number of an item is interchangeable with any other unit with the same
part number.
Descriptive documentation, specifications, and schematics also must match the
associated unit. Changes must run the gantlet of a Change Control Board (CCB)
and also may require government concurrence in some cases. The object of the
scrutiny is assurance that safety, litigation prevention, environmental
concerns, and quality are high in priority.
System Design
System design always is concerned with what the performance of the system will
be after the subsystems are integrated into a complete, functioning entity. How
well subsystems and components will work together with respect to the
integrated set of customer requirements overrides optimum performance of any
subsystem.
Subsystem designers always must be harnessed to the system goals through the
subsystem specifications whose requirements are allocated and tailored to
support that goal.
Without adequate system level requirements definitions, the individual
specialty designers will make their own assumptions about subystem design
requirements and may seriously err with neglect of non-performance parameters.
Some organizations have learned, through bitter experience with the extensive
delays for redesign and interminable verification test periods before customer
acceptance, that even moderately complex system designs require substantial
System Engineering from the start.
Support equipment is part of the overall system and therefore not less
important than the prime equipment to ultimate customer satisfaction. Their
design is to different requirements, however.
System level requirements should not levy specific solutions on subsystem
designers and software engineers. In the exercise of this truth, it is often
forgotten that the system requirements should cause and may define specific
testing to verify their fulfillment.
If support for system testing drives the design of prime and support equipment
beyond what would have been implemented with only subsystem performance
characteristics of interest, the job of verifying system readiness for delivery
becomes so much easier and quicker that the extra front end work is more than
paid for through rework costs not being necessary. System level accessible
built-in test verification features are particularly invaluable, but must be
designed in from the start or system level testing is virtually impossible.
The best long-term solution is to adopt the following rule:
A system performance or design requirement is most legitimate only if
it can be proven during system testing against the standards either provided or
implied by the customer needs for a product fulfilling that requirement.
System Integration and Verification
Integration is where system decomposition is reversed, where the hardware
components and firmware expressions of software begin to come together in a
support equipment integration environment, so the system perspective continues
to be necessary. Integration activity is a mix of bottom up and top down
functional/performance requirements and interface documentation verification.
Composition proceeds with assemblies becoming subassemblies which are further
integrated and verified into subsystems which are integrated and verified to
become the final system. The overall process of top-down design and bottom-up
integration now looks like a "V" in diagram form.
In larger organizations, the associated activities are divided into engineering
(development) testing for predicting performance, qualification testing
for verifying the design, system integration testing, and acceptance
testing for verifying minimum performance of every deliverable unit.
Acceptance testing of many military systems includes exercising the test unit
during cycles of rapid change between temperature extremes, after a soak time,
and vibration along most susceptible axis. This is called Environmental Stress
Screening (ESS). This "shake and bake" environmental testing for
manufacturing screening is designed to induce any incipient "infant
mortality" and thus preclude early failures of installed and operating
preproduction prototype units. Mean Time Between Failures experienced by the
customer then is likely to exceed the value predicted for the developed system
during design reliability calculations.
Development of requirements verification and maintenance testing requirements
is integral to the System Engineering process for system design. The same
skills required of primary system designers are necessary to design test equipment for
system level verification and validation.
System Engineering Management
As with other management assignments, your purpose is to make it possible for
others to accomplish the mission of your organization. Your selection should
have been based on ability to communicate the systems perspective across many
technical disciplines, not on your performance as a technical designer or as a
project engineer dealing with schedules and budgets. If you think of System
Engineering professionals only as better educated factory robots, you lose.
System engineers are just individuals whose personalities, strengths and
weaknesses you must learn in order to make the most appropriate assignments
for their most effective achievements in support of the organization goals
applied in a typically hostile environment. When you don't provide challenge
in meaningful assignments and appropriate support, in terms of resources,
tools, empowerment and direction, you are responsible for much of the blame
for their supposed ongoing mediocrity. The foregoing applies as much to the
Chief Engineer of the project System Engineering and system test teams as to
the functional director of a System Engineering group.
Objectives of the System Engineering function are:
- Provide project management with information necessary for effective
guidance of overall system development and fulfillment of cost and schedule
objectives through balanced use of resources.
- Assist development of plans and goals for logical project integration into
programs.
- Assist functional System Engineering management with planning for future
project team support needs, such as applying its practices to improvement of
its own processes.
- Keep current with technological advances in areas relating to the
disciplines encompassed by typical or anticipated projects. (Perhaps some of
the "success attributes" will be transferrable to your project tasks.)
- Execute the top-down decomposition and bottom-up integration process steps
as effectively as possible by maintaining the global system perspective and
configuration baselines.
The project system engineer also must balance cost and schedule against the
technical performance objectives, provided that critical system performance
minimums are not jeopardized by reported shortfalls in subsystem or component
reliability or in some other performance factor. Attempting to simultaneously
minimize all of project Technical, Schedule, and Cost Risk aspects is
impossible when attempt is made to agressively schedule technical
accomplishment at a defined low cost. When you fix two of the elements, the
third becomes relatively indeterminant.
This brings up the issue of describing the work required to define new system
development efforts. Part of project tailoring must be the schedule for the
data list, so the laudable urge for parallel work does not provide for
simultaneous delivery of serial process products. Work packages will consist
of cost and schedule estimates for development and release of requirements
specifications and the "level of effort" required to fulfill the
objective of an adequate system design.
Estimates based on the actual expenditures during previous "similar"
projects may not be directly applicable. For military projects, the Seller
Requirements Data List is different between the military services for similar
equipment and evolves over time. The degree of difficulty in attaining
technical objectives may differ substantially. (It is prudent to have a
technical performance criteria budget with significant slack for the same
reason as in project cost and schedule predictions. Uncertainties will at
least as often work against you as for you.)
Major, complex systems development projects never proceed cleanly from
one System Engineering step to the next. Discoveries of errors of omission or
commission drive changes to requirements which affect one or more somethings
at some stage of development, forcing return to an earlier stage for the
affected item(s). This is most likely when users and maintainers are left out
of the initial concept design development, when Concurrent Engineering
principles were not heeded. Some parts will become advanced and some retarded,
with respect to the initial plan, very early in the system development cycle.
Sometimes, unwarranted optimism led to setting of performance objectives that
happen to be physically impossible within the cost and space constraints
required by feasible technologies.
This is one reason Risk Management is considered a System Engineering
Management responsibility. An analysis may call for data that leads to
rethinking the original problem and radically changing requirements well
before the variances would show up under an earned value process, saving
enormous cost and schedule delays that would come from rework from redesign.
Complex subsystems will suffer from similar problems because they are systems
at the next lower level of perspective.
A checklist, with questions tailored to the system or subsystem at hand as
well as to the step being assessed, helps you keep your system perspective
during design reviews.
Further, the more the overall environment supports effective System Engineering,
the greater the likelihood that the resulting system design will fulfill its
requirements effectively. Any metrics used to measure the performance of the
process should motivate improvement either in the deliverable system or in the
System Engineering output products. No metric is preferable to a bad metric in
a supportive environment, because you then work under a minimum of high-
documentation management controls.
Applying even the desirable System Engineering approaches as contractually
required or mandatory procedures will make them as suspect as regular
management control procedures in the view of working engineers. That is,
"management by exception" controls applied to surface and highlight
deficiencies frequently are misused to assign only blame, so the more
experienced engineers will find ways to comply at a minimal level. When
working engineers become defensive and begin to hide all but the most glaring
problems, to avoid undesirable attentions of higher management, suboptimum
system designs are virtually guaranteed. So, what can you do?
Fortunately, the practices of Risk Management and Technical Performance
Measurement provide inherent controls on the System Engineering process while
being generally acceptable to working engineers. Just as individual components
are assembled into subsystems which are then integrated into complete systems,
Risk Assessment begins at the lowest design levels. Those technical judgments
are integrated into subsystem level assessments which in turn form the system
level Risk Assessments. (See the tutorial on Risk Assessment for additional
detail regarding constituent factors.)
Each higher level must increasingly employ a greater proportion of the
total systems perspective. When pressure is applied by people whose
success is measured by meeting delivery date milestones without corresponding
attention to quality or performance of the product, integrity of the risk
judgments by technical and systems people may be compromised. Complex systems
development takes so long that the people who must integrate subsystems and
patch the whole thing into a system often are not the original requirements
specifiers and designers.
Short term thinking is affordable only to those who don't expect to live with
the consequences of their decisions. When budgets are tight, management
pressure to take shortcuts can become even stronger. Pride in output product
stays with members of teams who maintain involvement through the complete
development and operational performance verification tasks.
The greatest likelihood, however, is that the early team members will
"meet" the initial project milestones with deficient documents and
designs, yet be praised as heros and be sent on to the next glamorous project.
The supposed second-stringers that remain will have to do the grotty job of
making it all work and to quietly accept the expressed or implied blame for
system shortcomings. By that time, budgets and schedules are overrun and upper
management is not interested in requirements deficiencies.
The only information of interest to management in that situation is how the
current team will get the system delivered soon at minimum cost, regardless of
performance problems. When a critical system is very late, some performance
requirements may be waived by the customer.
After all that, don't be surprised that some team member's judgment bias will
be towards choices likely to be made by more ambitious, politically conscious
people.
Unless System Engineering management helps the process, the rewards for keeping
the systems perspective are more likely to be penalties. It takes a very
self-confident systems engineer to risk walking the bed of hot coals just
described. Professional pride must exceed personal ambition by a large margin.
Junior design engineers should be assigned to the projects for learning from
the experienced people, even at the risk that they will develop the appropriate
System Engineering attitude and approach. They need to understand it, but not
for functioning within the everyone is a System Engineer paradigm. The
design organization rewards are not likely to be for maintaining the systems
perspective.
The tool to help management retain control while maintaining the systems
perspective is Technical Performance Measurement. Design deficiancies which
may not seem important when reviewed at the subassembly level may assume much
greater importance when reviewed for impact at the systems level where its
mission accomplishment criticality becomes apparent.
To alleviate the usual problem of delivering "garbage," such as key
specification documents that are mere skeletons filled with TBDs just to meet
early project milestones, Systems Engineering management must recognize that
technical uncertainties at the start of complex development projects coupled
with the inevitable changes prevent casting schedules in concrete.
You win some and you lose some, if schedules are realistic. You lose them all
if unbridled optimism substitutes for experienced judgment and no slack for
contingencies is allowed in the tasks.
Poor design remains long after memory of congratulations for meeting a
schedule milestone has faded.
The Japanese, in their "lean" production approach, assign a program
manager and engineering team for the life of the project. They obviously apply
Concurrent Engineering because the majority of changes occur during design,
with only a few at readiness for production. They practice System Engineering
by front loading the project to determine what to design as early as possible.
There is a lesson here.
It helps to begin analyses with the customer/user perspective and remember that
system engineers see to it that their system requirements are met without
being concerned about how it was done in the hardware. You don't care how it
was done, be it smoke, mirrors, or magic, so long as you can verify fulfillment
with scientifically repeatable tests. Adequate technology meets the customer's
objectives, not application of the theoretically best possible solution.
An incremental improvement to an existing approach may be the solution.
Remember, it is system rather than subsystem optimization for which you are
shooting. Does the output value justify application of the input to the
process?
Successful system engineers should be capable of becoming some of the more
successful project engineers/managers because only then do they really
know what is required to bring a system from initial concept to fielded
reality and can avoid a short-sighted and limited perspective that leads to
sub-optimum decisions. Hard-won system development experience contributes to
smoothest integration of knowledge from multiple disciplines and to better
modeling of systems. It is recognition that advancing any technological state
of the art brings forth unanticipated problems. It is recognition that
starting, stopping, abnormal conditions, and implementation factors should
receive nearly as much attention as design effort for normal operation. This
assists arguing the necessity for a larger contingency factor in development
schedules and budgets despite lack of clairvoyence necessary to provide hard
supporting numbers. It is realistic prudence rather than "planning for
failure."
The reverse job change is less successful, because assignment is to leadership
without the seasoning of hands-on experience. Simple observation of others
doing it is not enough to develop the necessary perspective for proper
tailoring of the System Engineering process to the project at hand. Seeing
it without having done it is like watching a travelogue on television. A
few concepts penetrate but only at the intellectual level. It is not
internalized "knowledge," so tasks seem easier than they are and
budgets are chopped accordingly.
Balancing budgets and schedules is easier than managing and coordinating a
design that balances competing technical objectives because physical reality
need not be a design constraint on wishful thinking in the former tasks.
If you have any questions on this tutorial subject, please contact the author as
listed below. A response will occur and entries in the FAQs section may result.
This tutorial is presented by:
Optants Documented Decision Support Co.
(ODDSCO)
297 Casitas Bulevar
Los Gatos, CA 95032-1119
(408) 379-6448 FAX: (Same, by arrangement)
Return to Home Page
E-mail:
Tutorial Author: jonesjh@optants.com
Other subjects: consult@optants.com