Where and When to Insert DE
Does It Matter Where My Program Starts?
ACQ 101 says the earlier the better, but don’t use that as an excuse to do nothing. Look for rational inject points such as mods which can be done digitally even if the program is in the dark ages of 2-D paper drawings. Focus on applying DE methods to an existing program to analyze failures or issues, for additional AoAs during the lifecycle, when the environment or operations change, including threat changes, and for change impact analysis (ECBs and CCBs).
Acquisition Category (ACAT) Considerations
DE methods are appropriate for all ACAT programs, and for non-ACAT efforts including prototyping. Models allow the effort to progress beyond a prototype accurately and efficiently.
Insertion Into an Acquisition Lifecycle
Ideally, using DE methods Pre-MDD and throughout the lifecycle to create a digital thread yields the highest value, but DE methods can be applied at any point in the lifecycle. Focus on modeling system aspects that are broken or need to be changed. Also consider building a digital twin of the final as built system, even if DE methods have not been used throughout the entire lifecycle. This will facilitate successful sustainment, including environment, operations, and requirements changes during operations, and follow-on systems.
How to Implement DE
Acquisition Program Goals
It is important initially to begin defining the output goals and analysis types required for the Acquisition Program early on, continually revisiting them as the Acquisition Program progresses. Using this approach allows the program to reuse available process models or create custom versions based on the specific needs of Program Management.
Implementation requires understanding of government and contactor technical data flows, current digital capabilities and gaps, workforce competency and infrastructure needs. Contractual considerations include IDE, Model Centric Technical Review, IP, System Model (SM) Delivery, SM Industry Standards Compliance, and Model Interfaces and Formats.
- Oversight & Review
- Logistics & Sustainment
Use standard and/or open source interfaces when possible. Look to use either/both before the considering the use of a proprietary interface.
- Input / Output
- Data Formats / Data Transfer
- Symbology / Syllabary (i.e., ᎣᏏᏲ)
Input & Output Requirements
- Use standardized and/or open data transfer types when possible, avoiding proprietary types, increasing future program / enterprise agility.
General Guidance on Executing Digital Engineering
The current AF acquisition construct does not enable enterprise management and the speed of relevancy:
- No management of cross-cutting capability investment
- Duplicative work to manage external stakeholders/agencies/partners
- Multiple levels of review inhibiting approval
Current AF Engineering Practices are:
- Disparate, nonstandard stovepiped data bases of mostly 2D & text data
- Empirical Analysis & Independent data analysis by function and mission – FM, PK, EN, AQ, SL, User
- Limited disparate MBE models, no digital thread, no enterprise view
- Limited disparate enterprise and mission modeling
- Mostly Paper/PDF and limited model-based CDRLs, reviews, TBLs, deliverables
- Shifting from single mission focus to an enterprise focus, will realize operational enhancements, acquisition agility, and resource efficiencies and create substantial improvements in speed, cost, and performance.
DE will facilitate this and enable:
- Single source of authoritative truth
- Enterprise Integrated Master Schedule (IMS)
- Internal and external enterprise digital thread with internally and externally integrated system model. including all functions and stakeholders (FM, PK, EN, AQ, SL, User, Contractor)
- Model-based Collaboration Environment to support Acquisition and Define/Maintain Technical Baseline and Technical Data Packages
- Continuous insight and improvement to enable agile adjustment as required to respond to changing threats, environment, operations and requirements. to provide enterprise capabilities to Warfighters
Effective Digital Engineering Implementation Requires:
- Assessment of cultural impacts and socialize enterprise focus and model-based acquisition benefits
- Assessment of workforce DE competency, and development plans to address gaps
- Development/adoption/implementation of DE standards and metrics
- Development, implementation, and Configuration Managed of integrated models
Cultural Challenges and Mitigations
People worry about their obsolescence and job security whenever a new technology is introduced.
Mitigation: Commit to providing the training needed to all affected.
“That’s not how WE do it!”
People become comfortable with the status quo; even if the current process is broken, it is considered better than a new one forced on them.
Mitigation: Engage stakeholders in adapting the new tools and processes to their work. Show them how the new way of working will make their jobs easier, better, and more interesting.
The learning curve
New tools need new processes; these take time to learn and master. The learning curve is not just personal, organizations are affected as well.
Mitigation: Have a technology insertion plan. Start with a pilot project to develop a core of expertise; have the pilot team help train and mentor. Adapt processes as experience and the team grows.
“This is unknown territory, what if we get lost?”
Any new approach brings potential for mistakes and some mistakes will be made.
Mitigation: Reach out to the larger MBSE community; learn from each other’s mistakes and share experiences.
Increases workload for critical staff
Limited program resources.
Mitigation: Cross training and mentoring.
Requires MBSE-ready skills
Both the government team and the contractors/offerors need working knowledge of modeling languages, object-oriented design, MBSE methods, tools, processes, best practices, etc.
Mitigation: Acquiring MBSE skills will need to be hired in or by training existing employees. Typically, training will be required at different levels and may need to be repeated as technology and tools or processes change.
Process Challenges and Mitigations
Information needs to move easily between tools, from process to process, and from person to person.
Mitigation: Using standard data-exchange formations eases transfers between tools, but the interfaces between processes and people need to be defined and managed. Model the process; not just the product.
Using standard data-exchange format eases transfers between tools; however, some will not directly support any of the chosen interfaces.
Mitigation: Customer interface tools and/or processes will be needed. Generalized automated translations are preferred, since they can be performed frequently and consistently
The new processes for MBSE will have roles that do not match prior processes. Some roles may disappear and others will be created. People’s jobs will change.
Mitigation: Define the roles and responsibilities in the new processes. Provide additional training where needed. Adapt these as the organization’s experience in MBSE grows.
“How do we know it’s working”
Established processes have known measures of progress and success with the experience to back them up. New processes need new ways to measure those properties.
Mitigation: Develop measures of process and the criteria for success. Use them, evaluate them, and adapt those that are not effective. Document this to build a knowledge base.
Current milestones, entry and exit criteria, and reviewed products will change with MBSE.
Mitigation: Start with the current milestones. Adapt criteria to fit new data formats as needed. As experience grows, change the milestones and data reviewed as makes sense for all stakeholders.
Continuous model management
Properly maintaining the model(s) as a single source of truth will require processes to ensure synchronization among models, configuration management, and version control.
Mitigation: Early on, establish and enforce processes to ensure the model is properly maintained, configuration controlled, and version controlled. Use the model to generate content for and drive system, hardware, and software documentation.
Lack of metrics and measurements
Currently there is not a well-established and commonly accepted set of measures for applying model-based approach on programs and systems.
Mitigation: Work is ongoing to extend current standard software measurement concepts and techniques, such as quality, size, effort, and cost for model-based approaches.
Change in acquisition deliverables
Traditionally, deliverables from acquisitions are document-centric. Acquisition employing MBSE will need to request models and tools as contract deliverables.
Mitigation: Ensure RFPs and contract proposals have mechanisms to ensure access and deliverables for models and tools.
Tool Cost Challenges and Mitigations
Initial investment: core tools
Infrastructure costs are higher to pay for new tools, tool maintenance, training and platforms. Each stakeholder may have their own suite of MBSE tools.
Mitigation: Establish the policy early for which tools each stakeholder will need and how the tools will be provided, installed, and purchased. Include these costs in the budget to better track expenditures and justify expenses.
Initial investment: support tools
MBSE datasets must permit exchange with other supporting engineering and analysis tools.
Mitigation: The approach for data exchange should be established early and implemented in the contracts.
Initial investment: hardware and software
Additional hardware and software maybe needed that are not part of the core MBSE tool purchases, such as network and workstation upgrades.
Mitigation: The approach for providing the infrastructure should be established early. Have stakeholders assess their infrastructure needs.
Initial investment: training
A core competency in the tools must be established. Formal training may need to be supplemented by hiring new talent and/or executing pilot projects.
Mitigation: Each stakeholder needs to create a transition plan for MBSE and determine expected costs.
All the tools and infrastructure have maintenance costs that may be greater than before. This includes training.
Mitigation: With time, the savings in development costs should offset the maintenance costs, and eventually the initial investment.
Maturity and Interoperability of Tool Challenges and Mitigations
Incompatible or proprietary data formats
Tools often use proprietary data formats or incompatible data formats that must be translated prior to use.
Mitigation: Include data format or data export format in tool selection and evaluation criteria. Test and pilot tool integration prior to use on acquisition.
Creating relationships among different views, or perspectives
Tools must support a diverse range of models and analyses for domains within a system or development process, where each domain applies specialized views and tools to support capturing and analyzing data within those views.
Mitigation: Adopt tools that can exchange data in tool environments that can support multifaceted views and models. Customize data exchange when needed
Large number of tools to select from (mature and unsupported)
Numerous tools are required for a complete end-to-end tool chain, and there are many tool choices. It can be difficult to predict which tools will continue to mature and which tools will become unsupported.
Mitigation: Evaluate program needs periodically. Watch the market for tools that can meet program needs.
Versioning and configuration control of tools
Software tools are periodically updated. Numerous stakeholders within the supply chain continuously access and manipulate data, potentially utilizing different versions of a tool.
Mitigation: Establish a process within the program that clearly identifies the rules of engagement for tool versioning and configuration management.
Semantics of data is lost during the exchange between tools.
Mitigation: Establish an ontology or common data model that support terminology and data relationships within your program.
Model Integration Challenges and Mitigations
Lack of model integration techniques
Integrating models is difficult because different languages, different levels of abstraction, different naming. conventions, etc.
Mitigation: Modeling standards and guides can help. Research and testing to develop integration techniques is needed.
Protecting contractor proprietary information and mission-sensitive information in models
Some information in models may be sensitive and should be handled appropriately in integrated models where multiple people and teams have access to the models.
Mitigation: Establish and enforce role-based access controls to models. Establish processes to ensure that sensitive data is not integrated into model.
Ensuring accountability of changes
On a shared model, it can be easy for users to change the model.
Mitigation: Need role-based access control to models. Establish and enforce processes to ensure model changes are controlled.
Lack of incremental model change techniques
Managing incremental model change (i.e., when a change made in one model impacts other models and only the impacted areas should be changed) is difficult.
Mitigation: Research is needed into incremental model change techniques. Strong processes and communication among teams can help.
Standards and Metrics
Industry 4.0 – the Industrial Internet of Things (IIoT) is driving value with increased speed and cost reduction – examples:
- Product design and development with 20% to 50% decreased time to market
- 85% better forecasts of supply chain capabilities
- 10% to 20% reduction in costs associated with ensuring manufacturing quality
Scaling-up pilot programs to capture the full value of innovative technology requires an action plan addressing 3 questions:
- Is the digital strategy driven by measurable business needs?
- Do you understand your organization’s digital capabilities and have a plan to fill any gaps?
- Do you [or your contactors] have the technology to scale-up, such as having the machines, automation, quality assurance, and distribution capabilities to take advantage of Industry 4.0 productivity improvements?
Faster production is squandered if finished parts stack up in the shipping department.
–Charles Cassidy and Kevin Goering, McKinsey and Co.
Before implementing DE
1. Determine metrics that will measure DE effectiveness and drive the improvements you want to achieve with DE – such as, but not limited to:
- Engineering Change Order (ECO) impact assessment time and effectiveness for all functions – EN, PK, FM, SL.
- Engineering Review Board (ECB) and Change control Board (CCB) approval review and approval time; note that ECB and CCB review and approvals may become continuous and real-time rather than event based
- Technical Evaluation time for source sections and contract modifications
- Source Selection time
- Contractor proposal response time
- Contract change order time
- Technology Insertion impact assessment and implementation time
- AoA time, and frequency and number of alternatives investigated – initial and throughout development/production
- Technical Review preparation, review and approval times – PDR/CDR, FCA/PCA; note that reviews and approvals may become continuous and real-time rather than event based
- Number of liens and waivers – time to resolve liens and number of waivers
- Technology Readiness Assessment (TRA) and Manufacturing Readiness assessment (MRA) or Independent Technical Risk Assessment (ITRA) time and effectiveness
- Design and Production time (contractor)
- Nonconformance volume and time to resolve
- Test time and effectiveness
- Acquisition documentation time and CM – SEP, TEMP, TRD, etc. – can be model-driven and continuously synchronized
- Architecture definition and synchronization – can be model-driven and continuously synchronized
2. Determine data flow throughout acquisition/development/production/operations/sustainment life cycle; include contractor data flow and government/contractor data touch points.
Determine data needed from the contractor and formats; specify open standards instead of tool or proprietary standards whenever feasible. Include DE deliverables in contract. The following figures define Typical Technical & Product Data Definitions & Relationships and Typical Prime Contractor Technical Data & Configuration Flow. Note that all technical data must be under CM, and that Product Data must also be in the Technical Baseline.