Maximilian Stark (mail@dakror.de), WS2019
Stand: 22.01.2019
Advanced Topics of Software Engineering
Context of Software Engineering
Characteristics of Software Systems in different domains
- Reactive systems
- Properties
- Interactivity
- Nonterminating processes
- Interrupt-driven
- State-dependent response
- Environment-oriented response
- Parallel processes
- Stringent real-time requirements
- Embedded Systems
- Properties
- Processing
- Communication
- Storage
- User Interface
- Quality attributes
- Dependability
- Efficiency
- Real-time usage
- Real-time systems (RTS)
- Deadline dependency
- Importance of predictability
- Processing of Events
- Information systems
- Collection and delivery of information and knowledge
- Data intensive platforms
- Business internal systems (ERP, etc...)
- Quality attributes
- Dependability
- Scalability
- Stability
- Cyber-physical systems
- European view: Socio-technical systems, integrating IS, ES & humans
- US view: New term for ES
From requirements to system design
Software Architecture
- Efficiency
- Risk minimization
- Communication means
- Knowledge management
Model of Software Architecture
- Mission
- Systems
- Environment
- Architecture
- Architectural Description
- Rationale
- Stakeholder
- Concerns
- Viewpoints
- 4+1 view model
- Logical view: end-user & designers
- Implementation view: programmers
- Deployment view: system engineers
- Process view: system integrators
- Use-case view
- Views
- Models
- Library Viewpoints
Models
- Goal: Abstraction
- Types
- Object model
- Functional model
- Dynamic model
- Data model
- Attributes
- Reduction
- Pragmatism
- Mapping
- Modeling process
- Understand
- Conceptualize
- Abstract
- Define
- Construct
- Evaluate
- Refine
- repeat
Modularity
- Unit of measurement for separability of components
- System decomposition
- Goals
- Information hiding
- Understandability increase
- Work distribution
- Complexity reduction (divide and conquer)
- Functional decomposition
- Modular decomposition
- Types of decomposition
- black-box model: functional (de-)composition
- white-box model: constructional (de-)composition
Component-based software engineering (CBSE)
- Component characteristics
- Interaction through public interfaces
- Independent executable entities
- Standardized
- Documented
- Composable
- Deployable
- Component model
- Interface
- Usage information
- Deployment and use
- Contras
- Questionable trustworthiness
- Certification
- Emergent property prediction
- Requirement trade-offs
- Microservices
- Independently deployed components
- Language-agnostic communication (REST)
Design by Contract (DbC)
- Dependable and robust OOP software
- Increased code reuse
- Contract between Client and Supplier
- Core principles
- Non-redudancy: No testing of own precondition
- Reasonable preconditions
- Failure model
- Disciplined exception handling
- Exception sinplicity
- Specifications of ubiquitous interfaces
- Preconditions
- Postconditions
- Invariants
- Side effects
- In practice:
assert
Defensive programming
- Strong verification of system inputs and state
- More work than DbC
- Many redundant checks
Dependency structure matrix (DSM)
- Column depends on row
- Partitioning
- Transformation into block triangular form
- Collapsing of mutually dependent components into composite entity
- Elimination of cycles
Guidelines for modular design
- Low coupling, high cohesion
- Single responsibility principle
- Never more than one reason to change a class
- Achieve cohesion at package level
- Separation of concerns
- Minimize responsibilities-per-component ratio
- Functional cohesion
- Open/closed principle
- Open for extensions
- Closed for modification
- Liskov substitution principle
- Subtypes must be fully substitutable for base type
- Interface-segregation principle
- No forced dependency on unneeded interfaces
- Change anticipation
- Limited repetition
- Types of duplication
- Imposed
- Inadvertent
- Impatient
- Inter-developer
Coupling and Cohesion
- Coupling
- Quantification of external dependencies
- Types from low to high
- Data: Pure data passing
- Stamp: Complete data structure transmission, sharing of data format
- Control: Absolute dependency on outputs of other component
- External: I/O
- Common: shared data between components
- Content: duplicate content with need to cross-update across components
- Cohesion
- Quantification of internal interdependencies
- Types from low to high
- Coincidental
- Logical
- Temporal
- Procedural
- Communicational
- Sequential
- Functional
- Hard to quantify
- Analysis of description in natural language
- LCOM (Lack of Cohesion in Methods)
- Number of method pairs that do not use the same attributes
- Good values indicate good design, bad values are meaningless
Formal definition of system structure
- Systems as tuple $S:= (C,I,CON)$
- Components $C$
- Interfaces $I$
- Environment $env \in C$
- Connections $CON \subseteq I \ times I$
- Functions
- $\mathrm{parent}(c): C \mapsto C$: Parent of component or $env$ for root
- $\mathrm{assigned}(i): I \mapsto C$: Component for given interface
- $\mathrm{connected}(c): CON \mapsto (I \times I)$: Connection between interfaces
Quality
- Internal
- External
- Attack surface
- Performance
- Intrinsic
- Design-time
- Portability
- Maintainability
- Run-time
- Usability
- Reliability
- Efficiency
- Functionality
Antipatterns
- Causes: The 7 deadly sins of software
- Haste
- Apathy
- Narrow-mindedness
- Sloth
- Avarice
- Ignorance
- Pride (Hubris)
- Blob: Single magic universal class doing everything
- Functional decomposition: Lack of OOP
- Auto-generated stovepipe: Ad-hoc repairs of ever-growing legacy support
- Golden Hammer: One tool for everything, even if bad choice
- Design by Committee
Reuse
- Challenges / Issues
- Technical issues
- Organizational issues
- Conflict between flexibility and stability
- Principles
- Modularity
- Loose coupling & high cohesion
- Information hiding
- Separation of concerns
- Types
- Opportunistic / Ad-hoc
- Not scalable
- Clone-and-own paradigm
- Lack of structure
- Clone detection
- Planned / Structured
- Software product lines
- Software construction kits with stable core architecture
- Managed variability modeling
- (Software) Frameworks
- Dedicated library projects for 3rd site use
- Variability
- External: Different artifacts for user / customer
- Internal: Different internal components
- Managed Variability
- Definition of variability at domain engineering
- Exploitation of variability in application engineering
- Moment of variability resolution: "Binding time"
- Orthogonal variability modeling
- Variability of a software product line in regards to other software development models
Software Product Line Engineering (PLE)
- Proactive development for reuse
- "Family Engineering" - Design of products with mass variability options
- Specializations
- Platform
- Environment
- Functionality
- Process
- Process
- Requirements on $n$ individual systems
- Family engineering: Information integration
- Product line infrastructure
- Application engineering: Information specialization
- $m$ resulting individual systems
- Scoping
- Aspects
- Products
- Domains
- Features
- Assets
- Process
- Identify
- Prioritize
- Optimize
Reference Architecture
- Abstract software architecture for specific application area
- Definition of software elements and their interactions
- =Blueprints
- Types
- Functional: Structuring required functionality
- Logical: Layer and component definitions
- Technical: Tech-stack definitions
- Examples
- Database-centric: Mainframes
- Message-oriented: RMI
- Object-oriented: .NET
- Service-oriented: WSDL
Testability
- Conflict with low time-to-market
- General testability scenario
- Source
- Stimulus
- Artifact / Environment
- Response
- Response measure
- Definitions in the literature
- Facilitation of testing (efficiency)
- Facilitation of revealing faults (effectiveness)
- RIPR Model: Requirements for failure reveal
- Reachability: Fault can be reached
- Infection: Fault corrupts program state
- Propagation: Faulty state gets carried to corrupt final state
- Revealability: Final state contains faults
- Other focus areas (requirements testability)
- Measurement of effort (unproven)
- Properties of programs affecting testability in order of significance
- Observability
- Controllability
- Complexity / Simplicity
- Isolateability (Coupling & Cohesion)
- Assessment methods
- Theoretical
- Measurement of information loss
- Implicit: Same result when given different inputs
- Explicit: Loss of some calculated values by design
- Domain/range ratio (DRR)
- Porr testability if $|input domain| >> |output domain|$
- Black-box testability
- Relating to single function domain
- Correlation with implicit loss
- Mutation-based
- Injection of defect into program
- Execution of random testing
- Measurement of likelihood of output change due to mutation
- High likelihood = High testability
Safety
- Freedom of risk
- System failure types
- Cascading failure
- Common cause failure (CCF)
- Risk reduction
- Avoidance of systematic faults during design
- Tolerance of systematic faults at runtime
- Tolerance of random faults at runtime
- Functional safety
- hazards & risks purely through operation
- Lifecycle approach
- Integration of safety into design
- Systematic hazard idenfitication & analysis
- Protection layers
- ISO 26202 Safety lifecycle model
- Concept Phase
- Item definition → External measures
- Initiation of safety lifecycle
- Hazard & risk assessment → Controllability
- Functional safety concept → Allocation to other technologies
- Development Phase
- Product development & Safety validation
- Production Planning
- Operation Planning
- Functional safety assessment
- Release for production
- Production / Operation
- Production
- Operation
- Safety analysis
- Qualitative
- Identification of failures
- Knowledge about relevant fault types required
- Example: HAZOP
- Quantitative: Fault frequency predictions
- Inductive Analysis Methods
- Bottom-up (from causes to effects)
- Failure Mode and Effects Analysis (FMEA)
- Deductive analysis Methods: Top-down
- Fault Tree Analysis (FTA)
Failure Mode and Effects Analysis (FMEA)
- Uncommon in software engineering
- System-level analysis
- Establishing (software) test goals
- Table columns per fault
- Failure mode
- Failure effects
- Causes
- Detection / Prevention measures
- Severity (SEV) (1-10=hazardous without warning)
- Occurrence (OCC) (1-10=persistent)
- Detection (DET) (1-10=unlikely)
- Risk priority number (RPN) = SEV OCC DET
Fault Tree Analysis (FTA)
- Goal: Identification of system failure conditions
- Graphical representation of causal relationships
- Done when design / architecture is available
- Requirement of system understanding, after FMEA
- Reversed attack tree
- Procedure
- System plan as input
- System definition under scrutiny (boundaries)
- Definition of undesired top event
- Recursive identification of series of events leading up to top level event
- Cut set identification: Set of events leading to failure
- Events
- Primary events
- Basic events: No precursor, probabilistic
- Undeveloped events: No major effect on the system
- External events: Expected events, not faults
- Intermediate events: AND/OR gate link
- Expanded events: Reference to whole other subtree
Model-Based Design
- Advantages
- Separation of concerns
- Model in the Loop (Simulation)
- Understandability
- Reusability
- Code-Generation
- Analysis: Verification and Validation
- Components
- Operational context
- Interfaces
- Behavior
- Inner structure
Model consistency
- Consistent-by-construction: One metamodel of entire system
- Consistent-by-check: Agreement of consistency between different view-types across tools
ISO 26262 Component model
- Components
- Item: System(s) under consideration
- System implements function(s)
- Systems are (composed of) elements
- Non-atomic elements are components
- Atomic elements are software parts / hardware units
- Properties
- Modular construction
- Generic levels of abstraction
- Inter-level allocation
- Distributed development
- Reusable elements
Modeling Viewpoints
- Requirements
- Functional
- Logical
- Reactive behavior
- Computational behavior
- Platform independent
- Technical
- Hardware
- Software
- Dynamic: Runtime
- Static: Design-time
- Deployment logic model: Mapping logical to technical
Security
- Core properties: CIA
- Confidentiality
- Integrity
- Availability
- Other desirable properties
- Non-repudiation
- Authenticity
- Auditability
- Accountability
- Privacy
- Anonymity
- Design principles
- Least privilege
- Complete mediation
- Secure, fail-safe defaults, Permission refusal by default
- Compartmentalization
- Minimum exposure
- Trade-offs
- Performance
- Usability
- Time to market
- Cost objectives
Distributed System Architectures
Database-centric
Operational database
- Online transaction processing (OLTP)
Data warehouse (DW)
- Subject oriented
- Better data analysis tools
- data homogenization from varying internal sources
- Core business intelligence data storage
- Online analytical processing (OLAP)
- Extract, Transform, Load (ETL)
- Procedure for storing multidimensional data in DW
- Problems
- Different sources
- Different formats
- Missing data / gaps
- Query performance of DW
- Architectures
- Central: Single DW
- Federated
- Data marts: distributed subdata nodes
- Virtual DW
- Tiered
- Hierarchical storage in aggregated data cubes
- Materialized central DW
Big data
- Central Vs
- volume
- velocity
- variety
- veracity
- value
- Sharding strategies (subdata assignments and routing)
- Key Value stores
- Redis
- Various data types
- In-RAM database
- Bigtable
- Mapping of row, column key and timestamp to arbitrary byte array
- Lexical sorting of rows
- Column hierarchy notation
- Document-oriented
- Graph-database
- Core consistency rule: no broken links
- Joins as first class citizen
- Neo4J
Message-oriented
- Message intents
- Industry Standard: Java Message service
- pub sub with in/out queues
- Message queues
- point-to-point pattern: 1:1 conversation per message
- pub/sub
- datatype-based
- shared queues: local distribution
- Message broker challenges
- Scalability
- Security
- Monitoring & debugging
- Maintenance
- Reactive systems
- Properties
- Responsive
- Resilient
- Elastic
- Message-driven
- Actor model
- Purely reactive modeling encompassing
- processing
- storage
- communication
- Possible actions upon message receive
- Creation of new actors
- Sending of messages to known actors
- Designation of handling procedures for upcoming message
Object-oriented
Object broker
- Object oriented RPC
- CORBA: Common Object Request Broker Architecture
- language agnostic standardized specification
- gRPC (+protobuf)
Component-based
Java EE
- Multi tier architecture
- Client tier
- Web tier
- Business tier
- Enterprise information system tier (ERP)
- Application server
- System resource mediation
- Life-cycle and deployment management
- Communication and security management
- Transaction and persistence management
- Functionality encapsulated into containers
- Application client container
- EJB container
- Web container
- Enterprise Java Beans (EJB)
- RMI for interaction
- Types
- Session (transient)
- Stateful
- Stateless
- Singleton
- Message driven
- Entities
- Contents
- Business logic implementation
- Business interfaces
- Helper classes
- Client access
- no-interface view: Exposure of public methods to (local) client
- business interface
- Client obtainment
- Dependency injection
- Java Naming and Directory Interface (JNDI) lookup
AUTOSAR
- Automotive Open System Architecture
- Standardized automotive reference architecture
- Focus on non-functional requirements
- Modularity
- Scalability
- Portability
- Reusability
- Architecture workflow
- Modular software-components (SW-C)
- Virtual functional bus (VFB): communication between SW-C
- System constraint and ECU descriptions
- Embedding on ECUs (engine control units)
- ECU stack
- Runtime environment (RTE)
- Module layers
- Application layer
- AUTOSAR RTE layer
- Basic Software layer
- Micro-controller abstraction layer
- Connection paradigms
- Client/Server
- Sender/Receiver a la pub/sub
- Standardized module interfaces
Android
- Architecture
- Layers
- Application
- Framework
- Library
- Runtime
- Kernel
- Information hiding
- high extensibility
- Sandboxing of Apps into individual VMs: Zygote
OSGi
- Open services gateway initiative
- Specificationas for dynamic component systems
- Components
- Bundles: actual developed components
- Services: dynamic connection layer for bundles
- Life-Cycle: bundle management
- Modules: import/export declarations for bundles
- Security
- Execution environments
- Driving forces
- Reduced complexity (information hiding)
- Reuse
- Real World (dynamic architecture)
- Easy development
- Dynamic updates
- Adaptive
- Transparency
- Versioning
- Simplicity
- Small size
- Speed
- Laziness
- Security
- Humbleness
- Non-intrusiveness
- Runs everywhere
- Widely used
Service-oriented
SOA
- Process orchestration of enterprise-level business services
- Solution of specific problems in concrete scenarios
- Multiple languages
- Multi-governance
- Strategic level: management of company
- Operational level: across departments
- Technical level: Technical integrity
- Interoperability via standard
- Independence from vendor or technology
- Loose coupling
- Drawbacks
- Lacking ecosystem
- Scalability
- Unfunctional browsing
- High complexity
- Reduced agility
- Principal architecture
- Application frontend
- Service
- Contract
- Interface(s)
- Implementation
- Service repository
- Service bus
- Roles
- Service consumer
- Service provider
- Service registry
- Principles
- Loose coupling
- Modularization
- Service contract
- Discoverability
- Abstraction and autonomy
- Reusability
- Composability
- Statelessness
- Web services
SOAP
- XML based
- Technology agnostic
- Verbose (arguably bloated)
- Structure
- Usage variations
- Interaction styles
- Document-style: exchange of hierarchical document data
- RPC-style: parameterized call and response
- Encoding styles
- Binding: definition of correct message-wrapping in a given transport protocol
- Web service description language (WSDL)
- Specification of service interfaces
- Difference from IDLs: Specification of required service access mechanism
- Structure
- Abstract definitions
- Types
- Messages
- Operations: simple exchange of messages
- Port types: logical collection of related operations
- Concrete definitions
- Bindings
- Services & ports
Representational state transfer (REST)
- Stateless
- Cache-support
- Focused NFRs
- Performance of component interaction
- Ecosystem scalability
- Interface simplicity
- Visibility
- Language-agnosticism
- Null-style
- Empty set of constraints
- Starting point for REST-description
- Description of system without component boundaries
- Client-Server
- Separation of UI from data storage
- Increased portability
- Improved scalability
- Independent component evolution
- Uniform interfaces
- Uniform identification scheme
- URI
- URL (location)
- URN (name)
- Structure
- Scheme
- Authority
- Path
- Query
- Fragment
- Uniform representation of exchanged information
- Decoupling of implementations from the services they provide
- Trade-off
- Hypermedia as the engine of application state (HATEOAS)
- MIME-Types: xml, json, etc
- Code-on-demand: Downloadable extension scripts
Microservices
- Conways law: Design of systems inevitably resembles team structure
- Products instead of projects: Full lifecycle responsibility of team
- Characteristics
- Decentralized governance
- Decentralized data management
- Problem: significant temporal coupling
- Varying world views among services
- Infrastructure automation
- Design for failure
- Evolutionary design
- Patterns
- Data management: database-per service / shared db
- Communication: service discovery / API gateway
- Deployment: service-per-container
- Operation: circuit breaker, distributed tracing
- UI: client side UI composition
From source code to deployment
- Version control
- SVN
- Centralized
- Storage of file based diffs
- Git
- Distributed
- Snapshots of miniature file system
- Continuous integration (CI)
- Risk reduction
- Quality control
- Health reports
- Availability of stable release
- Continuous deployment (CD)
- Continuous delivery
- Software engineering approach to ensure release-ready software at any point in time
- Short development cycles
- Benefits
- Time to market
- Customer satisfaction
- Productivity & Efficiency
- Building the right product
- Immediate deployment to production
- Deployment pipelines
- Individual business strategies
- Smoke-testing
- Deploy to production-mirror
- Deployment automation patterns
- Repository: all relevant data under version control
- Scripted deployment: full deployment process in scripts
- Single command deployment: single invocation to deploy
- Unified deployment: single script with support for multiple environments and platforms
- Template verifier: Single template file as base for all target environment properties
- Externalized configuration: extraction of application variables into build-time properties
- Tokenize configuration: Replacement of certain values through externalized configuration
- Remote deployment: Usage of centralized machine or cluster for deployment
- Blue green deployment
- Usage of production mirror
- Final stages of testing after deployment on mirror
- Flipping of routing for mirror and actual production
- Virtualization
- Cloud
- Goals
- Availability & Reliability
- Reduced investments and proportional costs
- Stability
- Actors
- Consumer
- Provider
- Auditor: independent cloud service evaluation party
- Broker: negotiator between consumer and provider
- Carrier: intermediary for transport and connectivity of consumer and provider
- Characteristics
- On-demand self-service
- Broad network access
- Resource pooling
- Rapid elasticity
- Measured service
- Forms
- Deployment models
- Public cloud
- Private cloud
- Outsourced private cloud
- Community cloud
- Hybrid
- Architectures
- Workload distribution
- Resource pooling
- Dynamic scalability
- Horizontally
- Vertically
- Relocation
- Service load balancing
- cloud usage monitor
- Resource cluster
- Resource replication