Risk Containers - A Help or Hindrance to Practitioners?

Abstract

Finding problems at the design stage reduces the cost to resolve them. Previous studies have indicated that error-proneness risks can be isolated into risk containers created from architectural designs, to help mitigate such risks early on. Here we describe an ongoing experiment to establish whether presenting designs as a collection of such containers helps practitioners manage the isolated risks. Participants must identify cyclic dependencies that could result in errorproneness and assess the impact of design changes. The emerging results suggest it takes participants longer to locate cyclic dependencies in collections of container diagrams than it does in a single diagram representing the whole design. Participants who reviewed collections of container diagrams tended to identify more cyclic dependencies correctly than those using a single diagram. Although, the results suggest that presenting a design as a collection of containers has no overall bearing on a participant's ability to correctly identify impacts of design changes, in cases where changes span multiple container diagrams no errors in change impact detection were observed. Errors were observed when assessing the same change using a single diagram representing the whole design.

Evaluating the effectiveness of risk containers to isolate change propagation

“…Bouwers et al (2010) explain that smaller architectural chunks can be easily understood. Their view is supported by our experimental results (Leigh et al, 2019): participants find it easier to locate cyclic dependencies in smaller risk containers than in the overall architecture diagram.…”

“…Thus, greater complexity in design structures will increase the chances of software developers misunderstanding and not fully appreciating all of the discrete program modifications needed to correctly propagate an intended change. We presented some evidence to support this view in Leigh et al (2019): when asked to identify the impact of design changes, participants made no errors if changes spanned multiple simpler container diagrams, but some participants assessing the same change on a single more complex diagram made errors. There have been further participants since that publication, but it remains the case that just one single error was observed in cases where eight participants were asked to impact changes spanning multiple container diagrams.…”