A Process for COTS Software Product EvaluationComella-Dorda, S.; Dean, John; Morris, E.; Oberndorf, P.Contact us / Contactez nous: nparc.cisti@nrc-cnrc.gc.ca. Abstract. The growing use of commercial products in large systems makes evaluation and selection of appropriate products an increasingly essential activity. However, many organizations struggle in their attempts to select an appropriate product for use in systems. As part of a cooperative effort, the Software Engineering Institute (SEI) and the National Research Council Canada (NRC) have defined a tailorable software product evaluation process that can support organizations in making carefully reasoned and sound product decisions. This paper describes that process.

The fundamentals of good decision-making are, first, a clear understanding of the decision itself and second t he availability of properly focused information to support the decision. Decision-making techniques help with both these problems. However, the techniques should be thought of as aids to decision-making and not the substitutes for it. Numerous decision-making techniques have been proposed as effective methods of ranking software products for selection for use as components in large-scale systems. Many of these techniques have been developed and successfully applied in other arenas and have been either used directly or adapted to be applied to COTS product evaluation and selection. This paper will show that many of these techniques are not valid when applied in this manner. We will describe an alternate requirements-driven technique that could be more effective.

Maintaining large software systems based on CommercialOff-The-Shelf (COTS) components is a major cost driver for these systems. Maintenance includes activities from component replacement to trouble-shooting and configuration management. The maintenance costs for COTS based software systems can be reduced by building systems according to specific design criteria. This paper identifies the major activities of a system maintainer, describes the properties that can be designed into a system to facilitate these activities, and outlines a checklist of items that can be verified during a design or code review, or during the evaluation of a COTS components in order to guarantee these properties are built into the system. The verification is illustrated using a photo imaging system that is currently under development.

No abstract

In early June of 2000 a COTS Workshop entitled "Continuing Collaborations for Successful COTS Development" was held in Limerick, Ireland in conjunction with ICSE 2000. The purpose of the workshop was to collect experience reports regarding the use of commercial off-the-shelf (COTS) software to build systems, identify best-practices for the use of COTS software, and to establish a research agenda for those researchers interested in COTS-based software systems. This one and a half day workshop was an extension of the work begun during the workshop entitled "Ensuring Successful COTS Development" held in conjunction with ICSE '99. Results from that workshop demonstrated that there were a number of common research areas, including acquisition, planning and management, architecture and implementation, and evaluation and testing, for which researchers saw the possibility of collaboration. These areas included specific topics such as estimating the effort required to implement COTS-based systems, classification of architectural styles, and certification of COTS products for reliability and safety. The group will reconvene at ICSE'01 (www.csr.uvic.ca/icse2001) to discuss further the results achieved.The ICSE 2000 Workshop had about 26 participants and was formatted as a combination of plenary sessions and small breakout groups that worked on specific issues related to COTS-based systems. The breakout groups investigated the impact of COTS software usage in the following areas: Economic and financial issues. Requirements definition Software engineering process. Integration, maintenance and system management. Business models. Each breakout group tried to identify the current state of the art in COTS software usage as well as open questions that could provide the basis for further research in the coming years. Each group was responsible for producing a written summary of their discussions which are included, without major editing, below. A more complete description of the workshop, as well as all the participants' position papers, can be found at:http://seg.iit.nrc.ca/projects/cots/icse2000wkshp/index.html