Software evolution explained

Software evolution is the continual development of a piece of software after its initial release to address changing stakeholder and/or market requirements. Software evolution is important because organizations invest large amounts of money in their software and are completely dependent on this software. Software evolution helps software adapt to changing businesses requirements, fix defects, and integrate with other changing systems in a software system environment.

General introduction

Fred Brooks, in his key book The Mythical Man-Month,^[1] states that over 90% of the costs of a typical system arise in the maintenance phase, and that any successful piece of software will inevitably be maintained.

In fact, Agile methods stem from maintenance-like activities in and around web based technologies, where the bulk of the capability comes from frameworks and standards.

Software maintenance addresses bug fixes and minor enhancements, while software evolution focuses on adaptation and migration.

Software technologies will continue to develop. These changes will require new laws and theories to be created and justified. Some models as well would require additional aspects in developing future programs. Innovations and improvements do increase unexpected form of software development. The maintenance issues also would probably change as to adapt to the evolution of the future software. Software processes are themselves evolving, after going through learning and refinements, it is always improve their efficiency and effectiveness.^[2]

Basic concepts

The need for software evolution comes from the fact that no one is able to predict how user requirements will evolve a priori .^[3] In other words, the existing systems are never complete and continue to evolve. As they evolve, the complexity of the systems will grow unless there is a better solution available to solve these issues. The main objectives of software evolution are ensuring functional relevance, reliability and flexibility of the system. Software evolution can be fully manual (based on changes by software engineers), partially automated (e.g. using refactoring tools) or fully automated.

Software evolution has been greatly impacted by the Internet:

• the rapid growth of World Wide Web and Internet Resources make it easier for users and engineers to find related information.
• open source development where anybody could download the source codes and hence modify it has enabled fast and parallel evolution (through forks).

Types of software maintenance

E.B. Swanson initially identified the three categories of maintenance: corrective, adaptive, and perfective. Four categories of software were then catalogued by Lientz and Swanson (1980).^[4] These have since been updated and normalized internationally in the ISO/IEC 14764:2006:^[5]

• Corrective maintenance: Reactive modification of a software product performed after delivery to correct discovered problems;
• Adaptive maintenance: Modification of a software product performed after delivery to keep a software product usable in a changed or changing environment;
• Perfective maintenance: Modification of a software product after delivery to improve performance or maintainability;
• Preventive maintenance: Modification of a software product after delivery to detect and correct latent faults in the software product before they become effective faults.

All of the preceding take place when there is a known requirement for change.

Although these categories were supplemented by many authors like Warren et al. (1999)^[6] and Chapin (2001),^[7] the ISO/IEC 14764:2006 international standard has kept the basic four categories.

More recently the description of software maintenance and evolution has been done using ontologies (Kitchenham et al. (1999),^[8] Deridder (2002),^[9] Vizcaíno (2003),^[10] Dias (2003),^[11] and Ruiz (2004)^[12]), which enrich the description of the many evolution activities.

Stage model

Current trends and practices are projected forward using a new model of software evolution called the staged model.^[13] Staged model was introduced to replace conventional analysis which is less suitable for modern software development is rapid changing due to its difficulties of hard to contribute in software evolution. There are five distinct stages contribute in simple staged model (Initial development, Evolution, Servicing, Phase-out, and Close-down).

• According to K.H.Bennett and V.T Rajlich,^[13] the key contribution is to separate the 'maintenance' phase into an evolution stage followed by a servicing and phase out stages. The first version of software system which is lacking some features will be developed during initial development or also known as alpha stage.^[13] However, the architecture has already been possessed during this stage will bring for any future changes or amendments. Most references in this stage will base on scenarios or case study. Knowledge has defined as another important outcome of initial development. Such knowledge including the knowledge of application domain, user requirements, business rules, policies, solutions, algorithm, etc. Knowledge also seems as the important factor for the subsequent phase of evolution.
• Once the previous stage completed successfully (and must be completed successfully before entering next stage), the next stage would be evolution. Users tend to change their requirements as well as they prefer to see some improvements or changes. Due to this factor, the software industry is facing the challenges of rapid changes environment. Hence the goal of evolution is to adapt the application to the ever-changing user requirements and operating environment.^[13] During the previous stage, the first version application created might contain a lot of faults, and those faults will be fixed during evolution stage based on more specified and accurate requirements due to the case study or scenarios.
• The software will continuously evolve until it is no longer evolvable and then enter stage of servicing (also known as software maturity). During this stage, only minor changes will be done.
• Next stage which is phase-out, there is no more servicing available for that particular software. However, the software still in production.
• Lastly, close-down. The software use is disconnected or discontinued^[13] and the users are directed towards a replacement.^[13]

Lehman's Laws of Software Evolution

See main article: Lehman's laws of software evolution. Prof. Meir M. Lehman, who worked at Imperial College London from 1972 to 2002, and his colleagues have identified a set of behaviours in the evolution of proprietary software. These behaviours (or observations) are known as Lehman's Laws. He refers to E-type systems as ones that arewritten to perform some real-world activity. The behavior of such systems is strongly linked to the environment in which it runs, and such a system needs to adapt to varying requirements and circumstances in that environment. The eight laws are:

1. (1974) "Continuing Change" — an E-type system must be continually adapted or it becomes progressively less satisfactory^[14]
2. (1974) "Increasing Complexity" — as an E-type system evolves, its complexity increases unless work is done to maintain or reduce it
3. (1980) "Self Regulation" — E-type system evolution processes are self-regulating with the distribution of product and process measures close to normal
4. (1978) "Conservation of Organisational Stability (invariant work rate)" - the average effective global activity rate in an evolving E-type system is invariant over the product's lifetime
5. (1978) "Conservation of Familiarity" — as an E-type system evolves, all associated with it, developers, sales personnel and users, for example, must maintain mastery of its content and behaviour to achieve satisfactory evolution. Excessive growth diminishes that mastery. Hence the average incremental growth remains invariant as the system evolves.
6. (1991) "Continuing Growth" — the functional content of an E-type system must be continually increased to maintain user satisfaction over its lifetime
7. (1996) "Declining Quality" — the quality of an E-type system will appear to be declining unless it is rigorously maintained and adapted to operational environment changes
8. (1996) "Feedback System" (first stated 1974, formalised as law 1996) — E-type evolution processes constitute multi-level, multi-loop, multi-agent feedback systems and must be treated as such to achieve significant improvement over any reasonable base^[15]

It is worth mentioning that the applicability of all of these laws for all types of software systems has been studied by several researchers. For example, see a presentation by Nanjangud C Narendra^[16] where he describes a case study of an enterprise Agile project in the light of Lehman’s laws of software evolution. Some empirical observations coming from the study of open source software development appear to challenge some of the laws .

The laws predict that the need for functional change in a software system is inevitable, and not a consequence of incomplete or incorrect analysis of requirements or bad programming. They state that there are limits to what a software development team can achieve in terms of safely implementing changes and new functionality.

Maturity Models specific to software evolution have been developed to improve processes, and help to ensure continuous rejuvenation of the software as it evolves iteratively.

The "global process" that is made by the many stakeholders (e.g. developers, users, their managers) has many feedback loops. The evolution speed is a function of the feedback loop structure and other characteristics of the global system. Process simulation techniques, such as system dynamics can be useful in understanding and managing such global process.

Software evolution is not likely to be Darwinian, Lamarckian or Baldwinian, but an important phenomenon on its own. Given the increasing dependence on software at all levels of society and economy, the successful evolution of software is becoming increasingly critical. This is an important topic of research that hasn't received much attention.

The evolution of software, because of its rapid path in comparison to other man-made entities, was seen by Lehman as the "fruit fly" of the study of the evolution of artificial systems.

See also

• Software entropy
• Meir M. Lehman
• Darwinian evolution
• Lamarckian evolution
• Baldwinian evolution

Available tools

• LibVCS4j A Java library that allows existing tools to analyse the evolution of software systems by providing a common API for different version control systems and issue trackers.

Further reading

• Andrea Capiluppi, Jesus M.Gonzalez Barahona, Israel Herraiz, Gregorio Robles, Adapting the "Staged Model for Software Evolution" to FLOSS
• Mark C. Paulk, A History of the Capability Maturity Model Software

Notes and References

1. [Fred Brooks]
2. aeddy; ref: Understanding Open Source Software Evolution Walt Scacchi Institute for Software Research
3. News: K. H. . Bennett . V. T. . Rajlich . R. Mohamad . Mazrul . Legacy System: Coping with success . . 1995 . 19–23.
4. Lientz, B.P. and Swanson, E.B., Software Maintenance Management, A Study Of The Maintenance Of Computer Application Software In 487 Data Processing Organizations. Addison-Wesley, Reading MA, 1980.
5. [ISO/IEC]
6. The evolution of websites. Paul Warren. Cornelia Boldyreff. Malcolm Munro. Proceedings of the Seventh International Workshop on Program Comprehension. 178–185. 1999. IEEE.
7. Types of software evolution and software maintenance. Ned Chapin. Joanne E Hale. Khaled Md Khan. Juan F Ramil. Wui-Gee Tan. Journal of Software Maintenance and Evolution: Research and Practice. 13. 1. 3–30. 2001. 10.1002/smr.220.
8. Towards an ontology of software maintenance. Barbara Kitchenham . Barbara Kitchenham. Guilherme Travassos. Anneliese von Mayrhauser. Frank Niessink. Norman Schneidewind. Janice Singer. Shingo Takada. Risto Vehvilainen. Hongji Yang. Hongji Yang. Journal of Software Maintenance: Research and Practice. 11. 6. 365–389. 1999. 10.1002/(SICI)1096-908X(199911/12)11:6<365::AID-SMR200>3.0.CO;2-W. 10945/55140. free. free.
9. A concept-oriented approach to support software maintenance and reuse activities. Dirk Deridder. Proceedings of the 5th Joint Conference on Knowledge Based Software Engineering. 2002.
10. A multi-agent system for knowledge management in software maintenance. Aurora Vizcaíno. Jesús Favela. Mario Piattini. Knowledge-Based Intelligent Information and Engineering Systems. 415–421. 2003. Springer.
11. Organizing the knowledge used in software maintenance. Márcio Dias. Nicolas Anquetil. Káthia de Oliveira. Journal of Universal Computer Science. 9. 7. 641–658. 2003.
12. An ontology for the management of software maintenance projects. Francisco Ruiz. Aurora Vizcaíno. Mario Piattini. Félix García. International Journal of Software Engineering and Knowledge Engineering. 14. 3. 323–349. 2004. 10.1142/S0218194004001646. 15592498.
13. A Staged Model for the Software Life Cycle. Computer. Bennett. Keith. 2000-07-01. 66–71. Rajlich. Václav. IEEE Computer Society. 10.1109/2.869374. 2000. 33. 7. 7547412. 2020-05-23.
14. 10.1016/0164-1212(79)90022-0. Lehman. M. M.. On Understanding Laws, Evolution, and Conservation in the Large-Program Life Cycle. 1980. Journal of Systems and Software. 1. 213–221 .
15. [Lehman's laws of software evolution]
16. Web site: Software Evolution in Agile Development . Nanjangud . Narendra . InfoQ . 29 April 2011 . 19 March 2016 .