Christopher G. Wilcox - Academic Record


	Christopher G. Wilcox

Software Philosophy

One of my interests is how cultural factors interact with traditional software engineering practices. Here's a short introduction to the topic:

Many software projects run over budget and past deadlines, often without producing software that remotely meets user expectations. Since the inception of software development, the same issues arise over and over again. Which factors contribute to the efficient production of high quality software? Why do software projects go astray? The field of software engineering was created to answer such questions. Has the effort been successful? Many working engineers and academics remain unconvinced that software development has evolved into a rigorous engineering discipline.

While many theories explain the current state of affairs, in my mind they fail to address the fundamental issue. My view is that one of the predominant factors in software engineering is the culture and behavior of its practitioners. To discuss such an assertion, we need a concise definition of cultural factors and how they differ from other concerns. My simple taxonomy divides software development concerns into three separate but related areas: technical, environmental, and cultural.

Technical factors consist of the advances that we normally associate with computer science: programming languages, compiler theory, parallel programming, object-oriented (OO) and aspect-oriented (AOP) techniques, database theory, and so on. My list includes everything that lies within the theoretical framework of computer science, with the exception of software engineering. Technical factors exert a strong influence on software development practices, as witnessed by how quickly theory becomes reality in many areas of computer science.

Environmental factors are the real world implementation of the techniques discussed above. Such factors comprise the milieu in which software engineering is performance at any given point in time. The most obvious of these are hardware platforms, operating systems, and compilers and other development tools. Theoretical advances are interesting, but no significant programming can be done until the theory is manifested into working tools. The recent profusion of parallel programming models provides an excellent example of how this works.

Cultural factors encompass the behavior and practices of software developers and managers. The process of developing software is essentially a culture practice, as so elegantly portrayed in "The Mythical Man-Month" by Fred Brooks. As such, it is heavily influenced by the attitudes, biases, and backgrounds of the participants. Within most organizations, a process is followed when making software. Software processes range from ad hoc to carefully managed development lifecycles as described in the software engineering literature. I contend that cultural factors exert an unduly strong influence on the project success or failure.

All of the factors just described are closely related. As mentioned, technical factors drive the environment for software development. Throughout the history of computer science we have seen technical advances appear in actual products very shortly after (or even before) new ideas are published. Environmental factors often exert a strong influence over software development practices. As an example, the development world is currently dividing sharply between desktop and mobile platforms. In a circular fashion, cultural factors often control the adoption of new technologies, which in turn influences the practice of software development within an organization. The real question in all this is what are the cultural factors that contribute to success? I have compiled my own list:

Prototype user interface and minimal functionality early in the project cycle to promote everyone's agreement on the product definition. Insist on early customer involvement.

Adhere to modular programming, layered architecture, and object-oriented design. Architect carefully, and share the block diagram project-wide. Cohesion is good, coupling is bad.

Assign a limited number of architects, and allow them to control and review key interfaces to avoid architectural degradation. Complexity is particularly costly at the interface level.

Have a minimal level of process maturity, including at the very least requirements analysis, written documentation, progress tracking, defect reporting, source control, etc.

Make sure you have a comprehensive test strategy, dedicate engineers to testing, allow them to have attitude, perform testing at all levels, and make sure everyone uses the product.

Formally evaluate tools and processes, get everyone in the same room and on the same page, embrace technical advances when possible and appropriate. Use the best tools available.

Hire according to a rigorous interview process and objective criteria, don't just hire your friends. Get a broad range of experience and skills.

Along with this list I include a list of worst practices. For example, why are programmers still trying to write portable C, when Java essentially solves the portability problem? Why are programmers still using C arrays when C++ template containers are so much easier and more reliable? Why do managers still make the same mistakes pointed out by Dr. Brooks decades ago? Here's my short list of pitfalls:

Avoiding the pain of understanding customer requirements becuase you think you know better, by never talking with users and never writing down requirements.

Implementing all of the best practices ever discovered, including every bit of a comprehensive model, or whatever process the guru of the day recommends.

Doing things the same way you've done them before, no matter what. Being afraid of change, and never modifying process because it might upset someone.

Following the waterfall model, while claiming to have an iterative methodology. The main symptom of this is not being able to show software until late in the lifecycle.

Hacking that compromises the architecture to get the product out the door. At least consider the impact on maintenance, reliability, etc. down the road. You can promise to fix it later, but you won't!

Performance tuning for the sake of showing how smart you are. Only invest in optimization that is clearly needed to fulfill product requirements, justified by meaningful benchmarks.