The Source Code Is The Design

A traditional manufacturing product cycle has a design phase and a manufacturing phase. If we want to draw an analogy between a manufacturing product cycle and a programming product cycle, the programming activity is analogous to the design phase, while the compilation is analogous to the manufacturing phase. Note that compilation is practically free, as opposed to the manufacturing phase in a manufacturing product cycle.

Many have insisted on treating programming like a manufacturing activity. The main reason for the confusion is that TheSourceCodeIsTheProduct is an illusion caused by the expensiveness of producing source code, and the inexpensiveness of compiling executables (ManufacturingIsExpensiveButCompilingIsCheap?).

TheSourceCodeIsTheProduct is an invalid assumption. I think this is the big revelation, which might explain why people shout TheSourceCodeIsTheDesign (when maybe perhaps they should be shouting TheSourceCodeIsNotTheProduct?).

See WhatIsSoftwareDesign, which was the seed that started this page, and which discusses JackReeves' article of the same name.

This resonates with some thoughts about project management. It seems like a common error in software project management is in treating all the stuff that developers do as a manufacturing activity. This seems to be a driver behind a lot of poor management decision making, and unrealistic goal setting.

Maybe if software managers understood better (I could just leave it there ;-) that software product (executables) may indeed be manufactured in any quantity in a short time, flawlessly and cheaply by the machine, they wouldn't try so hard to map all the activities upstream from final build onto a production line.

I guess everyone from FredBrooks on has being saying these things, but MichaelFeathers' comment really crystallized this idea for me.

(If you are a software development manager reading this, be assured I'm talking about someone else.) -- KeithBraithwaite

Since programming is a design activity, not a manufacturing activity, you shouldn't automatically think you can treat programmers like assembly line workers. TheSourceCodeIsTheDesign might help explain why throwing more programmers at a project that is behind schedule will hurt it, not help it. -- francis

A software developer is more akin to an artist than an assembly line worker, since they are designing in writing source code. It takes time, consideration, and in some cases, a bit of skill. Creation can not be rushed. Throwing more people into the design mix can be counterproductive, unless one of them takes the reigns and becomes the architect. More people can be helpful to get the work done, but first you must determine what needs to be done in terms of the design. -- JasonBurkert

The position of the article was not that source code makes all other documentation obsolete, it is simply that the act of programming is designing. To be quite honest, I sometimes fear that people who read about XP superficially may get the impression that it is "let's go in like gangbusters and hack something up." Especially those who come from places with formal trails through a development process, and the managers chant "what do you mean you don't have a pile of documents??" You don't really hear the word design thrown around in XP. Yet, I do believe, from experience, that WardAndKent are right. You do just have to try things out in code. It is how you solidify and validate design. TheSourceCodeIsTheDesign.

The thing about TheSourceCodeIsTheDesign is that it recognizes that coding is part of designing. The difference between someone who can step up to the computer and program/design something well without having elaborately diagrammed it, and a hack who paints himself into corners is really one of experience, talent, and motivation to learn. OK, but what about the LessAbleProgrammer?

The artificial line between programming and design causes tremendous waste. Everyone thinks that software is intrinsically different. That it requires a completely different approach from anything else in engineering. It isn't different, we just misunderstood it. We believed that coding and testing were not part of design. They are just like prototyping on a breadboard. Part of design.

This is one of the TwoIrreparableMistakesOfTheSoftwareField, IMHO. -- MichaelFeathers

This just occurred to me, but if TheSourceCodeIsTheDesign, then are UML diagrams, etc., merely different views of the design?

Shouldn't the tools be used to extract the diagrams from the source code (I know most tools have this option, but using it is usually frowned upon)? Use Cases may be a problem, but isn't it also a problem when reading source code in determining what end user functions it can perform. Finally, should refactoring show up as cleaner diagrams?

Anyone else care to speculate on how TheSourceCodeIsTheDesign principle could reverse the relationship between UML and source code? [ consider that TheMapIsNotTheTerritory ]

-- WayneMack

ReverseEngineering is all about this reversed relationship, trying to find out how to distill design representations (potentially UML) from the source code.

-- ArieVanDeursen

I think the Reeves article touches on a very interesting and subtle phenomenon in human value systems in which intermediate effects come to be valued as much as their end effects, and it gets difficult to tell exactly where the value lies. When the translation of source code to machine code is highly predictable, it's natural to shift attention to the source, even though no machine can run it directly. But similarly, machine code has no value when the machine is broken. Because of the high availability of working machines, we don't have to focus there. Where we have to focus is all that matters.

If I look very carefully at anything I value, I see a complex chain of effects feeding it. Stable systems quickly become taken for granted, so we place our value tokens on the parts of the changing world that matter. If a design is something other than a product valued for itself, then it is merely some effect that is also a cause in a system stable enough for us to predict a value outcome.

Most computer systems are stable from the compiler down to the execution of binary instructions. Therefore, it's natural to think of "product" as the artifact just above that base. That's language source code. The system that produces that artifact is still quite unpredictable, so it's not likely we'll shift our orientation. Artifacts more abstract, be they whatever, will constitute a nebulous range of "designs" or "specifications". I think it's the lack of a direct and strictly repeatable translation of these artifacts that characterizes them. ("Specification" is interesting nomenclature, because it's really the lack of specific-ness that keeps it from being the product.) Nor is it possible to sort out all the possible forms in this category and give them names.

It seems wrong to say that any one of these tenuously connected artifacts is the design. Unless, of course, it's the only document you have <s>. So, while Reeves has caused me to think in more detail about the chain of effects present in software development, he hasn't convinced me that the source is the design and not the product, practically speaking. But then he didn't claim to have discovered something of practical value, right?

-- WaldenMathews

On Dichotomy

"Advocates ... hold that the definition of a concept specifies its meaning in its entirety. Objectivists hold that the units integrated by the concept constitute its meaning." -- From: http://enlightenment.supersaturated.com/essays/text/ioe1/09.html

Literate Programming tried to balance the equation between Design and Source Code by having a single document depicting both design and source code (TeX and Mathematica are good examples of this kind of programming). Even though this style was not prominently picked up by the mainstream software community, of late, the tool LEO (http://leo.sourceforge.net) has done a remarkable job in this arena. A single XML file contains both the document and the code with a GUI front end depicting the software structure as a tree. You can selectively extract source code or comments from this document. Unlike any CASE tool, Leo does not use any extra intelligence to generate its own source code. You get what you put in there in the first place. In any realistic development the code always evolves from the first cut before it is fully functional. The beauty with Leo is that all these changes can be imported back into the original Leo document.

-- Hemanth P.S.

I believe that there is an interesting comparison to be made with electronic engineering and chip design. Proponents of CASE tools often promise to make software engineering more like electronic engineering, with UML diagrams analogous to circuit diagrams. However the design for complex digital chips is actually done using a programming language - the two leading languages are Verilog, which is similar to C, and VHDL which is similar to Algol. To quote a manual on Verilog (found at http://www.eg.bucknell.edu/~cs320/1995-fall/verilog-manual.html):

Digital systems are highly complex. At their most detailed level, they may consists of millions of elements, i. e., transistors or logic gates. Therefore, for large digital systems, gate-level design is dead. For many decades, logic schematics served as the lingua franca of logic design, but not any more. Today, hardware complexity has grown to such a degree that a schematic with logic gates is almost useless as it shows only a web of connectivity and not the functionality of design. Since the 1970s, Computer engineers and electrical engineers have moved toward hardware description languages (HDLs). The most prominent modern HDLs in industry are Verilog and VHDL. Verilog is the top HDL used by over 10,000 designers at such hardware vendors as Sun Microsystems, Apple Computer and Motorola. Industrial designers like Verilog. It works.

So in chip design, The Source Code Really Is The Design.

-- DaveKirby

Except now Object-Oriented HDLs are coming out such as http://www.aracnet.com/~ptkwt/ruby_stuff/RHDL/ based on Ruby. A UML tool that could generate Ruby could then be used to control the design of the HDL code. So it comes full circle.

Agree in Principle, Disagree on Semantics

I propose that the real issue is that design is not really a beneficial activity in software development, and to say "The Source Code Is The Design" is trying to use semantics to gloss over the issue.

I feel this is an important distinction if the goal is to remove the "design" stage from the software development process. Rather than being afraid of being accused of "not doing design", we need to turn the debate around to be "Why should we do design?"

For some tasks, it may be much more cost effective to create a design and evaluate the design before building the actual product. For software, this is not the case. For years, software has struggled to come up with something to use for "design." We had flow charts, PDL, Data Flow Diagrams, prose descriptions, and now UML. With software, however, it takes as much time to create the design as the actual software; the design is more difficult to validate than the actual software; and the simplifying assumptions made in the design are often the critical issues to evaluate in the software. For these reasons, it simply is not cost effective to design, iteratively correct the design, then write the software (and then iteratively correct the software). It is better to start with the software and iteratively correct it.

I believe it is time to explicitly state the long held secret of software, we do not need to do design; design is ineffective and costly.

-- WayneMack

See http://www.martinfowler.com/articles/designDead.html for a very different take on the need for design.

I am talking to a VC. They want to see something that shows we can do the software part of the project. Should I say, "So sorry, the source code will be the design?" Of course not.

A VultureCapitalist isn't going to be interested in SW design documents - source code or reams of dead trees. VCs are interested in whether or not you have a good business plan and business model. Of course, there may be bad VCs that consider themselves able to pass judgement on software specifications, but that's just bad lending practice and has little to do with good engineering practice.

A set of UML designs is not going to convince a VC of anything. Working software is useful, but as the previous writer has noted, the VC wants to see proof that the software can be sold for a profit. Tell a VC that you've gone through your initial capital to create paper diagram of what you want to do and he will probably just walk out the door.

If the source code is the design, then what are the UML diagrams and the project specifications that are created before the coding to be called? A design is a plan. From the programmers view, the source code is the plan because with that you can determine how your program will behave. On the other hand, if the source code is the design, then it is very difficult to communicate the design to someone except for the involved programmers. Maybe one can say that the source code is a view of the design? Just like design documents, UML diagrams and text descriptions are other views of the design?

UML diagrams and other associated paperwork (if any) are documentation about the design. A documentation can have different views as appropriate to shed some light (if any) on various aspects of the object described. However, the source code has a privileged status: it is not just "documentation about", it is the object itself.

You could, perhaps more precisely, consider the Source Code to be a concrete expression of the design--the design itself being an abstract thing. As such is is the most accurate expression, for the UML diagrams or other non-executable documentation may not be in sync with the implementation.
There are lots of partial points here... Some developers actually like to "design" and document - and there is value in that. Design is the sanity check and collaboration that happens on a white board, the reduction of logic from a large K-map(yes - these apply at times - does anyone remember what they are?) to a code approach, and the thing you do anytime you get out pencil and paper to work a problem because the IDE in front of you doesn't allow for it. There is also value in moving quickly to code for an experiment... experience has shown the need for both. And the reality is that there are still XP practitioners who avoid any design efforts or artifacts to a fault. There are Architecture classes where people are trying to choose the LDAP product and the App server before they have found out who the customer is... too much predisposition to action is too common... streamline steps, trash the unnecessary ones, but don't just skip them because they aren't interesting to you.

XP supposedly is about collaboration - SO why is it just about collaborating only with other developers? How do you describe your product to the customer's IT department? The question is more about limiting the artifacts and maintenance of them, and then the communication to other parties that NEED to know... support will not read code, sometimes the deliverables are more than just code, VCs actually do "technical due-diligence" - not just "business due-diligence" - and deciding you know enough about "vulture capitalists" to discount their use of consultants who actually might want to see design - I hope you aren't expecting to woo any investors or market to anyone with technical savvy.

I think too many developers have been ducking the communication skill problem - if you can't present your "design" in meaningful terms to those who would support it, use it, invest in it... you have come up short on the developer skillset. Jobs include the ability to communicate to the business. Knowing which artifacts matter, when and how to use them(minimally - absolutely; not at all - I don't think so!), when informal capture can be enough, what you can reverse out through tools and WHAT YOU CANNOT, how to convey to others, and how not to get bogged down by it... not simply evading that part of the job... comes from experience and collaboration and a predisposition for aggression+quality... that is what we should be talking about.

The binaries are actually the PRIMARY deliverable - not the source - but not necessarily the only one; Also it is not inconceivable that non-code models can be used to generate generate binaries. MDA is way young and perhaps not destined to produce much... but I have worked on projects where, thru code, we took further coding out of the loop! Code that produces code - is a trend - I wouldn't get completely tied to source listings as an excuse for documentation... just be sensible - and not evasive - about reality. If you think you know it all and that things are black and white - you are young.(Even Kent Beck on occasion (paraphrasing) has said you can be too extreme)

-Jeff Smith-

You make some nasty assumptions there by saying that the binaries are the primary deliverable, you're wrong. First, it's a mistake to assume there are binaries, you just excluded all the dynamic languages extremeo's love so much. Secondly, you're assuming that the product is where the value lies, but many products are produced for the purpose of being sold, not directly for a particular customer, and often the writers of that code are considering the company itself as the product, hoping to be bought out by a bigger company... who wants that source.

Secondly, when someone hires an expert to do something for them, it's a mistake to think he should justify his methods with them, frankly, if they had the competence to understand those things, they wouldn't have needed him in the first place. The problem isn't developers not communicating, it's business people thinking they can comprehend those things by reading a few documents and avoiding the fact that they don't have the necessary context to be able to communicate with the developer to make technical decisions. Technical decisions, architecture, etc, are not their decisions to make, they are the developers. Their only decision lies in which developer they hire, and in communicating the domain knowledge they'd like encoded into a program, beyond that... they must trust the expert, or risk bogging him down to the point of failure. Communication is certainly a skill, no disagreement here, but to place the blame on the developer seems a bit absurd, he was hired specifically because he can do something they can't, the burden is on them to not get in his way, and allow him to do his job. If anyone needs to be better at communicating, it's the business folk, they need to learn to listen, and acknowledge where their input is needed and wanted. I know of no developer who enjoys producing more documentation than code, and this is often exactly what business people expect. The code simply is the primary documentation of what a program does, that is after all why it's called the "SOURCE".

TheSourceCodeIsTheDesign also brings up interesting conclusions on what SoftwareDevelopment is about. Since we are continuously in design mode until release, aren't we always prototyping? Are we delivering RefinedPrototypes?, prototypes that have been deemed GoodEnough? May be, but don't let your VultureCapitalist know that!

An analogy that came to mind when reading the article was regarding OpenSource. OpenSource is about giving the right to use the design; if you have the tools (which in many cases are FreeSoftware) you can "build the product". A striking difference is what happens when a manufactured good's design is given away. In general, the public cannot use the design to "instantiate" the product without incurring considerably amounts of time/money.

References:

Code as Design: Three Essays by Jack W. Reeves http://www.developerdotstar.com/mag/articles/reeves_design_main.html

Also, for a complementary view look at Component Software by Clemens Szyperski ISBN: 0201745720, there is a very interesting exposition about what is and where the design of a software system resides.

TheFooIsTheBar: TheDesignIsTheSourceCode, TheSourceCodeIsTheSpecification, TheAtomIsTheDesign

More about documentation vs source code: ToNeedComments, SourceCodeAndModeling, TheWhatButNotTheWhy, SelfDocumentingCode, DocumentationBeyondTheSourceCode

See: WhatIsSourceCode, UnderstandingHotDraw, TheSourceCodeAndTheArchitecture, ProgrammingAintManufacturing, WhatIsSoftwareDesign, DesignOfTheDesign, ExtremeProgrammingAintPrototyping, SoftwareBlueprints

CategoryDesignIssues, CategoryDocumentation

EditText of this page (last edited April 18, 2006)
FindPage by browsing or searching

This page mirrored in ExtremeProgrammingRoadmap as of April 29, 2006