Collective disaster

"Your code might not be as valuable as you think."
Philomatics, The Value of Source Code, YouTube
Subtitle: Your code is worthless

The video is as cute as it is totally misguided. Not all code, just the poorly thought out and written code that is ubiquitous nowadays, in all guidance as in all practice, is actually a cost, not just worthless: a generally unperceived cost since that is 99% of the industry by now, indeed the very rationale that founds and sustains it is that that is the best we can do.

The fundamental misconception that the problem domain and the solution domain map one-to-one is but one ingredient of that yet-another global corporate-driven fraud and collective disaster.

Against Test-First

Against Test-First in synthesis:

CONS:

• negatively affects product quality:

• obliterates technical analysis/specification
  (which is the fundamental aberration, in theory and in practice)
  => conflating problem with solution domains;

• obliterates the separation coder/tester
  (akin to validating learning models against the training sets)
  => conflating QA with the methods of production;

• negatively affects product/process development:

• obliterates external/internal distinctions (up to the worst RAD)
  (yet what we develop is *what allows building* what we sell, not directly it: whence, together with reasons of intrinsic and utmost complexity, software production driven by the commercial/PM/QA functions, once upon a time rather auxiliary, is just doomed)
  => no development proper: no consolidation of assets, nor of (sane) processes, even less of competences and the chance itself of acquiring those competences;

• enforces the gradual local change (up to TDD)
  (yet the normality hypothesis for program code is false)
  => no restructuring, no global perspective;

• glorifies and institutionalises the trial and error
  => no improvement, no global sanity;

PROS:

• none known so far!

See also:

• the notion of complex system up to that of phase transition in particular;

• the need for "radical (global/strategic/discontinuous) change" vs "gradual (local/tactical/smooth) change": the history and need for BPR over TQM specifically.

[CREATED 2023-04-11 21:49 UTC+2]

Commercial vs production functions and projects

The commercial function, among other things, sells products and services to customers. The production function develops and keeps alive an internal system of competences and technologies which, among other things, enables delivering the products and services sold by the commercial function.

Commercial projects (aka external projects) should be driven in conjunction by a client account for the commercial side and a production analyst for the technical side (meaning roles and functions more than specific people). On the technical side, among other things, the analyst should deliver a requirements analysis which is the basis to place then follow any order to production.

On the other hand, production is strictly technical and wants full autonomy. To begin with, because what we develop is way more than just what we sell, hence the need for an autonomous and strictly technical management and resourcing of production projects (aka internal projects). And, at least as importantly, because, as with all production of complex systems, engineering facts about software and systems are for the most part so utterly counter-intuitive that good old common sense, even the good old common management sense, here is actually worse than just ineffective.

<rant>
Indeed, production driven by the commercial functions is certainly doomed, both on the production side and on the commercial side. And production driven by product or project management is a common incarnation of such an upside-down (dis)organisation. Not per chance 90%+ of software projects keeps badly underperforming. Nor this kind of (mis)management has anything to do with agility proper, which is rather and expressly about a lowest level of formality, certainly not about an upside-down flow of (in)competences and (ir)responsibilities.
</rant>

Definition of Agility

As far as technical definitions go, agility is the antipole of formality along a methodological dimension. Specifically, agility is "low formality", where by formality we mean high level of control, extensive documentation, and heavy bureaucracy.

Note that the agile-formal dimension is orthogonal to the process models, such that we can have agile waterfall, agile iterative, etc. just as we can have formal waterfall, formal iterative, etc. (Corollary: there is no such thing as agile vs. waterfall!)

Note that the agile-formal dimension is also orthogonal to the cowboy-disciplined dimension, such that we can be agile or formal, still we are required to be disciplined. (Indeed, this was the greatest achievement with agility, that before it we only had "cowboy" vs formal!)

On the "software crisis"

I would start from the fact that software production is an industrial, long-term endeavour: just think reuse (aka configuration management), which is only one component of a whole picture, and already that should remind not only the level of complexity (complexities!) involved, but, even more importantly, that an efficient and highly successful production unit (not just a successful project) is not an occasional target and cannot be!  (Indeed, that is why it was called "development".)

Currently, what I see is a general trend of naming "agility" what is actually a quest for the ultimate (worst) RAD and the ultimate (over-)simplification: and that is the clash of industrial vs. commercial [but, more and more so, also the overall economic and financial] cultures and strategies.  What has happened starting with the so called democratisation of programming is, so to speak, that our customers have taken our place!  In practice, building software has become feasible to everybody, with the result that most technical offices are literally up-side down, and the whole marketplace has been seriously [if not irremediably] diluted.

But, as noted initially, software production is a most complex engineering and requires highly sophisticated and specialised competences, to the point that plain common sense (the "magic recipes", invariably just out of the social-mediatic hat) just will not cope.  Indeed, software engineering is the *most* complex engineering that there is, to the point that the majority of its fundamental notions are genuinely counter-intuitive: as a result, plain common sense not only is utterly inadequate, it is actually doomed to failure.

Bottom line, I believe most of today's agility is a cover-up for what is in fact pure plain RAD, and the software crisis is a myth and a spell that can be (easily, to say) dis-solved: just there is an essential distinction to make, between proper and improper software production, with the vast majority of the software endeavours nowadays belonging to the latter category. Then, and just to begin with, the very shape of the industry-specific performance surveys would change... In fact, I would claim that proper software developers (and, scaling up, proper software production units) do, already today, consistently provide near to optimal results, all environmental circumstances considered!

Agile vs. waterfall?!

[This post in the Software Craftsmanship Google Group (*) has cost me getting banned from the group: I am reposting it here to denounce how compromised the situation is and to keep the discussion open.]

Agile vs. waterfall is another one we should clearly stand against: it's just the marketing of a false dichotomy, but it is as wrong and a damage as it can be because it promotes massive misconceptions not only about the software process but about the very state of the art.

Rather than setting up primary schools on the art of tying shoelaces, we'd better focus on getting the basics straight.

(*) Agile vs. waterfall ?! (Software Craftsmanship Google Group)

Why the Software Industry is sucking more and more

With inflation and pollution come ignorance and even more confusion...

To begin with, the A in the ABC of software production:

It is not about doing it right, it is about doing the right thing!

Software is about the conceptual side and only marginally about the technicalities. About using our mind, sensibility and intelligence to provide an effective and reliable product and service to people.

This is certainly clear to our customers: Information Technology is a wonderful tool, yet an enabler, never a driver [except that even this is now failing, courtesy progress up to Industry 4.0 (2023-11-09)]. This is seldom clear to present-day software managers and even developers, who should rather realise that the matter in question is predominantly conceptual even at the lowest levels.

Yet another argument: against test-driven

Mr Jason Gorman, in his post "Test-driven Enterprise Architecture" (September 16, 2008) [*], writes:

I've discovered that the most effective way to align business goals with the software and systems we deliver is through heirachies of tests, not through pretty pictures and dotted lines.

I have to disagree, essentially. It is certainly true that most of those dotted lines get drawn without criterion, which is indeed what makes the difference between an *analysis* and a bunch of dotted lines!

On the other hand, tests cannot drive solution development: the problem domain and the solution domain just do not map one-to-one to each other, rather the latter is a sub-product of the former. In practice, as information engineering teaches, the solution domain emerges out of the opportunities for automation identified within process and data models at the business level. Here is a distinction, just like the distinction between syntax and semantics, that must not be neglected, otherwise not only the understanding but the very quality we are after is gone, with all that in fact counts.

[*] http://parlezuml.com/blog/?postid=693

Italian Agile Day 2007

agileday.it (2007)

Great event in Bologna, with some 300 participants and sessions going for the whole day across three different tracks, from classic sessions for beginners, to open space sessions for advanced discussion.

Very interesting relations and ideas were exposed, with most of the speakers sharing real life experiences. Most of the discussion revolved around how to introduce agility in a team, how to track agile projects, how to apply agility to legacy code, how to agile contracts, and even something about agility and ISO 9001.

I was very pleased to see how agility is gaining momentum even in Italy, despite our industry has been almost sinked in the last few decades, and the IT field is then suffering for the lack of serious engagement.

Below a personal list of concepts I have noted down for my future meditation and adoption (this list simply reflects my current state of thinking and professional needs; DEV=development, PM=project management, LGY=legacy):

1. DEV - Make stories/tests from bugs

2. PM  - Reduce the work in progress

3. LGY - Refactoring cycle: 1) Select; 2) Break dependencies; 3) Write tests; 4) Refactor

4. LGY - Test used/modified code only

5. Use a wiki for tracking

6. Use a Progressive contract as an umbrella/framework contract over multiple Fixed Price contracts for new development phases; use Time & Materials for maintenance and small changes (max 2 days).

The one thing I would criticize does not relate to the event itself, it relates to the content of most relations. Even Tim Mackinnon, the keynote speaker and well known agility pioneer, said it: you need OO technologies to be agile. IMHO, that is plain wrong, and we should simply be talking about component based development, that is applying the most basic engineering design principle: modularity. This is not a cavil, the nowadays widespread emphasis on OO technologies is making a lot of damages, mainly among the younger professionals, all in the name of this idiotic idea that you can ultimately solve problems with tools-based/frameworks-based development. Welcome to the shop...