Finally if you’re looking on advice on how to break software into modules based on making your API surfaces easier to use for yourself and other programmers, have a look at A Philosophy of Software Design. It’s not comprehensive, but it’s concise and relatively easy to read: https://books.google.ca/books/about/A_Philosophy_of_Software...
> An unfortunate and unintended side effect of the Manifesto for Agile Software Development is that many teams have stopped or scaled back their diagramming and documentation efforts
This is tragic and too common, in my experience.
It's like code should document itself but it rarely does. And then we're back to transmitting oral knowledge to every new hire.
Huh, I hadn't made this connection, and it still may be a case of ad hoc ergo propter hoc. But they sure seem to go hand in hand.
I think the first team on which I ever experienced Agile– working on a CAD program–was quite good with documentation. Ever since then–about 8 years–it's been almost all bad. All my work since then has been on iOS, so I maybe difference in domains has something to do with it too.
I often say code should "document itself," but I guess what I really mean is that the code should be readable and understandable, without obfuscating elements like anonymous identifiers, single letter variables etc, or needing comments explaining _how_. Comments should explain higher-order ideas like _why_. But, no one part of code can document the whole _system_ to which it belongs, nor can e.g. Git commit logs (which I also highly value). For that, I absolutely need diagrams and technical writing.
> But, no one part of code can document the whole _system_ to which it belongs, nor can e.g. Git commit logs (which I also highly value). For that, I absolutely need diagrams and technical writing.
Yes, that's exactly why I use the maps analogy when describing the C4 model. If you were teleported into a random part of the country where you live, you'd find it very time consuming to figure out where you are, and where you want to go. The maps on your smart phone allow you to zoom-out and get some context.
It's the same with code. If you join a team with a million lines of code, simply exploring the code to figure out what's going on is an incredibly time consuming exercise. The diagrams and technical writing provide some zoomed-out context, so you can better explore the codebase.
The trick, of course, is to not document everything. Travel guidebooks don't explain everything about the place you're visiting, only the highlights. Software documentation should do the same.
> The trick, of course, is to not document everything. Travel guidebooks don't explain everything about the place you're visiting, only the highlights. Software documentation should do the same.
This makes a lot of sense to me, I think I would do well to remember it, thanks!
I’m not sure if it’s worse. There’s been a documentation gap as long as I’ve been thinking and working on this (1995+).
I don’t think it’s complete but I think I see more documentation and diagrams so it out automatically from projects than the stuff that was consciously written by librarians or tech writers (or created as a result of following a CASE tool or Rational’s UML stuff).
I have seen far fewer garbage diagrams that confused. If I had to choose between a pretty diagram and a hacked up json file that describes relationships, I’d take the semistructured data any day. I can always write a graph viz on top of the data that is always in sync, but it’s really expensive to try to spit out data from a graphic.
Too complex. Architecture descriptions should make sense to everyone on the project. It should not require you to learn a new design language or notation to use.
Unfortunately it won't do much for an architect's ego but, if the same simplistic approach to diagrams is applied to all levels of abstraction, it just may provide a thing all stakeholders can reason about and agree to.
As some background, the rationale behind creating the C4 model was to create software architecture diagrams that would be easy for anybody on the project team to understand. The example diagram you linked to is fairly typical of what most teams seem to do today unfortunately, and exhibits the following problems: ambiguously named boxes, unclear responsibilities, unexplained symbols, ambiguous relationships between elements, no key/legend, unclear abstractions, etc.
To solve these problems, you do need some sort of common language. Switching to UML potentially solves many of these problems, but teams don't want to do this for various reasons. As noted on the website, the C4 model is an abstraction-first approach (with only 4 levels to learn), although the simple notational tips do aid comprehension.
This I agree with, other than prescribing n levels - many projects don't need more than 1, most only need 3 (conceptual, logical, physical).
Note that people in the protection (life insurance) industry will see no ambiguity in this diagram. Note also that this diagram is contained in a Vision/Scope document that provides disambiguation (each box is described), responsibilities (and all stakeholders) defined, and all abstractions. Haven't anonymized that document so it's not online. Obviously no one on the team works alone, so it's made clear to all.
Largely based on the Microsoft Solutions Framework from many years ago because it's still the best tool in this space. And by god after 20 years as an architect and before that 14 as a developer I've seen my share.
I respect the effort brought to bear here, but there are existing solutions and frameworks out there, from the terrible (6 Sigma or EDS' BCL Framework) to the excellent (MSF 3, parts of the UML).
I've been a dev for about as long as you were and keep flirting with the idea of moving into a dedicated SA space. How do you feel about the current trend towards togaf and ITIL? Are the useful, useless, good to cherry pick?
Also did you enjoy your last 20 yrs as a SA compared to if you stayed in some type of development? I'm quite interested in your back story as there's not a lot of seasoned SAs where I live.
Both TOGAF and ITIL certifications are a great way to land a job as an architect, but (especially true in the civil service) these documents often become the deliverable, project outcome be damned.
Just getting certified will alter your thinking in a good way though. It shows you in what kind of environment and how many large enterprises work, and there's use in that.
Yes I enjoyed it. You need to keep yourself honest though. Once async/await came along I doggedly stuck to message queues because it's what I knew. And then I wrote an app using async/await and felt like a fool. It can be humbling.
And I stay in development by writing a lot of code at home, sometimes at work to demonstrate a thing. An SA who doesn't maintain dev skills is not as effective as one that does.
I moved into EA about 5 years ago, and my next move would've been CTO, except that I resigend last year to write software I hope to make a living from. I'm lucky in that I managed to sell a solution I developed about 12 years ago for myself to an enterprise for enough money to not have to work for the next 10 years at least.
I like like architecture/design, and perpetually live in the persuit of perfect technical documention (what I wrote 10 years ago and thought of as good was... terrible), so of course I'd recommend it.
I like C4 because the diagrams are easy to understand irrespective of the industry and those diagrams cover an abstraction span that's impossible to cover through one diagram. I didn't get the feeling that it has to be 4 diagrams and I pick and choose myself.
I get it that the example you gave works for your project, but if I took that and transplanted it into mine I'd expect a lot of questions at least about the relationships (arrows) and probably also about the responsibilities of the components (boxes).
Having a textual description of the above certainly helps and should be done, but people tend to focus on the diagrams and ignore the text in my experience.
> I didn't get the feeling that it has to be 4 diagrams and I pick and choose myself.
Correct. I get the feeling that most teams using the C4 model tend to stick to the top two levels - System Context and Containers (these levels also "age" slower than the others too).
> but there are existing solutions and frameworks out there, from the terrible (6 Sigma or EDS' BCL Framework) to the excellent (MSF 3, parts of the UML).
That's a fair point, and I'm occasionally asked why I'm "reinventing" UML too. Again, that's a totally valid question. If teams are using one of the existing approaches/notations, and it's working for them, I recommend that they stick with it.
But the majority of the teams I work with either (1) have no diagrams/documentation at all or (2) use collections of ad hoc "boxes and lines" diagrams ... usually "because we're agile". In many cases UML, ArchiMate, SysML, MSF, etc are not even on their radar. In others the teams are simply not interested in using these existing approaches, for a number of reasons.
Too complex. Architecture descriptions should make sense to everyone on the project. It should not require you to learn a new design language or notation to use.
That seems a little strong. We're talking about a potentially complicated system in a technical field. Some standardised notation and terminology could be helpful to keep the visuals concise and unambiguous. I agree that this looks far too complicated, though.
I’ve tried and used uml for 20 years. It’s hard to learn and a different way of thinking for many staff. I see a pattern of the business analyst / product owner /non-tech lead hiring someone to make uml for ver. That abstraction costs money and time and creates confusion.
So I think it’s not a worthwhile investment. The non-techs can’t create or edit and typically can’t read. The techs don’t use it for what they are creating.
I’d rather spend time on docs in the actual system components or teaching non-techs enough to read autogenerated docs or walk through code that actually is the system. Then couple that with block diagrams written on whiteboards or powerpoints or whatever. If you get to the point you need notations it’s too complicated and point them to the appropriate source.
This isn’t easy. But I think it’s easier than pretending that both sides learning an artificial middle language (uml, etc).
Because construction analogies are stereotypical if you are building a building. A big, expensive skyscraper. If your construction workers speak English and your financiers and architects speak French. I think you’re better off investing in hiring some people who speak both languages than in making them both speak German to work on blueprints.
Personally I find UML (in its full, standardised form) is still a lot more complicated than is normally necessary too, but it does include versions of several useful ideas.
I have often seen a group of developers discussing something and then someone sketches a simple diagram showing entities and relationships, transitions of a finite state machine, or message-passing similar to what UML would call a sequence or communication diagram.
However, in my entire career, I have never seen anyone actually draw out fully annotated UML diagrams using the official notation, nor any instance of some of the numerous diagram types UML specifies. And that includes working at places where everyone had been given UML training!
I've seen a number of organisations where everybody has been given UML training, and mandated to use a specific UML tool. In reality though, everybody is just abusing UML class diagrams as a way to draw ad hoc "boxes and lines" diagrams instead! :-)
The C4 model doesn't seem to mandate any particular notation, it's merely lines and boxes and you can use whatever notation you prefer.
"The C4 model doesn't prescribe any particular notation. A simple notation that works well on whiteboards, paper, sticky notes, index cards and a variety of diagraming tools is as follows."
Curious to know what makes you think it's too complex? What am I missing?
Disagree. It might not come across so great in written form but I watched this person give a presentation on this model and it was one of the best I’d ever seen. Probably the most logical way to explain a complex system. And then he rolled out the library he had written that could generate these diagrams automagically from Spring code... mind blown.
Your example is less clear to me compared to the C4 method. It's missing descriptions for relationships for one. Secondly, a single diagram can't tell the whole story.
In case, like me, the headline makes you think it's related, this is not the same as Pieter Hintjen's C4 development process, the "Collective Code Construction Contract":
UML for the last level does not seem particularly adapted for functional languages or languages with a sophisticated module system though. Anyone knows of some conventional description language that would be a better fit, or is this a case where the code (higher level functions, often times forming some DSL, or the higher level module interfaces) is legitimately the actual description then?
I like to draw software. Nothing tells people what's going on at a glance better than a bunch of boxes and arrows.
Some 5-10 years ago when UML went out of vogue (for good reasons; much of it is overspecified and overcomplicated), and around the same time FP started getting traction, I think we accidentally lost something.
I like FP and I also like using FP principles in not-strictly-FP language's such as JavaScript, but I miss OO's drawability.
If anyone knows of a compelling diagram syntax for FP style software, I'm deeply interested.
The best I've seen so far is abusing class diagrams for modules, which kinda sorta works when you follow a convention (common in eg Elixir) of having each module export one struct type plus some functions that operate on that struct. It's basically OO without inheritance that way so not surprising that it fits. But that feels a bit like shoehorning my FP into class diagrams rather than anything else. Any takers?
> Some 5-10 years ago when UML went out of vogue (for good reasons; much of it is overspecified and overcomplicated)
Perhaps UML ceased to be a fad for some, but to this day UML is still the best tool (and practically only tool) that helps describe and document code effectively and efficiently, from requirements to what classes comprised a submodule.
The "oversimplified and overcomplicated" jab makes no sense at all. UML is used to provide a view of a system without needing to go too much into detail (that's what code is for), and just because a bunch of people did a great job putting up a specification aimed at software developers intending to develop UML editora and code generators it does not mean it is overcomplicated". Anyone can use UML extensively if they can draw rectangles and lines on a piece of paper, or click and drag around in an editor. That's hardly a downside or a sign it's "overcomplicated".
I take it you never took a UML certification exam. Even the most useful UML diagram, class diagrams, has so much syntax that you can express all of C++'s weird ownership quirks plus some quirks no actual programming language has.
UML activity diagrams let you specify, in full detail, the semantics of a trivial 3-line constructor using a ridiculously large amount of arrows and boxes. You can get good at this (even though it serves no purpose whatsoever) and get "UML Certified" stamps on your forehead.
UML use case diagrams, hardly useful for anything at all, have a rule that you can put the name of the actor either above or below the actor stick figure. The name of a use case (an ellipsis), however, may only be above or inside the ellipsis, but not below. Put the name below the ellipsis and your UML is syntactically incorrect. Volumes of books were written about nonsense like this. The UML spec is many hundreds of pages. If you don't think this is overspecified and overcomplicated then I want some of what you're smoking.
I believe it is these aspects of UML that made it unpopular among the "get things done" crowd. I agree that UML class diagrams and UML state charts, when you just use the normal, useful parts of them, are super useful and currently underrated.
If the UML people had remembered they were making a communication tool and not a programming tool, then maybe so many people hadn't run away screaming.
> I take it you never took a UML certification exam.
You're either very disingenuous or deeply confused. You don't need to hold a Master's degree in English to hold a conversation with English speakers. You don't need academic certification to use a tool to communicate ideas.
It makes no sense to assert that the usefulness and ease of use of a language is demonstrated by how far and how extensively it may be covered in an academic setting.
I don't understand why you insist on disagreeing so violently when we're mostly on the same page. In fact I don't disagree with a single word in your comment, except the part about disingenuous which baffles me.
We agree that UML has a subset of common features that are super useful to communicate about software. That doesn't make UML as a whole any less overcomplicated and overspecified, though!
Most people agree that languages like C++ or Scala are large and complicated. The fact that you can choose to write C++ code that only uses functions and structs and pointers doesn't change anything about that.
I said nothing about UML's ease of use. I said something about its drop in popularity and about the swathes of useless features UML fans promoted when it was at the peak of its hype cycle. I mean, I assume you agree that UML is less popular now than it used to be?
Sidenote: nothing about UML was ever intended to be academic, I'm not sure where you got that term from all of a sudden. It was all very industry driven. People wanted to do "model driven development" and realized if only UML could have super precise semantics, you could generate entire programs from UML. This actually works and serious amounts of software have been painstakingly written this way. It's totally nuts.
> If the UML people had remembered they were making a communication tool and not a programming tool, then maybe so many people hadn't run away screaming.
The problem is OMG turns every analytical tool into programming tools, sacrificing focus on the core purpose; they did the same thing (with similar effect) with BPMN.
it was too tempting honestly and a natural tension of having computers manipulating graphs .. at one point you want to link the graph with the concrete artefact automatically
It's unfortunately not true. A large fraction of the UML creators come from the formal specification camp. They always wanted UML to be a specification language, instead of a visual communication tool for developers. Therefore, most of it is way too fine-grained for getting a birds-eye view. For example, you can't have a sequence diagram with components as messages sources and destinations, only with objects.
Of course, you can just ignore the UML semantics and draw something that looks like UML. That might be still useful. However, it then isn't UML anymore.
> Perhaps UML ceased to be a fad for some, but to this day UML is still the best tool (and practically only tool) that helps describe and document code effectively and efficiently, from requirements to what classes comprised a submodule.
UML sucks for requirements (outside of, maybe, sequence diagrams); BPMN—which OMG overcomplicated the same way it did UML—is far superior for most requirements-relates diagramming, and honestly DFD’s and minispecs in the style of (Yourdon, et al.) old-school structured analysis beat UML for requirements.
For classes in a module, well, despite the wide availability of tools that will generate UML from code for that, the dominant mechanism seems to be autogenerated text documentation. That suggests people don't find UML adding a lot of value in that role.
UML is, however, pretty much the ultimate tool for communicating the structure of an object-oriented system to someone familiar with UML but not the implementation language. And UML sequence diagrams are, if not necessarily the best, at least a useful and popular way of showing interaction patterns even outside of OOP.
Perhaps UML ceased to be a fad for some, but to this day UML is still the best tool (and practically only tool) that helps describe and document code effectively and efficiently, from requirements to what classes comprised a submodule.
What if your system isn't designed around OO-style classes?
A significant proportion of the development work my businesses have been doing recently involves systems where reliability is at a premium. They include some quite intricate logic and they will be deployed in environments where fixing bugs later can be very expensive. Based on that experience, you throw out the negative aspects of "enterprise" development pretty quickly when quality actually matters, right behind a large set of Agile-related buzzwords.
Well, functional languages really only distinguish between data and functions. They usually also lack control flow structures, because they are composed of expressions rather that statements.
+1 for DFDs ... UML was designed with OO programming languages in mind, and doesn't really fit well for diagramming functional languages (unless you start creating your own custom UML profile, which I don't see many people doing).
From everything that I have seen, any software architecture modeling model that relegates dynamic behavior to a 'supplementary' status is suboptimal. One of the ways that waterfall fails is that the architects come up with an imposing, apparently comprehensive static structure that overlooks various issues that arise when it is put in motion. These points of contention often do not become apparent until one is trying to integrate the parts into a coherent system.
Software design is always an interation between static and dynamic matters -- use informs structure, and structure informs use.
> any software architecture modeling model that relegates dynamic behavior to a 'supplementary' status is suboptimal
Another explanation is that the static structure diagrams are what give the C4 model its name: Context, Containers, Components, and Code. These diagrams existed for a number of years, before the addition of the other diagram types, which themselves supplement the static structure diagrams.
That said, I don't find myself using the dynamic diagram type all that often. For software systems of any size/scale, I'm not going to create dynamic diagrams for every use case/user story/feature/etc. Instead, I'll use dynamic diagrams sparingly to describe overall patterns in use, significant use cases only, or complicated sets of interactions between things.
To prevent the issues you mention about overlooking issues, especially when doing any up front design, I recommend "dry-running" the significant usage scenarios across the static structure diagrams and/or running a risk identification exercise (e.g. risk-storming). Or you could use a formal architecture evaluation technique.
UML - I know next to nothing about UML - but what I do know is the language was invented first and then people came around and tried to give semantics to the language. Well, in other words what that means is that the language was invented first and it really didn't mean anything. And then, later on, people came around to try to figure out what it meant. Well, that's not the way to design a specification language. The importance of a specification language is to specify something precisely, and therefore what you write - the specification you write - has to have a precise, rigorous meaning. - Leslie Lamport
UML: a language that was invented first and then people came around to try to get semantics. - Leslie Lamport
UML: fuzzy pictures of boxes and arrows. - Leslie Lamport
People use UML, things like UML, to model programs, but it's not clear how to translate them in to sequences of states, for concurrency. If you cannot translate them in to sequences of states, it means you don't understand them, and it may mean that there's nothing there. You know, there are lots of people selling snake-oil, drawing boxes and arrows that make you feel good, but ultimately have no real meaning. If something is really meaningful you should be able to express it in mathematics. - Leslie Lamport
When a giant like Leslie Lamport says he knows next to something about a subject but then gives a view anyway, it's probably wise to consider that he might just be being diplomatic. His comments about ambiguities in UML and the difference between that and a rigorous mathematical specification are reasonable in my experience (as someone who has both done significant work with high reliability code and formal methods and also worked in enterprise world with UML and all that).
I'm a big fan of this model - I've changed all my work architecture models to C4 and they make sense instantly to other engineers. The plugin for draw.io is really easy to use as well.
Nice approach but it seems to be rather focused on that modern web or mobile app. How about a system with redundancies? Is a DB replication group a single container? Or do I have to know exactly the number of DB nodes?
With the C4 model, you typically treat your database schema as a "container", and save the deployment/infrastructure concerns (e.g. replication) for a separate deployment diagram.
Also relevant, the Design Structure Matrix (DSM): https://dsmweb.org/ and https://mitpress.mit.edu/books/design-structure-matrix-metho... (mostly learning by example, not all software-relevant, available as an e-Textbook on Amazon) If you code in Java, IntelliJ has this diagram included.
Finally if you’re looking on advice on how to break software into modules based on making your API surfaces easier to use for yourself and other programmers, have a look at A Philosophy of Software Design. It’s not comprehensive, but it’s concise and relatively easy to read: https://books.google.ca/books/about/A_Philosophy_of_Software...