Kuro5hin.org: technology and culture, from the trenches
create account | help/FAQ | contact | links | search | IRC | site news
[ Everything | Diaries | Technology | Science | Culture | Politics | Media | News | Internet | Op-Ed | Fiction | Meta | MLP ]
We need your support: buy an ad | premium membership

[P]
Foundations of Software Architecture

By Herring in Technology
Wed Mar 26, 2003 at 11:13:44 AM EST
Tags: Software (all tags)
Software

My job specification describes me as a Software Architect. A lot of people who I deal with haven't heard the term before and don't really know what it involves. Up until recently, the role wasn't really widely accepted but as systems grow more complex with the need to integrate multiple technologies, legacy systems and third party components, a job which is rather more than the old "technical lead" has emerged.


What is software architecture?

Good question. I've heard many suggested definitions over the years: from "a way of paying programmers more without them having to do project management" to "reducing the possibilities for unnecessary creativity". The first definition is insulting to architects, the second is insulting to the people who have to turn the scribblings of these madmen into an actual application. twh270 gave me a more serious text:
A software architect is someone who knows how to put together a system that satisfies the fundamental requirements of its stakeholders. He knows software technologies; processes; understands project and software lifecycles; etc.

The eXtreme Programming camp have a simpler definition of software architecture: "a complete waste of everybody's time and money". I have to disagree with that - just from the point of view of job security.

Different organisations have different definitions for the role of architect. In fact different architects within the same organisation have different definitions. There should be one thing they have in common: the ability to see the bigger picture. OK, writing applications is fine, splitting the application up into components even is fine. The issue comes with interoperability and deployment - particularly with web applications. The application needs an infrastructure to run on and it needs to be secured.

The "Software Architect" could be seen as an ex-programmer in a nice office (I wish) surrounded by CASE tools and big, colour printers dishing out the True System Design in uncorruptable UML to the unquestioning code-monkeys. I hope that this isn't the case. Most of the job is about communication - which involves a lot of listening.

Current methods

For years people have been trying to build up standards around this poorly defined field. John Zachman first published on the subject in 1987. The Zachman Framework has been in existence for many years now. The Open Group is in the eighth year of it's TOGAF project (The Open Group Architecture Framework). Not a lot of mainstream interest was generated for a while, but IEEE 1471-2000 (sorry, you have to pay for this document) was an important step in acceptance of the field.

All these approaches have the idea of viewpoints and stakeholders. A "stakeholder" is somebody who has an interest in the system, e.g.: the project owner, the user, the developer. A stakeholder has a "viewpoint" on the system. Zachman defines six viewpoints for each of five stakeholders. I'm not going to go through them all as they are defined on the website, but an example would be "who" (people) which for the system owner is the people involved in the process and for the designer might be the use-case model. People can argue about the correctness of Zachman (for instance, the separation of data and function might annoy OO purists) but it does provide a guide for what should be done.

OMG MDA (Model Driven Architecture - not an exclamation made by LK5 people on a night out) is a UML based method for producing the artifacts required by Zachman or TOGAF. It relies on the idea of starting with a Platform Independent Model (PIM) which captures the basics with no reference to specific technology. For instance you would refer to a generic "RPC" facility rather than specifying CORBA/RMI/DCOM. This drives the Platform Specific Model (PSM). The principle being that specific technology could, and does, change over time. I have some personal misgivings about this for two reasons:it is yet one more level of modelling and you don't see a lot of System 390s dumped in the trash. If you're going to spend 7 figure sums on hardware, the last thing you're going to worry about is having to tweak a few UML diagrams.

Holy Wars and pragmatism

Q: What's the difference between a terrorist and a methodologist?
A: You can negotiate with a terrorist.

OK, that may be vaguely amusing, but it does represent a significant issue in software development: religious wars. Wars over languages, wars over notations, relational vs OO, OO vs components, XP vs waterfall, "enterprise architecture" vs JFDI.

OMG MDA would be enough to drive the average XP fanatic into a frenzy - doubling the amount of Big Design Up Front which gets done. Similarly, XP would cause the average project manager (most of whom still live in the waterfall age) to keel over with a premature stroke.

The "purists" of whatever camp frequently seem to ignore something very basic about what we, as software developers, are trying to achieve. In descending order of importance:
  • Produce a tool to solve a genuine problem. Software is not an end in itself.
  • Produce a secure, reliable system.
  • Produce a system which can be maintained and extended

  • Something I wouldn't put on the list is: "have a really nice looking class diagram". An elegant design is great but if the system doesn't achieve anything worthwhile then that's not really success.

    Does this mean that all of these methods are false? Well, no, not really. There is some wisdom to be found in all of them. Nobody can really disagree that frequent testing and heavy user involvement contribute highly to a project's success. Similarly, it's clear that a multi-tiered system, linking to legacy apps on 3 different platforms needs some sort of planning and design up front. Much as we may dislike them, project managers are there to make sure we don't spend years tweaking the UI on a system which is already good enough.

    Over the years of attending conferences, studying books etc. I haven't ever really seen a method which is just plain wrong (OK, we know waterfall doesn't work on big projects). I have seen methods be applied inappropriately. Use judgement - if development could be done by following a big book of rules then we could write a program to do it.

    Summary

    The term "architect" brings us back to the old building analogy - which some advocates of XP would say was broken. It may be badly dented but it's the only analogy we have. When constructing systems, we do have to bear in mind how they will fit in with existing infrastructure and how they will interact with current and future systems. Yes, on large systems, we do need the equivalent of a stable foundation on which to build.

    With buildings, then some things do need to be defined up front - such as how many floors. Some things don't - such as the colour you paint the walls. The problem with software is that it's not always clear which things fall into which category. For instance, it's easy to change text on a web site, but what if your "user" wants the ability to change it every hour? The boundary between the structural and the cosmetic has moved. Your web server may reside on the same DMZ as Big Critical Business App so there's no way that Security will allow random users to arbitrarily mess with HTML/JSPs/ASPs. The application has completely changed - you're adding some sort of content management facility with the associated complexity.

    Glossary

    Waterfall: The traditional method for developing systems where you do all of the analysis, followed by all of the design, followed by all of the development, followed by all of the testing. Heavily reliant upon the assumption that requirements do not change during the course of development. Project managers frequently assume that this is the case because it's the only way to cost the project up front.

    eXtreme Programming (XP): This is almost the antithesis of the waterfall model. XP assumes that requirements will change and that this is not necessarily a bad thing.

    Unified Modelling Language (UML): Several years ago there were competing notations for designing Object Oriented systems. UML brought together three of the leading creators of these notations to create what became the standard.

    Sponsors

    Voxel dot net
    o Managed Hosting
    o VoxCAST Content Delivery
    o Raw Infrastructure

    Login

    Poll
    Software Architecture
    o Sounds interesting 15%
    o Already familiar with it 53%
    o Sounds like a theoretical exercise with little practical value 13%
    o I am an eXtreme Programmer - take your diagrams and shove them 17%

    Votes: 52
    Results | Other Polls

    Related Links
    o twh270
    o Zachman Framework
    o TOGAF
    o OMG MDA
    o broken
    o eXtreme Programming
    o Unified Modelling Language
    o Also by Herring


    Display: Sort:
    Foundations of Software Architecture | 77 comments (59 topical, 18 editorial, 1 hidden)
    3 Simple questions (5.00 / 5) (#10)
    by treefrog on Wed Mar 26, 2003 at 06:56:43 AM EST

    'The eXtreme Programming camp have a simpler definition of software architecture: "a complete waste of everybody's time and money". I have to disagree with that - just from the point of view of job security.'
    I'd have to say that I think your description of XP is a bit too, well, extreme

    A large part of my role is software architecture, and on the whole I am a proponent of the XP way of doing things - although I am willing to mix and match from both camps (agile and waterfall) as necessary.

    Even in XP or other agile methods, software architecture is necessary. You need to be able to set out.

    • What are going to do What part of the system functionality are you going to build?
    • How you are going to do it What language/languages are you going to use. Which technologies will you use. What level is the system expected to scale to? How do the constraints fit with the capabilities of the team you have (or how do they affect the team you are to need)?
    • How will you know when you have got there How are you going to confirm that the system works as planned?
    Without these basics, you can't (and shouldn't) really start. Unfortunately, too many projects start with even these simple questions unanswered. It is no wonder that the softwre industry is littered with failures!

    Best regards, treefrog


    Twin fin swallowtail fish. You don't see many of those these days - rare as gold dust Customs officer to Treefrog

    Fair point (5.00 / 1) (#11)
    by Herring on Wed Mar 26, 2003 at 07:02:55 AM EST

    I was paraphrasing an eXtremist who I heard speak at a conference last year. There are a lot of good things in XP but, as with many things, I think we're seeing the thesis/antithese/synthesis routine play out here:
    Thesis: waterfall
    Anithesis: XP
    Synthesis: ?


    Say lol what again motherfucker, say lol what again, I dare you, no I double dare you
    [ Parent ]
    And that is why... (4.00 / 3) (#29)
    by lukme on Wed Mar 26, 2003 at 11:43:07 AM EST

    the most inovative software is written by experts who know their field and learn programming to solve a problem that they have.


    -----------------------------------
    It's awfully hard to fly with eagles when you're a turkey.
    [ Parent ]
    View from the trenches (5.00 / 2) (#20)
    by n8f8 on Wed Mar 26, 2003 at 09:32:57 AM EST

    It boils down to two kinds of people. Those who learned to program and like doing it. Those who learned to program and discovered they hate it. The latter become "Configuration Managment Specialists", "Consultants", and in some cases "Software Architects". In all fairness I've seen "Software Architects" on both sides.

     

    Sig: (This will get posted after your comments)

    Hmm. (5.00 / 5) (#21)
    by Herring on Wed Mar 26, 2003 at 09:42:28 AM EST

    I still really enjoy programming (on the rare occaisons I'm allowed to). I've seen people who call themselves "software architects" who don't have a good understanding of the technology. I've also seen the aftermath...

    To return to the (possibly broken) building analogy, building architects need to understand the properties of materials. Few people have attempted to build styrofoam skyscrapers. Sadly, people have attempted to build the sofware equivalent.

    One more point: I tend to distinguish between "programming" and "software development". The distinction is between producing "really neat" code and "really useful" code.


    Say lol what again motherfucker, say lol what again, I dare you, no I double dare you
    [ Parent ]
    What's your background? (5.00 / 1) (#26)
    by sien on Wed Mar 26, 2003 at 10:47:43 AM EST

    I've worked with a Software Architect who was trained as one at school and he was worse than useless. However, you seem like you've worked in other positions. Could you talk about your background and also say how you think Software Architects should be trained?

    Also, do you think Architects are valuable for all projects? It seems that one of the problems when people talk about software engineering is that they talk about all aspects of writing code from writing a half-day script to writing an OS and then talk about process. This is analagous to saying that writing an email should have the same process as writing a text-book and and a novel. The XP people seem to be particularly guilty of this.

    [ Parent ]

    Background (5.00 / 4) (#28)
    by Herring on Wed Mar 26, 2003 at 11:05:25 AM EST

    What I've done most of I suppose is C++ development. I've also done network setup and hardware support in the dim and distant past. It's when I got into distributed processing systems a few years back that the "big system" thing became important. When you start banging your head against DCOM/CORBA and associated performance, scalability and security issues then life becomes complex. I think it's better to come up through the development route - if, as a developer, someone is good at seeing beyond the immediate functional goal,then it seems a logical route.

    Training? Not sure. Most of the training I've had is on specific technologies so I understand the implications of using them. I go to the architecture conferences as often as I can. Talking to other people trying to solve the same problems is very useful.

    On small projects, there's no need to fill in all the Zachman boxes. Part of my role though is to keep half an eye on what people are doing to save re-inventing the wheel and with a view to future interoperability.

    Oh, and there's one really important thing to remember: your network guys know more about the network than you do; your Java techies know more about Java than you do; your DBAs know more about database tuning than you do etc. Ask them.


    Say lol what again motherfucker, say lol what again, I dare you, no I double dare you
    [ Parent ]
    Three years ago they didn't. <nt> (none / 0) (#34)
    by jforan on Wed Mar 26, 2003 at 12:09:58 PM EST


    I hops to be barley workin'.
    [ Parent ]
    Misconceptions about Methodology (5.00 / 3) (#30)
    by mberteig on Wed Mar 26, 2003 at 11:44:38 AM EST

    XP is explicitly designed for smaller projects (2 to 15 people or so), with a high level of feature movement. Most of the literature on XP acknowledges this in a very up-front way.

    There are other methodologies designed for other project sizes. The "crystal" methodologies are "shrink to fit" in that they can be selected and adjusted based on the needs of a specific project and team.

    Even RUP, one of the purveyors of "big design up front", is an adjustable methodology. Not very many people think there is a one-size-fits-all methodology. Those who do think this way tend to be trained as project managers. (The discipline of project management comes from construction and thus they love waterfall.)




    Agile Advice - How and Why to Work Agile
    [ Parent ]
    RUP (4.66 / 3) (#31)
    by Herring on Wed Mar 26, 2003 at 11:48:59 AM EST

    Reminds me of DSDM - lots of small waterfalls.

    I have had arguments with XP zealots who claim that "you shouldn't do big projects". Well it's fair that big projects are inherently more risky and have more communications issues (see Brookes) but sometimes you just have to bit the bullet.


    Say lol what again motherfucker, say lol what again, I dare you, no I double dare you
    [ Parent ]
    Small vs. Large Projects (5.00 / 2) (#35)
    by mberteig on Wed Mar 26, 2003 at 12:15:28 PM EST

    I agree with you, but I would add that if the work can be done as a set of small projects, that will be a contributing factor to reducing risk. One of the things that very few methodologies address effectively is the maintenance of a software system. The definition of a project is part of the problem (because a project ends at some point in time). Many software systems are very long-lived and need active work even after the "project" is completed. Pete McBreen discusses this in his book called "Software Craftsmanship".




    Agile Advice - How and Why to Work Agile
    [ Parent ]
    I have a love-hate relationship with programming. (5.00 / 2) (#33)
    by jforan on Wed Mar 26, 2003 at 12:06:49 PM EST

    Unfortunately, as projects get larger, there is less typing/coding and more bigger-picture-crapwork.

    As a "System Archtect" myself, I love the typing/coding.  I hate the all the other crap.  But I agree with the article author - you will never have a useful 7 figure software installation without a large portion of other crap.

    I specifically love it when the designing thread running in my brain can keep ahead of the typing thread, so that after having cruised through implementing a complex interface for the first time, the outcome is beautiful, complete, useful, and well documented.  In big projects, rarely does one get to do this, as there are so many implications for every piece of code that is written.

    Jeff

    I hops to be barley workin'.
    [ Parent ]

    The one kind inspires the other (5.00 / 2) (#38)
    by ethereal on Wed Mar 26, 2003 at 01:54:53 PM EST

    I.e. after you've been on a project that was poorly architected, had poor or nonexisting software configuration management, etc., some people will be inspired to do those things because the lack of those things are what prevent you from maximizing the fun of coding. And that's how I fell into it - I'd rather be coding (and, in fact, on coding days I'm always the happiest at the end of the day) but I absolutely hate to put out the forest fires of lost code or totally wasted effort. From that perspective, architecture or configuration management are risk management - you have to do some of them in order to make sure that the absolute worst case scenario doesn't happen when everybody starts coding. They reduce the fun level of the day, but overall they prevent severely unfun days from happening.

    Kind of in a communist manifesto-type way, once all developers are tremendously skilled, dedicated, and strong communicators, the need for software architecture and configuration management will go away. But until then somebody has to head off the train wrecks on larger projects via architectural-type activities.

    --

    Stand up for your right to not believe: Americans United for Separation of Church and State
    [ Parent ]

    Lost code and wasted effort (none / 0) (#47)
    by NFW on Wed Mar 26, 2003 at 03:56:02 PM EST

    That's the sort of thing that got me into an architecture role. Coding is fun, but less so when you're guessing at requirements and after seeing wrong guesses become wasted time. I made a big stink about getting our requirements on paper before we started coding, and next thing I knew nobody understood the system like I did (having put it all on paper) and I'd been promoted.

    The best part about having agreed-upon designs on paper is that when the time comes to write code, it just flows from your fingertips. Fingers on the keyboard, nose to the monitor, pound away, and when you're done, it works. When you already have answers for 90% of the tough questions, you can really focus on coding in a way that isn't possible when you're questioning requirements and design decisions every step of the way. That level of focus makes it much more fun, IMO/IME.

    My last company was fortunate in that we could get stable requirements up front, though.


    --
    Got birds?


    [ Parent ]

    agreed - requirements depend on the customer (none / 0) (#71)
    by ethereal on Thu Mar 27, 2003 at 12:48:03 PM EST

    The larger the system, hopefully the more willing the customer will be to commit to a spec up front. I also have worked in that environment and prefer it.

    --

    Stand up for your right to not believe: Americans United for Separation of Church and State
    [ Parent ]

    This is boring. (1.42 / 7) (#22)
    by tkatchev on Wed Mar 26, 2003 at 09:53:41 AM EST

    Tell me again, how is this related to Iraq?

    :)

       -- Signed, Lev Andropoff, cosmonaut.

    Holy Christ you're right (none / 0) (#25)
    by AmberEyes on Wed Mar 26, 2003 at 10:16:19 AM EST

    I got to the Zachman Framework stuff, and my eyes glazed over. Thank god I have a scroll-whell equipped mouse.

    -AmberEyes


    "But you [AmberEyes] have never admitted defeat your entire life, so why should you start now. It seems the only perfect human being since Jesus Christ himself is in our presence." -my Uncle Dean
    [ Parent ]
    Actually (5.00 / 1) (#36)
    by jabber on Wed Mar 26, 2003 at 01:05:13 PM EST

    If you look at the middle east as "the problem", and the war as an "implementation effort", then the parallels start falling into place.

    Once the implementation of the solution is complete, which will almost certainly be both over-due and over-budget, we'll move over into "maintenance" phase.

    I certainly hope that the USKAian governments and forces did a good deal of upfront architectural design before transitioning to the implementation phase. I hope their design favors simplified maintenance rather than a speedy implementation.

    I also sincerely hope that the process they applied is neither the waterfall nor XP. Personally, I'm a fan of phased spiral development with FDD and AOP being applied as things come up.

    I further hope that proper risk management plans have been drafted and prioritized, so that as requirements change, stakeholder resolve waivers, milestone deadlines whoosh by unmet and so on, our army of implementers isn't sentenced to a "death march" followed by round after round of downsizing.

    See, it's got a whole lot to do with Iraq. In fact, I think it should be reposted to Op-Ed/Politics.

    [TINK5C] |"Is K5 my kapusta intellectual teddy bear?"| "Yes"
    [ Parent ]

    As Webster slowly weeps (5.00 / 6) (#23)
    by RyoCokey on Wed Mar 26, 2003 at 10:00:00 AM EST

    Foundations of Software Architecture

    At least spell the title right.



    Pacifism in this poor world in which we live -- this lost world -- means that we desert the people who need our greatest help.
    -- Francis Schaeffer,
    Fuck [n/t] (5.00 / 2) (#24)
    by Herring on Wed Mar 26, 2003 at 10:03:31 AM EST



    Say lol what again motherfucker, say lol what again, I dare you, no I double dare you
    [ Parent ]
    Dammit, now we look like Slashdot -nt (5.00 / 2) (#39)
    by czth on Wed Mar 26, 2003 at 01:55:59 PM EST



    [ Parent ]
    come off it (5.00 / 1) (#41)
    by werner on Wed Mar 26, 2003 at 02:06:41 PM EST

    Webster couldn't spell. He especially had problems with "-re" and "-ou-" ;)

    [ Parent ]
    For the Record (5.00 / 9) (#32)
    by mberteig on Wed Mar 26, 2003 at 12:03:31 PM EST

    which some advocates of XP would say was broken

    Since that link points at my article, I can only assume that you think (1) that I am an advocate of XP and (2) that I represent other advocates of XP. Unfortunately neither is correct.

    Although on occasion I will advocate the use of XP (on small projects with a high degree of requirements risk), I will also advocate RUP, simple waterfall, or absolutely no defined process depending on the project and team. If one puts together a team of superstars who have all worked successfully together in the past, then you don't really need a methodology to get done the things that you yourself point out as being the whole point of softare:

    • Produce a tool to solve a genuine problem. Software is not an end in itself.
    • Produce a secure, reliable system.
    • Produce a system which can be maintained and extended

    Software architecture is an intermediate work product which has a supporting role for the above mentioned goals. In and of itself, it is useless (regrettably, since I too am billed as a software "architect"). The software architecture documents do not solve a problem, they do not guarentee a secure reliable system, and they do not guarentee a system which can be maintained and extended. These things are only accomplished by having "good" code - code that performs useful work, is bug-free, and which is clearly written. A good software architecture document can help a team get to good code, but sometimes it is not necessary, and sometimes it can actually get in the way.

    A methodology that reduces the role of the software architect (like XP), is simply defining a way of doing things that (hopefully) provides a different set of mechanisms to replace the useful aspects of software architecture. For example, the XP practices of pair programming and collective code ownership can reduce the need for intermediate documentation products (such as a software architecture document).




    Agile Advice - How and Why to Work Agile
    I agree with this post (5.00 / 1) (#40)
    by Herring on Wed Mar 26, 2003 at 02:00:06 PM EST

    Sorry if I misinterpreted your article.

    Yes, it's sometimes difficult to supress a laugh when someone hands you a diagram with four boxes and six arrows and announces "that's the architecture". Ultimately the system is the deliverable - creating reams of paper to cover your arse might help come performance review but doesn't necessarily get useful systems built.

    I'm not "anti XP", I just distrust fundamentalists of any stripe.


    Say lol what again motherfucker, say lol what again, I dare you, no I double dare you
    [ Parent ]
    SA's responsibility (5.00 / 3) (#37)
    by Mazzaroth on Wed Mar 26, 2003 at 01:20:56 PM EST

    In my mind (and according to the projects I did work on) the Software Architect has the responsibility to:
    • Make large-scale technological decisions by choosing, for example, the architectural patterns (will I use a blackboard, a 2,3,n-tier, pipeline, etc)
    • Identify and properly manage technological risks
    • Make sure functional and non-functional requirements will be rightly and efficiently fulfilled
    • Determine components and how they will interact
    • Make buy/adapt/develop research and recommendations
    • Enforce naming conventions, coding standards, code walkthrough and reviews (at least initially)

    to name a few responsibilities.
    Anyway...

    Great article! (5.00 / 1) (#42)
    by Ricochet Rita on Wed Mar 26, 2003 at 02:22:44 PM EST

    And I'm firmly in the practicing XP Camp! IMO, you gave fair treatment to 'our' way of doing things, without resorting to name-calling or preaching.

    And you're correct that abandoning the traditional waterfall method was a very hard sell to our cumtomers (who happen to be the government). In our case, a compromise of sorts was reached. We still provide some high-level designs, up front, but without many details--we build them a model as we're putting together our initial 'story cards.' Part of this modification of traditional XP, stems from our customers' inability to state their requirements, instead waiting for to tell them what they want or need.

    It was this frustrating behavior that nearly scrapped the whole software development process, back in our pre-XP days. It was (and still is) so unbelievably hard to get any of the GS's to make a decision, under their management climate. They might someday *shocked look* be held accountable for it!

    R

    FABRICATUS DIEM, PVNC!

    Thanks (5.00 / 2) (#43)
    by Herring on Wed Mar 26, 2003 at 02:53:40 PM EST

    Unfortunately, where I work, the whole software development process (and, in fact all projects) have been hijacked by old-school project managers. The idea of "keep going as long as we're delivering ROI" is a total anathema to them. They want end dates, costs the whole caboodle up front.

    I was toying with the idea of getting the main protaganists in this movement together and playing "XP preacher" to them just to see how they'd react. Funnily enough, the projects I've been involved in which were genuinely successful were ones with an "on site user" and a load of prototyping. Mind you, there was quite a lot of BDUF as well ...

    The stuff I work on wouldn't be easy to XP. For a start, the bureaucracy involved in getting things rolled to the web can take six weeks (capacity planning, security etc.) - which cripples your ability to deliver often.


    Say lol what again motherfucker, say lol what again, I dare you, no I double dare you
    [ Parent ]
    Analogies (4.00 / 1) (#44)
    by Salamander on Wed Mar 26, 2003 at 02:57:33 PM EST

    I'm also a software architect, and I think the analogy with building architecture is a good one. If you want to build a house, one of the first things you need is someone to turn the buyer's vague ideas about what they want into a concrete framework - this many rooms, of these sizes, connected this way, with windows and doorways and staircases and visual features like so. The architect has to have enough of a structural engineer's knowledge to understand what's possible in terms of materials and techniques, and enough of an artist's eye not to produce a birdcage or a cave, and so on. S/he does not need to have a particular idea of which couches and kitchen equipment are going to be in the house, or precisely what color the paint will be.

    Similarly, with software, the architect is concerned with how the product breaks down into components and how those components interact, but not with things like like which RPC method or STL container to use except as those choices do or do not support the overall structure. Just as masons and carpenters and electricians and plumbers are expected to know how to turn a spec into a physical object that meets certain requirements, so are individual programmers. A large part of a software architect's job is to make it possible for the individual programmers (of which, ideally, they are one) to make their choices and avoid the equivalent of a sewer pipe hanging over the middle of the living room, without each and every programmer having to check with each and every other for such things. After all, we all know that N^2 communication patterns suck...right?

    Yes, that means that software architects often spend way too much of their time in meetings arguing over requirements and interfaces. Believe me, it's not because they'd rather be doing that than coding. I positively loathe the fact that my job keeps me too busy with that other crap to maintain my accustomed status as the "top gun" coder in the building, but it's better for the company that I spend time doing that so somebody else can keep their mind on coding than to keep me coding and end up with that sewer pipe over the living room.

    Yeah, you can build a tree-house without a plan, but don't try it with a real house - let alone an office building or factory. Someone needs to keep track of the big picture, and you can't leave that entirely to the business-school types. It's better to have someone involved in that who actually knows the difference between a driver and a server.



    buildings and the waterfall model (5.00 / 1) (#64)
    by upper on Thu Mar 27, 2003 at 05:47:52 AM EST

    The biggest problem I see with the analogy of software architecture and construction architecture is that building projects are almost always organized as waterfalls. That is, each project is thought of as a self-contained entity which goes through requirements, plans, and various stages of construction and produces the finished building.

    Buildings get modified repeatedly, of course, and the net effect of a series of projects resembles spiral development. But each project is done as if it's the last one, and the building is often used for some time before the next project starts. It isn't really spiral development, though. It's more like a series of major releases each developed by waterfall. Each project has a plan, usually to fix the worst problems with what's there. In this way, office buildings and factories do get built with no overall plan. It may be more expensive and painful than having the right plan at the start, but it is adaptive and self-correcting. I recommend "How Buildings Learn", by Stuart Brand, which is a study of this.

    A better analogy would be to the construction of larger units -- housing developments, university and corporate campus, and urban infrastructure. With housing developments, there's often a plan to build in phases, with clear opportunities to replan between phases. University and corporate campuses and urban infrastructure systems often have a grand plan, but they often change it as they go. Building is done in response to short-term needs, but with an eye to future development, and it never really stops.

    Interestingly, the suggestion is being made that buildings should be built in a manner resembling XP. The best-known proponent of this is Christopher Alexander -- his books "A Pattern Language" and "The Timeless Way of Building" are widely known. In fact, XP is reportedly the result of Kent Beck and Ward Cunningham's attempts to apply Alexander's ideas to software.

    [ Parent ]

    Pair-brickies! (none / 0) (#65)
    by Herring on Thu Mar 27, 2003 at 05:58:26 AM EST

    Two brickies, one trowel?

    Sorry - couldn't resist.


    Say lol what again motherfucker, say lol what again, I dare you, no I double dare you
    [ Parent ]
    Council workers (none / 0) (#67)
    by treefrog on Thu Mar 27, 2003 at 06:43:15 AM EST

    -enough said!
    Twin fin swallowtail fish. You don't see many of those these days - rare as gold dust Customs officer to Treefrog
    [ Parent ]
    one downside of architects (5.00 / 1) (#45)
    by radish on Wed Mar 26, 2003 at 03:21:43 PM EST

    philosophically I agree with ethereal's take on this, but I've come to the conclusion that the procedural risk of having an official architect is pretty high.  having an architect per se centralizes a lot of design activities and makes it too easy for the requirements to change in midstream, on the assumption that it's the architect's problem to make it all work out.

    coupla illustrations:

    I got sucked into the architect role at a job where I had originally been hired as a developer, just because I had the most hands-on experience.  I saw the featuritis problem coming and went to great lengths to design for flexibility.  unfortunately my emphasis on flexibility ultimately bit me in the ass, allowing so many bells and whistles to get added after the fact that it seriously reduced the amount of time available for each one, and leading to some rather ill-conceived and poorly implemented features.  it wasn't until later that I even figured out where I'd gone wrong.  

    I also had a gig at a company so committed to Big Design Up Front that over four years they've gone through three (count em!) distinct architectures for their single (still unfinished) product.  I'm not making this up.  I worked there for one year during year three (architecture #2), long enough to realize that the real problem - incoherent requirements - remained hidden and ignored because the architect kept faithfully trying to squeeze the ongoing ravings of the execs into the framework he had originally come up with.  this may have been the wrong approach, but I can't really blame him, and I have to say that from the strictly technical viewpoint he was a fucking good architect.

    the common factor in this and several other BDUF experiences is that as soon as the Big Design is done, the architect becomes a conduit for execs, managers or clients to request changes (of the sort that are actually easy with agile methodologies but much harder with BDUF) without paying the price in planning.  if I hadn't been there to translate unrealistic requests into plausible development tasks most of the unnecessary features would have died quietly at the managerial level instead of competing for developer attention.  

    It's a balancing act (none / 0) (#46)
    by Herring on Wed Mar 26, 2003 at 03:38:49 PM EST

    You're pulled three ways:
    Users: We've changed our mind, it has to do this
    Developers: We just want to cut code
    Infrastructure: Don't even think about changing anything

    One advantage of the project based approach is that whenever users try to fuck with you, you can wave numbers at them (another 3 months and 100,000 for that feature).

    I spend too much of my time doing analysis (I think). It's hard to tease out what's important vs what is somebody's "great idea". The key thing is to find the right user rather than the useless guy they try to palm you off with because he's not contributing anything else.


    Say lol what again motherfucker, say lol what again, I dare you, no I double dare you
    [ Parent ]
    Not far enough (1.00 / 1) (#48)
    by gleather on Wed Mar 26, 2003 at 04:21:56 PM EST

    One step removed from useless, patronizing management scum is not far enough.

    brief comment (2.25 / 4) (#49)
    by phred on Wed Mar 26, 2003 at 04:27:20 PM EST

    Keeping in mind that I am a convenience store clerk, I'd still like to say that software architects are generally useless fucks unless they have 10 years (at least) experience in systems programming.

    Funny (none / 0) (#50)
    by mberteig on Wed Mar 26, 2003 at 04:34:44 PM EST

    I think this is really funny... and in a wierd way I totally agree.




    Agile Advice - How and Why to Work Agile
    [ Parent ]
    depends what project you're building (none / 0) (#55)
    by hardcorejon on Wed Mar 26, 2003 at 08:00:31 PM EST

    In my experience, people with heavy systems programming experience tend to do too much early optimization and over-optimization while missing the big picture and misunderstanding the ultimate end-users' needs. That's a broad generalization, but there it is.

    Now, that being said, some projects definitely do need lots of optimization attention and the kind of skills that a solid systems programmer brings to the table. Unfortunately, I don't see many of those projects anymore. Most projects are purely at the application layer, systems-level stuff is all done by libraries and 3rd-party code. We assemble apps from components rather than building everything from scratch.

    - jonathan.

    [ Parent ]
    few more comments (none / 0) (#72)
    by phred on Thu Mar 27, 2003 at 02:00:16 PM EST

    In my experience, people with heavy systems programming experience tend to do too much early optimization and over-optimization while missing the big picture and misunderstanding the ultimate end-users' needs. That's a broad generalization, but there it is.

    I disagree. Systems programming implies that theres going to be opportunities to optimize, but in no way does it have to be done first. It should be allowed for. Factoring software in such a way that allows for optimization without impacting system correctness is a valuable skill that systems programming experience offers. Systems software is all about correctness in the face of complexity. Architecture is about leading a team of highly skilled systems programmers to successful implementation of large software projects.

    Or, at least it used to be ;-)

    Seems like what you guys are discussing is "architecture", ie., like MCSE's discuss "engineering", so my bad for not getting a better feel for this. Carry on!

    [ Parent ]

    Heh (4.66 / 3) (#51)
    by trhurler on Wed Mar 26, 2003 at 05:44:00 PM EST

    People who remove themselves from programming for more than a few months at a time and yet think they're qualified to tell programmers what to do are amusing. Similarly, programmers who think code is all that matters are amusing. As someone who both programs AND designs systems at a higher level, I find all these petty arguments over details of process to be truly pathetic; at the end of the day, if you have the right people and they communicate, the job will get done, and if not, you will lose.

    --
    'God dammit, your posts make me hard.' --LilDebbie

    .NET (2.00 / 2) (#52)
    by Silent Chris on Wed Mar 26, 2003 at 07:20:16 PM EST

    I'd be curious to hear what you think of .NET.  I've had limited experience with it, but it's been pretty positive.  It seems to have been developed for developers, and going from idea to app is much easier than the old MS way of doing things.  

    In particular, I'm curious to hear how you handle integration with legacy apps/hardware.  This is one thing I haven't explored with .NET, but I heard is relatively decent.

    It's kind of sad that the best thing MS has ever produced is the least hyped.

    We haven't used it in anger (none / 0) (#68)
    by Herring on Thu Mar 27, 2003 at 06:55:30 AM EST

    We've recently undergone a takemerger and the push is away from MS towards J2EE now (except that noddy client/server apps are still largely VB).

    In theory the integration should be fairly easy. If you're relying on things like SOAP from two different vendors just working "out of the box" then you're probably going to be in trouble.

    I'm taking a "wait and see" attitude at the moment.


    Say lol what again motherfucker, say lol what again, I dare you, no I double dare you
    [ Parent ]
    ooo. You have a job "specification" (4.50 / 2) (#53)
    by porkchop_d_clown on Wed Mar 26, 2003 at 07:30:33 PM EST

    Most of us Archeeetects have to make do with mere job "descriptions"!

    Sorry, but despite being a software architect in my current position, I don't accord it a lot of respect; simply because of the point you mention - it's a dumping ground for a lot "methodologists" who think in theory without any corresponding practice.

    My rules for SA are straightforward:

    1. When working on an existing project, use the existing design. If you don't like it, tough.
    2. When embarking on a new project, keep in mind that it has to be supported by people who don't know your funky new methodology and tools. If you can do it with a methodology and tools the support people understand, do it.
    3. If, by some miracle, you fall into a green field and there are no barriers to hold you back, well, have fun - but remember you're going to have to deliver running code at some point; have a fall back plan in case your spiffy new methodology falls short.

    --
    You can lead a horse to water, but you can't make him go off the high dive.


    Writing style (none / 0) (#59)
    by Herring on Thu Mar 27, 2003 at 03:43:28 AM EST

    It did say "my job description describes" but that read awkwardly.

    I'd never expect a team to immediately fall into a new way of working and be instantly successful. It takes time. Way back, I remember trying to drag a team from C to C++. I came to the conclusion that some people would be writing Application::Run for years.


    Say lol what again motherfucker, say lol what again, I dare you, no I double dare you
    [ Parent ]
    Heh. I had trouble burning that bridge myself. (none / 0) (#69)
    by porkchop_d_clown on Thu Mar 27, 2003 at 08:35:03 AM EST

    I remember staring at a collection of 3x5 cards for a month, each with one supposed "object" on it, trying to think of an object oriented way to approach my project before it suddenly clicked.


    --
    You can lead a horse to water, but you can't make him go off the high dive.


    [ Parent ]
    the only thing you can count on.... (5.00 / 2) (#54)
    by hardcorejon on Wed Mar 26, 2003 at 07:46:34 PM EST

    ...is that there is NEVER any single methodology or approach that will work best.

    Just like nutrition and dieting, what works for one person in one set of circumstances can be vastly different for someone else, whether they are in the same circumstances or not.

    As a software architect, you've got to bring together LOTS of (sometimes conflicting) factors:
    • What sorts of methodologies do your developers adhere to?
    • Do they adhere rigidly or are they flexible?
    • Do they enjoy designing/coding solo or in small groups?
    • Are they very experienced, or are they very junior developers?
    • Is there a healthy mix of skill levels?
      Will there be a lot of ego-clashing going on? Other personality conflicts?

    • Do your product managers (or execs or whatever) *really* know what they want? Or are they groping in the dark?
    • How technical are they? Will you have to constantly be telling them their pipe dreams are totally infeasible?

    • What's the size and scope of the project?
      Do you have enough people on your team to tackle it?
    • Are the skill sets your team brings to bear a good match for the problem you're trying to solve?
    • Where are the truly difficult parts of the design going to be? How many such parts are there?
    There are certainly many other questions you'll need to ask, some more important than others, depending on the situation at hand.

    The most important thing is that (a) the structure/modularity/flexibility of the overall architecture and (b) the design/implementation strategy you choose must be a good fit for both your team and the product to be built. Match one and miss another, and either your developers or your stakeholders (or both) will be pissed off when things go haywire.

    There is no "one right way" to do things. Some good general advice is to use common sense, be a sincere listener, make team-consensus decisions whenever possible, avoid political situations, and make sure you don't forget to have fun! ...of course, that advice could apply to just about anything! :)

    - jonathan.

    Booch on Architecture (4.66 / 3) (#56)
    by FuzzyOne on Wed Mar 26, 2003 at 11:08:21 PM EST

    Wade through all the UML sales stuff, and Grady Booch pretty much confirms all your points here.
    The life of a software architect is a long (and sometimes painful succession) of suboptimal decisions made partly in the dark.
    He goes on to describe the software architect:
  • Not just a top level designer--Needs to ensure feasibility
  • Not the "project manager"--but joined at the hip
  • Not a technology expert--worries about the purpose of the architecture and the fit
  • Not a lone scientist--a communicator
  • IMHO, what makes a great software architect is the balanced presence of these traits (in moderation):
  • Passion for architectural integrity
  • Infectious enthusiasm for the software
  • Patience and desire to teach
  • Breadth of expertise (as opposed to deep expertise in a narrow area)
  • Ability to roll up their sleeves and get things done
  • Wisdom in making tradeoffs
  • A lot of dysfunctional software architects I've encountered overemphasize one of those traits. People reinforce what they're rewarded for. So you end up with influential architects who become caricatures of the above traits:
  • The "professor" who treats every encounter with a peer or underling as a chance to give a lecture. Also known as the "pontificator".
  • The purist who can't come to grips with compromise, never ships code, and wonders why people have stopped listening to him. Lives in dry, hands-off isolation.
  • The prodigy coder who managed to turn awesome productivity into an architect role, yet drags down the productivity of everyone else by never teaching or distributing work. Lives in wet, hands-on isolation.
  • The troll--good in every other way, but incapable of inspiring others, possibly out of disgust for the job. My suspicion is that most of these people have a secret ambition to do something else and are in fact sociopathic about getting to that other life. (Managers trapped in an architect's body?)


  • [ Parent ]
    Your wrong... (4.00 / 1) (#77)
    by ThreadSafe on Mon Mar 31, 2003 at 02:21:50 AM EST

    There IS a single methodology that works best. It just one level higher abstraction than UML, Waterfall, XP etc... It's called not being a short sighted, un-discerning fuckwit who gets caught up with buzz words whether your a designer or a programmer! (malice was not directed at you)

    Make a clone of me. And fucking listen to it! - Faik
    [ Parent ]

    My Definition (4.00 / 1) (#57)
    by bugmaster on Thu Mar 27, 2003 at 01:53:24 AM EST

    I always saw the architect as the one man who knows how all of the pieces of the project are supposed to work together. For example, the architect does not necessarily know what "setUserPreferenceTimeout" does, but he knows that the preference module is accessed by the news, advertisement, and presentation modules.

    The architect does not necessarily impose any specific design choices with an iron fist -- he is just the man to talk to if you want to know how module Foo (which was written by Joe the retired intern) is supposed to work.
    >|<*:=

    No, actually I do use the iron fist technique...nt (none / 0) (#58)
    by gnovos on Thu Mar 27, 2003 at 02:39:15 AM EST



    A Haiku: "fuck you fuck you fuck/you fuck you fuck you fuck you/fuck you fuck you snow" - JChen
    [ Parent ]
    Functional Vs Technical Architects (4.00 / 2) (#60)
    by Boricle on Thu Mar 27, 2003 at 04:26:20 AM EST

    Great article - very well written, very concise.

    I've found that there are three areas of Architecture relating to Software Engineering.

    One is Functional Architects - who often also fit into the Business Analysis area. These are the people who take a look at the massive bucket of user requirements and somehow transform them into nice neat functional packages. This like changing an accounting system into areas such as Accs Payable, Accounts Recievable, Payroll, Journaling, General Ledger etc. I've seen good functional architects take pages of assorted ramblings and produce very elegant and concise representations of the system - the kind of representations that can be used to further drive the development of specific functional areas of the application. Quite often these people have either moved from the business area into IT or are people who long ago moved out of hands on development.

    The second area - Technical Architects (Sometimes also Systems Architects) - these are the people who know that for a system requiring x thousands of transactions a day, for x amount of distributed clients, based on the Functional Requirements (from the Functional Architect) tell you if you need n-tier, client - server, thin client, messaging, RPC, transaction gateways, etc. These guys are responsible for putting together the design of the technical infrastructure.

    The last area - what I think of as the Software Architects - although could also be referred to as Code Architects or Code Designers - are the people who put together the software code infrastructure that all the developers workinside - increasingly these architectures are prefabricated in vendor products like websphere, but in very large applications these are the people who decide on the elegant algorithims, who are really big on Design Patterns.

    After this there I think that there is an associated area of the Super / Uber / Alpha / Artist Developer - this is the person who comes along and in smaller specific areas works the magic that takes your 1st year stupid bubble sort algorithim of array data and changes it into a sorted tree structure, how makes that process only take 5 minutes instead of 3 hours.

    A lot of this depends on the scale of the systems being built. The bigger the system gets, the more the need is there for the components of the Architecture - Functional, Technical, Software to become more specialised.

    Conversely, as systems get smaller, and as bigger systems get simpler to develop the role described as "Software Architect" becomes more of a Jack Of All Trades of a good senior allrounder - someone who can atlk to a client and refine functional details, who knows enough about hardware / infrastructure software to design the infrastructure, and (although this is only really on smaller projects) may know enough to help with the coding.

    As the systems get bigger, the knowledge and time involved in any one of the branches of the architecture area gets so big and demanding that I can't imagine anyone having enough time to do all of them. And as other posters have mentioned, coding is something that needs regular practice to remain proficient.

    I know on one project we locked the Project Manager / Technical / Functional Architect out of the source code system - not because he wasn't any good at coding, but because he spent so much time doing the other roles (this was a moderately sized project - 6 people, 1 year) that they didn't have the detailed knowledge of the code to be effective (read: avoid breaking things).

    Cheers.

    Boris.

    We don't tend to split (none / 0) (#62)
    by Herring on Thu Mar 27, 2003 at 05:34:39 AM EST

    So yeah, I have to do a bit of everything. Unfortunately we have a lack of skilled people to do analysis. The risk of sending someone with no technology skills to do that job is obvious ...

    I've had to learn a fair bit about the business as well, but then I've been at this company over 8 years so that's a long time to pick it up.


    Say lol what again motherfucker, say lol what again, I dare you, no I double dare you
    [ Parent ]
    All I want to know is... (3.00 / 1) (#61)
    by GRiNGO on Thu Mar 27, 2003 at 04:57:33 AM EST

    ...how much do you get paid?

    --
    "I send you to Baghdad a long time. Nobody find you. Do they care, buddy?" - Three Kings


    Obvious answer (none / 0) (#63)
    by Herring on Thu Mar 27, 2003 at 05:35:08 AM EST

    Not enough.


    Say lol what again motherfucker, say lol what again, I dare you, no I double dare you
    [ Parent ]
    ha.. (none / 0) (#76)
    by ThreadSafe on Mon Mar 31, 2003 at 02:18:31 AM EST

    nice sig.. hadn't heard that one before.

    Sorry (nothing to do with topic)

    Make a clone of me. And fucking listen to it! - Faik
    [ Parent ]

    If you really want to know (none / 0) (#66)
    by gazbo on Thu Mar 27, 2003 at 06:06:40 AM EST

    The data is available. But I'm not rude enough to tell.

    -----
    Topless, revealing, nude pics and vids of Zora Suleman! Upskirt and down blouse! Cleavage!
    Hardcore ZORA SULEMAN pics!

    [ Parent ]

    Gotta love all this (3.50 / 4) (#70)
    by Alhazred on Thu Mar 27, 2003 at 09:52:03 AM EST

    See, this is why software 'engineering' is still in the Dark Ages (and believe you me, its in the DARK AGES big time...). Endless discussions about whether or not developers are 'architects' or 'engineers' or 'authors' or whatever nice new fad comes along. Meanwhile software projects still suffer somewhere between a 70 and 90 percent failure rate.

    Basically, who gives a rat's patootie? I could care less if you call yourself an architect, and analyst, or a lead engineer. I guess 'analyst' is out of vogue these days, though to be honest 'systems analysis' was always at least a description that was fairly unique to IT.

    In any case, how about some real progress in the quality of software?
    That is not dead which may eternal lie And with strange aeons death itself may die.

    XP works for us, at least. (5.00 / 1) (#73)
    by dolo on Thu Mar 27, 2003 at 10:10:28 PM EST

    I am a developer in a small consulting firm. I truly believe in the XP formula because it does save time. Time is the only reason we even use computers, so it's pretty important. Time can be the difference between 98 Visa accts hacked and 8 million.

    Nobody can see the future. You can sit down and plan a system from ground up but by the time it's implemented using the trickle down waterfall of information via lengthy meetings and protocol (ie red tape), the system needs changing because of an unforeseen factor (like the OS becomes deprecated). It's amazing how many of those limiting factors exist. You can not plan for the future, but you can react to the present, and XP always wins initiative over Waterfall when it comes time to fix something that needs fixing. XP is perfect for reaction times and that is why we embody it in our corporate Mission Statement. Extreme Programming keeps data intact and the customer moving. It also glues directly to customer needs. A bonus is that XP seems to evade typical weasel tactics utilized by lazy employees who like to sponge. This is because there is always implementation work to do, and changes to make. That keeps them very busy, but at least they are satisfied (for the moment... wait I think one is sneaking up on me with microwave popcorn. Nope it's just a custodian and her custodian cart and headphones... or is it?).

    The reason I think that Waterfall is so popular is because it involves lots of project meetings. Meetings that can take months. And then there are follow up meetings. These make a juicy project take even longer and that can cost a lot of money, with punitive ROI.

    With XP, we are working with the client from go, and we are tailoring the system to them, like skin.

    The only real drawback to XP is possible security holes, but canned apps can be your bouncers, and that's good enough for development, although ironic as most security apps use waterfall. Oh well at least it's good for something.

    Is he talking about a spec? (5.00 / 1) (#74)
    by dh003i on Thu Mar 27, 2003 at 10:39:13 PM EST

    As in, what Joel spent several paragraphs talking about here?

    Social Security is a pyramid scam.

    Software Architecture (none / 0) (#75)
    by quasipalm on Fri Mar 28, 2003 at 02:11:30 PM EST

    "He knows software technologies; processes; understands project and software lifecycles; etc."

    What if he's a she?

    (hi)
    Foundations of Software Architecture | 77 comments (59 topical, 18 editorial, 1 hidden)
    Display: Sort:

    kuro5hin.org

    [XML]
    All trademarks and copyrights on this page are owned by their respective companies. The Rest 2000 - Present Kuro5hin.org Inc.
    See our legalese page for copyright policies. Please also read our Privacy Policy.
    Kuro5hin.org is powered by Free Software, including Apache, Perl, and Linux, The Scoop Engine that runs this site is freely available, under the terms of the GPL.
    Need some help? Email help@kuro5hin.org.
    My heart's the long stairs.

    Powered by Scoop create account | help/FAQ | mission | links | search | IRC | YOU choose the stories!