Software Architecture
Archived Posts from this Category
Archived Posts from this Category
Posted by Chris Sterling on 06 Aug 2009 | Tagged as: Acceptance Testing, Agile, Architecture, DotNet, General, Java, Open Source, Scrum, Software Architecture, TDD, User Stories, XP
Earlier this year I did a presentation on Executable Specficiations for AgilePalooza conference. There is information about working with legacy code, commercial off-the-shelf (COTS) systems, and Acceptance Test-Driven Development (ATDD) using automated acceptance testing tools. Also, the presentation lists types of automated acceptance testing tools out there along with actual names of tools and what they are best used for on projects. Hope it is interesting to you.
Comments Off on Executable Specifications - Presentation from AgilePalooza
Posted by Chris Sterling on 12 Jun 2009 | Tagged as: Agile, Architecture, General, Leadership, Product Owner, Scrum, Software Architecture, User Stories, XP
During an interview at the Better Software conference this week, I mentioned that I thought maintaining a single list of work prioritized by the business was important for our industry to improve. The following text is an excerpt, in first draft form, from chapter 2, “Architecture Integrity”, of my upcoming book “Architecture in an Agile Organization” that is relevant to this topic.
A healthy Product Backlog contains not only new features but also technical requirements to support the next Product Backlog items in priority. The development team is expected to review the Product Backlog frequently to provide cost estimates for Product Backlog items to help the Product Owner prioritize based on both cost and benefit. During the development team’s review they will suggest Product Backlog items that are missing from a technical point of view in the product definition. These technical Product Backlog items will be discussed with the Product Owner so they are able to prioritize them effectively. It is essential that the development team explain why each technical Product Backlog item is important to support other upcoming items. If this is not expressed the Product Owner could be inclined to demote those technical Product Backlog items because the value of them is not understood enough.
When issues are found in the software they usually come in three varieties:
Enhancements are requests for improvements to the software based on the perspective of the user. These improvements should be prioritized based on their value against all other Product Backlog items. An enhancement and defect issues are hard to distinguish from one another at times but that is fine. Defects are system faults that users are able to workaround but should not be happening from a user experience and/or technical perspective. Since we are able to work around these defects and the development team is able to provide a fix in the next iteration they are placed into the Product Backlog and prioritized against all other Product Backlog items. Fires are defects that cannot be worked around and cause serious problems for users of the software. In healthy software applications, fires should be a rare occurrence. It is important that a fix is implemented right away and may interrupt the work getting done in the current iteration.
Figure: This flow diagram describes how an identified issue is triaged in an iterative and incremental approach such as Scrum. If the issue is an enhancement or defect that can be worked around for now it goes into the Product Backlog and prioritized based on its value. If the issue is a “fire” or defect that cannot be worked around then it must be fixed immediately. The development will put the fire into their current list of work in the Sprint Backlog and figure out how it effects their commitment to deliver on their current iteration.
Defects are captured in bug databases in many organizations. Teams using the Scrum process to manage their software development find themsves in a predicament. Do we continue working from two lists, the Product Backlog of desired features and bug database? Or do we incorporate items from the bug database into the Product Backlog? I suggest all teams should incorporate defects into the Product Backlog. There are multiple reasons to do this:
Ultimately, the development is able to maintain alignment with business priorities for the software. Priorities are decided based upon the full reality of the software, new features and defects included.
A single work queue, such as the Product Backlog, provides visibility to the development team about current expectations about upcoming work. The Product Owner should share the Product Backlog with the development team and discuss the direction of the product beyond the next iteration. As the development team gains knowledge about the direction of the product they will be able to provide input into the Product Backlog. This visibility is important so Product Backlog items do not surprise the development team when it is too late to prepare a proper solution.
Posted by Chris Sterling on 19 May 2009 | Tagged as: Acceptance Testing, Agile, Architecture, General, Leadership, Open Source, Product Owner, Scrum, Software Architecture, TDD, Travel, User Stories, XP
AgilePalooza is a one day Agile conference on Friday May 29th at the San Francisco State University downtown campus. There will be two tracks: Learning Agility and Advancing Agility.
“Learning Agility” will be presentation style whereas “Advancing Agility” will use the open space format.
Speakers include David Hussman (DevJam), Chris Sterling (SolutionsIQ), Luke Hohmann (Enthiosys), Lee Henson (VersionOne Services) with special guests Ainsley Nies (open space co-facilitator) and Ian Culling (VersionOne CTO). When not presenting for the “Learning Agility” track speakers will participate in the open space.
Space is limited and the cost is low so please register soon if you would like to attend. For more information or to register please visit www.AgilePalooza.com.
Where?
San Francisco State University Downtown Campus, 835 Market Street, San Francisco. (Powell street BART station in the Westfield Center) - Map/Directions
When?
9am-5pm (check-in and continental breakfast starts at 8am)
Cost?
$69
Register here: www.AgilePalooza.com
Comments Off on AgilePalooza in San Francisco May 29th
Posted by Chris Sterling on 17 May 2009 | Tagged as: Agile, Architecture, General, IASA, Leadership, Product Owner, Scrum, Software Architecture, XP
I have been using a tool for some time with clients and teams to find out what software quality attributes the product development team should focus on in the project. ISO standard 9621 describes the quality attributes found in software. The following image shows the 6 categories and specific attributes contained within them.
Before I knew about this standard I would discuss how different quality attributes are in conflict with each other. For instance, if we write code that is focused on performance we lose some maintainability. It will be more difficult to read code focused on performance because readability will be sacrificed to some extent.
Now that I’ve known about ISO 9621 for many years, I made a simple spreadsheet tool to interview Product Owners, stakeholders, and software development team members with. Here is the Excel spreadsheet form of this tool:
Software Quality Attributes Rating Tool
Although a bunch of smart people have come up with ISO 9621, I found that modifying the software quality attributes rated in the tool worked more effectively with interviewees. This tool is not to decide what software attributes will be present in the software product getting developed. It is used to identify which software quality factors the team should put more emphasis on when making trade-off decisions during the project. Here is a list of the software quality attributes used in the tool:
Using the tool is quite simple:
Please let me know how you use this tool. There have been many teams that have taken up this tool and found it to be helpful for communicating what is most important from a software quality attibutes point of view in the project. Hope it does the same for you.
Posted by Chris Sterling on 27 Mar 2009 | Tagged as: Acceptance Testing, Agile, Architecture, Distributed Computing, DotNet, General, IASA, Java, Jini/JavaSpaces, Leadership, Maven, Open Source, Podcasts, Product Owner, Ruby, Scrum, Software Architecture, TDD, User Stories, XP
I have uploaded the talk I did at SD West 2009 on Yahoo! Video and here it is:
Comments Off on My Talk @ SD West 2009 on “Managing Software Debt”
Posted by Chris Sterling on 19 Jan 2009 | Tagged as: Agile, Architecture, General, IASA, Leadership, Product Owner, Scrum, Software Architecture, TDD, XP
Talk about a loaded term. Even the term itself, “architecture”, when used in the Agile community can start a heated discussion. When I was coordinator of the International Association of Software Architects Puget Sound chapter, the discussions about “what is architecture” caused passionate debate. I am sure this entry will get some interesting comments, as well.
I don’t believe that this entry will solve the question but I hope to at least give some focus for teams who are dealing with the structural integrity of their applications. First off, a definition of “architecture”:
architecture: noun - “the art or practice of designing and constructing…” or “the complex or carefully designed structure of something” or “the conceptual structure and logical organization of a computer or computer-based system”
Each of these definitions interests me but to put them to practice is quite difficult. They are left to interpretation about what is design, construction, structure, conceptual, and logical. Earlier interpretations of this term interpreted these to mean the high-level structure, architectural style, and finally the stuff that is too expensive to get wrong.
It seems to me that my interpretation is a bit different. Working with tools, practices, and processes over the past 15 years has guided me to a different conclusion. I believe that architecture is how it is SAID:
Structure - how the pieces (components) create a solution (application)
Alignment - the degree to which the application or enterprise structures align to current business objectives and strategy
Integrity - the components that provide stability to the structure (application or enterprise) (ie. automated tests and builds, service-level agreements, network infrastructure, etc…)
Design - a way to conceptually communicate the planned or implemented structure of a component, application, or enterprise (ie. system of names, XP practice of Metaphor, information radiators, etc…)
Using these as the basic building blocks for teams to focus their efforts can be helpful.
The structure is the reality of the solution’s construction. If the structure is too brittle or complex to support future needs then the structure is not sound. If you have been in the software development industry for even short period of time you have probably seen applications that are too brittle or complex.
If the application or enterprise structures are not aligned with current business needs then the value of those structures have deteriorated. We sometimes keep a specific architecture and force-fit new business needs into it because it once was the right architecture to have or we paid a bunch of money for it. Continual restructuring of our architecture to meet today’s business needs is important.
Providing supports to our structure allows for it withstand changes in the environment such as new business needs and updates in technology. Automated tests and builds help us keep the structural integrity of our applications intact while these changes are introduced into our applications and enterprise.
Communicating the conceptual structure of a component, application, or enterprise is important because it is common for new people to work on them and for those structures to interact with other components and applications. Getting someone up to speed on a component or application involves verbal, tactile, written, and visual examples. Much of what is needed can be kept in or close to the codebase along with conversations with existing developers. It is important to understand how components and applications are currently structure conceptually so we can discuss their interactions with other components and applications. For instance, should we connect via library, SOAP, RPC, or REST?
When I think of application architecture I want to focus on just a few principles:
If the structure is not able to change as the needs of our business change then the solution will become a liability. If someone other than those who are constructing components and applications are deciding on architecture decisions then there will be important information lost in translation between the two. If we keep using the same design hammer (ie. always using 3-tier or IoC for everything) then we are not allowing the strongest solutions to emerge. The value of architecture can usually be described by the cost incurred if it is not taken care of.
Well, let the onslaught of comments commence. I have put out my ideas on architecture. There are many more out there to discuss and I am sure I will hear some of them. I will finish this entry with a quote from Martin Fowler’s “Who Needs an Architect?” article:
“I define architecture as a word we use when we want to talk about design but want to puff it up to make it sound important.”
Posted by Chris Sterling on 15 Jan 2009 | Tagged as: Acceptance Testing, Agile, Architecture, General, Leadership, Open Source, Product Owner, Scrum, Software Architecture, TDD, Travel, User Stories, XP
I’d like to invite you to join me at …
SD West 2009 Conference & Expo
March 9–13
Santa Clara Convention Center, Santa Clara, CA
http://www.SDExpo.com/
I’m pleased to announce that I’ll be teaching the following session at SD West 2009:
“Managing Software Debt: Continuing to Deliver High Value as Systems Age” on Friday, March 13th from 3:30-5:00pm
SD West is where the software development community gathers to learn about the latest business-critical technologies, network with peers, connect with innovative vendors and get inspiration from industry visionaries. The comprehensive conference program covers today’s most important topics including cloud computing, concurrent programming, dynamic languages, agile processes, security, testing and much more.
Super early bird conference discounts of up to $400 expire Friday, January 16. As a speaker, I can also offer you an additional $100 off the VIP Pass. Simply register at http://www.SDExpo.com with the code 9ESPK to get your discount.
Check out all the excellent educational opportunities SD West 2009 has to offer at http://www.SDExpo.com.
I look forward to seeing you in Santa Clara!
Posted by Chris Sterling on 03 Jan 2009 | Tagged as: Acceptance Testing, Agile, Architecture, General, Leadership, Product Owner, Scrum, Software Architecture, User Stories, XP
The thought of less documentation is appealing to many in the software industry. Reducing the specificity of our requirements can have tremendous value but some go too far. One of the big myths about Agile software development is that Agile means no more documentation. This was not the purpose of the Agile Manifesto value “Working software over comprehensive documentation”. The ideas was that only the documentation that supports the creation of working software should be created. There are many reasons for software to have documentation:
Since the idea of Agile software development is not to remove all documentation from software delivery then what is the right amount of documentation? Of course, the answer is it depends but we won’t leave it at that. This article will focus on one area of documentation and that is specifying what users want.
User stories were popularized by Mike Cohn with his book “User Stories Applied”. Many people do not read and take in all of what is provided in Mike’s book before using user stories to capture user desired functionality. Instead they may be introduced to it on the Internet, through a seminar, or in training. The appeal of user stories is they are short statements that are easy to write and fit on 1 index card. The problem, as Ron Jeffries explains it in his article “Card, Conversation, Confirmation”, is the statement written on an index card is not enough information for the development to implement without providing some clarity through conversation with someone who represents the user’s desires. So the first way that user stories “GO WILD!” is:
Stakeholders expecting that a statement written on an index card is enough for the team to implement what they desired in the software
And this can be more generalized as:
User stories inappropriately used and becoming just bad specifications
Development must have conversations with people knowledgable about the user story domain to clarify the desired functionality. And as Ron rightly describes, conversations may be supplemented with documentation when needed. This could be in the form of an user interface design, usability study, business rules, acceptance tests, and any other format necessary for the development team to implement the user story satisfactorily. During the conversation new understanding and clarity of the user story specifics will be drawn out. The next way user stories “GO WILD!” is:
Acceptance criteria (or “Confirmation” as Ron puts it) for the user story is not captured and many times forgotten during implementation
It is important that the development team and domain experts write down important nuggets of knowledge about the desired functionality. This could be as simple as writing it down as bullet points on the back of the index card or creating skeletons of acceptance test cases.
Sometimes teams come across user stories that seem either too ambiguous or potentially mammoth in size. There are some great indicators in the user story text that I have found over the years while working with them over the years. Here are a few ways that user stories “GO WILD!” along with ways to deal with them when they do:
Using conjunctions within the user story statement
Whenever I see an “and” or “or” or “if” or “until” or… well you get the picture… this is an indicator to me that more than one piece of functionality is being described. This is an opportunity to split the user story into multiple user stories usually right where the conjunction was put in.
Describing too much about “how” to implement the user story
It has been common in my experience to see business analysts or technical writers creating user stories to support a Scrum Product Owner or stakeholder group in defining their desired features. You might see phrases such as “when they click on … button”, “drop down”, and “using Excel” which are specific about the “how”. In the past, these people were asked to describe the functionality so it was clear for programmers to code and testers to verify. Now we are asking them to write user stories that describe “what” the user wants and not “how” to implement it. This is a problem since they have used the “how” to make clear requirements for the development team to implement. Agile teams are expected to take more responsibility in defining “how” the software is implemented. This includes providing their input on specifying and designing the software. Many times I have seen specification of features get simplified or enhanced through the conversation between development team and domain experts. We want this to occur more often since these interactions will increase software features delivered and improve the product for its users.
Can’t describe the value provided to the user for the desired functionality
There has been more than one time that I have asked “why are we implementing this user story” and was not provided a satisfactory answer. I may have gotten an answer like “because it is defined in the requirements document”, “because it was prioritized high”, or “because the CEO wanted it”. All of these may be good enough reasons to implement the functionality but it is less likely that the development or domain experts will implement it masterfully given the lack of understand around what the user wants to do with this functionality. Understanding how a user story fits into the bigger picture is important since it will enhance the conceptual integrity of the software overall. When software is implemented by generating many disconnected components and integrating them together into a cohesive package the software tends to become inconsistent in its usage with lots of hidden but potentially useful functionality. Ask yourself when writing a user story why would the user want this functionality. You can use the user story template:
As a [user role] I want to [do something] so that [I gain some benefit or value]
By address the “so that” clause you will provide more understanding of how the functionality fits into the larger picture. Or you may find out that it is not useful at all. In that case, you can throw it away and work on something of more value to your users.
Writing technical user stories in which the value is not understood by the Product Owner or stakeholders
Technical user stories tend to get deprioritized since they are not easily assessed for value by most non-technical people. I have found that it is essential to describe why we have new technical needs or what the cost of not addressing a system’s quality attribute such as scalability could be. I have written an earlier post on this regarding a term coined by Mike Cohn called “abuse stories”. The idea is to capture the cost of not addressing an architectural or infrastructure need by describing it from an abuser of the software’s point of view. Writing technical needs in a user story format is not necessary but may be helpful while building trust between the development team and stakeholders. Sometimes project stakeholders believe that time and money is getting wasted on technical perfection. By writing the abuse story a development team can show that these technical needs are not just wasted efforts but provide value to the software. It is hoped that over time a team can just let their Product Owner or stakeholder group that there is a technical need that must be prioritized before particular functionality should be implemented and that will be enough.
Now that you know some ways that your user stories can “GO WILD!” it is time to tame those user stories so that your software can be the best it can be. Happy story writing! And, oh yeah, happy new year 2009!
Posted by Chris Sterling on 17 Dec 2008 | Tagged as: Architecture, Distributed Computing, General, Java, Maven, Open Source, Ruby, Software Architecture
This is a bit off my usual topics on this blog but I am a heavy open source user and this article is something that I hope gets to more enterprise operations, managers and executives. I have been using and deploying production available applications using open source tools, libraries, and platforms for over 12 years now. Open source tools can do almost anything commercial products are able to do and have transformed the software industry in that time span. The list given in the article contains open source projects that I would recommend and have used in the past either directly or indirectly including *nix tools and libraries shown.
I would like to add to this listing with some of the tools I have come to use often:
This is some of the open source that I have and still use on my projects. What are your favorites that were not on the list?
Posted by Chris Sterling on 13 Dec 2008 | Tagged as: Acceptance Testing, Agile, Architecture, General, Java, Scrum, Software Architecture, TDD, XP
For multiple years now I have thrown around the name “Executable Design” to describe Test-Driven Development (TDD) and how it is used for design rather than a test-centric tool. The name itself causes problems for those who are initially introduced to the technique. As a coach I was looking for a way to introduce it without stereotyping it as extra tests inhibiting more code getting delivered.
From my readings of multiple books, articles, and blog postings along with my own experiences with TDD the content of what I am about to distill is not new. This post is entirely about explaining the technique in a way that garners interest quickly. There are multiple pieces to “Executable Design” beyond the basic process of:
These statements and the technique is the basis for practicing Executable Design but are not sufficient for describing the value and nuance of the practice. Not that I will be able to present it sufficiently in a single blog post but I want to present the basic principles.
While in a meeting with a team recently we were presented with a question I have heard often:
“Why should we use TDD?”
There are many reasons but generic reasoning alone is not sufficient. We discussed the safety net that good code coverage creates. We discussed the reason system tests do not take the place of unit tests. Then we started to touch on design and this is where it got interesting (and usually it does about this time for me). Before I can describe the rest of this discussion I want to present what lead up to this meeting.
A coach that I highly respect seemed a bit preoccupied one day when he wandered into my team’s area. I asked him what was going on and he told me that some of his issues with the current team he was coaching. He wondered why they were not consistently using TDD in their day-to-day development. The team had allowed a card saying “We do TDD” onto their Working Agreement and were not adhering to it.
I happened to know a bit about the background of this project that our company has been working on for over 2 1/2 years. There is a significant legacy codebase developed over many more years with poor design, multiple open source libraries included, and heavy logic built into relational database stored procedures. Also, just recently management on the client’s side had changed significantly and caused quite a shake up in terms of their participation and guidance of release deliverables. Yet the team was supposed to deliver on a date with certain features that were not well defined. This lead me to discuss the following situations that a coach can find their way into:
The current team situation seemed to be more like #3 above and therefore the lack of success in helping the team adopt TDD did not surprise me. Also, I started to play devil’s advocate and provide a list of reasons for this team NOT to do TDD:
This is not the full list but you get the picture. Don’t get me wrong, the list above begs to me the need for Executable Design but if the team does not have significant experience to implement it effectively it could seem overhead with little benefit to show for it.
After discussing this and more stuff that I won’t go into he told me about a couple of things that he can do to help the team. One of them was to work on minimizing the reasons for not doing Executable Design by discussing them with their ScrumMaster and actioning them on the impediments list. Some of those actions would go to upper management who get together each day and resolve impediments at an organizational level. One of the actions was to get our CTO and myself into a room with the team so they can ask the question “why should we do TDD?”.
Now we are in the room and most of the team members had been exposed to TDD through pairing sessions. Some of them had some ideas about where TDD was useful and why they thought it was not on this project. During the discussion one of the team members brought up a great discussion point:
“One of the problems with our use of TDD is that we are not using it for improving the design. If we just create unit tests to test the way the code is structured right now it will not do any good. In fact, it seems like we are wasting time putting in unit tests and system tests since they are not helping us implement new functionality faster.”
This team member had just said in the first sentence what I instinctually think when approaching a code base. The reason to do TDD is not just to create code coverage but to force design improvement as the code is being written. This is why I call TDD and its best known principles and practices of applying it Executable Design. If you are not improving the design of the application then you are not doing Executable Design. You might be just adding tests.
Some of the principles I have found to help me in applying Executable Design effectively are (and most, if not all, of these are not something I came up with):
Try these out on your own or with your team and see how they work for you. Modify as necessary and always look for improvements. There are many thought leaders in the Agile community that have written down important principles that may work for you and your team.
And finally, now that I have filled an entire blog post with “Executable Design” what do people think about the name? It has worked for me in the past to explain the basic nature of TDD so I will use it either way unless others have better names that I can steal?