Chris Sterling’s Blog

Earlier this year I did a presentation on Executable Specficiations for AgilePalooza conference. There is information about working with legacy code, commercial off-the-shelf (COTS) systems, and Acceptance Test-Driven Development (ATDD) using automated acceptance testing tools. Also, the presentation lists types of automated acceptance testing tools out there along with actual names of tools and what they are best used for on projects. Hope it is interesting to you.

Comments Off on Executable Specifications - Presentation from AgilePalooza

During an interview at the Better Software conference this week, I mentioned that I thought maintaining a single list of work prioritized by the business was important for our industry to improve. The following text is an excerpt, in first draft form, from chapter 2, “Architecture Integrity”, of my upcoming book “Architecture in an Agile Organization” that is relevant to this topic.

---

In Scrum, the Product Backlog is a list of desired features prioritized by the Product Owner. Prioritization is done by the Product Owner to optimize value of the software delivered to the users. Instead of prioritizing within levels of importance such as high, medium, and low the Product Owner must prioritize in “stack-rank” order. This means that no Product Backlog item has the same priority as another item. This form of prioritization gives clear directions to development teams about what the most important feature is next to work on.

A healthy Product Backlog contains not only new features but also technical requirements to support the next Product Backlog items in priority. The development team is expected to review the Product Backlog frequently to provide cost estimates for Product Backlog items to help the Product Owner prioritize based on both cost and benefit. During the development team’s review they will suggest Product Backlog items that are missing from a technical point of view in the product definition. These technical Product Backlog items will be discussed with the Product Owner so they are able to prioritize them effectively. It is essential that the development team explain why each technical Product Backlog item is important to support other upcoming items. If this is not expressed the Product Owner could be inclined to demote those technical Product Backlog items because the value of them is not understood enough.

When issues are found in the software they usually come in three varieties:

• Enhancement – a suggestion for improving the current state of the software from a user’s point of view
• Defect – a system fault that occurs while using the software
• Fire – a system fault that is inhibiting significant usage scenarios in the software and must be fixed immediately

Enhancements are requests for improvements to the software based on the perspective of the user. These improvements should be prioritized based on their value against all other Product Backlog items. An enhancement and defect issues are hard to distinguish from one another at times but that is fine. Defects are system faults that users are able to workaround but should not be happening from a user experience and/or technical perspective. Since we are able to work around these defects and the development team is able to provide a fix in the next iteration they are placed into the Product Backlog and prioritized against all other Product Backlog items. Fires are defects that cannot be worked around and cause serious problems for users of the software. In healthy software applications, fires should be a rare occurrence. It is important that a fix is implemented right away and may interrupt the work getting done in the current iteration.

Figure: This flow diagram describes how an identified issue is triaged in an iterative and incremental approach such as Scrum. If the issue is an enhancement or defect that can be worked around for now it goes into the Product Backlog and prioritized based on its value. If the issue is a “fire” or defect that cannot be worked around then it must be fixed immediately. The development will put the fire into their current list of work in the Sprint Backlog and figure out how it effects their commitment to deliver on their current iteration.

Defects are captured in bug databases in many organizations. Teams using the Scrum process to manage their software development find themsves in a predicament. Do we continue working from two lists, the Product Backlog of desired features and bug database? Or do we incorporate items from the bug database into the Product Backlog? I suggest all teams should incorporate defects into the Product Backlog. There are multiple reasons to do this:

• Development team does not make priority decisions for the business stakeholders. If the development team decides to work on defects rather than desired features then they are deciding the defects are higher priority than the desired features without including the Product Owner in the decision-making process.
• Minimizes the amount of hidden work done by the development team. By providing visibility into the defects, the development team is being transparent about the state of the software they are working on.
• Provides the Product Owner with an opportunity to prioritize the defect lower. The Product Owner could decide the defect is not important enough to fix at this time.
• Allows development team and Product Owner to design a solution for dealing with defects. From time to time the defect provides insight into a design flaw that the Product Owner would like to improve.

Ultimately, the development is able to maintain alignment with business priorities for the software. Priorities are decided based upon the full reality of the software, new features and defects included.

A single work queue, such as the Product Backlog, provides visibility to the development team about current expectations about upcoming work. The Product Owner should share the Product Backlog with the development team and discuss the direction of the product beyond the next iteration. As the development team gains knowledge about the direction of the product they will be able to provide input into the Product Backlog. This visibility is important so Product Backlog items do not surprise the development team when it is too late to prepare a proper solution.

4 Comments »

AgilePalooza is a one day Agile conference on Friday May 29th at the San Francisco State University downtown campus. There will be two tracks: Learning Agility and Advancing Agility.

“Learning Agility” will be presentation style whereas “Advancing Agility” will use the open space format.

Speakers include David Hussman (DevJam), Chris Sterling (SolutionsIQ), Luke Hohmann (Enthiosys), Lee Henson (VersionOne Services) with special guests Ainsley Nies (open space co-facilitator) and Ian Culling (VersionOne CTO). When not presenting for the “Learning Agility” track speakers will participate in the open space.

Space is limited and the cost is low so please register soon if you would like to attend. For more information or to register please visit www.AgilePalooza.com.

Where?

San Francisco State University Downtown Campus, 835 Market Street, San Francisco. (Powell street BART station in the Westfield Center) - Map/Directions

When?

9am-5pm (check-in and continental breakfast starts at 8am)

Cost?

$69

Register here: www.AgilePalooza.com

Comments Off on AgilePalooza in San Francisco May 29th

I have been using a tool for some time with clients and teams to find out what software quality attributes the product development team should focus on in the project. ISO standard 9621 describes the quality attributes found in software. The following image shows the 6 categories and specific attributes contained within them.

Before I knew about this standard I would discuss how different quality attributes are in conflict with each other. For instance, if we write code that is focused on performance we lose some maintainability. It will be more difficult to read code focused on performance because readability will be sacrificed to some extent.

Now that I’ve known about ISO 9621 for many years, I made a simple spreadsheet tool to interview Product Owners, stakeholders, and software development team members with. Here is the Excel spreadsheet form of this tool:

Software Quality Attributes Rating Tool

Although a bunch of smart people have come up with ISO 9621, I found that modifying the software quality attributes rated in the tool worked more effectively with interviewees. This tool is not to decide what software attributes will be present in the software product getting developed. It is used to identify which software quality factors the team should put more emphasis on when making trade-off decisions during the project. Here is a list of the software quality attributes used in the tool:

• Suitability - Functionality is suitable to all end users
• Interoperability - Functionality interoperates with other systems easily
• Compliance - Functionality is compliant with applicable regulatory guidelines
• Security - System is secure: confidentiality, integrity, availability, accountability and assurance
• Maturity - System components are proven stable by others
• Fault Tolerance - System continues operating properly in the event of failure by one or more of its components
• Recoverability - System recovers from failures in surrounding environment
• Understandability - Able to use system with little training
• Learnability - Supports learning of system functionality with little external interfacing
• Operability - Ability to keep a system in a functioning and operating condition
• Performance - Perceived response is immediate
• Scalability - Able to handle increased usage on the appropriate amount of resources
• Analyzability - Ability to figure out how the system functions
• Changeability - Ability to change the system components to meet new business needs
• Testability - Ability to create repeatable and specific tests of the system and potential for some to be automated
• Adaptability - Ability to change out system component functionality quickly
• Installability - Ease of installation and reinstallation
• Conformance - Conformance to industry and operational standards
• Replaceability - Ability to replace system in the future

Using the tool is quite simple:

1. Emphasis Ranking: Have the Product Owner score each of the software quality factors from 1-5 (1 being less applicable and 5 being more applicable) in terms of the software product.
2. Business Stakeholders identify 3 Must Haves - Ask the business stakeholders what are the 3 software quality factors they want most and stack rank them from 1 to 3
3. Software Development Team 3 Must Haves - Ask the software development team what are the 3 software quality factors they want most and stack rank them from 1 to 3
4. Capture Notes about Decisions - Use the notes column to capture any specific decisions about the software quality factors that would help the entire software product team come to consensus.
5. Discuss Must Haves across Groups - Get the Product Owner, business stakeholders, and software development team members together to discuss and come to consensus on which software quality attributes they will focus on for this project.

Please let me know how you use this tool. There have been many teams that have taken up this tool and found it to be helpful for communicating what is most important from a software quality attibutes point of view in the project. Hope it does the same for you.

4 Comments »

I have uploaded the talk I did at SD West 2009 on Yahoo! Video and here it is:

Comments Off on My Talk @ SD West 2009 on “Managing Software Debt”

The hopefully-not-anticlimatic second event of the still-pretty-new BeyondAgile is happening on Thursday:

“Agile Challenges Clinic/Swap Meet”
Thursday, 26 March 2009
6:30 to 8:30 p.m.
Locations on the Eastside and Seattle!
Read on for agenda and location information, or visit our Google Group at
http://groups.google.com/group/beyondagile

The Blurb
At the first event, everyone (over forty people) created a backlog of ideas, suggestions, and work items. Among them were two suggestions that gave rise to this event’s focus: “a bring-a-problem” session where everyone helps everyone with their challenges. We’ve extended the idea of a clinic, where you bring a problem to an expert and get help, to a swap meet (or potluck), were everyone brings something and gets something. So come if you think you’re brimming over with answers and expertise, and come if you’re wanting other perspectives or advice on how to tangle with the problems that bedevil your days.

We’ll also form a program team, a group that’ll work proactively to plan interesting and exciting events in sufficient time for the blurb-writer–that’s me–to write and distribute the blurb a few weeks before the event. If you’re wondering who’d be good at doing that, go look in a mirror—it’s you we need!

Second Event Agenda

• Welcome and Announcements (10 min.)
• Formation of Program Team (10 min.)
• “Agile Clinic/Swap Meet (85 min.)
• Giveaway Drawing (5 min.)
• Meeting Retrospective (10 min.)
• Socializing and Breakout/Breakup/Beer

Event Locations

Eastside: SolutionsIQ
First Floor Training Center
10785 Willows Road NE, Suite 250
Redmond, WA
NOTE: directions and picture of building at http://www.solutionsiq.com/about-us/contact-us.php?Look for the tent signs and the blue BeyondAgile logo

Seattle: Amdocs Digital Commerce Division (formerly QPass)
Greece Room
2211 Elliott Ave Suite 400
Seattle, WA

Questions?
Contact us at [email protected], or visit our Google Group, or contribute to the nascent Wiki at beyondagile.org.

Why come?
Here’s your chance to be in on the beginning of an exciting new collaboration of the Puget-Sound area (and beyond!) agile-interested. And if that’s not enough, we’ll hold a drawing to win exciting and valuable prizes!

What’s BeyondAgile?
It’s the combination and follow-on to a number of the agile-oriented user and interest groups that have operated in the Puget Sound area. Representatives of the Seattle XP User Group and Seattle Scrum came together in December to try and combine the efforts of all of us who care about and use agile methods.

Why does BeyondAgile exist?
o Find best practices among all agile processes
o Build a bigger agile community
o Take advantage of overlaps between existing groups
o Expand beyond meetings and provide a place to collaborate
o Align software development and business
o Explore cutting-edge ideas and techniques

What is BeyondAgile like?
That, in large part, is up to you. A primary reason for consolidating our efforts is to broaden our base of support, capability, and leadership. We envision a more active, multi-faceted organization that does more than just host talking heads. We’ll gather at least once a month on the 4th Thursday of the month, and probably more often, once you figure out what other events you’d like.

Where do BeyondAgile events happen?
Our goal is to have many answers to this. As a result, we’ve worked to remove the impediment offered by bridges and commuting: for our monthly meetings, we hold our events in both Eastside and Seattle locations, through the semi-magic of videocasting. We’re experimenting: we’ve thought of trying to alternate the “real” physical meeting between sides of the lake. At this point, we’re only at the stage of using a semi-okayish video link between the two locations, and try *very* hard to make the meeting balanced between locations. That’s harder than it seems; come and help us work it out! We’re still dreaming of enabling people anywhere to attend through streaming video, even after the meeting’s already happened, but we need more knowledge, resources, and volunteers before that’s going to happen.

I have question for you.
Great! Visit the Google Group (BeyondAgile) or send a message to [email protected]

Comments Off on It’s BeyondAgile: people making software that works

Talk about a loaded term. Even the term itself, “architecture”, when used in the Agile community can start a heated discussion. When I was coordinator of the International Association of Software Architects Puget Sound chapter, the discussions about “what is architecture” caused passionate debate. I am sure this entry will get some interesting comments, as well.

I don’t believe that this entry will solve the question but I hope to at least give some focus for teams who are dealing with the structural integrity of their applications. First off, a definition of “architecture”:

architecture: noun - “the art or practice of designing and constructing…” or “the complex or carefully designed structure of something” or “the conceptual structure and logical organization of a computer or computer-based system”

Each of these definitions interests me but to put them to practice is quite difficult. They are left to interpretation about what is design, construction, structure, conceptual, and logical. Earlier interpretations of this term interpreted these to mean the high-level structure, architectural style, and finally the stuff that is too expensive to get wrong.

It seems to me that my interpretation is a bit different. Working with tools, practices, and processes over the past 15 years has guided me to a different conclusion. I believe that architecture is how it is SAID:

Structure - how the pieces (components) create a solution (application)
Alignment - the degree to which the application or enterprise structures align to current business objectives and strategy
Integrity - the components that provide stability to the structure (application or enterprise) (ie. automated tests and builds, service-level agreements, network infrastructure, etc…)
Design - a way to conceptually communicate the planned or implemented structure of a component, application, or enterprise (ie. system of names, XP practice of Metaphor, information radiators, etc…)

Using these as the basic building blocks for teams to focus their efforts can be helpful.

The structure is the reality of the solution’s construction. If the structure is too brittle or complex to support future needs then the structure is not sound. If you have been in the software development industry for even short period of time you have probably seen applications that are too brittle or complex.

If the application or enterprise structures are not aligned with current business needs then the value of those structures have deteriorated. We sometimes keep a specific architecture and force-fit new business needs into it because it once was the right architecture to have or we paid a bunch of money for it. Continual restructuring of our architecture to meet today’s business needs is important.

Providing supports to our structure allows for it withstand changes in the environment such as new business needs and updates in technology. Automated tests and builds help us keep the structural integrity of our applications intact while these changes are introduced into our applications and enterprise.

Communicating the conceptual structure of a component, application, or enterprise is important because it is common for new people to work on them and for those structures to interact with other components and applications. Getting someone up to speed on a component or application involves verbal, tactile, written, and visual examples. Much of what is needed can be kept in or close to the codebase along with conversations with existing developers. It is important to understand how components and applications are currently structure conceptually so we can discuss their interactions with other components and applications. For instance, should we connect via library, SOAP, RPC, or REST?

When I think of application architecture I want to focus on just a few principles:

• Application architecture is about changeability, not longevity
• Application architecture decisions should be made closest to where they are implemented as possible
• Application architecture design is not only about our ability to design a solution but also knowing what components, applications, and solutions already exist
• The value for change in a component’s, application’s, or enterprise’s architecture is directly proportional to the cost of not addressing

If the structure is not able to change as the needs of our business change then the solution will become a liability. If someone other than those who are constructing components and applications are deciding on architecture decisions then there will be important information lost in translation between the two. If we keep using the same design hammer (ie. always using 3-tier or IoC for everything) then we are not allowing the strongest solutions to emerge. The value of architecture can usually be described by the cost incurred if it is not taken care of.

Well, let the onslaught of comments commence. I have put out my ideas on architecture. There are many more out there to discuss and I am sure I will hear some of them. I will finish this entry with a quote from Martin Fowler’s “Who Needs an Architect?” article:

“I define architecture as a word we use when we want to talk about design but want to puff it up to make it sound important.”

1 Comment »

I’d like to invite you to join me at …

SD West 2009 Conference & Expo
March 9–13
Santa Clara Convention Center, Santa Clara, CA
http://www.SDExpo.com/

I’m pleased to announce that I’ll be teaching the following session at SD West 2009:

“Managing Software Debt: Continuing to Deliver High Value as Systems Age” on Friday, March 13th from 3:30-5:00pm

SD West is where the software development community gathers to learn about the latest business-critical technologies, network with peers, connect with innovative vendors and get inspiration from industry visionaries. The comprehensive conference program covers today’s most important topics including cloud computing, concurrent programming, dynamic languages, agile processes, security, testing and much more.

Super early bird conference discounts of up to $400 expire Friday, January 16. As a speaker, I can also offer you an additional $100 off the VIP Pass. Simply register at http://www.SDExpo.com with the code 9ESPK to get your discount.

Check out all the excellent educational opportunities SD West 2009 has to offer at http://www.SDExpo.com.

I look forward to seeing you in Santa Clara!

2 Comments »

The thought of less documentation is appealing to many in the software industry. Reducing the specificity of our requirements can have tremendous value but some go too far. One of the big myths about Agile software development is that Agile means no more documentation. This was not the purpose of the Agile Manifesto value “Working software over comprehensive documentation”. The ideas was that only the documentation that supports the creation of working software should be created. There are many reasons for software to have documentation:

• Maintainability
• User manuals
• Training
• Communication
• etc…

Since the idea of Agile software development is not to remove all documentation from software delivery then what is the right amount of documentation? Of course, the answer is it depends but we won’t leave it at that. This article will focus on one area of documentation and that is specifying what users want.

User stories were popularized by Mike Cohn with his book “User Stories Applied”. Many people do not read and take in all of what is provided in Mike’s book before using user stories to capture user desired functionality. Instead they may be introduced to it on the Internet, through a seminar, or in training. The appeal of user stories is they are short statements that are easy to write and fit on 1 index card. The problem, as Ron Jeffries explains it in his article “Card, Conversation, Confirmation”, is the statement written on an index card is not enough information for the development to implement without providing some clarity through conversation with someone who represents the user’s desires. So the first way that user stories “GO WILD!” is:

Stakeholders expecting that a statement written on an index card is enough for the team to implement what they desired in the software

And this can be more generalized as:

User stories inappropriately used and becoming just bad specifications

Development must have conversations with people knowledgable about the user story domain to clarify the desired functionality. And as Ron rightly describes, conversations may be supplemented with documentation when needed. This could be in the form of an user interface design, usability study, business rules, acceptance tests, and any other format necessary for the development team to implement the user story satisfactorily. During the conversation new understanding and clarity of the user story specifics will be drawn out. The next way user stories “GO WILD!” is:

Acceptance criteria (or “Confirmation” as Ron puts it) for the user story is not captured and many times forgotten during implementation

It is important that the development team and domain experts write down important nuggets of knowledge about the desired functionality. This could be as simple as writing it down as bullet points on the back of the index card or creating skeletons of acceptance test cases.

Sometimes teams come across user stories that seem either too ambiguous or potentially mammoth in size. There are some great indicators in the user story text that I have found over the years while working with them over the years. Here are a few ways that user stories “GO WILD!” along with ways to deal with them when they do:

Using conjunctions within the user story statement

Whenever I see an “and” or “or” or “if” or “until” or… well you get the picture… this is an indicator to me that more than one piece of functionality is being described. This is an opportunity to split the user story into multiple user stories usually right where the conjunction was put in.

Describing too much about “how” to implement the user story

It has been common in my experience to see business analysts or technical writers creating user stories to support a Scrum Product Owner or stakeholder group in defining their desired features. You might see phrases such as “when they click on … button”, “drop down”, and “using Excel” which are specific about the “how”. In the past, these people were asked to describe the functionality so it was clear for programmers to code and testers to verify. Now we are asking them to write user stories that describe “what” the user wants and not “how” to implement it. This is a problem since they have used the “how” to make clear requirements for the development team to implement. Agile teams are expected to take more responsibility in defining “how” the software is implemented. This includes providing their input on specifying and designing the software. Many times I have seen specification of features get simplified or enhanced through the conversation between development team and domain experts. We want this to occur more often since these interactions will increase software features delivered and improve the product for its users.

Can’t describe the value provided to the user for the desired functionality

There has been more than one time that I have asked “why are we implementing this user story” and was not provided a satisfactory answer. I may have gotten an answer like “because it is defined in the requirements document”, “because it was prioritized high”, or “because the CEO wanted it”. All of these may be good enough reasons to implement the functionality but it is less likely that the development or domain experts will implement it masterfully given the lack of understand around what the user wants to do with this functionality. Understanding how a user story fits into the bigger picture is important since it will enhance the conceptual integrity of the software overall. When software is implemented by generating many disconnected components and integrating them together into a cohesive package the software tends to become inconsistent in its usage with lots of hidden but potentially useful functionality. Ask yourself when writing a user story why would the user want this functionality. You can use the user story template:

As a [user role] I want to [do something] so that [I gain some benefit or value]

By address the “so that” clause you will provide more understanding of how the functionality fits into the larger picture. Or you may find out that it is not useful at all. In that case, you can throw it away and work on something of more value to your users.

Writing technical user stories in which the value is not understood by the Product Owner or stakeholders

Technical user stories tend to get deprioritized since they are not easily assessed for value by most non-technical people. I have found that it is essential to describe why we have new technical needs or what the cost of not addressing a system’s quality attribute such as scalability could be. I have written an earlier post on this regarding a term coined by Mike Cohn called “abuse stories”. The idea is to capture the cost of not addressing an architectural or infrastructure need by describing it from an abuser of the software’s point of view. Writing technical needs in a user story format is not necessary but may be helpful while building trust between the development team and stakeholders. Sometimes project stakeholders believe that time and money is getting wasted on technical perfection. By writing the abuse story a development team can show that these technical needs are not just wasted efforts but provide value to the software. It is hoped that over time a team can just let their Product Owner or stakeholder group that there is a technical need that must be prioritized before particular functionality should be implemented and that will be enough.

Now that you know some ways that your user stories can “GO WILD!” it is time to tame those user stories so that your software can be the best it can be. Happy story writing! And, oh yeah, happy new year 2009!

4 Comments »

This is a bit off my usual topics on this blog but I am a heavy open source user and this article is something that I hope gets to more enterprise operations, managers and executives. I have been using and deploying production available applications using open source tools, libraries, and platforms for over 12 years now. Open source tools can do almost anything commercial products are able to do and have transformed the software industry in that time span. The list given in the article contains open source projects that I would recommend and have used in the past either directly or indirectly including *nix tools and libraries shown.

I would like to add to this listing with some of the tools I have come to use often:

• Maven 2.x+ (http://maven.apache.org/)
• JBoss (http://www.jboss.org/)
• Rio/Jini/Apache River (http://incubator.apache.org/river/RIVER/index.html)
• Apache Commons (http://commons.apache.org/)
• Subversion (http://subversion.tigris.org/)
• Apache Web Server (http://httpd.apache.org/)
• Bouncy Castle (http://www.bouncycastle.org/)
• Time and Money (http://timeandmoney.sourceforge.net/)
• Spring Framework (http://www.springframework.org/)
• Hadoop (http://hadoop.apache.org/)
• Ruby on Rails (http://www.rubyonrails.org/)

This is some of the open source that I have and still use on my projects. What are your favorites that were not on the list?

3 Comments »