Agile Software Qualities

“People Don’t Shop for Organizational Change”

2011-09-28T13:28:00.000-04:00

I was talking about coaching experiences yesterday with Skip Angel. I said that I try to get management who feel they want to implement Agile practices to understand that, before too long, they will need to face organizational, not just team practice, changes. At one point in our discussion, Skip said the phrase that is the title of this blog and he is quite right. When management starts to look for improvements in their product development efforts, they usually do not put out RFPs for organizational change. Someone normally decides, based on peer discussion or reading, that some method, technique, practice, etc. seems to be working for some other folks and that it might be what they need, so they ask for something along those lines. Something like Agile, for example.

In this regard, Agile “transformations” are no different than the organizational efforts in the 80s and 90s to implement ISO 9001, CMM, ITIL, COBIT, etc. That is, small changes at individual or team levels are possible, but the real benefit of pursuing any of these is to look at their impact organizationally. Even when companies do this, there seems to be the feeling that the desired improvement will occur with minimal impact, i.e., with little actual change required.

This usually results in attempts to immediately “tailor” the chosen approach and produce a “hybrid” version of what has been selected. The reasons for this are varied, but seem to boil down to organizational (un)readiness for the change the approach asks be pursued when it expands beyond team/functional area impacts. This, in turn, is often manifested by team level training that focuses on practices and techniques without conveying what I believe to be an understanding of the values and principles behind the practices. The resulting tailoring and hybrid practices can then miss the point of the originally recommended practices, often losing the impact intended. At some point, people begin to wonder why the desired improvements are not happening or happening fast enough or at the level of impact desired/expected.

I think one reason organizations ending up in this situation is that they, indeed, didn’t start out “shopping for organizational change.”

The Agile Manifesto as "Immutable, Sacred Text"*

2010-11-27T11:18:00.001-05:00

Periodically, over the past 6-7 years, the issue of whether the Agile Manifesto** requires changing comes up and a variety of points are made suggesting/justifying why it seems this is needed. Recently, some of the new and repeated points have been:

Surely learning has taken place in the almost 10 years since the Manifesto was created which would influence a different set of ideas to be stated now than then.
The work being done by people before the Manifesto existed wasn’t about “agility” but was about more effective, productive software development.
By the very nature of Agile ideas, we should be prepared to change the Manifesto and not keep it the same since that was just a point in time perception and both perceptions and needs change.
Most of the ideas in the Manifesto can be found elsewhere, especially in Lean ideas, so they should not be regarded so permanently.
The Value statements are so easy to agree with that they are, essentially, “content-free.”
The Principles are redundant in many aspects (indeed one basically repeats a Manifesto Value) and could be reduced to a much smaller set without losing any meaning/effect.
Misunderstanding/misuse of the Manifesto (as stated) is now an impediment to new people learning about Agile ideas so we should move away from it as a focal point.

And the list can go on and has at various times.

There are, of course, lists of actual suggestions for changes/additions to the Manifesto which are even longer and, in themselves, certainly contain useful, important ideas. One just has to look at the early material on each method that was around at the turn of the century to see how many values, principles, ideas, etc. existed which are not explicitly mentioned in the Manifesto. So it was even obvious back in 2001 that the Manifesto did not say everything one could be expected (to need) to know on the subject.

Among others, Ron Jeffries has said that the Manifesto is a point in time statement of belief by a specific group of people. After years of experience and interaction with one another, they came to feel that they wanted to say something about ideas they shared which were, as suggested above, just a subset of the ideas they held.

I don’t believe the people at the Snowbird meeting where the Manifesto was created thought it was some document that should be amended, extended and always be the authority for thinking on the subject of better software development. I do believe they felt the ideas were a solid set of guidance, an effective starting point for changing how software development was being done, for the most part, up to that time. And, if groups/organizations pursue a new approach to development, the Manifesto’s elements can be looked upon as useful touchstones for comparing chosen techniques and practices to ideas those people in 2001 felt held important guidance for doing so.

So when calls to change (or even abandon) the Manifesto are made, while I am interested to hear what people would offer as changes, additions or replacements to/for the Manifesto, I have not seen anything that, to me, materially alters what is there.*** Much of the impetus to change the Manifesto sometimes sounds as if, to get other ideas accepted, the Manifesto must be reduced in importance in some way or shown to need “fixing.” Perhaps the strong attraction to the Manifesto just shows that the ideas can ring very true to many people.

If the fear is that latching onto the Manifesto (and misunderstanding it) holds people back and prevents new learning from taking place, perhaps that is our problem to solve, not by trying to change the Manifesto in some way, but in more effectively presenting and explaining it to those new to it while extending the knowledge that can be associated with it. I do not believe we can blame the Manifesto for people not “getting it” because it seems to me that this happens to every idea once it breaks out into the mainstream of awareness.

If something has any validity, it will attract people, but they will surely try to adapt it to what they feel is their need. Much of that adaptation is always a divergence from what the original ideas intended. I don’t think it is possible to expect this not to occur unless one produces a document as ponderous as much formal legislation, standards, etc. material. Of course, even when you try to do that, the interpretation is still going to be there, sometimes worse than before you started.

So, by all means don’t stop at the Manifesto, but don’t assume it is somehow the problem or the impediment to growth.

[* The title is derived from a Tweet by Jason Yip.]

[** When I say “Manifesto,” I include both its explicit Values and the associated 12 Principles that followed unless a distinction is otherwise made.]

[*** Efforts such as the Manifesto for Software Craftsmanship and the APLN’s Declaration of Interdependence seem to me to be good examples of making the point about other important ideas without suggestion the Agile Manifesto’s are now flawed or necessarily passé.]

Letting Them Figure It Out

2010-11-26T07:18:00.001-05:00

A few days ago, there was a brief exchange on Twitter about the Scrum advice to let teams figure things out for themselves where there isn’t explicit Scrum guidance on a topic. Since there isn’t explicit Scrum guidance about a lot of things teams encounter that leaves a lot of “figuring out” for teams to do, including adopting practices from other Agile approaches like XP, Crystal, DSDM, etc.

There seemed to be an initial implication that leaving teams to do this could be wasteful, perhaps even detrimental, given the assumption that they really don’t have a good answer for their situation and might not, by consensus, come up with one. I suggested that “guiding" teams to figure things out seemed to me a useful way for a coach/ScrumMaster to fulfill that role than just "letting" the team figure it out. If the coach/SM sees a possibly useful direction for the team to go in, guiding the team that way through the “right” questions would be an appropriate approach.

Given that, I said I didn’t mind teams figuring things out for themselves, just not doing so by expecting them to reinvent all knowledge to accomplish that. Thus, if the coach/SM knows of techniques/practices that have worked for other teams, suggesting those ideas and asking the team what they think about them still seemed to be allowing teams to make their own decisions while still being helpful in a non-directive manner.

So that’s gotten me thinking about the subject of doing this sort of thing as a coach/SM as it has been my way of performing these roles. Indeed, it has been my way of teaching people about things for decades, even before I was involved in software development. I’ve always associated this with employing the Socratic Method, i.e.,

A pedagogical technique in which a teacher does not give information directly but instead asks a series of questions, with the result that the student comes either to the desired knowledge by answering the questions or to a deeper awareness of the limits of knowledge. [From http://www.answers.com/topic/socratic-method.]

I don’t go as far as “Socratic irony” with this. That is, I don’t feign/pose ignorance. Indeed, after using the Socratic approach a bit, I’m fairly certain the students/teams I’ve been engaged with don’t believe that’s the case.

But this raises the following questions:

How much information about techniques and practices might a team need before they start working together so that there are not too many open issues to “figure out”?
Indeed, how many open issues are “too many” for them?
How many times or how seriously should a coach/SM allow a team to “fail” in such decision-making (since allowing teams to fail, a bit, is typical advice given to coaches/SMs/managers in an Agile context)?

On the latter point, one thing I can say is that learning through even a little failure can be stressful when teams are up against deadlines. I also think, given such deadlines, both teams and people around them seem much less inclined to accept the “fail a little” approach.

Now the kind of “deadlines” I am thinking of are not the kind the team commits to themselves through reasonable estimation and planning. I’m thinking of the all too common situation where an Agile approach is taken in the context of already fixed schedules and, more or less, fixed functionality expectations.

Not only is this kind of situation inherently a major impediment to learning through some failure, even though a Socratic approach, it is an impediment to the whole idea of teams “figuring it out” for themselves whether serious failure would be involved or not.

With fixed schedule and functionality (as well as fixed staffing) usually comes 100% “resource utilization” assumptions as the way those outside the team(s) assume planning and estimation will be done.

I believe every team I have ever been involve with, Agile-based or not, has been somewhat astonished at my suggestions that

they only presume a 6, not 8, hour productive day in their estimation, so a 30, not 40, hour week;
within this 30 hour week they commit to less than 100% of that in functionality terms, especially when they are new to this approach;
they do not presume to commit overtime at the outset as a way to deal with the fixed schedule and functionality.

The first point is something I learned doing estimation over 30 years ago. The second has been influenced by my Agile training/experiences. The last is also a nearly 30 year old belief based on watching teams do this, implicitly, if not explicitly, and coming up with unreasonable commitments as a result.

So, it seems to me that trying to let teams figure things out for themselves isn’t something that will be successfully achieved outside the context of having some other things in place that give the teams the “space” in which to believe they can/should expect to do this. Otherwise, they are going to look for someone else to solve the problem (and hang the result on) given their very natural perception that they don’t have the time to take for much team reflection and a Socratic type of guidance.

Deadlines, But No Tempo

2010-11-04T10:43:00.000-04:00

I have only recently started to present the idea I comment upon below when I teach classes and otherwise discuss development work with people. I’m not sure why it took me so long to recognize this comparison between the traditional phased-sequential (i.e., waterfall) and Agile approaches to development. But it seems an important point.

Traditional phased-sequential approaches to development have many deadlines/milestones as targets for work, but specify no expected tempo for getting work done between those milestones. Agile methods show considerable concern for establishing a work tempo which replaces most of the date-driven milestones in traditional approaches.

To me, the focus on milestones is less desirable than a predictable tempo because distance between milestones will vary from project to project based on the overall size of the work to be done by the final project delivery milestone. Agile’s main milestone is the iteration which is always (supposed to be) the same length. So teams get used to producing some defined result, regularly, at this tempo.

In traditional milestone-driven projects, the weekly/bi-weekly status meeting might be viewed as establishing some sort of tempo, but what is expected to be done between such events is not a delivery of value/product. So I do not view status meetings as having the same tempo impact as iterations.

A predictable tempo seems to me to encourage more of a team approach to work. Milestones allow more individual work expectations using milestones as coordination points. Of course, the time between such coordination points allows a lot of work to be done before coordination occurs and, as noted above, the time between such points will not necessarily be consistent within or between projects. I believe the inconsistency, extended time periods, and individual task focus work against effective coordination.

At least, it places greater burden on individuals to establish coordination behavior ad hoc. On the one hand, coordination is expected to occur. On the other, individuals view any work tempo to be personal. Coordination efforts in this context seem more difficult because they are not collaboration within an agreed upon tempo, but negotiation between individually accustomed to tempos. In such negotiation, some people are almost guaranteed to feel they have “lost”. (And, truthfully, this may be why some people find moving to an Agile approach problematic. The tempo of an iteration is not their natural one and, though it is not the other team members who have imposed the iteration concept on them, someone has.)

This leads me to believe that some talk about individual work styles and tempos is an important part of the “forming” stage for a team. Everyone much fit into the iteration length’s tempo. Discussing how to support and help one another do that, then, seems to me to be an important aspect of team-building in an Agile context.

I don’t hear people talk about this much, but I do believe I see consequences of it in daily work discussions, ability to deliver on commitments, etc. Because there are consequences for not overtly recognizing this, I think it should be discussed when teams are getting started. And certainly think the concept of milestones vs tempo is important for an organization to discuss when deciding to embark on an Agile approach to their development.

Last Quotes From Twitter

2010-11-04T08:05:00.000-04:00

Not because folks aren't saying great things still, but I think I've run this idea into the ground and doing this last, very short, set will be a way to start my blog up again after several months of regrettable inactivity.

@FunnyOneLiners - A mighty oak is the result of a nut that held it's ground.

Abraham Williams - Sometimes I write a letter on paper with a pen then burn it laughing about how Google must be crying over information it will never index.

Albert Einstein (via Jason Yip) - The formulation of a problem is often more essential than its solution...

Albrecht & Zemke (via @ASQ) - People do not just buy things, they also buy expectations.

Bob Marshall - Agile successes - or not; echoes of Rashomon?

Dale Emery – The values (then the principles, then the practices) stand at the brink and wave goodbye as the name moves on. Michael Bolton - I said a couple years back that Agile hasn't crossed the chasm; it's mostly fallen in. But the name made it across.

David Joyce - Toyota manager induction doesn’t take place in a room but instead 12 weeks are spent in the work being coached on how to solve problems.

G. K. Chesterton (via Jason Yip) - There are two ways to get enough: One is to continue to accumulate more and more. The other is to desire less.

Glyn Lumley - The manner in which employees treat customers is determined, in part, by the norms for handling internal conflict and frustration.

James Bach - If you don't have a clear goal in life-- then watch out!-- you might wander and learn cool things that no one anticipated. Payson Hall (replied with this from Tolkien) - Not all those who wander are lost.

Jay Arthur - At one time, customers wanted you to be better, faster and cheaper. Now, they want everything free, perfect and now.

Michael James (actually from a mailing list talking about Scrum and Kanban) - I'm not sure it matters too much whether someone edits with vi or emacs, as the real challenge is what's in the file.

Gad...Yet More Quotes from Twitter

2010-06-28T16:39:00.001-04:00

Been too long since I posted anything. But, fortunately, I have been very busy with work and still digging out from under a move to a much smaller place. I said it a few months ago, but I hope to turn posting to this blog into a more regular habit.
--------------------------------------------------------------------------------------

@funnyoneliners - When asked "What would you bring with you to a deserted island", how come no one ever replies, "A boat."?

Alan Cooper - Art always contains craft, but not vice versa.

Bob Martin - Any meaningful certification must be hard to acquire with a significant chance of failure. I wonder what the CSD failure rate will be...

Bob Martin - Once any word, like "agile" or "Object" or "Structured" becomes synonymous with "Good" it has lost all meaning.

Bob Martin - Second law of meetings: Meetings are for decisions. If you don't know what's to be decided, don't go. If no decision is proposed, leave.

Brad Burt - In the 'Art of the Effect' in magic, less is almost always MORE. It has to do with purity and thus the ease of a watchers acquisition of what has happened. Amazement is knowledge of a kind.

Brian Marick - The original sin of Agile is that small nimble companies don't pay consultants enough, so all the attention is to the lumbering dinosaurs. James Shore - Such truth in @marick's tweet. Explains "Agile" plan tools, watering down of Agile, desire to "scale" mediocrity. Not only reason, natch. [Brian noted “original sin” term not right one. Other noted consultant focus wasn’t real point, it was “dinosaur” focus.]

Brian Marick (at goosgaggle ) - design happens in writing the test - implementation is just stenography.

David Hussman - Many companies need to slow down to get more done.

Dawn Cannon - The purpose of planning is not to prevent uncertainty, but rather to plan how to deal with uncertainty.

Esther Derby - managers are designers of the experience of work.

Esther Derby - Resistance = "Other people are not doing things I want them to do w/ the speed or enthusiasm that I desire."

FunnyOneLiners - It's not easy taking my problems one at a time when they refuse to get in line.

Hannu Kokko - Partial conditions of satisfaction for a demo: understandable, valuable for customer, wow-factor, shows progress

Hillel Glazer (heard at SE{G NA) - Proposal to add a new value to manifesto: "we value improvement over compliance"

Ian E. Savage - Certifications are only as good as the certifying agency. Got it. Let’s move on, shall we.

James Bach - A lot of my choices are about arranging life so instead of having to control my anger, I don't have anger that needs controlling.

James Bach (at #accu2010 via Tom Gilb) - 'it is Testers Job to not be fooled by What is fooling everyone else'. 'testing is about discovering the significance of the requirements'. ‘The spec is a rumor! Wouldn't you like to know what the code ACTUALLY does before you ship?’

James Bach (at StarEast via Matt Heusser) - "bad rigor is following instructions without understanding them", or "pathetic compliance"

Jared Richardson – Overheard: Jenga development. You're not sure which piece will bring it all down, so you resist any changes no matter what.

Jason Gorman (via Hannu Kokko & Kent Beck) - Knowledge of languages and APIs no more makes you a developer than knowledge of anatomy makes you a doctor.

Jerry Weinberg - When managers don't understand the work, they tend to reward the appearance of work. (long hours, piles of paper, ...)

Joseph Angel Alcala - Apologizing always doesn't mean you wrong and others right...means you respect relationship more than your ego.

Ken Schwaber (in a recent podcast, reported by Tobias Meyer) - Scrum is less a silver bullet and more an enema.

Kent Beck (via Jon Udell) - There's no difference I can see but there's obviously a difference the computer can see. I hate that.

Marcus Ahnve (via Lasse Koskela) - Feedback without conversation is criticism.

Mark Levison (re: Agile retrospectives) - I ask "what needs improvement" and focus is on discovering issues. Next I ask "What does team have energy to improve". Then they create action plan.

Michael Bolton - The *number* of test cases that are passing or failing has no more meaning than the *number* of ideas you had in the shower.

Michael Feathers (via Jurgen Appelo) - If you don't give your software a shape, it's going to get a shape anyway. The reason most of our code sucks is that we don't take ourselves seriously as users of our code. Constraints drive design. TDD is an example of a constraint (testable code), and design thrives in constraints.

Morrell (via Jesse Fewell at InfoTech2010) - your team doesn't care how much you know, until they know how much you care.

Naresh Jain - Little things make big things possible. Close attention to fine details brings out excellence in your craft.

Payson Hall - Pondering process improvement/decay cycle oscillation: +Pain > +motivation > +rigor > -pain > -motivation > -rigor > +pain... (repeat)

Peter Scholtes (via Glyn Lumley) - Explaining why change is important will not make the change happen.

Phillip G Armour - and testing is also about finding out how to find out something you don't know you don't know.

Stephen King (via Peter Murphy) - If you don't have the time to read, you don't have the time or the tools to write.

Stephen Mann (via Anna Nachesa) - Love the old management saying ... meetings - where minutes are saved and hours are lost.

Tom Kealey - Instead of asking the question "how agile are we?" try "how are we agile?" The difference is profound.

Woody Williams - The absence of failure is not an indication of success.

Newer Set of Quotes from Twitter

2010-03-14T11:20:00.001-04:00

Many things happening over the past couple of months, including becoming a grandfather. But also preparing for what looks to be a lengthy Agile/Scrum coaching assignment. So here's yet another set of quotes from Twitter posts in, again, alphabetical order by first name.
-------------------------------------------------------------------------------

Alan Cooper - Lean is related to "efficiency" which seems to me an industrial age concept. Bob Marshall - Imo Lean is much more about *effectiveness* than efficiency, hence my #rightshifting campaign, not least.

Alan Shalloway - if your team is populated by heroes your process will require them.

Alan Shalloway - sometimes being heroic is just another way of avoiding prioritization. Scott Duncan - Or is a sign of not recognizing the need to prioritize?

Ash Maurya - Build [features] in response to a signal from the "customer", and otherwise rest or improve.

Bob Martin (on liking one-week iterations) - "not much can go wrong in a week"

Brian Foote (in response to Alan Shalloway - There is a big difference between eliminating waste & not creating it in the first place.) - Fasting vs. Sanitation

Brian Marick - If you have one really great idea and express it together with many dumb ideas, the dumb ideas get a free ride in ppl's minds. This is bad.

Brian Marick - It's swell to be compulsive about "delivering value" but let's realize that "value" isn't a real thing out there in the world. "Value" is not an objective fact. It is a prediction, made by some people, about reactions of other people, over some time. George Dinwiddie - I think "value" is a subjective fact. It, of course, begs the question of whose value takes precedence. Brian Marick in response to Mike Hill - different words have different favorite tricks. "Value"s favorite is making opinion look like measurement. Brian Foote - "value" shines at not sounding like a waste of money. "business" too. Whereas, anything with "re-" in front of it sounds like a wasteful do-over. Refactor, Revise, Repair, Reengineer. Mike Sutton - re-birth, re-new, re-'why can we compete with the latest blah?' revisiting the old is a fact of life, we should embrace it.

Carlton Matthews - when you are not living the life you dreamed you tend to envy the lives of others.

Daniel Elliott (via Bob Martin) - Cope suggests a surgeon who has washed his hands still needs to concentrate on the scalpel (or architecture in our case!)

Dave Nicolette - Velocity is a measure of a team's capacity to deliver, not a measure of activity. But velocity depends on having fixed-length time-boxed iterations. For a continuous flow process, I prefer cumulative flow.

David Hussman - I find it interesting that software people tend to talk about "training" where educators talk about "teaching". Your thoughts are welcome. Ron Jeffries - training is possible, teaching is not. neither word is good. one can, at best, provide an occasion where learning is possible.

Delavigne & Rob (via Glyn Lumley) - "We want to get Deming off the quality shelf and have him recognised for his contributions to a unique philosophy of life"

Derek Wood - Any accommodation we make to doing Scrum well is indicative of an impediment that will need to be resolved at some point.

J.B.Rainsberger - Irony alert: Most organizations learning Scrum are actually practising Waterfall in a manner quite similar to what Royce intended.

Jeff Patton - Is the "definition of done" actually a "definition of built"? "Built" is when I get software, "done" is when I get benefit.

Jonathan Bach - As a manager, my mantra always is: "You may report to me, but really, I work for you."

Jonathan Bach - Scrabble analogy #1: Rarely can your most valuable test ideas (letters Q&Z) be easily put into words. Scrabble analogy #2: Sometimes a test (a word) has more value when it's placed on the boundaries (triple word score). Scrabble analogy #3: You get a bonus if you use all 7 letters to form a word (diverse techniques in one test is powerful).

Jonathan Bach - There are no certified musicians, but we know skilled ones from unskilled ones. Still, execs just want sheet music, not the sound.

Joshua Kerievsky - Faster velocity != better software development. Velocity metrics are a powerfully distracting illusion. Story point estimation is a confusing waste of time. Discussion, feature fat removal, bargain hunting are way more useful.

Jurgen Appelo - Separating process (Scrum) from development (TDD etc) is a fine example of loose coupling. Developers should appreciate that.

Mark O Oakes - A company's ability to collectively learn faster than its competitors may be only sustainable competitive advantage

Mary & Tom Poppendieck (from Leading Lean Software Development) via Jason Yip – It is the overhead incurred to enable auditability that induces the waste, not the standards themselves.

Mary & Tom Poppendieck (from Leading Lean Software Development) via Jason Yip – Randomly giving patients medication until they get better would not even be considered. And yet, we randomly give our work processes medicine, adding on more and more 'best practices,' until the processes seem to get better.

Mary & Tom Poppendieck (from Leading Lean Software Development) via Jason Yip – The advantage which a commander thinks he can attain through continued personal intervention is largely illusory. By engaging in it he assumes a task that really belongs to others, whose effectiveness he thus destroys. He also multiplies his own tasks to a point where he can longer fulfill the whole of them. (actually a quote from Helmuth Von Moltke)

Matthew Heusser - my Fathinlaw, a quality engineer at Ford, once told me the quality engineering discipline existed to product cmpny from execs

Michael Bolton - Often enough, bugs aren't mistakes. Sometimes they're differences of opinion, discoveries, dilemmas, etc.

Mike Sutton - i just got reminded of something that i forgot - thank you @netparr - agility is *not* about getting everything right, but being able to respond to them quickly and correctly when they go wrong.

Nancy Duarte (via @johnnieb99, via Brian Stallings) - "If a slide contains more than 75 words, it has become a document."

Naresh Jain - Increasing velocity is one thing and making your team more productive is another. Don't confuse one for another.

Naresh Jain - Instead of asking how much its going to cost & how long its going to take, ask with 2 ppl for 2 months what can I get?

Rosabeth Kanter - Change is a threat when done TO people, an opportunity when done BY them.

Scott Adams (via David Hussman via maiasylba) - Creativity is allowing yourself to make mistakes. Design is knowing which ones to keep.

Steve Jobs (via Scott Williams) - Your time is limited, don’t waste it living someone else’s life.

Steven M. Smith - Effort chases measurement. Measurement chases the discussable. Core org diseases aren't discussable. Diseases elude cure (effort).

Tim Ottinger - Axis: simple<---->complex is a count of structural parts & linkages, unaffected by naming, syntax, obviousness of solution. Axis: clear<----->perplexing is a measure of how easily one may comprehend the solution, aside from structural parts/linkages. A very brief routine may be simple and clear, or may be simple and cryptic. A long routine may be complicated and clear, or may be complicated and impenetrable. We find a certain synergy working in our favor in short, clear, simple routines. I would go so far as to say that brevity with clarity = elegance.

Even Newer Set of Quotes from Twitter

2010-01-14T10:19:00.000-05:00

Been far too long since I lasted posted anything, but there was a lot of moving going on in December and some other things so far in January. More moving is left, but I am going to promise to post more often. However, as I have another collection of quotes from folks on twitter, I thought I'd put them out before the list gets too long:

Alan Shalloway - a process tells you to fix your problems is different than a process that gives you information on how to fix it.

Alan Shalloway - There is a big difference between eliminating waste & not creating it in the first place.

Alistair Cockburn (reported by Biran Marick) - Remember talk from about "micro-techniques". If I recall right, he said his observation of ppl like @kentbeck and @wardcunningham was that they gained great speed by doing many small things extremely well and quickly. Alistair Cockburn - Right & I could never develop that idea because micro-techniques come in xtremely context senstitive bushel-basket-loads not onezies.

Bob Marshall - Imo Red Bead is profound because it has many layers; vain hope; delusions (all round); (false) pride; introspection; etc. Don Reinertsen - Profound and possibly dangerous. It intentionally creates a situation where the player's actions have no effect on outcome. Bob Marshall - Yes. But that's kinda the point, really? From Deming's perspective... Don Reinertsen - To some extent players are taught to accept variation rather than improving the process. (e.g. batch size reduction) Bob Marshall - Yes. But that's reality for most orgs and most workers. Variants on Red Bead can explore more "enlightened" scenarios. Don Reinertsen - The game certainly proves the methods Deming dislikes won't work. It also incorrectly shows SPC doesn't help outcomes. I think it would be more profound if outcomes had both controllable and variable components. Teach the middle way... Bob Marshall - For me, Red Bead is very Zen, in that it's point is left for the participants and audience to draw out themselves. I concur Red Bead fails to show benefits of SPC: In coaching, "ARC" reminds us Awareness -> Responsibility -> Commitment. I participated in an enlightening 15min workshop at Agile 2008 with teams and dice that could show SPC well (!RedBead). Karl Scotland - Red Beads useful to change thinking on purpose/demand/outcomes. Intentionally exaggerated scenario is powerful.

Bob Martin - humans test creatively. Machines test reliably. Both are vital. Confusing the two is disaster.

Carlton Matthews - overheard someone use this quote, "Don't ask me to cook dinner if I can't buy the groceries"
Daniel Lapin (via Carlton Matthews) - The complacent are always conquered by the committed and the diffident are always defeated by the determined.

Dave Rooney - Fair enough... knowledge is indeed power. Scott Duncan - And applied knowledge is powerful. I think more people know what to do than do it. How to transmit belief plus knowledge?

David Anderson - in next decade aim to change focus to "manage rules of game" not "manage work" or "manage people" - Deming system design.

Don Reinertsen (on “value”) - Economic view also allows Gilb's subjective dimension. A thirsty man will pay more for what is objectively the same water.

Gary Hamel (from a retweet by Bob Marshall) - "In the absence of purpose, only thing that will disrupt the status quo is pain." http://tinyurl.com/yjes5zn

Jeff Bezos (via Jason Yip) - There are two kinds of companies, those that work to charge more and those that work to charge less.

Jochen Schuchardt - We sometimes equate project mgmt with the visible planning artifacts. But the heart of it is people and relationships.

Johanna Rothman (from her blog, ptd to from Twitter) - good interviews should make a candidate (and an interviewer) think, not sweat [preceeded by “Good interviews do not surprise people. Good interviews build rapport with a candidate, learn about a candidate, preferably with behavior-description questions and auditions. Maybe with hypothetical questions. Maybe with a meta-question.”

Jon Bach - FB is an album for friends, Twitter is a coffee shop for colleagues.

Kent Beck - there's a world of difference between telling me what to do and helping me learn by telling me what to do.

Max Planck (via Michelle Sliger) - A new scientific truth does not triumph by convincing its opponents and making them see the light, but rather because its opponents eventually die, and a new generation grows up that is familiar with it.

Michael Feathers - I still think the best way to predict the future is to have friends in time zones ahead of you.

Michael Feathers - If you want to learn from experts listen to their questions more than their answers.

Ralph W Emerson (Via Alan Shalloway) - "As to methods there may be a million and then some, but principles are few. The man who grasps principles can successfully select his own methods. The man who tries methods, ignoring principles, is sure to have trouble."

Ramsey Show (via Carlton Matthews) - "Savings has to be done on purpose. Debt can just happen."

Rob Myers - so the msg could be "slow down for quality & use quality to speed up"

Ron Jeffries - If we write a great tester's tests before even writing the software, and ship when they run, have we found or prevented defects?

Ron Jeffries - It is becoming clear that w sufficient disrespect, sarcasm, recalcitrance, we can stop any/t good from ever happening again.

Roy Atkinson (via Noami Karten) - If each of us holds up a little bit of the world, it will weigh none of us down.

Scott Duncan - Change may be hard as 1 change often causes/requires another, then another. What looked like absorbable change cascades.

Scott Duncan - I think we can/should talk about both what is valuable to do & how best to do it. Being clear about what both mean. Don't see a need to create a dichotomy between the what & how as long as we realize the latter serves the former. Perhaps an uber-value [is] "We value being clear about what is important to accomplish over how we achieve that end"?

Seth Godin (not from, but pointed to from, Twitter) - Organizations thrive on their ability to allow individuals to remain faceless. It permits them to act badly, not in the interest of their customers.

Skip Angel - Decided that bug repositories are evil. Gives teams excuse 2 defer defects not show-stoppers. Don't all bugs cause workarounds 4 team/users?

Skip Angel - While #scrum is based on empirical sys 4 learning, u must have values & principles 2 guide ur learning. Agile Manifesto does that. Chris Sterling - agreed. I am more interested in teams learning & changing rather than being "right"; a team will get better when focused on both.

Stephen Parry (via Grant Rule) - "If you measure your business using averages, don't be surprised to find yourself running an average business.”

Tobias Mayer - IMO waste is defined as 'work that adds no value for any stakeholder' (others restrict to 'for customer'). Much refactoring is waste ie rework.. due to commiting too early. Lunch breaks are not work, hence not waste.

Tobias Meyer - Are agreements the same as rules, & if not, what's different? I've always found group rules to be unnecessary, in fact detrimental. Dale Emery - Agreements come from people who agree. Not all rules come from (explicit, negotiated) agreement.

W. Edwards Deming (Via Glyn Lumley) - 'Examples teach nothing unless they are studied with the aid of theory. Most people merely search for examples .. to copy them'

Ward Cunningham - Estimating is the non-problem that know-nothings spent decades trying to solve.

New Set of Quotes from Twitter

2009-12-02T06:35:00.002-05:00

[Been occupied with packing and moving since my last post until a couple days ago, though more packing and moving needed. Hopefully I'll get on a daily posting schedule before too long.]

As with the others, this list of Twitter tweets is sorted by first name. (Hmmm...a thought just struck me. By posting these collections, am I engaging in mass retweeting? Guess not since it isn't on Twitter. But there must be a name for this...or should be, I guess.)

Alan Shalloway - people are often confusing predictability with control. if you do, you won't believe you can control your dev process. being in control is not a bad thing. being controlled by someone else is.

Alan Shalloway - the more i work w/ lrg orgs (300+ devs) the more i believe that any approach that focuses solely on the team or is bottom up is doomed. top down does not mean top down control. but rather top down enabling of working on the right things in the right way.

Alan Shalloway (series of quotes) - Issue isn't "when does agile work?", rather "how agile can i be?" This implies a continuum. Going to iterations is a break. Scrum is evolutionary in perspective but revolutionary in implementation (making it hard). Kanban is revolutionary in perspective but evolutionary in implementation. Many (most?) people in Agile community think Agile==Scrum. So when Scrum won't work they think Agile won't. At conferences people say "we're doing agile" when they mean they are learning scrum.

Andreas Boes at XP Days Germany 09 on “The New Industrialization” (via Alistair Cockburn) – “Industrialization" converted subjective process to objective process (remove individual's capabilities). Taylor reduced 'processes' to invidual activities and actions. Taylor was a micro-methodologist (studied microtechniques). Alistair Cockburn - I think modern agile ppl are doing Taylorist work: study top performers, decompose, teach to others. That's exactly what we're doing! (I disagree with [Boes] - he says programmers are black box. but nano-TDD opens that box. I think agilists are whitewashing a secret Taylorist agenda, and it worries me.) what does he mean with “separate subjectivity from individuality”? I need a translation of Taylor’s 3 ground rules. Boes - agile brings in "living process"; which also lean brings in, too. diff : Taylor = optimize before announcing the process; Lean = optimize after deploying the process. Agile brings self-organization; Lean brings repeatable processes. Agile = collective learning & knowledge capture; this makes the "objectivity" and visibility needed. What does agile mean for the average worker? we haven't thought through it (bad things to follow?) Alistair Cockburn - Boes/audience Taylor tried to diminish the human, sw tries to augment (I worry that parts-replaceable programmers goes the wrong way.

Bob MacNeal (more on being self-directing) - 1) Responsibility - given or takes responsibility for results and 2) Autonomy (free to determine, plan & schedule activities).

Chris Sterling - scrum is a learning system; modify tools, practices, & process when your team learns something.

Dave Rooney - Agile is a collection of those 'things that work' put together in a synergistic way... whole is greater than sum of parts.

David Anderson - Kaizen done right is statistically based systems analysis often using SPC. Diana Larsen - Retros done right get at diff probs than SPC - esp human issues & + deviance. Well-run retrospectives incl subjective & objective data, as relevant to retro focus, + analysis & action. David Anderson - retros tend to project level focused. higher maturity orgs will have an organization level focus across teams/projects. This is not guidance or opinion. This is field reported cases. You need to be open to ask why? and challenge your beliefs. Paul Dyson (via Rachel Davies) - Like all agile practices (or practices in general), there is a risk of 'cargo cult execution' with retrospectives.

Gerald Weinberg - Definition of Bureaucracy: Each thing is in control, but everything is out of control.

James Bach - Saying ISTQB certification makes a tester better is like saying a chefs hat makes you a good cook. I had a woman in my class who studied for two years to be a baker. Then she studied for a few days to get a tester certification. Why are donuts worth two years of school, and testing worth no training at all? Because testing is easier to fake than donuts.

Jason Yip - Crisis is required to provoke deep change only if top management has a monopoly on setting strategy.

Jean Tabaka asked about 100-130 chars to describe Kanban and some responses were: Karl Scotland - Map value stream, visualize, limit WIP, establish cadence. Reduce WIP to improve value flow & individual fulfillment. David Anderson - visualize flow, limit WIP to encourage evolutionary change towards lean outcome, high maturity culture.

Jurgen Appelo - Introverts are not shy. Introverts just prefer low-noise communication. [Added by Dave Rooney in a retweet - Introvert == shy is common misconception]

Karl Scotland - If your process is designed to expose dysfunction, what do you do when your process becomes the dysfunction?

Karl Scotland (asked) - Are retrospectives a form of Schewart/Deming Cycle? David Anderson (replied) - I find most retrospective guidance to suggest subjective, anecdotal feedback rather than objective data-based Deming style info. SEI classifies OID based on subjective, anecdotal evidence as low maturity even though OID is a ML5 process area. Deming's method would be considered high maturity OID/CAR ML5 by the SEI. Typical retros are a low maturity precursor to PDSA. meanwhile, @kjscotland and I are only reporting facts from field. High maturity kanban teams tend to drop retros as waste.

Vasco Duarte - Ppl talk a lot about business value, but they forget that for most people business value is totally subjective! (i.e. unquantifiable)

Vince Lombardi (via Jason Yip) - Perfection is not attainable, but if we chase perfection we can catch excellence.

"The System," "They" and "Policy"

2009-11-22T10:02:00.002-05:00

This was such a funny/weird situation involving communication with a customer that I just had to pass it along.

First, some setup information. In August of 2008 we changed cell phone company to fit in with the last job I had. So no contact from the prior provider since then. That is, until two days ago...

A 4-page (2 sheet) bill from the provider (one of the big four) shows up saying its from the Oct 10-Nov 9 Bill Period with a Nov 13 Bill Date. First page shows

Oct 13 Tax Adjustment ............................................. -$1.81
New Charges ...........................................................   $1.81
                                                                 Total Due    $0.00

On the payment remit slip is says "DO NOT SEND PAYMENT. This amount will be credited to your next bill. $0.00."

On page 4 (after 2 pages of legal stuff and ads for buying more services) it says

Charges
Toleance.................................................................. $1.81
                                                                        Total $1.81
(and that is the only thing on the page except the "4of 4" page number, the company logo, and billing acct number and dates).

Now the fun begins since I wanted to know what this was all about after 14 months of not being with this provider. So I call the customer service number from the first page of the bill and explain all this to the person who answers. I tell them I'm trying to find out why I got this and what it means given I have not been a customer since August of 2008.

The person was quite nice but said her records of our account doin't show any such credit/charges. But she said we didn't owe anything, so just ignore it. But, again, I asked why did I get this and what does it mean. Since she seemed to feel I should not care, I asked for a supervisor.

The customer service supervisor was also quite nice, but could not explain it, i.e., nothing showing on any records she could see. She guessed it was a credit left over after we closed the account. So why, I asked, did it take 14 months to contact us and why what was the charge for that cancelled out the credit. She did not know and put me on hold a few times trying to find someone who might know. Finally, she sent me to their finance/accounting folks.

That person, also very nice, suggested "they" must have noticed this credit recently so "the system" sent me a letter letting me know. I said, it wasn't a letter, it was a bill and asked what a "Tolerance" charge was. I never got an answer to that last question. But it seems, since we closed the account and paid the final bill, somehow "they" decided we were owed a $1.81 for some overpayment of tax. However, it is the provider's "policy" not to send out checks for less that $5. But the finance person could also not really explain much beyond this and did not show anything in their records either specific to this "bill" being sent.

So, apparently, to clear the account, "the system" or some "they" decided to send a bill to acknowledge the credit and the charge in that credit amount to make the account $0 since it was not the provider's intent to actually reimburse the credit amount given it was under $5.

This silliness, of course, was to satisfy some legal and accounting rules that I didn't know or care about.

But a couple things struck me about all this, besides the silliness:

None of the people could access any system of information that would allow them to find out what this was all about. (The Finance person was basically guessing what happenbed based on "policy" rules, not based on any information about the actual account.)
I wonder if this was something that was done to many people to clear out old accounts, i.e., the cost to do this and then to deal with people like myself calling up to find out why, if so, would clearly be a substantial waste of time, money and company credibility.
It would be interesting to know how much the provider makes each year keeping all credits under $5 (and what legal loophole makes this possible) since they waste the postage and time sending out silly $0.00 balance bills anyway.

Like Arsenio used to say, makes you wanna' go "Hmmmm."

Expectations Around Uncertainty

2009-11-16T13:34:00.000-05:00

Back in September (10th), Mike Cottmeyer posted “Managing Expectations about Uncertainty” and noted that traditional project management views it as important to “manage uncertainty out of the project.” On the other hand, Agile efforts “[r]ather than managing OUT uncertainty” choose “managing FOR uncertainty.” I like that phrasing of the Agile approach. Mike did point out that “both worldviews have a place depending upon your context and problem domain” and that it is “up to us [as Project Managers] to recognize the nature of the projects we are working on and choose the strategy most likely yield a desirable outcome.”

I am inclined to say that "uncertainty" early in a project can be divided into (a) things we cannot (now) know and (b) things we should be able to know. I believe the more traditional approach takes the latter as its view. That is, the traditional view is that “due diligence” should be able to reduce uncertainty to nearly zero, leaving few (or no) things unknown. Thus, Agile’s approach to "embrace uncertainty" suggests irresponsible risk-taking in the traditional view because insufficient effort is expended to eliminate all the uncertainty possible. In this view, uncertainty is lack of knowledge that should be corrected by better initial effort. I believe the Agile approach looks at (a) as its view of early uncertainty. That is, there are things we really cannot know early on, may not be able to know until work has been done and feedback collected, or may not end up needing to know by the time we get there.

Now the word “uncertain” suggests being aware of something but not totally sure about it. If you are totally unaware of something (or it is something that is truly unknowable), talking of being “certain” or “uncertain” makes little sense. The traditional view of “uncertainty” carries a lot of weight, then, within that worldview if it means things we are aware of but do not understand deeply enough. Consider the large number of lists of potential project risks and failure causes that have been compiled over the years. In effect, they say, “Look, all of these things have been noted in the past and could impact your project. You need to explore them and become ‘certain’ about whether or not they have meaning/impact on your work.” Hence, “due diligence” involves being thorough about considering all these factors since we can and “should have known” this early on during planning for our project.

The Agile view is that it is wasteful to try to drive out all uncertainty early because it cannot be done. This appears to the more traditional view as irresponsible for the reasons noted above. An Agile approach relies more on short delivery cycles, than detailed up front planning, to address uncertainty. As with many other things, an Agile approach advocates an incremental, iterative way of addressing uncertainty, increasing detail as the events requiring it loom closer and moving from an implication of "early" to merely "before." From an Agile perspective, “due diligence” includes avoiding wasteful anticipation of risks/problems as much as responsible consideration of them.

This is not to say all early consideration of risk/uncertainty is to be avoided. However, from an Agile perspective, the details regarding how certain issues should be addressed can be delayed until more knowledge is available to the project. Agile projects move ahead with what is known while information on what isn’t known is developed.

In the end, of course, if an Agile project goes bad because of an unplanned for issue, the traditional view can say “See, we told you.” Equally, if a traditional project never encounters issues it expends effort to make plans for, the Agile view can say “See, we told you.” I am reminded of a talk Kent Beck gave at XP2006 in Oulu, Finland where he discussed “responsible development.” I think “certainty” and “due diligence” are things which walk that line of what is and is not “responsible.”

Notes on the ASQ Software Division’s ICSQ 2009

2009-11-12T19:43:00.003-05:00

Each year, the Software Division of the American Society for Quality (ASQ) holds their International Conference on Software Quality. This year, it was held at the Hilton in Northbrook, Illinois on November 10-11 (with a tutorial day on the 9th).

What follows are my notes on the sessions I attended and the Agile “debate” in which I represented the “pro” Agile side. Other than keynotes, sessions were run in parallel with 4 tracks going on at the same time. My notes, therefore, represent the one session I attended of four going on simultaneously.

One thing to be aware of is that attendees at ICSQ’s are often from regulated industries and firms doing government related contracting where formal, standards-driven quality approaches are the rule.

Tuesday, November 10, 2009

Keynote – Bill Curtis on “Quality in Multi-Tiered IT Applications”

Bill Curtis has been a researcher, practitioner and chief scientist in software methods and individual performance for many decades. He has worked at ITT, MCC (Austin, TX research consortium), SEI (as head of the Process program), TeraQuest (process assessment and improvement), and now at CAST Software. I have known Bill over the years during his time at MCC, SEI and TeraQuest, in particular coordinating (and applying the results of) his research activity in software design expertise for the company where I was working at that time.

Curtis started saying, “We’re moving beyond just what smarts and knowledge can handle.” By this, he meant the systems and their interactions have evolved (and continue to evolve) to where product (code) quality ideas are not enough to manage the desired results. Expertise in application level quality, i.e., how all the components interact, is what has the largest impact on system quality today. Quoting an ACM article (Jackson, “A Direct Path to Dependable Software,” CACM v. 54, no. 4), “The correctness of code is rarely the weakest link.”

Curtis pointed to problems with design choices that “pass (functional) tests,” but are (at best) inadvisable practice when scaled and must address non-functional production requirements. Presaging the end of day keynote by Joe Jarzombek, Curtis said that we need to be able to make dependability, assurance, etc. “cases” about our systems. That is, we should be able to provide evidence to support arguments that justify belief in claims about such non-functional requirements.

Curtis offered a few other ideas such as:

addressing people’s ability to understand a system when changes must be made since he said 50% of the change effort in maintenance is devoted just to figuring out what the system, not an individual module, is doing;
allowing (and training) testing groups to do true QA, indeed do Quality Engineering, which would require a broader involvement of personnel from testing organizations in the full lifecycle of work as well as not regarding test groups as “entry-level” positions;
understanding the COO’s view on the need to standardize ways of doing things an organization does not compete on.

Finally, Curtis mentioned the Consortium for IT Software Quality “sponsored by a partnership between the Software Engineering Institute (SEI) at Carnegie Mellon University and the Object Management Group (OMG) to combine their industry-leading strengths in developing software-related standards and appraiser licensing programs.” As one of its activities, Curtis said, it will work to create more operational definitions of software (non-functional) quality characteristics (i.e., the “ilities”). The ISO 25000 series, which supplanted ISO 9126, has definitions, but the CISQ’s work suggests they are not viewed as operational enough.

Tom Roth – Psychology of Software Quality

Roth’s background is in embedded avionics software quality assurance where he is in charge of overseeing reviews, internal audits, testing, etc.

Roth started by saying we should “think of QA as people trying to influence the behavior of other people developing software.” Hence, his talk was about influencing people with regard to quality and the importance of QA knowing how to develop trust in developers since not everything can be reviewed. (Interestingly, an article in the ASQ’s main magazine, Quality Progress, for this month, is entitled “Trust, But Verify.”) But Roth cautioned against “enjoying the power you have” as an arbiter of quality, using knowledge of psychology to establish a collaborative relationship with development groups/management.

In discussing inspections and reviews, Roth noted that software engineering formalities, such as Fagan inspections, impart a level of discipline and, potentially, a group sharing of responsibility, which may not exist with individuals alone. Indeed, inspections turn the deliverable being inspected over to the group and the individual does not bear the full brunt of making sure quality is present in that deliverable. From an Agile perspective, I was thinking that, after a while, such discipline should become more internalized and less dependent on external rigor.

Some of the things Roth touched on were how:

in relationships, differences often attract while similarities comfort, but trying to force sameness can destroy the attraction;
inappropriate habits exert tremendous influence on further (and, perhaps even worse, expanded) inappropriate behavior;
we are not 2, 3 or 4 different people, despite external appearances under different social circumstances, i.e., there is a congruence in behavior which, especially under stress, will reveal itself;
people working alone can spend 2/3 of their time evaluating alternatives and 1/3 implementing a chosen alternative while two people working together reverse the balance, effectively quadrupling the productivity of one person alone;
behavior [engineering] leads attitude [morality] - you can tell people what to do but not how/what to think, so work on behaviors/practices and allow the thinking to come along on its own.

The last two struck me as quite interesting, of course, from an Agile perspective.

Ed Weller – Getting Management to Listen

There are many talks that have been given over the years about how to talk to management. Ed Weller covered some of the same terrain in terms of speaking from a cost/dollars perspective. However, he did offer some specific ideas related to managers who are:

used to technical “change agents” (1) underestimating the cost of implementation, (2) overestimating improvement benefits, and (3) introducing risk management is not comfortable with;
faced with “Problem Saturation”, i.e., “consumed solving today’s problems” with “no time for next week’s (month’s/year’s) problem.”

Weller’s suggestion was to focus on data on the cost of rework, pre/post ship defects, and, in general, poor quality. From a lean/agile perspective, this means showing management how they can reduce waste in the software process.

Rebecca Staton-Reinstein – Using A Cost of Quality Model to Drive Improvement

This was a fairly standard talk on CoQ models elements. Some of the audience interaction and comments were of interest, especially regarding the difficulties in doing CoQ calculations:

collecting data accepted as “accurate” enough to truly prove/show improvement ROI is very difficult for an organization that does not have some level of process discipline and decent data capture capability;
such models, on the cost avoidance side, are talking about things that haven’t happened, yet, requiring (accepted) historical data to show prior trends that could be reasonably extrapolated to the future;
belief in quality as a matter of personal “morality” or “will” (i.e., we have problems because people just don’t try hard enough to do the job “right”) rather than something addressable through an engineering approach;
being able to take quality data and relate it to schedule and budget impact.

Then, at some point during the talk, the following thought struck me: if you do things with urgency, you won’t have to do them in a rush.

Keynote – Joe Jarzombek, National Software [Security] Assurance effort from DHS

Joe Jarzombek directs the Department of Homeland Security’s program on Software Assurance and has been doing this, with little budget, for over 4-1/2 years. I met Joe through my activities on the IEEE Software and Systems Engineering Standards Committee when he was still consulting within the Dept. of Defense. (Before that, he was an active duty Lt. Colonel with the Army serving at the Pentagon.) Joe’s job is to promote an interest in and work actively toward securing the infrastructure of the USA from cyber attack. To do this over the years he has brought together academic institutions, government agencies (DHS, Dept. of Commerce, Dept. of Energy, and DoD), non-profit agencies, and commercial organizations to work on a variety of efforts in tools, techniques, guidance, educational programs, standards (working with IEEE and ISO), etc.

Joe’s talk is one I have heard over his many years with DHS. He updates it regularly with the latest status on the efforts noted above. And, in case it is not otherwise obvious, the “assurance” focus of this work is on writing secure software and securing infrastructure computing.

As most of the printed materials which arise from the efforts of the participants he has brought together are produced under government funding, they are freely available at the Build Security In website under the “DHS SwA Web Site” link. Another good source of material on Software Assurance is articles from issues of Crosstalk (the Journal of Defense Software Engineering) which are also freely available. And, though a few years old, a July 31, 2007 “State of the Art Report on Software Security Assurance,” is also available.

Wednesday, November 11, 2009

Keynote – Edy Liongosari “Everything’s Elastic”

Liongosari directs work at Accenture Technology Labs and spoke about the changing landscape of computing as it moves from traditional computers to mobile devices. Most of the trends he noted (e.g., cloud computing) were not new, however, some of the implications and data were interesting.

For example, 30% of “smart” phones are owned by people with family incomes at or below $30,000. For them, this was their computing platform in the sense that they did their “computing” through internet access to sources of information, data, and applications. (On the latter point, Liongosari noted that there were some 100,000 iPhone applications available.) From a third-world perspective, Liongosari noted that, despite wide-spread cell-phone use in the developed countries, cell technology was even more prevalent in the third-world where land-line phones, computers, bank accounts, etc. were not at all common or available. Indeed, there were places, he said, where people “barely able to feed themselves” had cell phones.

Liongosari also spent some time talking about how large organizations were beginning to use cloud capability to get work done in fractions of the time it would have taken them to set up in house infrastructure to handle the same level of computing. He even noted an insurance firm (unnamed) that uploaded data to the cloud, performed massive analysis and downloaded the data and results a few hours later, “renting” the time and resources.

From a social computing perspective, he talked about how companies were starting to use such ideas (if not the most well-known social sites) in “harnessing the power of the crowd” to collect ideas and trends. Some examples were IBM’s Bluehouse, Adobe’s Cocomo, and Dell’s Ideastorm.

Another point made was how people in the workforce from teens to late twenties had a view of free access to computing resources and what this means when they are in company environments. Liongosari also noted the relative lack of concern people in this age group have for the idea of privacy, which is a concern (and mentioned in Joe Jarzombek’s talk) with regard to cloud computing.

While I listened to this keynote, another thought came to me: IT is moving from meaning “institutional” to “individual” technology, even more than the PC represented such a move since, it is not just owning your own computing resources now, but having an individual “right” to information that is starting to dominate thinking.

Tim Olson – Lean Principles and Process Models

Tim Olson has considerable background in process (improvement) consulting, having worked at the SEI early on in the CMM program, being involved with the Juran Institute, having Six Sigma experience, and, most recently, working with Lean.

Olson started by relating to an example from Deming about the danger of trying to replicate success by others through simply copying their apparent behavior without understanding the context and principles behind the practices. This resonated greatly with me because it is an issue I have seen with Agile adoption when companies learn practices and techniques without an understanding of Agile Values and Principles.

For the most part, Olson’s talk was about basic Value-Stream and Value-Stream Mapping ideas. However, he noted the lack of automated tools to make the mapping easier. He did note that, in discussion with Toyota, that they used walls and boards, without automated tools. But Olson noted that his process definition approach, focused on diagrammatic, rather than textual, process description has led him to apply process modeling/mapping tools to value-stream work.

He did caution, however, that simply trying to transplant Lean manufacturing ideas to software has been a problem in various cases since this has resulted in removing “non-value-added” activities such as configuration management.

Siegfried Zopf – Pitfalls in Globally Distributed Projects

Zopf is from Austria and discussed issues and problems he has observed working in multi-national, multi-cultural project situations.

Zopf began by making a distinction between distributed and outsourced situations, then, between minimally and large-scale outsourcing. Overall, it seemed as though he was emphasizing awareness of the level of appropriate control to apply to the different combinations of situations. For example, in a minimally responsible outsourcing situation – the work is confined to low-level design and coding – risk is low and pulling the work back in-house is not a problem, but the financial savings are lower. On the other hand, there is great financial advantage in outsourcing greater parts of the work, but much more local project management required in the outsourced locations. Zopf suggested allowing for 15-20% cost for such a “beachhead” operation.

Zopf also, in connection with any larger outsourcing effort, noted how planning must account for extra costs in project management, communication, travel, documentation, and knowledge transfer that would not exist in a fully local project. Thus, a company cannot take an estimation done assuming in-house work, then simply portion out parts of the work, “transplanting” the estimate and plans without adjusting for the differences.

And, for distribution matters in general, whether outsourcing or not, there are still issues related to national and cultural differences regardless of whether or not it is the “same” company. A couple examples of what he discussed are:

two groups, for each of whom English is a second language, and the problems they can have trying to communicate using English which are not the case when at least one group is native English speakers;
monochronistic and polychronistic cultures where the former view time as linear, compartmentalized, and value punctuality and the latter view time as more fluid, schedule multiple things at the same time, and even expect people/things to be late.

One final point in distributing work (with or without outsourcing) is process mismatch. Specifically when working with a high maturity (using the CMMI scale, Level 5) organization, that organization will find it difficult working with another organization that is not at least Level 3. In the reverse direction, a low maturity organization may find it frustrating working with the expectations and pace of a high maturity one.

Ron McClintic – Performance Testing

Ron was the “con” side in the Agile “debate” (which I describe below) and has a lot of years of testing/QA experience working for and in the GE Capital environment. He currently works with applications that collect and analyze data on truck fleets using a combination of hardware, embedded software, and more traditional application software. However, the multiple vendor networked environment he works in matches quite well with the multi-tiered issues Bill Curtis discussed.

His talk could be considered a “case study” since he went into detail about the various points for performance testing that his efforts need to address, from the lowest levels of (vendor-supplied) software doing network and database optimization and capacity adjustments to customer GUIs and response expectations. On the latter he noted work done to determine thresholds for response time from the minimum one would ever need which matched the time for a person to perceive a screen change up to the upper limit before a customer would abandon a (web) application and go elsewhere. It turns out the latter is about 4x the tolerated time frame. The problem is that, for different client situations, the tolerated times vary.

As an example of a difficult performance/response issue to address, one client reported significant performance problems before 10am each day, but which seemed to go away at or about 10am. After a few weeks of probing and testing, it was discovered that 10am local time was when the company’s European offices, mostly in the UK, ended their day, a fact not previously revealed to Ron’s group. The moral being not all issues are really technical ones even though they have technical impacts.

One statement Ron made rang very true to me, especially in light of one of the points Bill Curtis made, and that had to do with using testing and test tools in an “investigative,” rather than ”reactive,” manner. I think, overall, this is an important way to view the real value testing and test staff can have to an organization as opposed to the, often, “last ditch” quality control function often performed. QA (testing and other true QA functions) should be to supply information to the rest of the organization about the status of progress and quality of work.

The Agile “Debate” – Ron McClintic & Scott Duncan

The other talks/keynotes described above occurred in the order I’ve listed them. This “debate” actually occurred Tuesday afternoon just before Joe Jarzombek’s keynote. I’ve saved it for last since I am commenting on my own (and Ron’s) session. Ron had proposed this and I had been recommended to him by the Conference Chair, Mark Neal, whom I have worked with before on Software Division activities.

We started the session with me going over, briefly, the Agile Values and Principles and stating that I felt this is what “defines” Agile. Since the various methods preceded the Snowbird meeting, the term “Agile,” applied to software development, occurred at that meeting and with regard to the Vs & Ps. So, for me, that means the Vs & Ps are what define “Agile” while practices and techniques are examples of possible implementation approaches. Ron had no real problem with this as he noted he agreed with these ideas. His objection to Agile came from two sources, he felt:

it resulted in overall “suboptimization” of a project because it focused only on optimization of the development piece, and
it did not focus on actually profitability of a company that sells to the marketplace, defining value as just the delivery of the working software.

Thus, his argument was that a more traditional approach to projects that accounted for the full product lifecycle, including longer-term maintenance and support costs, was more appropriate. He also felt there had been no appropriately trials of similar project situations that had collected data to show the benefit of a well-conducted traditional effort compared to an Agile one.

He and the audience had stories to tell about “purist” insistences from consults arguing against the responsibility for Agile teams to be concerned with such issues as they were matters for the business beyond the development team. What I was hearing were stories of:

“teams” without appropriate business collaboration or all the skills needed to do the work or
projects where the organization, itself, isolated groups outside of development from trying to pursue/accommodate an Agile approach, insisting on more formal handoffs or
developers insisting that going back to fix work not done completely or well in the first place constituted “refactoring” or
the usually litany of refusal to document, measure, and/or plan.

Indeed, in one case that Ron noted, developers were writing code without requirements. I had to ask him how they got the okay to be developing anything with “no requirements” and, then, suggest this was an “Agile” approach.

A couple audience members also brought up the book “eXtreme Programming Refactored” and its claims of failure for the C3 project.

What I found was that people were exceedingly receptive to an explanation of the Values and Principles and accepted practices, seeing how wrong it was to characterize many of these behaviors as “Agile,” rather than merely ad hoc.

Of course, throughout the Conference, there were stories and discussions about this same sort of thing happening to other ideas once they had “crossed the chasm.” Mark Paulk, for example, was there discussing various process improvement ideas as well as his research work at Carnegie Mellon University with Scrum. He and I sat at lunch with a number of people who were at this “debate” or had other Agile contact and discussed how similar things were after a while for the CMM (and now for the CMMI) with people ascribing “requirements” status to guidance material and pushing their own idea of process “rightness” in assessing companies.

So I have left this “debate” topic until the end, and am not going into great detail about what was said because, overall, it demonstrated the attraction the Manifesto’s Values and associated Principles have for people. It also demonstrated, to me at least, the need that exists for people to understand how practices are intended to implement those Vs & Ps and not simply copy (or think they are copying) them without such understanding.

And Still More Quotes from Twitter

2009-11-09T09:38:00.001-05:00

Alan Shalloway - Scrum's power lies in its collaboration, co-location & feedback. thinking it lies in its practices is what ossifies/limits it.

Alan Shalloway - The Leader must be pulling the org thru the change process, not pushing it thru. When a leader is pushing, the org's, people don't know if they are being pushed toward something better or off a cliff- they do not have the concern if leaders are out in front and pulling.

Alistair Cockburn - Crystal = frequent delivery + close communication + reflective improvement => self-awareness; Ladder: techniques -> principles -> metaskills

Bob MacNeal - Maybe "self-direction" is one characteristic that distinguishes a team from a group. Scott Duncan - Collaboration would be another. Been in group's that related well, but each did their own thing and were structured that way.

Carl Sagan (via Kevlin Henney) - Intellectual brilliance is no guarantee against being dead wrong.

Chris McMahon - to improve performance, what is THE ONE THING we can do right now? Ron Jeffries - they already know the one thing. Everyone does.

Dale Emery - Passion and respect are not inherently at odds, though they can seem to be if you don't know how to express both at once.

Daryl Kulak - Project estimating is just wishing to two decimal places.

Dave Ramsey (via Carlton Matthews and @E-Mealz) - When your outgo exceeds your income, your upkeep becomes your downfall.

David Hussman - Sometimes Twitter seems like the world first and largest virtual fortune cookie.

Elizabeth Hendrickson - It is hard to Speak the Truth, and speak it diplomatically enough to be heard, when people want Comforting Lies.

Elizabeth Hendrickson (reported by Chris Sterling at PNSQC) - "The definition of Agile is in the results" (deliver value frequently at sustainable pace) "Flexibility is a side-effect of being agile."

Glyn Lumley - Anybody's job should not merely be to "do it right" but to do it better.

Hillel Glazer - Real engineers *do* like GOOD processes. Those that don't are posers. If you made it through college/university w/out any processes you probably weren't in an engineering school. Check your diploma.

J.B. Rainsberger - I find it hard to value Customer Collaboration without valuing whatever the customer decides is value.

Jacob Yip - Stop the line != stop and fix. Stop and contain. Fix when you understand why, which may require longer analysis.

Jason Yip - A car is a system; individual parts are not. Extreme Programming is a system; individual practices are not.

Jason Yip - Paradoxically, if you truly value people, you tend to focus on the work, not on the people.

Jeff Patton - agile people: is iteration process iteration- repeat the same steps in cycles, or product iteration: reconsider and rework prod decisions?

Jeff Patton - Agile-resistant teams hate all the meetings. Experienced agile teams love all the collaborative work.

Jeff Patton - Requirements are the boundary between what I get to decide and what you get to decide. It's a fuzzy discussion, or DMZ.

Jim Highsmith - Agility is the ability to think and learn rather than blindly following a recipe. Brian Marick - No, agility is not "the ability to think and learn rather than blindly following a recipe". Let's not equate "agility" with "good". The ability to think and learn is part of Agile and 8 zillion other things. Joshua Kerievsky - @marick You've focused on the think/learn part while the important part is not "blindly following a recipe." (even an Agile one). Brian Marick - @JoshuaKerievsky I suppose it needs saying. But I think it would be good if Agile thought leaders thought more about software.

Jim Knowlton (at PNSQC via Matt Dressman) - "date, but don't marry, your framework."

John D. Cook (actually from his blog pointed to by Jurgen Appelo) - I’ve said that someone has too much time on their hands, but not since I read Meyer’s post. I see now that the phrase is often a sour grapes response to creativity. I don’t want to do that anymore.

John Goodsen - Iterations are the dogma and waste of Agile. Flow and pull make iterations irrelevant. Are you certified to manage waste?

John Seddon (via Bob Marshall) - Less of the wrong thing is not the right thing.

John Seddon (via Bob Marshall) - Measure what is important to customers, not auditors.

Mike Sutton - The larger the organisation the thinner the thread that connects work to value.

Naresh Jain - Code is not an asset its a liability. Tacit knowledge gained building the product is the asset.

Nat Pryce - New manifesto: while we value valuing things other than value, we value valuing value more.

Pat Reed (via George Dinwiddie) - Projects fail at the beginning, not at the end.

Payson Hall - On reflection, key takeaway from #Risk Conference last month was definition of #project risk as "Uncertainty that matters".

Peter R. Scholtes (via Glyn Lumley) - Why do you hire dead wood? Or, why do you hire live wood and kill it! (From The leader's handbook: making things happen, getting things done‎).

Peter Scholtes (via Glyn Lumley) - Bureaucracy is a form of waste.. people with no real work to do impose needless tasks on those with real work to do.

Scott Duncan - I see "giving" offense as different from "taking" it. I can avoid the former, but likely not the latter.

Scott Duncan - In general we do not expect perfection, but in particular, we do.

Scott Duncan - Waste: Write on board; copy to paper; transcribe to elec doc; have a bunch of people review to check for mistakes, etc.

Tim Ottinger - Between the placebo effect and the hawthorne effect it is hard to know anything about anything involving human beings.

Tim Ottinger - Biggest tip for remote pairing: Latitude hurts, longitude kills. Common hours are helpful. [Actually, a general comment about distributed teams that I’ve heard various places.]

Virginia Satir - Understanding and clarity, not agreement, is what's important in dialogue.

“What do you think of …?”

2009-11-09T09:16:00.000-05:00

Long before I became involved with Agile ideas, I was doing internal (and some external) consulting in more traditional (software) quality and process improvement. And many years before that, even before I got involved with software, I taught some at the college level. My style, I came to learn, was “Socratic” in that I would use questions to encourage (self) learning by my classes, clients, and audiences rather than be too direct, too often with “answers.”

I think this has been helpful as an Agile coach/trainer since, over the years, I do believe leading a horse to water will get most of them to drink. That is, most people are reasonable enough to arrive at decent conclusions if given the information that will allow them to do this. This does not mean people will not have years of experience pointing them in certain directions than my questions may be hoping to encourage. I also find that it is easier, in a group, to get more useful learning to occur using this question-driven approach since the group learns from itself in a sense.

In another context in a Twitter post I stated you can control what you give, but not what others take. With at least some common base of agreed upon information, I do think you can encourage people to take and experience them asking you to give more. And what I do give, I try to do so by offering alternatives when I think they exist.

The title of this post suggests what I typically do that seems to work. I’ll ask people to direct attention to some data or situation and ask them what they think of it. In general with any data related to quality/process, I suggest people take an “it’s ten o’clock, do you know where your children are”* approach. That is, I encourage them to know what’s going on to the greatest extent reasonable at that time, then consider whether they are or are not okay with that being the situation.

That’s it. Nothing dramatic. Just something that I find has worked for me over the years in many different contexts.

(By the way, I have even used this approach to teach 8yr olds, and up, about Newton’s laws of motion where they derive F=MA and eventually come to understand why satellites stay up in the air. Takes about one 30-45 minute class. And, no, they don’t all remember everything, but they do end up learning about learning and that they, in fact, can learn about some seemingly complex things.)

*For those not familiar with this phrase, it was used in a US public service announcement on TV to urge parents to know what their kids are up to and where they are late at night.

Burndown & Control Charts

2009-11-08T08:27:00.005-05:00

The teams I’ve worked with have used burndown charts to track task hours remaining during their iterations. For them, the burndown baseline represents the optimal pace a team would need to be on to complete all work for the iteration. Assuming, of course, that all the work contributes to completing committed stories, the burndown chart helps indicate how well the team is doing in meeting the iteration goal. I say “helps” as the burndown is not the entire truth. (Some teams have tracked story points in a burndown, but, as that produces a stair-step chart, most teams have used a task hours chart for their iterations. Later I’ll mention how some teams track story completion as well.)

Coming from a more traditional quality background originally, I view the burndown chart as a simplistic form of a classic control chart. The baseline is like the central line on a control chart. We have no upper or lower limits since we are not doing statistical sampling, of course. We are tracking actual data completely. But there are some similarities in how a control chart is used and how we can view the actual iteration progress line compared to the baseline.

If the actual progress line hovers/fluctuates right around the baseline, the team is on track to complete the iteration goal. If the actual progress line is above the baseline constantly, it could mean the team is not headed to complete their Sprint commitment. If the actual progress line is below the baseline constantly, it may mean the team is headed to complete their Sprint commitment somewhat early. However, being not too far above or below the line is likely nothing to worry about if the trend is consistent. If a team’s ability to estimate and commit is effective, they should not be too far (or for too long) above, or below, the line.

On the other hand being below the baseline and heading even further below or being above the baseline and heading even further above it, should be case to consider taking some action:

A progress line that is below the baseline and increasingly headed down means the team is ahead of their schedule for completing tasks and getting further ahead. This may or may not be good news. If things are going very well during the iteration, the team might discuss with the customer/Product Owner/etc. the possibility of taking on more work. However, this pattern could suggest some tasks being skipped or downgraded in time. That ought to be looked into as well to be sure everyone understands what the pattern means.
A progress line that is above the baseline and increasingly headed up means the team is behind their schedule for completing tasks and getting further behind. This is not good news as this pattern suggests some tasks being added or increased in time. This ought to be looked into to find out why work is not converging on the iteration goal.

Earlier I said the task burndown chart is not the complete story and used the word “helps.” Like a control chart, the burndown chart is an indicator of whether or not further consideration needs to be given to team progress. Because of the statistical nature of control charts, deviations of certain kinds from the center line, in either direction, are reasons to investigate the cause of that deviation. The assumption is that this baseline represents expected results of sampling the production process and deviations either way could be either good or bad news, but, in either case, cause to look deeper into what is happening.

The same goes for the burndown chart, but there is certainly more to know about iteration progress since completion of task hours does not, by itself, mean completion of stories which is the iteration goal. One could be completing many hours of task effort, even be below the baseline, and not have completed a single story. This can happen if a mini-Waterfall is occurring in the iteration, with testing tasks bunching toward the end of the iteration.

One thing a couple teams I’ve worked with have done is to put story completion targets along their baseline, then note when each story actually gets completed. This gives both a daily progress indication based on the tasks and an iteration goal indication based on when stories show completion. If teams size stories effectively, a story should be getting completed every few days, at least.

Now teams that are communicating well among their members usually know all these things without the chart. But the visual representation of what the team “knows” is a useful reminder and lets people outside the team see what is happening. The visibility/transparency provided by the burndown chart is important and, for me, is its basic value since it offers everyone the opportunity to understand iteration progress and discuss it objectively.

Common Project Risks and Agile Mitigations, Part 8, Epilogue

2009-10-27T06:33:00.001-04:00

Part 7 closed out the discussion of issues that the various sources identified as contributing to project failure. As promised at the end of that Part, this Part looks back on the first 7 parts and offers some summary comments. I’ve also added some further bibliography entries that I encountered them while doing this series. While they were not used to compile the list of failure issues, they do discuss project failure issues.

As implied in Part 1, the series was inspired by my having reviewed a number of surveys and articles on why projects failures occur. It seemed to me that the Agile Values and Principles, if adopted using a number of generally accepted Agile practices and techniques, could mitigate many of these issues. While an Agile approach cannot pretend to solve all the identified problems, its communication and feedback model for team-based work is far more likely to surface and deal directly with such problems when they are usually less severe and less costly to address.

However, these project failure risks have existed, in some cases, for decades. People have been dealing with them long before the Agile Manifesto was created. Over the years, the failure rate has been declining, as some of the source surveys have noted. The recommended solutions have been codified in various bodies of knowledge (e.g., PMIBoK, SWEBoK, CSQE BoK), models (e.g., CMM®/CMMI®), and standards (e.g., ISO 9001, IEEE S2ESC, ISO/IEC JTC1/SC7). These approaches to the problems have been (often unfairly) characterized by voluminous documentation, significant cost, and bureaucracy. Agile methods have (equally as unfairly) been characterized by applicability only to small, low-risk efforts. The BoK, models, and standards approach is a reflection of what people believe to be currently understood “best” practices derived from those used in other (non-software) engineering and project domains. Agile methods reflect the view that a less complex, less prescriptive approach is possible which relies less on detailed process guidance and more on process transparency and person-to-person communication.

What is interesting, however, is the position often taken by both approaches with regard to why projects employing them continue to have problems. From a phase-based, sequential project perspective, the view is that such problems can be avoided if only project personnel would follow the existing, accepted, and well-defined project management and software engineering BoKs, models, and standards. That is, projects fail because people do not follow known methods and procedures effectively, i.e., they do not carry out such guidance “right.” Interestingly, when an Agile project is not successful, we often hear the same things, i.e., it failed because people did not do Agile (or a specific method) “right.”

So the question, in either case, is: can it merely be a matter of doing what is predetermined to be “right,” whether traditional or Agile? And, why is it so hard to do what is “right” to achieve the results expected from doing so in either case? Proponents of an Agile approach say the BoK/model/standards approach suffers from too much overhead, inapplicable detail, and inflexibility causing an inability to respond promptly to the circumstances and changes that lead to failures. Proponents of the BoK/model/standards approach say Agile methods suffer from ad hoc behavior, short-term thinking, and transfer of significant risk responsibility to the business and customers.

What may be most true of either approach is that a formulaic application of their practices and techniques, with insufficient knowledge and understanding of their basic principles can produce an inflexibility (or inability) to respond when circumstances require. It may be that the “best” approach would be to ensure that the guidance in the BoKs, models, and standards is understood by those working on projects, but that the Agile Values and Principles are used as the structure within which to apply that knowledge.

Related Bibliographical Items

Ellis, Keith, Business Analysis Benchmark – 2009. IAG Consulting, 2009. http://www.iag.biz/register.html (free, but must register with an email address)
Pedersen, Alf A., “Project management failure - Why projects fail,” from his Database Design Resource website. http://www.databasedesign-resource.com/project-management-failure.html
Software Engineering Institute, Insights on Program Success, A Joint Publication of the Software Engineering Institute and the Systems and Software Consortium, Inc., October 2009, CMU/SEI-2009-SR-023. http://www.sei.cmu.edu/reports/09sr023.pdf
Ward, J. LeRoy, “What Troubled Projects Are Costing Us,” from PMStudent blog (Josh Nankivel). http://pmstudent.com/what-troubled-projects-are-costing-us/
Winters, Frank, “The Top 10 reasons Projects Fail,” a 13-part series from Gantthead.com. Part 1 is at http://www.gantthead.com/content/articles/147229.cfm with links to all other parts.

Common Project Risks and Agile Mitigations, Part 7, Everything Else

2009-10-25T08:11:00.001-04:00

Part 6 addressed the topics of quality and stakeholders. This part will address the remaining categories of issues that the various sources identified as contributing to project failure. They involve issues with and between people, availability of information, failure to apply lessons learned, testing process and vendor performance.

Remaining Issues

While a number of the issues noted below have impacts in many other categories, there are some specific instances if problems that were not assigned to other areas and mentioned on their own in the various surveys of problem failure risks. The remaining categories/issues are:

tension in social relationships; people (often respected staff) who, for one reason or another, block progress.
missing, or delayed access to, information resulting in untimely decision-making. (Shari Pfleeger [May] notes the common situation that “there isn’t one person who has an overview of the whole project.”)
not learning from past problems and/or ignoring the need to act on what has been learned.
inadequate procedures, strategies, and tools to be used in testing; late involvement of the test team in the project.
poor vendor performance delivering on a contract; for vendors of contract staff, a concern was offshore-outsourcing relationships as engineers sometimes must train colleagues who do the same work for much less pay. (Vendor contract issues were not mentioned much, but were a high % item when they were.)

Applicable Agile Values and Principles

Regarding people issues, Agile’s emphasis on transparency, regular improvement, and a team-based approach, while not automatically solving problems, will surface them sooner and in a more constructive atmosphere when it can be less costly to address them.

Agile’s expectation of frequent, direct communication between development team, business staff, and stakeholders, while not solving all information availability problems, can reduce them significantly and surface them earlier. Agile’s approach can also mitigate “stonewalling” around information that can occur in phase-based, sequential approaches.

The inspection and adaptation potential in daily meetings and regular retrospectives greatly increases the opportunity for, and expectation of, improvement. Reflection occurs close to, if not immediately after, an opportunity to learn and improve. Improvements, then, take place or are identified almost immediately rather than waiting, sometimes months, before consideration of them and any action is taken in a traditional project cycle.

Testing is given first-class status by most Agile methods and test staff are very much expected to be a part of the development team. In this way a quality focus not only drives development work but confirms its success in meeting customer functionality expectations. Agile’s expectation of working software as the primary measure of project progress and success, backed by a strong regression test suite, not only moves concern for quality to the forefront of the project but help ensure that changes can be made with confidence. There is also significant emphasis on test automation in Agile methods.

[A side note here regarding the latter. In 1989-90, I conducted some interviews with many large, technology-based companies in the US regarding the technologies they found most important in addressing quality. I asked what 3 things, if they were denied use of them, would have the most negative impact on their development work/quality. Two of the things mentioned most frequently were automated source code control and automated builds. Today, these two are almost always found present in development organizations. The third thing mentioned was automated regression testing. Interestingly, today, test automation is still a subject of much concern, even after 20 years.]

With regard to vendor performance, Agile Values and Principles do not directly address the issues noted. However, Agile’s team structure, communication expectations, collaboration emphasis, and success-focused contracting all provide a basis for working more effectively with vendors as with other participants and stakeholders. At the very least, an Agile approach will surface the issues of everyone’s performance from the very beginning of the project and offer the potential for a more constructive, early solution to problems when they are far less costly to address.

[One thing I have personally noted in working on some large, distributed efforts, was the lack of Agile experience of and training offered to off-shore teams. I believe this was because, at least in those instances, the on-shore company making use of off-shore, contract staff had been working with the off-shore companies in traditional project situations for many years and just stayed with them. The problem was the lack of training off-shore folks were given though many on-shore people went to some level of Agile training. More than once, as a consultant myself, I was asked to train off-shore personnel when their lack of experience/familiarity with Agile methods became clear.]

A final Part 8 will be posted within the next few days where I look back on the first 7 parts and make some summary comments.

Common Project Risks and Agile Mitigations, Part 6, Quality & Stakeholders

2009-10-23T07:36:00.006-04:00

Part 5 addressed the category of technology. This part will address the next two most frequently mentioned categories of issues that can lead to project failure: quality and stakeholders.

Quality Related Issues

The particular issues related to quality mentioned in the various survey sources include:

general lack of focus on quality (i.e., assurance and control), leading to critical problems where quality and reliability end up being unacceptably poor.
No practical way to show software meets non-functional criteria (i.e., “ilities”) or that the delivered software will "work" for the user.
Misunderstanding of the role of quality assurance, giving it a secondary status to other activities and viewing it as just being about testing.

Applicable Agile Values and Principles

Agile’s major contributions to addressing quality issues are:

the insistence on working software as the actual measure of progress (and success) on a project;
early and frequent demonstration of functionality to the customer;
clear acceptance criteria that define what it means for functionality to be “done.”

All of these work together to help reduce the possibility that what is built diverges from what the customer needs/expects and to increase the likelihood that what is built functions acceptably. Various development practices are used to implement these three main goals (e.g., TDD, continuous integration, pair programming, shared code ownership).

The brevity of this discussion on quality is not meant to trivialize its importance, just to indicate that an Agile approach has fairly straight-forward quality-related goals and techniques to met those goals.

Stakeholder Related Issues

It might have been expected that stakeholder related issues would be a bit higher, but some of the more frequently mentioned items have stakeholder impacts associated with them. The particular issues related to stakeholder attitudes and behaviors mentioned in the various survey sources include:

Lack of sufficient user input/involvement leading to or as a result of most requirements coming from management or other user “proxies” who do not (or rarely) use the existing system.
Stakeholders unable to agree on answers related to requirements, resources, etc. and the consequences of these things, who then “paper over their differences” early on, pushing problems into development where they are revealed when software implementation concerns demand a specific answer.
Fear that project threatens their job: their influence (even working conditions) in the organization because many software projects result in one group assuming power and another losing it.
Stakeholders who are partners on this project may be competitors in other areas.

Applicable Agile Values and Principles

Quite frankly, in my view, there aren’t significant Agile ideas that specifically address the last two of these issues. Certainly, the Value of “customer collaboration” applies here, but does not, in itself or in combination with any of the Principles, offer a solution to serious stakeholder personal or competitive differences.

As to the second issue, Agile advice is to give the development team a single individual to “represent” the customer(s) and deal with the differences before the team gets its direction on functionality and priorities. That person (e.g., Product Owner) deals with the stakeholder diversities. Not really a solution in itself as it just hands the issue to the business, saying it needs to handle this if the development team is going to be able to be as “agile” as desired.

With multiple, a perhaps disparate, stakeholders, there is not Agile “magic” to wipe away the problems. Early and frequent demonstration of working software, though, can help focus stakeholder attention on the work, giving them concrete functionality to consider. Being concrete often makes it easier to overcome, or at least discuss somewhat more openly, concerns of and differences between stakeholders.

The first issue is best addressed through Agile’s goal that there be regular, frequent involvement by the customer in discussing functionality and reviewing iteration output. However, this will not prevent the “customer” view from being dominated more by management than actual system user perspectives. Again, it is left up to the business side of the relationship to ensure the information the development team receives represents the actual needs of the business in a way that will allow the most benefit to be derived from the work produced.

[Note: In the case of both quality and stakeholder negotiation matters, there is a great deal of existing wisdom and practice that is not Agile-specific in nature, but certainly consistent with Agile Values and Principles. Indeed, one could say the same about all of the issues mentioned in this series. Certainly, the goal of the series has not been to suggest only Agile ideas exist to address them, just that Agile has something to contribute in the way it suggests conducting product development.]

My First “Agile” Experience in the Early 80s

2009-10-22T16:26:00.000-04:00

Of course, it wasn’t actually Agile according to any currently understood practices and techniques. But it did have aspects of some of the ideas now viewed as Agile and was the experience that came to mind when I first read Kent beck’s 1st edition of eXtreme Programming Explained. I’ll describe the working environment, the product (a bit) and how we interacted. You can judge for yourself where the experience fit on any scale of being or not being Agile.

We were doing a commercial 4GL product (non-procedural language and associated database) known as RAMISII. It was written in FORTRAN and IBM 370 assembler so, of course, ran on IBM mainframes. There were 9 of us in the development group. Two focused mainly on the database code (and an access module to IMS DB/DC so the “language” could do reporting from those databases as if they were our own database). Two did “support” and non-development (regression) testing. So five of us worked on the language/reporting side. There were also a couple documentation folks who worked under the Marketing group which had 4-5 other folks in it. Marketing was our internal “customer,” though we never used that term or really thought of them that way formally.

Once year, after relational databases started catching on but before DB2 was big in the IBM environment, it was decided we would redo the reporting/language side of the product to add substantial functionality, including language constructs to allow relational operations for reporting extract purposes. Since we were going to change some 70% of the system, it was also decided we’d get rid of all the FORTRAN and replace it with Assembler. That’ll give you an idea of the technical scope of what the five of us would tackle over the course of 9 months or so. The 9 month target was because it was early summer and the next major customer gathering/conference was the next Spring, so we needed to have a major release ready then as we did each Spring.

[An aside on why assembler and not C, Pascal, even PL/I. We had to stick with languages that IBM was currently supporting on mainframes and an IBM C compiler was many years away at that point. But we could have used PL/I except that, at least back then, there were dynamic libraries associated with it that the customer would already have had to have, i.e., they had to be using PL/I, to then run our delivered system. Or the customer would have to license those libraries separately from IBM. Or we would have had to license them and pay IBM for each copy of our system that we released with them. Company senior management did not want to get involved with any of these concerns. So no PL/I.]

As to our environment, besides just saying it as assembler and IBM mainframes. Our (two) machines were mid-sized mainframes running VM/CMS (a virtual memory system that allowed each of us to feel like we had a machine all to ourselves). We worked on, wait for it, TI Silent 700s, i.e., thermal printer terminals. No CRTs as it was before even 3270s were widely used. (A year or so later, we had to get into supporting 3270 full-screen, but, at that time, it was command-line interfacing.) Each of us had our own private office in a nice building in the suburbs of Princeton, NJ. Of the five of us, 3 were relatively new to the company (<1 year), one was just a bit over a year, and I had been there between 2-3 years. (The database access and support folks were more senior than I.) Everyone, save one very junior person, had several years of experience at that point (i.e., 5+ at least) in IBM or Honeywell mainframe environments with assembler (and other languages).

Our immediate bosses were really fine folks with development experience from an operations research background. There was no obvious hierarchy among us. (Indeed. while I was there, I was "promoted" 3 times and never knew it. It never affected how I related to or was noticebly viewed by anyone, at least as far as I could see.) We each, and our managers, valued good ideas and enjoyed one another’s ideas as there was always lots of room for people to contribute. The product was thousands of lines of code. I can’t remember exactly how big but we had some 50-60 uniquely named FORTRAN and assembler modules that ranged from a couple hundred to a thousand lines each. Back then, it was a large footprint product that employed dynamically loaded overlays to limit memory use.

That summer, those of us in development spent a month or so determining every module we felt we’d have to touch based on some very high-level specs from Marketing. These specs came from the last Spring’s meeting where customers would annually “vote” on the features they wanted from a long list compiled from everything they all had been submitting for the 6 months before that. As this was pre-32 bit addressing, we also had some decisions to make about how to restructure our pointer approach for our report record sorting. One of the features desired was the ability to print summary data before all the data actually printed on the page, so we had look-ahead selection and calculation issues to deal with so we could create, carry and sort summary data records within the detailed records.

I have no recollection of how this started, but near the end of the summer, we started having regular meetings with the Marketing and documentation folks to flesh out functionality in more detail. We never had a formal requirements spec, but the user/support manuals became our “spec” as were interactively decided on features, new language syntax, etc. We would not have new functionality to show at each meeting, but we had ideas to present and questions to cover so Marketing and documentation knew, throughout, what we were up to and what it was going to look like. (Being syntax and command-line focused helped a lot since we did not have GUIs and full-screen formatting to worry about.)

After a couple months, though, we did have features to show since we had updated/rewritten enough to have threads of functionality able to work. We could also start testing functionality, at least within development, though we were also sharing what we were doing (as was Marketing and documentation) with the support group so they could work on tests at their level to incorporate into the test suite. (We would give them ours, but they’d always enhance them and then move them into their large test suite where they thought the new tests would work best with existing test cases.)

As we saw how our design ideas were working out, we began to offer to Marketing ideas we had on features, related to ones clients had requested, that we felt could “fall out” of work we were doing with little or no additional effort. If Marketing liked them and management accepted our evidence that it would not impact the Spring client meeting demo date, we’d put them in. An example of this were various summary calculations and totals we saw we could add if Marketing felt they’d be useful to clients. In some cases, we suggested ideas the clients had literally suggested but which were lower on the voting list than ones the Marketing identified as committed to the release. So, as we began to announce what features would be in the release, end customers were really happy to see everything they had voted for making it as well as other things they had not expected we would deliver.

A couple months before the meeting, all the functionality was done as was all the user documentation. Actually, the documentation had been settled upon a month before that. The doc folks were just doing their own formatting and example insertion work. Examples often came from our test examples. In deed, tests we used often came from ideas Marketing had for typical user request streams. During the last two months, we started working more with performance improvements and internal structural changes. No new functionality was being worked. All defects found during that time were due to internal issues and had no functionality impacts, i.e., no missing or misunderstood requirements.

At the Spring meeting, customers seemed really excited by the work we had done. They loved all the functionality as well as hearing, despite all that was added, that both memory footprint had been reduced and performance had been improved. There was also an explosion of functionality ideas from clients coming out of the meeting. (As the formal release date was early summer each year, we were able to incorporate a number of these successfully before the actual release, so clients did not have to wait a year for them.)

In the process of making all these changes, we introduced no major defects and, in fact, found two serious defects that had been plaguing us on large reports and on customer system configuration changes, i.e., whether they allocated dynamic memory above or below the base load address of the main system modules.

We never had a name for how we operated. It clearly wasn’t “Agile” in many aspects, but I think we hit every Manifesto Value dead on and several of the Principles.

Common Project Risks and Agile Mitigations, Part 5, Technology

2009-10-19T06:30:00.001-04:00

Part 4 addressed the topic of creating an initial plan (i.e., planning and estimation). This part will address a category of issues that can lead to project failure which was surprisingly high in the number of times mentioned: technology.

Technology Related Issues

The particular issues related to technology mentioned in the various survey sources include two types. The first is application technology, revealing itself through

Complex technical details (often hard to recognize or address early in a project)
Lack of sufficient technology competence to address the complexity.

The second is development process technology, revealing itself through

Poor up-front (inflexible, inadequate) architectural planning
Lack of effective, practical ways to show one approach is better than another.

Another item also mentioned in connection with these was that technical decisions may be made by people without expertise in the domain, but with the managerial authority to make those decisions. One could argue that that this is not a technology matter, per se, as non-technical decisions could as easily be made under the same circumstances. But it was mentioned, so I wanted to note it.

Applicable Agile Values and Principles

The Agile Values and Principles that seem to apply are those associated with individual & team performance and design approaches as well as frequent software delivery.

With regard to application technology, if a commitment has to be made early to such directions before other work occurs, such commitments, if “wrong,” will unquestionably have a strong, negative impact on project success. The Agile approach is characterized by two phrases often seen in Agile literature: making decisions at “the last responsible moment” and focusing on “the simplest thing that could possibly work.” Key here, of course, are “responsible” and “could possibly work.” While Agile “embraces change” and seeks to “inspect and adapt,” that does not mean things are done ad hoc. It would be irresponsible not to make decisions using the best information possible, at that moment. However, to make progress and gain more information usually requires moving ahead, incrementally, at the point when a decision must be made (“the last responsible moment”) using that information in the most direct, clear manner known at that time (“simplest thing that could possibly work”). (Naturally, in a very complex domain, “the simplest thing” is still going to be complex, but likely better to deal with than a more complicated complex thing.)

Regarding technical competence, especially in software, it can be quite difficult to assess because the evidence is often buried within the code and not obvious, especially in phased, sequential efforts, until very late in the project schedule when various pieces are integrated into the larger system. Agile techniques such as collective code ownership, pairing and general visibility within the team are one way this information can be revealed sooner rather than later. Also the Principles of commitment to continuing technical growth and regular reflection on how to work better provide a path toward increasing individual, team and project competence on a regular basis.

These latter Agile techniques and Principles apply equally as much to the issue of development process technology, including architectural planning. There is always the question of how much up front work is necessary before useful progress can begin. The answer is usually less than might otherwise be expected if working software is delivered early and frequently thereafter since there will be concrete evidence of what “works” and what does not. In a complex project, what this will require is close communication between any architecture groups and development teams and a willingness of the former to work iteratively and incrementally rather than in a purely phased, sequential manner.

This latter point is often the most challenging organizational issue faced early in an Agile adoption effort since adoption efforts often start within development teams, perhaps with some test participation, but often without direct involvement of other technology-based teams. Architectural teams are one; configuration and change control teams are another; very often build and deployment teams are a third. Failure to incorporate these areas into the Agile effort can produce, at best, delays and limitations on the velocity achieved by teams since other groups usually work in a less iterative manner. They will engage in more up-front planning, expect less frequent changes and follow well-defined timelines for availability later in a typical project cycle as they do for non-Agile projects.

And More Quotes From Twitter

2009-10-17T09:21:00.000-04:00

A fourth installment of interesting things I've captured from Twitter that people have said (including some of my own) either individually or as part of a thread of discussion (again sorted by first name):

Alan Shalloway - biggest difference between adopting Lean now from adopting agile 10 yrs ago is customers r now more open-minded than th consultant community

Bob Marshall - Common insight from Reinertsen, Goldratt, Seddon: Manage *queues*, not schedules, capacity, efficiency or costs.

David Hussman - Detailed requirements are poorly written tests or what I call "tests in disguise".

David Hussman: Project communities are bonded by common goals not by percentage of availability on org charts.

Declan Whelan - I tire of people saying we were successful w/o agile as if success is binary. Agility fosters the ability to expand success.

Eric Hoffer (via Steve Freeman) - “Every great cause begins as a movement, becomes a business, and eventually degenerates into a racket.”

Gloria Steinem (via Suhas Walanjoo) - The truth will set you free. But first, it will piss you off.

James Bach - 3-word critical thinking process: Huh? Really? So? (question meaning, then question fact, then question significance)

Jason Gorman - 1. Hire good programmers. 2. Give them clear goals. 3. Give regular constructive feedback. 4. Stay out of the way!

Jason Yip - Isn't it kind of weird that Japan has a Deming prize and the US has a Shingo prize?

Jean Tabaka - Scrum vs Kanban vs other systems/certification debates distract vs focus. Continuous systems improvement jazzes me. Pick & focus.

Marcin Niebudeck - What I find the most difficult in #agile transition is not the legacy code, but the legacy people. For legacy code we have already good engineering techniques from #xp. What do we have for legacy people?

Mark Twain (via Will Green) - Never argue with an idiot. He will drag you down to his level and beat you with his experience. Scott Duncan - A variation "Never argue with an idiot. Onlookers may not be able to tell the difference." Both from Twain, I think.

Niklas Bjørnerstedt (via Kent Beck blog) - A good team can learn a new domain much faster than a bad one can learn good practices.

Scott Duncan - I am just not convinced the way to combat dysfunction is with more easily dismissed forms of dysfunction.

Scott Duncan (From the movie "Nightwatch") - "It's easier for a man to destroy the light within himself than to defeat the darkness all around him." (Anna Nachesa said the actual translation from the original Russian is: “It's always easier to put out the light within yourself than to cast away the darkness outside.”)

Tim Ottinger - I think that the agile motto should be "Building a better next week."

Timothy L. Johnson (via Josh Nankivel from Twitter)- Changing the world, surprisingly, looks a lot like living your life... day to day... with purpose... with focus... and with love. And there are days when looking at yourself in the mirror at the end of it all... and smiling... is really the best accomplishment. (28 September 2009)

Tobias Mayer - Scrum & XP are incomparable. Scrum is a framework for organizational change, XP for individual craftsmanship.

Unknown (via Kim Coles) - "The world is so fast that there are days when the person who says it can't be done is interrupted by the person doing it.”

Vasco Duarte - My def: "a method scales iff the effort needed to manage "things" grows at a slower rate than the number of "things"."

Will Rogers (via Ainsley Nies) - “Even if you're on the right track, you'll get run over if you just sit there.”

William W. (Woody) Williams - Highly motivated, productive people working in the wrong direction do huge damage in a short time.

Willie Colon (via Roy Atkinson) - The capacity to learn is a gift; The ability to learn is a skill; The WILLINGNESS to learn is a choice.

Yves Hanoulle - "A shared vision is about shared state, not about a shared statement."

Common Project Risks and Agile Mitigations, Part 4, Project & Risk Management

2009-10-12T07:49:00.000-04:00

Part 3 addressed the topic of creating an initial plan (i.e., planning and estimation). This part will address related categories of issues that can lead to project failure: project and risk management.

Project Management Related Issues

To hearken back a bit to the last part, consider some thoughts from Tom DeMarco [May] in noting how a “Lean and Mean” emphasis in companies has “goaded” managers and staff into unreasonable estimates and the need for overtime work. “Any failure will be viewed as a direct result of underperformance,” DeMarco says. However, underperformance is “not even a significant [project failure] factor” for most projects compared to simply having initially unattainable goals.

Now consider some remarks from Ed Yourdon, quoted in [May] saying, “Nobody seems to acknowledge that disaster is approaching” even when they recognize it. “There is no early warning signal.” May goes on to state Yourdon’s belief that, “Until more organizations abandon waterfall-style development in favor of processes that demand early working code or prototypes…this scenario will continue to be familiar.”

The particular issues related to project and risk management mentioned in the various survey sources include:

Failure to apply (or understand) essential project management techniques and/or project control systems (e.g., tracking, measurement).
Inadequate visibility of progress (not just resources used) generally due to status reporting being misleading (e.g., generally more optimistic than pessimistic) or not having effective means of tracing a software development from requirements to completed code.
Not proactively managing risk or basing actions on “catching up” later (e.g., counting on overtime to handle contingencies).
Lack of knowledge of the actual probability of success, late failure warning signals, and reduction in resources needed for the project.

The first of the issues noted above argues for traditional project management capabilities being needed. Yet, though such capabilities might, on the surface, be present, the latter three problems could still occur because of the things DeMarco and Yourdon mention. Part 3 mentioned some ways to try to address DeMarco’s concerns. In this part, Yourdon’s concern should also be addressed.

Applicable Agile Values and Principles

Again, all the Agile Values and most of the Principles seem to apply in one way or the other.

The first problem noted above could just as easily be an issue in an Agile project if the techniques and principles for how to conduct one are either not applied or understood. The key in an Agile project is that these techniques are very near-term, direct and based on visible indicators of what is happening every day rather than at weekly/monthly status reporting occasions. This promotes the Agile Principle of simplicity, especially in process and tools, applied to project management by taking advantage of close, regular communication among project stakeholders and regular reflection on and adjustment to plans and project conduct. Because of all this, an Agile project will not engage in “Project Management Chicken” where everyone waits to see who will be the first to have to admit they are behind schedule.

Another important concept is how Agile projects measure progress based on working software rather than tasks accomplished. While it is true that an iteration burndown may track task, the ultimate measure of success will be completion of the committed to functionality and its acceptance by the customer. This is Agile’s “highest priority” and, as in planning, employs short timeframes (e.g., a few weeks) as the cycle for ensuring project direction meets customer expectations. And, as the demonstration of achievement of iteration goals is a very visible event, surprises are very unlikely if stakeholders maintain involvement with the team(s).

Finally, as noted in Part 3, planning is organized to make changes easier, less costly, and very visible. Agile project management, then, is focused on how to best respond to change and address risks right away. Indeed, a risk-based approach to managing projects is very much in keeping with Agile principles. However, rather than try to defend against change and risk, Agile projects take advantage of the near-term focus and visibility to accommodate change and surface risks early. (Jim Highsmith has some good advice related to what he calls a “Planning and Scanning” approach to managing projects, encouraging us to “expand our anticipation practices to include both planning (working with what we know), and scanning (looking ahead to learn the unknown as quickly as possible).”)

Regarding surfacing issues early, I have experienced managers responding to an Agile approach with the feeling, at least initially, that all they hear about day after day are the impediments the team is dealing with and the issues that need to be addressed. However, after a while, they see that these are issues that usually get dealt with immediately and which go on all the time, but are often not made visible in traditional projects. It is the visibility of the issues that initially produces the discomfort but, as time progresses, then produces greater confidence that teams are managing work well and ensuring stakeholders are kept informed. In this way, if large issues emerge, they are recognized earlier and addressed more promptly. This does not necessarily make them automatically easier to deal with, but does mean you’ll be dealing with them earlier, rather than later, increasing the likelihood of dealing with them at the least cost to the project.

Common Project Risks and Agile Mitigations, Part 3, Planning & Estimation

2009-10-08T11:27:00.000-04:00

Part 2 covered issues related to requirements as the number one issue plaguing projects. This part will address two of the next most frequently mentioned category of issues that can lead to project failure: planning and estimation. Planning was the next most frequently mentioned category, but estimation, which is highly related to planning, was also high on the list. Therefore, these two have been combined to be discussed together.

Planning & Estimation Related Issues

As with requirements, the fact that planning is a problem area should come as no surprise. Making plans, getting agreement to them, managing the project using the plan, and making changes to a plan all involve substantial effort. Later, project management will be discussed. But, for this post, the focus will be on creating a plan.

The particular issues related to plan creation/initiation mentioned in the various survey sources include:

Allowing a plan to diverge from project reality, or to be created which diverges from that reality in the first place, due to lack of estimation experience, estimation under pressure, rejection of reasonable estimates (usually resulting in underestimation), ignoring the obvious (e.g., back-of-the-envelope calculations), and/or not revising plans if scope changes.
Failure to plan, adequately or at all, to consider all project activities, to address risk management, or to inadequately document decisions and commitments (leading to later disagreements, disappointments, and costly rework).
Infrequent project milestones (i.e., project "chunks" too large), allowing too much time to elapse between opportunities to validate that work done meets intended needs.

A telling comment from Watts Humphrey (as quoted in [May]) is that “any plan [project managers] put together won’t meet the [desired release] date, so they can’t plan.” Additionally, May asks states, “It is unfair to call a project a failure if it fails to meet budget and schedule goals that were inherently unattainable. Attempts to circumvent a project’s natural minimum limits will backfire. This problem occurs any time someone ‘makes up a number and won’t listen to anyone about how long other projects took,’ said Capers Jones.”

There are also famous quotes (mostly from the military) about planning and what happens when plans start to be executed such as “a battleplan never survives contact with the enemy” and “plans are not important, but planning is everything.” Interestingly, the formality, rigor, and hierarchical control structure of the military is classic, but the military also realizes how important it is for the people on the front lines to be prepared to inspect and adapt. Remember Clint Eastwood’s comment as Gunny Highway in “Heartbreak Ridge” regarding plans and response to situations in the field: “Improvise. Adapt. Overcome.”

Applicable Agile Values and Principles

As with the requirements category, most of the Values and Principles also seem to apply to planning and estimation.

The key in Agile-based planning (as with requirements specification) is simplicity born of close collaboration with the stakeholder(s) and frequent validation that work is meeting needs. Because of the iterative and incremental nature of an Agile approach, detail is reserved for the near-term with progressively less detail the further out the planning goes. This approach is taken to reduce the initial cost of detailed plans which, in all likelihood, will be changing anyway.

Because of this willingness to change plans if stakeholder needs and priorities change, Agile planning (and validation of the planning), as an activity, occurs frequently (i.e., daily, every few weeks). In this way, the plan will not diverge far from reality, can be adapted easily when changes occur, and will not consume significant effort to maintain. Most importantly, all planning is based on the regular, frequent delivery of working functionality as the basis for assessing the validity of the development activity.

Common Project Risks and Agile Mitigations, Part 2, Requirements

2009-10-03T10:13:00.002-04:00

As noted in Part 1, this series is intended to “identify the various reasons cited for project problems and suggest how elements of an Agile approach might minimize the risk of them occurring.” That first part set some background to where the categories of project difficulty have come from and identified a list of some 17 such categories. A number of the latter categories mentioned far less often in the sources of failure data and observations. So I’m going to cover the categories from most to least frequently cited in the belief that’s how people would like to see the parts in this series emerge.

This part will address the most frequently mentioned category of issues that can lead to project failure: requirements.

Requirements Related Issues

It probably comes as no surprise that requirements related issues are at the top of the list. After all, there would be no project unless someone felt a need to have some work done; however, problems eliciting and defining requirements “completely” are likely no surprise. The traditional phased-sequential process expects someone to explain everything they want, analysts to translate that into formats (often text-based) understandable to all others who need to know, and rigorous control over changes. This is aggravated by the fact that many “someones” (i.e., stakeholders) may be involved. (Stakeholder issues related to project problems are covered later in this series and, no doubt, contribute to requirements being at the top.)

The particular issues related to requirements mentioned in the various survey sources seem to fall into two groups:

First, late discovery of information that was assumed to be true or was not known when work began.

Discovery that there is no more need for certain requirements (or the system as a whole) to be developed.
Late recognition of the seriousness of requirements repercussions.
Mid/late-development changes in requirements and scope.

Second, misperception in what could be expected and how satisfaction of expectations would be determined.

Inadequate acceptance criteria, which results in "poor-quality" delivered software.
Incomplete, ambiguous, inconsistent, and/or unmeasurable requirements (and other project objectives).
Unrealistic Expectations.

One could look at this list and also say the items are communication problems. That is, inability to be capable of communicating what is needed early enough to avoid the problems described. That would be true enough. But it is also clear that change constitutes a good bit of the concern over the role of requirements in project problems. Finally, it seems that “knowledge” is another possible theme. All of these are focused here on requirements and contribute to project instability of one sort or another.

A couple of things are suggested by the above two groups of issues. In the first, much effort is usually expended based on assuming, one way or the other, that the information available after “due diligence” has occurred is, in fact, accurate. In the second, even without significant change, if criteria for assessing satisfaction of requirements are inadequate or applied only after much effort has been expended, disagreements are likely as to whether the result satisfies what was intended.

A large, up-front investment in very detailed descriptions of functionality and stringent change control procedures are often not justified by later project occurrences and lead to both of the difficulties noted above. We are asking much of stakeholders to insist they can, and should, be fully explicit up front and incur large costs when changes become necessary. However, knowing this, stakeholders must acknowledge that to avoid such problems both regular, direct communication and effective prioritization of requirements will be necessary on their part.

An interesting remark comes from Watts Humphrey [May] who has said, “You can’t design a process that assumes [requirements] are stable” so “learning what the requirements really are while building the product” should be expected.

Applicable Agile Values and Principles

When it comes to this category of issues, every Agile Value and most of the Principles apply.

From a change perspective, an Agile approach seeks a development process designed to adapt to, rather than resist, change. It does this by focusing on working software rather than documentation as its measure of achievement. This does not mean all documentation is rejected, just that it be kept as simple and direct as possible in leading to working software.

Up front specifications are kept simple (e.g., stories) with stakeholder involvement used to replace extensive documentation passed from one person/group to another. Given short iterations and frequent demonstrations of working software, simplicity and brevity in specifications is possible. Simplicity in requirements is also achieved through emphasis on the importance of test cases as executable requirements specifications. Hence, less is written in static text/diagrams and more in executable scripts.

Given that individuals on a project (e.g., developers, testers, product owners, etc.) work on effective, direct interaction between one another (on a daily basis), collaboration can replace extensive written specifications. Then, with software demonstrated and delivered within a few weeks of the start of the work (and every few weeks after that), there is significantly less chance that stakeholder expectations and development work will diverge greatly before a correction can be made.

This last point is an important one in reducing the cost of change and addressing the concern that full, up-front specification is needed to prevent disappointments at the end of the work. An Agile approach seeks validation of all aspects of the work on a frequent basis: every few hours, every day, every few weeks.

All of this, combined, does not guarantee changes will not be required that can be costly. Nothing can do that unless change is prohibited from occurring completely. In some cases, that may be possible, but usually only if the time from beginning of work until the end is short anyway. What an Agile approach can ensure is that change impacts, possible requirements misinterpretations, and issues of dissatisfaction with functionality can be identified early and resolved as inexpensively as possible, all without significant cost having been incurred until such things are discovered.

Common Project Risks and Agile Mitigations, Part 1, Introduction

2009-09-30T09:33:00.002-04:00

There’s a lot of literature on software project failure rates and reasons. The intent of this series is to identify the various reasons cited for project problems and suggest how elements of an Agile approach might minimize the risk of them occurring.

How Much Failure?

Surveys of project “failure” rates have been going on for about 15 years. The 1994 Standish CHAOS Report is usually the first one cited in most discussions. After that, CHAOS Reports have come out every other year and there is data from sources such as Capers Jones, Computer Weekly, and KPMG. The CHAOS Reports tend to be the most widely quoted and certainly the most regular reporting of such data. Unfortunately, reports, from any of these sources, cannot easily be verified since their raw data, data sources, and methodologies are not generally available to other researchers/analysts.

A few years ago, Robert Glass [Glass] raised the question of just how much project failure there actually was. He noted the substantial reliance on the CHAOS Reports by many who discuss failure rates and criticized how people used those numbers. However, he did note that the three categories in the CHAOS Reports (i.e., cancelled, challenged and successful) had seen improvement in the percentages of projects falling into each. In 1994, the reported % rates were: 31, 53 and 16; in 2000: 23, 49 and 28; in 2006: 19, 46 and 35. Things clearly seem to be getter better, but, as Glass pointed out, they are “not figures to be proud of.”

Glass also makes a point about how “failure” is usually defined in such reports. Project failure means large cost or schedule overruns, late discovery of quality problems, or cancellation (for any reason). A “functionally brilliant” project, Glass says, that “misses its cost or schedule targets by 10 percent” could be categorized as a failure. (A recent examination of such data [El Emam] repeats many of Glass’ observations, including the improvement trend in project results.)

For the purposes of this series, I’d like to dispense with how much and where one would place a project in the CHAOS categories. Indeed, my title is intended to do away with the whole concept of “failure,” not because I don’t believe it happens, but because, like Glass, I think it is a relative term depending on your definition.

In discussing a Robbins-Gioia Survey (2001), IT Cortex [Cortex] says:

Project failure is not defined by objective criteria but by the perception of the respondents. The advantage of a perception is that it naturally integrates multiple aspects. Its obvious disadvantage is that it is inevitably partial: if the respondent has taken an active role in the project it will inevitably embellish the reality, whereas if the project has been "forced down his throat" he might cast a grimmer look at the project outcome.

Thus, I’m not here to tell you what you, or your management, or your organization, or your customer(s) should think constitutes “failure."

I do want to note, though, that lists of potential project risks are even larger than lists of project failure reasons. To all this, Capers Jones points out [Jones 96], “There are myriad ways to fail. … There are only a very few ways to succeed.” And though it is not mentioned directly in any of the lists of project failure reasons, Tim Lister points out [Lister] that, “The biggest risk an organization faces is lost opportunity, the failure to choose the right projects. So, value is every bit as important as cost (the plusses matter as much as the minuses) and your process for deciding what projects to do is more important than your process for how to do them.” But that’s a topic for another blog.

What I want to do in this series is list the things that typically challenge projects, increasing risk and tending toward less satisfactory results than might otherwise have been achieved. Since the various sources do not list problems in the same order, I won’t try to do that. Indeed, I have taken all the reasons given and categorized them in my own way. The frequency with which things are mentioned does offer some indication of their importance. So I am listing the problems based on my categories in order of the overall frequency of issues being mentioned in the sources. Briefly, these areas are:

• Requirements Related Issues
• Planning Related Issues
• Technology Related Issues
• Project Management Related Issues
• Estimation Related Issues
• Quality Related Issues
• Stakeholder Related Issues
• Risk Management Related Issues
• Management Related Issues
• People Related Issues
• Schedule Related Issues
• Communication Related Issues
• Resource Related Issues
• Lessons Learned Related Issues
• Process Related Issues
• Testing Related Issues
• Vendor Related Issues

Having addressed these problem categories briefly, I’ll point out what aspects of an Agile approach I believe could (help) minimize each of them.

Part 2 of this series will begin the discussion of the categories listed above.

References

[Armour] Armour, Phillip G. “Twenty Percent,” Communications of the ACM, June 2007 (vol. 50, no. 6), pp. 21-23.
[Cortex] IT Cortex. http://www.it-cortex.com/Stat_Failure_Rate.htm and (http://www.it-cortex.com/Stat_Failure_Cause.htm)
[El Emam] El Emamn, Khaled and A. Güneş Koru. “A Replicated Survey of IT Software Project Failures,” IEEE Software, Sept/Oct 2008 (vol. 25, no. 5), pp. 84-90.
[Evans] Evans, Michael W., Alex M. Abela, and Thomas Beltz. “Seven Characteristics of Dysfunctional Software Projects,” Crosstalk (The Journal of Defense Software Engineering), April 2002, pp. 16-20.
[Fairley] Fairley, Richard E. and Mary Jane Wilshire. “Why the Vasa Sank: 10 Problems and Some Antidotes for Software Projects,” IEEE Software, Mar/Apr 2003 (vol. 20, no. 2), pp. 18-25.
[Glass] Glass, Robert L. “IT Failure Rates – 70% 0r 10-15%?” IEEE Software, May/June 2005 (vol 22. no. 3), pp. 112 & 110-111
[Jones 96] Jones, Capers. Patterns of Software Systems Failure and Success, International Thompson Computer Press, Boston, MA, 1996.
[Jones 06] Jones, Capers. “Social and Technical Reasons for Software Project Failure,” Crosstalk (The Journal of Defense Software Engineering), June 2006, pp. 4-9.
[Lister] From slides from a talk.
May] May, Lorin J. “Major Causes of Software Project Failures,” Crosstalk (The Journal of Defense Software Engineering), July 1998, pp. 9-12.
[McConnell] McConnell, Steve. “The Nine Deadly Sins of Project Planning,” IEEE Software, Sept/Oct 2007 (vol. 18, no. 5), pp. 5-7.
[Reifer] Reifer, Donald J. “Software Management’s Seven Deadly Sins,” IEEE Software, Mar/Apr 2001 (vol. 18, no. 2), pp. 12-15.
[Rost] Rost, Johann. “Political reasons for Failed Software Projects,” IEEE Software, Nov/Dec 2004 (vol. 21, no. 6),pp. 104, 102-103.
[SD Times] Rubinstein, David. “Standish group report: There’s Less Development Chaos Today,” March 1, 2007 (found at http://www.sdtimes.com/content/article.aspx?ArticleID=30247).
[SPC] Software Productivity Center, Inc. “Root Causes Of The Most Common Project Problems,” http://www.spc.ca/resources/process/problems.htm
[Standish 94] The Standish Group Report, 1995, http://www.cs.nmt.edu/~cs328/reading/Standish.pdf
[Thayer] Thayer, Richard H., Arthur Pyster and Roger C. Wood, “The Challenge for Software Engineering Project management,” IEEE Computer, August 1980 (vol. 13, no. 8), pp. 51-59

What If Software Quality Got (Lots) Better?

2009-09-23T11:08:00.000-04:00

I have been reading various things about testing and the QA role on Agile teams. This question occurred to me the other day as a result, I believe. Just what would we do or how would things be different, in software development, if the overall quality of software increased by a lot? There are a number of aspects to how one could answer this question.

Presumably, relationships with customers would improve significantly and people might be inclined to spend more money on software expecting that they would not be as disappointed in the results. Note that I did say “as disappointed” since others things beyond raw quality would affect a customer’s view of the results. For most, improving “quality” would go beyond defects, but let’s just stick with that now. Let’s assume the results include acceptable usability and coverage of functionality and that what we are concerned with is failures of the software to perform that functionality. I don’t want to get too hung up on a definition of “quality” at this point.

Of course, another question may be by how much would the improvement be that I am thinking of (i.e., what does “lots” mean)? That’s one of the things I’m going to leave up to those who comment on this post. What would the change have to be for you to think it is significant compared to today?

One thing that I would expect to be different would be some change(s) in process. Indeed, such change(s) would likely be required to even get to “lots.” What change(s) would you see occurring to get there are as a consequence of having gotten there?

My next thought was, what would happen to our view of the role of test staff? Would they just be able to do more comprehensive testing (since more software would work correctly at a basic checking/testing level)? Would it mean they could do different testing than required if confidence was lower in potential quality? Or would your definition of “lots” mean so few defects would exist that (at least some portion of) test staff would be moved to some other kind(s) of work either related to quality control or quality assurance or a actual career change/move?

Finally, and this is certainly not anything new to think about, what would we need to make a big enough improvement in quality to call it “lots”? Now there are many things being done today related to quality improvement, but what new thing(s) might have to occur to make a leap to “lots”? Would it be just more diligent practice of what we already know and (try to) do? Or is there something significant in the nature of how we create and check/test software that is needed?

I’d be most interested in what people think.

The Problem Solving PM and Agile

2009-09-20T17:52:00.027-04:00

I had been thinking about this topic and making notes periodically. Then, earlier today, I was reviewing a chapter from a book (in progress) on coaching. Some of what it says reminded me of the topic of what a well-regarded Project Manager may face in becoming involved with an Agile effort. The chapter describes, among other things, the need for Agile coaches to restrain themselves from immediately trying to solve team problems.

Now the chapter seems to be written mostly from the perspective of how a coach would view their own behavior based on having been a “go to” person when problems need solving. Such problem-solving skill is something valued in project management. Indeed, success in solving problems helps make a Project Manager valuable and well-respected. So what happens when this is true, i.e., in their work, their problem-solving, fix-it ability helps define their organizational value, and they become involved with an Agile team/project?

One thought I have heard expressed is that, of many traditional roles on projects, PMs can be good candidates to become ScrumMasters or Agile coaches. The book chapter deals very nicely with advice to a person finding themselves in such a role when it comes to their own behavior. It provides good examples of occasions when a coach might be tempted to try to fix a problem right away based on their experience and their view of how the issue should be addressed. The “take it to the team” theme is covered nicely in the chapter.

That addresses the inward looking aspect of the “fix-it” approach. What about the outward looking one? That is the inward looking view of those around and above the new coach in the organizational hierarchy? If part of a coach’s organizational value has been based on their problem-solving initiative, what happens when they begin not to dive in and solve problems but work to help the team(s) to do so in most cases? Will they be viewed as failing to meet expectations, even “dogging it” in their new role by not having ready answers when issues arise in the team or are brought to them (or the team) from outside the team?

One of the things a person moving into a coach role needs to realize is that they have to coach in both directions. Coaching outwardly to the rest of the organization seems to me to be at least as hard, if not harder, than coaching a team. Presumably, teams are expecting to get coaching and, if trained reasonably effectively in Agile concepts, have heard that they, as a team, will be expected to identify impediments and do what they can, first, to address them before expecting others to do so. So a coach that holds back a bit and tries to point the team toward their own solution is probably not going to be looked upon too strangely by the team.

What about coaches who have been viewed as the “fix it” people and now must deal with people used to bringing them problems from outside the team? Now, the problems are really for the team, hopefully the coach is aware of them (and the team). If not, that’s an issue the coach should be addressing. But what if the problems aren’t team issues but organizational ones thrown their way which will impinge on the new Agile approach, yet which others expect the coach and team to “adapt” to somehow and “fix”? (By the way, I have heard folks latch on to the “adapt” word and assume it means the team figures out how to deal with whatever gets thrown at them.)

What will it be like for the new coach to have to coach those that were previously their peers or managers? When those folks, used to bringing the coach problems to solve, find the coach using an Agile approach with them, it may be a shock initially. This may be more true if the other people are not expecting to be involved in any Agile process, There may even be resentment and a feeling that the new coach is trying to “manage” them. (I know there’s a lot of material or “managing your boss,” etc., but the openness of an Agile approach seems somewhat more direct than much of this advice often suggests.)

Most of my experience in coaching has involved being brought in from the outside, so I have had no prior PM fix-it expectations about my role. However, I will say that one thing that has helped me when I have coached other coaches who are in this position has been to work at least as much with the management and peers of these other coaches to help them understand the Agile approach. This includes the roles and responsibilities of teams and coaches (and management) as well as how issues and impediments are handled.

In particular, I try to get peers and managers to understand two things:

1) to get the most value out of a team’s effort, those outside the team should be prepared to become involved in more problem-solving (i.e., impediment clearing) than they may have previously expected, so teams can focus on meeting iteration functionality expectations;

2) an important goal of an Agile approach is to build teams that can effectively solve their own problems (but not all the organizational ones thrown at them), developing larger numbers of people who are, in fact, problem-solvers.

Indeed, this last point is a key reason the coach does not jump in, even if they can solve the problem. Having the team do it is usually a valuable experience in the team working to manage their own environment (to the extent they can). Agile or otherwise, relying on single individuals to be “problem-solvers” may mean you set up a number of single-points-of-failure should such people leave, transfer, etc.

None of what I have said, though, should be interpreted to mean that having individuals who can “get things done” is a bad thing. But my belief is that an Agile approach hopes those are the people who can, if they do not already, come to achieve this through the teams. This can lead to multiplying the number of people who develop skills in problem-solving that can be spread further through the organization. A benefit of taking an Agile approach is uncovering/developing people who are, after all, the kind companies say they want in the first place. Agile helps do that, but only if teams are given the chance to learn this. The temptation to find the “fix it” people and expect them, in the name of efficiency, to get it done faster rather than developing teams who can will, at the very least, retard Agile’s contribution to organizational improvement/growth and may, indeed, kill it.

Yet More Quotes from Twitter

2009-09-17T12:39:00.005-04:00

A third installment of interesting things I've captured from Twitter that people have said (including some of my own) either individually or as part of a thread of discussion (again sorted by first name):

Alistair Cockburn (not from Twitter but mentioned on it) - Management tells the workers to mutiny. The workers refuse. (A koan for agile development)

Andrew Meyer - The People, Process and Technology triangle, to me, is about balancing a path to successful execution of a project. Value is about determining if a project should be done.

Andrew Morgan - Be fearless with your day. Show up bold. Show up on purpose. Love every minute of life today. (Repeat daily until death)

Ben Rady - The first rule of productivity is accepting the fact that there are valuable things that you will just not have time for.

Benjamin Mitchell - Challenge of #kanban in s/w dev is to manage overlapping activities (analysis, dev) that may start work with partial information. Vasco Duarte - That's the exact same challenge that all processes face. #kanban is no exception. In #waterfall that was just ignored. #agile Benjamin Mitchell - Good point. It's also a difference when applying 'Lean' approaches to Prod Dev and Manufacturing. Goal of Prod Dev is to generate info. (which requires variation) to assist in making best future economic decisions so 'rework' can be ok. Goal of manufacturing is to produce within acceptable variation. Rework is generally waste. So depending on your view of s/w dev (manufact or prod dev) you'll take a diff view of managing 'rework'. My #kanban board surfaces this.

Bob Marshall - Is there any chance we can forego the "waterfall" epithet in favour of the more descriptive, objective, term "batch and queue"?

Bob Martin - to be a professional, you must not rush. keep your code clean. So clean it barely needs comments.

Brian Button - Is the most important skill in facilitating retrospective the ability to hear what isn't being said?

Dan Whelan – The Fifth Discipline and #systemsthinking has me thinking that agile/lean focuses too much on short-term value. Need to build learning orgs.

David Anderson - I find risk management is poorly executed. This is why I am going after it next as a root cause of #agile adoption failure. J.B. Rainsberger - Do you find risk management a sizable bottleneck? From here, it looks like not focusing on value limits everything. David Anderson - value is a complex notion. Board room wants predictable ROI more than revenue or profit maximization. if you ask "would you like more value?" answer: "yes". if you ask "would you like more value at risk of less predictable outcome?" answer: "No!" #agile appears to give less predictable outcome as traditional gives (false) impression of predictability. Hence #agile == risky! to remove this impediment we must show that #agile manages risk better than traditional to deliver more predictable outcome. J.B. Rainsberger - For most teams, most of the time, volatility of the cost of features depends on the design. Improve the design, less volatile. David Anderson - if you are using design to include analysis classification and architecture then i agree with you. Lean product design uses analysis classification & product architecture to design for variability/volatility. J.B. Rainsberger - I consider "architecture" simply to be design-in-the-large, and I don't know the term "analysis classification". I more mean that the crappier the design, the more the marginal cost of a feature depends on the state of the design.... Dan Whelan - I think of architecture as design decisions that are expensive to change. David Anderson - agree architecture is design-in-the-large. good analysis can classify functionality into areas of the design.

Deborah Hartmann Preuss - Toyota competency levels:: Assisted, Independent, Can make changes, Coach. #AgileOpen Thx! I like those levels!

Declan Whelan – I've had negative comments from mgrs and teams be de-motivated by unrealistic diagonal lines on burn-down charts. I do see the value of line in highlighting continuous flow. I find the shape of real actual curve sufficient for this, i.e. focus team on making curve flatter rather than having curve track to some ideal line. Deborah Hartmann Preuss - I have teams reflect on their own sprint "signatures" over time - improving? repeating same mistakes? Aim is consistency. I find that sprint burndown w/o task board is much less useful (& less used). If had to choose: task board. Lisa Crispin - I'm not a fan of burndown charts. Prefer task or kanban type board, visually you can see what tasks/types remain to be done. Scott Duncan - "focus team on making curve flatter" Flatter relative to what? Wouldn't that be some baseline even if it isn't drawn on the graph? "unrealistic diagonal line" I spend time coaching/training on what lines means, use of burndown, etc. Haven't had the issue. Declan Whelan - Flatter in that tangent of curve remains constant - i.e. only relative to itself. Perhaps splitting hairs ;).

Elizabeth Hendrickson – (On exercises about learning to learn) Simple & meta: have groups work together to solve a puzzle of some kind & debrief how they learned?

Elizabeth Hendrickson - Many have said this before. I'll say it again anyway: Source code, like inventory, is a liability, not an asset. Dave Rooney - Not sure I agree. Source code that hasn't been shipped to production is indeed inventory. Afterwards, it's documentation. Brian Foote - Tell us more. It would follow then that you feel that reusing source is like reusing diapers, noble sounding, but impractical. Chet Hendrickson - Good code can be a capital resource. 1 that allows us to build things of value. But it must be treated as one otherwise... Elizabeth Hendrickson - Consider: mgr insists on keeping 1/2-done feats in code base despite drag on productivity b/c they're "too valuable" to toss. Chet Hendrickson - reminds me of my grandfather's box of broken electric drills. He didn't need a new one, he had 5 already. I don't think it's all that audacious. Consider inventory as liability from a manufacturing perspective: http://bit.ly/YvQ1d.

Hillel Glazer - Consultant <> Contractor. Ensure you're being hired as a consultant, not a contractor, are you there to create outcomes or outputs?

James Bach - They way to kill curiosity is to *force* other people to learn what you think they will someday need to know.

John C. Maxwell - People change when they: HURT enuf that they have to; LEARN enuf that they want to; & RECEIVE enuf that they're able to.

Joshua Kerievsky - Low-grade sausages contain stuff you don't want to know about just like low-quality software.

Kathy Sierra - "self-promotion" need not be literal. Want people to see you're good at X? Promote "learning X" or "others I helped do X" or simply BE... X

Kathy Sierra - Clarification on word "useful": it does NOT rule out entertainment or tweeting your lunch menu. Making my day even .0001% better? Useful.

Kathy Sierra - Good directions to your house include how to know I'm on the right road and how to know recognize when I'm not (& what to do about THAT).

Lao-tzu (via Bob McNeal) - Different perception?... What the caterpillar calls the end, the rest of the world calls a butterfly.

Larry Weidel -There are LAWS, PRINCIPLES, and PREFERENCES. LAWS are ALWAYS true. PRINCIPLES are USUALLY true. PREFERENCES are up 2 U.

Michael Bolton - @cory_foy "OH: 'We just need to go faster'" No, no! Work smarter! No, no! Work harder!

Michael Bolton - If the last question is "Am I okay with having no more questions?", then you're done. If you're not okay with that, you're not.

Michael Bolton - Testing is what you do when designing a check, interpreting the result of a check, and learning. Spell checkers *help you test* spelling. J.B. Rainsberger - Here's a clue: automated "tests" replaced the error-prone "guru checks output" anti-pattern. The clue is right there. Michael Bolton - Not always. You DO need a guru (programmer, tester, business person) to design and interpret the results of the check. James Marcus Bach - Non-sapient work can, but not necessarily should be performed by computer. Checking is a poor substitute for testing. In general, when checking substitutes for testing, bad outcomes happen. But it has its place. Regression CHECKING != regression TESTING. Latter /investigates risk/ of change-related failure, requires *new* tests. Michael Bolton - You can call unit tests and ATDD "tests" if you like; I don't mind. But if you only *check* your product, you may not be *testing* it well. Testing is explorative (probing) & learning oriented. Checking is confirmative (verification & validation of what we know.

Michael Bolton - The problem with maturity models: they assess "maturity" on based conformance, instead of independence and adaptability.

Michael Bolton "Don't mistake requirements document for the requirement. Don't mistake process manual for the process".

Michelle Sliger - The importance of facing the truth and saying Yes 2 reality, then no 2 denial.

Mitch Kapor - "Is the good enough the enemy of the barely ok?" Brian Foote - Nope. The barely ok is the enemy of everything.

Paul Dyson (via William W. (Woody) Williams) - "Scrum Masters remove impediments, Project Managers prevent them occurring."

Serge Beaumont - Agile is techniques at the Shu level, a framework at the Ha level, and a culture at the Ri level.

Skip Angel - Coaching = Not afraid 2 pull off band-aid of workarounds 2 expose infection of problems under surface. May hurt but needed 4 org's health!

Steve Keating - When we throw mud at others, not only do we lose a lot of ground, we also get our hands dirty.

Steven Keating - People don't buy your product for the value you put into it. They buy it for the value they get out of it. Sell that way!

Steven Keating - People quit leaders not companies. If you lose employees you don't have an employee problem, you have a leadership problem.

Tanmay Vora - One of the biggest challenges for "Human Resources" is to do justice to "Human" part of it and not get into routine policies/processes!

Tim Ottinger - An expert is a person who knows what all the mistakes look like, and that you don't have to make them.

Vadim Zaytsev - Optimist: the glass is half full. Pessimist: the glass is half empty. Engineer: the glass is twice the required size. Tim Ottinger - _COST_ACCOUNTANT_: glass is too large, Engineer: glass has 100% safety margin.

My Thoughts on Certification (and some related topics)

2009-09-16T20:00:00.008-04:00

Recently, there has been increased talk in the Agile community about certifications, pro and con, though it appears mostly the latter. It has also been noted that the IEEE Computer Society will work on an exam for state licensing of software engineers. Since I have had some experience/contact with certifications (e.g., CSQE, CSM/CSP, PMP, CSDP, my wife's PA-C), I have some idea as to how they work and feel that any certification effort should be able to explain how it addresses some common certification characteristics.

What It Means to “Certify”

First, though, it seems to me to be important to define what it means to certify something/body as there are implications just in this definition. Common definitions for “certify” related to professional credentialing include:

a declaration by some individual, group, organization (i.e., the certifier) that
some other individual, group, organizations possesses/has demonstrated some
quality, characteristic, knowledge, ability, skill, or combination of these.

Thus, it requires a certifier to grant the certification to an applicant. Of course, the value of any certification depends on the credibility of the certifier e.g., a professional association of some sort. For this reason, there are even certifiers who certify other certifiers. For example, auditing firms (“registrars”) are themselves audited by certification bodies. These latter bodies attest to the fact that a registrar conducts audits according to some standard, usually created by a Standards Development Organization such as ISO.

So, for any certification of any kind to have an meaning/value whatsoever there must be trust and confidence in the certifier. What has to be trusted is that the certifier has actually been able to confirm that the individual, group, or organization being certified has met the criteria established for the certification. There are a few ways in which this trust can be established:

the certification criteria are public and easy to understand (at least by those familiar with the scope of the certification);
the certifier states and can demonstrate how they verify that the criteria have been met;
those certified are recognized as actually possessing the quality, characteristic, knowledge, ability, skill, or combination covered by the certification.

In the latter case, this means that significant question/evidence is not raised after the fact that those certified do not, in fact, meet the criteria, e.g., people certified to some skill/craft capability actually cannot apply that skill/craft as defined by the certification scope and criteria.

Scope

As mentioned above, one important matter is the scope covered by the certification.

The scope really defines what it is that is being certified. This is often defined by what is known as a “body of knowledge” which represents the domain covered by the certification. Some certifications have “guides” or outlines to perform such scope definition since actual bodies of knowledge are usually considered to be the entire corpus of published material available for the domain (e.g., the Project Management Institute’s PMBoK or the IEEE Computer Society’s SWEBoK for software engineering).

For certain very narrow certifications, a body of knowledge might be contained within some single, perhaps large, published source. But, one way or the other, a scope and body of knowledge need to be defined so they can be publically assessed for others to determine just what the scope of a certification would mean.

Another aspect of scope is whether the certification is largely based on demonstration of knowledge alone or application of that knowledge. This is where a number of people complain about some (levels of) certification that currently exists in the software field: it can be achieved by mostly passing a test (plus perhaps some minimal years of employment). Any professional certification (from medicine through hairdressing) usually involves some actual demonstration, before already certified professionals, that a person can apply the knowledge that may have been demonstrated through a test (or series of them).

The controversy will be because folks will disagree on the boundaries of that scope, especially in a skills-based certification. For example, the SWEBoK and ASQ’s Certified Software Quality Engineer BoK both note that there are important areas relative to being an effective employee and/or professional that are not covered by their BoK. They even admit there are technical and domain knowledge their exams don't cover that can and/or will matter in given work situations. So you cannot cover everything that could conceivably be important in working in a given situation.

Finally, it would also be necessary to indicate whether there are levels of certification, e.g., entry, experienced, expert or some such scale. Now each level could also be established as an independent certification on its own, of course. But it will need to be part of the definition of scope. Having different levels may help with the difference between demonstration of knowledge vs of application of that knowledge.

Nonetheless, it is important to be clear as to what scope any certification would cover.

Criteria

While not without controversy and not automatically simple to do, it can be much easier to define the scope than to actually certify one against it. Therefore, a second important matter is defining the criteria to be met to become certified. Different certifications have different criteria, but most professional certifications contain some form of the following types of criteria:

Evidence of some (a) training such as industry courses or formal school classes, (b) educational degree based on some curriculum of classes/ having been met, and/or (c) experience gained, often under supervision of already certified individuals.
Examination/testing independent of that associated with educational achievement since people in most fields can get the educational credentialing from many sources. Developing (and maintaining) these takes substantial time and effort.
Possible observational input from existing, certified holders of the certification based on the applicant. Not all certifications do this; however, most "professional" ones do.

The extent of any or all of these criteria may depend on any level of certification as well.

Now 2 is where must controversy exists in current software-based certifications. Most tests turn out to be multiple-choice efforts and focus on factual knowledge, not application of that knowledge. I do not see how the latter can be avoided to have any test be of real value. Of course, multiple test types could be used for multiple certification levels, e.g., a "trainee's" being largely or completely knowledge while more experienced levels having more applied expectations.

The key here is what out of all the Body of Knowledge should be covered in any test and how answers can be made "objective." Of course, even “objective” tests are “subjective” in that they represent someone’s opinion about what someone else should know. Can this even be avoided? And in judgment-based professions, can answers even be “objective” except in very narrow, technical ways?

The observational component of 3 could be like residency in medicine. Some one gets through medical school (which requires no small practical demonstration of ability) but are now to be observed by those who are already MDs. That is, for some period of time, applicants must be under the observation of or otherwise associated with someone who is already fully certified to practice the profession independently. Engineering has a somewhat different model as not everyone graduating from an engineering school ends up being licensed as a Professional Engineer. But, in one sense this is not dramatically different than their being other medical professionals certified to be other than a full MD, e.g., various nursing licenses, Physician Assistants.

Validating Scope & Criteria

Having determined what the scope and criteria will be, the next step is to validate these. This requires that there be openness in the definition of and rational for the scope and criteria and review by the professional community. The latter would be people who are currently believed to represent those who would deserve to be certified, i.e., expected to be able to meet all the criteria. (For an update to an existing certification, this would be people already certified under the prior criteria.)

The job of these people is to ensure the criteria are truly relevant to the scope the certification claims to cover such that most people who pass deserve to do so and few people who deserve to pass end up failing to do so. This can be done through peer review of the certification materials and knowledge base as well as these peer reviewers taking sample exams and assessing whether the results suggest the tests effectively cover the scope.

Verifying Criteria are Met

After this is done, the next step is to actually carry out the certification effort and verify whether the criteria are met by individual applicants or not. Concerns here are:

How is training, education and/or experience to be verified? Copies of certificates of class attendance? School Transcripts? Letters from institutions?
What pass/fail rate will exams have? Must 100% of all answers, results, etc. be "right" or is some, less than, 100% rate okay? And what is the correct <100%>
What kind of “objectivity” can be brought to bear on observational input? Multiple inputs rather than just one observer’s? Indeed, can this be (and do we want) purely “objective” input for this criterion? Is any input beyond the body of knowledge relevant for observational input?

For example, PMI actually audits some portion of applications on point 1.

Ongoing Certification Responsibility

Once an individual is certified, the question remains how long that certification is considered valid. For some certifications, there is yearly recertification required. Others require it every 2, 3, 5, or 6 years. This can take the form of accumulating continuing educational credits, demonstrating continuous practice in the profession, as well as, at one of the longer periods of time, taking another examination. All this needs to be defined as part of the (re)certification criteria so applicants, and others, will know what ongoing expectations exist.

------------------------------------------------------------------------------

There is a lot to consider and all of it may not be possible or make sense at this point. But I think these ideas cover most of what passes for meeting "certification" requirements in most fields. However, some other topics deserve a bit of mention as they are definitely related to certification.

Where Licensing Fits In

Licensing is a government activity. It is where some governmental authority grants some individual, group, organization a “right” of some sort. For example, in the USA, states license people to practice law, medicine, etc. This is usually a pro forma event, accompanied by licensing fees paid to the state, given that the state trusts the certifier. For example, in the USA, this would be the AMA for doctors or the ABA for lawyers. Licensing is not an issue I want to discuss here in any depth. I felt it was important, though, to make the distinction since not all certifications necessarily lead to licensing concerns. One criteria when they do is if the health, welfare or safety of the public can be affected by activities covered under the certification. The IEEE effort noted above is occurring because matters of software development in areas such as aerospace, medical devices, financial transactions, and the like can have significant impact on the public.

Training Separated from Certification

For true objectivity and ethical reasons, a certification body should not mandate/control its own training/education as the only source to meet its own criteria. In any certification program, money will always end up having to be involved, one way or the other. This is why a certification body should not have some vested interest in consulting/training related to the certification.

The same idea applies to a company that would both perform audits to some standard and supply training/consulting to that same standard. What are the odds that, if you pay for the latter, you're likely to fail the former? At least, that's the conflict of interest question. Can a group claim to effectively train/consult with a company on some standard, then turn around and fail them because they followed the training/consulting? You get the idea. Most audit certification bodies look dimly on registrars that have too close a relationship with training/consulting firms (or arms of the registrar firm).

Value of Certification: Who Cares?

Certainly, those who would make use of the products/services of organizations employing/using certified individuals could care how effective certification of those individuals is conducted or, indeed, that certification exists at all. Individuals might care about certification as a distinguishing characteristic for themselves compared to those who are not certified. Naturally, as noted above, the credibility of the certification will matter. As some note, people with too many certifications may raise questions with potential clients/employers who may wonder how valid such certifications are if a person can maintain so many of them concurrently.

If the goal of certification is to ensure (a loaded word itself) people can be (not even "are") more effective in a given domain covered by the certification scope, perhaps there are other ways to make people more effective rather than worrying about how to judge if they are? Not that the latter isn't important, just that the former might be easier to accomplish, i.e., designing/promoting excellence in education/training could be easier than excellence in certification.

[And as a final note, almost everything said above regarding certification of individuals could apply to assessment programs related to, for example, “how agile” individuals or organizations are.]

So why the plural?

2009-09-15T21:53:00.006-04:00

Last month, I noted that I had a blog over 2 years ago that was cut very short. I reposted a somewhat revised version of one of the posts, Here's the other initial post, also somewhat revised, explaining why I use "Qualities" plural form.

--------------------------------------------------------------------

A lot of websites and blogs use the singular and, at the time, I was fishing around for something that wasn't already in use. One day, I was also working on something related to the ISO standard (ISO 9126) on software product quality characteristics. At one point I was writing something along the lines of "the non-functional qualities against which software can be compared" and, then, later, "these software qualities." That was even longer ago and I had the idea of using "software qualities" as a website since then.

Since I mentioned ISO 9126 and since it was the "inspiration" for the title of the original blog (and then this one), let me say a few things about this standard in case you are not familiar with it. It addresses what are usually called categories of "non-functional requirements" related to software and which are often known, for short, as the "ilities" because of how many of them end, e.g., maintainability, reliability, portability, etc.

Now you can certainly argue with how they are organized or what terms are used -- I've collected a list of over 100 such terms that I use on one slide in one presentation I give just to point out that the names are not the really important thing. However, having some sort of "model" of quality attributes seems to me to be quite valuable in considering how you plan to achieve quality in software.

Very often, non-functional requirements do find their way into formal requirements specifications, but not always. Sometimes they come up in less formal discussion. Sometimes, and this is the dangerous part, they do not come up at all as they are assumed by a customer. Well, they do come up, but usually after the customer gets their first substantive look at or experience with the software and they say, "What do you mean it doesn't....?"

Being explicit about such characteristics when the requirements are being discussed can avoid a lot of anger and frustration (not to mention cost) later on. So having a "model" that addresses the kinds of non-functional characteristics important to a specific product (or release) is a way to address them in an organized fashion. It can also demonstrate some proactiveness on your part as a software developer.

--------------------------------------------------------------------

That's what I said back then and it's still behind my thinking these days, though, as I noted in my first blog post, I've adopted a more Kano model approach to the "Qualities" ideas which covers functional and non-functional requirements expectations.

There is No Definition of Agile – Hooey!

2009-09-14T16:09:00.000-04:00

Periodically, I’m at meetings or conferences and hear someone saying something like: “There is no definition of ‘Agile’. It’s a bunch of practices and techniques and methods, but no real definition exists. To that, I always say, “Hooey!”

Well, I don’t use that word. Instead I point out that before the Snowbird meeting in 2001, there was no definition for "Agile" with the capital "A". The Values in the Manifesto that came out of that meeting and the formulation of the Principles that followed shortly thereafter are, to me, what define "Agile." Matters of method vs practice vs technique are another issue, but the Values & Principles are what represent a definition to me.

Now you may say these are too vague. Indeed, I have had folks tell me their (or their clients) consider the Manifesto to be “content-free.” That is, the four Values are statements that hardly anyone could disagree with as generally desirable, but they have no substantive meaning. In asking further about this, it usually trails back to the Values & Principles not saying how to do anything, i.e., no specific practices or techniques are included.

Now, lots of definitions don't tell you how to do what the word(s) define. But the better explanation is simply that the Manifesto came together based on the common beliefs of many practicing individuals – each with their own (often related or similar) practices – about how software should be developed. As Alistair Cockburn as noted, this means Agile “has no ‘center’: There is no “center” to agile development. There’s only proximity of similar-but-different personal value systems coincidentally producing similar recommendations.

This does not, in my estimation, mean the Values and Principles do not serve to “define” Agile. They are, to me, a touchstone against which to measure what is done, not a description of how to do. Practices and behaviors that more closely adhere to the Values and Principles are more “Agile” than those that drift further from them. Indeed, my belief in this is why one of my first posts on this blog was entitled “Agile Training - Values & Principles Are Essential”.

[A Side-Note on Agile Assessment Models --

It’s also for this reason that I believe any form of assessment model related to an organization’s readiness for or practice of an Agile approach must use the Values and Principles as the top-level concerns. If you are familiar with the (CMM®/)CMMI® model, you know there are (Key) Process Areas, followed by Goals, followed by Practices. From any Agile perspective, the Values and Principles would be, in effect, the (K)PAs. The various methods’ practices and techniques would be the Practice level where what is stated are recommended ways to achieve the goals, but not required, per se. What is missing in Agile is a clear statement of the goal-level.

In performing an assessment, an organization would show that they achieve the Goals by following some identifiable practices/techniques. The Goals would need to be satisfied, but a variety of practices might be used to do so, though a given assessment model could offer examples of practices and techniques that would be considered effective in doing so.

I mention this, because there are efforts to define and use such Agile assessment models already and I am not sure they can work as well as would be hoped without such a set of Goals having been clearly defined. In some instances, it appears models have established practices as goals, i.e., if you do this or that practice, you get some sort of “points” for that. I would hope model creators spend time coming up with a clear statement of the Goal level.]

Why Organizations May be Uncomfortable Getting the Kind of Employees They Say They Want

2009-09-12T17:31:00.002-04:00

Organizations often say they want individuals to have the characteristics Agile calls self-managing/organizing, so why do organizations sometimes doubt/resist teams doing so? To me, an Agile approach would seem to encourage developing exactly the kinds of people organizations claim they want: empowered, skilled, motivated, responsible, concerned with quality, responsive to customers, etc.

I asked about this on Twitter. Here are some of the responses:

@skirk Because they (the resistors) don't understand the changes involved and are afraid. Change has to come from within.

@madwilliamflint Because in those instances organizations want 'self motivated' but NOT 'independent' which they see as unpredictable. Last thing they want is what they see as "rogue programmers." They mean "works w/o badgering."

@estherderby Managers lack knowledge of how to set appropriate boundaries & negotiate explicit decision-making authority, which is part of the reason they freak out. no boundaries --> unpredictable behavior.

@tottinge I think they want self-SUPERVISING members, not self-MANAGING members. They confuse the terms.

@YvesHanoulle I want my kids also to have an opinion and be independent. I hope to raise them so that they end up with a strong personality. But in the morning rush, when we are juggling to get them on time to school and ourself to work, it does not feel like the right time for me that they practice that. Managers also want their teams to become self-organizational because they heard it is better. But if they never had the experience of such a team, the first time feels very frustrating. Just like how I feel in the morning with my kids.

One idea I think came through the discussion was that statements about the kind of employees desired are more corporate in nature while resistance often comes individually. That is, what is said collectively and what people are comfortable doing individually are different things. It’s also likely that, while there is a desire for more initiative from workers, there are also concerns over when that occurs, e.g., a general license to make decisions about everything isn't desired.

I’m reminded of an Agile 2009 session I attended on “Boundary, Authority, Role and Tasks” which you might want to read.

What do you think are reasons why what Agile offers as potential people development might not be accepted readily?

An Accountability "Scale" and Agile Teams

2009-09-07T21:58:00.004-04:00

Over the weekend, I found some things from many years ago regarding "rules" for corporate survival. One of them was a list of phrases related to being accountable for one's actions and situation (and, at the other end of the "scale," acting the part of a victim). Now "motivational" posters have never been popular with me, but this list was interesting mainly because of the "scale" it represented. The list was seen on the office wall of one of the owning companies of a company I worked for at the time.

Make It Happen

Find Solutions

"Own It"

Acknowledge Reality

--- [Accountability starts above this point.] ---

Wait and Hope

--- [Being a "victim" starts below this point.] ---

"I can't" Excuses

Blame Others

Unaware and/or Unconscious

Again, the theory was that there was some motivational value in posting a list like this with its implied "scale" from worst (at the bottom) to best (at the top) attitude toward accountability.

So I offer it, not as motivational material, but perhaps as something useful in thinking about individual and team accountability postures as well as something around which an interesting discussion could occur. I know that this worked, in some instances, many years ago when I (and others) first saw and heard of it.

Of course, to do so means getting beyond the platitudinous responses, i.e., agree with the best and reject the worst along the scale. Then, there can be a useful discussion of the reasons an agile team (or members on the team) could/would adopt each of the positions along the scale.

And, years ago, one group of people where I worked created their own poster on a magnetic surface, put little pictures of each of themselves on individual magnets, and placed their magnets where they felt that day, moving them during the day depending on circumstances. Seeing where other people placed their magnets was a kind of invitation for others to ask them about why they were feeling that way. They did not keep this up for a long time, but it did open up people to talk more about how they were feeling and why.

More Quotes from Twitter

2009-09-04T11:51:00.005-04:00

Back on August 20th, I posted a long list of quotes from things people have posted through Twitter. Today, because I can tell it's going to be a lazy day for me, and for no other reason, I'm posting a second, shorter, set of such Twitter material. Again, listed in alphabetical order by first names and with some things by me.

?? (can’t remember where I heard it) – “If you fence people in, they become sheep.”

Alistair Cockburn - "Plans are great till they meet people"

Angelo Anolin - No matter how much bashing waterfall method gets, it has its share of successes prior to agile. Scott Duncan - Re: bashing waterfall & its "successes prior to agile" - 'course agile really suggests it isn't the method we should rely on.

anonymous (via Armond Mehrabian) - When there is in elephant in the room, introduce him.

Bob Marshall - Adopt the perspective that all sw development is a subset of product development and learn from PD folks like Reinertsen.

Bob Marshall - Limit work-in-process - not just development work but all work. Get Little’s Law working for you rather than against.

Bob Marshall - When I say (or think) "I have an idea" what I mean is "please look at the world again, from this new perspective, folks".

Brian Marick - Greater ease requires first moving in the direction of less ease. Shame, that.

Dave Rooney - Easier to make something small bigger, but not opposite. Scott Duncan - Yes...one of my main methodology pts! And also encourages what do I need to do better" thinking as opposed to "what do I want to throw out or get away without doing". Dave Rooney - Absolutely! By nature we're methodology "pack rats", loathe to get rid of any artifact or procedure "in case we need it"!

Dave Rooney - Yup - that's my business model!! :) Beginners are in the "shu" stage - I get them to "ha" and they fire me when they attain "ri".

David Anderson - @flowchainsensei the problem with Deming is folks refuse to believe that he's relevant to the knowledge work century. :-S Bob Marshall - @agilemanager True. Although I think that opinion's ltd to those (few) who've ever heard of him :-S And fewer want to own ways of working

Dee Hock (via Wally Bock)- "Haste never made time and waste never made abundance."

Dr. Seuss (via Mike Cottmeyer) - "Be who you are and say what you feel because those who mind don't matter and those who matter don't mind.”

Earl Everett - Scrum is not the name of the 'ball game', it's rugby. Played well, rugby is a very agile game, mentally & physically. Scott Duncan - True, but once you're asked where the word comes from, Rugby gets dragged into the discussion, unfortunately. But your point is good in that I think we should move the conversation from game to qualities as you describe. Earl Evertt - Also, rugby is a highly collaborative and fluid game, and leadership comes from different people at different times. Scott Duncan - Unlike many sports we get to see in the USA, Rugby is very much a team game. Even moreso than football/soccer, I think.

Esther Derby - having leadership strengths != being "the leader." teams need _leadership_ which can come from diff ppl at dif times.

Esther Derby - Myths about pay: Labor rate = labor cost. Labor rate is easy to count; labor cost, not always so easy. Definitely not the same thing.

George Dinwiddie - Carpenters don't argue whether a hammer or saw is the better tool. Why Lean vs Agile vs Kanban? Goal is to build a house. Tim Ottinger - Carpenters do argue about Case v. Cat v. Deere v. Holland. Also, saw v. hammer is clear-cut. flow v. piulse is harder. Scott Duncan – Carpenters don't argue hammer vs saw since they can't do the other's job. Individuals sure have their fav hammers, though.

Henry Ford (via Deborah Hartmann Preuss) - "If I'd asked my customers what they wanted, they'd have said 'a faster horse.'” Dale Emery - Maybe Ford didn't know to ask the next question: "If u had a faster horse, what would that do for u?"

J.B.Rainsberger (via David Hussman at Agile 2009, not Twitter) – “Drive the cost of failure to 0, so we can fail a million times.”

James Bach (and Michael Bolton) - We think a check [as opposed to a test] is "an observation with a decision rule that can be performed non-sapiently" example: junit asserts. So, a lot of what Agilists call testing, we would call checking. But to create checks and react to broken checks requires testing skill.

Joan Koerber-Walker - Why Do We Ignore "Best Practices"? Scott Duncan - 'Cause we don't think they are?

Michelle Sliger - Deming says don’t blame the people, it's the system. My question: who forged the sys? PEOPLE. Who's going to chg the sys? People. Let’s get busy. Dennis Stevens - I feel sorry for the system. People are always abusing it and gaming it. It just isn't nice. Skip Angel - One more: Many orgs expect mgmt to fix systems, but best ppl are those that are closest to problems system is trying to fix. Esther Derby - even when U can't chg the system, U can chg your response to the system from any point in org.

Mike Schubert - Agile Development isn't doing things faster - it's doing the right things sooner (which may make you appear ... faster).

Mitch Kapor - The perfect is the enemy of the good, & the good is the enemy of the good enough. Is the good enough the enemy of the barely ok?

Paula Thornton (via Grant Rule) - Toyota "not normal corp business model...they've learned how to learn. GM makes cars. Toyota makes people who make cars" Grant Rule - If Toyota has "learned how to learn" by "mak[ing] people who make cars" who in the software industry makes ppl who make effective systems?

Scott Duncan - And instead of a CSM test, what about a CSP-like statement that CSM "graduates" would have to submit and have judged?

Steven M Smith - A TEAM without the means to score product value is missing explicit agreement with their customer(s) about significance. A TEAM obsessed with product quality IME runs out of funds, which forces shipment, which results in a product with little value. A TEAM obsessed with time to market IME operates on hunches and ships quickly with the hope that its product has value.

Steven M Smith - Tell me my idea is wrong and I'll think that info is useful. Assist me to transform the idea so it's right and I'll feel helped.

Steven M Smith - When a TEAM reaches consensus, ideally IME each member has agreed to actively rather than passively support the decision. Most TEAMS don't use consensus to make decisions IME, which results in many members failing to actively support the decisions. a TEAM that uses consensus to make minor decisions IME wastes its time: Empower individuals or sub-teams to make minor decisions. a TEAM member is guilty of fraud when they participate in a consensus decision and refuse to support it to an outsider.

Sydney J. Harris (via Chris Moy) – “The whole purpose of education is to turn mirrors into windows."

Tanmay Vora - In the game of excellence, if you have "anger" and "ego" on your side, you don't need an opponent! :)

Tim Ottinger - How many orgs consider management "the art of getting people to work more"?

Tim Ottinger (actually from a workshop at Agile 2009) – “My backlog is terribly frustrating. The users? Abusive, berating. Id like it all first, but will work in small bursts, if you let me make money while waiting!”

Tom Gilb (via Benjamin Mitchell) - "A software programmer is not necessarily an engineer in the same way a bricklayer is not necessarily a construction engineer"

Vasco Duarte - Irony is having the first quiet moment of the day in a canteen with 200 other people after heavy work load in an office alone!

A Mononumerosis Example (and Measurement Scales)

2009-09-03T18:20:00.008-04:00

Mononumerosis, according to Wiktionary is “The oversimplification of a metric by using a single numerical value to characterize a complex phenomenon or system.”

My example involves the very common practice of doing surveys, asking people to rate something on a 1 to N scale, then reporting results using an “average” value for that something.

Really, there are two bad things at work here, from a statistics and survey perspective, I believe. Let me deal with the one that is not the main point of this post, but important for people to consider. Once again, Wikipedia covers the broad subject, but I’ll just explain it briefly.

The Scale Problem

Most 1 to N scales have nothing numeric about them, except for the fact that they use numbers as symbols for points on the scale. It would be better to use words to indicate what the points on the scale mean, since the scale is really ordinal, at best.

That is, from left to right or right to left each position is considered higher or lower (better or worse) in some “value” than those around it. But there is no guarantee the space between them is mathematically equal (or it would be at least an interval scale), so it cannot be a ratio scale, where multiplying and dividing are legitimate operations.

And, that’s the point, using numbers to represent an ordinal scale then adding up values and dividing to get an average is, technically, meaningless. (You can count instances of such points on a scale and report how many results you got for each point on the scale.)

Of course, this is done all the time on customer surveys, conference feedback forms, the ratings on Amazon, etc. And all such examples seem to end up with an average rating for the questions on the survey. (Amazon, at least, shows you the counts for each “star” value as well as the whole feedback statement from those doing the rating.)

Another issue with ordinal scales is that there is no way to be sure one person’s 3 really is the same as another person’s because the surveys often do not place any substantive interpretation on the points to help you judge where your sense of evaluation for that question would fit. But, enough of that…you get the idea.

The Results Representation Problem

This is the more serious issue with the mononumerosis question. Let’s even say you could legitimately do adding and dividing and get an average. Does that tell you an accurate story about what all the respondents taken together felt? Or does it represent some imaginary respondent’s evaluation?

Here’s a few examples.

Let’s say you have 3 data sets of 10 responses on a 1 to 5 scale each with the following numbers of each data set being the number of 1s, 2s, 3s, 4s and 5s for each data set:

Bell - 1, 2, 4, 2, 1
Flat - 2, 2, 2, 2, 2
Camel - 0, 5, 0, 5, 0

This will give you an average of “3" for each.

I’m sure you can see from the data itself that “3” isn’t the same as “3” isn’t the same as “3” when it comes to the actual sense of what responses to the question would mean.

Here they are graphed in two ways (both showing the same thing, but one might be more meaningful for you than the other):

Each shows the very different impression of what the sets of responses might mean.

My point is that it matters how data is represented based on its scale and distribution. So watch out for mononumerosis (and scales) when you are given survey results.

Iterative Development Happens in Your Head, not on the Calendar

2009-09-02T08:21:00.003-04:00

Just a quick note today that I actually posted as a "comment" on another site a week or so ago, but decided to update just a bit as I thought more about it.

One of the little things I note that holds groups back, technically, from be more effective in their Agile practices is their concept of what it means to do development iteratively and incrementally. I hear and read, quite often, that teams fall into mini-waterfalls during their iterations. They take days to complete individual pieces of functionality, resulting in testing often bunching up at the end of the iteration. When I have asked developers or testers about this, their view of being iterative and incremental seems to be applied at the iteration as a whole, not their own work. Thus, being iterative and incremental has a "calendar" focus.

What they are not doing is looking at the work as a series of daily "episodes" (as many people have described/called it). They are also not engaging in TDD and/or pairing, since the ~2 hour recommended pairing time frame and the test-first approach would drive them right to a shorter cycle of creating and testing software. However, getting people to implement either of these practices can take quite a bit of effort since they require a level of collaboration which people often have not experienced in the past.

So, if there is resistance getting people to pair or practice TDD, I suggest another thing they can try which allows them to work more individually, but still in an iterative/incremental fashion of short duration. The goal is to get more frequent delivery, within the iteration, by having people think in minutes or hours rather than days, including testers. I've done this by simply asking folks to considering the following as an approach to developing a "module":

1) Create a shell with just the interfaces to other modules and test that. (Ada, for example, used to be (I'm guessing still is) great at being able validate a whole system of modules like this to ensure all interfaces pass the right number of and type of variables, for instance.)

2) Create the I/O "calls" and test them. (Yes, I am not presupposing use of OO development as my first couple of experiences with Agile iterations involved COBOL and assembler on IBM mainframes.)

3) Create the "control structure" for the module and test that. (Perhaps the hardest part of the coding/testing effort because of the combinations involved, so I even suggest doing this in increments, usually from outside in.)

4) Add the sequential data manipulation operations and test them. (I also suggest doing this incrementally based on the control structure, usually from inside out, in this case.)

The result will be shorter and shorter periods of development and test, producing earlier confidence that something "works" and closer, more frequent, collaboration between developers and testers.

Now, I'll admit that a waterfall can still be said to exist, but it is more "mini" than before and is headed in the right direction.

This also takes a few iterations to get rolling, but people began to see what being iterative and incremental, from an Agile perspective, could mean instead of just at the iteration level.

[PS: And, yes, I know about vertical slicing and Alistair Cockburn's "Walking Skeleton" as other approaches. Also, great ways to get across the idea of working in short bursts. Which is best may depend on the team and what they respond to. The idea is to get them to respond and move to the shorter time frame as their mental model for the implementation cycle.]

Defining "Agile"

2009-09-01T14:36:00.004-04:00

Yeah, I know...hasn't this been argued to death?

Folks have criticized agile ideas by saying the whole approach is vague because there is no definition of what "Agile" means. Other folks fall back on the dictionary definitions of "quick and easy of movement" or something along those lines. Individuals known in the community have said there isn't, and likely shouldn't/can't be a definition, e.g.:

Scott Ambler has said - “the agile community will never settle on a common definition for what agile is and more than likely are smart enough not to even try.”

Alistair Cockburn has said - "finding the 'true center' of agile can’t be done. There is no 'center' to agile development. There’s only proximity of similar-but-different personal value systems coincidentally producing similar recommendations."

Of course, people have and are trying to define agile (or, at least, how agile one is), sometimes through standards work, sometimes through assessment and capability scales, sometimes through tests and certifications.

Alistair's point comes from how the term came into existence as a description for what a number of people felt they shared in common with regard to software development ideals. Since what emerged when those people met was from many people's perspectives, each with their own practices and techniques, a "center" wasn't likely.

But, to an extent, I must disagree that there is no definition of "Agile" since what that group of people did in 2001 was precisely that. When someone says, or I read someone state, that there is no definition of "Agile," I must point to the Manifesto's Values and Principles. Before those people created the Manifesto and the subsequent principles, there was no definition for "Agile" as we use it in software (and elsewhere for some of us) today.

It is my belief that the Manifesto's Vs & Ps define what Agile is. Agile is an adherence to those Vs & Ps in getting work done.

Now there are many practices and techniques one can employ that may be closer to or farther from the Vs & Ps. But that's about the "how Agile are you" question, not what it is.

Whenever I have taught Agile intro classes, I start with the Vs & Ps and try to make sure attendees understand what the Vs & Ps say (and what I think they mean in practice). When talking about specific methods and practices, my goal is always to point back to the Vs & Ps to say, in effect, "and this is how method X [and/or practice Y] implements Z" (Z being one or more Vs & Ps).

Now there are surely many methods and practices that can be used to implement given Vs & Ps, but I think there is a "tether" back to the Vs & Ps. It may be made out of Bungee material so, if you really stretch it far, you'll get snapped back.

Some people may cut that tether, either accidentally or on purpose, of course. When trying to fit something new into an organization, "tailoring" methods and practices seems quite the "practical" approach. But if you don't honor the tether, i.e., you cut it, you can end up rather far from the intent of the Vs & Ps and miss the whole point and value of the methods and practices.

I don't want this to sound like some "if you failed at Agile, you didn't do it right" rant. I certainly think, like anything, Agile might not work in your environment. I do think that, if you start off focused on the Vs & Ps, you'll have a better idea, sooner, if that is likely to be true. And you'll know whether any "adaptations" or "tailorings" you make are likely to draw you further away from the intent and benefit of the Vs & Ps.

So, really study the Vs & Ps. Talk them over with people in your organization. Talk to people doing coaching and training about what they think the Vs and Ps mean/imply. When you've done this, then make a commitment to some method/practices and try them out. Doing this investigation shouldn't take long, and you will certainly save time and money later by doing so. You'll also avoid antagonizing and stressing a lot of people in your organization by making sure they understand what Agile means.

One of the "Something Old" Variety

2009-08-31T20:07:00.004-04:00

Back in 2007, I started a blog called just “Software Qualities.” Not long after, I needed to change jobs and ended up in a rather restrictive IP situation which caused me to stop blogging after hardly any posts – 2 to be specific. So while I collect myself after Agile 2009 and having caught up on 4 days of meeting notes, I thought I’d repost at least one of the original posts. I've posted it as it was to avoid being revisionist since I don’t think that differently today. So here it is, over 2 years later.

--------------------------------------------------------------------------------

There are a lot of places where people can, and do, write about quality and about software quality in particular. What I hope to do with "Software Qualities" is write about (and have people write back about) ideas on what quality (mostly related to software, but not exclusively) means from a product, process, personal, professional, whatever perspective.

Why yet another place to go on about quality? Because I think it's very important and not just some theoretical subject (though there's plenty of theory out there).

What (finally) prompted me to start this blog was a fortune cookie message that said "Give to the world the best you have and the best will come back to you." Though it sounds a bit 60's "instant karma"-ish, I think this is quite a practical idea, though not always easy to do on a day-to-day basis.

I wrote a letter to the ASQ's Quality Progress editor a few years ago (and may someday expand into an actual article) which suggested that, in all the years of training, consulting, and speaking, I have never run into anyone who believes they consciously head to work intending to do a bad job. However, I've run into very few people who claim they go to work intending (or at least really knowing how or expecting) to do an excellent job. I also suggested that I thought most people end up do "the best they can" to "get by" that day as various pressures begin to weigh on them.

Then, recently, I was at a local ASQ section talk about management and leadership, which was actually quite good. In the course of the talk, the speaker stated that -- influenced by the style of management and/or leadership -- people will largely do just enough on their job to "stay out of trouble" and that more than 50 percent of people are "disengaged" on their jobs. The speaker stated this was based on formal research and it seemed to match my experiences with people's answers during my training, consulting, and speaking.

Again, nobody intends to do a "bad" job, but whether they really feel they can do their "best," whatever that means to them, let alone do an "excellent" job, whatever that means to them, seems to be a real question.

On the other hand, Deming suggested people "doing their best" was insufficient to achieve quality as he meant it.[1] I've worked several places where that was, indeed, the performance value and suggesting that it was insufficient would produce a reaction somewhat like, "Well, if we cannot rely on people doing their best to get the quality we want, what in the world can we rely on?" I would claim that this approach suggested that, if we were failing to get the quality we wanted, it was because, somewhere out there in the organization, there were people not "doing their best" and this pushed pursuit of quality into a "moral" rather than "engineering" direction. Maybe this is why people just work to "get by" because failure says something very disturbing to them personally give the moral approach to quality that, I believe, it does not have to.

Just some initial thoughts to get this rolling, then.

Do you agree with any of this? Do you experience this yourself or do you see it in others? What do you think "doing your best" has to do with achieving "quality"?

[1] Deming's original statement was “It is not enough to just do your best or work hard. You must know what to work on.”

Agile 2009 Notes - Thursday

2009-08-30T21:37:00.004-04:00

The following material describes the sessions I attended.

Thursday AM –

Dan Mezick – “Boundary, Authority, Role and Tasks”

As I note in my Tuesday summary, Dan had said this session would be an expansion of some of the ideas from the more general Group Relations talk on Tuesday. Specifically, he spoke about four key elements in team/group performance and how “negotiation” about them (due to lack of clarity) creates/is waste. Vagueness about them causes anxiety and competition as two examples of wasteful energy. I found this session more strongly rooted in ideas I feel could be used and Dan made a point of discussing the clarity or vagueness around Scrum ideas as examples.

One by one, the four elements were addressed:

Boundary – defined as the “container for work” and was a topic Jurgen Appelo also mentioned in his Tuesday talk on complexity. Boundaries can be defined in terms of time (e.g., deadlines), physical space, territory, roles, resources, responsibilities, tasks, and resource access. Dan stated that the latter, resource access, can have a great deal to do with project success or failure. Waste can occur when boundaries remain fuzzy, though certainly flexibility in boundaries can be helpful. That is, rather than a hard line for a boundary there may be an area of broader boundary definition that can be tolerated and used effectively. But, ultimately there will have to be a point which is “outside” the boundary in order to use the boundary to define the work parameters. There is also what Dan called a “boundary culture” defined by how rigid or flexible the boundary is.

Authority – defined as “the right to do work” which can be formal (i.e., delegated) or personal (i.e., how one steps into and take more formal authority). Lack of clarity about authority can result in postponement of decisions (and Alistair noted in his keynote that this causes waste as people wait to see who will decide or what will be decided). This lack of clarity was described as “what you think they think” and “what they think you think.”

Role – definition depends on boundary, authority and task definitions. However, as with authority, there are formal roles (e.g., job descriptions) and informal ones taken on to fill gaps that formal ones do not indentify. What roles a person chooses to take depends a great deal, as one might imagine, on their personality and/or self-image.

Task – is defined, as Dan sad, by being unique to the current state of work, influenced as it is by time and personal difference. This is because, no matter how similar in name or general concept a task may be, it likely has never been done before exactly as it needs to be this time.

After this discussion, Dan went over Scrum’s roles, artifacts and ceremonies and we discussed how clearly or vaguely they are defined. For example, there are 3 defined Scrum roles: Scrum Master, Product Owner, and Team. The first two have specifically defined responsibilities, though exactly how they carry them out (and what else they may do) is not defined. The Team is a collection of many people with specialties and knowledge (and probably job titles), but Scrum does not define any of these. Thus, the Team must come to some understanding of how more specific definition will be accomplished and done in terms of boundaries, authority and tasks. The same is true for Scrum artifacts and ceremonies: there are some specific ones identified, but a large part of what they contain and how they are done is not specifically defined in Scrum.

Dan used a good word throughout this discussion of Scrum in describing what the official Scrum literature says. He called the information “canonical” and avoided the word “pure.” I liked this since the former carries with it the sense of being authoritative or accepted from a particular source which can be defined. The term “pure,” on the other hand, implies being free from impurity and somewhat restrictive, even carrying some moral tone. (And when one speaks of a person being a “agile purist,” there is a decidedly negative sense in which that is used, implying impracticality and unnecessary rigidness.)

Rod Claar and Doug Shimp – “May the Forces Be with You, Exploring the Forces Driving and Restraining Agile”

This was another workshop-style session addressing the forces that contribute to adoption and rejection of an agile approach. The room was divided into two teams: one to address the driving forces (in favor of adoption) and one to address the restraining forces (holding back adoption). There were also a pair of “judges” (not Rod or Doug) as we would be preparing presentations which they would rate on impact of the selected force and style of presentation.

I ended up on the team addressing the drivers for adoption. Each team spent some time coming up with a set of forces. Ours had 7-8, but we eventually had to choose the 3 top ones in our estimation and present each one, in turn, alternating with the restraining forces team. Forms if presentation included a limerick and then various forms of skits illustrating the forces. There were a number of clever ideas fort representing the forces visually, almost allegorically in some ways.

A few comments made during the session, sometimes in response to presentations, were:

“Will any methodology work if it is not possible to find/identify the needs of the customer appropriately?”
“Is every force restraining Agile really about lack of/inability to collaborate” (which depends on communication)?
Regarding distributed teams: “Latitude hurts; longitude kills.”
“Sharks work alone; dolphins, in teams” and can take on sharks that way.
“Everything thrown into a shark tank dies.”

Thursday PM -

Christian Gruber & Lisa Moore – “Coach Aikido: Lessons and support for abused coaches in hostile environments”

This session was another workshop format with 6-7 people in a group and consisted of group discussions around so scenarios we were given. While we were discussing, Christian and Lisa circulate among the groups, made notes on what they heard, and offered some suggestions on occasion. Then there was a bit of debriefing by each group around the scenarios which, at the start, were the same one, but then each team selected another from the remaining set.

Before we began however, Christian and Lisa described and demonstrated, in slow-motion, basic aikido concepts which are based on “control of the opponent’s violence for the sake of their spirit.” So it is an approach which does not involve combat, but rather a way to diffuse/redirect force by “blending” with your opponent. From a coaching perspective, the goal was to come up with ideas for how to use this non-confrontational approach to overcome negativity and resistance in agile settings.

Aikido’s “approach” is to understand your attacker and try to “guide them into a new posture or situation…dissipating their violence (for their sake)…while also exiting cleanly and safely” yourself. This requires taking “the correct stance.” We were also asked to consider four “mental postures” a person may take: beginner, deeply rooted, leadable and no-mind.

The first scenario describe a single, senior member of a team who criticizes all aspects of the agile approach and proceeds to do as he wishes irrespective of the team. The Scrum Master is not his manager and he is “protected politically” by his actual manager. His knowledge is also needed by the team. We started out discussion in the abstract, but a couple people had such issue (or had them in the past) and we ended up focusing on the real-life situation, which had more details that could be brought to bear on how to handle such a situation. There was some discussion in our team about flattery, when various ways to reach out to the person and bring them into the group were discussed. One or two people felt the attention they would get might be seen as catering to their, possible, attention-getting behavior by others on the team. In one of our real-world situations, it was noted that others were wondering why they could not be allowed to behave as independently as this person. Some of the recommended ideas, in the workshop notes, involved eliciting information from the person about what they see as the usefulness of their work methods. It as also noted that a coach would need to look broader than just this person to see if there is some encouragement coming from elsewhere in the organization suggesting doubts about an agile approach.

The next scenario our team selected involved a team where several members felt negatively about pair programming and how it “invaded their space” and wasted individually productive time. These people did pair, but didn’t engage well and made those they paired with “miserable.” One idea offered within our team had to do with trying to provide information/evidence of how this approach had benefited others (hopefully in that organization but from other companies if need be). Another idea was simply not to push on this issue, offering other forms of pairing and cooperative work or, if all else failed, not requiring the people to pair. This approach was suggested as it was felt there were larger, more critical practices to worry about in all likelihood. Our real world examples suggested the latter was true and needed more attention than pairing.

Jesse Fewell – “Growing PMI Using Agile”

Jesse has been at the forefront of building an agile community within the Project Management Institute (PMI). This talk described how this was accomplished, using an agile approach to the community’s work with the PMI headquarters staff. His slides have notes which cover his talk, so I won’t try to summarize all of that. Indeed, as an example, it was interesting to listen to, but the real payoff is what lies ahead as this community can now begin to approach local PMI chapters and bring agile speakers/instructors to them. The one phrase Jesse used that has stuck with me is “contagious culture of commitment.” This describes the dedication and effort he and others have put forth to make this all happen and how that attracted support from PMI headquarters through example.

Jesse also spoke on Wednesday evening at the ThoughtWorks office not far from the Conference hotel where many people showed up to hear the formal announcement of the PMI-Agile effort being launched. There were other talks/discussions at this event and I actually spent a good bit of time in a room with 10-15 people and video hookups to ThoughtWorks office in India and China. We discussed various aspects of widely distributed teams, the challenges this presents, and ways people have worked to address those challenges.

Brian Marick – “4 Challenges, 5 Guiding Values”

This was a clearly well-planned and practiced presentation as Brian delivered it clearly and succinctly.

His 4 “challenges” were:

Open Workspace – or rather the lack thereof which represents an impediment to remove. The typical agile workspace may be viewed by others as messy, noisy, undisciplined, uncontrolled and, in general, “unprofessional.” Brian told a story of a coach that, when the organization refused to move cube walls, came in over a weekend and personally reorganized the space by disassembling and re-assembling the walls. On Monday, they stated that they’d keep doing this, if the walls were reassembled, until they were fired.

Courage – The prior example Brian gave illustrates when it can take and the challenge presented in doing so. But Brian noted that as the transition to agile improves, less of this sort of courage becomes needed.

Naiveté – or, again, lack of it. Brian note the need to hold on to this “beginner’s mind” trait to fight objections over new ideas by withholding judgment about practices not tried (e.g., pairing, TDD).

Infrastructure Design – There is always some tension between building as robust base for functionality and getting to the functionality. Features may come in a way that ends up with the architecture being inadequate if too little thought goes into initial concern for it.

Regarding Working Software, Brian noted that if one can deliver this, people may be more willing to give you some slack in other ways to work as you wish. The goal is to deliver something better than the last time, i.e., “last week I could not do ‘X’, but this week I can.” In this way the value becomes clear and concrete.

His 5 values were:

Reactive vs Proactive – in this case being “reactive” was more about adaptation than allowing poor work to be done and than scrambling to “fix” it.

“Irritation” (leading to) Ease – something will drive the desire to improve and Brian discussed a “ready-to-hand” approach (which focuses on the goal) compared to a present-to-hand one (which focuses on the tool). He used the example of a hammer which we do not think about as we focusing on the act of hammering compared to a hammer with, for example, a loose head which we have some concern over lest the head fly off while hammering. In this context, Brian also talked about the practice of “shaving yaks” which represents starting out on a task only to find something else it depends on and then something that depends on, trying to find the perfect way when a good way right now will do. (The Yak’s hair enters the picture as a last step driven to though starting out with nothing like that in mind.)

Solidarity – Brian mentioned the well-known Golden Rule of “doing unto others as you would have them do unto you.” The problem, he said, is that other people are not you and that a Platinum Rule seems better where you “do unto others as they would have you do unto them.” At this point, however, he turned back to the story of the open workspace disassembly and noted that it would have been a far stronger example had the whole team joined in and stated they would quit. That would show solidarity.

Decency – Brian stated this quite simply as “treating others uncommonly well,” which ties back to the Platinum Rule.

Joy – Finally, Brian spoke of bringing true joy to work and noted how everyone can speak about the “one great project” they were on. These days, he said, what he hears more often is that they do agile because “at least my project doesn’t suck as much as it used to.” Hardly an encouraging sign for agile methods and their value.

One final idea that I noted was Brian’s statement that “if you are doing agile well, you can afford to be wrong” as the consequences of a decision are easily corrected.

[This ends my formal summary of the Agile 2009 sessions I attended. I may have more to say about my reaction to some of these ideas in other posts in the future, though.]

Agile 2009 Notes - Wednesday

2009-08-30T17:19:00.005-04:00

The following material describes the sessions I attended.

Wednesday AM –

Daryl Kulak (of Pillar Technology) – “5 Symptoms of Mechanical Agile and Being Change Ready”

Got up early to hear this talk since it was a vendor presentation scheduled for 7:30am in the breakfast area of the Hyatt and I was staying about 8 blocks away and it was raining and ….
In any event, Daryl identified “mechanistic” signs and, later, ideas to make an organization more change ready (that I have associated with the mechanistic practices though they were discussed separately):

Agile Expert Syndrome – people continue to do things because the expert brought in to help at one time did so. (Later on in the talk Daryl said “If you see a Best Practice by the side of the roads, kill it” and referred to Mary Poppendick’s “good practices in context” quote.

Separation of Decision-Making from Work – Daryl used an example of decisions made by a process group (SEPG) separated from the teams which resulted, not surprisingly, in much overhead and resulting in “seepage” as the pronunciation for the process group acronym. Daryl’s advice, of course, was to close the gap between decision-making and work.

Blame the [other people, team] Person, not the System – here, the advice was to remove boundaries and encourage connectedness between teams.

Just Tell Me What to do, Boss – the material on this topic basically recommended learning to value unstructuredness.

Competition Between People and/or Teams – didn’t fine a direct “answer” to this as the last change-related point was to “avoid over-engineering your requirements, people and processes.”

David Hussman – “Coaching and Producing Agility”

This was an all morning workshop on the role of the Agile coach and how to improve one’s coaching capability. Most of the time we worked in pairs (though “promiscuously” since we changed for each exercise like canonical pairing suggests). David divided the session into four (not necessarily equal) sections, which, in case you wonder, were using a music production theme as David’s former life was as a record producer and musician:

Coaching Personas (Your Coaching Style) – In this section, we were all asked to create/identify a persona for ourselves with a short phrase summarizing our style, a possible graphic to illustrate it, a series of words describing ourselves and a series of words identifying values we hold. Mine ended up being Socratic, Story-Telling, Scorpio with a graphic of "? my blog logo ?". My descriptive elements were: (Talkative), Organized, Inquisitive, [Musical], Teacher. My values were: Synthesis, Experimentation, Openness, Learning, Self-Direction.

So, to explain this, my approach to teaching and consulting has been to ask questions and guide people to answers rather than tell in as many cases as it seems reasonable to do so (a “Socratic” approach). I employ a lot of stories as examples since I’ve been around software development of various kinds and industries for over 37 years. And, though my astrological sign is Gemini, with Capricorn rising and moon in Ares, Scorpio is at my mid-heaven – I used to be in a band where one person’s wife was into astrology and she did my chart. So the Gemini represents the inquisitive, experimenting, open, change-ready aspect of my life. Capricorn represents my organizational inclination (which is often what people see over time). People’s mid-heaven sign usually represents their success mode, so Scorpio represents my synthesis approach where I try to take ideas from many places and produce some whole out of it that takes pieces from each. My logo represents this (see my first blog where I describe it).

The other elements of the persona I’ve covered more or less. Talkative has parens around it since I see that as something I need to contain as a coach and Musical is bracketed because I have a band/recording background and it just popped in there but I’m not sure how it fits exactly other than the band metaphor working better for me than team sports ones so common in business similes/metaphors.

Preproduction (Getting Ready to Produce) – This section addressed interviewing, which we practiced with one another by using real world project examples from our own experience. During this part of the session, David recommended a descriptive approach (”This is what I have seen work”) rather than a prescriptive one (“This is what you should do”) in doing our coaching. He also briefly touched on the Satir change model then collected typical agile practices together to show valuable, related groupings:

For Community-Teams: Chartering, Common Workspace, Information Radiators, Iteration 0
For Iterative Delivery: Burnup / Velocity, Acceptance Tests, Test Driven / Refactoring, Continuous Integration
For Products-Planning: Product Backlogs, Personas, User Stories, Release / Iteration Planning
For Tuning -Improving: Stand Up Meetings, Product Reviews, Retrospectives, Continuous Feedback

We completed this part of the session with a coaching plan development exercise to (1) take practices (what do you want to do?), (2) identify the value of each (why?), and (3) give an example of each (how might it work?). The goal was to arrive at a base approach each of us could use for coaching.

Finding Your Groove (Getting Productive) – David started this part of the session recommending a couple books: The Black Swan and This is Your Brain on Music, quoting from the latter:

“Groove is that quality that moves the song forward”
“When a song has a good groove, it invites us into a sonic world that we don’t want to leave.”

Five things help build “groove” in agile coaching: iteration planning, story-telling, stand ups, acceptance tests/reviews, and retrospectives/indicators. We spent time telling one another, again in pairs, stories to illustrate situations from real projects. David emphasized that each story told should be about somebody doing something of value. He also commented on the traditional story-writing format for agile requirements: he doesn’t like it because, I believe, he sees it as constraining ideas about expressing value, being ritualistic if treated dogmatically.

(Regarding story estimation he noted an idea I have heard before, which he said comes from ThoughtWorks, that people use a paper-rock-scissors approach to sizing since, you always have your materials with you.)

Keeping the Band Together (Staying Productive) – The final part of the session was about sustaining what is built through earlier coaching work. Davbid mentioned the traditional “5 whys” used to do root cause analysis, but recommended a “5 whats” approach since, psychologically, “whys” can lead to blaming of individuals. Of course, retrospectives were noted in this part of the session, including doing retrospectives on one’s own coaching using the coaching plan format of practice-value-example. Finally, David mentioned The Beginner’s Mind and the need to maintain curiosity about the work.

Wednesday PM –

Esther Derby and Diana Larsen – “Esther and Diana's Excellent Retrospective Adventures”

This was another long workshop, taking the entire afternoon. Most of it was based on Esther and Diana’s book Agile Retrospectives: Making Good Teams Great. The session began with an activity for teams of 5-7 people, of which there were ~7 in the session. We were all charged with building an “objet d’arte” with materials such as sheets of paper, colored pipe cleaners, stickers, etc. (but no tape). The “specs” for the project were that the result needed height, stability, and beauty. We were allowed to ask Esther to come over and ask her questions, as our “customer.” When we asked about height, she pulled out a sewing measurement tape to about 24”. When we asked about beauty, she mentioned she liked “motion.” Stability we took for granted as understanding (and perhaps lucked out there). Lots of “structures” resulted, but, as you might guess, the exercise was all about having something to use to practice retrospective technique.

Before we started doing so, however, we were provided with a proposed outline for a retrospective, beginning with Setting the Stage which included (1) identifying the focus/goal of the retrospective, (2) congratulating the team on their iteration, (3) checking in by getting everyone to say a word or two that described how their felt/reacted to the iteration, and (4) defining the agenda. The agenda would consist of: gathering data, generating insights, deciding priorities, and coming to a close. For each of the agenda elements, we performed an exercise.

For Gathering Data, each team generated a radar chart covering things like use of resources, customer satisfaction, quality of work life, etc. Each member of the team was asked to rate (0-10 scale along the arms of the chart) their experience with each element of the chart. We were then asked to identify the variances and commonalities among the results, discuss what we heard and saw, and recognize the high and low points for each element.

To Generate Insights, we were asked to identify what we felt were underlying causes for the major results (pro and con) using a four section matrix: what we felt good about, what we did not feel good about, what were ideas & insights, and what “bouquets” we wanted to give anyone. Each of us created stickies with things which could go in any of the portions of the matrix.

From this we moved to Deciding Priorities by listing actions in a table with the columns labeled: Action, Impact, Effort, Energy and Commitment. The last two deserve some clarification since they require people to indicate how dedicated they are to taking the action as a group and as individuals. The process was to list all the actions, then all the impacts for each action, then the effort expected for each action, followed by the energy people felt they had for the action. In case of apparent ties (since not all actions could reasonably be done in a single iteration), the energy column was to be used to break ties. However, people have to be willing to “sign up for” (commit to) doing any action, not just place votes and estimates on them. Of course, if people were not willing to ultimately commit to an action, what did their energy statement really mean? Finally, it was emphasized that we should end up being able to “take a card into the planning” for the next iteration that described the task to be performed (not as a story).

Coming to a Close involved doing a brief retrospective on the retrospective itself, revisiting commitments, and thanking the participants for their involvement.

Following this, there was some open discussion and question time. One question was who should facilitate retrospectives. A number of ideas were suggested: team coach, trained facilitator from outside the project, and rotation of team members. Esther and Diana recommended the latter and said it could help participation in the retrospectives since people would not want to withhold from participating since they would want others to be active when they facilitate. So this would help establish a more constructive retrospective “culture.”

One final idea about retrospectives and team interactions was that the absence of conflict is not harmony, but apathy.

To close, we were directed to a Yahoo group (retrospectives-subscribe@yahoogroups.com, mentioning this class) as it is a moderated group. Also a book by Sam Kaner (and others) called Facilitator's Guide to Participatory Decision-Making was recommended.

Agile 2009 Notes - Tuesday

2009-08-30T12:48:00.003-04:00

[Sorry I am behind on these, but I'll be catching up today and tomorrow.]

The following material describes the sessions I attended.

Tuesday AM –

Alistair Cockburn’s Keynote, “I Come to Bury Agile, Not to Praise it”

Alistair’s presentation began with a recitation of a revised version of Marc Antony’s speech over Caesar’s dead body from Shakespeare’s Julius Caesar. He said he did not feel Agile was really dead, but that it had [begun to] become absorbed in the mainstream of how many groups do software. Like a large, melted iceberg, he said, Agile was now part of the ocean. As such, Agile has begun facing classic software development problems related to large team design efforts. To address this situation, Alistair made three main points related to (1) software development as a cooperative game (a classic theme of his), (2) craftsmanship (a recent “movement” in the Agile community), and (3) lean concepts.

Game – Alistair showed his matrix of competitive and cooperative game types along the scale of finite, open-ended and infinite time periods. Software development is a cooperative game which is finite (in length) and goal-directed. Alistair note that software development had two goals which could conflict: delivery the software [on time and on budget with acceptable quality] and “setting up for the next game.” The latter meant making sure the work done made it possible to do the next work. This involves effort that will pay off in the future, but not necessarily now. Thus, there will be a cost added to the current work that may show no immediate value and, thus, be hard to justify in the short-term.

A significant problem in software as a game is that “positions don’t repeat” (often), so strategies for playing the game must be adaptive. Being adaptive, however, means being able to communicate and understand effectively. Alistair described another of his classic themes: how the best communication is face-to-face and how distance between people who need to communicate has a significant cost that is often not accounted for in project structures. Distance, he said, affects whether people can detect the need to communicate, care enough to do so, and be effective in making it happen.

Craft – Alistair listed 7 major crafts: deciding what to build, managing people and projects, modeling, designing an external view, large-scale (architecture) design, fine-scale (programming) design, and validation. At this point, Alistair reviewed another of his classic themes, the shu-ha-ri learning stages where one learns technique, collects more technique and finally invents/blends techniques.

Lean – Finally, Alistair talked about work flow and bottlenecks, noting that software’s “inventory” that moved through the development process is “the unvalidated decision.” Lean aims for “continuous flow” and people waiting on decisions create the bottlenecks. [One problem with such waiting is that, to move along the expected schedule, people will often “make up” decisions that either go unknown or are learned of late and require expensive rework.]

Unlike manufacturing, software development requires feedback/learning as design is knowledge acquisition. Alistair spoke about waterfall as a late-learning approach to developing software since, until a coherent system delivery for testing occurs, there is not much information about the actual state of what will be delivered. Agile methods, in contrast, incur cost earlier to learn earlier through continuous integration such that a good idea of the state of the work occurs early enough to make risk-reduction decisions. This allows business to “trim the tail,” that is, make a decision to deliver on time (or early) or delay delivery to get more (or better) functionality.

In closing, Alistair said that 21st century development will use the game, craft and lean ideas he noted.

Tamara Sulaiman – “Tips & Techniques for Implementing an Agile Program Across Distributed Teams”

Most of this talk was based on ideas from John P. Kotter’s book Leading Change in which he identifies 8 change concepts:

1) Establish a sense of urgency by “going for emotions” and being clear about the passion for change.

2) Create a guiding coalition of people from across the whole organization who are credible and committed to change. They should be willing to model agile behaviors for the rest of the organization, i.e., being cross-functional, running their meetings/work with one another in an agile fashion, being visible in the coaching/training given to other people in the organization, and maintaining a backlog of transition tasks with a Product Owner. [Audience member suggested forming, not a PMO but an AMO: Agile Mentoring Office.]

3) Develop a vision and strategy by showing what success in the new way would be/look like.

4) Communicate the change vision through slogans, graphics, posters, wiki pages, a glossary of new terms and how they relate to older ones. [There could clearly be some argument about this as it is how every “flavor of the month” program gets done in companies. Having point 2) clear would make this “marketing” acceptable and likely effective. Otherwise, it will be ridiculed.]

5) Empower broad-based action to allow change impact to go beyond development teams and IT as well as to provide an environment where it is okay to fail as long as the learning is clear and progress continues.

6) Generate short-term wins [to show progress] and use a wiki where teams can write about their experiences.

7) Consolidate gains and produce more change using lean, process flow, etc.

8) Anchor the change in the culture, producing a high-trust organization.

Overall, though the material was interesting, the talk was not that much about agile-specific issues in distributed teams. It was more a focus on organization-wide change management, which could be applied to any methodology, for example.

Luke Hohmann – “Leveraging Collaborative Tools with Distributed Customer Teams”

Again, while some interesting ideas and pointers to other resources, this talk did not address substantive distributed team issues. Some important concepts mentioned, however, were:

- Collaboration is always about the goals, so being clear about goals matters. [Collaboration is not simply ad hoc “cooperation” among people.]

- Effective goals are expressed in verbs, i.e., action.

- Understand how fast a customer can accept the output of agile teams. [Cannot collaborate well if the work cannot be shared effectively.]

Hohmann pointed the audience to a Harvard Business Review article entitled “In Praise of Hierarchy,” noting that there are many effective uses for hierarchy, authority not being one of them but being the one most often associated with hierarchy. Hierarchy can be very helpful in handling complexity, for example.

Tuesday PM –

Jurgen Appelo – “What (Else) Can Agile Learn from Complexity?”

Jurgen has one of the most popular blogs in Europe (related to software development and management) and his talk was an interesting exploration of ideas related to complexity. Basically, Jurgen took a series of quotations about complexity and related them to various ideas in agile methods. He began, as might be expected, with some quotes from Ken Schwaber, Jeff Sutherland, and Jim Highsmith discussing very direct links with agile methods. He then displayed a detailed graphic of the history of complexity science from people like Norbert Weiner through work being done to explore web/e-science ideas.

During the talk, he referenced numerous books and articles, including one in particular on social complexity and management. Jurgen is, himself, working on a book related to management in an agile context as he is CIO of a software development company in the Netherlands. Though the article describes 4 types of complexity, Jurgen’s emphasis was on social complexity because of its complex, unordered, and human-centric focus. This led him to some discussion of self-organization which is key in agile development philosophy. Interestingly, he asked whether an agile team, being coached, is really self-organizing, pointing to the Wikipedia definition of self-organization as including the phrase “without being guided or managed by an outside source.”

Jurgen then described various complexity-related principles, including:

Darkness Principle – systems elements are ignorant of overall system behavior since all system complexity cannot be present in each element; thus no single person (e.g., project manager) can monitor and control a whole system since they cannot know everything [and should learn to make use of self-organization and self-management in teams to help manage the complexity].

Law of Requisite Variety – to have a stable system the states of any control mechanism must equal or exceed that of the system under control and one (project) manager is less complex than the project being managed.

Boundaries and Conditions – self-organization requires that it operate within a boundary which defines the “self” being organized, thus agile management is an important part of agile development in setting such boundaries, i.e., teams do not get to make all decisions about everything.

Hierarchy Principle – “complex natural phenomena are organized into hierarchies wherein each level is made up of several integrated systems,” harkening back to Hohmann’s statement about the value of hierarchy and using a Scrum of Scrums as an example of one that can grow naturally rather than being imposed. Jurgen also noted a “patchwork” approach to Scrum relationships without hierarchy (and referenced Mike Cohn’s Mountain Goat Software site for further discussion of this).

Group Size – where it was noted that some research has pointed to ‘8’ as the most likely group size to lead to deadlock.

Specialization – many advantages accrue in complex systems when there is some level of specialization/division of labor.

Power Laws – which are related to that fact that there is a high chance of small issues and a low chance of large ones, suggesting that prediction of velocity into the future “includes an (impossible) estimate of the size of unknown problems.”

Dependence on Context – “the method to manage the project is embedded in the context and one must allow the emergence of such a method through interaction between the actors and the environment,” which led Jurgen to say that “ScrumButs are natural and necessary.”

Fitness Landscapes – is about how we “create the environment,” it is not “separate from us,” or, quoting a Spanish phrase, “My friend, there is no road, You make the road as you walk.” (This all relates to how a released system will affect, and change, the environment into which it is released.)

Incomprehensibility – states that “there is no accurate representation of the system which is simpler than the system itself,” so any model of it will be wrong, though, as George Box has said, “all models are wrong, but some are useful.”

Stephen Palmer – “Working with Large Backlogs”

Palmer discussed a variety of methods for backlog “triage” and management, including epics, themes, hierarchies, pruning, demand management, and extended kanban. He directed the audience to http://www.limitedwipsociety.org/, in particular, with regard to the last approach and mentioned some ideas from David Anderson on time-to-market psychology, e.g., on a 6 week project with 2 week deliveries, there is not much schedule concern, but on a 3 month project with 1 month deliveries, schedule will matter.

Dan Mezick – “Group Relations and Social Systems”

This was basically a talk about group relations theory and how it could be applied to agile teams (to some extent). Dan noted that he had a talk Thursday AM covering the ideas (of boundaries, authority, roles and tasks) in more detail and I’ll cover that talk in Thursday’s summary. This talk began by noting work by Wilfred Bion and his book Experiences in Groups (1961) which covers the main theories in this area. In particular, Dan stated that true groups demonstrate “interdependence of tasks and fate” since survival is a key group driver. For example, he noted that wolves, on their own, must survive on meager food sources (i.e., small animals) while those in packs have richer sources (i.e., moose, caribou). A key issue in maintaining this survival and accomplishing tasks is avoidance of distractions since distractions = waste. Unfortunately, group relations can produce a great deal of waste, but agile methods, like Scrum, employ various “ceremonies,” “artifacts,” and “practices” to “short circuit inattention and drift.” This is done by leveraging “inattentional blindness” which research has indicated means people with a clear focus on activities of immediate concern can block out other things which might place a claim on their attention. In a sense, this supports the idea of “we see and hear what we expect.”

Dan then discussed planning which he said is a prediction, but “prediction is a judgment” and “judgment is a belief” and “belief is a filter” and “filters can distort reality.” All belief demands attention, which can produce waste if the attention is not on the reality of what needs to be done at the moment. Impediments in Scrum are one kind of distraction.

Dan continued by noting several resources/references related to group relations such as LeBon’s study (1895) of loss of individual identity in crowds, making them more easily influenced. Crowds exhibit “system-level emotions that are inherently primitive” as well as seeking dependence on leadership. Indeed, a group will usually seek out leadership that “voices” its main desire. Work by McDougal (1920s) focused on smaller groups who were task-oriented (not the unorganized crowds LeBon studied). Work by Kurt Lewin (1940s) on field theory influenced Bion and was the pre-cursor to modern group relations work. For Lewin a field is “the totality of coexisting facts which are conceived of as mutually interdependent” such as Scrum’s goal of co-location, its roles, its artifacts, and its “ceremonies” (e.g., stand-ups, planning meeting, review, retrospective). Dan also noted Tuckman’s (1965) “forming, storming, norming, performing” model of small group development to which Tuckman added “adjourning” (1977). Tuckman’s work frequently mentioned Bion’s.

Dan nexted spoke about attention and comparing Scrum to more waterfall approaches. In Scrum, “you cannot drift very far from stated task[s].” Waterfall approaches are “not an ‘attention harness’ in any sense of that word” since “all sorts of attention is diverted away from stated task[s], lengthening” their duration. In Scrum all distractions are treated equally, “they are waste.”

Dan finished up by discussing group relations “conferences” (which sounded more like intensive week-long workshops). He also pointed us to the Washington-Baltimore Center for the Study of Group Relations which he said has a variety of good resources on group relations.

Agile 2009 Notes - Monday

2009-08-24T23:52:00.006-04:00

The following material describes the sessions I attended.

Monday AM –

Research Stage

Research reports started Monday at 9AM before other sessions, but were located out of the way of all other session areas. Also, some people at information areas claimed sessions didn’t start until 11AM.

I spent the morning at two research presentations because the format this year drew in a great deal of audience participation as opposed to just a series of 45-60 minute talks. Dr. Jael Dublinksy explained that a goal of the Research Stage this year was to solicit a lot of input from industry folks. I believe it worked for those industry people present, but, again, being a out of the way(and mind) of people did diminish attendance.

The first presentation was by Rosalva E. Gallardo-Valencia on “How Agile Teams validate Requirements.” Team activity was observed and artifacts were collected for ~2 days, including stand-ups and Sprint Planning sessions. Overall, the study showed that, while standard validation approaches were not used (or even possible) in an agile environment, there as substantial validation performed at various points during team and team/PO interaction.
Interestingly, it was noted that the PO and written up a “checklist” of details about story information which was not shared with the teams. Instead, at the Planning Meeting, the PO would talk about this information and respond to team questions while the ScrumMaster worked to write the stories to match the discussion. I asked why, if all this material had already been written (even in brief form), did people feel index cards had to be created. There were a variety of audience opinions, but there really was no answer from the study why this occurred.
It was noted that the teams wrote jUnit tests before coding and that Fitnesse was used for acceptance testing.

After the presentation, there was a period of feedback to the presenter on work of the study. But then, the audience was asked to form into groups and consider research topics that they might like to see pursued. Our group had the following ideas submitted by various members of the group:

1) Classification of team sizes and recommendations for what team sizes/characteristics might be appropriate for different development situations. Dimensions to be examined were: trust, distribution of team, new vs “rewrite” type work, and degree of team integration, including senior management.

2) What concrete benefits does agile provide an organization, e.g., lower costs, higher customer satisfaction, improved time-to-market, market success (revenue).

3) Quantify an organizations degree of “agileness”

4) Effective approaches for introducing agile – path to adoption.

The next presentation was by Dr. Stuart Mitchell based on the work of Rashina Hoda regarding “Agile Undercover” where some organizations wanted to use agile but customers “couldn’t have cared less about what methods were used.”

Dr. Mitchell noted that a grounded theory approach (Glaser rather than Strauss style) was being used for data collection. Observation, without making conclusive statements was a characteristic of how this study was done.

Some outsourcing teams in India were studied. They wanted to use an agile approach, but most of their customers did not care to adapt to such an approach. Ms. Hoda interviewed a person from each of ~8 teams. It seemed that many ended up with customer proxies to allow their teams to try to function in as agile a manner as they wanted. Some teams did do periodic demos to actual customers, but some did not.

I suggested that the most interesting research work related to this might be to observe how the proxy role made it easier for the team as to behave as they wanted while the customer did not have to be aware of how the team was getting work done.

Monday PM –

Elizabeth Hendrickson and Chris Sims on Design of Simulations and Games

This was an all afternoon session that asked attendees, in teams, to design “games” of various kinds that could be used to allow help groups learn (presumably about agile methods, values, etc.). There were board games designed; some involving pure interaction among participants; one involving joint creation of sentences; etc. Each, however, was developed around a different goal. One being “Truth and honesty in planning.” Once teams had been given some time to design a game, members of other teams would come by to “playtest” and provide feedback. Teams could revise their games and get further feedback.

Many fascinating observations occurred about what the games “taught” as well as what contribution to that learning the participants brought compared to the games as originally envisioned. Some of the main ideas that came from the session as a whole (some offered directly and others arising out of the session itself) were:

1) The debriefing around playing the game(s) is where the main value occurs, not in the game(s) themselves.

2) Using the term ”simulation” rather than “game” might make the experience of using games more acceptable in some organizations.

3) It’s better to layer complexity on a simple game than to try to remove it from a more complex one.

4) Consider the incorporation of obstacles, choices and randomness in games.

5) Interesting results come from players not sharing the same mental model of what the game is about. (Two examples were a sentence-building game and a balloon passing one.)

6) One good goal of a game is to see how participants achieve alignment, perhaps by evolving the game and its rules themselves to reach a goal they determine to be the point of the game.

7) Game debriefing is like a retrospective for which some suggested questions ideas were asking what happened (in game play); what did players do and feel; what surprised people? In this regard Elizabeth and Chris suggested being very observant as game facilitators, noting what people did and how the game changed, then using these observations to ask what the participants thought about the things observed.

8) Ask participants to “map” what the learned/observed in game play to real world situations and how they could take action in their environment to implement what was learned.

All in all, it was a very interesting day.

What’s your agile methods “elevator speech”?

2009-08-22T07:30:00.001-04:00

Sorry I missed a post for yesterday, but I had a bit to do to complete getting ready to leave later today for Agile 2009.

Today’s post will be short, but deliberately so as it is about asking you to submit your favorite agile “elevator speech.”

In case you haven’t heard the term before, an “elevator speech” is a brief explanation of something to get the listener interested in hearing more or reasonably informed about the subject in a short time. The “elevator” part comes from someone asking some thing like, “What if you were on an elevator with your boss and they asked you ‘I’ve been hearing a lot about . What’s it about?’ What would you say in 60 seconds, so your boss would walk away with a reasonable idea?”

So what if you were in the elevator and your boss said, “I’ve been hearing a lot about agile methods. What are they all about?” What would you say to explain agile methods in 60 seconds?

Here’s one sample (but by no means great) idea:

"Agile methods are an approach to development and delivery of software in smaller, shorter increments. They do this by using techniques that improve communication bandwidth & frequency using direct contact among customers, developers & the product to get continuous feedback. They also employ development approaches that try to be open to and prepared for change by facilitating change rather than trying to prevent it and developing incrementally so change is easier to manage."

What would you say in about that many words?

Quotes (from Twitter)

2009-08-20T15:13:00.004-04:00

The the past 2-3 months that I've been on Twitter, I've been saving Tweets that, at the time, struck in particular when they appeared. As the list was getting long, I thought I'd better post these now, before it got ridiculously long. Some are quotes, not from Twitter users, but quotes other Twitter users thought were interesting from authors, philosophers, etc. There's even a few here of my own. :-) [In some cases there are follow-ups combined with the original which are not, therefore, in alphabetical order by the author's name.]

Abraham Maslow (via ASQ) - "Secrecy, censorship, dishonesty, & blocking of communication threatens all the basic needs."

Alan Shalloway - Doing meta thinking is good (see my book design patterns explained). "going meta" means doing meta thinking w/o validation ;)

Alan Shalloway - practices followed as solutions are dangerous. Practices followed as examples of understood principles are a basis of learning.

Albert Einstein - Not everything that can be counted counts and not everything that counts can be counted.

Alistair Cockburn - The center of agile development is to deliver running, tested features to users and collect feedback.

Alistair Cockburn - We need shared understanding at this point, not shared common sense.

Alistair Cockburn: OH "a large dev team has a voracious appetite for requirements". Food for thought.

Andrew Clay Shafer - In a mechanical system, friction is the most common cause of lost work. Friction seems like a useful metaphor.

Ari Tikka - Standardized work... in Finnish I like word "vakioitu". Has the flavor of being constant instead of forced to standard.

August Turak - Maximum motivation emerges from the peer pressure of a team working toward a common mission.

Bas Vodde - I'm a positive person, but gradually losing hope that large companies can get rid of their self-destroying habits. Esther Derby - plus structures and policies reinforce current behavior, sometimes punish new behavior. Bas Vodde - Yes, and people reinforce each others behavior all the time. A nice system build to keep it in status quo.

Ben Simo - If you learn from /waste/ and /rework/, is it really waste?

Ben Simo - SW doesn't handle ambiguity like people but SW systems definitely can be complex. Plus, human users r part of those systems.

Benjamin Mitchell - A new team has set up a Kanban board with an explicit column for 'blocked'. Amazing how quickly kanban boards provide useful information.

Bill Graham - “It’s only work if there’s something else you’d rather be doing.”

Bob Marshall - Locating Sw Dev in IT seems dumb to me (these days). But the wider point: who owns Product Dev?

Bob Marshall - The one and only question you need to ask agile teams: "What measures are you using to understand and improve performance?

Bob Martin - Scrum+XP+Lean+Kanban+CSM+DSDM+TDD+BDD+CSP+... = murky Agilebet soup. Maybe it's time to rethink this.

Brian Marick - like to think of AR⊗TA as part of new wave: "economy of trust", "smaller co's w/ intense collaboration", products instead of finance, etc.

Brian Marick - Courage isn't needed for those things once you've constructed an environment where making mistakes isn't scary. So I'd put courage as a temporary value, a stepping stone.

Brian Marick - I detect a certain tendency for craftsmanship to become narcissistic, about the individual's heroic journey toward mastery.

Brian Marick - I think of arxta as a back to basics movement, where basics are pretty much XP attitude & the original scrappiness of Scrum.

Brian Marick - Interesting to think about changes required by "Done means it's in the user's hands. Nothing less."

Brian Marick - Much social thought today is about making sure no one gets something for nothing (welfare queens!) and bad ppl suffer (smokers: perish). (Esther Derby - Those seem almost like tribal values.) What gets lost is persuading/allowing ppl to give something for nothing (open source, social capital) so that ordinary people benefit. Being jealous of resources is human reproduction model (small # of children, huge investment in each). Alternative is our maple: huge # of seeds, waste cheerfully accepted. Maple doesn't care that some seeds wasted on undeserving ground.

Brian Marick - People who think they're on a hero's journey tend to disregard the ordinary schmucks around them.

Brian Marick - The idea that working with other people is risky, that it requires lowering some sort of barrier or thinking differently (becoming humble).

Brian Marick - Thing about waterfall for programmers: you only have to acknowledge your gross lack of skill every few months. With Agile, it's *every day*. Ron Jeffries - true but i can't do much harm in a single day.

C. Northcote Parkinson - “Delay is the deadliest form of denial.” (Not from Twitter)

Charles Kingsley - We act as though comfort and luxury were the chief requirements of life, when all that we need to make us happy is something to be enthusiastic about.

Chris Sims - People think agile introduces uncertainty when it is actually just uncovering the uncertainty that has always been there.

Classic #quote - A user is somebody who tells you what they want the day you give them what they asked for.

Courtney Benson - "People don’t fail because they make mistakes. People fail because they don’t learn from their mistakes.” Chuck Musciano

Dan North - talk on learning at Better Software conf: "Use metrics as indicators, not targets."

Dave Pembs - Do not argue with an idiot. He will drag you down to his level and beat you with experience.

David Alfaro - "Individuals and interactions over processes and tools" So true! Docs=Potential Knowledge [hardly unleashed], People=Kinetic Knowledge.

David Alfaro - You will be amazed how powerful is to switch the immediate reaction "I disagree" to "I don't see it, help me see it"

David Anderson - TPS yes! Lean (Womack, Jones, Daniels) No! A theory of variation is missing from core Lean literature. So not Deming.

David Hussman - "what do you call a stand-up meeting with too many people? A stand-there meeting"

David Platt (via Michael Bolton) - “Your user is not you. Most people don't want to DRIVE somewhere, they want to BE somewhere."

Derrick Bailey - if you don't learn fr waste/rework, it's not only waste/rework, its complete failure. i expect learning fr waste/rework.

Doug Shimp - A well formed team often can solve problems faster than the surrounding business is able to apply.

Doug Shimp - Build your team protocols one at a time. Pay attention and adapt based on realities encounter and don't assume what is needed.

Doug Shimp - Great individual talent is often not enough. We need teams that are talented & humble enough 2 work together.

Doug Shimp - If you think it's expensive to hire a professional to do the job, wait until you hire an amateur.

Doug Shimp - Scrum often reveals that the business is not able to figure out which problems to solve.

Ed Yourdon - David Stephenson makes key point (at #e2conf1): "make customers co-creators"

Elizabeth Hendrickson - Seems to me that prof testers do 4 things well: observe, notice what can be varied, predict risks, & explore methodically.

Elizabeth Hendrickson -Trying to test the depths of the code thru the UI is like peering through the shower head to examine pipes in the basement.

Esther Derby - primary work of managers: establish conditions for teams to work in. develop people. work on the work system.

G.M. Weinberg - CENTER means know your own agenda and motivations. Get control of yourself first, so you can genuinely be of service to someone else. ENTER means you must enter someone else's system to help them. You can't bash your way in, and you can't force them to see things as you do. TURN means that we think in terms of making a nudge here and there. We can't expect to transform someone. They self-transform, if at all.

G.M. Weinberg - Instead of thinking, 'irrational', think 'rational from the perspective of a different set of values.

George Dinwiddie - "Talk to the card" We found that focus on the card wall helped bring focus to the standup.

George Dinwiddie - Yes, it's a pity. With mechanical products, the customer can admire the design and workmanship. Estimate long term value.

Gia Lyons - Marketing = Matchmaker. Sales = Dating. Services = Marriage. Support = Marriage Counseling.

Greg Vaughn - Agile offers more control. But you have to give up the illusion of predictability.

Henrik Kniberg - In Scrum, the Scrum Master's role is to create a great Team, and the Product Owner's role is to use that team to create a great Product.

Herm Albright - A positive attitude may not solve all your problems but it will annoy enough people to make it worth the effort.

Igor Macaubus - "Rules and procedures can be an insurance policy against disaster, and they prevent disaster. But they also assure mediocrity."

Immanuel Kant - Immaturity is the inability to use one's own understanding without the guidance of another.

J.B.Rainsberger - I find that the more authority I have over my own habits, the more I care about outcomes over dogma.

J.B.Rainsberger - Perhaps if more craftspeople worked more closely together, the narcissism would subside.

James Bach - Testing under no circumstances shows that the product will work. So we test mainly to discover how it will not work.

James Bach - When you see someone "resist change", realize from their point of view they are just applying self-discipline to do things right.

Jason Yip - If you had to scale down Agile to a few core skills that can be learned and applied without a huge commitment. What would that look like?

Jason Yip - The problem is not concentration of power; the problem is thinking based on authority rather than responsibility.

Jeff Patton - popular quote at yesterday's agile round table [Colorado somewhere] on good management: "organize their goals, not their work." - can't remember where I got it.

Jens Ohlig - pizza with the radius z and thickness a has the volume pi*z*z*a [Life is good]

Jim Highsmith - Agile managers understand that demanding certainty in the face of uncertainty is dysfunctional.

Jim Highsmith - Calculating value points. If the product mgr has no time to calculate value, the dev team has no time to calculate cost.

Jim Highsmith - Documentation is often the solution to a communications problem that can't be corrected with documentation.

Jim Highsmith - The best way to get a project done faster is to start sooner.

Joshua Kerievsky - An improvement is something that "enhances value or excellence." Don't ship features. Ship improvements.

Joshua Kerievsky - To release frequently, discover and build "acceptably incomplete" features that can ship today and evolve tomorrow. re: "acceptably incomplete" - one Customer deferred a good number of stories (8-10) that would "flesh out" a feature. In the end, those stories were never implemented - Customer realized that they didn't really need them after all. Dave Rooney - So, "acceptably incomplete" was actually "perfectly acceptable". Joshua Kerievsky - Completeness is overrated. We get more done faster by finding what is acceptable in its incompleteness. The core issue may be that we don't know what "complete" really is all the time (perhaps 'most' of the time). That's why the hardest and perhaps most valuable skill to master is Evolutionary Design.

Keith Braithwaite - No. I'd call src ctrl a good practice so good that it's mandatory until something better comes along—and I hope for something better. Ron Jeffries - so your concern with "best" is that someday something might be better? Keith Braithwaite - My concern is that a stated belief in the current way being best will lessen the likelihood of a better way being recognised.

Kent Beck - by "aspirations" i mean "who we are trying to be": software will improve when we aspire to be accountable, transparent, reliable. Ron Jeffries - well, yes, that and when we act in accord with those aspirations.

Kent Beck - Disagree that xp1e glorified programmers. Talking code when addressing the industry built on code isn't glorifying. The kind of thing i notice is when i say "the team" in 1e i mean "the programmers". In 2e we meant everybody influencing dev. "respect" as a value hasn't been widely accepted. Michael Feathers - The thing that was key in XP1E was the emphatic message "with these constraints, this works.” Up to that point, everyone was trying to solve the "general problem" of software dev. Rachel Davies - for me - XP1E was about listening to customers without compromising code quality - a way to achieve balance.

Kent Beck - i would be very glad to see a dramatic increase in aspirations for outcomes accompanied by a dramatic decrease in dogma.

Kent Beck - not quite a definition but... "quality results in a steady flow of value" or "a steady flow of value indicates the presence of quality".

Kent Beck - shu ha ri is generally a power trip for the teacher. empathy, engagement, and modeling work much better if you can deliver them.

Kevlin Henney - Use of "general" and "flexible" are design meeting smells.

LinusTorvalds - Real men don’t use backups, they post their stuff on a public FTP server & let the rest of the world make copies.

Luke Hohmann - Surveys are about getting answers to questions. #innovationgames are about shared exploration of complex issues to gain insight.

Malte Foegen (via Hillel Glazer) - "Replace 'process' with 'work' everywhere you see it & ppl will not get so hung-up on process."

Mark Graban - Toyota people taught me to shift my mindset from "we can't do that" to "we haven't figured out how to do that YET."

Mark Twain - The best way to cheer yourself up is to try to cheer somebody else up.

Martin Fowler - I'd rather someone thoughtfully does something counter to my advice than someone blindly follows it.

Martin L. Shoemaker - If you want to get things done around here, you have to learn to think outside the boss...

Matt Podwysocki - @KentBeck now what about the anti-for campaign? I could get behind that one too. Composable functions over explicit loops.

Michael Bolton - Serendipity is the discovery of something in the absence of a goal to discover it.

Michael Bolton - Testing is not quality assurance. Testing provides information to those who make decisions about quality assurance (programmers, managers).

Michael Bolton - That which is ubiquitous without being influential is in obsolescence. (from Mark Federman @ #Agile 2008, and maybe from McLuhan).

Michael Bolton - There is no test that is guaranteed to be needed only once. True; as is the converse. A test of some kind will help us tell the difference.

Michael Bolton -To me, "going meta" means "going above the current level of conversation to try to figure out what we're really talking about".

Michael Bolton: We get into BIG problems when we confuse MEASUREMENT which can be qua [l nt] itative with METRICS, which are functions, quantitative.

Michael Feathers - The test of a philosophy is the state the world would be in if it were followed fully. Most philosophies posit impossible worlds. Human nature is the one bit that many philosophies don't account for. Some pay lip service to it, but then conveniently ignore it.

Mike Wesely - "Statistics are often used as a drunken man uses a lamppost; for support rather than illumination."

Napoleon Bonaparte - Ten people who speak make more noise than ten thousand who are silent.

Nicole Radziwill - Alex just invented a new word: "for-gonna-get" - when you haven't forgotten about it yet, but you will forget something in the future.

Ohno - “Do not codify method" [because improvement is never-ending, and by writing it down, your process will ossify].

Paul Seibert - Are you agile? Look for adaptation in the face of things you did not expect?

Paul Seibert - The right people don't need to be managed. if you need to tightly manage someone, you've made a hiring mistake.

Randy Nelson (of Pixar) - Core skill of innovators is error recovery not failure avoidance. [Is agile more about the former compared to traditional approaches that may emphasize the latter?]

Ricardo Semler - "It's unfair to expect all employees to feel passionate about their work."

Rob England - Open content standards those I know of have failed. Open OK, but need money and editors. COBIT 5 might fly.

Ron Jeffries - an estimate is a guess in a clean shirt.

Ron Jeffries - OK. Here's the deal. The fact that you think you need tools to support your distributed development is a sign. Read the damn sign.

Ron Jeffries - There is a big difference between "I don't know a better way" and "there is no better way".

RonJeffries - @jwgrenning well put. TDD style finds mistakes, preventing defects.

Scott Bellware - That decay that plagues community efforts (agile, alt.net, etc) is only inevitable if the community fears principled community organization. With a values statement of code of conduct, a community becomes an *intentional community*. A community with defined values and protocols can afford more diversity than a community that eschews definition for the sake of diversity. A community's core values necessarily creates a core group. as long as that group doesn't devolve into a clique, it strengthens the whole. On top of creating definitions that permit a core, the group needs values and protocols to recognize and address cliquishness of the core.

Scott Duncan - I think we should talk about (what I think is) DeMarco's main point: focusing on the goal of creating software that changes/transforms and the conception vs construction aspect of software. Note also that he does not reject the idea of engineering software, so understanding what that may mean still seems important, though control, predictability & consistency are not most important to him.

Scott Duncan - My reading of the article suggests the pt is to focus on building transformational sw, not expect cmd & ctrl will be the best way to structure doing that, and rethink what the engineering of sw needs to mean in doing so.

Scott Duncan - Perhaps why some at #agileroots felt "real" agile is about code & coding techniques, feeling social "stuff" is a distraction. Alistair noted that people saying this weren't around when the Manifesto was crafted. Kent Beck - how odd. the social stuff is the point. technique supports relationships. Ron Jeffries - Yes, many techies think "social stuff" is distraction. But no: valuable skill. At same time, one puts effort where one's heart is. George Dinwiddie - Many business people also think "social stuff" is distraction.

Scott Duncan - Stability seems relative: depends on how much change one can absorb/comprehend in some period of time. Could it be that acceptance/concern over an agile approach is about one's relative sense of stability?

Scott McKain - “What gets measured is what gets done” is true but also “What gets measured gets emphasized by management.”

Seiche Sanders (ASQ) - I've never been a fan of the shooting-a-mouse-with-a-shotgun problem-solving approach. Propagates more problems/costs.

Shigeo Shingo - What is the Toyota Production System? When asked this question most people (80 percent) will echo the view of the average consumer and say: “It’s a kanban system”; another 15 percent may actually know how it functions in the factory and say: “It’s a production system”; only a very few (5 percent) really understand its purpose and say: “It’s a system for the absolute elimination of waste.” Some people imagine that Toyota has put on a smart new set of clothes, the kanban system, so they go out and purchase the same outfit and try it on. They quickly discover that they are much too fat to wear it! They must eliminate waste and make fundamental improvements in their production systems before techniques like kanban can be of any help. The Toyota production system is 80 percent waste elimination, 15 percent production system, and only 5 percent kanban. This confusion stems from a misunderstanding of the relationship between basic principles of production at Toyota and kanban as a technique to help implement those principles. – Shigeo Shingo, A Study of the Toyota Production System

Thomas Cagley - The world is not rational and expecting people or even groups of people to act rationally is confirmation of the thesis.

Tobias G Mayer - If teams are not collocated, are they dislocated?

Tom Gilb - DeMarco is really saying we must control value not merely cost and time.

Vadim Zaytzev - “Formal rules for comments are difficult enough to be easily forgotten to be included in a language standard” Michael D. Hill - aren't comments for precisely the stuff we can't express formally? if we could, they'd be called "code".

William W. (Woody) Williams - Agile development requires continuous planning. Waterfall requires constant re-planning.

William W. (Woody) Williams - Despite rumors to the contrary, Scrum is not a project management practice, it is a software delivery method.

William W. (Woody) Williams - It is infinitely better to mentor and train people - risking they leave - than to do nothing and risk they stay.

Developing Standards with an Open Source Model - Possible? Any Interest?

2009-08-19T15:08:00.004-04:00

Rob England (@theitskeptic) retweeted (and agreed with) a Tweet from Antonio Valle (@avallesalas) who said "Why not create an open-source alternative to ITIL?" I responded asking, "What about an entire open source standards movement?"

I have, in the past, been involved with both ISO and IEEE software/systems related standards work. Without getting into an entire explanation of how standards bodies work, one of the issues raised by people on the "outside" is that it is very costly to get standards and difficult to find out the real status of ongoing work. At a certain stage in standards work, drafts are no longer available to anyone outside the standards working groups and voting bodies.

To be involved in such groups costs money, not because there is a high membership fee. There are 2-3 meetings of each per year, which means travel & living costs plus time away from work for 2-3 days each time. This cost dwarfs any fees associated with membership. Consequently, membership in such organizations usually means corporate support of some sort. Therefore, most of the people involved represent corporate interests. In the case of the software/systems areas in which I was involved (process related), most participants were either government bodies or contractors to the government (or consultant groups associated with government work).

The argument for the cost of standards is that the bodies overseeing the work have legal and administrative costs to cover (besides publishing ones) to ensure the fairness of the standards work. This fairness is very important since contracts and business often depend on meeting standards, so making sure the standards do not unnecessarily favor certain vendors or industry groups is very important. (In the case of ISO standards, the issue is to ensure countries or blocks of countries are not so favored.)

However, standards are made by those who show up either to the write or vote upon them. The former carries a good bit of weight since voters rarely spend the time to re-write what comes out of a working committee, though comments can be extensive. Part of the fairness issue is to make sure all comments are addressed and dealt with appropriately and in an open manner.

It can takes years for a standard to go from a proposal for new work to an actual accepted document. So the cost can be significant to participate, in money and time.

Now one way to have an impact on this process, is to come to the bodies with a draft of a standard already in hand. Of course, that is usually done by some member of the group or some document "sponsored" by such a member. Once a document is created by or adopted by a standards body, the copyright belongs to that body, i.e., they "own" the document. From then on, they are the ones entitled to make changes (or not) to it.

My thought when I asked about an open source standards effort would be to see if a standards document could be produced that could get industry support but be developed in an open source manner. I did some work on an agile-related standard for IEEE where I attempted to use email rather than face-to-face meetings to try to get broad international participation. But it was not open source and consensus was difficult to impossible to achieve that way with the large number of people involved. An open source approach might be able to overcome this as the open source community has addressed software developed in a "community" fashion.

Do people who have had open source and/or standards experience feel this could work? One major question would be how the results are handled and appropriate "stability" introduced so the result can be reliably used as a reference that is not changing too frequently given how standards are used over many years and may become part of contractual considerations.

Engineering Practice and Bridge Design

2009-08-17T17:48:00.002-04:00

In April of 1986, ACM’s CACM (Communications of the ACM) published an article entitled “A Computer Science Perspective on Bridge Design” which was an interview of Gerald Fox (a bridge engineer) by Alfred Spector and David Gifford: http://portal.acm.org/citation.cfm?id=5684.6327&coll=ACM&dl=ACM&idx=J79&part=magazine&WantType=Magazines&title=Communications%20of%20the%20ACM&CFID=47665715&CFTOKEN=13076261

Since it requires a subscription to the ACM Digital Library to get access to this article, I thought I would highlight some points in the article, specifically where it discusses ideas that have some relevance to software design practice. It should be noted that all the material quoted comes from Fox’s statements as Spector and Gifford acted as interviewers only, though they did control the kinds of questions asked. The rest of the commentary is my own take on the material.

The first point made in the article is the significant difference in approach used for small vs large bridges. Small bridge design can get by with “simple calculations or experience.” Large ones usual employ “several alternative designs” during a “preliminary design phase”. These alternatives are usually presented to a client with one recommended over the others. “There would also usually be hearings to get the public’s reaction.” In software, the “public” could be considered end-users.

The next interesting comment made is that long-term cost is not a major concern since “initial cost is the primary thing clients look at today.” This concern for initial cost has an important impact on the materials and design proposed and accepted. However, the costs for all aspects of design usually pale in comparison to actual construction cost. This is similar for most manufactured products. Thus, a good bit of effort over the years has been spent in ways to optimize and improve the costs for construction/manufacturing.

One telling remark made by Fox was that everyone involved with the design phases “have always understood that the quality of the design is more important than its cost,” again, because of the huge cost of construction. It is noted that design cost is usually “less than 6 percent of total cost, and even less for larger bridges.” This represents one major difference between developing software (which is almost completely a design activity) and creating physical products.

The next major point is that, “There’s usually not a lot of very complex analysis involved, unless the project represents a significant departure from experience.” I believe this is an area where software development has continued to struggle for decades. Ensuring that necessary (domain and technical) experience exists and that this body of experience grows in some organized fashion may be one of the great limiting factors software development currently faces. It contributes greatly to the perceived complexity in software projects. In this regard, complexity from lack of knowledge of existing experience may even be considered an “accidental” (to use Fred Brooks’ term) element of software development.

Related to this is discussion about the existence and use of “standard values for the allowable stresses and loadings” which exist in basic bridge design. Engineering also benefits from much knowledge about the characteristics of the materials it works with, which is where a good bit of basic science comes into engineering. In contrast, much software is built starting from atoms/molecules rather than whole materials, though reusable libraries, patterns, O-O design are/have been attempts to raise the level of design. Fox does note, however, that “additional criteria” are usually needed for larger bridges, especially with regard to “natural phenomena” where a good bit of risk analysis is performed “to establish acceptable bounds.”

There is, of course, considerable modeling done for larger bridges, both mathematical and physical, both to determine “where joints, pins, and other connections are to be placed” and to consider dead and live loads. In software, we can think of these as interface and performance design. A key in both bridge and software design, though, is determining “how many situations to account for.” In particular, both must be concerned for combinations of conditions. Using one of Brooks’ terms again, we have an “essential” aspect of complexity in this regard, both for bridges and software.

One factor crucial in engineering design for physical products is the effect of component wear. For example, in bridge design, there is “fatigue” brought about by the stresses the bridge undergoes both through use and the impacts of natural phenomena. Again, combinations of both are important to consider. Concern for fatigue and component failure drives assessment of safety limits as well as maintenance considerations. The former are a design concern while the latter is usually not factored into costs, though the needs of such maintenance are well understood. In this regard, software has little concern as it does not “wear out” and repair maintenance of software is due to flaws in the original design, not flaws which develop “naturally” through use. Indeed, Fox points out how “inadequate inspections” of existing structures, are more likely to be the cause of failure than design/construction flaws. However, many examples of catastrophic failure are design/construction flaws, e.g., Tacoma Narrows Bridge and Kansas City Hyatt Regency walkways.

The interviewers next ask about the numbers of people involved in design and the use of pre-made components. Fox notes that too many people involved in design can “get in each other’s way, and it becomes more difficult to keep everyone up-to-date with changes”. But design can be divided up between groups with “one engineer in charge who would ensure all the groups were designing compatible structures.” Regarding standard components, Fox states the there is “very little…except for small-span bridges. Most elements are built up out of steel plates.” Fox does say that there are some standard sizes for “angles and I beams,” but, overall, “standardization is not very great in terms of components” for large bridges. Fox continues by saying that “economies of scale with a large bridge may make it more logical to use nonstandard components.” (Again, however, “components” are not atoms and molecules.)

Quite telling is Fox’s statement that, “It’s difficult to get every nuance of the design into the drawings” to be used for guiding construction and that contractors “may add extra material to the structure to reduce the stresses.” Fox points out the “tendency for the consulting engineering firm not to be involved in the construction process.” He later states that he feels this is something that needs to change. In software, we might compare this to separation between groups doing architectural and high-level design and those involved in actual low-level design and coding. In both engineering for physical products and software “on site” decisions are frequently made to get things to “fit” properly.

Continuing with this theme of change and design modification, Fox states that, even with computer design optimization, bridge design relies largely “on experience and trial and error” during the design phase. Fox does say computers allow engineers to “be more precise” about safety factors. Henry Petroski, who has written extensively on engineering practice and design, expressed concern that use of computer programs for safety optimizations posed some danger as they tended toward less rechecking of calculations, etc. as would have been done before their use. Indeed, he talks about how safety concerns ebb and flow relative to cost considerations. Cutting safety close (being more “precise”) combined with on-site modifications sometimes leads to failures which promote a new phase of over-engineering for safety until, after a while, cost concerns begin to drive the tolerances closer, until the next failure.

Fox says, regarding failure, that this usually occurs when “we extrapolate beyond out experience and models.” And as Petroski and Fox both note, it being a general principle of engineering, failure drives much of the learning, though not necessarily catastrophically so. Fox does not that, in general, bridge design is “conservative with our loading” since natural phenomena and ultimate use cannot be predicated exactly, e.g., wind and traffic. Fox says believe that, though “understanding of materials is also quite good,” a safety-factor of “1.8” is commonly used to account for “variability in loads or materials, as well as possible mistakes in the model.”

When it comes to design “correctness,” Fox points out that “good engineers check everything” and that this is “always done by an engineer who did not work on the original design.” When all design documentation is complete, another review occurs “by a senior engineer for completeness.” One of the tradeoffs made that is checked has to do with redundancy versus safety factors in the individual components. Redundancy can keep a structure from failing even if some element of it fails. But such redundancy is not used in certain bridge types because of the cost. This is where extra safety factors are used to “compensate for lack of redundancy.” Small bridges are going to be more redundant, due to cost, than larger ones. (It is interesting that Fox says how there are “perhaps 100 failures of old small bridges in any given year” though fatalities in such cases are rare.)

At the end of the article, there is a section entitled “Editors Conclusions” which list the following as “specific differences in the disciplines”:

1) Bridge design “is much more structured than computer systems design.”

2) Standardized, conservative specifications exist for many aspects of bridge design and components.

3) In terms of person-years, bridge designs are not as complex as software systems.

4) There is greater attention to reliability in bridge design and considerable checking of design specifications.

5) Analytical models are used more in bridge design and are more advanced.

6) Bridge design produces specifications that are “explicit” and “comprehensible” by many outside of the design group(s).

7) Design and construction of bridges is separated far more than in software where they can be no separation at all.

Now this is an old article, but many of the themes sound quite familiar, even today. Over the last 10 years or so, I have found it very informative and interesting to read about various forms of engineering practice and design. As noted above, books by Henry Petroski are often noted by others and a book entitled Discussion of the Method by Billy Vaughn Koen provides a combination of practical and philosophical examination of engineering problem-solving.

Agile Leadership - More about Agile or More About Leadership

2009-08-17T09:38:00.007-04:00

This topic came up on Twitter earlier this morning and it's 140 character limit seemed to me to not be allowing proper addressing of the topic. I may be that everyone can actually agree, but the shortened communication form is getting in the way. In any event here's the way the thread went:

Bob Marshall (@flowchainsensei) asked "Is Agile Leadership more about Agile, or Leadership?" I replied "Former. It's used as an adjective, subsetting the latter." Bob replied "Your answer is not congruent with my world view, not my experience."

To this I posted a set of three Tweets which, combined said "Once you put 'Agile' in front of 'leadership,' by definition, you are specializing some aspects of leadership. Now leadership is still paramount, but once you qualify it, I believe the focus becomes of those aspects of leadership congruent with Agile Values & Principles rather than all possible leadership ideas.

Anna Nachesa "(@ashalynd) posted the following two thoughts: "is this not the case when sub-definitions undermine the original meaning?" and "sort of 'rectification of names' a la Confucius, w the purpose to give the people a new shiny symbol instead of old & beaten one."

But I think the original issue with me may have been my misunderstanding of Bob's intention. I was reading his question as whether leadership, generically, was what "Agile leadership" was about or whether it was about Agile ideas (applied to leadership). So I said it was the former.

But I believe, and tried to say, that leadership was the main point; however, "Agile leadership" was that general idea focused on leadership ideas that fit with the overall Agile Values and Principles. So I think Bob and I may agree. (I do not think "Agile Leadership" necessarily undermines the idea of leadership overall as I believe Anna suggested. Of course, now I may have misunderstood her!)

What do you think? How is adding the word "Agile" to the overall idea of leadership either helpful or harmful (in that it can distract from or diminish the idea of leadership)?

[PS: A follow up Tweet from Bob Marshall clarifies that he was "using the term to indicate 'Leadership of Agile initiatives'." And I Tweeted back that "I would agree then that, once in an Agile initiative, leadership matters most as, hopefully, Agile Vs&Ps are honored."

Bob further noted "My intent in the question was to explore if Agile initiatives are best led by folks with agile dev exp., or leadership exp." I definitely agree its the latter given this either/or but those leading need to understand Agile Vs&Ps wll enough to communicate with others involved in the initiative.

Thoughts on what "Agile leadership" may mean are still welcomed.]

On (Early) Failure (and Iteration Lengths)

2009-08-15T18:20:00.001-04:00

“Fail Fast” is a commonly heard idea in Agile development. This idea is that, if there is some doubt about a design/implementation approach, try something and find out quickly if it works or not. In this way, you don’t wait for a long time to find out if there is going to be a problem.

Now at the coding level, this can be tried without too much resistance from others. But what about this approach in a larger context? On Twitter one ASQ (American Society for Quality) daily quality quote offered was from Charles Knight: “You need the ability to fail. You cannot innovate unless you are willing to accept mistakes." The question is, at what level can failure be accepted?

With a coding spike, you’d refactor or try something different and, hopefully, find an approach that would work. You may even leave yourself some slack in the iteration commitment for just such trial-and-error. A similar application of slack is usually recommended for accepting stories into the iteration. This is normally done by committing to about a 75-80% person/day level for everyone on the team as a contingency. Failure here may be a story (or two). The goal would be to discuss why this happened at the retrospective and work to eliminate the causes going forward.

These kinds of failures, could be absorbed and viewed as something to be expected. What about failure of an entire iteration? In particular, few or no stories were completed. Could that sort of failure be absorbed? Probably not well into an organization’s release effort. But what about at the very beginning? What expectation for successful iteration performance exists when an organization first starts out?

I think it is very important to employ a “fail fast” approach to the entire adoption effort. Failing fast is about learning, of course, not really failing as it “nothing achieved.” So I usually recommend to an organization that they start with, at most, two week iterations. In this way, teams gain experience with every aspect of iteration behavior and can repeat this a few times – three at least – to get used to the rhythm of an Agile approach.

This is not to say there would be acceptance for delivering no stories, but everyone involved should agree that the learning which goes on in these early iterations is as important as delivery of stories. Identification and elimination of impediments is very important at this early stage in adoption, so effective retrospectives are crucial.

The goal, of course, is to get better and better at iteration delivery capability, but this is achieved, not by quotas, “win one for the Gipper” attitudes, etc. It’s achieved by people on, and related to, the teams seeing how Agile Values and Principles, as well as a specific method’s practices and techniques, will function in the organization’s environment. This will lead, hopefully, to adjustments in that environment to allow all these Agile concepts to become understood, accepted, and practiced by everyone.

Small, co-located teams might be able to begin with one week iterations. But I believe even larger (up to 10-15 people) teams that are distributed can benefit from sticking to no more than two weeks. As long as everyone involved understands the important of the learning that will go on and works to apply that learning each iteration, I believe two weeks is advisable. After the teams have a good feeling for how individuals (and teams) interact (including with management), what estimation and delivery commitment makes sense, and what technology support they can reasonably expect, iteration length could be increased.

I do not believe these early iterations should be considered “practice” efforts, though. It may be what they are at one level, but the results should not be treated as throw-aways. That is, the delivered functionality should be treated with production-level quality expectations and expected to be the basis for future iteration work. That very little such production-level results may come out of the early iterations should not be the concern, however.

Finally, it’s important not to set these early iterations up as “failure” cushion, i.e., allow teams to think they can afford not to try to do the best they can. The same commitment and accountability should be expected of them as would be expected later on; however, besides evidence of delivered functionality, evidence of important lessons learned should be considered valuable results at this point.

It is likely everyone associated with the Agile adoption will learn things, not just those delivering functionality. If everyone feels everyone else is, indeed, developing important experience in Agile concepts and behaviors, I believe early “failures” of functionality delivery can be handled positively and without “panic.”

What are your experiences with early team “failures” and the response to them?

On "Pointing" Stories

2009-08-14T06:54:00.007-04:00

One of the, if not "the," most common way for agile teams to estimate work for an iteration is what is known as "Planning Poker." Perhaps the best explanation for this approach is to be found in Mike Cohn's book, Agile Estimation and Planning.

My experience when watching teams try this the first few times is that they have a hard time going from story to story, estimating the point value of each story. I think this is because each story is treated somewhat in isolation from the others. Even if a team considers, say, the top 10 stories in priority from the Product Backlog and has that set as a focus, it seems hard for them to feel they understand what they are doing in assigning points to a story.

The idea, of course, is that points represent a relative relationship between stories, not some absolute value that connects with some other, more familiar, value. Sometimes, the suggestion is made that 1-point represents 1 "ideal day" of effort. That is it means ~1 person/day of effort. This can be helpful because it isn't too detailed, but does give a team some reference to something they can relate to.

At Agile Roots this past June in Salt Lake City, I heard a "lightning talk" by Chris Sims that made a lot of sense on this topic. Actually, what he suggested is a very simple, classic technique for ordering things without trying to be numeric at all, at the outset. Very simply, he suggested just ordering the stories using a simply "insertion sort" approach:

1. Pick all the stories you want to deal with for the estimation session (as it could be iteration planning or planning for an entire release, i.e., looking at the whole Backlog, not just a few highest priority ones).

2. Take any 2 stories and decide which the team thinks is larger (or smaller) than the other in effort, but don't try to assign any specific "quantity" to this.

3. Take 1 more story and decide where it fits relative to the 2 you already now have ordered. It'll be smaller than both of them, larger than both of them, or fit in between the two of them.)

4. Now, just repeat the last step for every remaining story, fitting each story, in turn, into the ordering that exists. (Don't spend too much time worrying about whether a given story is, perhaps, equal to one(s) already out there. Just drop it into the ordering close to where it seems to fit.)

5. When you've done this with all the stories, you'll have a complete ordering of the stories, relative to one another, from smallest to largest (in terms of effort ), but will not have had to worry about exactly how "big" any story is.

Now you can start the process of assigning points to the stories:

6. Look at the lowest ranked story and decide if it is so small you want to assign it "0" (it can be done in much less than a day) or whether you want to call it a "1".

7. Go to the next smallest and decide if that one's about the same as the prior one (and assign it the same value) or it is larger enough to assign the next number using the scale you have selected. (The reference describes one such scale know as a Fibonnaci series. This scale is suggested since, as stories get larger, the jumps between numbers increase. The idea is that it is not useful to debate aboput 1 pt differences between larger stories since this whole approach is a relative estimation technique, not an absolute one.)

8. Continue doing this with all the stories until you complete the largest one.

9. Now you have a fully pointed set of stories.

The same decisions end up being made, but it's done in two, perhaps easier to comprehend, steps. The first sorting step may be easier to do than thinking about both the realtive ranking and the point value at the same time. With the ranking done, there is an already implied sizing and it is a matter of assigning the points knowing that the values will increase as you go through the ordered set of stories.

This approach does not relieve much of the burden of deciding whether a story is about the same, 2x, 3x, etc. as the last story. But, you can be sure that it is at least the same or larger given that you are now pointing from smallest to largest.

I've found this approach to help teams. They can usually do the initial sorting without a lot of concern because it is, truly, a relative relationship activity. Assigning points is still somewhat vague early on in a team's experience, but the already ordered set of stories helps a good deal rather than randomly picking stories (even in business priority order) and trying to assign points immediately.

Have others used this approach rather than the traditional Planning Poker approach described in the text linked to from above? Do you think this is too much work compared to the typical pointing approach?

Summary of CACM Article on Evolutionary Development and US Gov't

2009-08-13T07:31:00.003-04:00

I’ve been reading some folders of "old" articles this week and ran across “The Profession of IT - Evolutionary System Development” by Peter Denning, Chris Gunderson and Rick Hayes-Roth. It appeared in Communications of the ACM in December of 2008 on pages 29-31. The specific link to the article’s index page on the ACM’s Digital Library website is:

http://portal.acm.org/citation.cfm?id=1409360.1409371&coll=ACM&dl=ACM&idx=J79&part=magazine&WantType=Magazines&title=Communications%20of%20the%20ACM&CFID=47138439&CFTOKEN=79695566

Unfortunately, you need an ACM Digital Library subscription to access the whole article. There are References from the article listed on the index page which, in themselves, might be interesting reading, assuming you have the books listed and an IEEE Digital Library subscription. A couple of the references are available as PDF files describing U.S. government acquisition issues related to evolutionary and traditional development approaches.

What I’d like to do in this post is quote/summarize what seem to me to be the key things in the article.

To start with, here is the authors' last paragraph:

“All the evidence says that that evolutionary processes works for systems large and small, and that risk seeking is the fastest route to fitness. There is too much at stake to continue to allow us to be locked into a process that does not work.”

In the last sentence the “process that does not work” is the traditional phased, sequential acquisition and development approach. For many, it may be assumed that this is required for government contracting. However, the authors say “that practices for adaptability are allowed under government acquisition rules.”

More surprising, perhaps, is the following:

“The U.S. Government Accounting Office (GAO) has scolded the government on several occasions for its uncommitted lip service to agile processes.4 The GAO believes agile processes could significantly shorten time to delivery, reduce failure rate, and lower costs. Many people resist the GAO advice because they assume careful preplanning minimizes risk and maximizes dependability and usability. However, more leaders are pushing for agile acquisition because the track record of the normal process in dynamic environments is so dismal.”

The “4” at the end of the first sentence of the quote refers to one of the references for two of the available PDFs:

“GAO. Defense Acquisitions: Assessments of Selected Weapons Programs. Report GAO -06-391 (Mar. 2006); http://www.gao.gov/new.items/d06391.pdf, and Information Technology: DOD Needs to Ensure That Navy Marine Corps Intranet Program Is Meeting Goals and Satisfying Customers. Report GAO -07- 51. (Dec. 2006); http://www.gao.gov/new.items/d0751.pdf. (Dec. 2006); http://www.gao.gov/new.items/d0751.pdf.”

Regarding the “sweet-spots” (my use of the term, not the authors) for both traditional and Agile approaches, the authors say traditional software engineering seems to have agreed that:
“preplanning is best for large systems where reliability and risk-avoidance are prime concerns, and agile is best for small to medium systems where adaptability and user friendliness are prime concerns.”

This idea has appeared in various places over the past 4-5 years. However, the authors “challenge that conclusion. Preplanning is ceasing to be a viable option for large systems. Moreover, many small systems aim to be ultra-reliable.”

However, the authors admit that there are challenges to using agile methods on government projects apart from beliefs about their appropriateness. In particular, the authors say that there are “organizational impediments to information sharing” in government projects and reference Cao, L. and Balascubramaniam, R. Agile software development: Ad hoc practice or sound principles? IEEE Pro (Mar.–Apr. 2007), 41–47. [Note: The second author’s last name is Balasubramaniam, no “c” but the final “m” is correct; and IEEE Pro refers to IEEE IT Professional.]

There are some other details and some examples of projects and results, including a parallel implementation trial by the World Wide Consortium for the Grid (W2COG). This summary, I believe, and the references available, should help you start to pursue this topic further if you are interested.

As always, I would be glad to hear comments from folks with government contracting experience using agile methods especially discussing impediments overcome and how that was accomplished.

Agile for Non-Software Work

2009-08-12T08:27:00.001-04:00

Not long after I encountered the Agile Manifesto, it struck me that the Values and Principles could be applied to work other than software development. I’ve encountered some resistance to that view as people feel it was software development experience which resulted in the Manifesto and that the validity of its Values and Principles is based on their effectiveness for software.Now going from the Manifesto to actual practices and techniques outside of software development would be, I agree, a harder next step. The Manifesto was preceded by software development methods that had already validated a number of practices. (It is true that not long after the Manifesto was created, people experienced in project management came together to create a Declaration of Interdependence which sought to extend the Manifesto’s concepts into a broader project management context not limited to software. However, many of these people were associated with the Agile community and had some existing software ties.)

What if we take the statement of the Values and Principles and substitute “product” for software, understanding that “product” could also include “service”? We would get one change in the Values and just a few more in the Principles. One Principle would need a bit more tweeking to remove the software-like sound: “The best results and product artifacts emerge from self-organizing teams.”

I believe the words “development” and “developer” could be left as is since one does develop a product or service. But if these terms grate on you, I’m sure they could be reasonably changed.
The point is, though, that there is nothing in the modified Values or Principles which suggests they can not apply perfectly outside of a software context. I have done this not to belabor the obvious when one makes a few simply word substitutions, but to make a larger point that I’m sure others have noticed.

Perhaps the next major growth in agile adoption could be to more broadly incorporate non-software organizations in working with the agile community as well as encourage companies where agile exists in their IT work to move application of the Values and Principle out of their IT organizations and into other areas of their businesses? In any Agile conferences, workshops and seminars, it is clear that many sessions have to do with matters that go beyond IT-only concerns. There are many talks about the social/cultural aspects of agile adoption. In fact, most of the well-known people in the Agile community talk about such matters more than technical practices. Look at the session titles and descriptions for the upcoming Agile 2009 conference for people with highly recognizable names in the Agile community.

I think more overt recognition of this might allow for an interesting (re)structuring of future Agile related events. What would be needed is for practices and techniques (and tools) not based on software development to be discussed at such events. (There has been recent growth of discussion around Lean/Kanban and Agile practices. However, this is more a matter of bringing non-software-specific practices and tools into the software context, I believe.)

I think this could be helpful in Agile’s growth since some adoption efforts seem to “stall” in IT, even if somewhat successful, because translation to areas outside of IT is not always direct and those areas, understandably, don’t see how the technical practices fit. Other parts of an organization need translation for, and practices related to, their own domains, though such practices, especially at the outset, would be expected to be informed by existing software-based ones.

It would be interesting to hear from people who have experience applying agile concepts to non-IT parts of a company and/or completely non-software domains. (Indeed, I just saw that Dean Leffingwell has written a very nice piece about this exact topic, using the word "solutions" where I used "product" above: http://www.informit.com/articles/article.aspx?p=1383182).

My (Current) Plans for Agile 2009

2009-08-11T06:40:00.002-04:00

While I work on some other posting ideas, I thought I'd provide my current plan for sessions I expect to attend at Agile 2009 in a few weeks.

In general, I have chosen to attend sessions to hear people I have not heard before (with a few exceptions: Cockburn, Patton, Jeffries & Hendrickson, Sutherland, Pixton, Hussman, Marick). And, as is clear from the chart below, I have some conflicts, especially the first day. Not sure what last minute conditions may drive my final decision. (Sometimes, I'll admit, I make such choices based on the least travel distance between time slots.)

Monday

11:00-12:30 The Agile Playground Mayer
What makes this Agile ours Sanders, Patton
14:00-14:45 What is an Agile Proj Mgr? Fewell, Reed
Creating Agile Simulations & Games for Coaches & Consultants Henrickson, Sims
14:45-15:30 Lets stop calling it "agile" Vodde, Mak
Why (so many) Testers (still) hate Agile Beaton, Bennett
16:00-16:17:30 Fully Distributed Scrum Sutherland, Schoonheim

Tuesday

11:00-11:45 Implementing Agile Across Distributed Teams Smits, Sulaiman
Release Planning (The Small Card Game) Hendrickson, Jeffries
11:45-12:30 Collaborative Tools with Distributed Customer Teams Hohmann
14:00-17:30 Nano-Incremental Development Cockburn

Wednesday
09:00-10:30 How to Bootstrap a Hyperproductive Team Sutherland, Downey
Producing Software Groove David Hussman
11:00-12:30 Help me to see... corporate culture Cyment, Mayer
14:00-15:30 Creating a Culture of Trust Pixton
16:00-17:30 Building Shared Responsibility Teams Behrens

Thursday
09:00-10:30 Only Dead Agilists Don’t Ask Questions Jepsen
11:00-12:30 May the Forces Be With You Claar, Shimp
14:45-15:30 Coach Aikido Gruber
16:00-16:45 Growing PMI using Agile Fewell
16:45-17:30 Eight Guiding Values Marick

What's your session plan?

It All Goes Back to Deming, et al

2009-08-10T15:30:00.010-04:00

The discussion about Agile, Lean and Kanban is relatively new compared to how long all of these ideas have been actively practiced/discussed individually. I recall a statement by Alistair Cockburn a few years ago (I think at an Agile 200n Conference, but can't swear to it) where he first noted how much similarity there was between Lean ideas and Agile ones. Of course, the Poppendiecks have been writing and speaking about application of Lean ideas to software for many years.

What strikes me in all ofthis (and other organizational culture, change/improvement, and quality-related topics) is how it all seems to end up connecting right back to Deming, in particular, and other "early" quality figures. What industry, in the USA at least, seems to have latched onto, though, were the technical/statistical practices advocated by such people along with some process ones. The people and value-centered ideas seem to have gotten far less attention in actual practice.

One small example unrelated to agile methods, per se, is how all the Capability Maturity Models got a lot of attention, except the People CMM. The P-CMM 1.0 version came out in 1998, followed by the 2.0 version 2001. A 2.0 2nd ed just came out in July. This document is about organizing the human resource systems in an organization to support and treat people in a manner that would complement the more technical process goals of the other CMMs.

Another example was some debate that went on between sessions and at the receptionat Agile Roots 2009 where the technical practices aspects of Agile methods were characterized as the "real" Agile roots. Again, Alistair not, when I mentioned it to him, that folks saying this obviously weren't the ones present when the Manifesto was written as the social/cultural ideas were the basis for it. Reading the Manifesto's values and principles, it seems hard for me to imagine otherwise. I found it strange that such debate went on at an Agile conference, but, perhaps, upon reflection, it isn't so surprising.

I have also been told that the Manifesto has been described by some customers as "content-free," meaning that there was nothing there to really disagree with as it was all rather social "motherhood" sounding. So, even when the social/cultural focus is recognized, it seems people do not find it substantive enough to consider as important as technical practices.

But, like the Manifesto, Deming's 14 Points are largely about organizational cultural change:

1) Create constancy of purpose toward improvement of product and service, with the aim to become competitive and stay in business, and to provide jobs.

2) Adopt the new philosophy. We are in a new economic age. Western management must awaken to the challenge, must learn their responsibilities, and take on leadership for change.

3) Cease dependence on inspection to achieve quality. Eliminate the need for inspection on a mass basis by building quality into the product in the first place.

4) End the practice of awarding business on the basis of price tag. Instead, minimize total cost. Move towards a single supplier for any one item, on a long-term relationship of loyalty and trust.

5) Improve constantly and forever the system of production and service, to improve quality and productivity, and thus constantly decrease costs.

6) Institute training on the job.

7) Institute leadership (see Point 12 and Ch. 8 of "Out of the Crisis"). The aim of supervision should be to help people and machines and gadgets to do a better job. Supervision of management is in need of overhaul, as well as supervision of production workers.

8) Drive out fear, so that everyone may work effectively for the company. (See Ch. 3 of "Out of the Crisis").

9) Break down barriers between departments. People in research, design, sales, and production must work as a team, to foresee problems of production and in use that may be encountered with the product or service.

10) Eliminate slogans, exhortations, and targets for the work force asking for zero defects and new levels of productivity. Such exhortations only create adversarial relationships, as the bulk of the causes of low quality and low productivity belong to the system and thus lie beyond the power of the work force.

11) Eliminate work standards (quotas) on the factory floor. Substitute leadership. b. Eliminate management by objective. Eliminate management by numbers, numerical goals. Substitute leadership.

12) Remove barriers that rob the hourly worker of his right to pride of workmanship. The responsibility of supervisors must be changed from sheer numbers to quality. b. Remove barriers that rob people in management and in engineering of their right to pride of workmanship. This means, inter alia," abolishment of the annual or merit rating and of management by objective (See Ch. 3 of "Out of the Crisis").

13) Institute a vigorous program of education and self-improvement.

14) Put everybody in the company to work to accomplish the transformation. The transformation is everybody's job.

I'd argue that points 1, 2, 6, 7, 8, 9, 10, 12, 13 and 14 are absolutely about social/cultural matters involved in organizational change. Points 3, 4, 5 and 11 each have some practice-specific ideas, though there are clearly social/cultural overtones.

I have also never seen an process improvement approach that is not some form of Shewhart's PDCA cycle modified, popularized and renamed by Deming as Plan-Do-Study-Act.

This is not to denegrate the Agile, Lean and Kanban ideas, but to point out their very valid heritage which has been recognized and used in most other industries for decades. I believe these newer approaches to work, taken together, speak unequivocally about how

- people need to (be allowed to) collaborate,
- communication and feedback provide visibility and reduce error,
- change is to be viewed and handled to mitigate risk,
- delivery of business value should drive work,
- continuous reflection and improvement supports quality.

And they do so through a structure that very much follows what Deming and others started telling us a half-century, and more, ago.

Agile Encourages the Kind of Employee Companies Claim They Want

2009-08-09T07:56:00.002-04:00

I've used the title of this post in talking to managers about agile on several occasions as I believe this is just what an agile approach does. The values and principles of Agile methods encourage the kind of attitudes and behaviors that most organization's want.

These include, but are not limited to

1) Personal responsibility for one's actions and integrity in day-to-day interactions;
3) Visibility into and openness regarding the status of work;
4) Collaborative relationships and effective communication within and between teams;
6) Self-direction and initiative in organizing work and handling issues;
7) Continuous improvement (in skills and process);
8) A quality focus and dedication to provide business/customer value.

If an agile approach is effectively pursued, an organization will see people adopting such attitudes and behaviors. Those who do not will not be successful in an agile environment.

Perhaps the best part of an agile approach, is that, within a relatively short time (usually no more than 3-5 iterations of effort), it will be clear who seems willing and able to adopt such attitudes and behaviors and who is/can not.

This is not to say that in this time frame, everyone who wants to will have succeeded in demonstrating such results. But it should be clear who really feels comfortable with and wants such "culture" change in the organization.

For this reason, I would think any organization who is truly serious about having the kinds of employees they say they want would give an agile approach serious consideration.

Agile Training - Values & Principles Are Essential

2009-08-08T16:55:00.004-04:00

(Just a short post today to put a stake in the ground for future posts.)

I believe it to be essential that any training for people (relatively) new to an agile approach include coverage of the values and principles, not just specific method practices and techniques.

Since most organizations are likely not to be able to instantly switch to a full agile approach, they will usually pursue some sort of "tailoring" as necessary to accommodate their situation. If they do not understand the Manifesto's values and principles, such "tailoring" may miss the point of the practice/technique being tailored. This will likely result in losing or, at least, diminishing the benefit intended by the original practice.

Substituting/changing practices may end up looking like the same thing is being accomplished on a surface level, but the larger value/intent behind the practice may be lost.

I have been brought in to coach/train organizations that have been pursuing an agile approach for a while. They often they feel they are not getting the benefit/experience they expected. The problems they bring me in to help address usually stem from an inadequate initial understanding of agile concepts. The organization has had training in method specifics, but seems not to have understood the broader context. The adaptations made to the method's practices (or invention of practices they felt were "missing" from their training) usually do not support an effective agile adoption.

At some point, such change/tailoring will have drifted them far enough from the agile values and principles that it hardly seems appropriate to keep calling what they are doing "agile." It may be an improvement over their old approach(es), but usually falls short of what they expected pursuing agile adoption to provide.

Because of this, I feel it is essential for people conducting agile-related training to make sure the values and principles are covered as the basis for any practice/technique training. In this way, an organization should, at least, be better able to understand the implications of the changes they feel they need to make and what they may be sacrificing by doing so.

(As I noted a couple days ago, I have guest blogged on this topic at Jurgen Appelo's blog and I expect to say more about this here in the future.)

On Anyone Being A Scrummaster

2009-08-07T07:33:00.003-04:00

There have been a few Tweets and posts on this, the latest of which was by John Sutcliffe. It rather sums up the issue. Certainly, any person, with the right characteristics, could become an effective Scrum Master.

What I think makes it hard for people in various roles are the expectations for their role which already exist when they seek to become a Scrum Master. Some of those expectations are their own while some come from their peers, managers, subordinates.

Sutcliffe mentions "guiding, facilitating, understanding, trusting, leading" as traits needed to be a Scrum Master (specifically in the context of a manager). If someone demonstrates these traits, then others will already have an expectation that this is how the person functions, so they will be more willing to "let" that person be a Scrum Master. That is, they will more readily accept that person in a Scrum Master role since they will already have some belief in that person's fitness for the role.

What can make it harder for a person in an "authority" role to become a Scrum Master is that they may have been more directive (even if benevolently) than a Scrum Master should be. They may be used to being an authority figure, being a person to whom others defer for decisions (organizational or technical). That can be hard to get away from because the person and others are used to expecting that.

To remove this from just the question of managers being Scrum Masters, I have seen lead technical folks have trouble in the Scrum Master role as well. Their problem was the feeling that they were "giving up" what they liked about their former role, to become a Scrum Master. What they liked, what they had been rewarded for, what gave them a sense of contribution to the organization, was making the decisions about and directing the technical work. Being hands-on as a technical lead was how they defined themselves and how they had been defined by others.

Analysts sometimes have an easier time than technical leads, but if they are more technical than business Analysts, they may have the same "withdrawal" concerns.

What I think should be clear when it comes to whether a person can be a Scrum Master or not is that:

1. They have been given a good idea of what the Scrum Master role requires;
2. They are not led to believe the Scrum Master role is "a way to get management experience";
3. They voluntarily take on the role, i.e., are not volunteered by others.

I have seen all these things happen to the detriment of both the Scrum Master’s experience and the team's. Were any of these disasters? No, people rose to the occasion and did their best given the circumstances. Unfortunately, that, plus limited training in agile generally, made it easier to think things were going okay.

An Open Invitation for Ideas

2009-08-06T10:27:00.001-04:00

If there are topics related to the theme/philosophy of this blog that you would like to see addressed, suggest them in a comment to this brief post. I certainly will have ideas of my own for things to post, but I'd be very happy to post ideas on things of specific interest to you. So leave a comment and I'll try to post on that topic so a discussion can begin around it.

This Blog, That Logo

2009-08-06T07:36:00.000-04:00

This blog will be devoted to all things Agile as well as traditional quality/process ideas as they apply to the goals of an Agile approach to product/service creation.

The logo represents the iterative cycles of an agile approach combined with the lines of the Kano quality model. Combined they form an "A", "S" and "Q" (of a sort) for Agile Software Qualities.

The overall philosophy of this blog is that the values and principles of the Agile Manifesto, combined with various "traditional" software process and quality concepts, provide powerful guidance for valuable product and service creation.

In particular, effective adoption of an Agile approach depends upon a clear understanding of the Agile values and principles, not just the practices and techniques of particular methods. Since many organizations will be unable to adopt a fully Agile approach instantly, they will make certain adaptations to it, especially at the practice/technique level. Without an understanding of the values and principles, these "tailorings" can seriously impede the organization's successful adoption of an Agile approach by losing the actual benefits intended by the practices. (I have guest blogged on this topic at Jurgen Appelo's blog.)

So welcome...I hope you share some of my philosophy, but encourage your comments (on this and future posts) whether you do or not.