Agile 2009 Notes - Tuesday

[Sorry I am behind on these, but I'll be catching up today and tomorrow.]

The following material describes the sessions I attended.

Tuesday AM –

Alistair Cockburn’s Keynote, “I Come to Bury Agile, Not to Praise it”

Alistair’s presentation began with a recitation of a revised version of Marc Antony’s speech over Caesar’s dead body from Shakespeare’s Julius Caesar. He said he did not feel Agile was really dead, but that it had [begun to] become absorbed in the mainstream of how many groups do software. Like a large, melted iceberg, he said, Agile was now part of the ocean. As such, Agile has begun facing classic software development problems related to large team design efforts. To address this situation, Alistair made three main points related to (1) software development as a cooperative game (a classic theme of his), (2) craftsmanship (a recent “movement” in the Agile community), and (3) lean concepts.

Game – Alistair showed his matrix of competitive and cooperative game types along the scale of finite, open-ended and infinite time periods. Software development is a cooperative game which is finite (in length) and goal-directed. Alistair note that software development had two goals which could conflict: delivery the software [on time and on budget with acceptable quality] and “setting up for the next game.” The latter meant making sure the work done made it possible to do the next work. This involves effort that will pay off in the future, but not necessarily now. Thus, there will be a cost added to the current work that may show no immediate value and, thus, be hard to justify in the short-term.

A significant problem in software as a game is that “positions don’t repeat” (often), so strategies for playing the game must be adaptive. Being adaptive, however, means being able to communicate and understand effectively. Alistair described another of his classic themes: how the best communication is face-to-face and how distance between people who need to communicate has a significant cost that is often not accounted for in project structures. Distance, he said, affects whether people can detect the need to communicate, care enough to do so, and be effective in making it happen.

Craft – Alistair listed 7 major crafts: deciding what to build, managing people and projects, modeling, designing an external view, large-scale (architecture) design, fine-scale (programming) design, and validation. At this point, Alistair reviewed another of his classic themes, the shu-ha-ri learning stages where one learns technique, collects more technique and finally invents/blends techniques.

Lean – Finally, Alistair talked about work flow and bottlenecks, noting that software’s “inventory” that moved through the development process is “the unvalidated decision.” Lean aims for “continuous flow” and people waiting on decisions create the bottlenecks. [One problem with such waiting is that, to move along the expected schedule, people will often “make up” decisions that either go unknown or are learned of late and require expensive rework.]

Unlike manufacturing, software development requires feedback/learning as design is knowledge acquisition. Alistair spoke about waterfall as a late-learning approach to developing software since, until a coherent system delivery for testing occurs, there is not much information about the actual state of what will be delivered. Agile methods, in contrast, incur cost earlier to learn earlier through continuous integration such that a good idea of the state of the work occurs early enough to make risk-reduction decisions. This allows business to “trim the tail,” that is, make a decision to deliver on time (or early) or delay delivery to get more (or better) functionality.

In closing, Alistair said that 21st century development will use the game, craft and lean ideas he noted.

Tamara Sulaiman – “Tips & Techniques for Implementing an Agile Program Across Distributed Teams”

Most of this talk was based on ideas from John P. Kotter’s book Leading Change in which he identifies 8 change concepts:

1) Establish a sense of urgency by “going for emotions” and being clear about the passion for change.

2) Create a guiding coalition of people from across the whole organization who are credible and committed to change. They should be willing to model agile behaviors for the rest of the organization, i.e., being cross-functional, running their meetings/work with one another in an agile fashion, being visible in the coaching/training given to other people in the organization, and maintaining a backlog of transition tasks with a Product Owner. [Audience member suggested forming, not a PMO but an AMO: Agile Mentoring Office.]

3) Develop a vision and strategy by showing what success in the new way would be/look like.

4) Communicate the change vision through slogans, graphics, posters, wiki pages, a glossary of new terms and how they relate to older ones. [There could clearly be some argument about this as it is how every “flavor of the month” program gets done in companies. Having point 2) clear would make this “marketing” acceptable and likely effective. Otherwise, it will be ridiculed.]

5) Empower broad-based action to allow change impact to go beyond development teams and IT as well as to provide an environment where it is okay to fail as long as the learning is clear and progress continues.

6) Generate short-term wins [to show progress] and use a wiki where teams can write about their experiences.

7) Consolidate gains and produce more change using lean, process flow, etc.

8) Anchor the change in the culture, producing a high-trust organization.

Overall, though the material was interesting, the talk was not that much about agile-specific issues in distributed teams. It was more a focus on organization-wide change management, which could be applied to any methodology, for example.

Luke Hohmann – “Leveraging Collaborative Tools with Distributed Customer Teams”

Again, while some interesting ideas and pointers to other resources, this talk did not address substantive distributed team issues. Some important concepts mentioned, however, were:

- Collaboration is always about the goals, so being clear about goals matters. [Collaboration is not simply ad hoc “cooperation” among people.]

- Effective goals are expressed in verbs, i.e., action.

- Understand how fast a customer can accept the output of agile teams. [Cannot collaborate well if the work cannot be shared effectively.]

Hohmann pointed the audience to a Harvard Business Review article entitled “In Praise of Hierarchy,” noting that there are many effective uses for hierarchy, authority not being one of them but being the one most often associated with hierarchy. Hierarchy can be very helpful in handling complexity, for example.

Tuesday PM –

Jurgen Appelo – “What (Else) Can Agile Learn from Complexity?”

Jurgen has one of the most popular blogs in Europe (related to software development and management) and his talk was an interesting exploration of ideas related to complexity. Basically, Jurgen took a series of quotations about complexity and related them to various ideas in agile methods. He began, as might be expected, with some quotes from Ken Schwaber, Jeff Sutherland, and Jim Highsmith discussing very direct links with agile methods. He then displayed a detailed graphic of the history of complexity science from people like Norbert Weiner through work being done to explore web/e-science ideas.

During the talk, he referenced numerous books and articles, including one in particular on social complexity and management. Jurgen is, himself, working on a book related to management in an agile context as he is CIO of a software development company in the Netherlands. Though the article describes 4 types of complexity, Jurgen’s emphasis was on social complexity because of its complex, unordered, and human-centric focus. This led him to some discussion of self-organization which is key in agile development philosophy. Interestingly, he asked whether an agile team, being coached, is really self-organizing, pointing to the Wikipedia definition of self-organization as including the phrase “without being guided or managed by an outside source.”

Jurgen then described various complexity-related principles, including:

Darkness Principle – systems elements are ignorant of overall system behavior since all system complexity cannot be present in each element; thus no single person (e.g., project manager) can monitor and control a whole system since they cannot know everything [and should learn to make use of self-organization and self-management in teams to help manage the complexity].

Law of Requisite Variety – to have a stable system the states of any control mechanism must equal or exceed that of the system under control and one (project) manager is less complex than the project being managed.

Boundaries and Conditions – self-organization requires that it operate within a boundary which defines the “self” being organized, thus agile management is an important part of agile development in setting such boundaries, i.e., teams do not get to make all decisions about everything.

Hierarchy Principle – “complex natural phenomena are organized into hierarchies wherein each level is made up of several integrated systems,” harkening back to Hohmann’s statement about the value of hierarchy and using a Scrum of Scrums as an example of one that can grow naturally rather than being imposed. Jurgen also noted a “patchwork” approach to Scrum relationships without hierarchy (and referenced Mike Cohn’s Mountain Goat Software site for further discussion of this).

Group Size – where it was noted that some research has pointed to ‘8’ as the most likely group size to lead to deadlock.

Specialization – many advantages accrue in complex systems when there is some level of specialization/division of labor.

Power Laws – which are related to that fact that there is a high chance of small issues and a low chance of large ones, suggesting that prediction of velocity into the future “includes an (impossible) estimate of the size of unknown problems.”

Dependence on Context – “the method to manage the project is embedded in the context and one must allow the emergence of such a method through interaction between the actors and the environment,” which led Jurgen to say that “ScrumButs are natural and necessary.”

Fitness Landscapes – is about how we “create the environment,” it is not “separate from us,” or, quoting a Spanish phrase, “My friend, there is no road, You make the road as you walk.” (This all relates to how a released system will affect, and change, the environment into which it is released.)

Incomprehensibility – states that “there is no accurate representation of the system which is simpler than the system itself,” so any model of it will be wrong, though, as George Box has said, “all models are wrong, but some are useful.”

Stephen Palmer – “Working with Large Backlogs”

Palmer discussed a variety of methods for backlog “triage” and management, including epics, themes, hierarchies, pruning, demand management, and extended kanban. He directed the audience to http://www.limitedwipsociety.org/, in particular, with regard to the last approach and mentioned some ideas from David Anderson on time-to-market psychology, e.g., on a 6 week project with 2 week deliveries, there is not much schedule concern, but on a 3 month project with 1 month deliveries, schedule will matter.

Dan Mezick – “Group Relations and Social Systems”

This was basically a talk about group relations theory and how it could be applied to agile teams (to some extent). Dan noted that he had a talk Thursday AM covering the ideas (of boundaries, authority, roles and tasks) in more detail and I’ll cover that talk in Thursday’s summary. This talk began by noting work by Wilfred Bion and his book Experiences in Groups (1961) which covers the main theories in this area. In particular, Dan stated that true groups demonstrate “interdependence of tasks and fate” since survival is a key group driver. For example, he noted that wolves, on their own, must survive on meager food sources (i.e., small animals) while those in packs have richer sources (i.e., moose, caribou). A key issue in maintaining this survival and accomplishing tasks is avoidance of distractions since distractions = waste. Unfortunately, group relations can produce a great deal of waste, but agile methods, like Scrum, employ various “ceremonies,” “artifacts,” and “practices” to “short circuit inattention and drift.” This is done by leveraging “inattentional blindness” which research has indicated means people with a clear focus on activities of immediate concern can block out other things which might place a claim on their attention. In a sense, this supports the idea of “we see and hear what we expect.”

Dan then discussed planning which he said is a prediction, but “prediction is a judgment” and “judgment is a belief” and “belief is a filter” and “filters can distort reality.” All belief demands attention, which can produce waste if the attention is not on the reality of what needs to be done at the moment. Impediments in Scrum are one kind of distraction.

Dan continued by noting several resources/references related to group relations such as LeBon’s study (1895) of loss of individual identity in crowds, making them more easily influenced. Crowds exhibit “system-level emotions that are inherently primitive” as well as seeking dependence on leadership. Indeed, a group will usually seek out leadership that “voices” its main desire. Work by McDougal (1920s) focused on smaller groups who were task-oriented (not the unorganized crowds LeBon studied). Work by Kurt Lewin (1940s) on field theory influenced Bion and was the pre-cursor to modern group relations work. For Lewin a field is “the totality of coexisting facts which are conceived of as mutually interdependent” such as Scrum’s goal of co-location, its roles, its artifacts, and its “ceremonies” (e.g., stand-ups, planning meeting, review, retrospective). Dan also noted Tuckman’s (1965) “forming, storming, norming, performing” model of small group development to which Tuckman added “adjourning” (1977). Tuckman’s work frequently mentioned Bion’s.

Dan nexted spoke about attention and comparing Scrum to more waterfall approaches. In Scrum, “you cannot drift very far from stated task[s].” Waterfall approaches are “not an ‘attention harness’ in any sense of that word” since “all sorts of attention is diverted away from stated task[s], lengthening” their duration. In Scrum all distractions are treated equally, “they are waste.”

Dan finished up by discussing group relations “conferences” (which sounded more like intensive week-long workshops). He also pointed us to the Washington-Baltimore Center for the Study of Group Relations which he said has a variety of good resources on group relations.