30 March 2009

Mitigating Mediocrity in Middle Management

James Shore just wrote a post on "mediocrity" which nicely explains why Agile doesn't always scale well: It isn't Agile that's being scaled, but corporate inertia. If an organization isn't going to commit to doing things differently, they have succeeded only at mediocrity.

Some of the follow-up comments got me to thinking about the problem. People were asking how to get middle management on-board. I think a few of the questioners realized that finger-pointing, and arguments, and directives to "Just do it!" are counter-productive. It's not that managers don't see the value in Agile methods, but it's not clear how their day-to-day behavior is supposed to change.

We need a little root-cause analysis here. Businesses are run by people, and people are motivated by a variety of things: Self-interest, compassion, fear, joy, confusion. Change is unnerving, particularly when one is not the instigator of the change.

I find that people resist change at work when work itself is a source of stability in their lives (e.g., they would rather be at work than at home), and/or they see their job as just a job, and not a productive, change-generating career. In traditional organizational structures, middle management is detached from the two most dramatic regions of the org chart.

So if I'm a middle manager, and I like my stability, I resist the change. If I would like to be part of the change, I see my own involvement as limited. And, do I even have a role (and a job!) in this new world order???

The good news is that, on a truly agile project, there is plenty of interesting work, and learning, for everyone. Our first step is to stop thinking of them as managers, and start thinking of them as leaders.

I once heard a very good quote which helped me out of a career rut. I'll paraphrase: "Not everyone gets to do what they love [C'mon, I'd be co-authoring novels with Larry Niven and Vernor Vinge. In Maui. Shaka, Bra!]; but we must love what we do."

That's a non-mediocre goal, without being an impossible stretch towards perfection. You help the team by helping yourself grow, and you help yourself by helping the team.

I can't count the number of times I have met a manager who says, somewhat forlornly, "I used to write code..." Leadership involves some wonderful skills that we can learn and be proud of, but we may want to reconsider the "one ladder" approach of moving our most talented technical people into pure management roles.

Nor should we necessarily get our managers directly from a pool of young, non-technical, certified PMIs. We need to be picky, and find people who are familiar with technical endeavors and are willing to lead people towards a common goal.

Maybe that's why "coach" has such appeal for me: It's technical, and people-oriented. I help other people (developers, testers, managers, directors, VPs, everyone) find ways to enjoy what they do while successfully building quality software products. I do love what I do!

Do you? (If not, we can help.)

Further Reading

Jim's post: http://jamesshore.com/Blog/Stumbling-Through-Mediocrity.html

Agile Bob's comments regarding Jim's post: http://www.agileforall.com/2009/03/30/new-to-agile-dont-settle-for-mediocrity/

28 March 2009

A Fresh Perspective on Shu-Ha-Ri

Shu-Ha-Ri is a learning model. It is one of those timeless formulations that is both pragmatic and universally applicable. Like many good, ancient models, it is elegant and simple. Of course, misapplication of such a model can lead to oversimplifications and missed details.

I recently stumbled upon a passage in a book that made me think of Shu-Ha-Ri, although the author did not mention it explicitly. I'll get to that, but first...

What is Shu-Ha-Ri?
Shu-Ha-Ri is a very old way of looking at the way we learn and teach new things. Shu-Ha-Ri divides understanding into three levels, Shu (hold), Ha (break), and Ri (leave).

In more detail (and perhaps colored by my own interpretation):

Shu: The beginner, or apprentice, needs (and usually wants) to be given the rules, so that he or she can be productive. The Shu practitioner follows the rules, and wants to know when the crayon is moving outside the lines. Creative interpretations by a Shu practitioner can lead to disaster. At this level, finding a good teacher is essential.

Ha: The explorer, or journeyman, knows the rules, and when and how to bend them. There is more creativity involved. Exploration "outside the lines" by the Ha practitioner usually results in a high degree of craftsmanship. Self-learning can occur, but the practitioner is also thirstily seeking varied mentors.

Ri: The expert, or master, is not hindered by the rules. In fact, the master practitioner can create new rules, and sometimes these rules seem to conflict with old rules. Some have suggested, tongue-in-cheek, that this level can be summarized as "Rules? What rules?" This is not accurate: The master of a practice can see the underlying truths behind the rules, and thus does not need to consciously think about the rules. This does not imply that a master can do whatever she wants. The master's actions embody the truth discovered within the context of the practice. Being an expert at something does not make one perfect at anything.

No One is Perfect

No one is an expert at everything, and only a newborn is a beginner at everything. Each of us has a level of proficiency in a variety of skills. It can take a lifetime to reach mastery for even a single practice.

No one stays at one level. A beginner can have a creative insight that turns out very well. An expert can forget and relearn the simplest and oldest rules. I'm told that if you think you've mastered something, you probably haven't; and if you say you've mastered something, you certainly haven't.

A Strange Loop
Experts (Ri) may have a hard time teaching beginners (Shu). (I think it was James Shore who pointed this out to me while we were building a course together.) Training comes from at most one level "higher" than the intended audience. Masters (Ri) can provide excellent examples by how they live their craft, but are not always able (or willing) to communicate effectively with an apprentice.

So those who can train beginners in a practice have learned another skill: Training. It's not enough to be an expert alligator wrestler if you want to train others to wrestle alligators.

Since becoming an "Agile Trainer," I've realized that there is a whole new dimension to my interests. I try to absorb anything I can regarding software-development, process, and teaching.

Learning to be a teacher... Is that Shu-Ha-Ri applied to Shu-Ha-Ri?

Taking Our Places
I recently read a pleasant little book called Taking Our Places: The Buddhist Path to Truly Growing Up by Norm Fischer. In it, there's a section about teachers and teaching, and he outlines three levels:

Literal: We follow the rules to the letter.

Compassionate: "...we may find it necessary to go beyond rules, laws, or customs, motivated...by compassion, which sometimes causes us to transcend the literal level of good conduct for the purpose of helping others." [Fischer, p. 182]

Ultimate: "...we come to appreciate that the [rules] are deeper than we had ever imagined; so deep that they can never be completely understood." [Fischer, p. 182]

Sounds a lot like Shu-Ha-Ri, yes? Fischer never mentions Shu-Ha-Ri, but I suspect the parallels are not coincidental.

I'm quite inspired by the implications: I notice that the level that may be most suitable for trainers like myself (Ha) maps to Fischer's level where compassion is the primary motivation.

Further Reading
There has been a lot of "tweeting" recently regarding Shu-Ha-Ri:
http://search.twitter.com/search?q=shu-ha-ri
Wikipedia has a (very) brief introduction:
http://en.wikipedia.org/wiki/Shuhari
Ward's good old (original) wiki has one of the best introductions I could find on the web:

http://c2.com/cgi/wiki?ShuHaRi
Alistair Cockburn made some valuable clarifications:

http://alistair.cockburn.us/Shu+Ha+Ri

18 March 2009

Ward Cunningham's Debt Metaphor Isn't a Metaphor

Last month, Ward Cunningham posted an excellent video on YouTube regarding refactoring and "debt". If you haven't seen it, I have it for you right here...




But is it simply a metaphor, or is he describing a real fiscal debt associated with software development activities?

The form of technical debt that Ward is referring to is noted by developers as poor, brittle design. Since a poor design is one that's difficult to change or maintain, it will take developers longer to add features, and their changes will more likely result in defects. They will tip-toe slowly and cautiously within (or around) untested code, and even the best developer will inadvertently introduce the occasional defect.

Isn't developer time expensive? Of course. So, in effect, Design Debt is real debt. I wouldn't suggest you create a liability account in QuickBooks: It's difficult to measure the actual dollar value, and it manifests itself in subtle and even contradictory ways. For example, remember the old story about the manager who proclaimed, "if only my developers could type faster!"? That's akin to saying "if only I could use my credit card faster!"

Design Debt feeds into another form of debt which is easier for the business to see: Quality Debt. This can be quantified as the number and severity of defects experienced by users. This metric has a fairly direct relationship with the financial impact of lost sales, time spent on support calls, developer time spent debugging the product, and on and on (and on).

Yet how many teams just assume that Quality Debt is "the nature of software development"? Well, doctors were using rattles and snake venom 50 years into the profession of medicine. Aren't you glad they're not still clinging to those archaic practices?

Unfortunately, what we usually try to do may just make it worse...

The functionality grows, the complexity grows, and it all has to be tested. We hire more testers, or give ourselves longer for the test "phase" of an agile iteration. (Yes, this "mini-waterfall" approach has been noted on many "agile-ish" teams. If waterfalls are bad, why are more of them better???)

And the stack of manual test-cases grows and grows, until the poor test team is working late at night to get them all tested: Point-click-type-point-click-type... and even our infallible testers make mistakes!

Or, the team runs only the "critical" tests each iteration, and saves the whole suite for the pre-release shake-down cruise. But which tests--which product features--are not critical? And waiting until the end...well, that sounds a lot like waterfall again, and also sounds like a Home Equity Loan, if you get my meaning.

The third form of debt is Testing Debt. The anecdotal measurement of Testing Debt is the height of the stack of printed manual test-cases. A better measure would be the time it takes to run the entire regression suite. How long after a change does it take the team to know that they haven't broken anything they've already built?
On the University of Michigan Transplant Center's two-year "OTIS2" rewrite of their aging Organ Transplant Information System; and after weekly reminders that "a mistake could kill a patient"; the answer was 15 minutes.

My goal isn't to make you feel bad in comparison, dear reader, but to point out what's truly possible without any technical debt.
Is Testing Debt real financial debt? Well, there's testers' time and salary. Also, Testing Debt again feeds into Quality Debt. If you don't assure that there are no bugs, your customer will find them for you.

But It's 0% APR, Right?

There is interest. You know what you get when you add features to buggy code? Yeah, more buggy features.

Thanks! You've Ruined My Day. Now What?

First, in the immortal words of Douglas Adams: Don't panic. How does anyone get out of debt?

1. Make regular payments.
2. Stop accruing more debt.

Examples of how agile teams can pay down technical debt:

Design Debt: When tasking and estimating stories, the team should consider the refactorings necessary to get any new story designed correctly. Those refactorings become explicit tasks.

Quality Debt: Set aside time within each iteration to explore and fix high-priority defects, or create a story for a group of related bugs. On a well-running agile project, each reported defect becomes a story of its own.

Testing Debt: Set aside time within each iteration for testers to automate those "critical" tests, or a particularly onerous set of tests. If you're not ready for that, create a story for the team (testers and developers working together) to explore automation tools, including record/playback tools as well as ATDD tools such as Fit. Yes, you're going to have to slow down the introduction of additional stories/features.

Ways to avoid new debt:

Design Debt: Developers, adopt Test-Driven Development (TDD) and refactor in tiny increments as you identify seemingly insignificant trends towards poor design. Whenever a task or story requires a new variation in your design, first refactor the appropriate piece of the design into one that follows the Open-Closed Principle for that variation. (That's as concise as I can be without going into gory detail here. Explaining this technique requires lots of UML, at least, and I'd probably rather use a video medium.)

Quality Debt: Pin down desired behavior immediately with fast, automated tests. Adopt TDD and ATDD (Acceptance-Test TDD) practices to keep away any new defects. Avoid feeding Quality Debt from the other two forms of debt.

Testing Debt: Stop writing manual test cases, perhaps today. As soon as practical, build acceptance tests for stories using something allowing test-first automation such as Fit, Robot Framework, or Cucumber. It's also up to the team to find ways to make those tests run fast.

04 March 2009

Let 'Em Eat Cake

The Case of the Missing Piece of Birthday Cake

Recently I spoke with a friend (let's call him Pete) who manages a software project, and he was telling me that his division had recently stopped paying for employee birthday cakes. More accurately, they had discontinued the funding of petty cash expenses. He apparently told his team "Would you rather have jobs, or cakes?"

Pete's small, profitable division of a struggling gargantuan company had been a small, profitable company of its own. It was sold, and the founders had all departed with well-deserved wealth.

Employee birthday cakes were a carry-over from the good old independent days. So the thinking in upper management may have been this: "It's unfair to the other divisions to be giving this perk only to the new acquisition, and we can't afford to give such a perk to all divisions."

Pete's solution was to collect a small bit of funds from each person on the team, so that the tradition could continue. It sounds reasonable, and Pete seemed pleased with himself. But then he continued...

There is one guy on the team (Carl) who conveniently forgets to pay in, yet eats a piece of cake, "or possibly two!" On one recent occasion, Sally (another team member) had purchased the cake with her own funds, and "now she is still going around trying to collect from everyone else," including Carl the Cake Thief.

Pete finished his story with: "Carl is lazy, and he's been in trouble before, and people talk about him behind his back, but he has a lot of product experience, and I can't let him go..."

Biting My Tongue


At the time that Pete and I had this conversation, I was acting as host, not consultant, and kept quiet. Pete was on vacation, and simply needed to vent. But my response would not have been as simple as this post's title would suggest.

It's true that I would encourage Pete to find a way to continue the birthday cake tradition. Sharing a meal is a natural team-building activity, and small, simple rituals are good for morale. But imagine how Carl's perceived pettiness is generating bad feelings. And surely Sally has some critical task, more important than collecting dollar bills and gossiping about Carl's second slice of cake.

And did Pete really say "...jobs, or cakes?" to his team, during a difficult transition and a very deep recession? I can't imagine how one could deliver those words without it seeming like an ultimatum, and thus putting everyone on the defensive.

I see two management anti-patterns at work here. First is the "Penny Wise, Pound Foolish" approach taken by upper management: cutting off a cost without regard to the long-term effects on profitability. How often do we need to see the proof that employee morale has a strong impact on corporate profitability?

The second misstep is Pete's attempt to make things better by "Treating Symptoms."

Go Lean

I'd start with two principles straight from the Lean playbook: Look to eliminate the greatest waste, and respect your people.

Had upper management worked to identify the largest waste in the whole organization, cakes would probably not have been at the top of the list. And if they were, what about the costs of chipping away at a profitable division by removing its small rituals?

Whatever is at the top, the next step is to use Root-Cause Analysis: A heavy phrase for a simple technique of repeatedly asking "Why?" and looking deeper each time until you find the underlying "root cause." Perhaps these cakes were originally Black Hound cakes overnighted from New York...

Then we try one or more of the proposed solutions: For example, Pete (who is a fine baker) could start baking the cakes from a boxed mix.

Much of the burden of change probably falls upon poor Pete. He needs to bring any issues that will affect the performance of the team directly to his superiors. Honest communication must flow in both directions in order for a team to succeed.

For a manager from an acquired company, during a monster recession, telling the boss that a proposed change carries risks may seem like political suicide. Rightfully, Pete is concerned about keeping his own job. But for the long-term health of his team, his company, and his own career, this is the very point where Pete needs to take a few risks and show off his leadership skills. He needs to keep the team running smoothly and generating valuable results, not quietly bickering over a slice of cake.

One of his first tasks should be to talk privately to Carl, the cake-eatin' code-cowboy. Is Carl having tax-time troubles after exercising his stock options? Does he need to hang on to every dollar, and eat every calorie he sees because there's no food at home?

Was Carl the top technical guy before the acquisition, and now he's buried somewhere in a 20-layer org chart?

There are people who are truly, simply lazy, but more often people are just exhausted, depressed, bored, or unnerved by all the organizational change. Pete needs to find out what's motivating Carl, and Carl should be encouraged to find his place in the new organization, or perhaps look elsewhere.

Touchy-Feely Psycho-babble?

I tend to give people who are causing trouble the benefit of the doubt. When people are able to see how their actions impact their organization, the people around them, and their own careers, they tend to behave helpfully and professionally.

Has it ever backfired? Oh, yes. I've worked with people who repeatedly ignored requests to alter unprofessional behavior. (Yes, we fired them. We also adjusted our hiring practices, but I'll save that story for another post...)

The successes will always far outnumber the failures. Getting a key player to enjoy his work again is much more valuable in the long-run. Were I to focus on the failures, I might stop trusting people, and immediately fire poor performers. Results: The team would lose all trust in me, and would stop providing critical information regarding the project. Deeper failure lies in that direction.

Mitigation

What if Pete talked with Carl and gave him a second chance, but Carl doesn't try to improve, or abruptly quits? Remember, Carl was a key part of the product team prior to the acquisition.

Well, it would be too late for Pete, then, unless he had the foresight to mitigate the impact of Carl's departure.

No single team-member should be able to effectively ruin a team by quitting. To allow such a situation to arise is what I would call "Believing in Superheroes," another managerial anti-pattern. When the team first discovers that there's a player who is pivotal to the success of the team, that player needs to take on an apprentice. (That simple act, making Carl a coach, could be all Carl needs to regain his enthusiasm.)

On Agile teams, we have people working closely together, and sharing ideas and techniques. No one has to be able to do everything. In fact, the team doesn't even have to have two experts for a particular topic. Just one expert, and someone else who knows where to look, what to learn, and how to fix minor problems. In a crisis, the team can pick up the pieces and move on.

Leadership Emerges

Within days, I had another visitor with a similar story: A Business Analyst visiting from the Midwest (we'll call her Betty) was discussing the economy, and she told me that her team had added up the cost of "Wednesday's Bagel Breakfast" and had voted to stop expensing it. The next Wednesday, there were still fresh bagels. Apparently the managers had decided to pay for "Wednesday Bagels" out of their own pockets.

Perhaps management was impressed by the team's self-organizing attempt to reduce waste. And perhaps the managers realized that their success was by virtue of their team's success.

Uh, Just Checking

Rob (paraphrased): "Bagels every week?"
Betty: "Or other pastries."
Rob: "Is the team successful?"
Betty: "Yes!"
Rob: "You know this?"
Betty: "Of course. Why work on an unsuccessful project?"
Rob: "Do you like your job?"
Betty: "Yep!"

I'm not surprised. Pete, are you listening?