Not so very long ago, I made my living writing code. My colleagues and I did our best to understand what our customers needed, and to write code that was easy for other programmers to understand, solid, defect free.  When our managers asked us how long it would take to create a new feature or modify and existing program, we studied the problem, considered how much code we’d need to write or change, how we’d integrate the new functions with existing functions, and what sort of testing would allow us to feel confident our work was complete and correct.

Our managers were often not satisfied with our estimates.  They wished we could deliver working software more quickly.  They suggested we were exaggerating how complicated the work was.  They hinted we were padding our estimates, and spoke sternly of work expanding to fill the available time.  Then they revised the estimates to fit their desires and handed down a due date.

Strangely enough, the work usually took longer than the managers wished it would.  Our managers contemplated the gap between wished-for and actual delivery and pronounced us not sufficiently productive.

And so the quest to improve our productivity began.  First it was CASE tools.  They were supposed to improve productivity by an order of magnitude (they did not).  Then came Rapid Application Development (apparently the managers stopped reading after “Rapid”).  The next improvement, Object Oriented programming, was supposed to save the day (it did change the way we think about programming).  When I left that company, the productivity pill was a development methodology that took up an entire 4-foot bookshelf.

And the quest to improve productivity in software organizations marches on.  More and more often, the quest includes some flavor of Agile Methods, and a promise of hyper-productivity, quick delivery, and change for free.

Agile methods can help achieve business results, get software out the door faster, adjust to new priorities, and dramatically reduce the number of errors that reach the customer.

Agile encourages close collaboration with a product owner or customer (whether that’s an internal customer proxy, product owner or an end-user of the software). This helps teams understand the customer and his context, so they can make better decisions about feature design. Short iterations and frequent demonstrations of working features keeps development close to customer needs and wants.

Agile methods encourage planning based on demonstrated capacity, using agile methods can help prevent cost overruns and help managers make decisions to continue funding feature development–or not. Iteration planning and tracking story points helps teams and management understand capacity. Without this information, plans are merely wishes.  Armed with such information, managers have the opportunity to review progress and viability at the end of every iteration.

Working in short iterations to produce working software can allow a company to realize revenue early; when teams finish small chunks of features each iteration, it’s no longer necessary to wait until all the features are completed to test and deploy. Every feature slice is tested and ready for customer acceptance at the end of the iteration. When the slices add up to a marketable set, managers can choose to release.

Agile engineering practices such as automated unit and customer tests, pair programming, and frequent integration find errors early which reduces the number of defects released to the customer. That results in lower support costs, find and fix costs and bad press generated by buggy products. Practices such as simple design, Test Driven Development and refactoring keep the design flexible and clean, avoiding design degradation and escalating costs for future changes.

Many teams who use agile methods are building strong cross-functional teams and report that their satisfaction with work-life and work/life balance is higher. Working at a sustainable pace and re-establishing pride in work bring intrinsic motivation. That makes it easier to attract and retain employees.

But when agile methods—extreme programming, scrum, kanban or some combination there of—are imposed as a way to “fix” perceived programmer productivity problems, the effort almost always fails.  Sheep-dip training programs, “going agile” by decree and cherry-picking the agile practices that don’t seem difficult won’t achieve great results.

No method, or set of methods can improve results unless managers also change the way they do their work.

Managers need to get organizational systems working.  Managers must do the hard work of understanding the market, their customers, and make difficult choices about what work to do and what work not to do.  Managers need to study how teams and organizations really function and then create systems and environments that don’t hinder people from doing work.

Writing software is hard.  Agile methods can help.  But they are not a silver bullet.  And no method can improve productivity unless management also changes.