Volume 4, Issue 2
Reprinted from Product Development Best Practices Report

EFFECTIVE R&D MEASUREMENT ONE OF THE KEYS TO SUCCESS AT HEWLETT-PACKARD

With $31.5 billion in sales, $2.4 billion in net revenue, more than 100,000 employees, and R&D costs that run almost 10% of net revenue, the folks at Hewlett-Packard need to manage a fascinating business dilemma: on the one hand they need to adhere to a disciplined measurement system, on the other they need to keep alive the tremendously strong intuitive capability that has been a hallmark of the CA-based company since its inception. Since measurement devices are one of their core product areas, along with computing systems and peripherals, and communications devices, it's no surprise that they are keenly aware of this.

It's a stiff challenge: they operate in a highly dynamic business environment, with rapidly evolving markets, fierce competition, relentlessly changing technology, and customer requirements that can change course almost as swiftly as a teenager's plans. On top of this, while primary research is centralized at Hewlett-Packard Laboratories, product development is carried out in more than 100 decentralized product divisions, scattered around the world. While each division is a profit center, they are tasked with effective coordination with each other and with HP Labs. Without effective measurement, in no time at all the company's R&D costs would start to eat their lunch.

Says HP project management consultant Tom Kendrick, "Although we measure a lot of things, we are essentially an intuitively managed company. We don't let the measurements get in the way of what we feel are good business decisions. We use measurement to help the business decision making process, not to perform it."

Adds corporate Quality Systems program manager Ron Benton, "We are intuitive, but we are also rigorously financially managed, We have been growing at roughly 25% the last three years, probably the fastest-growing large company in the world. My guess is that the half life of an HP product division is probably 22 to 24 months. There is so much change and renewal going on that it affects what you do and how you prepare to measure; if your measurement systems are designed around a given organizational construct and that construct goes away, you have to ask yourself about the value of that measurement investment. And measurement is an investment: it doesn't come free."

Kendrick agrees, noting that most investments in R&D measurement tend to yield only intermittent or small improvements, "Investments in R&D measurement are only systematically effective when you implement them in the context of processes and responsibilities that are well-integrated into the measures, where it is supporting business decisions and objectives, where it is supporting desired behaviors, and where the metrics program is aimed primarily at improvement, not punishment."

Characteristics of High Performance R&D Measurement Systems

Based on HP's experience, Benton identifies what he calls "seven sins" of R&D measurement that lead to failure: only one person (typically the lab manager) designs the system, the metrics aren't tied to business objectives or key product development decisions, the system requires too much overhead to work, achieving measurement goals doesn't lead to improved customer satisfaction, the metrics don't motivate, they are used as "report cards", and they don't really explain the reality of what is happening.

Observes Benton, "My definition of failure here is the measurement doesn't survive the tenure of the key sponsor. When somebody says, 'I need to have a measurement system,' they are already on the wrong track. They usually have a management system problem for which measurements are the symptom."

So what are the characteristics of measurement systems that work? They support achievement of business objectives, monitoring performance of a critical few and focusing on the customer at every turn. They promote behaviors that improve performance, aligning with reward and recognition systems, and linking results to improvements for all organizational levels. Going back to Deming, they de-personalize and de-criminalize errors, and reward good reporting of bad results.

They also support effective decision making and organizational learning, integrating with the organization's management systems, presenting balanced views of cross-functional performance, and providing value for all customers of the measurement system. Over time, they eliminate inappropriate or unused measures.

R&D Management Hierarchy

Product development at HP takes place in a three-tiered hierarchy. The top, the business management system level, focuses on environmental scanning, strategic direction setting, organizational alignment, managing stakeholder value and strategic relationships, and overseeing operational management. The middle, the product generation process and R&D strategic management level, stays on top of market/customer/channel research, portfolio and product family management, technology and platform management, competitive analysis, and resource and process management. The bottom, the project and program management systems level, zeroes in on project planning and budgeting, risk management, product definition, life cycle execution, and predictable outcomes.

The following objectives and measures for the top determine the company's overall hoshin or breakthrough goals:

market and customer leadership (market share, order growth rate, loyalty and satisfaction);

financial excellence (return on assets, net profit, inventory-sales ratio);

strategic direction (scenario trigger status and new markets outlook);

best place to work (employee satisfaction and retention, depth of employee strategic understanding and strategic alignment).

Kendrick identifies five desired behaviors that go with the top level objectives and measures: functional areas and teams cooperate to foster a strong corporate identity; business decisions reflect deep customer understanding; investments balance short- and long-term business health; the organization learns constructively from both successes and failures; and innovation is fostered, not inhibited by management.

In Kendrick's words, senior management "fractions off" from its work and cascades this down to the middle, whose objectives and measures provide most of the meat of overall R&D effectiveness. Benton clusters these into three areas:

business results (financial performance - portfolio return, portfolio profitability, and percent of revenue from new products; market performance - order growth rate, projected share; customer results - competitive quality, new customers attraction, share of new market);

product generation renewal (technology distribution - R&D and marketing investment balance; platform renewal - platform effectiveness and efficiency; and technical competencies - core capability skill match, key employee retention, and intellectual property strength);

World Class product generation (internal/external competitiveness - aggregate product generation process cycle time, ramp-to-volume time, new product introduction rate, and new product introduction leadership; product generation process predictability - platform effectiveness and efficiency, aggregate schedule progress; and investment productivity - expense ratios and platform efficiency).

Five desired organizational behaviors go along with these objectives and measures: performance of the cross-functional product generation process is optimized for the business; projects are launched rapidly (and canceled promptly and cleanly, when necessary); overtime and idle time are minimized; contributors reflect high motivation and care toward current projects and each other: and long-term strategic work is balanced with short-term evolutionary projects.

At the project/program management level, says Kendrick, objectives and measures focus on three areas:

clear, aligned deliverables (change control - volume of modifications; product definition - assessment measures);

efficient resource utilization (staffing - overtime, idle time)

project performance (predictability versus plans - resources, schedule, task closure rate; and business and technical risk management).

The measurement system targets six basic behaviors at this level: project and sub-process (marketing, finance, manufacturing, etc.) managers identify, assess, and take appropriate risks; innovation is encouraged within well-understood business and customer requirements; careful planning, execution, and communication are rewarded more than project heroism and fire fighting; employee burn-out is rare; contributors move enthusiastically from completed project to follow-on projects; and work and rewards are spread evenly among project teams.

The bottom line on R&D measurement: stay strategic. Measuring too many things or the wrong things makes little more sense than flying blind. But at HP, with 40% of revenues coming from last year's new products, and 60% to 70% from products introduced in the last two years, good, clear metrics matter. Summarizes Kendrick, "Effective metrics have these three characteristics: they are part of something bigger, support the kind of behavior you're trying to foster, and facilitate good business decision making, helping you get answers to critical questions." PD

Key Learnings:

Measurement is part of a larger objective, a means to an end, not an end in itself. Measurement always affects behavior.
One-size-fits-all measurement systems don't make sense - effective measures should be tailored to the desired results for a given environment.
Measurement supports better decision making - it does not replace good judgment and intuition.PD

Copyright 1997 The Management Roundtable, Inc.