Parallel R&D Paths Revisited
Faculty Research Working Paper Series

F.M. Scherer
Harvard Kennedy School

June 2011 RWP11-022
The views expressed in the HKS Faculty Research Working Paper Series are those of the author(s) and do not necessarily reflect those of the John F. Kennedy School of Government or of Harvard University. Faculty Research Working Papers have not undergone formal review and approval. Such papers are included in this series to elicit feedback and to encourage debate on important public policy challenges. Copyright belongs to the author(s). Papers may be downloaded for personal use only.
www.hks.harvard.edu

PARALLEL R&D PATHS REVISITED F. M. Scherer Harvard University Abstract This paper revisits the logic of pursuing parallel R&D paths when there is uncertainty as to which approaches will succeed technically and/or economically. Previous findings by Richard Nelson and the present author are reviewed. A further analysis then seeks to determine how sensitive optimal strategies are to parameter variations and the extent to which parallel and series strategies are integrated. It pays to support more approaches, the deeper the stream of benefits is and the lower is the probability of success with a single approach. Higher profits are obtained with combinations of parallel and series strategies, but the differences are small when the number of series trial periods is extended from two to larger numbers. A "dartboard experiment" shows that when uncertainty pertains mainly to outcome values and the distribution of values is skew-distributed, the optimal number of trials is inversely related to the cost per trial. John F. Kennedy School of Government Harvard University Cambridge, MA 02138 mike_scherer@harvard.edu

1

PARALLEL R&D PATHS REVISITED F. M. Scherer June 2011 Revision 1. Introduction

This paper revisits the role pursuing parallel paths -i.e., supporting simultaneously a diversity of experiments or product designs to hedge against uncertainties in securing a desired technological result -- plays in research and development strategy. Some of the more novel points advanced here come from my early publications and from unpublished lectures in a course on the economics of technological innovation and economic growth, taught repeatedly between 1982 and 2009. Some are newer, worked out to fill lacunae I had left in past treatments. My introduction to the parallel paths strategy came in joint research with M. J. Peck on advanced weapons research and development. In our (1962) book, Peck and I proposed, expanding upon a suggestion we heard first at Bell Telephone Laboratories, that the scheduling an R&D project entailed a tradeoff between speed of development and cost. Figure 1 reproduces our diagram introducing the basic concept. One could accelerate the expected completion date of a development project, but (within efficient tradeoff curve segment NR) only by incurring higher cost. Time could be saved by assigning more talent to the effort, but only subject to diminishing marginal returns; by overlapping tests before the first step has yielded all the information useful in a later step; and through hedging against uncertainty by supporting parallel research or development approaches. The essence of the parallel paths strategy was (Peck and Scherer, 1962), p. 261: ... operating simultaneously two or more approaches to the step, test, or problem to insure that at least one approach will hit the mark at the earliest possible moment. We proposed a crude calculus-based solution to the time-cost tradeoff problem, finding an optimum where the (negative) slope of the convex time-cost tradeoff function was equal to the time derivative of a function measuring the expected military value

2

from successful completion of the research and development project. The more valuable the completed weapon system, the more rapidly it should be developed, among other things through the use of parallel paths. There is no systematic evidence on how widely parallel paths strategies are used, but there is reason to believe the best-known cases are not atypical. Thomas Edison is said to have tested 1,600 materials for his electric lamp filaments before focusing on a carbonized design. In 1934 DuPont synthesized 81 different polyamide compounds in its quest for what eventually became nylon. Five were carried into further experiments. 1 A pioneer in science-guided rational drug design explored 367 different molecules before finding one with good prospects for suppressing the human body's rejection of artificial organ transplants. 2 Peck and I observed that the military authorities authorized parallel paths, sometimes in head-to-head competitions and sometimes more informally, in their quest to develop new fighters, bombers, and guided missiles. More recently, several design-stage alternatives were supported, and two full-scale prototypes were built for competitive evaluation during the 1990s, as precursors to what eventually became the F-22 "Raptor" advanced tactical fighter and the F-35 joint strike fighter. In perhaps the most famous case of all, U.S. defense authorities initially supported five different approaches to the problem of producing fissionable material for an atomic bomb, each expected in May 1942 to cost approximately $100 million, and four were sustained into production during the atomic bomb development effort. 3 And two different bomb designs --a gun-barrel design using uranium and an implosion device using plutonium -- were supported to the 1 2 . Hounshell and Smith (1986), p. 259. . Werth (1994), p. 251.

3 . See e.g. Hewlett and Anderson (1962, vol. I); and Rhodes (1986). Hitch and McKean (1960, p. 249) write that "... the method that succeeded in producing the material for the first bomb was regarded at first as among the least promising..." They do not identify the successful method or pinpoint the timing. In fact, the five methods' perceived prospects changed ranks several times between 1940 and 1943. In December 1941, the best alternatives were considered to be gaseous diffusion, which was combined with electromagnetic separation to produce material for the Hiroshima bomb, and centrifugal separation, which proved intractable and whose production plant authorization was cancelled in November 1942. It later became low-income nations' preferred method. 3

end of the War and afterward, the former exploded over Hiroshima and the latter at Nagasaki. After completing my work with Peck, I sought to extend insights from the time-cost tradeoff concept, initially making my mathematical treatment more general and more rigorous in the context of government-supported R&D (Scherer, 1965) and then building models showing how and when competition accelerated the pace of innovation in the civilian sector (Scherer, 1967). The time-cost tradeoff approach was not without critics. At an informal meeting in the early 1960s, General Bernard Schriever, head of the U.S. Air Force's ballistic missile development program, insisted that there was no tradeoff: the quickest approach was also the least expensive. His implicit emphasis was on what we called the overhead effect, characterized by segment MN in Figure 1. There were also rumblings of skepticism from another important West Coast institution, the RAND Corporation. After my early work on timecost tradeoffs was completed, economist Thomas Marschak published (1967) a rich set of what might be called impossibility theorems suggesting that parallel paths strategies could lead to an inverse convex relationship between development time and cost, but that important exceptions could also exist. 4 2. Richard Nelson's Contribution

A more focused and widely-disseminated contribution came from Richard R. Nelson (1961), who had been a colleague of Marschak at RAND. Nelson emphasized the beneficial role of parallel approaches in the face of R&D uncertainties, concluding (p. 363) that "we should be wary in damning the wastefulness of independent and competitive efforts" and more generally that "the number of alternative inventors ... should be greater, the greater the demand for the invention." Nelson motivates his more general model with a simple numerical illustration. The objective is to develop a successful new fighter aircraft. At the outset, it is uncertain which of various alternative designs is likely to be successful. A prediction of future success is obtained by building and testflying one or more prototypes embodying proposed designs. Each 4 . For later, more specific, modelling approaches, see Abernathy and Rosenbloom (1968) and (1969). 4

prototype effort costs $10 million (an astounding but plausible number, compared to present-day fighter aircraft development program costs in the billions of dollars!) and takes 20 months. Prototype tests can reveal any given design to be a more desirable Type I, with an expected further development cost to successful completion of $50 million over an additional time span of 20 months, or a less desirable Type II, for which the additional development effort is expected to cost $100 million and take 50 additional months. The a priori probability of a Type I outcome is 0.40 and for Type II 0.60. If only one design proves after prototype tests to be of Type I, it is carried into final development. If parallel prototype paths are pursued and more than one design proves to be a Type I, one of the successes is chosen randomly for further development. If none is a Type I, one of the Type II prototypes is selected randomly for highcost final development. Weighting outcomes by probabilities, Nelson computes the expected values of total development cost and time for alternative strategies -- supporting only a single first-stage prototype, chosen at random, or pursuing from two to five parallel prototype development paths. The outcomes, given his assumptions, are shown in Figure 2. Authorizing a single prototype leads to an expected total cost of $90 million and a probability-weighted time to completion of 58 months; with two approaches in parallel, development is not only faster -- 50.8 months -- but less expensive ($88 million). Choosing between one and two paths, there is no tradeoff: the two-path strategy is dominant. For the specific values chosen, a partial counterexample to my assumption of tradeoff curve convexity is demonstrated. Nelson's initially articulated criterion is "achieving a given objective at minimum cost," so it would appear that the two-path strategy is optimal. However, he recognizes that delay can also be costly. For strategies involving more than two parallel paths, time is reduced, but only at higher expected cost. A tradeoff materializes. Assuming (p. 360) that a month's delay in effect costs $1 million (i.e., that the Air Force is willing to spend an extra $1 million for each month saved), he finds the least-cost strategy to be pursuing three parallel paths. Nelson's pioneering analysis makes a compelling case for the potential attractiveness of parallel R&D paths strategies. There are, however, two noteworthy problems.

5

First, his "cost of delay" or (in footnote 14) indifference curve relating delay cost to time 5 overlooks a rather general point. R&D is normally an investment made for the purpose of securing future benefits, which, as in most investment problems, accrue over a period of time. When the project is completed, one taps into a stream of benefits. What one loses by taking longer to complete one's development is the benefit from that stream during the period of delay. As Peck and I formulated crudely (smoothing unique and short-duration but uncertain combat needs with the probability of a combat situation) and I proposed more rigorously in 1965, the problem of optimal development timing consists of maximizing the difference between discounted benefits and R&D costs, i.e., (1) Max H ∫ v(t) e-rt dt - C(T); T

where v(t) is the depth of the benefits stream at time t, r is a conventional time discount rate (or in business problems, the so-called "hurdle rate"), C(T) is a convex inverse function relating development cost to the expected time for development completion, T is the time when the development is completed and benefits begin flowing in, and H is the decision-maker's time horizon. To be sure, erratic benefit stream configurations might require the single-valued time cost assumption of Nelson and Marschak, but equation (1) is more general and more consistent with the accepted literature on capital investment. Second, Nelson's analysis compares only options entailing multiple but simultaneous prototype paths against the one-path alternative. He ignores alternative series scheduling strategies. Given his assumptions, a series strategy would build and test one prototype, determine at the end of 20 months whether it is a Type I or Type II, commence full-scale development if it is a Type I, and (here is the difference) shut down the first project and begin a second prototype project if the first prototype is found to be a Type II. The same decision rule could be followed when test results are obtained from the second prototype, and so on for as many iterations in series as one wishes to entertain. Where 0.6 is the probability of a Type II outcome for a single prototype and 0.4 the probability of a 5 . See also Marschak (1967), pp. 207-210, who views delay as a cost and postulates indifference curves to resolve tradeoffs. 6

Type I outcome, the expected cost of completion for a threestage series strategy (with full-scale development proceeding regardless of the third-stage results, if it is reached) is: (2) E(T) = $10 million + (0.4)x $50 million [1st stage success] + (0.6 x $10 million) [2nd prototype after failure] + (0.6 x 0.4) x $50 million [2nd stage success) + (0.62 x $10 million) [3rd proto. after 2nd failure] + (0.62 x 0.4) x $50 million [3rd stage success] + (0.63 x $100) million [3rd stage failure] = $80.4 million.

By a similar probability-weighted calculation, one finds that the expected time to completion with a three-stage strategy is 65.7 months. Although the expected time to completion is longer, the expected cost with the three-stage series strategy is lower than with any of the parallel paths strategies. The tradeoff is restored. Figure 3 adds to Figure 2 three series time-cost tradeoff outcomes, for two-stage, four-stage, and six-stage strategies. Again, one sees that the tradeoff is restored, the only anomaly being the higher cost with a one-stage strategy than with two parallel paths. Whether one would choose to use a series strategy instead of a parallel paths strategy depends upon the depth of the benefits stream tapped when the R&D project is successful. The deeper the benefits stream, the more the optimum moves to the northwest into multiple parallel paths. For shallow expected benefits streams, series scheduling could be optimal. Small-scale research in a university setting characteristically emphasizes the series approach. However, for important problems -- those whose solution will tap deep benefits streams -- multiple investigators are likely to be working in parallel, quite possibly competitively. Thus, realworld behavior --to be sure, not necessarily optimal -- may combine series and parallel strategies. See Scherer (2010), pp. 568-569. 3. An Intermediate Step

To set the stage for my work on how rivalry in the private sector affects the speed of innovation, I considered it essential first to reaffirm that there was indeed a tradeoff between development time and cost, especially in the context of

7

parallel path vs. series strategies under uncertainty. Several approaches to the problem were analyzed and published in my (1966) paper. 6 That paper verified a robust time-cost tradeoff under uncertainty. Having resolved that question to my satisfaction, essentially at the sub-optimizing research strategy level, I was able to simplify my analysis of optimal R&D strategies under rivalry by assuming the time-cost tradeoff function to be deterministic. The paper established several propositions, summarized as follows: (1) When each alternative research project has the same probability of success and the same cost, and when, for a series strategy, equal numbers of projects are scheduled per time period, a convex time-cost tradeoff function exists. (2) Under the assumptions of (1), the equal projects per time period strategy was not optimal, although the time-cost tradeoff continued to exist. Rather, costs were reduced, all else equal, by scheduling relatively few projects in the first period and then increasing the number of projects progressively in later periods. In a dynamic programming example with an equal individual project success probability of 0.05 and a cumulative success probability target of 0.95, the optimal number of projects in six successive periods was 6, 7, 8, 10, 12, and 17. However, costs were not highly sensitive to modest deviations from this optimal pattern. (3) When individual project success probabilities and/or costs differ, one schedules first in a parallel-series strategy the projects with the highest success probabilities. The negative time-cost tradeoff persists. Letting project costs differ too complicated the analysis beyond the bounds of known computational feasibility. (4) When completing one or more projects generates information that increases the success prospects of subsequent projects, the case for series scheduling is strengthened without eliminating the existence of a time-cost tradeoff. (5) Simultaneous cross-project learning (cross fertilization) or other scope economies strengthens the case for parallel paths and might reverse the time-cost tradeoff for 6 . Material inadvertently omitted from the original version was published in the September 1966 edition of the same journal. 8

excessively serial strategies. (6) All of the above analyses assumed that successful trials are near-perfect substitutes of approximately equal utility, so in a series strategy, testing ceases when one success has been achieved. When outcomes are differentiated so that additional outcomes have value, the case for series scheduling is weakened but not eliminated and the case for parallel scheduling is strengthened. We return to this piece of unfinished business in a later section. 4. Finding the Global Optimum All of these analyses were focused narrowly on testing, and in most cases supporting, the existence of a negatively sloped time-cost tradeoff function under significant uncertainty. Once that function is established, the problem remains of finding the time-cost combination that maximizes the expected surplus of benefits over R&D costs. A simple but fairly general analysis affirmed that the deeper the stream of benefits tapped following successful R&D project completion, the more one optimally emphasized saving time over saving cost. To achieve insight into how sensitive net profits were to the pursuit of parallel and series strategies, a quantitative analysis of diverse success probability, benefit stream depths, and scheduling strategies was conducted. Where t was a running time variable, bt was the dollar value of the benefits realizable in the tth time period contingent upon success, M was the cost per research approach, q was the probability that any given approach would fail (like M, assumed constant), N was the number of approaches originally scheduled (subject to reduction if an early success emerged), and r was the discount rate, the objective was to maximize net expected present value V, defined as: (3) V = T ∑ [1 - qN(t-1)/T] [1 / (1+r)] bt t=2 H + ∑ [1 - qN] [1 / (1+r)t] bt t=T+1 T - ∑ [qN(t-1)/T] [1 / (1+r)t-1] (NM/T) t=1

9

with respect to N and T. The first term is the benefits received contingent upon early success in the planned experimentation period, the second the benefits after completed experimentation has (with some probability) yielded success, and the third the discounted present value of R&D costs. Figure 4 reproduces the main results of the numerical search for optimal solutions, where each experiment is conducted within a single year, the cost per experiment is $1,000, potential benefits are measured in thousands of dollars per year, they continue out to year 25, and the time discount rate r is 0.06. 7 Verifying prior insights, one found that the optimal number of independent research projects increases monotonically with the depth of the benefits stream bt. The article reported that V was relatively insensitive to the diverse combinations of N (the total number of planned experiments, if early success were not achieved) and T (the number of periods over which experimentation might continue). In other words, getting the total number of scheduled experiments right was much more important than the way they were scheduled over time. But getting N/T right was important, as Figure 4 shows. Finally, changes in the net benefit-maximizing N/T were more sensitive to the depth of the benefit stream, the lower the probability of success in a single experiment was -- i.e., the greater the uncertainty. As I reconsidered the relevance of these results to an analysis of uncertainty-hedging in pharmaceutical R&D, 8 I realized that one needed to know more about the relationship between N and T, that is, on the extent of reliance upon series as compared to parallel scheduling of individual projects. I have extended the analysis for a plausible array of scheduling assumptions. The computations were done mainly for the case in which scheduling was expected to be most sensitive to differences in benefit stream depths, i.e., with a low 0.01 probability of success. This is akin to conditions in preclinical animal model tests to discover therapeutically interesting pharmaceutical molecules before testing in humans begins. Profit-maximizing solutions were found by inspection 7 . Benefits were assumed to begin flowing in at the earliest in year 2 and to be discounted at the end of the year in which they were realized. 8 . Scherer (2010). 10

from profit vectors for alternative numbers of trials per period, given differing benefit stream depths (a relatively easy task in the era of spreadsheets). For the series strategies, the number of trials scheduled per period was equal, contrary to insight (2) from my 1966 tradeoff paper. Figure 5 shows that the discounted profit-maximizing number of trials scheduled per period (N/T) differs widely, depending upon whether all tests are scheduled for the first period as compared to being spread conditionally over two, three, or four successive periods. When everything is done in the first period, by far the largest number of trials in a period is scheduled. The more periods over which the trials are spread, the smaller is the profit-maximizing number of trials per period -- in the hope that an early success will alleviate the need for later trials. 9 The optimal total number of projected trials increases two-to-threefold, however, as one moves from a onestage to a four-stage strategy. If one is unlucky and no success is achieved in early stages, the larger number of trials will cost more than does the single-stage strategy. But that multi-stage cost is reduced by the probability that success will be achieved earlier and the later trials will be unnecessary. Figure 6 tests for the sensitivity of total discounted profits, i.e., expected benefits minus expected R&D costs, to alternative series scheduling strategies. Spreading trials, whose number is optimized for the series strategy chosen, over two periods is substantially more profitable in the net than running all trials simultaneously in the first period. This is so even in the absence of learning from unsuccessful tests, as assumed throughout this computation. However, profits are not very sensitive to moving from two to three or four trial periods. Thus, a bit of series scheduling appears to be a good thing, but diminishing returns set in rapidly. To be sure, a disadvantage of spreading any given number of trials (actually variable among series alternatives in the Figure 6 computation) over more periods is a longer expected period of development. For bt = 25, the expected development time, given the profit-maximizing choice of trial numbers, varies with the number of periods over which the trials are 9 . The curves are discontinuous below bt of 10, since no parallel or series strategies yielded positive net profits for benefits of $7,500 per year -- the next lowest value for which optima were calculated -- or less. 11

spread (i.e., with increasing recourse to series scheduling) as follows: E(T) All in first period Two periods Three periods Four periods 1.00 1.42 1.70 1.92 years years years years

Since series scheduling tends to be less costly, all else equal, this implies again the existence of a time-cost tradeoff. In the computation conducted, this tradeoff is taken into account explicitly by choosing trial numbers that maximize net profits, i.e., discounted benefits less discounted R&D costs, compensating for waiting longer on average with more protracted series strategies to tap the benefits stream. 10 Because calculating the optimal number of trials is fraught with estimation uncertainties in real-world practice, Figure 7 tests for sensitivity to a crude second-best strategy: conducting under any of four different series assumptions the number of trials per period optimal for a two-period strategy. 11 Again, the profit difference between a one-period and two-period strategy is substantial. But for a larger number of periods, the profit sacrifice from using this second-best strategy is even smaller than under the assumptions of Figure 6. Clearly, impressive profits are realized in most of the cases analyzed, even though there is substantial duplication of R&D costs. They rise nonlinearly, needless to say, with the depth of the annual benefits stream. For perspective, the discounted present value (at 6 percent simple end-of-year interest) of benefits starting in year 3 and ending in year 25 is 273.75 (thousands of dollars) with annual benefits of 25 10 . There is also a third variable. Since costs are lower with series strategies, all else equal, more trials are conducted when early trials yield no successes, and as a result, at the end of the sequence, the cumulative probability of success is higher. For bt = 25, the total number of trials in the worst case is 108 for a fully parallel strategy (T = 1), 174 for two stages, 222 for three stages, and 264 for four stages. The cumulative success probabilities are correspondingly 0.662, 0.826, 0.893, and 0.930. A higher cumulative success probability, like lower trial costs, enhances net profits, which is also taken into account in the computations. 11 . For the one, three, and four-stage strategies, these are not the profit-maximizing strategies. 12

(thousands) per year and 547 with benefits of 50 per year. Thus, for the least profitable single-stage strategy summarized by Figure 6, net profits (after the deduction of R&D costs) are 32 percent of maximum attainable benefits in the bt = 25 case and 58 percent in the bt = 50 case. 12 Figure 8 broadens the perspective to show net profits from fully parallel (single-year) R&D strategies for a broader array of success probabilities. Quite plausibly, one observes much higher net profits from parallel paths strategies with singletrial success probabilities greater than our initially assumed 0.01, with commensurately smaller optimal trial numbers. The profit increases as success probabilities are raised from 0.05 to 0.20 are considerably smaller than those for increases from 0.01 to 0.05. At bt = 25, net profits with optimal fully parallel paths and a single-trial success probability of 0.20 are 93 percent of maximum attainable benefits, calculated as in the previous paragraph. One sees too that parallel paths strategies yield positive net profits for benefit stream depths considerably lower than 10 (thousand) per year, which was the approximate breakeven threshold with a single-trial success probability of only 0.01. 5. Multiple and Diverse Payoff Cases

The analyses presented thus far have assumed consistently that there is uncertainty as to which research project or task will yield a good solution, but once a single solution is found, it suffices and the investigation can end. In effect, the main uncertainty is scientific or technological. But as recognized in generalization (6) above, a parallel paths investigation may yield more than one good solution. Consumer tastes differ, and a particular solution may satisfy one set of consumer wants while others better meet other consumers' wants. Moreover, there is abundant evidence that some research and development results, though technically successful, elicit relatively little consumer demand while others turn out to be "blockbusters." The distribution of profits from technically successful and hence marketed new products and processes has been found consistently to be highly skew; that is, most commercialized inventions have low payoffs, but a few have high payoffs. The top ten percent of innovations, ranked by profitability, account for from 48 to 12 . Net profits as a percent of actual discounted benefits, the latter reduced by less than unit probabilities of ultimate success, are necessarily smaller. See also Scherer (2010). 13

93 percent of total sample profits. 13 Further investigation revealed that the statistical distribution of high-technology payoffs most commonly approximates log normality. 14 That is, where N(0) is a random variable distributed normally with mean of zero and variance of 1, the distribution of profits is approximated by: (1) D(P) = k X
N(0)

,

where P is the value of profits, D() is a distribution function, and k and X are scaling parameters. To illustrate how parallel paths strategies cope with highly skew payoff distributions, we extend here a "dartboard experiment" published in more limited form in 2007. 15 The choice of R&D projects is analogized to throwing darts at a dartboard, the cells of which are the various payoffs contingent upon research and marketing success. In the experiments reported here, each dartboard contains 100 possible payoffs, assumed to be log normally distributed according to equation (1) above, with X= 10 and k = 1000 (e.g., dollars, multiplied by whatever further scaling parameter is suited to market conditions). The number of parallel paths, i.e., "throws" at the dartboard, varied from 5 to 100 per experiment. The payoff matrix coordinate "hit" on any given throw was random, with equal probability for any of the 100 possible coordinates. R&D costs per "throw" were allowed to vary from zero to $12,000. The strategies were purely parallel, that is, no allowance was made for series strategies in which a smaller number of throws was attempted in a first stage, followed by further stages if success goals were not attained. Under conditions of certainty, i.e., perfect aim, the decision-maker would throw a single dart at each payoff matrix cell (i.e., dartboard locus) until every cell with a payoff exceeding R&D costs is struck. Given the log normal distribution assumed, the average number of throws with varying costs per throw (i.e., R&D costs per approach) was as follows:

13 14 15

. Scherer and Harhoff (2000). . Harhoff and Scherer (2003), pp. 279-310. . F. M. Scherer (2007). 14

R&D Cost $12,000 10,000 8,000 6,000 4,000 2,000 0

Number of Throws 15 17 19 22 29 39 100

Since every cell yields a positive payoff, dart-throwing with perfect aim continues when throws (R&D paths) are costless until all 100 cells of the dartboard are covered. To achieve reasonably general results in the face of widely varying (skew-distributed) payoffs, 40 full experiments -- a value determined by computing constraints -- were carried out. For each experiment, a new set of 100 payoffs distributed according to equation (1) was generated, taking care to choose a different normal distribution "seed" for each iteration. As expected, right-hand tail values varied widely across experiments. The largest single extreme payoff value was $1,065,124; the minimax (i.e., the lowest maximum across 40 experiments) was $58,010; the mean among the 40 experiments' maxima was $334,532. Thus, substantial and, not surprisingly, still skewed variability was encountered. 16 At the other extreme, zero payoffs were not feasible, but some of the minima were less than $1.00. Averaging all payoffs across all 40 experiments, the mean single-trial payoff was $7,032. Figure 1 summarizes the results from the 40 experiments, with the number of trials per experiment ranging from 5 to 100. The values graphed are total payoffs for a given number of trials, averaged across all 40 experiments, less total R&D costs, i.e., the assumed cost per trial times the number of trials. One sees that with low R&D costs -- i.e., $4,000 per trial or less -- average net payoffs are maximized by extending the number of trials to at least 100 and presumably (given the value cutoff) more, i.e., attempting (given duplicates, unsuccessfully) to hit every cell on the dartboard. Even in the extreme case of zero R&D cost, many more trials will be 16 . For those who doubt that random sampling from skew distributions can generate such widely varying results, see the whole-pharmaceutical industry simulation in Scherer and Harhoff (2000. 11, pp. 562-564); and Nordhaus (1989). 15

undertaken than in the certainty (perfect aim) scenario. With R&D costs of $6,000 or more per trial, the extreme variability of tail observations leads to somewhat erratic results. Evidently, the trials were on average particularly lucky within the 20-trial set; with both high and low R&D costs, a local maximum is found. With R&D costs of $6,000, there are two local maxima -- one with 20 trials and an average net payoff of $120,650, and a maximum maximorum at 50 trials (many more than in the certainty case) and an average net payoff of $149,829 after deduction of $300,000 total R&D cost per experiment. 17 With still higher R&D costs, the 20-trial strategy dominates, e.g., at R&D costs of $8,000 per trial, with mean net payoffs of $80,650 for 20 trials compared to $62,979 for 40 trials. 18 Given the great variability of payoffs stemming from the log normal distribution, the most one can say with confidence is that when R&D costs per trial are such that the average net payoff across all trials begins to approach break-even, the strategy maximizing the expected value of net payoffs lies somewhere between 15 and 40 trials. At the lower extreme of this range, the parallel paths count differs less than the number of trials with perfect aim. In every experiment, additional "hits" on the same payoff cell were tallied as adding no incremental value, reflecting the real-world case when, say, two virtually identical products are launched into the same product characteristics space niche, with each product sharing in the payoff realizable within that niche. 19 In experiments with 100 trials, the average number of duplicated "hits" was on the order of 36, and even with only five trials, occasional double hits were recorded. That some payoff cells are not exploited explains why the optimal number of trials exceeds 100 with low R&D costs per trial: one keeps trying in the hope of hitting untapped payoffs. A key assumption in all experiments is that the each 17 . In the earlier (Scherer 2007) dartboard experiment with identical distribution parameters, local maxima appeared with 25 trials. Such variability is common with highly skew distributions. 18 . Given the high variability with the highly skew log normal distribution, one cannot rule out a near-zero surplus of payoffs over R&D costs with less lucky trials. 19 . In roughly one case out of 200, zero cell-locater values were also possible. They were treated as if the dart thrower missed the dartboard altogether. 16

trial's "hit" location was statistically independent of other trials. This assumption could be violated in the real world when the targeting of individual trials is positively correlated, e.g., when a single organization launches multiple parallel trials but favors certain broad technical approaches over others. If the number of multiple "hits" is increased for this reason, average payoffs (less R&D costs) will be reduced for a given number of trials. 6. Conclusion

That uncertainty is an important feature of research and development is a truism. The uncertainties are of two main types: technological, i.e., whether a particular approach "works," and demand-driven, i.e., how consumers respond to the technical solutions achieved. For both kinds of uncertainties, parallel paths strategies are a significant coping approach. They may be adopted by a single firm or government agency seeking to meet a market need with new technology, or by the market, i.e., when numerous firms more or less simultaneously pursue their own approaches to meeting a perceived market need. In either case, the analyses above yield some strong clues as to effective strategies. Most importantly, the higher the value of individual successes for a given quantum of uncertainty and cost per trial, the more parallel paths should be pursued. 20 And the greater the uncertainty for a given solution value -- i.e., the lower the probability of single-trial success or the more skew the distribution of market value outcomes -- the more parallel paths one should optimally pursue. The experiments reported in this paper suggest that pure parallel paths strategies are not always optimal in their own right, especially when expected payoffs contingent upon success are modest. Then some combination of parallel and series strategies is likely to be warranted, especially when researchers can learn from their failures and when some approaches are considered more likely ex ante to succeed than others. The quantitative experiments reported in this paper do not yield specific solutions for individual R&D decision-making situations. However, they point to the kinds of strategy options R&D managers should evaluate as they pursue their important work. 20 . When social benefits exceed the private benefits appropriable by innovators, as is commonly the case, a larger number of parallel paths is socially optimal than is profitmaximizing for individual market participants, although competition to be a first mover may drive the two closer. See Scherer (2010). 17

References Abernathy, William J., and Rosenbloom, Richard, 1968. "Parallel and Sequential R&D Strategies." IEEE Transactions on Engineering Management, vol. 15. Abernathy, William J., and Rosenbloom, Richard, 1969. "Parallel Strategies in Development Projects." Management Science, vol. 15, pp. 485-505. Harhoff, Dietmar, and Scherer, F. M., 2003. "Exploring the Tail of Patented Invention Value Distributions," in Ove Granstrand, ed., Economics, Law, and Intellectual Property. Boston: Kluwer, pp. 279-310. Hewlett, R. G., and Anderson, O. E., 1962. The New World. Vol. I. Pennsylvania State University Press, University Park, PA. Hitch, Charles J., and McKean, Roland, 1960. The Economics of Defense in the Nuclear Age. Harvard University Press, Cambridge, MA. Hounshell, David A., and Smith, John Kelly, 1986. and Corporate Strategy. Simon & Schuster, New York. Science

Marschak, Thomas, 1967. "Toward a Normative Theory of Development." In Marschak et al., eds., Strategy for R&D: Studies in the Microeconomics of Development. Springer-Verlag, New York. Nelson, Richard R., 1961. "Uncertainty, Learning, and the Economics of Parallel Research and Development." Review of Economics and Statistics, vol. 43, pp. 351-368. Nordhaus, William, 1989. "Comment," in Brookings Papers on Economic Activity (Microeconomics), pp. 320-325. Peck, Merton J., and Scherer, F. M., 1962. The Weapons Acquisition Process: An Economic Analysis. Harvard Business School Division of Research, Boston, MA. Rhodes, Richard, 1986. Simon & Schuster, New York. The Making of the Atomic Bomb.

18

Scherer, F. M., 1965. "Government Research and Development Programs." In Robert Dorfman, ed., Measuring Benefits of Government Investments, pp. 12-57. Brookings Institution, Washington DC. Scherer, F. M., 1966. "Time-Cost Tradeoffs in Uncertain Empirical Research Projects." Naval Research Logistics Quarterly, vol. 13, pp. 71-82. Scherer, F. M., 1967. Allocation under Rivalry." 81, pp. 359-394. "Research and Development Resource Quarterly Journal of Economics, vol.

Scherer, F. M., 2007. "Schumpeter and the MicroFoundations of Endogenous Growth." In Hanusch, Horst, and Pyka, Andreas, eds. The Elgar Companion to Neo-Schumpeterian Economics, pp. 671-687. Edward Elgar, Cheltenham UK., Scherer, F. M., 2010. "Pharmaceutical Innovation," in Hall, Bronwyn, and Rosenberg, Nathan, eds. Handbook on the Economics of Technical Change. Elsevier, pp. 540-574. Scherer, F. M., and Harhoff, Dietmar, 2000. "Technology Policy for a World of Skew-Distributed Outcomes." Research Policy, vol. 29, pp. 559-566. Werth, Barry, 1994. Schuster, New York. The Billion Dollar Molecule. Simon &

19