> bjbj jj-,l
~~~"X44" v2222$" '$0~22222
:2K 2~2~22~~2t!4B"a 0 W$vFW$""
~~[Discussion paper for Symposium on Complex Systems Engineering, RAND Corporation, January 11-12 2007. This consists of various sections drawn from my book Extending Ourselves: Computational Science, Empiricism, and Scientific Method (Oxford, 2004). Computer simulations are discussed in section 4.1; the earlier sections are on related issues. A more recent paper `Computational Economics, which discusses computational complexity, agent based models, and the relation between simulations and computational incompressibility, will be available at HYPERLINK "http://www.virginia.edu/~pwh2a/" http://www.virginia.edu/~pwh2a/ after December 10th.]
Paul Humphreys
3.5. `The Same Equations Have the Same Solutions: Re-Organizing the Sciences
`The degree of relationship to real forest fires [of the `forest fire model] remains unknown, but the model has been used with some success to model the spreading of measles in the population of the island of Bornholm and of the Faroe Islands...To be consistent with the literature we will use the terminology of trees and fires. The harm in doing so is no greater than in using sandpile language for...cellular automata.
The drive for computational tractability underlies the enormous importance of a relatively small number of computational templates in the quantitatively oriented sciences. In physics, there is the mathematically convenient fact that three fundamental kinds of partial differential equations elliptic (e.g. Laplace's equation), parabolic (e.g. the diffusion equation), and hyperbolic (e.g. the wave equation) are used to model an enormous variety of physical phenomena.This emphasis on a few central models is not restricted to physics. William Feller, the statistician, makes a similar point about statistical models:
We have here [for the Poisson distribution] a special case of the remarkable fact that there exist a few distributions of great universality which occur in a surprisingly great variety of problems. The three principal distributions, with ramifications throughout probability theory, are the binomial distribution, the normal distribution...and the Poisson distribution...
So, too, in engineering:
It develops that the equations for most field problems of interest to engineers take no more than five or six characteristic forms. It therefore appears logical to classify engineering field problems according to the form of the characteristic equations and to discuss the method of solution of each category as a whole.
Clearly, the practical advantages of this versatility of equation forms is enormous - science would be vastly more difficult if each distinct phenomenon had a different mathematical representation. As Feynman put it in his characteristically pithy fashion `The same equations have the same solutions. As a master computationalist, Feynman knew that command of a repertoire of mathematical skills paid off many times over in areas that were often quite remote from the original applications. From the philosophical perspective, the ability to use and reuse known equation forms across disciplinary boundaries is crucial because the emphasis on sameness of mathematical form has significant consequences for how we conceive of scientific domains.
Where do the appropriate boundaries lie between the different sciences and how should we organize them? In various nineteenth century treatises such as those of Auguste Comte and William Whewell one can find neat diagrams of a scientific hierarchy with physics at the base, chemistry above physics, biology above chemistry, and niches here and there for geology, botany, and other well-established disciplines. Things became a little hazy once the social sciences were reached, but the basic ordering principle was clear: it was the concrete subject matter which determined the boundaries of the specific sciences, and the ordering relation was based on the idea that the fundamental entities of a higher level science are composed of entities from lower level sciences. We do not have to agree either with Whewells and Comtes specific orderings or with the general ordering principle of part/whole inclusion that if the basic elements of one discipline are made up from entities of another, then the latter is more fundamental to the hierarchy than is the former to agree that subject matter usually is what counts when we decide how scientific disciplines should be organized.
This focus on the concrete subject matter as determining the boundaries of a science is straightforward if one is an experimentalist. To manipulate the chemical world, you must deal with chemicals, which are often hard and recalcitrant task-masters, and what can and cannot be done by a given set of techniques is determined by that antecedently fixed subject matter. The same holds in other fields. Try as you might, you cannot affect planetary orbits by musical intervention. Kepler, at least in considering the music of the spheres, just got his subject matters confused.
Things are different in the theoretical realm. Here, reorganization is possible and it happens frequently. The schisms that happen periodically in economics certainly have to do with subject matter, inasmuch as the disputes concern what it is important to study, but they have as much to do with methods and goals. Political economy shears away from econometrics and allies itself with political science because it has goals which are quite peripheral to those working on non-Archimedean measurement structures. Astronomy in Ptolemys time was inseparable from, in fact was a branch of, mathematics. Subject matter and method are not wholly separable elementary particle physics and cosmogony now have overlapping theoretical interests due to the common bond of highly energetic interactions (they are both interested in processes that involve extraordinarily high energies; cosmogony in those that happened to occur naturally near the temporal beginnings of our universe, elementary particle physics more generally) but theoretical alliances tend to be flexible. Departmental boundaries can also be fluid for a variety of bureaucratic reasons. Faith and reason once co-existed in departments of religion and philosophy, some of which remain in an uneasy co-existence. The sub-disciplines of hydrology, ecology, geology, meteorology and climatology, and some other areas have been merged into departments of environmental science, with somewhat better reason. One can thus find a number of principles to classify the sciences, some of which are congenial to realists, others to anti-realists.
A quite different kind of organization can be based on computational templates. Percolation theory (of which Ising models are a particular example see section 5.3) can be applied to phenomena as varied as the spread of fungal infections in orchards, the spread of forest fires, the synchronization of firefly flashing, and ferromagnetism. Agent based models are being applied to systems as varied as financial markets and biological systems developing under evolutionary pressures. Very general models using directed acyclic graphs, g-computational methods, or structural equation models are applied to economic, epidemiological, sociological, and educational phenomena. All of these models transcend the traditional boundaries between the sciences, often quite radically. The contemporary set of methods that goes under the name of `complexity theory is predicated on the methodological view that a common set of computationally based models is applicable to complex systems in a largely subject-independent manner.
Some phenomena that are covered by the same equations have something in common. For example, the Rayleigh-Taylor instability is found in astrophysics, magnetohydrodynamics, boiling, and detonation. In contrast, there are phenomena that seem to have nothing in common 'physically'. The motion of a flexible string embedded in a thin sheet of rubber is described by the Klein-Gordon equation:
(2 - (1/c2)(2/(t2 = k
but so too is the wave equation for a relativistic spinless particle in a null electromagnetic field. The statistical models mentioned by Feller cover an astonishing variety of scientific subject matter which in terms of content have nothing in common save for a common abstract structure.
This all suggests that, rather than looking to the material subject matter to classify the sciences or to bureaucratic convenience, parts of theoretical science can be re-classified on the basis of which computational templates they use. This kind of reorganization has already happened to a certain extent in some computationally intensive endeavours, including the areas already mentioned, and lies behind much of the movement known generally as complexity theory. Does this mean that the subject matter then becomes irrelevant to the employment of the common templates? Not at all. The primary motivation for using many of these models across disciplinary boundaries is their established computational track record. But as we shall see, the construction and adjustment processes underlying the templates requires that attention be paid to the substantive, subject-dependent, assumptions that led to their adoption. Without that local, subject-dependent process, the use of these templates is largely unmotivated and it is this tension between the cross-disciplinary use of the formal templates and their implicit subject-dependence that lends to them interest as units of analysis.
The idea of an abstract argument form has long been used in logic, although there the orientation has been towards the general form and not the specific instances. A generalization of that idea is Philip Kitchers concept of a general argument pattern. A general argument pattern, as its name suggests, consists of a skeleton argument form together with sets of filling instructions sets of directions for filling in the abstract argument form with specific content. An example is the use in post-Daltonian chemistry of atomic weights in explaining why specific compounds of two substances have constant weight ratios of those substances. Computational templates differ from general argument patterns in at least two ways. The issue of mathematical tractability is not a concern for the latter, whereas it is the major focus for the former. Secondly, general argument patterns provide the basis for Kitchers account of explanatory unification in an enlightening way. Yet the orientation, judging from the examples, seems to be towards unification within the existing disciplinary organization of science, in contrast to the position advocated here, which is that computational templates provide a clear kind of unity across disciplines. Although unification accounts of explanation are insufficiently realist for my tastes, it would be interesting for advocates of that approach to explore the implications for explanatory unification of the approach suggested here, not the least because it has profound implications for the issue of theoretical reduction. These computational commonalities upset the traditional hierarchy Gibbs models from physics are used in economics, economic models are used in biology, biological models are used in linguistics, and so on. `Theory longer parallels subject matter, although it has not been cast entirely adrift.
3.10. Templates Are Not Always Built On Laws or Theories
The templates we have considered so far have been constructed on the basis of assumptions, some of which are justified by an explicit appeal to scientific laws. In the first and third derivations, each contained a law, the conservation of heat and the conservation of fluid, respectively. Other templates make no use of laws. Consider an utterly simple construction, for which the resulting computational template is that of a Poisson process. Suppose we are willing to make these assumptions about the stochastic process:
a) During a small interval of time, the probability of observing one event in that interval is proportional to the length of that interval,
b) the probability of two or more events in a small interval is small and goes to zero rapidly as the length of the interval goes to zero,
c) the number of events in a given time interval is independent of the number of arrivals in any disjoint interval,
d) the number of events in a given interval depends only upon the length of the interval and not its location.
These are informal assumptions, but ones that can be given precise mathematical representations. From those representations it is possible to derive the exact form of the probability distribution covering the output from the process: P(N=k) = e- k/k!. This template is of great generality, and it has been used to represent phenomena as varied as the number of organisms per unit volume in a dilute solution, the number of telephone calls passing through a switchboard, the number of cars arriving per unit time at a remote border post, the number of radioactive disintegrations per unit time, chromosome interchanges in cells, flying bomb hits on London during World War II, the number of fish caught per hour on a given river, the rate of light bulb failures, and many others. Some of these applications, such as the flying bomb hits, provide reasonably accurate predictions of the observed data, whereas others give at best approximations to the empirical results. Leaving aside the issue of accuracy, one conclusion to be drawn from this example is that templates can be constructed in the absence of anything that looks like a systematic theory about a specific domain of objects. It is implausible to claim that in the case of the flying bomb application there exists a systematic theory of aerodynamics that justifies the use of each assumption and in this and in other cases, the Poisson assumptions can be justified purely on the basis of empirical counts. Theories need play no role in constructing templates.
And so also in the case of laws. In many cases of template construction, laws will enter into the construction process, but in the Poisson example just described it is not plausible in the least that any of the assumptions given count as laws. Although one might say that the individual outcomes are explained by virtue of falling under a law-like distribution, if the fact to be explained is that the system itself follows a Poisson distribution, that fact is not understood in virtue of its being the conclusion in a derivation within which at least one law plays an essential role. Furthermore, even if we consider the distribution that covers the process to itself be a law, that law is not itself understood in terms of further laws, but in terms of structural facts about the system. This clearly indicates that scientific understanding can take place in the absence of laws.
The point is not that there are no scientific laws -- there are. There are fundamental laws such as the conservation of charge in elementary particle reactions and decays, the conservation of isospin in strong interactions, and the conservation of charge conjugation parity in strong and electromagnetic interactions, to name only three. These laws are, to the best of our current knowledge, true and hence exceptionless. There are others, such as the conservation of baryon number, which hold universally except for situations that can be explicitly specified in this case proton decay and these exceptions have not to date been observed. Combine this with our point that accurate and widely used templates can be constructed without the need for laws and the combination of the two lead us to the conclusion that there are laws but that they need not be the starting point for applying science to the world.
We can now address a central issue for templates. How is it that exactly the same template can be successfully used on completely different subject matters? This is a different question than the one that famously asks 'What explains the unreasonable effectiveness of mathematics in representing the world?'. That older question cannot simply be about why mathematics applies to the world, for there would be a straightforward answer to it, which is for the most part, it does not. Mathematical models in science generally do not apply directly to the world, but only to a refined version of it. Because the space of consistent mathematical structures is much larger than the space of idealized physical structures, it is not at all surprising that one can find, or construct, objects in the former realm that apply to the latter. Accept the idealizations, approximations, and abstractions involved in the physical models and it is inevitable that some mathematical model will work.
A slightly more interesting interpretation of what is meant by the question suggests that what is surprising is that parts of the world are such that relatively simple mathematical models fit them closely, even without much idealization. An answer to that version of the question, perhaps not much more interesting than the first answer, is that a selection bias has been in operation and that the era of simple mathematics effectively modelling parts of the world is drawing to a close. It is possible that new areas of investigation will lend themselves to simple models, but the evidence is that within existing areas of investigation, the domain of simple models has been extensively mined to the point where the rewards are slim.
There is a third version of the older question: why do parts of mathematics which were invented for the purposes of pure mathematics model the natural world so well? One answer to this question is simply a variant of our answer to the first question: much mathematics has been constructed and it is perhaps not surprising that we can find some parts of it that happen to apply to selected features of the world. A related and more accurate answer relies on a variant of our second slogan from section 3.2 we simply force-fit our models to the available mathematics because which forms of mathematics are available constitutes one of the constraints within which we must work.
My question is different from these older questions. Given that mathematics does apply to the world, why do the same mathematical models apply to parts of the world that seem in many respects to be completely different from one another? On a hypothetico-deductive approach to templates, this fact appears quite mysterious. In contrast, on the constructivist account of templates we have suggested there is a ready explanation. Templates are usually not constructed on the basis of specific theories, but by using components which are highly general, such as conservation laws and mathematical approximations, and it would be a mistake to identify many of these components with a specific theory or subject matter. The Poisson derivation is an extreme example of this, for it uses only purely structural features of the systems, and those features can often be justified on the basis of simple empirical data without any need for theory for example the fact that the probability of two cars arriving at a border post within a short time interval is small. That is why such templates are applicable across so many different domains. The role of subject-specific theory and laws is non-existent in these extreme cases, except when such things are used to justify the adoption of one or more of the assumptions used in the construction. Most template constructions are more specific than this, but in the case of the diffusion equation, the independence from theory is most clearly seen in the third derivation, where nothing that is theory-specific is used. Even in the more specific constructions, it is the fact that mere fragments of a theory are used that makes for the great generality in the following way. According to our account in section 2.5, objects and their types are identified with property clusters, with the clusters usually being of modest size. When, for example, a conservation law is invoked, the specific nature of the conserved quantity is often irrelevant and certainly the full set of properties constituting the domain of the theory at hand is never used. All that matters is the mathematical form of the function and that there is no net loss or gain of the relevant quantity. It is this welcome flexibility of application for the templates that allows us to take the templates as our focus rather than specific theories when investigating computational science.
3.11. The Role of Subject-Specific Knowledge in the Construction and Evaluation of Templates
`In individual cases it is necessary to choose an appropriate sigma-algebra and construct a probability measure on it. The procedure varies from case to case and it is impossible to describe a general method.
There is a curious tension involved in the use of computational templates. As we have seen, computational templates straddle many different areas of science in the sense that many of them can be successfully applied to a wide variety of disparate subject matters. Take the basic idea of fitness landscapes, a technique pioneered by Sewall Wright, J.B.S. Haldane, and R.A. Fisher. The underlying techniques can be applied to biological evolution, the evolution of economies, and the survival strategies of corporations. The mathematical techniques developed for modelling the simulated annealing of metals were adapted to provide the basis for the revival of neural network research. Spin glass models, originally devised in physics to model solid state phenomena, have been adopted by complexity theorists for a wide range of phenomena, including modelling political coalitions. Cellular automata, invented by von Neumann as a device to represent self-reproducing automata, have been adapted, elaborated, and applied to agent-based models in sociology, economics, and anthropology.
On the other hand, a considerable amount of subject-specific knowledge is needed in order to effectively fit the templates to actual cases. The choice of which force function or probability distribution should be used to bring a theoretical template down to the level of a computational template requires knowledge of the subject matter the `theory does not contain that information and more than inferential skills are needed in the construction of the computational template.
In general, matter constrains method. By this I mean that the intrinsic nature of a given phenomenon renders certain methods and forms of inquiry impotent for that subject matter, whereas other subject matterss will yield to those means. Heating plays a key role in polymerase chain reaction processes, but virtually none in studying radioactive decay; randomized controlled trials are the gold standard in epidemiological investigations, but are unnecessary for electromagnetic experiments; careful attention to statistical analysis is required in mouse models of carcinogenesis, but such analysis is often casual window-dressing in physics. Subject matter does not determine method too many other factors enter for that to be true but the philosophical dream for much of this century has been that logical, or at least formal, methods could, in a subject matter independent way, illuminate some, if not all, of scientific method. Although these subject-independent approaches methods achieved impressive results and research continues in that tradition, there are quite serious limitations on the extent to which those accounts are capable of success.
Consider a specific example which I shall elaborate in some detail to bring out the various ways in which knowledge of a variety of subject matters is needed in the construction and correction of the computational template. It is an important issue for the United States Environmental Protection Agency whether increased particulate matter concentrations in the atmosphere cause an increase in human mortality and morbidity. Particulate matter comes from a variety of sources such as diesel exhaust, industrial smoke, dust from construction sites, burning waste, forest fires, and so forth. Particulate matter (PM) comes in various sizes, and it is conjectured that smaller particles (of less than 10m in diameter) are a greater health hazard than are larger particles. The current air quality standard is that the concentration of such particles, denoted by PM10, should not go above 150g/m3 averaged over a 24 hour period. Suppose we want to arrive at a model that will allow us to represent the causal relations between ambient levels of PM10 and morbidity in order to predict the latter. There are some obvious subject-specific choices that have to be made before constructing the model: How to measure morbidity? Some effects of PM10 exposure are acute cardiopulmonary fatalities, emergency room asthmatic admissions, and reduced respiratory function. What is the appropriate time lag between exposure and effect? The usual choice is 0-4 days. Which potential confounders should be controlled for? Temperature affects the onset of asthmatic episodes, as does the season, flu epidemics lead to increased hospital admissions for pulmonary distress, and the co-pollutants sulphur dioxide, carbon monoxide, and ozone, which are by products of PM emissions, can also affect respiratory health.
There are less obvious items of knowledge that must be used, some of which are `common sense and others of which are more subject specific, but ignorance of which can easily ruin our evaluation of the template. PM levels in cigarette smoke are many times higher than the ambient levels in air and so can swamp the effects of exhaust emissions. This would seem to present a straightforward case for controlling for cigarette smoking in the model except that chronic obstructive pulmonary disease, one of the measures of morbidity, is not exacerbated by smoking in many individuals. Perhaps second-hand smoking could be a problem, for smoking a pack of cigarettes raises the ambient PM levels by 20g/m3. Yet if the individual is in a fully air conditioned building, the ambient levels are lowered by 42g/m3 thus complicating the issue. Air quality monitors are located outdoors, yet on average people spend more than 90% of their time indoors. In the elderly, it is even more. The air quality meters, being scattered, are not representative of local levels in many cases and educated interpolations must be made. The time lag for asthmatic admissions is about 41 hours, close to the upper bound of many of the time series correlations and so a number of PM-related asthmatic admissions will occur too late to be caught by such studies. Finally, ecological inferences made from aggregate data on exposure to individual exposure levels are frequently invalid. It is thus not only impossible (or perhaps irresponsible) to assess a given model of the links between PM levels and morbidity without a considerable amount of subject-specific knowledge being brought to bear on the assessment, it is also quite clear that adjustments to defective models will not be made on grounds of convention but on the basis of informed knowledge of the area, with a strong sense of the rank order of the defects.
There do exist cases in which the evidence in favour of a causal connection between variables is so strong that the models are robust under quite significant epistemic defects, but this is unusual. For example, Doll and Peto's epidemiological work which established a causal link between smoking and lung cancer was accepted despite the absence of any widely accepted mechanism linking smoking and lung cancer because of the highly invariant nature of the link across populations and contexts.
Such examples, which are unusual, do not undermine the general point that a high level of subject-specific knowledge is required to justify causal inferences. The need for such knowledge means neither that general conclusions about science are impossible nor that general philosophy of science is ill-conceived. Philosophy of science does not reduce to case studies because the morals drawn here about computational science transcend the particular cases considered. In a similar way statistics, which also has ambitions to great generality, requires subject-specific knowledge to be applied in a thoughtful way, but it has, nonetheless, been quite successful in providing general methodological maxims.
3.14. Computational Models
We have discussed at some length the concept of a template and features associated with it. We can bring together these features by introducing the concept of a computational model. A computational model has six components:
(1) A computational template, often consisting of a differential equation or a set of such together with the appropriate boundary or initial conditions types. Integral equations, difference equations, simultaneous equations, iterative equations, and other formal apparatus can also be used. These syntactic objects provide the basic computational form of the model.
(2) The Construction Assumptions.
(3) The Correction Set
(4) An interpretation.
(5) The initial justification of the template.
(6) An output representation, which can be a data array, graphical, a general solution in the form of a function, or a number of other forms. The output representation plays a key role in computer simulations, as we shall see in the next chapter.
The sextuple then constitutes a computational model, which can be an autonomous object of study. If we consider only the formal template, there is no essential subject-dependency of the models, but as the assumptions, correction set, interpretation and justification are specified, we recover some of the more traditional, subject-specific, organizational structure of the sciences that is lost with the versatility of the templates. With these models, scientific knowledge is contained in and conveyed through the entire sextuple the formal template is simply one component amongst many. Although each component in the sextuple can be the object of focussed attention, it will be rare that it can be evaluated without considering at least some of the remaining components.
Often, the construction of a template that is not computationally tractable will precede its use in a computational model, sometimes by a considerable period of time. For example, in 1942, Kolmogorov developed a now widely used two equation model of turbulence, but it was not until the 1970s that the models could be applied, when computational methods were finally developed to implement the equations. It is also common for a reciprocal process of adjustment between proto-models and simulation runs to result in the development of a properly articulated computational model. The links that have to be constructed between a computational model and its consequences as represented by the solution space are not always just `more of the same' in the sense that it requires only an ingenious application of pre-existing rules of inference, as the standard picture of axiomatically formulated theories would lead us to believe. The task will ordinarily be composed of a complex and interconnected set of factors including (i) the development of concrete computational devices of sufficient power, (ii) the justification, and occasionally development, of approximation methods, (iii) the construction of algorithms and programs to implement (ii), and (iv) idealizations. We can represent the stages in the construction and refinement of computational models by Figure 3.1. (To preserve clarity, links between an analytic solution and a simulation based upon that solution have been omitted.)
[Insert Fig. 3.1 exactly here]
4.1. A Definition.
`The acceptance by medicinal chemists of molecular modelling was favored by the fact that the structure-activity correlations are represented by 3-D visualizations of molecular structures and not by mathematical equations.
Because our goal is to establish the various ways in which computational science involves new scientific methods, it will be helpful to examine a few representative areas that most researchers would agree fall under the heading. In this chapter I shall focus on the important sub-area of computer simulations. When discussing these activities, we must be aware of a number of things. The entire field of computer simulations, like computational science, is relatively new and is rapidly evolving. Unlike wellentrenched fields, techniques that are now widely used may well be of minor interest twenty years hence as developments in computer architecture, numerical methods, and software routines occur. Because of this, I shall discuss only general issues that apply to a broad variety of approaches and which seem to have more than a temporary status. A second thing to note is that at present some of the methods are highly problemspecific. Methods that work for one kind of simulation are often not applicable to, or work less efficiently for, other simulations and arguments to general conclusions are thus doubly hard to come by.
We begin with a very simple preliminary definition of the general category of computational science :
Computational science consists in the development, exploration and implementation of computational models (in the sense given in section 3.13) of non-mathematical systems using concrete computational devices.
At the level of this preliminary definition, computational science has been around for a very long time indeed. Humans are concrete computational devices, so the calculation of the motion of Venus by ancient Greek astronomers using Ptolemaic theories counts as computational science. So does the process of working out the area of a field using an abacus, and the computing of projectile trajectories with the aid of mechanical hand calculators. In the 1940s, the modelling of nuclear explosions by some of the earliest electronic computers constituted the cutting edge of computational science. Nowadays, there is an entire range of sophisticated computational aids, ranging from student software packages for computational chemistry to the most demanding climate modelling carried out on the latest supercomputer. There is a continuum of extrapolation here, just as there is a continuum in extending our observational powers from the simplest optical microscopes to the currently most powerful magnifying devices.
The simple definition given above does not, therefore, capture what is distinctive about contemporary computational science, even though it does cover an enormous variety of methods falling into the domain of computational science. These include modelling, prediction, design, discovery, and analysis of systems; discrete state and continuous state simulations; deterministic and stochastic simulations; subject dependent methods and general methods; difference equation methods; ordinary and partial differential equation methods; agent-based modelling; analog, hybrid, and digital implementations; Monte Carlo methods; molecular dynamics; Brownian dynamics; semi-empirical methods of computational chemistry, chaos theory, and a whole host of other methods that fall under the loose heading of computational science.
Computer simulations form a special but very important subclass of computational science. They are, of course, widely used to implement mathematical models that are analytically intractable. But they are also used when numerical experiments are more appropriate than empirical experiments for practical or ethical reasons. Thus, some empirical experiments are potentially too costly, such as intervening in the U.S. economy, and are hence replaced by simulations; some are too uncertain in their outcomes at various stages of theoretical research, so that such things as the containment of controlled fusion in a magnetic bottle are better simulated than created; some are too time consuming, such as following the movement of real sand dunes, so that simulations are used to compress the natural time scale; others are politically unacceptable, so that nuclear explosions are simulated rather than investigated by setting off real, dirty, devices; or they are ethically undesirable, so that we protect amphibians by replacing the dissection of real frogs with simulated dissections or the diffusion of drugs in a community is simulated using epidemiological models. Practically impossible experiments include rotating the angle of sight of galaxies, reproducing the formation of thin disks around black holes, investigating oil reservoir flows, earthquake modelling, and many others. In other cases simulation is used as a pre-experimental technique, when trial runs of an experiment are performed numerically in order to optimize the design of the physical experiment.
We can approach the task of defining computer simulations by looking at the difference between a representation and a simulation. Simulations rely on an underlying computational model or are themselves models of a system, and hence either involve or are themselves representations. So what is the relation between a computational model, a simulation, and a representation? In the first article that I wrote on simulations, I left open what counted as a simulation and used what I called a `working definition: `A computer simulation is any computer-implemented method for exploring the properties of mathematical models where analytic methods are unavailable. This working definition was both too narrow and too broad. In an important article, Stephan Hartmann has persuasively argued that the definition is too narrow in that there are many simulations of processes for which analytically solvable models are available. More importantly, he has provided the core of what now seems to me to be the correct definition of computer simulations. Hartmanns own position needs a little supplementation, but his principal insight is correct. He writes:
`A model is called dynamic, if it ... includes assumptions about the time-evolution of the system. ... Simulations are closely related to dynamic models. More concretely, a simulation results when the equations of the underlying dynamic model are solved. This model is designed to imitate the time evolution of a real system. To put it another way, a simulation imitates one process by another process. In this definition, the term process refers solely to some object or system whose state changes in time.[footnote omitted] If the simulation is run on a computer, it is called a computer simulation.
It is the idea of one process imitating another that is the key idea here. A specific example will help. Suppose that one is computationally simulating the orbital motion of a planet. This will consist in successive computations of the planet's state (position and velocity) at discrete time intervals, using the mathematical equations that constitute a model of the orbital kinematics and an arrangement of the outputs from these computations into a representation of the planet's motion. In this particular example, it is the behavior of the system that is being simulated and there is no explicit concern about whether the processes that generate the behavior are accurately modelled.
The core of the simulation is the dynamical working out of the successive positions and velocities of the planet. To capture the special nature of computer simulations, we need to focus on the devices by means of which the computational science is carried out. Included in the class of concrete computational devices are both digital and analog computers; that is, analog devices that are used for computational purposes and not just as physical models. In different examples there could even be non-computational simulations that result in dynamical representations as, for example, when a film is taken of a scale model of an airframe in a wind tunnel test, but we are not concerned here with that wider class of simulations, interesting as they are. It is important that the simulation is actually carried out on a concrete device, for mere abstract representations of computations do not count as falling within the realm of computational science. This is because important constraints occur in physical computational devices that are omitted in abstract accounts of computation, such as bounded memory storage, access speed, truncation errors, and so on, yet these are crucially important in real computational science. I do, however, include computations run on virtual machines in the class of concrete computations. Furthermore, the whole process between data input and output must be run on a computer in order for it to be a computer simulation. In contrast, the more general area of computational science can involve computers in only some stages in the process, with the others being done more traditionally 'by hand'.
It is the fact that this solution process is dynamic that crucially differentiates it from the computational model that underlies it, for logical and mathematical representations are essentially static. I shall call this dynamic part of the process the core simulation. Each individual computation in the core simulation could, from the in principle perspective, be viewed as a replacement for the human calculations which are usually infeasible in practice, if not individually then sequentially. In our planetary example, this core simulation leads to specific numerical values of position and velocity. The computational process is thus an amplification instrument it is speeding up what an unaided human could do but this takes the method into an extended realm in a way that closely resembles the way that moving into the realm of the humanly unobservable does for instruments. We shall explore these similarities in section 4.3.
It is with the representation of the output that one of the key methodological differences of simulations emerges. Consider three different ways of representing the results of the core simulation. In the first way, the results are displayed in a numerical table. In the second way, the results are represented graphically by an elliptical shape. In the third way, the output is displayed dynamically via an elliptical motion. If the results of the core simulation are displayed in a numerical array, that array will be a static representation of the planets motion, as will be a display in the form of a static ellipse, whereas if the results are graphically displayed as an elliptical motion, it will then constitute a dynamical representation.
We can thus formulate the following definition:
System S provides a core simulation of behavior B just in case S dynamically produces solutions to a computational model which correctly represents, either dynamically or statically, B. If in addition the computational model used by S correctly represents the mechanisms by means of which the real system R produces B, then S provides a core simulation of system R with respect to B.
It is important in static representations of behavior that the successive solutions which have been computed be arranged in an order that correctly represents the successive positions of the planet in its orbit. Thus, if the solutions were computed in a random order and the numerical array of the solutions presented them as such, this would not be a simulation of the planets orbital motion but, at best, of something else. In contrast, if the solutions were computed in parallel, or in an order different from the order in which they occur in the actual orbit but they were then systematically re-ordered so as to conform to the actual order in the output representation, we would have a simulation of the orbital behavior, although not of the planetary system because the underlying computational processes misrepresent the dynamics of the real system. So the process that constitutes the simulation consists of two linked processes the process of calculating the solutions to the model, within which the order of calculations is important for simulations of the system but unimportant for simulations of its behavior, and the process of presenting them in a way that is either a static or a dynamic representation of the motion of the real planet, within which the order of representation is crucial.
The internal clock in a computer simulation is thus crucial for arriving at the correct ordering in core simulations of systems and in the dynamical representations of their behavior. Simulations of static systems, for example those in equilibrium, can be viewed as degenerate cases of dynamic simulations, in part because successive states need to be calculated to simulate the unchanging state of the system.
When both a core simulation (which is always dynamical) and a dynamical representation are present, we have a full computer simulation of the planets motion. In the early days of computer simulations, static numerical arrays were all that was available and it would seem unreasonable to disallow these pioneering efforts as simulations. Nevertheless, the difference between types of output turns out to be important for understanding why simulations have introduced a distinctively different method into computational science. For the definitions we have given up to this point appear to give us no reason to claim that computer simulations are essentially different from methods of numerical mathematics. Numerical mathematics is the subject concerned with obtaining numerical values of the solutions to a given mathematical problem; numerical methods is that part of numerical mathematics concerned with finding an approximate, feasible, solution to a given problem; and numerical analysis has as its principal task the theoretical analysis of numerical methods and the computed solutions,
with particular emphasis on the error between the computed solution and the exact solution.
We cannot identify numerical mathematics with computational simulations, even though the former play a central role in many areas of the latter because, as we have seen, there is a dynamical aspect to simulations that is absent in numerical mathematics. A more important issue concerns the form of the outputs. A purely automated science of the kind discussed in section 1.2 would attribute no special status to dynamical or graphical representations of the output. It is only when we focus on humans as scientists that the importance of the output representation is apparent, because the information contained in vast numerical arrays is cognitively inaccessible to humans until a conversion instrument is available.
Approximately 60% of the human brains sensory input comes from vision. Because the human eye has a non-linear response to colour, brightness and colour manipulation can often be used in displays to bring out features that might be hidden in a `natural image. Similarly, grey-scale images often provide a better representation of structure than does a coloured image, as do colour and contrast enhancement and edge detection software. It is partly because of the importance of qualitative features such as these for humans that computer visualizations have become so widely used. There has been a reliance on graphical models in the past recall the famous photograph of Watson and Crick in front of the molecular model of DNA and the elementary school representation of a magnetic field by iron filings but visual representations have usually been downplayed as merely useful heuristic devices. Yet graphical representations are not simply useful, they are in many cases necessary because of the overwhelming amount of data generated by modern instruments, a fact we identified in section 1.2 as the quantity of data issue. A flight simulator on which pilots are trained would be significantly lacking in its simulation capabilities if the `view from the `window was represented in terms of a massive numerical data array. Such data displays in numerical form are impossible to assimilate, whereas the right kind of graphical displays are, perceptually, much easier to understand. To give just one example, a relatively crude finite difference model of the flow of gas near a black hole will, over 10,000 time steps, generate a solution comprising 1.25 billion numerical values. This is why the results of simulations are often presented in the form of dynamic films rather than by static photographs, revealing one limitation of traditional scientific journals with respect to these new methods. Such dynamic presentations have the additional advantage of our being able to see which structures are stable over time. This is also true of many agent-based models if we display the results of such models numerically or statically, we cannot `see the higher-order emergent patterns that result from the interactions between the agents. For the purposes of human science the output representation is more than a heuristic device, it becomes a part of the simulation itself. It is not the extension of human computational abilities alone that produces a new kind of scientific method but the combination of that extension and a conversion of the mode of presentation.
The appeal to perceptual ease may disturb philosophical readers who will be inclined to identify the simulation with the equivalence class of all simulations that have the same core simulation. We have also become used to arguments in which it is routinely assumed that transformations of graphical representations into propositional representations and vice versa can be performed. Because of the greater generality of propositional representations, they are taken to be the primary object of study. There is, of course, a sense in which this view is correct, because it is the computer code that causes, via the hardware, pixel illumination. Sometimes these outputs are in the shape of formal symbols (formal languages rely only on the geometrical shape of the symbols for individuation) and sometimes they are in the shape of an image. But this transformational view concedes too much to an in principle approach. Because one of the goals of science is human understanding, how the simulation output is represented is of great epistemological import when visual outputs provide considerably greater levels of understanding than do other kinds. The output of the instrument must serve one of the primary goals of science, which is to produce increased understanding of the phenomena being investigated. Increases in human understanding are obviously not always facilitated by propositional representations and in some cases are precluded altogether. The form of the representation can profoundly affect our understanding of a problem and because understanding is an epistemic concept, this is not at root a practical matter but an epistemological one.
PAGE 150
. Jensen 1998, pp.65-66.
. Feller 1968a, p. 156. PAGE 1
. Karplus 1959, p. 11.
.Feynman 1965, section 12-1 My attention was drawn to this wonderful slogan by Gould and Tobochnik 1988, p.620.
.Comte 1869, Whewell 1840.
. Arthur et al 1997.
.For other examples, see Epstein 1997, Lecture 4.
. Introduced in Kitcher 1989.
.See the remarks on p. 447 of Kitcher 1989 regarding calculations of the chemical bond.
. For a simple exposition of Poisson processes, see any standard text on probability. One clear account can be found in Barr and Zehna 1971, pp. 344ff.
.See e.g. Huntsberger and Billingsley 1973, p.122; Feller 1968a, pp. 156ff..
. For more on this see Humphreys 2000.
.With, perhaps, the additional twist that, as van Fraassens excellent 1989 has suggested, talk of laws in fundamental physics should be replaced by talk of symmetries.
.It is easy to imagine situations in which this would not be true and in which the Poisson model would not hold, as when cars travel in armed convoys for security reasons.
.Feller 1968b, p.115.
. Hopfield and Tank 1986.
.See e.g. Mezard et al 1987.
.For evidence on the last, see Humphreys 1976, Appendix.
. These and other cautions can be found in Abbey et al. 1995, Gamble and Lewis 1996, Gamble 1998 amongst others.
. A different example comes from traditional physics. Here is what one formerly standard text says about using Laplace's equation to model electrostatic phenomena:
`If electrostatic problems always involved localized discrete or continuous distribution of charge with no boundary surfaces, the general solution (1.17) [i.e. the simple integration of the charge density over all charges in the universe] would be the most convenient and straightforward solution to any problem. There would be no need of Poisson's or Laplace's equation. In actual fact, of course, many, if not most, of the problems of electrostatics involve finite regions of space, with or without charge inside, and with prescribed boundary conditions on the bounding surfaces. These boundary conditions may be simulated...but (1.17) becomes inconvenient as a means of calculating the potential, except in simple cases... ... The question arises as to what are the boundary conditions appropriate for Poisson's (or Laplace's) equation in order that a unique and wellbehaved solution (i.e. physically reasonable) solution exist inside the bounded region. Physical experience leads us to believe that specification of the potential on a closed surface...defines a unique potential problem. This is called a Dirichlet problem... (Jackson 1962, 14-16) So again `physical experience subjectmatterspecific knowledge lies behind the use of a standard model (and enhancing it for more complex applications).
.This fact is explicitly acknowledged in Pearl 2000.
. Kolmogorov 1942
.Cohen 1996, p.4
.When I was a schoolboy, a very elderly gentleman came to our science class to talk. He seemed weighed down by the problems of old age. The topic of his talk was the development during World War I of forward mounted machine guns that could fire through the propellors of aeroplanes, the synchronization mechanisms of which were adapted from mechanical calculators. As he regaled us with stories of miscalculations that resulted in pilots shooting off their own propellors and plummeting to their deaths, he began to pull from his jacket pockets various gears, levers, and springs, all machined from steel and brass. As his talk progressed, he became progressively more sprightly and cheerful, the talk ending with a large pile of metal on the table and the speaker considerably less bent.
. One of the first philosophers to have discussed simulations explicitly is Sayre 1965. (See also Crosson and Sayre 1963). As the importance of this field becomes recognized amongst philosophers, I hope that his priority in this area will be properly recognized.
. Humphreys 1991, p. 501.
. Hartmann 1996.
.In correspondence with Fritz Rohrlich and Ron Laymon dated June 29 1990, I wrote `...Im not sure that we ought to exclude as a simulation some computer run that analytically works out the trajectory of a falling body in a vacuum under a uniform gravitational force, for example. Eventually, I settled in the published paper for a narrower conception, wrongly as it turned out.
. Hartmann 1996, p. 5
.There is a different use of the term `simulation' that is standard in computer science, which is
PAGE 1
also closely tied to the concept of a virtual machine. `A virtual machine is a `machine' that owes its existence solely to a program that runs (perhaps with other intervening stages) on a real, physical machine and causes it to imitate the usually more complex machine to which we address our instructions. Such high level programming languages as LISP, PROLOG, and POPII thus define virtual machines. (Clark l989, p.12) This is not the sense in which the term `simulation is to be taken here, although there are connections that can be made, and the sense just discussed is perhaps better called `emulation' rather than `simulation'. Nor do I intend this account to cover what is called `simulation theory' in cognitive science, wherein understanding of others comes through projectively imagining ourselves in their place.
.Simulations of boundary value problems are frequently concerned with static phenomena, as in applications of Poisson s equation (2u(x,y,z) = (x,y,z).
.There is a danger of semantic confusion arising in this area. The output of a simulation is often displayed and it is then said "That's a simulation of X". But that is simply a shorthand for referring to the combined process of core simulation and output representation we have just described.
.Russ 1990, p.1.
.Issues involving human/data interfaces constitute an important area of research. It has been said that if Apollo XIII had been equipped with monitors that displayed critical data in a more easily assimilated form, the near disaster to that mission would have been much easier to deal with.
.For some striking examples in more traditional formats, see Tufte 1983, 1990.
. Smarr 1985, p. 404
PAGE 1
$%RSTst3wx
!p#q#$$$$$$$$$$ $"$*$,$0$;&<&zzrzrzzz jB*phB*mHphsHB*H*mHphsH jB*ph!j0JB*H*UmH phsH 5B*\mH phsH j0JB*H*Uph B*phB*H*mH phsH 0JmH sH jB*UmH phsH jB*UmH phsH B*mH phsH ,y
i!o#.$=&>&**'2`2d
Hdd`d]m<&>&++../.44X7Y7888
88;;xAyAAAERRYY[[\\0`aachhXhZhiijjppqqltmtvxxxxxzOb:BB*OJQJ\^JphB*H*phB*H*phj0JB*H*Uph6B*H*]mH phsH B*H*mH phsH 6B*]mH phsH !j0JB*H*UmH phsH B*mH phsH B*ph:`2m4455g6h6>zA{AEHHKxNyNW3XYYD]&_ccll#vxd`dd`xxzz{{F}h}}}}~:34G(Ԏ͒+d`d$da$dd`BC4FGH&'([\Þ˟̟ ɠA 8GֲEFcd[˩˙˩˩˩˩˩˙˙˓˙˩˩˙˙˙˩˩˩˩˩˩H*mH sH 6]mH sH 5\mH sH j0JH*UmH sH OJQJ^JmH sH 5OJQJ\^JmH sH 5B*OJQJ\^JphB*OJQJ\^Jph(j0JB*H*OJQJU\^Jph4+,
ȥɥ89زHI[\]d$da$$d`a$d`[\]^cdghmnp345PQfgh !wxstu>ҳҳҳҳҳ5OJQJ\^JmH sH mHnHsH ujUmH sH mH sH j0JUmH sH mHnHu jU#5B*OJQJ\^JmH phsH F]ijklm4Pf wtd
$d
$$& #$a$t>tczHz&v
$
$ddd`d>?@stucdz{|HJLLNPfhz|&'(vwŴᣔ5H*OJQJ\^JmH sH j5OJQJ\^JmH sH !5OJQJ\^JmHnHsH u#j5OJQJU\^JmH sH 5OJQJ\^J5OJQJ\^JmH sH mH sH j0JUmH sH 7mHnHsH umH sH jUmH sH . 00P/ =!"#$%. 00P/ =!"#$%. 00P/ =!"#$%. 00P/ =!"#$%. 00P/ =!"#$%. 00P/ =!"#$%. 00P/ =!"#$%. 00P/ =!"#$%. 00P/ =!"#$%. 00P/ =!"#$%. 00P/ =!"#$%. 00P/ =!"#$%. 00P/ =!"#$%. 00P/ =!"#$%. 00P/ =!"#$%.00P/ =!"#$%. 00P/ =!"#$%. 00P/ =!"#$%. 00P/ =!"#$%. 00P/ =!"#$%. 00P/ =!"#$%. 00P/ =!"#$%. 00P/ =!"#$%. 00P/ =!"#$%. 00P/ =!"#$% 1h/ =!"#$%DyK http://www.virginia.edu/~pwh2a/yK@http://www.virginia.edu/~pwh2a/
i4@4NormalCJ_HmH sH tH <A@<Default Paragraph Font4&`4Footnote Reference.U@. Hyperlink>*B*phw
P&(.3H::lKQTTxZ*V,K=]
!=[
,BLgy
P +. T%0K:AIGQ\clpӂҊ Z,j0` P@p0` Py io S%T%,,.k//j0006J:K:=AADHGIGPQQQVW~\\ccjllooppr8rOrfrrs
w|}}}k҂ӂъҊݗ اYZy`+,-9:;=WX{ 6izG^zDDWi3JF\]hk0000@`000000@`000@`00@`00000@`00@`000@0@0@0@0@0@`0@0@0@`0@0@0@`0@0@0@`0@0@0@0@`0@0@0@0@`0@0@`0@0@0@`0@00@0@`0@0@0@0@0@0@0@0@0@00@00@00@0@@0@@0@0@0@@0@@0@0@0@0@@0@0@@0@0@0@@0@@0@`00@0@0@0
@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0
<&B[>{~`2x+]t|}$SsjX
!4:<rxz!').!t!T!Temdr#e
l
GMW\HP~qrt{}~ &)&I'R'((11
;;;;vCC;HFH#S)S9S@STT)U2UXXcc!k'kkkww{{X}_}ɉωv~)1)1-<=~zch#-KT'AJGQku7Az49INkdrJ
W
UaMX5B~qP \ 2!:!""##&&n/q///m0p0111172>2'4:46666m9s9<<??:?C?@@ B.BDDUI_IzOOfTpTTT\\^^``=`E`cbwbggclullmnnooqrttuu'w2w~ ~nxǂтƉωċ͋ӏۏ~{}*;JWl{\b+6ס$եæ̦]j
!`dõ2;-<=[p#57;{HO{>GX^joKQHSk333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333-<=DVFgk C:\talks2\RAND paper.doc bC:\Documents and Settings\paul\Application Data\Microsoft\Word\AutoRecovery save of RAND paper.asd bC:\Documents and Settings\paul\Application Data\Microsoft\Word\AutoRecovery save of RAND paper.asd@Wrr 12^_`efj0@0$0L@080t@0f0h0@0p0@00@UnknownGz Times New Roman5Symbol3&z Arialoz Times New Roman TURTimes New RomanO1CourierCourier New"qh;;T`!2023 Oh+'0\
$
0<DLT3ss sssNormal rm2rmMicrosoft Word 9.0@캃@f*?@kB՜.+,D՜.+,4hp
UVa HS/CSw`T 3Title 8@_PID_HLINKSAxz3 http://www.virginia.edu/~pwh2a/
!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~Root Entry FB94BData
1TableW$WordDocumentSummaryInformation(DocumentSummaryInformation8CompObjjObjectPoolB94BB94B
FMicrosoft Word Document
MSWordDocWord.Document.89q*