Critics also attempted to differentiate connectionism from rule governed systems. However, bothcome under the more general description of systems of functions, with traditional rules differing in allowing only discrete states. This limitation leads to the brittleness which plagued expert systems. Fuzzy rule systems, which allow continuous valued logics, have recently become popular because they exhibit less brittleness. On one interpretation, neural networks are fuzzy rule systems, with summation of activation being a weighted rule and output activations corresponding to probabilities. Discrete logic is a subset of connectionism, with the discreteness implemented by threshold activation functions.
Latimer and others argued that connectionism forces the theorist to be explicit abo ut the structure of his or her model. This is a welcome virtue for any researcher who has tried to derive predictions from a verbal model. Oliphant criticises connectionist models on the grounds that their explicitness does not guarantee their internal consistency. However, the objection is true of any formal model, and its main force is to argue for careful analysis of the behaviour of a model. I would claim that suchproblems have become common because of the ease with which any model can be simulated on acomputer. Many papers present simulation results without attempting to understand the behaviour ofthe system and its parameters.
Both Andrews and Latimer note that explicitness can be a problem when modellers differentiate theory relevant and theory irrelevant components. Putative theory irrelevant components usually arise from hacks which the researcher did not consider before they attempted to implement their model as a computer program. Theory irrelevance is a claim of invariance over a class of implementations. If the connectionist model is not to share the verbal model's flexibility of interpretation, the invariance claim must be justified.
As a researcher concerned with subtle aspects of behaviour such as the higher moments of reaction time distribution (e.g. Mewhort., Braun, & Heathcote, 1992 ) I think that implementation details are important: Plausible implementation in neural wetware can provide useful constraints on models. This is not to deny the usefulness of analyses at computational and algorithmic levels (e.g.Humphreys, Wiles, & Dennis, 1994). However, converging constraints can be derived from implementation issues such as Fledman and Ballard's (1982) 100 step limit, locality of weight update rules (making backpropogation implausible as a learning algorithm) or the ubiquity of lateral inhibitory processing in the brain. Lateral inhibition is particularly useful in modelling competition and choice (e.g. Heathcote, 1993) and has lately been found necessary for previously exclusively feedforward models (e.g. Plaut & McClelland, 1993).
As a researcher concerned with reaction time, I would like to support Kohonen's claim in his invited address to the Fifth Australian Conference on Neural Networks that a connectionist unit's behaviour should be characterised in terms of a differential equation. For example the activation updatefunction used by classical PDP models (1) is the steady state solution to a differential equation with activation decay (2).
Similarly Cohen, Dunbar, and McClelland's (1990) cascade update equation, used to predict reaction time in the Stroop paradigm (3), is an Euler's approximation to (2) with a time constant t (see Equation4).
The sigmoid transfer function commonly used in PDP models (5) can be implemented in a moreneurally plausible differential equation which separates inhibitory (J-) and excitatory (J+) inputs (6). The steady state solution to Equation (6) reveals a sigmoid function bounded on the interval between -1 and 1 (7).
Differential equations make reaction time a natural prediction of connectionist models. A decay term removes the problem of autonomous reset between trials and is useful in modelling sequential dependencies. Finally, the explicit analyses of psychological models recommended above can be performed with the powerful tools available for the analysis of dynamical systems.
Feldman, J. A., & Ballard, D. H. (1982). Connectionist models and their properties. Cognitive Science, 6, 205-254.
Fodor, J. A., & Pylyshyn, S. (1989). Connectionism and cognitive architecture: A critical analysis. Cognition, 28, 3-71.
Heathcote, A. (1993) Episodic ART: A model of episodic recognition memory. Proceedings of the Fifth Australian Conference on Neural Networks.
Hornik, Stinchcombe, & White (1989).Multilayer feedforward networks are universal approximators. Neural Networks, 2, 359-366.
Humphreys, M. S., Wiles, J., & Dennis, S. (1994). Data structures and access processes: A first approximation to a Theory of Human Memory. Unpublished Manuscript.
Mewhort, D. J. K., Braun, J. G., & Heathcote, A. (1992). Response-time distributions and the Stroop task: A test of Cohen, Dunbar, and McClelland's (1990) parallel distributed processing model., Journal of Experimental Psychology: Human Perception and Performance, 18, 872-882.
Plaut, D. & McClelland, J. L. (1993). Generalisation with componential attractors: Word and nonword reading in an attractornetwork. Proceedings of the 15th Annual Conference of the Cognitive Science Society. Hillsdale, NJ, Earlbaum.