Computing with structured connectionist networks

Feldman, Jerome A.; Fanty, Mark; Goddard, Nigel; Lynne, Kenton J.

doi:10.1145/42372.42378

Cited by 89 publications

(21 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The conceptual roots of this approach, which may be called structured connectionism, include the old-style connectionism (Feldman & Ballard, 1982;Feldman, Fanty, Goddard, & Lynne, 1988) and the augmented transition network (ATN) methods in natural language processing (Woods, 1970). Structured connectionist networks can directly represent sequential order and can support functional abstraction, as indicated, for instance, by the newer work on embodied construction grammar (Chang, Feldman, & Narayanan, 2004), where units stand for concept frames such as 'walk' and directed connections represent the sequential ordering of units.…”

Section: Structured Connectionism Reduxmentioning

confidence: 99%

The minority report: some common assumptions to reconsider in the modelling of the brain and behaviour

Edelman

2015

Journal of Experimental & Theoretical Artificial Intelligen

View full text Add to dashboard Cite

Reverse-engineering the brain involves adopting and testing a hierarchy of working hypotheses regarding the computational problems that it solves, the representations and algorithms that it employs and the manner in which these are implemented. Because problem-level assumptions set the course for the entire research programme, it is particularly important to be open to the possibility that we have them wrong, but tacit algorithm-and implementation-level hypotheses can also benefit from occasional scrutiny. This paper focuses on the extent to which our computational understanding of how the brain works is shaped by three such rarely discussed assumptions, which span the levels of Marr's hierarchy: (i) that animal behaviour amounts to a series of stimulus/response bouts, (ii) that learning can be adequately modelled as being driven by the optimisation of a fixed objective function and (iii) that massively parallel, uniformly connected layered or recurrent network architectures suffice to support learning and behaviour. In comparison, a more realistic approach acknowledges that animal behaviour in the wild is characterised by dynamically branching serial order and is often agentic rather than reactive. Arguably, such behaviour calls for open-ended learning of world structure and may require a neural architecture that includes precisely wired circuits reflecting the serial and branching structure of behavioural tasks.

show abstract

Section: Structured Connectionism Reduxmentioning

confidence: 99%

The minority report: some common assumptions to reconsider in the modelling of the brain and behaviour

Edelman

2015

Journal of Experimental & Theoretical Artificial Intelligen

View full text Add to dashboard Cite

show abstract

“…FIG is built as a "structured connectionist" (Feldman et al 1988) (or "localist connectionist" or spreading activation) system for four main reasons. First, it suits the task; since parallel computation and the numeric combination of evidence are required.…”

Section: Characteristics Of the Modelmentioning

confidence: 99%

A parallel approach to syntax for generation

Ward

1992

Artificial Intelligence

View full text Add to dashboard Cite

To produce good utterances from non-trivial inputs a natural language generator should consider many words in parallel, which raises the question of how to handle syntax in a parallel generator. If a generator is incremental and centered on the task of word choice, then the role of syntax is merely to help evaluate the appropriateness of words. One way to do this is to represent syntactic knowledge as an inventory of "syntactic constructions" and to have many constructions active in parallel at run-time. If this is done then the syntactic form of utterances can be emergent, resulting from synergy among constructions, and there is no need to build up or manipulate representations of syntactic structure. This approach is implemented in FIG, an incremental generator based on spreading activation, in which syntactic knowledge is represented in the same network as world knowledge and lexical knowledge.

show abstract

“…Second, PDP networks (again, neural networks in general) typically lack built-in structure (Feldman et at., 1988). For instance, in PDP networks, the structural components responsible for incorrect system performance are "identified" by back propagation of corrective feedback to individual connections in the network, and the system structure is "modified" by adjusting the connection weights in the direction of steepest gradient descent in the error space.…”

Section: I Umentioning

confidence: 99%

High-Level Connectionist Models

Pollack¹

1989

View full text Add to dashboard Cite

The major achievement of this first seniannum was the significant revision and extension of the Recursive Auto-Associative Memory (RAAM) work for publication in the journal Artificial Intelligence. Included as an appendix to this report, the article includes several new elements:% 1) Background z ( , r 51 V erThe work was more clearly set into the area oTiistributed representations, machine learning, and the adequacy of the connectionist approach for high-level cognitive modelingi 2) New Experiment-RAAM was applied to finding compact representations for sequences of letters, 3) AnalysisThe developed representations were analyzed as features which range from categorical to distinctive. Categorical features distinguish between conceptual categories while distinctive features vary within categories and discriminate or label the members. The representations were also analyzed geometrically) A P'>-4) ApplicationsFeasibility studies were performed and described on inference by association, and on using RAAM-generated patterns along with cascaded networks for natural language parsing. Both of these remain long-term goals of the project. \ There are several other areas that are currently being explored, and which should be written up in the second semiannum: Discrete Analog SystemsOne problem for most recurrent or sequential work i a-connectionism is the default assumption of real arithmetic implemented in floating point. This means that states (or internal representations) are yery imprecise, as there is no equality test. We have been experimenting with a:6 activanon function based upon the inverse of Cantor's function, shown below, which is a sigmoid-shaped step-function, and have be 4XIII .-,.-2-able to use standard neural network learning algorithms with it. One result so far is a RAAM which exactly reconstructs its trees. Inductive Inference J. Feldman set out (on an electronic bulletin board) the problem of inductive inference of finite state automata from language examples as a possible benchmark for connectionist networks. This is now a very active area, with research ongoing at CMU, Toronto, and UMass. Sequential Cascaded Networks had already shown some promise in this area, on the parity and balanced parenthesis languages. With a simple modification, they have worked on more complex test cases (from a 1982 paper by M. Tomita). 1,0- 75--Chaotic Behavior One of problems that plague modem connectionist learning algorithms is that gradient descent is susceptible to local minima. This has been discounted by the originators of Back-Propagation, but it is generally known that "sometimes it converges and sometimes it doesn't." It is also known that if all weights start at 0, or any other constant, the networks won't converge. The default initial condition for the technique has thus been to start with small random weights. In the first part of Kolen & Goel's paper, they show that if the weights aren't small, a large percentage of initial conditions lead to non-convergence. We have examined this question in more detail...

show abstract

Computing with structured connectionist networks

Cited by 89 publications

References 17 publications

The minority report: some common assumptions to reconsider in the modelling of the brain and behaviour

The minority report: some common assumptions to reconsider in the modelling of the brain and behaviour

A parallel approach to syntax for generation

High-Level Connectionist Models

Contact Info

Product

Resources

About