Weekly QuEST Discussion Topics, 22 June

QuEST 22 June 2018

As some of the team is about to interact with our CSAIL colleagues we want to spend some time this week talking about narratives and specifically the MIT effort Genesis.

The Genesis Story Understanding and Story Telling System
A 21st Century Step toward Artificial Intelligence
Patrick Henry Winston

Story understanding is an important differentiator of human intelligence, perhaps the most important differentiator.

  • The Genesis system was built to model and explore aspects of story understanding using simply expressed,100 sentence stories drawn from sources ranging from Shakespeare’s plays to fairy tales.
  • I describe Genesis at work as it reflects on its reading,
  • searching for concepts,
  • reads stories with controllable allegiances and cultural biases,
  • models personality traits,
  • answers basic questions about why and when,
  • notes concept onsets,
  • anticipating trouble,
  • calculates similarity using concepts,
  • models question-driven interpretation,
  • aligns similar stories for analogical reasoning,
  • develops summaries, and
  • tells and persuades using a reader model.
  • Since a key starting point for Genesis is the START system we might spend some time to cover some aspects of it:
  • Natural Language Annotations for Question Answering*
    Boris Katz, Gary Borchardt and Sue Felshin
  • This paper presents strategies and lessons learned from the use of natural language annotations to facilitate question answering in the START information access system.
  • START [Katz, 1997; Katz, 1990] is a publicly-accessible information access system that has been available for use on the Internet since 1993 (http://start.csail.mit.edu/).
  • START answers natural language questions by presenting components of text and multi-media information drawn from a set of information resources that are hosted locally or accessed remotely through the Internet.
  • These resources contain structured, semi-structured and unstructured information.
  • START targets high precision in its question answering, and in large part, START’s ability to respond to questions derives from its use of natural language annotations as a mechanism by which questions are matched to candidate answers.
  • When new information resources are incorporated for use by START, natural language annotations are often composed manually—usually at an abstract level— then associated with various information components.
  • While the START effort has also explored a range of techniques for automatic generation of annotations, this paper focuses on the use of, and benefits derived from, manually composed annotations within START and its affiliated systems.
Categories: Uncategorized

Weekly QuEST Discussion Topics, 15 June

QuEST June 15, 2018

As we continue our preparing for our impending interactions with our CSAIL colleagues we want to venture into a discussion on narratives. We’ve previously discussed this topic for example:

Toward a Computational Model of Narrative
George Lakoff, Srini Narayanan v5
International Computer Science Institute and
University of California at Berkeley

• Narratives structure our understanding of the world and of ourselves. They exploit the shared cognitive structures of human motivations, goals, actions, events, and outcomes.

• We report on a computational model that is motivated by results in neural computation and captures fine-grained, context sensitive information about human goals, processes, actions, policies, and outcomes.

• We describe the use of the model in the context of a pilot system that is able to interpret simple stories and narrative fragments in the domain of international politics and economics.

• We identify problems with the pilot system and outline extensions required to incorporate several crucial dimensions of narrative structure.

Categories: Uncategorized

Weekly QuEST Discussion Topics, 1 June

QuEST 1 June 2018

As we move to DIY AI – do-it-yourself Artificial Intelligence – we want to consider a range of issues.  A recent interaction with a colleague, Ken Forbus of Northwestern, included a relevant discussion on some of his team’s work.

  • In this article I argue that achieving human-level AI is equivalent to learning how to create sufficiently smart software social organisms.
  • This implies that no single test will be sufficient to measure progress.
  • Instead, evaluations should be organized around showing increasing abilities to participate in our culture, as apprentices.
  • This provides multiple dimensions within which progress can be measured,
  • including how well different interaction modalities can be used,
  • what range of domains can be tackled,
  • what human-normed levels of knowledge they are able to acquire,
  • as well as others.
  • I begin by motivating the idea of software social organisms, drawing on ideas from other areas of cognitive science, and provide an analysis of the substrate capabilities that are needed in social organisms in terms closer to what is needed for computational modeling.
  • Finally, the implications for evaluation are discussed.

We also have some colleagues from MIT coming to see us and us going to see them – in prep for those interactions we want to make the group aware of some relevant work:

The Art of the Propagator

Alexey Radul and Gerald Jay Sussman

Computer Science and Artificial Intelligence Laboratory

Technical Report

massachusetts inst i t u t e o f technology, cambridge , ma 02139 usa — www. c s a il.mit.edu

MIT-CSAIL-TR-2009-002 January 26, 2009

We develop a programming model built on the idea that the basic computational elements are autonomous machines interconnected b shared cells through which they communicate. Each machine continuousl examines the cells it is interested in, and adds information to some based on deductions it can make from information from the others. This model makes it easy to smoothly combine expression oriented and constraint-based programming; it also easily accommodate implicit incremental distributed search in ordinary programs.

This work builds on the original research of Guy Lewis Steel Jr. [19] and was developed more recently with the help of Chris Hanson.

System building using Genesis’s box-and-wire mechanism

Patrick H.Winston

7October 2014

The Genesis Manifesto:

Story Understanding and Human Intelligence

Patrick Henry Winston and Dylan Holmes

May 1, 2018


We believe we must construct biologically plausible computational models of human story understanding if we are to develop a computational account of human intelligence. We argue that building a story-understanding system exposes computational imperatives associated with human competences such as question answering, mental modeling, culturally biased story interpretation, story-based hypothetical reasoning, and self-aware problem solving. We explain that we believe such human competences rest on a uniquely human ability to construct complex, highly nested symbolic descriptions.

We illustrate our approach to modeling human story understanding by describing the development of the Genesis story understanding system and by explaining how Genesis goes about understanding short, 20- to 100-sentence stories expressed in English. The stories include, for example, summaries of plays, such as Shakespeare’s Macbeth; fairy tales, such as Hansel and Gretel ; and contemporary conflicts, such as the 2007 Estonia–Russia cyberwar.

We explain how we ensure that work on Genesis is scientifically grounded, we identify representative questions to be answered by empirical science, and we note why story understanding has much to offer not only to Artificial Intelligence but also to fields such as business, defense, design, economics, education, humanities, law, linguistics, neuroscience, philosophy, psychology, medicine, and politics.

Categories: Uncategorized

Weekly QuEST Discussion Topics, 25 May

QuEST 25 May 2018

Some of the team has been interested in the impact of ‘pre-training’ and along those lines this week we will review some recent work by our colleagues at Facebook:

 Mahajan, Dhruv, Ross Girshick, Vignesh Ramanathan, Kaiming He, Manohar Paluri, Yixuan Li, Ashwin Bharambe, and Laurens van der Maaten. “Exploring the Limits of Weakly Supervised Pretraining.” arXiv preprint arXiv:1805.00932(2018).


Categories: Uncategorized

Weekly QuEST Discussion Topics, 18 May

QuEST 18 May 2018

We want to start this week discussing some of the left over topics from last week – specifically some more background information to answer some of the questions that people asked as we shut down on the connection between physiological neurons and artificial neural networks – we will provide some background and then use the Limulus vision system to explain some choices in common models.

We also might provide a quick review of some information about the human visual system.

We also want to discuss some recent interactions on RNN / LSTMs:


The fall of RNN / LSTM

It only took 2 more years, but today we can definitely say:

“Drop your RNN and LSTM, they are no good!”

But do not take our words for it, also see evidence that Attention based networks are used more and more by GoogleFacebook,Salesforce, to name a few. All these companies have replaced RNN and variants for attention based models, and it is just the beginning. RNN have the days counted in all applications, because they require more resources to train and run than attention-based models. See this post for more info.

Categories: Uncategorized

Weekly QuEST Discussion Topics and News, 11 May

QuEST 11 May 2018


This week we will have our colleague Dr. Mike M. present some background material on unsupervised learning. 

Unsupervised learning can be Useful for Finding Intrinsic Classes or the Underlying Structure of the Data

•       Find relationship between input patterns:

–       Premise – similar inputs naturally cluster

•       The similarity measure chosen will determine the effectiveness of the Algorithm

Ideally, data consist of clusters, small variances within clusters, all points within a cluster ‘similar,’ and often represented by cluster center

Set of codewords = codebook; categorize new data by membership (which cluster does the new piece of data belong to?)

Categories: Uncategorized

Weekly QuEST Discussion Topics, 20 Apr

April 20, 2018 Leave a comment

QuEST 20 April 2018

We will have our colleague Prof Bert P provide lead us into a discussion of other approaches to AI (other than neural) that will provide us some of the requirements for our ‘representation’ concerns.  The material from last week that we didn’t get to – see below will be posted.

We want to focus our technical discussions this week on context.  We  have defined some of the characteristics we seek in 3rd wave AI and that included the ability to use/exploit context was listed.  We will start by reviewing previously discussed information about vision systems as an example of perception is all about context, to include Mach bands, negative color after images and also provide the details of what we referred to last week in the experiments that demonstrated the mammalian visual systems uses Gabor function models.  We will also demonstrate how the Limulus visual system can generate similar artifacts, again driving home the point it isn’t about the stimulus in isolation it requires context.

We then want to review material we’ve discussed on ‘context’, specifically material provided by our colleague Mitch Kokar.  This will provide us a path to discuss representations, specifically tools like sematic networks and OWL and statistical relationship learning.

Finally this leads us to a discussion on the difference between semantic Web and semantic interpretation.

  • The Semantic Web is a convention for formal representation languages that lets software services interactwith each other “without needing artificial intelligence.”11
  • The problem of understanding human speech and writing – the semantic interpretation problem-is quite different from the problem of software service interoperability.

–     Semantic interpretation deals with imprecise, ambiguous natural languages, whereas service interoperability deals with making data precise enough that the programs operating on the data will function effectively.

  • Unfortunately, the fact that the word “semantic” appears in both “Semantic Web” and “semantic interpretation“ means that the two problems have often been conflated, causing needless and endless consternation and confusion.

Eventually in later meetings we will planning domain definition language (PDDL) that is used to standardize AI planning language and its relationship to OWL ontological solutions.



Attachments area

Categories: Uncategorized