Home > Uncategorized > Weekly QuEST Discussion Topics and News, 31 Mar

Weekly QuEST Discussion Topics and News, 31 Mar

QuEST March 31, 2017

This week’s discussion will include first allowing Sean M. to make some additional points on remote viewing and then a continuation of our discussion on ‘aligning’ multiple agents.  On the latter topic the issue is for example a recent news article on Netflix changing their user ratings from 1-5 stars to thumbs up or down.  They found that change resulted in a 200 % increase in reviews but also noted that although viewers would provide high star ratings to for example artsy films they were more likely to watch lesser graded fun films.  Why the disconnect?  Similar to the disconnect in polling in the election last year?  Clearly the vocabulary for communicating between the human agent and the computer scoring is broken if in fact the computer hopes to estimate human response via the score.  This discussion also leads us back to the agent-to-agent communication issue.

The recent article: ‘Bots are learning to chat in their own language’

Born in Ukraine and raised in Toronto, the 31-year-old is now a visiting researcher at OpenAI, the artificial intelligence lab started by Tesla founder Elon Musk and Y combinator president Sam Altman. There, Mordatch is exploring a new path to machines that can not only converse with humans, but with each other. He’s building virtual worlds where software bots learn to create their own language out of necessity.

And the related technical article:

Emergence of grounded compositional language in Multi-agent populations:  Mordatch / abbeel

By capturing statistical patterns in large corpora, machine learning has enabled significant advances in natural language processing, including in machine translation, question answering, and sentiment analysis. However, for agents to intelligently interact with humans, simply capturing the statistical patterns is insufficient. In this paper we investigate if, and how, grounded compositional language can emerge as a means to achieve goals in multi-agent populations. Towards this end, we propose a multi-agent learning environment and learning methods that bring about emergence of a basic compositional language.

This language is represented as streams of abstract discrete symbols uttered by agents over time, but nonetheless has a coherent structure that possesses a defined vocabulary and syntax. We also observe emergence of non-verbal communication such as pointing and guiding when language communication is unavailable.


Learning Multiagent Communication
with Backpropagation
Sainbayar Sukhbaatar
Dept. of Computer Science
Courant Institute, New York University
Arthur Szlam
Facebook AI Research
New York
Rob Fergus
Facebook AI Research
New York


arXiv:1605.07736v2 [cs.LG] 31 Oct 2016

29th Conference on Neural Information Processing Systems (NIPS 2016), Barcelona, Spain



  • Many tasks in AI require the collaboration of multiple agents. Typically, the communication protocol between agents is manually specified and not altered during training. In this paper we explore a simple neural model, called CommNet, that uses continuous communication for fully cooperative tasks. The model consists of multiple agents and the communication between them is learned alongside their policy. We apply this model to a diverse set of tasks, demonstrating the ability of the agents to learn to communicate amongst themselves, yielding improved performance over non-communicative agents and baselines. In some cases, it is possible to interpret the language devised by the agents, revealing simple but effective strategies for solving the task at hand.


Also on agent to agent communication – I’ve presented before the view that although not in reality two distinct agents – one view of the dual model is to adopt that metaphor.  A recent article provided by our colleague Teresa H – provides a vehicle to renew that discussion:


The Manipulation of Pace within

Endurance Sport

Sabrina Skorski 1* and Chris R. Abbiss 2


Frontiers in Physiology | www.frontiersin.org   February 2017 | Volume 8 | Article 102


In any athletic event, the ability to appropriately distribute energy is essential to prevent premature fatigue prior to the completion of the event. In sport science literature this is termed “pacing.” Within the past decade, research aiming to better understand the underlying mechanisms influencing the selection of an athlete’s pacing during exercise has dramatically increased. It is suggested that pacing is a combination of anticipation, knowledge of the end-point, prior experience and sensory feedback. In order to better understand the role each of these factors have in the regulation of pace, studies have often manipulated various conditions known to influence performance such as the feedback provided to participants, the starting strategy or environmental conditions. As with all research there are several factors that should be considered in the interpretation of results from these studies. Thus, this review aims at discussing the pacing literature examining the manipulation of: (i) energy expenditure and pacing strategies, (ii) kinematics or biomechanics, (iii) exercise environment, and (iv) fatigue developmentnews summary (47)

Categories: Uncategorized
  1. tdhawkes
    March 31, 2017 at 9:18 pm

    Dear QuEST attendees, Teresa Hawkes, Ph.D. here.

    The few minutes of QuEST I heard today have sparked many mind-wheels turning. I was a poet and dancer before I was a scientist. Please accept my occasional metaphors 😉

    There is a path diagram for that set of neurophysiological activations that we experience as 1) consciousness, 2) dreaming, and other states such as 3) coma. I am compiling proposed path diagrams from what is known of the temporal sequencing of microcircuit activations during unconscious (UC) and conscious experiences (C) including dreaming (UC), memory (C+UC), decision-making (C+UC), visual perception (mostly UC), etc.

    Why is this relevant to QuEST objectives?

    What path diagram would programmers draw for the calculations carried out by a computer relative to their program instructions?

    I propose we have a human and a computer do the same decision-making problem (see Robert Patterson’s work).

    The human would be evaluated in terms of accuracy and ERP matrices during a dense array encephalographic recording (dEEG) as the problem was being solved. Key ERPs: would include early P’s and N’s (latency < 100) and categories defined by common literature and methods (see Steven J. Luck's work)..

    Can we trace the computer's algorithm at key points during instruction performance and plot a timeseries? We would also have the computer's accuracy and reaction time to accurate response, as well as the human's.

    We could then compare their RT, accuracy, and path diagrams for similarities and differences when calculating the same problem.

    What do you think?

  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: