Archive
Weekly QuEST Discussion Topics, 20 Dec
QuEST 20 Dec 2019
This will be the last QuEST meeting of the year (after Friday the next meeting will be the Kabrisky lecture on 10 Jan 2020, where we will do our annual ‘What is QuEST’ lecture in honor of one of the QuEST founders Dr. Matt Kabrisky) – all are welcome as always and no prior exposure to ideas on consciousness needed especially for the Kabrisky lecture.
For the 20 Dec 2019 meeting (we might cut this short as many have already started the holidays) we want to continue our discussion on the move towards using as motivation for solving some current AI limitations what is known about consciousness. The Keynote at NeurIPS by Prof Bengio:
https://slideslive.com/38921750/from-system-1-deep-learning-to-system-2-deep-learning
Past progress in DL has concentrated mostly on learning from a static data set, mostly for perception task and other Sys1 tasks which are done intuitively and unconsciously by humans. However in recent years a shift in research direction and new tools such as soft attention and progress in deep rl and are opening the door to the development of novel dep architectures and training frameworks for addressing Sys2 tasks (which are done consciously), such as reasoning planning capturing causality and obtaining systematic generalization in natural language process and other applications. Such as expansion of DL from Sys1 task to Sys2 tasks is important to achieve the old deep learning goal of discovering high level abstract representations because we argue that sys2 requirements will put pressure on representational learning to discover the kind of high level concepts which humans manipulate with language. We argue that towards this objective, soft attention mechanisms constitute a key ingredient to focus computation on a few concepts at a time (a conscious thought) as per the consciousness prior and its associated assumption that many high level dependencies can be approximately captured by a sparse factor graph. We also argue how the agent perspective in deep learning can help put more constraints on the learned representations to capture affordances, causal variables and model transitions in the environment. Finally we propose that meta learning the modularization aspect of the consciousness prior and the agent perspective on representation learning should facilitate reuse of learned components in novel ways even if statistically improbable, as in counterfactuals) enabling more powerful forms of compositional generalization (ie out of distribution generalization based on the hypothesis of localized in time space and concept space changes in the environment due to interventions of agents.
Similarly there is a related spectrum article.
Our discussion a couple of weeks back on ‘culture enabled AI’ leads us to discuss Muffins / Badger / Magagent / xAI – a range of topics all related to the issue of ‘alignment’ between agents – the key technical challenge for culture enabled AI. We’ve had several occasions that were crystalized in a recent discussion with colleagues with the tag lines (no attribution):
Just in time, just enough and just for me!
What does it require to determine what an agent needs at a given time so you are providing ‘just in time’ – how do you figure out what is the topic of the agent’s current mental state – then attack the issue of how much of what you as an agent have available to you that might be relevant to that agent what is just enough information / knowledge to provide and all that requires some understanding of just for me (the receiving agent’s)
We want to think through this issue from the perspectives of a system providing a human information just in time / just enough / tuned just for that particular human but also from the perspective of a computer agent providing to another computer agent what it should provide exactly when it should provide it and only what it needs and tuned to that receiving computer agent.
First our colleague Cameron L provided us with a link to the work Badger:
BADGER: Learning to (Learn [Learning Algorithms] through Multi-Agent Communication)
- In this work, we propose a novel memory-based multi-agent meta-learning architecture and learning procedure that allows for learning of a shared communication policy that enables the emergence of rapid adaptation to new and unseen environments by learning to learn learning algorithms through communication.
- Behavior, adaptation and learning to adapt emerges from the interactions of homogeneous experts inside a single agent.
- The proposed architecture should allow for generalization beyond the level seen in existing methods, in part due to the use of a single policy shared by all experts within the agent as well as the inherent modularity of ‘Badger’.
An architecture and a learning procedure where:
- · An agent is made up of many experts
- · All experts share the same communication policy (expert policy), but have different internal memory states
- · There are two levels of learning, an inner loop (with a communication stage) and an outer loop
- · Inner loop – Agent’s behavior and adaptation should emerge as a result of experts communicating between each other. Experts send messages (of any complexity) to each other and update their internal states based on observations/messages and their internal state from the previous time-step. Expert policy (comm policy) is fixed and does not change during the inner loop·
Exhibiting the following novel properties:
- · Roles of experts and connectivity among them assigned dynamically at inference time
- · Learned communication protocol with context dependent messages of varied complexity
- · Generalizes to different numbers and types of inputs/outputs
- · Can be trained to handle variations in architecture during both training and testing
- Initial empirical results show generalization and scalability along the spectrum of learning types.
- Inner loop loss need not even be a proper loss function. It can be any kind of structured feedback guiding the adaptation during the agent’s lifetime ~*** megagent?
- · Outer loop – An expert policy (comm policy) is discovered over generations of agents, ensuring that strategies that find solutions to problems in diverse environments can quickly emerge in the inner loop
- · Agent’s objective is to adapt fast to novel tasks
This may require us to divert into some discussion on metal learning – and specifically meta reinforcement learning.
This is in some ways of interest to our group’s effort led by Prof Bert P – Muffins – we won’t get out in front of the team’s publication of that work – but do want to introduce aspects of the Megagent that we need to relate to Badger:
- A complication to inter-agent interaction occurs when the agents learn (change their own functionality), when new agents are introduced, or existing agents functionality modified.
- This research focuses on creating a general use multi-agent system, Middleware Unifying Framework For Independent Nodes System (MUFFINS), and implementing a mechanism, the Megagent, that addresses the interaction challenges.
- The Megagent provides the ability for agents to assess their performance per data source and to improve it with transformations based on feedback.
xAI by Gunning / Aha
Dramatic success in machine learning has led to a new wave of AI applications (for example, transportation, security, medicine, finance, defense) that offer tremendous benefits but often cannot explain their decisions and actions to human users. DARPA’s explainable artificial intelligence (XAI) program endeavors to create AI systems whose learned models and decisions can be understood and appropriately trusted by end users. ** this last part is where we’ve focused – it is not always necessary to understand how the AI cognition works – but it is critical to make AI solutions that are trustworth and enable appropriate trust by other agents **
Weekly QuEST Discussion Topics, 13 Dec
QuEST 13 Dec 2019
Some of our colleagues at Neurips forwarded us this link:
https://slideslive.com/38921750/from-system-1-deep-learning-to-system-2-deep-learning
it is a keynote talk by Yoshua Bengio – Turing Award winner for Deep Learning clearly heading towards an area related to our QuEST interest.
Past progress in DL has concentrated mostly on learning from a static data set, mostly for perception task and other Sys1 tasks which are done intuitively and unconsciously by humans. However in recent years a shift in research direction and new tools such as soft attention and progress in deep rl and are opening the door to the development of novel dep architectures and training frameworks for addressing Sys2 tasks (which are done consciously), such as reasoning planning capturing causality and obtaining systematic generalization in natural language process and other applications. Such as expansion of DL from Sys1 task to Sys2 tasks is important to achieve the old deep learning goal of discovering high level abstract representations because we argue that sys2 requirements will put pressure on representational learning to discover the kind of high level concepts which humans manipulate with language. We argue that towards this objective, soft attention mechanisms constitute a key ingredient to focus computation on a few concepts at a time (a conscious thought) as per the consciousness prior and its associated assumption that many high level dependencies can be approximately captured by a sparse factor graph. We also argue how the agent perspective in deep learning can help put more constraints on the learned representations to capture affordances, causal variables and model transitions in the environment. Finally we propose that meta learning the modularization aspect of the consciousness prior and the agent perspective on representation learning should facilitate reuse of learned components in novel ways even if statistically improbable, as in counterfactuals) enabling more powerful forms of compositional generalization (ie out of distribution generalization based on the hypothesis of localized in time space and concept space changes in the environment due to interventions of agents.
Similarly there is a related spectrum article.
Our discussion last week on ‘culture enabled AI’ leads us this week to discuss Muffins / Badger / Magagent / xAI – a range of topics all related to the issue of ‘alignment’ between agents – the key technical challenge for culture enabled AI. We’ve had several occasions that were crystalized in a recent discussion with colleagues with the tag lines (no attribution):
Just in time, just enough and just for me!
What does it require to determine what an agent needs at a given time so you are providing ‘just in time’ – how do you figure out what is the topic of the agent’s current mental state – then attack the issue of how much of what you as an agent have available to you that might be relevant to that agent what is just enough information / knowledge to provide and all that requires some understanding of just for me (the receiving agent’s)
We want to think through this issue from the perspectives of a system providing a human information just in time / just enough / tuned just for that particular human but also from the perspective of a computer agent providing to another computer agent what it should provide exactly when it should provide it and only what it needs and tuned to that receiving computer agent.
First our colleague Cameron L provided us with a link to the work Badger:
BADGER: Learning to (Learn [Learning Algorithms] through Multi-Agent Communication)
- In this work, we propose a novel memory-based multi-agent meta-learning architecture and learning procedure that allows for learning of a shared communication policy that enables the emergence of rapid adaptation to new and unseen environments by learning to learn learning algorithms through communication.
- Behavior, adaptation and learning to adapt emerges from the interactions of homogeneous experts inside a single agent.
- The proposed architecture should allow for generalization beyond the level seen in existing methods, in part due to the use of a single policy shared by all experts within the agent as well as the inherent modularity of ‘Badger’.
An architecture and a learning procedure where:
- · An agent is made up of many experts
- · All experts share the same communication policy (expert policy), but have different internal memory states
- · There are two levels of learning, an inner loop (with a communication stage) and an outer loop
- · Inner loop – Agent’s behavior and adaptation should emerge as a result of experts communicating between each other. Experts send messages (of any complexity) to each other and update their internal states based on observations/messages and their internal state from the previous time-step. Expert policy (comm policy) is fixed and does not change during the inner loop·
Exhibiting the following novel properties:
- · Roles of experts and connectivity among them assigned dynamically at inference time
- · Learned communication protocol with context dependent messages of varied complexity
- · Generalizes to different numbers and types of inputs/outputs
- · Can be trained to handle variations in architecture during both training and testing
- Initial empirical results show generalization and scalability along the spectrum of learning types.
- Inner loop loss need not even be a proper loss function. It can be any kind of structured feedback guiding the adaptation during the agent’s lifetime ~*** megagent?
- · Outer loop – An expert policy (comm policy) is discovered over generations of agents, ensuring that strategies that find solutions to problems in diverse environments can quickly emerge in the inner loop
- · Agent’s objective is to adapt fast to novel tasks
This may require us to divert into some discussion on metal learning – and specifically meta reinforcement learning.
This is in some ways of interest to our group’s effort led by Prof Bert P – Muffins – we won’t get out in front of the team’s publication of that work – but do want to introduce aspects of the Megagent that we need to relate to Badger:
- A complication to inter-agent interaction occurs when the agents learn (change their own functionality), when new agents are introduced, or existing agents functionality modified.
- This research focuses on creating a general use multi-agent system, Middleware Unifying Framework For Independent Nodes System (MUFFINS), and implementing a mechanism, the Megagent, that addresses the interaction challenges.
- The Megagent provides the ability for agents to assess their performance per data source and to improve it with transformations based on feedback.
xAI by Gunning / Aha
Dramatic success in machine learning has led to a new wave of AI applications (for example, transportation, security, medicine, finance, defense) that offer tremendous benefits but often cannot explain their decisions and actions to human users. DARPA’s explainable artificial intelligence (XAI) program endeavors to create AI systems whose learned models and decisions can be understood and appropriately trusted by end users. ** this last part is where we’ve focused – it is not always necessary to understand how the AI cognition works – but it is critical to make AI solutions that are trustworth and enable appropriate trust by other agents **
Weekly QuEST Discussion Topics, 6 Dec
Dr. Jared Culbertson will lead this week’s discussion. Cap will contribute to the discussion based on a related effort he is tackling.
Predictive, informative models of urban environments have historically been challenging to develop due to the high complexity and interconnections between characteristics such as size, poverty, violence, infrastructure, mobility, economics, etc. Recent work from the Santa Fe Institute led by Luis Bettencourt has proposed some simple scaling laws derived from analyzing data from a large number of cities across the world that seem to be robust across cultural, geographic, and political boundaries. We want to understand how this work might be useful, for example, in guiding urban policy decisions and resource allocations.
Boston Area Research Initiative: https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/SE2CIX
Bettencourt Science paper (attached)
Quest Group's Blog 