Weekly QuEST Discussion Topics and News, 21 Apr

April 20, 2017 Leave a comment

QuEST 21 April 2017

Last week we began an extremely interesting discussion on ‘Autonomy’.  We used our recently written FAQ (frequently asked questions) on the topic where we generated a self-consistent set of definitions to make our discussions on capabilities and capability gaps more precise.

We started with the idea of locking down terms relevant to multi-agent system of systems (SoS) where we might need for these agents collaborating (note this could be humans and machines in the SoS).  Do these agents have to be intelligent?  Do these agents need to communicate meaning? Do they need to communicate knowledge?  What is understanding?  To have that discussion we used:

1.1         Definitions & Foundational Concepts

1.1.1        What is intelligence? What is artificial intelligence?

Intelligence is the ability to gather observations, generate knowledge, and appropriately apply that knowledge to accomplish tasks. Artificial Intelligence (AI) is a machine that possesses intelligence.

1.1.2        What is an Autonomous system’s (AS’s) internal representation?

Current AS’s are programmed to complete tasks using different procedures.  The AS’s internal representation is how the agent structures what it knows about the world, its knowledge (what the AS uses to take observations and generate meaning), how the agent structures its meaning and its understanding.  For example, the programmed model used inside of the AS for its knowledge-base.  The knowledge base can change as the AS acquires more knowledge or as the AS further manipulates existing knowledge to create new knowledge.

1.1.3        What is meaning?  Do machines generate meaning?

Meaning is what changes in an Airman’s or Autonomous System’s (AS’s) internal representation as a result of some stimuli.  It is the meaning of the stimuli to that the Airman or the System. When you, the Airman, look at an American flag, the sequence of thoughts and emotions that it evokes in you, is the meaning of that experience to you at that moment. When the image is shown to a computer, and if the pixel intensities evoked some programed changes in that computers program, then that is the meaning of that flag to that computer (the AS). Here we see that the AS generates meaning that is completely different than what an Airmen does. The change in the AS’s internal representation, as a result of how it is programmed, is the meaning to the AS. The meaning of a stimulus is the agent specific representational changes evoked by that stimulus in that agent.  The update to the representation, evoked by the data, is the meaning of the stimulus to this agent.  Meaning is NOT just the posting into the representation of the data it is all the resulting changes to the representation.  For example, the evoking of tacit knowledge or a modification of the ongoing simulation (consciousness) or even the updating of the agent’s knowledge resulting from the stimuli is included in the meaning of a stimulus to an agent.  Meaning is not static and changes over time.  The meaning of a stimulus is different for a given agent depending on when it is presented to the agent.

1.1.4        What is understanding?  Do machines understand?

Understanding is an estimation of whether an AS’s meaning will result in it acceptably accomplishing a task. Understanding occurs if it raises an evaluating Airman or evaluating AS’s belief that the performing AS will respond acceptably. Meaning is the change in an AS’s internal representation resulting from a query (presentation of a stimulus). Understanding is the impact of the meaning resulting in the expectation of successful accomplishment of a particular task.

1.1.5        What is knowledge?

Knowledge is what is used to generate the meaning of stimuli for a given agent.  Historically knowledge comes from the species capturing and encoding via evolution in genetics, experience by an individual animal or animals via culture communicating knowledge to other members of the same species (culture).  With the advances in machine learning it is a reasonable argument that most of the knowledge that will be generated in the world in the future will be done by machines.

1.1.6        What is thinking? Do machines think?

Thinking is the process used to manipulate an AS’s internal representation; a generation of meaning, where meaning is the change in the internal representation resulting from a stimuli. If an AS can change or manipulate its internal representation, then it can think.

1.1.7        What is reasoning? Do machines reason?

Reasoning is thinking in the context of a task.  Reasoning is the ability to think about what is perceived and the actions to take to complete a task. If the system updates its internal representation, it generates meaning, and is doing reasoning when that thinking is associated with accomplishing a task. If the system’s approach is not generating the required ‘meaning’ to acceptably accomplish the task, it is not reasoning appropriately.

We will start this week by opening up to questions associated with this framework of the FAQ terms relevant to systems consisting of multiple agents (humans and computers).  As the framework is new to many it is prudent to rehash them so we can begin to ‘chunk’ them into common use.  It turns out this alone is a challenge – since it is so easy to lose the relationship between the terms as we will use them.  So we will spend some time attempting to come up with an approach to bringing others up to speed on our use of these terms to facilitate conversations.  We didn’t get to it last week so one of the deliverables out of this week’s conversation / discussion will be a simple to understand example thread to capture where we are in making autonomous systems and what the world will look like when we actually deliver these systems in this simple example thread.

The one example that I’ve recently been using is putting into a hotel room Alexa / Siri / Cortana … and having it be a ubiquitous aid.  For example, handling on-demand the HVAC (temp / air in the room) and the audio visual (channel location / movie options / radio …), local information to include weather / transportation / exercise / eating…  The discussion is not to build the widgets that facilitate the physical / cyber connectivity but building the joint cognitive solutions – that is what is necessary in the Alexa representation to facilitate her to be able to understand a set of request she has not been programmed to accomplish.  This will provide the machinery to move to the main topic.

The major focus again this week is on the expectation that solutions for many of the mission capabilities we seek will require an Agile/autonomous System of systems (ASoS).  Agility in this phrase is meant to capture the dynamic nature of the composition of the SoS as well as the dynamic nature of the range of tasks this SoS needs to accomplish.  This system (made up of both human and computer agents) has to solve the issue of collaboration between its agents.  Collaboration will require inter-agent communication.  We seek to have agile communication versus having to standardize a communication protocol to maintain maximum agility.  We expect agents will join and depart from these collaborations and some of the required mission capabilities will not be pre-defined.  It seems logical that these agents have to be intelligent.  Do we need these agents to be able to share knowledge or meaning or both?  What is required for two agents to be able to share knowledge or meaning?  Where do goals and intent fit in our framework?  The goal of collaboration is to accomplish some task that requires the ASoS have an understanding, meaning associated with expected successful completion of the task.  What is required for multiple agents to collaboratively achieve understanding for a given task?

I have several articles and a string of email threads to help guide the discussion.  One classic stream is associated with how to make automation a team player with human members of a team from Klein:

Ten Challenges for Making
Automation a “Team Player”
in Joint Human-Agent Activity – gary Klein …

  • We propose 10 challenges for making automation into effective “team players” when they interact with people in significant ways. Our analysis is based on some of theprinciples of human-centered computing that we have developed individually and jointly over the years, and is adapted from a more comprehensive examination of common ground and coordination … We define joint activity as an extended set of actionsthat are carried out by an ensemble of people who are coordinating with each other.1,2
  • Joint activity involves at least four basic requirements.
    All the participants must:
  • • Enter into an agreement, which we call a Basic Compact, that the participants intend to work together
  • • Be mutually predictable in their actions
  • • Be mutually directable
  • • Maintain common ground

The discussion we want to have with respect to the Klein article is how to take his challenges and map them to our framework so we can understand where our gaps are particularly troublesome.

Learning Multiagent Communication with Backpropagation
Sainbayar Sukhbaatar
Dept. of Computer Science
Courant Institute, New York University …

  • Many tasks in AI require the collaboration of multiple agents. Typically, thecommunication protocol between agents is manually specified and not altered during training. In this paper we explore a simple neural model, called CommNet, that uses continuous communication for fully cooperative tasks.
  • The model consists of multiple agents and the communication between them is learned alongside their policy. We apply this model to a diverse set of tasks, demonstrating the ability of the agents to learn to communicate amongst themselves, yielding improved performance over non-communicative agents and baselines.
  • In some cases, it is possible to interpret the language devised by the agents, revealing simple but effective strategies for solving the task at hand.

Emergence of Grounded Compositional Language in Multi-Agent Populations
Igor Mordatch

arXiv:1703.04908v1 [cs.AI] 15 Mar 2017

It Begins: Bots Are Learning to Chat in Their Own Language

Igor Mordatch is working to build machines that can carry on a conversation. That’s something so many people are working on. In Silicon Valley, chatbot is now a bona fide buzzword. But Mordatch is different. He’s not a linguist. He doesn’t deal in the AI techniques that typically reach for language. He’s a roboticist who began his career as an animator. He spent time at Pixar and worked on Toy Story 3, in between stints as an academic at places like Stanford and the University of Washington, where he taught robots to move like humans. “Creating movement from scratch is what I was always interested in,” he says. Now, all this expertise is coming together in an unexpected way

news summary (50)

Categories: Uncategorized

Weekly QuEST Discussion Topics and News, 14 Apr

April 13, 2017 Leave a comment

QuEST 14 April 2017

My sincere apologies for last week – a family medical emergency resulted in a late notice cancellation and a government iphone email application failure resulted in my communication of that situation not occur successfully.

Many of us are focused on ‘Autonomy’.  To that end we’ve written a FAQ (frequently asked questions) on the topic where we generated a self-consistent set of definitions to make our discussions on capabilities and capability gaps more precise.  We will start this week with a presentation of the FAQ terms relevant to systems consisting of multiple agents (humans and computers).  It turns out this alone is a challenge – since it is so easy to lose the relationship between the terms.  So we will spend some time attempting to come up with an approach to bringing others up to speed on our use of these terms to facilitate conversations.  One of the deliverables out of this week’s conversation / discussion will be a simple to understand example thread to capture where we are in making autonomous systems and what the world will look like when we actually deliver these systems in this simple example thread.

The one example that I’ve recently been using is putting into a hotel room Alexa / Siri / Cortana … and having it be a ubiquitous aid.  For example, handling the HVAC (temp / air in the room) and the audio visual (channel location / movie options …), local information to include weather / transportation / exercise / eating.  The discussion is not to build the widgets that facilitate the physical / cyber connectivity but building the joint cognitive solutions.  The will provide the machinery to move to the main topic.

The major focus this week is on the expectation that solutions for many of the mission capabilities we seek will require an Agile/autonomous System of systems (ASoS).  This system (made up of both human and computer agents) has to solve the issue of collaboration between its agents.  Collaboration will require inter-agent communication.  We seek to have agile communication versus having to standardize a communication protocol to maintain maximum agility.  We expect agents will join and depart from these collaborations and some of the required mission capabilities will not be pre-defined.  Do we need these agents to be able to share knowledge or meaning or both?  What is required for two agents to be able to share knowledge or meaning?  Since the goal of collaboration is to accomplish some task that requires the ASoS has an understanding, meaning associated with expected successful completion of the task.  What is required for multiple agents to collaboratively achieve understanding for a given task?

I have several articles and a string of email threads to help guide the discussion:

Ten Challenges for Making
Automation a “Team Player”
in Joint Human-Agent Activity – gary Klein …

  • We propose 10 challenges for making automation into effective “team players” when they interact with people in significant ways. Our analysis is based on some of theprinciples of human-centered computing that we have developed individually and jointly over the years, and is adapted from a more comprehensive examination of common ground and coordination … We define joint activity as an extended set of actionsthat are carried out by an ensemble of people who are coordinating with each other.1,2
  • Joint activity involves at least four basic requirements.
    All the participants must:
  • • Enter into an agreement, which we call a Basic Compact, that the participants intend to work together
  • • Be mutually predictable in their actions
  • • Be mutually directable
  • • Maintain common ground

Learning Multiagent Communication
with Backpropagation
Sainbayar Sukhbaatar
Dept. of Computer Science
Courant Institute, New York University …

  • Many tasks in AI require the collaboration of multiple agents. Typically, thecommunication protocol between agents is manually specified and not altered during training. In this paper we explore a simple neural model, called CommNet, that uses continuous communication for fully cooperative tasks.
  • The model consists of multiple agents and the communication between them is learned alongside their policy. We apply this model to a diverse set of tasks, demonstrating the ability of the agents to learn to communicate amongst themselves, yielding improved performance over non-communicative agents and baselines.
  • In some cases, it is possible to interpret the language devised by the agents, revealing simple but effective strategies for solving the task at hand.

Emergence of Grounded Compositional Language in Multi-Agent Populations
Igor Mordatch

arXiv:1703.04908v1 [cs.AI] 15 Mar 2017

It Begins: Bots Are Learning to Chat in Their Own Language

Igor Mordatch is working to build machines that can carry on a conversation. That’s something so many people are working on. In Silicon Valley, chatbot is now a bona fide buzzword. But Mordatch is different. He’s not a linguist. He doesn’t deal in the AI techniques that typically reach for language. He’s a roboticist who began his career as an animator. He spent time at Pixar and worked on Toy Story 3, in between stints as an academic at places like Stanford and the University of Washington, where he taught robots to move like humans. “Creating movement from scratch is what I was always interested in,” he says. Now, all this expertise is coming together in an unexpected way

news summary (49)

Categories: Uncategorized

Weekly QuEST Discussion Topics and News, 7 Apr

QuEST 7 April 2017

This is a bonus QuEST meeting this week – I had expected to lose this week due to travel but it got a reprieve – the downside is my voice is weak so many will have to pick up the slack during the discussion but we’ve had several email threads that I can’t pass up the opportunity to advance our thoughts.

Many of us are focused on ‘Autonomy’.  To that end we’ve written a FAQ on the topic where we generated a self-consistent set of definitions to make our discussions on capabilities and capability gaps more precise.  We will start this week with a presentation of the FAQ terms relevant to systems consisting of multiple agents (humans and computers).  It turns out this alone is a challenge – since it is so easy to lose the relationship between the terms.  So we will spend some time attempting to come up with an approach to bringing others up to speed on our use of these terms to facilitate conversations.  The will provide the machinery to move to the main topic.

The major focus this week is on the expectation that solutions for many of the mission capabilities we seek will require an Agile System of systems (ASoS).  This system (made up of both human and computer agents) has to solve the issue of collaboration between its agents.  Collaboration will require inter-agent communication.  We seek to have agile communication versus having to standardize a communication protocol to maintain maximum agility.  We expect agents will join and depart from these collaborations and some of the required mission capabilities will not be pre-defined.  Do we need these agents to be able to share knowledge or meaning or both?  What is required for two agents to be able to share knowledge or meaning?  Since the goal of collaboration is to accomplish some task that requires the ASoS has an understanding, meaning associated with expected successful completion of the task.  What is required for multiple agents to collaboratively achieve understanding for a given task?

Autonomy FAQ 88ABW-2017-0021 (1)news summary (48)

Categories: Uncategorized

Weekly QuEST Discussion Topics and News, 31 Mar

March 30, 2017 1 comment

QuEST March 31, 2017

This week’s discussion will include first allowing Sean M. to make some additional points on remote viewing and then a continuation of our discussion on ‘aligning’ multiple agents.  On the latter topic the issue is for example a recent news article on Netflix changing their user ratings from 1-5 stars to thumbs up or down.  They found that change resulted in a 200 % increase in reviews but also noted that although viewers would provide high star ratings to for example artsy films they were more likely to watch lesser graded fun films.  Why the disconnect?  Similar to the disconnect in polling in the election last year?  Clearly the vocabulary for communicating between the human agent and the computer scoring is broken if in fact the computer hopes to estimate human response via the score.  This discussion also leads us back to the agent-to-agent communication issue.

The recent article: ‘Bots are learning to chat in their own language’

Born in Ukraine and raised in Toronto, the 31-year-old is now a visiting researcher at OpenAI, the artificial intelligence lab started by Tesla founder Elon Musk and Y combinator president Sam Altman. There, Mordatch is exploring a new path to machines that can not only converse with humans, but with each other. He’s building virtual worlds where software bots learn to create their own language out of necessity.

And the related technical article:

Emergence of grounded compositional language in Multi-agent populations:  Mordatch / abbeel

By capturing statistical patterns in large corpora, machine learning has enabled significant advances in natural language processing, including in machine translation, question answering, and sentiment analysis. However, for agents to intelligently interact with humans, simply capturing the statistical patterns is insufficient. In this paper we investigate if, and how, grounded compositional language can emerge as a means to achieve goals in multi-agent populations. Towards this end, we propose a multi-agent learning environment and learning methods that bring about emergence of a basic compositional language.

This language is represented as streams of abstract discrete symbols uttered by agents over time, but nonetheless has a coherent structure that possesses a defined vocabulary and syntax. We also observe emergence of non-verbal communication such as pointing and guiding when language communication is unavailable.

 

Learning Multiagent Communication
with Backpropagation
Sainbayar Sukhbaatar
Dept. of Computer Science
Courant Institute, New York University
sainbar@cs.nyu.edu
Arthur Szlam
Facebook AI Research
New York
aszlam@fb.com
Rob Fergus
Facebook AI Research
New York
robfergus@fb.com

 

arXiv:1605.07736v2 [cs.LG] 31 Oct 2016

29th Conference on Neural Information Processing Systems (NIPS 2016), Barcelona, Spain

 

 

  • Many tasks in AI require the collaboration of multiple agents. Typically, the communication protocol between agents is manually specified and not altered during training. In this paper we explore a simple neural model, called CommNet, that uses continuous communication for fully cooperative tasks. The model consists of multiple agents and the communication between them is learned alongside their policy. We apply this model to a diverse set of tasks, demonstrating the ability of the agents to learn to communicate amongst themselves, yielding improved performance over non-communicative agents and baselines. In some cases, it is possible to interpret the language devised by the agents, revealing simple but effective strategies for solving the task at hand.

 

Also on agent to agent communication – I’ve presented before the view that although not in reality two distinct agents – one view of the dual model is to adopt that metaphor.  A recent article provided by our colleague Teresa H – provides a vehicle to renew that discussion:

 

The Manipulation of Pace within

Endurance Sport

Sabrina Skorski 1* and Chris R. Abbiss 2

 

Frontiers in Physiology | www.frontiersin.org   February 2017 | Volume 8 | Article 102

 

In any athletic event, the ability to appropriately distribute energy is essential to prevent premature fatigue prior to the completion of the event. In sport science literature this is termed “pacing.” Within the past decade, research aiming to better understand the underlying mechanisms influencing the selection of an athlete’s pacing during exercise has dramatically increased. It is suggested that pacing is a combination of anticipation, knowledge of the end-point, prior experience and sensory feedback. In order to better understand the role each of these factors have in the regulation of pace, studies have often manipulated various conditions known to influence performance such as the feedback provided to participants, the starting strategy or environmental conditions. As with all research there are several factors that should be considered in the interpretation of results from these studies. Thus, this review aims at discussing the pacing literature examining the manipulation of: (i) energy expenditure and pacing strategies, (ii) kinematics or biomechanics, (iii) exercise environment, and (iv) fatigue developmentnews summary (47)

Categories: Uncategorized

Weekly QuEST Discussion Topics and News, 24 Mar

March 23, 2017 Leave a comment

QuEST March 24 2017

 

This week we will have our colleague Sean M provide us his insights into the topic of Remote Viewing and relate those to the QuEST interest in impacting the ‘subconscious’ of the human teammates to our computer based QuEST agents for more effective group (human-machine) decisions:

 

What engineering advantage can we obtain against the mission of QuEST by taking a serious look at the US government-sponsored “psychic spy” program?  In the words of a former Army unit member, Major David Morehouse, this was “one of the intelligence services’ most controversial, misunderstood, and often feared special access programs.”  In this discussion Sean Mahoney will attempt to demystify the subject.

 

In September of 1995 the public learned of a nearly 23 yearlong intelligence gathering program that utilized a purely human centered ISR technique called ‘remote viewing’.  Remote viewing, as it was used by the military, was developed at the Stanford Research Institute (now SRI International) under contract by the CIA in the 1970s.  It is a teachable, structured pen-and-paper process by which a person can interview their own subconscious mind about a pre-selected target, that their conscious mind is blind to, and report data on that target.

 

Since declassification, many former military remote viewers wrote books and created training programs describing the methodologies they used successfully throughout the life of the government program. A community has sprung up around the practice with thousands of people across the globe actively applying remote viewing to various uses. There is now an International Remote Viewing Association (IRVA), 2 magazines, an annual conference, and a few professional consulting groups that offer remote viewing services for anything from missing persons or property, to legal cases, to research and development efforts. Through books, formal training, conference attendance, and lots of practice, Sean has learned several different methodologies of remote viewing as they have been taught by former military unit members.  Sean will present on his experiences with remote viewing and what he feels it reveals about intuitive cognition and the nature of consciousness. Please join us for this interesting discussion.

news summary (46)

Categories: Uncategorized

Weekly QuEST Discussion Topics and News, 17 Mar

March 16, 2017 Leave a comment

QuEST 17 March 2017

Again there were several interesting email conversation threads going on this week:

We want to hit briefly the article from last week – this thread was initiated by Trevor and Todd from our Sensing dendrite:

Why does deep and cheap learning work so well?
Henry W. Lin and Max Tegmark
Dept. of Physics, Harvard University, Cambridge, MA 02138 and
Dept. of Physics & MIT Kavli Institute, Massachusetts Institute of Technology, Cambridge, MA 02139

arXiv:1608.08225v2 [cond-mat.dis-nn] 28 Sep 2016

  • We show how the success of deep learning depends not only on mathematics but also on physics: although well-knownmathematical theorems guarantee that neural networks can approximate arbitrary functions well, the class of functions of practical interest can be approximated through “cheap learning” with exponentially fewer parameters than generic ones, because they have simplifying properties tracing back to the laws of physics.
  • The exceptional simplicity of physics-based functions hinges on properties such as symmetry, locality, compositionality and polynomial log-probability, and we explore how these properties translate into exceptionally simple neural networks approximating both natural phenomena such as images and abstract representations thereof such as drawings.
  • We further argue that when the statistical process generating the data is of a certain hierarchical form prevalent in physics and machine-learning, a deep neural network can be more efficient than a shallow one.
  • We formalize these claims using information theory and discuss the relation to renormalization group procedures. We prove various “no-flattening theorems” showing when such efficient deep networks cannot be accurately approximated by shallow ones without efficiency loss: flattening even linear functions can be costly, and flattening polynomials is exponentially expensive; we use group theoretic techniques to show that n variables cannot be multiplied using fewer than 2^n neurons in a single hidden layer.

A related topic to this first one:

In another email thread this week we were asked about:

http://www.sciencemag.org/news/2017/03/brainlike-computers-are-black-box-scientists-are-finally-peering-inside?utm_campaign=news_daily_2017-03-07&et_rid=54802259&et_cid=1203472

Brainlike computers are a black box. Scientists are finally peering inside

By Jackie SnowMar. 7, 2017 , 3:15 PM

Last month, Facebook announced software that could simply look at a photo and tell, for example, whether it was a picture of a cat or a dog. A related program identifies cancerous skin lesions as well as trained dermatologists can. Both technologies are based on neural networks, sophisticated computer algorithms at the cutting edge of artificial intelligence (AI)—but even their developers aren’t sure exactly how they work. Now, researchers have found a way to “look” at neural networks in action and see how they draw conclusions.

Neural networks, also called neural nets, are loosely based on the brain’s use of layers of neurons working together. Like the human brain, they aren’t hard-wired to produce a specific result—they “learn” on training sets of data, making and reinforcing connections between multiple inputs. A neural net might have a layer of neurons that look at pixels and a layer that looks at edges, like the outline of a person against a background. After being trained on thousands or millions of data points, a neural network algorithm will come up with its own rules on how to process new data. But it’s unclear what the algorithm is using from those data to come to its conclusions.

“Neural nets are fascinating mathematical models,” says Wojciech Samek, a researcher at Fraunhofer Institute for Telecommunications at the Heinrich Hertz Institute in Berlin. “They outperform classical methods in many fields, but are often used in a black box manner.”

In an attempt to unlock this black box, Samek and his colleagues created software that can go through such networks backward in order to see where a certain decision was made, and how strongly this decision influenced the results.Their method, which they will describe this month at the Centre of Office Automation and Information Technology and Telecommunication conference in Hanover, Germany, enables researchers to measure how much individual inputs, like pixels of an image, contribute to the overall conclusion. Pixels and areas are then given a numerical score for their importance. With that information, researchers can create visualizations that impose a mask over the image. The mask is most bright where the pixels are important and darkest in regions that have little or no effect on the neural net’s output.

For example, the software was used on two neural nets trained to recognize horses. One neural net was using the body shape to determine whether it was horse. The other, however, was looking at copyright symbols on the images that were associated with horse association websites.

This work could improve neural networks, Samek suggests. That includes helping reduce the amount of data needed, one of the biggest problems in AI development, by focusing in on what the neural nets need. It could also help investigate errors when they occur in results, like misclassifying objects in an image.

Other researchers are working on similar processes to look into how algorithms make decisions, including neural nets for visuals as well as text. Continued research is important as algorithms make more decisions in our daily lives, says Sara Watson, a technology critic with the Berkman Klein Center for Internet & Society at Harvard University. The public needs tools to be able to understand how AI makes decisions. Algorithms, far from being perfect arbitrators of truth, are only as good as the data they’re given, she notes.

In a notorious neural network mess up, Google tagged a black woman as a gorilla in its photos application. Even more serious discrimination has been called into question in software that provides risk scores that some courts use to determine whether a criminal is likely to reoffend, with at least one study showing black defendants are given a higher risk score than white defendants for similar crimes. “It comes down to the importance of making machines, and the entities that employ them, accountable for their outputs,” Watson says

 

Not attempting to be dismissive but:

Cathy is pulling the technical article – but from the text in the news article this appears to be a rehash of something we invented in 1990:

 

  • Ruck, D. W., Rogers, S., Kabrisky, M., “Feature Selection Using a Multilayer Perceptron”, Journal of Neural Network Computing, Vol 2 (2), pp 40-48, Fall 1990.

 

When you use a supervised learning system with a mean squared error objective function and differentiable nonlinear neurons – then you can solve the partial differential equations to extract ‘saliency’ – that is you can work through any decision and rank order the inputs to decide an ‘order’ to their impact – in 1990 we weren’t doing representational learning (like with deep neural networks – we didn’t have enough data or compute power) but the equations are the same we just put in features extracted with our computer vision algorithms that were suggested by human radiologists – then after trained when we put in a new mammogram we could extract which features dominated the decision to call something cancer or normal

 

We’ve recently in deep neural networks done similar things in our captioning work to decide what aspects of an image or video a particular linguistic expression is evoked from – for example in a dog chasing Frisbee picture we can back project to find where in the image are the pixels that evoked the word Frisbee – this has cracked the black box somewhat also

 

So both of these suggest to me this news article is just stating what we know (although in general a black box these deep systems can provide us some aspects of their ‘meaning’ that we can understand – this will be a focus of the new start at DARPA xAI – for explainable AI) but again I will review the technical article and if there is more there I will provide an addendum to this email

 

We now have the technical article – I don’t think our response above is far off except for the approach is based on Taylor expansion versus our approach – the ideas are the same and the importance of the problem is good – in a very important way they extend our sensitivity analysis as a special case of their more general Taylor approach:

Pattern Recognition 65 (2017) 211–222

Explaining nonlinear classification decisions with deep Taylor

decomposition

Grégoire Montavona,⁎, Sebastian Lapuschkinb, Alexander Binderc, Wojciech Samekb,⁎,

Klaus-Robert Müllera,d,⁎⁎

a Department of Electrical Engineering & Computer Science, Technische Universität Berlin, Marchstr. 23, Berlin 10587, Germany

b Department of Video Coding & Analytics, Fraunhofer Heinrich Hertz Institute, Einsteinufer 37, Berlin 10587, Germany

c Information Systems Technology & Design, Singapore University of Technology and Design, 8 Somapah Road, Building 1, Level 5, 487372, Singapore

d Department of Brain & Cognitive Engineering, Korea University, Anam-dong 5ga, Seongbuk-gu, Seoul 136-713, South Korea

Nonlinear methods such as Deep Neural Networks (DNNs) are the gold standard for various challenging machine learning problems such as image recognition. Although these methods perform impressively well, they have a significant disadvantage, the lack of transparency, limiting the interpretability of the solution and thus the scope of application in practice. Especially DNNs act as black boxes due to their multilayer nonlinear structure. In this paper we introduce a novel methodology for interpreting generic multilayer neural networks by decomposing the network classification decision into contributions of its input elements. Although our focus is on image classification, the method is applicable to a broad set of input data, learning tasks and network architectures. Our method called deep Taylor decomposition efficiently utilizes the structure of the network by backpropagating the explanations from the output to the input layer. We evaluate the proposed method empirically on the MNIST and ILSVRC data sets.

 

With respect to applications of deep systems:

Another thread was tied to my prior life in using AI/ML for medical detection / diagnosis – thus the article below:

Detecting Cancer Metastases on
Gigapixel Pathology Images
Yun Liu1?, Krishna Gadepalli1, Mohammad Norouzi1, George E. Dahl1,
Timo Kohlberger1, Aleksey Boyko1, Subhashini Venugopalan2??,
Aleksei Timofeev2, Philip Q. Nelson2, Greg S. Corrado1, Jason D. Hipp3,
Lily Peng1, and Martin C. Stumpe1
fliuyun,mnorouzi,gdahl,lhpeng,mstumpeg@google.com
1Google Brain, 2Google Inc, 3Verily Life Sciences,
Mountain View, CA, USA

  • Each year, the treatment decisions for more than 230; 000 breast cancer patients in the U.S. hinge on whether the cancer has metastasized away from the breast.
  • Metastasis detection is currently performed by pathologists reviewing large expanses of biological tissues. This process is labor intensive and error-prone.
  • We present a framework to automatically detect and localize tumors as small as 100 x 100 pixels in gigapixel microscopy images sized 100,000 x 100,000 pixels.
  • Our method leverages a convolutional neural network (CNN) architecture and obtains state-of-the-art results on the Camelyon16 dataset in the challenging lesion-level tumor detection task.
  • At 8 false positives per image, we detect 92:4% of the tumors, relative to 82:7% by the previous best automated approach.
  • For comparison, a human pathologist attempting exhaustive search achieved 73:2% sensitivity.
  • We achieve image-level AUC scores above 97% on both the Camelyon16 test set and an independent set of 110 slides.
  • In addition, we discover that two slides in the Camelyon16 training set were erroneously labeled normal.
  • Our approach could considerably reduce false negative rates in metastasis detection.

 

As another application:

on March 9, 2017http://science.sciencemag.org/ Downloaded from

 

DeepStack: Expert-level artificial intelligence in heads-up no-limit poker
Matej Moravčík,1,2* Martin Schmid,1,2* Neil Burch,1 Viliam Lisý,1,3 Dustin Morrill,1 Nolan Bard,1 Trevor Davis,1 Kevin Waugh,1 Michael Johanson,1 Michael Bowling1

 

  • Artificial intelligence has seen several breakthroughs in recent years, with games often serving as milestones. A common feature of these games is that players have perfect information. Poker is the quintessential game of imperfect information, and a longstanding challenge problem in artificial intelligence. We introduce DeepStack, an algorithm for imperfect information settings. It combines recursive reasoning to handle information asymmetry, decomposition to focus computation on the relevant decision, and a form of intuition that is automatically learned from self-play using deep learning. In a study involving 44,000 hands of poker, DeepStack defeated with statistical significance professional poker players in heads-up no-limit Texas hold’em. The approach is theoretically sound and is shown to produce more difficult to exploit strategies than prior approaches.

 

One of our main topics of multiple agents with objective functions commonality – from our sensing and AFIT mathematics dendrite:

How do we know that your “red” looks ** this is a quale statement – what you perceive in your consciousness is what I perceive **  the same as my “red”? For all we know, your “red” looks like my “blue.” In fact, for all we know your “red” looks nothing like any of my colors at all! If colors are just internal labels  ** labels here is not meant to imply the word – it is meant to describe the representation internal in a vocabulary of conscious thought ** , then as long as everything gets labeled, why should your brain and my brain use the same labels?  ** as long as we can align – why would they have to be the same – keep in mind the Kaku view – philosophers waste our time with these thought problems – he speaks to the ‘what is life concern that has disappeared’ – BUT OUR INTEREST IN OBJECTIVE FUNCTIONS THAT START WITH SIMILAR MACHINERY WHAT CAN I SAY ABOUT THE RESULTING REPRESENTATION – CAN I MAKE A STATEMENT ON WHAT CHAREACTERISTICS OF YOUR RED MY RED HAS TO HAVE? – IN THE RED CASE I WOULD CONTEND THAT THE RELATIONSHIPS BETWEEN YOUR RED AND YOUR BLUE HAVE TO BE REPLICATED – AND BY THE TIME I CONSTRAIN THE RELATIONSHIPS WITH SO MANY QUALIA IT CONSTRAINS THE MEANING (probably should say the comprehension – how qualia are related and how they can interact) TO BE THE SAME WHERE THE MEANING IS THE CHANGES TO THE REPRESENTATION RESULTING FROM THE STIMULI  – **

*** CAP POSITS:  THE MEANING / understanding / comprehension THE WAY WE DEFINE IN QUEST OF YOUR RED IS THE SAME AS THE MEANING OF MY RED ** ?understanding and maybe comprehension versus meaning – jared **really your representation / resulting meaning / understanding / resulting comprehension (relationships and ways the situations can interact – Representation is how agent structured knowledge meaning understanding and comprehension – recall we defined comprehension when we were defining qualia / situations – as something that can be comprehended as a whole – comprehended was defined as being able to discern how it is related to or how it can interact with other situations –

 

A situation is any part of the agent centric internal representation which can be comprehended as a whole by that agent through defining how it interacts with or is related to other parts of the representation in that agent.

We will define comprehended by defining how it interacts or is related to other situations via linking (and types of links).

interacting with other things we mean that the situations have properties or relate to other situations.” *** we would say  can and must be linked to other ‘situations’  = ‘other qualila’ = other chunks***

 

as I thought about this – and thought about the word doc – is your red my red – and my posit – that the meaning you generate for red is the same as what I generate for my red – your comment on I need to use understanding versus meaning – putting all this together I was forced to own up to what I need to stay ‘alignable’ – we don’t have to have the same exact changes to the representation (use our deep learning metaphor – I don’t care if all the activations are the same) – but what I have to maintain between agents for ‘red’ to be alignable is that the relationships to other situations is maintained between the respective agents – when I went down this path it reminded me of how we defined situations – and how we had to clear up the word comprehension – I’m happy to change the word comprehension to understanding – for now I have restricted comprehension to be understanding associated with the task of relationships / interactions with other situations

 

Another thread has continued to advance this week and related to objective function ‘red’ thread is the with interactions between our Airmen sensors autonomy team and our AFIT autonomy team – with the focus on ‘chat-bots’ – the idea that the future is all about these ‘AI bots’ versus apps – and that QuEST chat-bots might provide an avenue where knowledge of the developing representations that capture aspects of consciousness are key to solving the very tough problem of bots that accomplish the type of meaning-making required for many applications – and might be the key to bot-to-bot communication without have to strictly manually define a communication protocol.

Part of the interest of this thread is multi-modal communications – that is the reason the material below was inserted into the thread:

Snap Makes a Bet on the Cultural Supremacy of the Camera

  • https://www.nytimes.com/2017/03/08/technology/snap-makes-a-bet-on-the-cultural-supremacy-of-the-camera.html?_r=0
  • The rising dependence on cameras is changing the way we communicate. Credit Doug Chayka
  • If you’re watching Snap’s stock ticker, stop. The company that makes Snapchat, the popularphoto-messaging app, has been having a volatile few days after its rocket-fueled initial public offering last week.
  • But Snap’s success or failure isn’t going to be determined this week or even this year. This is a company that’s betting on a long-term trend: the rise and eventual global dominance of visual culture.
  • Snap calls itself a camera company. That’s a bit cute, considering that it only just released an actual camera, the Spectacles sunglasses, late last year. Snap will probably build other kinds of cameras, including potentially a drone.
  • But it’s best to take Snap’s camera company claim seriously, not literally. Snap does not necessarily mean that its primary business will be selling a bunch of camera hardware. It’s not going to turn into Nikon, Polaroid or GoPro. Instead it’s hit on something deeper and more important. Through both its hardware and software, Snap wants to enable the cultural supremacy of the camera, to make it at least as important to our daily lives as the keyboard.  ** profound point – camera / visual media as important as keyboard in communicating with other agents people and machines **

 

arXiv:1605.07736v2 [cs.LG] 31 Oct 2016

29th Conference on Neural Information Processing Systems (NIPS 2016), Barcelona, Spain

 

Learning Multiagent Communication
with Backpropagation
Sainbayar Sukhbaatar
Dept. of Computer Science
Courant Institute, New York University
sainbar@cs.nyu.edu
Arthur Szlam
Facebook AI Research
New York
aszlam@fb.com
Rob Fergus
Facebook AI Research
New York
robfergus@fb.com

  • Many tasks in AI require the collaboration of multiple agents. Typically, the

communication protocol between agents is manually specified and not altered during training. In this paper we explore a simple neural model, called CommNet, that uses continuous communication for fully cooperative tasks. The model consists of multiple agents and the communication between them is learned alongside their policy. We apply this model to a diverse set of tasks, demonstrating the ability of the agents to learn to communicate amongst themselves, yielding improved performance over non-communicative agents and baselines. In some cases, it is possible to interpret the language devised by the agents, revealing simple but effective strategies for solving the task at hand.

 

 

 

We are always interested in new information on the neuro-physiology that might provide guidance in our engineering endeavors:

 

Dynamics of cortical dendritic membrane potential and spikes in freely behaving rats
Jason J. Moore,1,2* Pascal M. Ravassard,1,3 David Ho,1,2 Lavanya Acharya,1,4 Ashley L. Kees,1,2 Cliff Vuong,1,3 Mayank R. Mehta1,2,3,5

 

  • Neural activity in vivo is primarily measured using extracellular somatic spikes, which provide limited information about neural computation. Hence, it is necessary to record from neuronal dendrites, which generate dendritic action potentials (DAP) and profoundly influence neural computation and plasticity. We measured neocortical sub- and suprathreshold dendritic membrane potential (DMP) from putative distal-most dendrites using tetrodes in freely behaving rats over multiple days with a high degree of stability and sub-millisecond temporal resolution. DAP firing rates were several fold larger than somatic rates. DAP rates were modulated by subthreshold DMP fluctuations which were far larger than DAP amplitude, indicting hybrid, analog-digital coding in the dendrites. Parietal DAP and DMP exhibited egocentric spatial maps comparable to pyramidal neurons. These results have important implications for neural coding and plasticity.news summary (45)
Categories: Uncategorized

Weekly QuEST Discussion Topics and News, 10 Mar

QuEST 10 March 2017

There were several interesting email conversation threads going on this week:

We want to start this week by talking about Kaku’s book on the future of Mind – this thread was initiated by Eric B from our Rome dendrite of the QuEST neuron assembly:

The Future of the Mind

Book by Michio Kaku

 

The Future of the Mind: The Scientific Quest to Understand, Enhance, and Empower the Mind

 

[he lists three levels of consciousness and highlights many discussions on the subject, such as that of a split brain actions]

https://cosmosmagazine.com/social-sciences/mind-michio-kaku

Kaku outlines levels of consciousness that correspond to different degrees of complexity, from the simplest things like plants at Level 0 to we humans on Level III. The big difference with us is that we are self-aware. “Human consciousness is a specific form of consciousness that creates a model of the world and then simulates it in time,” he writes in the book. “This requires mediating and evaluating many feedback loops to make a decision to achieve a goal.”

The mind of Michio Kaku

What is a physicist doing weighing in on the mysteries of the mind? Tim Dean went to find out.

 

 

There are two great mysteries that overshadow all other mysteries in science – the origin of the universe and what sits on your shoulders, says Michio Kaku.

Mark Von Holden/WireImage/Getty images

 

Michio Kaku has an extraordinary mind. It loves nothing better than occupying itself untangling the mathematics of subatomic strings vibrating in 11 dimensions. “My wife always thinks something strange is happening because I stare out the window for hours,” he says. *** I walk my dog ** “That’s all I’m doing. I’m playing with equations in my head.” In his day job, Kaku works at the very fringes of physics. He made a name for himself as co-founder of string field theory, which seeks to complete Einstein’s unfinished business by unifying all the fundamental forces of the universe into a single grand equation. He regales the public with tales of multiverses, hyperspace and visions of a better future built by science.

Hyperbole is part of his style. He once urged humanity to consider escaping our universe, which several billions of years from now will be in its last throes, by slipping through a wormhole. Hardly a pressing concern for today, but such proclamations lasso attention and get people to think big.

Kaku certainly thinks big in his latest book, and there’s plenty of hyperbole. But The Future of the Mind is somewhat of a surprise. What is a theoretical physicist doing writing a book about the mind – a topic usually reserved for neuroscientists, psychologists and philosophers? As a philosopher myself I was curious to see if a physicist could shed some photons on the problem. I took the opportunity when he was in Sydney spruiking his new book to find out.

Comments from our colleague Mike Y on this thread:

A couple of comments…  There is a difference between consciousness and intelligence.  In principle, we can build machines (or zombies) with extreme intelligence.  But that does not make them conscious.  Consciousness, your subjective experience, depends upon the neuronal architecture of your brain and nervous system.  This determines what sensations, perceptions, and cognitions you can experience. (and what behavioral goals you will have).

 

I agree completely that the human neuronal architecture enables humans to represent limited, specific, aspects of the world and to “run” simulations with them.

 

I also agree that all animals developed sensory systems to help them in their evolutionary niche.  For elephants that is a sensory system the processes low intensity sounds that travel a long way through the earth.  For humans that is high resolution color vision that enables us to identify the state of fruit and other foods sources, and to accurately judge the social/emotional state of conspecifics.  And for dogs (and apparently rhinos) a smell based spatial representation of the world.

 

Then we want to hit the article – this thread was initiated by Trevor and Todd from our Sensing dendrite:

Why does deep and cheap learning work so well?
Henry W. Lin and Max Tegmark
Dept. of Physics, Harvard University, Cambridge, MA 02138 and
Dept. of Physics & MIT Kavli Institute, Massachusetts Institute of Technology, Cambridge, MA 02139

arXiv:1608.08225v2 [cond-mat.dis-nn] 28 Sep 2016

  • We show how the success of deep learning depends not only on mathematics but also on physics: although well-knownmathematical theorems guarantee that neural networks can approximate arbitrary functions well, the class of functions of practical interest can be approximated through “cheap learning” with exponentially fewer parameters than generic ones, because they have simplifying properties tracing back to the laws of physics.
  • The exceptional simplicity of physics-based functions hinges on properties such as symmetry, locality, compositionality and polynomial log-probability, and we explore how these properties translate into exceptionally simple neural networks approximating both natural phenomena such as images and abstract representations thereof such as drawings.
  • We further argue that when the statistical process generating the data is of a certain hierarchical form prevalent in physics and machine-learning, a deep neural network can be more efficient than a shallow one.
  • We formalize these claims using information theory and discuss the relation to renormalization group procedures. We prove various “no-flattening theorems” showing when such efficient deep networks cannot be accurately approximated by shallow ones without efficiency loss: flattening even linear functions can be costly, and flattening polynomials is exponentially expensive; we use group theoretic techniques to show that n variables cannot be multiplied using fewer than 2^n neurons in a single hidden layer.

 

On our topic of multiple agents with objective functions commonality – from our sensing and AFIT mathematics dendrite:

n  Seems like the below makes the argument of an objective function that might impact all of our perception of ‘red’ – if you will an objective function for color perception:  

To remind you of our interest:

This statement is completely consistent with our view of qualia.

Is there a reasonable argument that if two agents get exposed to the same set of

stimuli and have the same objective function that you can make some statement on the relationships

of their resulting representations? 

n   

n  Mark Changizi, Ph.D. Neuroscientist, Author of ‘Harnessed’ & ‘Vision Revolution’ 

n  How do we know that your “red” looks the same as my “red”? For all we know, your “red” looks like my “blue.” In fact, for all we know your “red” looks nothing like any of my colors at all! If colors are just internal labels, then as long as everything gets labeled, why should your brain and my brain use the same labels?

n  Richard Dawkins wrote a nice little piece on color, and along the way he asked these questions.

n  He also noted that not only can color labels differ in your and my brain, but perhaps the same color labels could be used in non-visual modalities of other animals. Bats, he notes, use audition for their spatial sense, and perhaps furry moths are heard as red, and leathery locusts as blue. Similarly, rhinoceroses may use olfaction for their spatial sense, and could perceive water as orange and rival male markings as gray.

n  … see next slide

n  The entirety of these links is, I submit, what determines the qualitative feel of the colors we see. If you and I largely share the same “perceptual network,” then we’ll have the same qualia. And if some other animal perceives some three-dimensional color space that differs radically in how it links to the other aspects of its mental life, then it won’t be like our color space… its perceptions will be an orange of a different color.

n  In fact, in my research I have provided evidence that our primate variety color vision evolved for seeing the color changes occurring on our faces and other naked spots. Our primate color vision is peculiar in its cone sensitivities (with the M and L cones having sensitivities that are uncomfortably close), but these peculiar cone sensitivities are just right for sensing the peculiar spectral modulations hemoglobin in the skin undergoes as the blood varies in oxygenation. Also, the naked-faced and naked-rumped primates are the ones with color vision; those primates without color vision have your typical mammalian furry face.

n  In essence, I have argued elsewhere that our color-vision eyes are oximeters like those found in hospital rooms, giving us the power to read off the emotions, moods and health of those around us.

n  On this new view of the origins of color vision, color is far from an arbitrary permutable labeling system. Our three-dimensional color space is steeped with links to emotions, moods and physiological states, as well as potentially to behaviors. For example, purple regions within color space are not merely a perceptual mix of blue and red, but are also steeped in physiological, emotional and behavioral implications — in this case, perhaps of a livid male ready to punch you.

n  http://www.huffingtonpost.com/mark-changizi-phd/perceiving-colors-differently_b_988244.html

a

 

http://www.newyorker.com/magazine/2017/02/27/why-facts-dont-change-our-minds?mbid=social_facebook

Why Facts Don’t Change Our Minds

New discoveries about the human mind show the limitations of reason.

 

From Adam a related thought on objective functions:  What I liked, that I thought was unique but also in agreement somewhat with the polyvagal theory I’ve been working through

 

If reason is designed to generate sound judgments, then it’s hard to conceive of a more serious design flaw than confirmation bias. Imagine, Mercier and Sperber suggest, a mouse that thinks the way we do. Such a mouse, “bent on confirming its belief that there are no cats around,” would soon be dinner. To the extent that confirmation bias leads people to dismiss evidence of new or underappreciated threats—the human equivalent of the cat around the corner—it’s a trait that should have been selected against. The fact that both we and it survive, Mercier and Sperber argue, proves that it must have some adaptive function, and that function, they maintain, is related to our “hypersociability.”

Mercier and Sperber prefer the term “myside bias.” Humans, they point out, aren’t randomly credulous. Presented with someone else’s argument, we’re quite adept at spotting the weaknesses. Almost invariably, the positions we’re blind about are our own.

 

10/02/2011 05:21 am ET | Updated Dec 02, 2011

Another thread has advanced this week with interactions between our Airmen sensors autonomy team and our AFIT autonomy team – with the focus on ‘chat-bots’ – the idea that the future is all about these ‘AI bots’ versus apps – and that QuEST chat-bots might provide an avenue where knowledge of the developing representations that capture aspects of consciousness are key to solving the very tough problem of bots that accomplish the type of meaning-making required for many applications

 

In another email thread this week initiated by our colleague Morley from our senior leader dendrite:

http://www.sciencemag.org/news/2017/03/brainlike-computers-are-black-box-scientists-are-finally-peering-inside?utm_campaign=news_daily_2017-03-07&et_rid=54802259&et_cid=1203472

Brainlike computers are a black box. Scientists are finally peering inside

By Jackie SnowMar. 7, 2017 , 3:15 PM

Last month, Facebook announced software that could simply look at a photo and tell, for example, whether it was a picture of a cat or a dog. A related program identifies cancerous skin lesions as well as trained dermatologists can. Both technologies are based on neural networks, sophisticated computer algorithms at the cutting edge of artificial intelligence (AI)—but even their developers aren’t sure exactly how they work. Now, researchers have found a way to “look” at neural networks in action and see how they draw conclusions.

Neural networks, also called neural nets, are loosely based on the brain’s use of layers of neurons working together. Like the human brain, they aren’t hard-wired to produce a specific result—they “learn” on training sets of data, making and reinforcing connections between multiple inputs. A neural net might have a layer of neurons that look at pixels and a layer that looks at edges, like the outline of a person against a background. After being trained on thousands or millions of data points, a neural network algorithm will come up with its own rules on how to process new data. But it’s unclear what the algorithm is using from those data to come to its conclusions.

“Neural nets are fascinating mathematical models,” says Wojciech Samek, a researcher at Fraunhofer Institute for Telecommunications at the Heinrich Hertz Institute in Berlin. “They outperform classical methods in many fields, but are often used in a black box manner.”

In an attempt to unlock this black box, Samek and his colleagues created software that can go through such networks backward in order to see where a certain decision was made, and how strongly this decision influenced the results.Their method, which they will describe this month at the Centre of Office Automation and Information Technology and Telecommunication conference in Hanover, Germany, enables researchers to measure how much individual inputs, like pixels of an image, contribute to the overall conclusion. Pixels and areas are then given a numerical score for their importance. With that information, researchers can create visualizations that impose a mask over the image. The mask is most bright where the pixels are important and darkest in regions that have little or no effect on the neural net’s output.

For example, the software was used on two neural nets trained to recognize horses. One neural net was using the body shape to determine whether it was horse. The other, however, was looking at copyright symbols on the images that were associated with horse association websites.

This work could improve neural networks, Samek suggests. That includes helping reduce the amount of data needed, one of the biggest problems in AI development, by focusing in on what the neural nets need. It could also help investigate errors when they occur in results, like misclassifying objects in an image.

Other researchers are working on similar processes to look into how algorithms make decisions, including neural nets for visuals as well as text. Continued research is important as algorithms make more decisions in our daily lives, says Sara Watson, a technology critic with the Berkman Klein Center for Internet & Society at Harvard University. The public needs tools to be able to understand how AI makes decisions. Algorithms, far from being perfect arbitrators of truth, are only as good as the data they’re given, she notes.

In a notorious neural network mess up, Google tagged a black woman as a gorilla in its photos application. Even more serious discrimination has been called into question in software that provides risk scores that some courts use to determine whether a criminal is likely to reoffend, with at least one study showing black defendants are given a higher risk score than white defendants for similar crimes. “It comes down to the importance of making machines, and the entities that employ them, accountable for their outputs,” Watson says

 

Not attempting to be dismissive but:

Cathy is pulling the technical article – but from the text in the news article this appears to be a rehash of something we invented in 1990:

 

  • Ruck, D. W., Rogers, S., Kabrisky, M., “Feature Selection Using a Multilayer Perceptron”, Journal of Neural Network Computing, Vol 2 (2), pp 40-48, Fall 1990.

 

When you use a supervised learning system with a mean squared error objective function and differentiable nonlinear neurons – then you can solve the partial differential equations to extract ‘saliency’ – that is you can work through any decision and rank order the inputs to decide an ‘order’ to their impact – in 1990 we weren’t doing representational learning (like with deep neural networks – we didn’t have enough data or compute power) but the equations are the same we just put in features extracted with our computer vision algorithms that were suggested by human radiologists – then after trained when we put in a new mammogram we could extract which features dominated the decision to call something cancer or normal

 

We’ve recently in deep neural networks done similar things in our captioning work to decide what aspects of an image or video a particular linguistic expression is evoked from – for example in a dog chasing Frisbee picture we can back project to find where in the image are the pixels that evoked the word Frisbee – this has cracked the black box somewhat also

 

So both of these suggest to me this news article is just stating what we know (although in general a black box these deep systems can provide us some aspects of their ‘meaning’ that we can understand – this will be a focus of the new start at DARPA xAI – for explainable AI) but again I will review the technical article and if there is more there I will provide an addendum to this email

 

We now have the technical article – I don’t think our response above is far off except for the approach is based on Taylor expansion versus our approach – the ideas are the same and the importance of the problem is good – in a very important way they extend our sensitivity analysis as a special case of their more general Taylor approach:

Pattern Recognition 65 (2017) 211–222

Explaining nonlinear classification decisions with deep Taylor

decomposition

Grégoire Montavona,⁎, Sebastian Lapuschkinb, Alexander Binderc, Wojciech Samekb,⁎,

Klaus-Robert Müllera,d,⁎⁎

a Department of Electrical Engineering & Computer Science, Technische Universität Berlin, Marchstr. 23, Berlin 10587, Germany

b Department of Video Coding & Analytics, Fraunhofer Heinrich Hertz Institute, Einsteinufer 37, Berlin 10587, Germany

c Information Systems Technology & Design, Singapore University of Technology and Design, 8 Somapah Road, Building 1, Level 5, 487372, Singapore

d Department of Brain & Cognitive Engineering, Korea University, Anam-dong 5ga, Seongbuk-gu, Seoul 136-713, South Korea

Nonlinear methods such as Deep Neural Networks (DNNs) are the gold standard for various challenging machine learning problems such as image recognition. Although these methods perform impressively well, they have a significant disadvantage, the lack of transparency, limiting the interpretability of the solution and thus the scope of application in practice. Especially DNNs act as black boxes due to their multilayer nonlinear structure. In this paper we introduce a novel methodology for interpreting generic multilayer neural networks by decomposing the network classification decision into contributions of its input elements. Although our focus is on image classification, the method is applicable to a broad set of input data, learning tasks and network architectures. Our method called deep Taylor decomposition efficiently utilizes the structure of the network by backpropagating the explanations from the output to the input layer. We evaluate the proposed method empirically on the MNIST and ILSVRC data sets.

news summary (44)

Categories: Uncategorized