Archive for May, 2013

Weekly QUEST Discussion Topics and News, May 31st

We defined a measure of intelligence associated with a task to be the performance function of the agent (or set of agents) for that task (example finding cancer in a set of mammograms where you have defined a particular performance function like the sensitivity -> probability of detecting cancer when it is present in the woman)

We went to great lengths in the WCCI material to define performance functions / tasks as critical prior to any discussion on intelligence or intelligence amplification

What I am working on this week is an approach where we can emphasize the critical importance of ‘alignment’ to maximizing IA – the thought is to do that I have to define a task and as part of the performance function I have to define the manner in which the human and computer are ‘fused’ to provide an answer in the arm that has both – and how they can function independently in arms where they are not combined – that will give us intelligence estimates for what we called ‘h’ and what we called ‘c’ in the ‘ari gesher’ blog discussion

I can have a particular human do a set of mammograms and get an estimate of the performance – f(t) = h, I can have an algorithm like Second Look execute on that same set of mammograms and get an estimate of the performance –f(t) = c,

Using the notation of the Ari blog these are the a(h) and the a(c) –

Now to get an estimate of a(h,c) I have to constrain the manner the two agents combine their analysis to make a decision – one of my patents – 6,115,488 defines very specifically how to combine detections to maximize sensitivity – and for this discussion that is the performance function – f, in that patent all the detections of the human alone survive the ‘fusion’ – (thus maximizing the sensitivity and also acknowledging the purpose of CAD is NOT to prevent ‘work-ups’ of regions that concern doctors but the purpose of CAD is to help docs find regions that may have missed but are detectable by a machine CAD) – so the human looks at the output of the computer CAD and decides whether he/she wants to add additional locations of concern to their original list of regions based upon the CAD detections – NOTE in this fusion the performance function can NOT be reduced by the addition of the computer aide. Thus the overall intelligence of the human computer combination as measured by the sensitivity performance function is greater than or equal to the human alone intelligence! So in the Ari blog notation a(h,c) >= a(h).

So my query this week for QUEST is how can we make statements associated with friction / alignment and its impact on cancers that were overlooked by the human that upon re-examination based on the fact that the CAD marked the location will maximize IA. In the fusion rule I’ve patented above the answer seems obvious to me. If the CAD marks something that the human upon re-examination can NOT align to (it doesn’t look like cancer) then the human will not add it to the list of regions from their original review and thus it can NOT increase the sensitivity.

Note: we can argue that the maximum IA would be if the computer aid provides answers that the human can’t decipher using human perceptual powers it just requires in a ‘fusion’ rule that is different. Such combinations may in fact NOT want maximal alignment with human cognition – but they have to be trusted by the human as a result of performance – that was the point of the Prof Kurzweil position in our conversation with him last week.

Weekly QUEST Discussion Topics and News May 31


Weekly QUEST Discussion Topics and News, May 24th

Last week we began a discussion on attempting to understand the total analytic capability for a given task of a human-computer team. We want to revisit that topic this week trying to lock down a QUEST position on the ‘mixing function’. This is the term in the framework that accounts for the intelligence amplification of the team over either the human or the computer working alone. We want to examine the factors that impact the mixing. In some of the literature we have seen the term ‘friction’ and we have used the term ‘alignment’ to describe the impact of the mixing on the overall analytical power of the human-computer team. It is suggested that optimal analytical systems are built to maximize the ability of the human to leverage the computer. As part of the discussion we want to revisit our Plenary talk we gave at the World Congress on Computational Intelligence – ‘Computing machinery and Intelligence Amplification’ where we attempted to capture a snapshot of our thoughts on how we were measuring IA in our Breast Cancer Detection products. Dr. Oxley will lead the discussion on the framework we used in the WCCI chapter.

Weekly QUEST Discussion Topics and News May 24

Weekly QUEST Discussion Topics and News, May 17

This week we will have to cut our meeting to a half hour – 12-12:30 due to a call we have at 1 with the Google Director of Engineering.

The topics are:

1.) The notes for the Ray Kurzweil call. The AF is seeking an Investment strategy that will lead to Game Changing (10-100x) improvements in mission capabilities – our specific interests are in the mission areas of command and control of our forces and in the Intelligence / Surveillance / Reconnaissance missions, so we’re seeking Mr. Kurzweil’s opinion on what breakthroughs will occur and when they will occur and what he would recommend our investment strategy should be (lead investor, fast adaptor of commercially developed solutions, watch for breakthroughs) – as an example -what would allow us to break out of our current ‘drowning in data’ problems

2.) The second topic is a series of documents associated with the Palantir’s Director of Forward Deployed Engineering, Shyam Sankar’s talk on intelligence augmentation – in it he notes – the stories behind two classic encounters between man and machine: the 1997 match in which IBM’s Deep Blue supercomputer defeated chess grandmaster Garry Kasparov, and a 2005 freestyle tournament in which two amateur players using three weak laptops defeated all comers, including grandmasters armed with supercomputers… he also notes – the relevance and impact of Human-Computer Symbiosis today with the emergence of Big Data and related technologies. Responding quickly to victims of the Haiti earthquake, making sense of complex documents found in an Al-Qaeda house, designing the 9/11 Memorial—these are all tasks that are best tackled by a nimble mind and powerful technology working in concert… – he suggest the focus should be – By building software in such a way that it reduces the friction experienced at the boundaries between the computing power, the analyst, and the source data… with the dramatic walk-away conclusion point : Weak human + machine + better process was superior to a strong computer alone and, more remarkably, superior to a strong human + machine + inferior process.

Weekly QUEST Discussion Topics May 17

Weekly QUEST Discussion Topics and News, May 10th

A Framework / Theory for Artificial Consciousness:

A framework is not a detailed hypothesis or set of hypotheses; rather, it is a suggested point of view for an attack on a scientific problem, often suggesting testable hypotheses – we will discuss how defining characteristics of consciousness can provide the guidance to developing a mathematical framework for QUEST. We will focus on which of the tenets / fundamental laws we’ve discussed are critical to embody in QUEST solutions and thus the mathematical framework must support.

THE ULTIMATE GOAL of a theory of consciousness is a simple and elegant set of fundamental laws, analogous to the fundamental laws of physics.
• What might the underlying fundamental laws be?
• There is no reason they should not be strongly constrained to account accurately for our own first-person experiences, aswell as the evidence from subjects’ reports.
• If we find a theory that fits the data better than any other theory of equal simplicity, we will have good reason to accept it.
• If the theory leads to the engineering of systems that demonstrate an engineering advantage then that will be a success!

Weekly QUEST Discussion Topics and News May 10th