Archive

Archive for the ‘Uncategorized’ Category

Weekly QuEST Discussion Topics, 8 Dec

December 7, 2017 Leave a comment

QuEST 8 Dec 2017

Many of our team is focused on ACT3 – Autonomy Capability Team 3.  We will focus this week’s discussion on ‘Events’ – specifically how does QuEST envision event detection.  We will start by defining and explaining in the context of several technical vectors situations and events and then discuss their implications.

n  situation is any part of the agent centric internal representation which can be comprehended as a whole by that agent through defining how it interacts with or is related to other parts of the representation in that agent.

n  We will define comprehended by defining how it interacts or is related to other situations via linking (and types of links).

n  interacting with other things we mean that the situations have properties or relate to other situations.” *** we would say  can and must be linked to other ‘situations’  = ‘other qualila’ = other chunks***

Then we will expand the discussion to events.

Christoffersen, Woods, and Blike (2007) provide a concise, contemporary definition of events in their modified version of Newtson’s unit marking procedure (UMP) (See Newtson, 1973).

          They describe events as a meaningful pattern of change in the environment of an observer, a definition that is grounded in studies of ecological psychology (e.g. Gibson, 1979; Warren and Shaw, 1985; McCabe and Balzano, 1986).

Let’s talk a soccer match – what is an event – note how from a coaches perspective

talking to a defender – versus a fan’s perspective looking at ESPN highlights

Meaningful could be that it results in an action by the agent – that action could be the agent expects to communicate the occurrence of that event to another agent or could be that agent notes the occurrence because it is know to be part of the defining sequence for other events

  • From our Eventstream team:
  • In the context of the EventStream, the underpinning for the conceptualization of “events” is the Christoffersen/Woods description of an event as a “(meaningful) pattern of change over time”. This suggests a definition of events in the PED environment that would read something like: operationally meaningful interpretations of the raw FMV feed data.
  • Given this definition, examples of events for a Vehicle Follow would include:
  • the vehicle beginning or ending transit,
  • a person entering or exiting the vehicle,
  • the driver or passenger of the vehicle interacting with another person outside the vehicle, etc.
  • Thus, an event would be approximately equivalent to a single row in the Excel target log (in some cases).
  • We would just extend the definition to be change over more than time – space and modality … – and can be comprehended (defines how can interacts with and how is related to other parts of world)

An event is any part of the agent centric internal representation which is comprehended as a whole by that agent through experiencing how it is interacting with AND is related to other parts of the representation in that agent and the agent assesses the saliency of this event (which is a situation) may require action / or communication to another agent.

The vocabulary that is used to experience events are qualia since all event detection is the result of type 2 processes – since by our definition above a key aspect of the definition is they may require action/ communication to another agent – ex) this event caused the agent to do that action …, AND the event itself is experienced thus is a Quale itself

We’ve recently found that event based representations are often more valuable than map based – but the real issue is how to formulate information in a manner that together with the human analyst accomplish some mission (sometimes it is object based, sometimes it is activity based but in general we need an approach that can accommodate forms for this operator doing this task at this moment with this data) – overlays are common but we are actively attacking the issue of not just using overlays because although they accelerate a current approach to analysis we are seeking to find what approach (example event based representation / processing) provides greater mission capability – that may or may not be fusing diverse sources/types of information into a map display

Advertisements
Categories: Uncategorized

Weekly QuEST Discussion Topics, 16 Nov

November 15, 2017 Leave a comment

QuEST – 16 November 2017

 

We are honored to have John Launchbury (bio below) speak at QuEST on Thursday – we had exciting discussions with him on representations and our goals in QuEST.

 

 

Bio

Dr. John Launchbury rejoined Galois in September 2017 as the Chief Scientist focused on collaborating with government and industry leaders to fundamentally improve the security of cyber-physical systems. He also leads Galois’s involvement with industry partners looking to leverage applied formal mathematical techniques to make functional guarantees about the software their teams develop.

 

Prior to rejoining Galois in 2017, John was the director of the Information Innovation Office (I2O) at DARPA, where he led nation-scale investments in cryptography, cybersecurity for vehicles and other embedded systems, data privacy, and artificial intelligence.

 

Dr. Launchbury received first-class honors in mathematics from Oxford University in 1985, holds a Ph.D. in computing science from the University of Glasgow and won the British Computer Society’s distinguished dissertation prize. In 2010, Dr. Launchbury was inducted as a Fellow of the Association for Computing Machinery (ACM).

Categories: Uncategorized

Weekly QuEST Discussion Topics, 10 Nov

November 9, 2017 Leave a comment

QuEST 10 Nov 2017

We will have a meeting on Thursday at noon due to the holiday – and similarly next week due to travel commitments.

This week we have Prof Ox presenting some background material on Dr. Vapnik’s approach to statistical pattern recognition / regression. The reason we need to establish some basic understanding is we are investigating bring Dr. Vapnik on-site to work with us in enhancing our current solutions to building more flexible systems.

Categories: Uncategorized

Weekly QuEST Discussion Topics, 3 Nov

November 2, 2017 Leave a comment

A group of us have embarked upon a mission to build some QuEST agents for use in a range of applications (ISR, Air-to-Air Mission Effect chain, Business processes).  What we’ve noticed is people tripping over words (to be expected) but when you are attempting to actually code computers you have to clear up issue / confusion. What I mean is people who code AI solutions say one thing and people who have been in our discussions argue against the instantiation using our terminology concluding that the implementation misses the ‘magic’.  Keep in mind any sufficiently advanced technology will appear as magic (one of Arthur C. Clarke’s 3 adages / sometimes known as Clarke’s 3 laws).

What is our artificial conscious construct look like in a computer.  We will start with a reminder of the key defining characteristics of QuEST agents and specifically focus on the issue of how do you make / code the conscious representation.  What is new /different than all the other AI work going on?

The second topic is to focus on a specific application – captioning what is going on in a video:

  • Video captioning and semantic description is a research area that with the advent of deep learning has gained widespread interest in recent years.
  • Despite the increased number of publications and methods in previous years that address this problem, there is an increasing need for a thorough study and survey of recent methodologies and algorithms dedicated to this problem.
  • In order to mitigate such lack of information, we present a study of video captioning methods throughout recent years and identify current issues and trends of modern techniques focused in this area.
  • We also introduce a novel multiple decoder framework for automatic semantic description of label video sequences
Categories: Uncategorized

Weekly QuEST Discussion Topics and News, 27 Oct

October 26, 2017 Leave a comment

QuEST 27 Oct 2017

A group of us have embarked upon a mission to build some QuEST agents for use in a range of applications (ISR, Air-to-Air Mission Effect chain, Business processes).  What we’ve noticed is people tripping over words (to be expected) but when you are attempting to actually code computers you have to clear up issue / confusion. What I mean is people who code AI solutions say one thing and people who have been in our discussions argue against the instantiation using our terminology concluding that the implementation misses the ‘magic’.  Keep in mind any sufficiently advanced technology will appear as magic (one of Arthur C. Clarke’s 3 adages / sometimes known as Clarke’s 3 laws).

What is our artificial conscious construct look like in a computer.  We will start with a reminder of the key defining characteristics of QuEST agents and specifically focus on the issue of how do you make / code the conscious representation.  What is new /different than all the other AI work going on?

The second topic is to focus on a specific application – captioning what is going on in a video:

  • Video captioning and semantic description is a research area that with the advent of deep learning has gained widespread interest in recent years.
  • Despite the increased number of publications and methods in previous years that address this problem, there is an increasing need for a thorough study and survey of recent methodologies and algorithms dedicated to this problem.
  • In order to mitigate such lack of information, we present a study of video captioning methods throughout recent years and identify current issues and trends of modern techniques focused in this area.
  • We also introduce a novel multiple decoder framework for automatic semantic description of label video sequences

news summary (71)

 

Categories: Uncategorized

Weekly QuEST Discussion Topics and News, 13 Oct

October 12, 2017 Leave a comment

QuEST Friday 13th October

Some of us have been having a side-bar discussion on meaning – specifically related to the idea of ‘conversations as a platform’.  To expound on that issue we want to revisit some prior discussions on ‘big-data’.

For example:  Big Data – QuEST perspectives v11 short deck – AC3 inserts (AFRL conscious content curation).

  • From the IQT (In-Q-tel) quarterly (vol 7 no 2) fall 2015 issue – discusses “Artificial Intelligence gets Real”.
  • Predictions with Big Data By Devavrat Shah:

–     We know how to collect massive amounts of data (e.g., web scraping, social media, mobile phones),

–     how to store it efficiently to enable queries at scale (e.g., Hadoop File System, Cassandra) and

–     how to perform computation (analytics) at scale with it (e.g., Hadoop, MapReduce).

–     And we can sometimes visualize it (e.g., New York Times visualizations).

But from a QuEST perspective:

  • Current approaches to big-data bring extremely valuable insights – even in very large data sets with low information density
  • These approaches do so finding correlations
  • Most often they don’t attempt to answer questions on causation
  • QuEST seeks to deliver a simulation based deliberation approach (not correlation not causation)

–     degrees of freedom for simulation possibly chosen via ‘big-data’ infrastructure

  • Using the situated simulation consciousness provides an alternative to the issues above – you don’t have to have the experiences and been able to articulate a model to be able to understand causation – BUT – you also don’t have to have experience all of the data to be able to relate to prior data – the simulation approach provides something between or maybe outside of those – better than both?

The second topic follows this reasoning specifically with modeling relationships:

Modeling Relationships in Referential Expressions
with Compositional Modular Networks
Hu – UC Berkeley

  • People often refer to entities in an image in terms of their relationships with other entities.
  • For example, the black cat sitting under the table refers to both a black cat entity and its relationship with another table entity.
  • Understanding these relationships is essential for interpreting and grounding such natural language expressions.

Most prior work focuses on either grounding entire referential expressions holistically to one region, or localizing relationships based on a fixed set of categories

From our prior discussions on meaning:

  • Meaning, value and such like, are not intrinsic properties of things in the way that their mass or shape is.
  • They are relational properties.
  • Meaning is use, as Wittgenstein put it.
  • Meaning is not intrinsic, as Dennett has put it.
  • And here’s the point: if you know everything there is to know about that web, then you know everything there is to know about the data.

And the precursor to that work:

Neural Module Networks
Jacob Andreas Marcus Rohrbach Trevor Darrell Dan Klein
University of California, Berkeley
{jda,rohrbach,trevor,klein}@eecs.berkeley.edu

  • Visual question answering is fundamentally compositional in nature—a question like where is the dog? Shares substructure with questions like what color is the dog?And where is the cat?
  • This paper seeks to simultaneously exploit the representational capacity of deep networks and the compositional linguistic structure of questions.
  • We describe a procedure for constructing and learning neural module networks, which compose collections of jointly-trained neural “modules” into deep networks for question answering.
  • Our approach decomposes questions into their linguistic substructures, and uses these structures to dynamically instantiate modular networks (with reusable components for recognizing dogs, classifying colors, etc.). 
  • The resulting compound networks are jointly trained.
  • We evaluate our approach on two challenging datasets for visual question answering, achieving state-of-the-art results on both the VQA natural image dataset and a new dataset of complex questions about abstract shapes.

new summary

Categories: Uncategorized

Weekly QuEST Discussion Topics, 6 Oct

October 5, 2017 Leave a comment

QuEST 6 Oct 2017

We want to start this week by discussing the paper:

The Consciousness Prior
Yoshua Bengio
Université de Montréal, MILA
September 26, 2017

arXiv:1709.08568v1 [cs.LG] 25 Sep 2017

  • new prior is proposed for representation learning, which can be combined with other priors in order to help disentangling abstract factors from each other.
  • It is inspired by the phenomenon of consciousness seen as the formation of a low-dimensional combination of a few concepts constituting a conscious thought, i.e., consciousness as awareness at a particular time instant. ** very consistent with our position of qualia as the vocabulary of conscious thoughts – and that it is a lower dimensional representation versus the data space **
  • This provides a powerful constraint on the representation in that such low-dimensional thought vectors can correspond to statements about reality which are either true, highly probable, or very useful for taking decisions.  ** to get a stable consistent and useful representation is the objective **
  • The fact that a few elements of the current state can be combined into such a predictive or useful statement is a strong constraint and deviates considerably from the maximum likelihood approaches to modeling data and how states unfold in the future based on an agent’s actions.

Instead of making predictions in the sensory (e.g. pixel) space, the consciousness priorallow the agent to make predictions in the abstract space, with only a few dimensions of that space being involved in each of these predictions

  • The consciousness prior also makes it natural to map conscious states to natural language utterances or to express classical AI knowledge in the form of facts and rules, although the conscious states may be richer than what can be expressed easily in the form of a sentence, a fact or a rule.
Categories: Uncategorized