Home > Uncategorized > Weekly QuEST Discussion Topics and News, 23 June

Weekly QuEST Discussion Topics and News, 23 June

QuEST 23 June 2017


We will start this week by responding to any issues people want to discuss reference to our previous meeting topic – Can machines be conscious?

We then want to discuss the idea of the ‘knowledge centric’ view of making machines conscious.  What we mean by that is we define knowledge as what is being used by a system to generate meaning.  The current limiting factor in machine generated knowledge is the resulting meaning the machines make is not rich enough for understanding – we define understanding to be meaning associated with expected successful accomplishment of a task.  If we want to expand the tasks that our machine agents can be expected to acceptably solve we have to expand the richness of the meaning they generate and thus we have to increase the complexity of the knowledge they create.  What are the stages of increasing knowledge complexity that will lead to autonomy?  We want to brainstorm a sequence of advances that would lead to system of systems that demonstrate peer, task and cognitive flexibility.

That leads to consideration of how that knowledge is represented and the topic below:

The paper by Achille / Soatto UCLA, arXiv:1706.01350v1 [cs.LG] 5 Jun 2017

On the emergence of invariance and disentangling in deep representations

Lots of interesting analysis in this article but what caught my eye was the discussion on properties of representations:

  • In many applications, the observed data x is high dimensional (e.g., images or video), while the task y is low-dimensional, e.g., a label or a coarsely quantized location. ** what if the task was a simulation – that was stable, consistent and useful – low dimensional?**
  • For this reason, instead of working directly with x, we want to use a representation z that captures all the information the data x contains about the task y, while also being simpler than the data itself.
  • Ideally, such a representation should be
  • (a) sufficient for the task y, i.e. I(y; z) = I(y; x), so that information about y is not lostamong all sufficient representations, it should be
  • (b) minimal, i.e. I(z; x) is minimized, so that it retains as little about x as possible, simplifying the role of the classifier; finally, it should be

(c) invariant to the effect of nuisances I(z; n) = 0, so that decisions based on the representation z will not overfit to spurious correlations between nuisances n and labels y present in the training dataset

  • Assuming such a representation exists, it would not be unique, since any bijective function preserves all these properties.
  • We can use this fact to our advantage and further aim to make the representation (d) maximally disentangled, i.e., TC(z) is minimal.
  • This simplifies the classifier rule, since no information is present in the complicated higher-order correlations between the components of z, a.k.a. “features.”
  • In short, an ideal representation of the data is a minimal sufficient invariant representation that is disentangled.
  • Inferring a representation that satisfies all these properties may seem daunting. However, in this section we show that we only need to enforce (a) sufficiency and (b) minimality, from which invariance and disentanglement follow naturally.
  • Between this and the next section, we will then show that sufficiency and minimality of the learned representation can be promoted easily through implicit or explicit regularization during the training process.

As we mature our view of how to work to these rich representation it brings up the discussion point of QuEST as a platform:


I would like to think through a QuEST solution that is a platform that uses existing front ends (application dependent by observation vendors) and existing big-data back ends like systems that follow the standard Crisp-DM approach like Amazon Web services … , and possibly a series of knowledge creation vendors  –


Independent of the representation used by a front end system that captures the observables and provides them to the QuEST agent – it becomes the quest agent’s job to take them and create two uses for them – the first is put them in the form to be used by the big-data solution (structure them so they can be used in the CRISP-DM process to find if there exists experiences stored – something close enough to them to provide the appropriate response) and the second form has to be consistent with our situated / simulation tenets – so they are provided to a ‘simulation’ system that attempts to ‘constrain’ the simulation that will generate the artificially conscious ‘imagined’ present that can complement the ‘big-data’ response – in fact the simulated data might be fed as ‘imagined observables’ into the back end – I would like to expand on this discussion

news summary (59)

Categories: Uncategorized
  1. No comments yet.
  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: