***************************************************************************************************************************************************************************************************************************************************************************************

When I started grad school, one of the first things I did was to adopt a kitten that had been born earlier that summer in the apartment across the hall. Now, 18 years (!?!) later, here is a Eulogy for my Cat, in the form of a couple of cat-related anecdotes from my studies.

In my first year of school I was getting into the science of spatial vision - acuity, contrast, retina, V1, etc. I learned that a cat's visual acuity is like 5x worse than a human's. So, naturally, I figured that everything must look *blurry* to a cat.

See, I lived with this creature, and I would often think, how do things appear to her? I remember walking home late one night thinking about how stars appear to me as points - must they appear to a cat as blurry spots? And I realized that this couldn't be the case.

When I take off my glasses and see a point of light, it looks blurry in that I can see that what should be a point is actually smudged across a region of the visual field. And to see that smudge as a smudge, I must be seeing details within the region it fills.

If something appears as a point, that means that it has no apparent interior. So in fact, the acuity limit is telling us about the spatial resolution of appearance. Some visible thing that's smaller than the acuity limit must appear as a point. 

Without glasses, a point is imaged on my retina in a smudge that's a good fraction of a degree across. But my true visual acuity is in the range of 2 minutes of arc; so I see that smudge as an extended, 2d thing. My cat wouldn't see the smudge - she'd just see a point.

Now, years later I'm still making use of that insight, for example when I have argued that peripheral vision can and should look just as sharp as foveal vision - it's for the same reasons that a star should look like a point to my cat.

Another one, now about my perspective rather than the cat's. Some version of this is familiar to any of us: you see a dark shadow at night - is it the cat? or is it something else? There was one instance of this that I puzzled over for years.

I had a pair of shoes that I would sometimes leave by the wall in the hallway of my apartment in Boston - now we're in my first postdoc, 10 years ago. Time and again, I would see those shoes out of the corner of my eye and think, "Cat!" - then foveate and see, "shoes!'.

What fascinated me was that after the first time, I knew what was going on. That is, the high-level visual part of me knew they were shoes. But over and over, my early visual system was fooled - that brown shape of a certain size, on the floor, was most likely "Cat".

I started to collect these cases, where peripherally-seen X makes me think I'm seeing Y, even once I know X is really X. I posted an good case here a while back: (https://twitter.com/AndrewHaun3/status/1261891022625398784). But the prototype will always be Shoes->Cat.

Anyways, before I was so theoretically rigid as I am today, I used to wonder: when I thought the shoes were my Cat, did they actually *appear* as a Cat would appear? Did my mistaken recognition subtly reshape the spatial patterns so as to make them a better fit for "Cat"?

It does seem like this sort of thing can happen to some degree, e.g. with Ryota Kanai's "healing grid" illusion. But wholesale, at the object level? How could we find out? It doesn't matter, because I don't think that's what's happening here.

What I think is happening here, rather, is that I am experiencing a spatial form that is shoes-shaped, shoes-textured, etc. And it's decidedly not cat-shaped, cat-textured, etc - it's not clear cut pareidolia, as where I might see the Cat-shapedness of a Cat-shaped cloud. 

With the shoes, they're the right size and location for a Cat, and my super-sensitive Cat-recognizers are activated, and I experience recognition-of-Cat at the same time I experience a shoes-shaped, shoes-textured spatial gestalt. It's not a wholly congruent experience.

Even now, this week, when my Cat is no more, I've already made the mistake of thinking that a shadow in the hall was "Cat"; that a light *thump* in the other room was "Cat". Now, not only is it a mistake, it's an impossible mistake. But the brain does what it does.

***************************************************************************************************************************************************************************************************************************************************************************************

Procrastination

 I have reached peaks of procrastination that even I might once have thought were too great, too high. I've done it with the help of 'working from home', this monstrous situation that allows me to practice the piano, play video games, read nonsense, and any other disastrous activity, any time I like.

Back when I worked at the lab, in an office, I could only procrastinate for so long before I was cornered and had no choice but to do-the-thing. I couldn't play the piano or watch Netflix on a Tuesday afternoon in the lab. I might avoid work for a while by reading internet garbage, but eventually that runs out - it really does, it just takes a few days - and I have to do-the-thing.

But this...

Since the beginning of the pandemic, I've learned a Chopin nocturne - pretty well - laid groundwork for several Rachmaninoff preludes, memorized the 3rd movement of Beethoven's 'Moonlight' Sonata (biggest thing I've memorized in.. decades, probably?), and now I'm getting the hang of the awesome last movement of his first Sonata. I'm getting pretty good at improvising random lines over 9 chords! It's great! 

And it's worthless and stupid and dangerous, since I'm not a pianist and I'm not getting paid to play the piano. I have papers to write, finish, revise; experiments to plan; blah blah blah.

A point on interpreting inattentional blindness studies

On Inattentional Agnosia

In a recently published study, Cohen et al showed that, under very naturalistic conditions (viewing 3D natural scene videos in VR), observers often fail to notice that the entire periphery of the visual stimulus has been rendered colorless, i.e. completely desaturated. Cohen et al conclude that the visual periphery is far less colorful than one might have thought. They state this conclusion in several ways:

“these results demonstrate a surprising lack of awareness of peripheral color in everyday life”

-here qualifying the phenomenon as ‘lack of awareness of peripheral color’. Later, they say it less ambiguously: “If color perception in the real world is indeed as sparse as our findings suggest, the final question to consider is how this can be. Why does it intuitively feel like we see so much color when our data suggest we see so little?”.

So, Cohen et al believe that their data suggest we see very little color. This particular claim is logically absurd, however. I explain why in the following:

Cohen et al clearly believe that the phenomenon they present is a case of inattentional blindness. Inattentional blindness phenomena are frequently encountered in visual experience, and are not difficult to bring about in experimental scenarios. Typically, an observer is shown some stimuli connected to an explicit or implicit task; during the course of the task, some unexpected stimulus is inserted, and the observer may fail to notice. These failures to notice are often very retrospectively surprising, since once the observer knows what to look for (once they’ve been debriefed) it is easy to see the missed stimulus. Experimenters often conclude from these results – both the failures to notice and the retrospective surprise - that, in one way or another, the observers (and the rest of us) must see far less than they think they do. But this is not the kind of conclusion that Cohen et al are drawing.

The most famous example of an inattentional blindness phenomenon has got to be Simons and Chabris’ gorilla. In their experiment, observers watched a video of several people playing a ball-passing game. The players move around constantly, throwing the ball back and forth; the observer’s task is to count the number of times that certain players catch the ball. With no warning, halfway through the video, a person in a gorilla suit wanders into the middle of the ballgame, stops and waves at the camera, and then wanders back out of the frame – the ball game continues. Many observers do not notice the gorilla at all!

Simons and Chabris used this and similar results to advance a version of the “we see less than we think” argument. But what if we transplant Cohen et al’s conclusion to the gorilla experiment?

Here is a sentence in general terms that describes both studies:

“Observers viewed a complex stimulus that engaged their visual attention. After a while, a change was introduced to the stimulus that was retrospectively obvious. Many observers did not notice the change.”

Now, here is the reasonable, broad conclusion (a la Simons & Chabris):

“We must not notice as many things as we would expect based on what seems to be obviously noticeable.”

And here is the unreasonable, specific conclusion (a la Cohen et al):

“We must always be having the kinds of experiences evoked by the changed stimulus.”

In the Cohen et al study, they replaced a colorful scene with a colorless scene; observers didn’t notice; so, according to their reasoning, we must actually be having colorless experiences all the time (or more precisely, experiences of “so little” color). Otherwise, the reasoning seems to go, we would notice the change from colorful to colorless. We don’t notice it because it was colorless all along (they do include a caveat that maybe it’s the other way around, that even the grayscale scene evokes a colorful ‘filled-in’ experience, but that doesn’t seem to be their favored interpretation).

For the gorilla study, a gorilla was introduced incongruously into a ball game; observers didn’t notice; so, we must actually be having experiences of incongruous gorillas all the time (or, maybe more precisely, experiences of gorillas in ball games?). Otherwise, we would notice the change from a no-gorilla to an incongruous-gorilla scene. I don’t want to go on with this because it’s obviously absurd. But isn’t it the same logic as the color argument?

The absurdity comes in part from arguing from a complete lack of evidence: they are taking absence of evidence (failure to notice the change) to be evidence of absence (of color experiences). The experiments they are doing have no bearing, it seems to me, on whether or not their observers are actually experiencing color in their peripheral vision.

But more than this, the absurd conclusion comes from a lack of engagement with the important concepts at play. Color, most importantly. What does it mean to see color? That's for another time, I guess.

Before I finish here, an attempt at charity:

Perhaps the logic Cohen et al would derive from Simons & Chabris is somewhere in between the broad & reasonable, and the narrow and unreasonable:

"If we do not notice something, we are not experiencing it."

This is a strong claim, which I know that Cohen et al and many others would more-or-less endorse. But it does demand some engagement with some basic questions: if one's experience is colorless, what is it like? Is it like experience of a monochrome scene? Why are shades of gray excluded from 'color' status? What is special about 'chromatic hues'? Is there really less to seeing a monochrome scene than there is to seeing a color scene? Think about it: if you are seeing a spot as blue, that precludes your seeing it as any shade of gray, just as much as it precludes it from being yellow or red or whatever. Each part of the visual field always - it seems, at least - has some color in the broad sense.

In fact, Cohen et al did find that subjects always noticed if all color, in the broad sense, was removed from the periphery, i.e. if it were replaced with a flat gray field. Which would seem to defeat their basic conclusion that we are not seeing color, or much color. So, again, what is supposed to be so special about chromatic hues? Interesting questions, definitely.

A dialogue! (old but good)

I ask you to point your eyes up towards the clear sky and to tell me what you see. “An expanse of blue,” you say. I nod and say, “Ah, so simple. Blue is just one thing – your visual experience is so simple. Is that surprising?” 

This doesn’t sound right to you. You shake your head. “No, it’s not simple. There’s blue everywhere. Every location is blue; every part of the space is blue. It’s clearly not simple – it’s a vast structure of blue spots. I’m not even sure how to describe it to you.”

“Well,” I say, “you’re most likely confabulating this description. Through your life experience with using vision, you know that the sky is extended spatially, and that if you move your eyes around you’ll still see blue, and you know from moment-to-moment that the last thing you saw was what you’re seeing right now – simple blue – so you are illuded into claiming you see an expanse of blue. But you actually see no such thing.”

“How can you claim this?” you ask. “Why should you doubt what I tell you?”

I shake my head sadly. “Subjective reports are known for their fallibility. People often claim to have seen things that they could not have seen; they claim their experiences have qualities that they cannot have. But I’ll suspend my disbelief for a bit. Can you convince me?”

You seem a little annoyed, but you nod. “Perhaps.”

“Okay. How many parts are there to this blue expanse?” I ask. You don’t know. We go through some basic tests and it seems that you can’t really tell me about more than a handful of spots at once – yet you persist in claiming that the actual number of blue ‘spots’ is vast.

“Are they all there at once?” You seem to think that they are. “Could it be that the parts are there only when you look for them?”

“No,” you say, “it feels like a big, continuous expanse of blue. It’s not a little searchlight.”

I proceed. “But I’m asking you to convince me of that, not just to tell me again and again. As far as I can tell, you can only report the color of a few spots at a time – a big ‘sky-sized’ spot, or a few little ‘point-sized’ spots. But your momentary capacity seems to be extremely small – where is this huge expanse? And how does it make any sense that you should experience such enormous complexity, but be able to interact with only a vanishingly small portion of it?”

You seem unsettled: “Why,” you ask, “would I claim to see an expanse of spots when I only see a few at a time? What do I gain by confabulation?”

“But it’s a meaningful confabulation – you are unaware of the limits or boundaries between your momentary visual experience, your memories of recent experiences, and your expectations of what future experiences will be like. The reports you generate are more a confusion of these different processes, rather than a confabulation.” I concede a word, but little else.

“Well then,” you say slowly, “what does this confusion feel like? Might it feel like an expanse of blue? Or do you assume that only the perceptual process constitutes experience?”

I miss your point. “But we’re exactly talking about perceptual experience – of seeing blue – don’t try to shift the goal posts.”

“No, we aren’t talking strictly about perceptual experience, though I do think my experience is fairly categorized as perceptual rather than memorial or expectative. You’re the one who introduced other ‘processes’ into the conversation.”

“Well, this can’t work,” I say. “I can concede the process thing, but this doesn’t address the reportability issue at all, and it’s highly implausible, even worse than if you put everything in ‘perception’. Are you claiming that you experience all your memories or all your expectations, at once?”

“No no,” you dismiss the idea with a wave of your hand. “I was just asking – what do you think such confusion should feel like?”

“Like what you’re feeling now,” I say.

You roll your eyes. “Come on. Clearly the confusion you’re suggesting should feel very different from what I am feeling – or no, from what I claim to be feeling. Otherwise we wouldn’t be having this debate.”

“Well, I can’t say exactly. You are experiencing what you have access to, and you can report what you have access to; so your experience must be of a narrow set of blue spots. And you claim otherwise because whenever you check other spots, you immediately begin to experience them – so you mistakenly believe that they were there all along. Your experience isn’t what you think it is – and it isn’t what you claim.”

You seem perplexed. “Does that mean that unless I am queried about my experience, I am not under this illusion? I only become mistaken when asked to describe what I’m experiencing?”

“Maybe?”

You decide to change tack. “Okay. Can you tell me what substantive difference there is between this illusory or mistaken experience and an actual experience of a blue expanse?”

“Well, it would be a huge difference – the illusory experience is actually very limited and consists of very few parts, including the few blue spots and a particular set of expectations and memories that lead you to claim that you see an expanse of blue. The actual experience of a blue expanse would be just that – many many more spots, and no necessary memory or expectation aspects, though you’d probably also have those in addition.”

“I can’t help but think,” you say carefully, “that you’re doing something slippery here. You want to know why I claim to see a blue expanse, and your explanation has to do with these non-perceptual processes and how they seamlessly support my very limited perceptual process. And you reject my explanation for my claim – that I really am experiencing a blue expanse – because I can’t report the whole expanse to you. But can I report all my memories and expectations to you? Do you know how to collect such data?”

“I think that the fact you’re able to so quickly report on what you see at randomly cued locations suggests that those processes must be at work.”

“Surely they are, and I can tell you that I do indeed have experiences of memory and expectation. But I’m wondering why you think you must reject my explanation but are satisfied with your own.”

“Certainly I’m not satisfied – there is much to learn. We really understand perception etc rather poorly at this stage.”

You shake your head.

EHS

Falling asleep two nights ago, I realized I was hearing a terrible noise, a roaring, screaming cacophony; but then, when I realized it, I also realized I wasn't hearing anything at all. For a while, maybe tens of seconds or a few minutes, I had been lying there listening to this noise, with a vaguely uncomfortable feeling, mind drifting - falling asleep, in other words. On attending to it, I seem to have broken myself out of a hypnogogic auditory hallucination - the experience faded after a few seconds of my concentrating on the fact that, actually, I wasn't hearing anything at all, and as far as I know it didn't come back, and I fell asleep.

The next day I told Jonathan about it, and he reminded me of "Exploding head syndrome", which is usually described as a sharper, more acute noise ("snapping of the brain" was a term coined by a doctor in 1920), but I think what I experienced fits. Maybe it's happened before - probably, I think - and either I proceeded to sleep without the lucid break, or I've forgotten the details. It doesn't happen often, at least.

The experience was basically like auditory imagery, similar quality as hearing music or voices in your head; but whereas those kinds of experiences generally seem (to me) to be endogenous and even self-willed (even when it's a tune you can't get out of your head), this sound felt out of my control. It was frightening, but as I attended to it it was clearly a completely internal experience - less real than tinnitus, but in retrospect I think more than just a typical auditory image. I say this because now, when I try to imagine the sound, I can get a rough idea of it but I can't really experience it as really as I was during the episode; and when I imagine a piece of music, or a voice, these also seem less substantial.

At any rate, once I attended to the experience, and reassured myself that in fact I was hearing nothing and that it was a hallucination, the cacophony dissolved into the regular silence of auditory imagining. Maybe it was an aftereffect of the long holiday weekend? Exhaustion?

Parahypnic Hallucinosis?

Remember this episode:
http://xuexixs.blogspot.com/2018/08/blinking-plants.html

That was my bout with what I termed 'gardener's hallucinosis', where I spent all day pulling clear weed and etc from the gardens, and wound up with vivid blinking hallucinations. I likened it to the 'eyes open' geometric hallucinations I sometimes had back in my migraine days.

Well I've noticed a few times lately something similar happening under a specific circumstance. The circumstance is: I fall asleep in my daughter's room as I read her to sleep. I wake up a couple of hours later, stumble to the bathroom to brush my teeth, and I go get in my bed.

It doesn't happen every time, but sometimes - and last night very vividly - on waking in this way I have vivid and complex geometric hallucinations. Fine-grained, colorless - much of the content is just of very, very fine beads or dots, flickering and moving - but mixed into it are larger-scale features. Last night, the features were like a high-pass Kandinsky painting: discs and long, smoothly-curving lines, all moving and twisting around randomly, but no particular surface colors except for grayness, or darkness.

I could see it all fairly clearly with eyes opened, until I turned on the bathroom light and then the experience faded.

Maybe not coincidentally, I have been having minor headaches lately, and I think there was one yesterday. My brain must be in a state?

Two Things

Got two emails last night, one right after the other:

1. I get to give my 'color across the visual field' talk at ASSC 23. Gonna put my mouth where my money is! I wonder how they'll take it.

2. I was actually awarded a NEI travel award by the VSS! When's the last time I won something? Yeah, I don't know either!

Perfect Cadence

Just a few days ago, I realized something about my simple mind. It had been bubbling up for a few months, but suddenly it's crystallized:

Start your melody with the dominant tone on the upbeat, then down on the tonic ("5-1"), and I will love your melody and it will stick in my mind forever. Apparently.

I am a lifelong piano player, but I've only recently really started to get acquainted with the general classical piano repertoire. So while I've been playing everything by Erik Satie since I was a teenager, only in the last 5-6 years have I started to play a lot of Chopin, Scriabin, Grieg, etc. Only a 2-3 years ago did I first start to learn Chopin's etudes.

One of the etudes I played for a while, maybe the most beautiful one and one of the most famous, is the Op.10 no.3 in E-major. It starts on the 5, then to the 1:



That is a nice melody! Happy but a little sad, reflective, contemplative. Then this summer, I start looking for Schumann pieces to play, and of course I rediscover his "Traumerei" in F:



Very nice melody, easy to play, similar mood, meaning (in my mind) to the Chopin etude. And it also starts from the 5 to the 1. I notice this and think, how neat, two melodies I like that start the same way - and just a half-tone apart!

Then this summer, I start listening to Grieg's lyric pieces, and I get the book and they're all fun to play, mostly not too difficult. One of my favorites is one of the simplest, the "Watchman's song":

Now you see it again, again in E major: 5-1 to start the melody. And again, similar mood. Now I start to realize something funny is going on. I'm not just cherry-picking here: these are three of my favorite piano pieces to play or hum to myself in the hallways in the last year or so. I also realize that my favorite Grieg melody, "Solveig's Song" from Peer Gynt, starts the same way (5 up to 1 on the downbeat), though it is in A minor.

Then, well, I forget about it - and last week, not sure why, I'm thinking about it again. My favorite tune from Satie's Gnossienes: no 5, in G major.



Not just the first two notes - all four of these pieces (Chopin, Schumann, Grieg, Satie) have the same first three notes, and they're all in nearby keys (Emaj, Fmaj, Emaj, Gmaj).

Once I noticed the Gnossienne, I start finding the pattern everywhere. First, many other great classical melodies, though mostly in minor keys (the major ones apparently stand out to me), and I realize I can just sit and they're so easy to find in my brain:

Beethoven's "Marmotte"; Smetana's "Moldau"; Grieg's "Solveig's song"; Faure's Sicilienne, Scriabin's Prelude 14 (which I learned 10 years ago, one of the first of his that I learned); the 'song without words' from Holst's folk song Suite, which was always my favorite part of that tune when the band played it in high school. Those are all in minor keys, but notice this too: whereas the major key ones all go down from the tonic (to the major seven, each one), the minor key tunes all go up! And except for the Sicilienne, they all go up to the 2 (it goes up to the 3).

And what do you think has always been my favorite leitmotif from Star Wars? The "Force Theme", in C minor, which goes up from the tonic to the 2!

Then we can do folk songs and pop songs; I would have immediately have listed my childhood favorites as O Shenandoah (Major, but goes up from the tonic) and "My Grandfather's Clock" (Major, goes down to the 7 like the four initial examples). Everyone's favorite 'Simple Gifts' (the Shaker hymn used by Aaron Copland). My favorite Simon Garfunkel song: El Condor Pasa (Minor, down to the seven from the tonic); a pretty good Queen song, though not my favorite, is "Who wants to live forever", in a minor key going up to the 2 from the tonic.

My two favorite Tom Waits tunes: "Downtown train" (outside another yellow moon) and "If I have to go", though those are so similar it might be that the former is a rewrite of the latter. And what Mozart melody, that I would always have said was his most beautiful, starts in exactly the same way (same first 4 notes) as those Tom Waits melodies?

It doesn't start on the upbeat, but it's the theme from the second movement of his clarinet concerto:



***

Now, starting your melody by going from the dominant to the tonic doesn't guarantee I'll love it. I quickly thought of two examples I don't really care for - just two, while it was so easy to think of the two-dozen beloved ones above: I don't enjoy Schumann's "The Happy Farmer", maybe because my mother's a piano teacher and I've heard it way, way, way too many times in my childhood. And Wagner's wedding march ("Treulich geführt") is just clichéd to death. But then, speaking of Wagner, there is main theme of the Tannhauser overture, which I remember listening to over and over again, for its inspirational feeling, back 2-3 years ago. Easily my favorite Wagner theme, though I had not thought of it in a while (and I have never listened to the actual opera, so I don't know the source of the theme - wikipedia says it's from the "Pilgrim's Chorus").

In sum it seems that there is a key in my brain, shaped like "5-1", and it lets you right into my eternal musical memory.

Gardener's Hallucinosis?

Interesting experience yesterday, Sunday.

Spent ~5hrs outside in the gardens, pulling weeds. Had done the same for a few hours on Saturday. When I came inside for good, about 5 or 6, I started to hallucinate during blinks - when I blinked my eyes, especially when not prepared for it, I would see images of the plants I had been pulling all day. Sometimes very clear, seeing leaves with their serrations and textures, and tendrils curling around - the images were coherent and (mutedly) colorful, seemingly randomly selected but each was a recognizable one of the real plants I had seen, mostly members of the 2 or 3 most common weeds of the day.

Sometimes the images were strong enough to be distracting, making it hard to see - or to recognize - what was actually before my eyes. But I think they were only actually visible during the blinks. I managed over time to notice some properties of the images - I could hold my eyes closed after an effective blink. It was still unclear to what extent I was *really* seeing the fine details, or whether the actual images were coarser and just 'suggesting', as in normal visual imagery, the fine details. Holding my eyes shut, it seemed that the form of the afterimages or noise, in the eyes-shut darkness, guided the structure of the hallucination: spots of afterimage seemed to appear as leaves, streaks as stems or tendrils. But it was not so clear as to be certain of this.

The experience lasted until I went to bed, 6 or 7 hours later, but it had attenuated by then. I slept and remembered several dreams that had nothing at all to do with plants (one I remember, now, was that my lab seemed to be based in the house I grew up in, and some newcomers were using space in the den - dream-Giulio warned me not to give them to much space, or they'll think they can take more). When I woke up this morning, the phenomenon returned for another hour or so, but is gone now.

The phenomenon resembled, to me, the kinds of hypnogogic hallucinations some people have after long, repeated activity ('the Tetris effect'), but I can't find reports of this in normal waking experience (albeit during blinks). I described it to Giulio and others, the physiological explanations are kind of clear, but as to why it happened to me and why it isn't much more common, that's an open, strange question.

Another thing it reminded me of, was back in the migraine days, seeing geometric web patterns after waking, sometimes during blinks. Similar kind of dynamic, but I don't think those experiences ever lasted more than minutes, definitely not many hours.

IIT and Star Trek

[I originally posted this to a Star Trek forum because I am a Star Trek nerd but here it is for better posterity:]

"Complex systems can sometimes behave in ways that are entirely unpredictable. The Human brain for example, might be described in terms of cellular functions and neurochemical interactions, but that description does not explain human consciousness, a capacity that far exceeds simple neural functions. Consciousness is an emergent property.” - Lt.Cmdr. Data

A post the other day about strong AI in ST provoked me to think about one of my pet theories, there in the title: Data is conscious, the Doctor is not, and other cases can be inferred from there. Sorry that this is super long, but if you guys don't read it I don't know who will, and my procrastination needs an outlet.

First, some definitions. Consciousness is a famously misunderstood term, defined differently from many different perspectives; my perspective is that of a psychologist/neuroscientist (because that is what I am), and I would define consciousness to mean “subjective phenomenal experience”. That is, if X is conscious, then there is “something it is like to be X”.

There are several other properties that often get mixed up with consciousness as I have defined it. Three, in particular, are important for the current topic: cognition, intelligence, and autonomy. This is a bit involved, but it’s necessary to set the scene (just wait, we’ll get to Data and the Doctor eventually):

Cognition is a functional concept, i.e. it is a particular suite of things that an information processing system does, specifically it is the type of generalized information processing that an intelligent autonomous organism does. Thinking, perceiving, planning, etc, all fall under the broad rubric of “cognition”. Humans are considered to have complex cognition, and they are conscious, and those two things tend to be strongly associated (your cognition ‘goes away’ when you lose consciousness, and so long as you are conscious, you seem to usually be ‘cognizing’ about things). But it is well known that there is unconscious cognition (for example, you are completely unaware of how you plan your movements through a room, or how your visual system binds an object and presents it as against a background, how you understand language, or how you retrieve memories, etc) - and some theorists even argue that cognition is entirely unconscious, and we experience only the superficial perceptual qualities that are evoked by cognitive mechanisms (I am not sure about that). We might just summarize cognition as “animal-style information processing”, which is categorically different from “what it’s like to be an animal”.

Intelligence is another property that might get mixed up with consciousness; it is generally considered, rather crudely, as “how well” some information processing system handles a natural task. While cognition is a qualitative property, intelligence is more quantitative. If a system handles a ‘cognitive' task better, it is more intelligent, regardless of how it achieved the result. Conceiving of intelligence in this way, we understand why intelligence tests usually measure multiple factors: an agent might be intelligent (or unintelligent) in many different ways, depending on just what kinds of demands are being assessed. “Strong AI” is the term usually used to refer to a system that has a general kind of intelligence that is of a level comparable to human intelligence - it can do what a human mind can do, about as well (or better). No such thing exists in our time, but there is little doubt that such systems will eventually be constructed. Just like with cognition, there is an obvious association between consciousness and intelligence - your intelligence ‘goes away’ when you lose consciousness, etc. But it seems problematic to suppose that someone who is more intelligent is more conscious (does their experience consist of “more qualia”? What exactly does it have more of, then?), and more likely that they are simply better-able to do certain types of tasks. And it is clear, to me at least, that conscious experience is possible in the absence of intelligent behavior: I might just lie down and stare at the sky, meditating with a clear mind - I’m not “doing” anything at all, making my intelligence irrelevant, but I’m still conscious.

Autonomy is the third property that might get mixed up with consciousness. We see a creature moving around in the environment, navigating obstacles, making choices, and we are inclined to see it as having a sort of inner life - until we learn that, no, it was remote-controlled all along, and then that apparent inner life vanishes. If a system makes its own decisions, if it is autonomous, then it has, for human observers at least, an intrinsic animacy (this tendency is the ultimate basis of many human religious practices), and many would identify this with consciousness. But this is clearly just an observer bias: we humans are autonomous, and we assume that we are all conscious (I am; you are like me in basic ways, so I assume you are too), and so we conflate autonomy with consciousness. But, again, we can conceive of counter-examples - a patient with locked-in syndrome has no autonomy, but they retain their consciousness; and an airplane on autopilot has real (if limited) autonomy, but maybe it’s really just a complex Kalman filter in action, and why should a Kalman filter be conscious (i.e. “autonomy as consciousness” just results in an endless regression of burden-shifting - it doesn’t explain anything)?

To reiterate, consciousness is “something it’s like to be” something - there’s something-it’s-like-to-be me, for example, and likewise for you. We can turn this property around and query objects in nature, and then it gets hard, and we come to our current problem (i.e. Data and the Doctor). Is there something-it’s-like-to-be a rock? Certainly not. A cabbage? Probably not. Your digestive system? Maybe, but probably not. A cat? Almost certainly. Another human? Definitely. An autonomous, intelligent, android with human-style cognition? Hmmm… What if it’s a hologram? Hmmm….

That list I just gave (rock; cabbage; etc) was an intuition pump: most of us will agree that a rock, or a cabbage, has no such thing as phenomenal consciousness; most of us will agree that animals and other humanoids do have such a thing. What makes an animal different from a rock? The answer is obvious: animals have brains. Natural science makes clear that human consciousness (as well as intelligence, etc) relies on the brain. Does this mean that there’s something special about neurons, or synapses, or neurotransmitters? Probably not, or, at least there’s no reason to suppose that those are the magic factors (The 24th century would agree with this; see Data’s quote at the top of this essay). Instead, neuroscientists believe that consciousness is a consequence of “the way the brain is put together”, i.e. the way its components are interconnected. This interconnection allows for dynamically flexible information processing, which gives the overt properties we have listed, but it also somehow permits existence of a subjective point of view - the conscious experience. Rocks and cabbages have no such system of dynamical interconnections, so they’re clearly out. Brains seem to be special in this regard: they are big masses of complex dynamical interconnection, and so they are conscious.

What I’m describing here is, roughly, something called the “dynamic core hypothesis”, which leads into my favored theory of consciousness: “integrated information theory”. You can read about these here:http://www.scholarpedia.org/article/Models_of_consciousness The upshot of these theories is that consciousness arises in a system that is densely interconnected with itself. It is important to note here that computer systems do not have this property - a computer ultimately is largely a feed-forward system, with its feedback channels limited to long courses through its architecture, so that any particular component is strictly feed-foward. A brain, by contrast, is “feedback everywhere” - if a neuron gets inputs from some other neurons, then it is almost certainly sending inputs back their way, and this recurrent architecture seems implemented at just about every scale. It’s not until you get to sensorimotor channels (like the optic nerves, or the spinal cord) that you find mostly-feed-forward structures in the brain, which explains why consciousness doesn’t depend on the peripheral nervous system (it’s ‘inputs and outputs’). Anyways, this kind of densely interconnected structure is hypothesized to be the basis of conscious experience; the fact that the structure also ‘processes information’ means that such systems will also be intelligent, etc, but these capacities are orthogonal to the actual structure of the system’s implementation.

So, Data. Maybe Data isn’t conscious, but just gives a great impression of a conscious being: he’s autonomous, he’s intelligent, he has a sophisticated cognitive apparatus. Maybe there’s nothing “inside” - ultimately, he’s just a bunch of software running on a robotic computer platform. People treat him like he’s conscious (Maddox excepted) just because of his convincing appearance and behavior. But I don’t think it’s an illusion - I think Data is indeed conscious. 

Data’s “positronic brain” is, in a sense, a computer; it’s artificial and made from artificial materials, it’s rated in operations per second, it easily interfaces with other more familiar kinds of computers. But these are really superficial properties, and Data’s brain is different from a computer in the ways that really matter. It is specifically designed to mimic the structure of a human brain; there are numerous references throughout TNG that suggest that Data’s brain consisted critically of a massive network of interconnected fibers or filaments, intentionally comparable to the interconnected neurons of a biological brain (data often refers to these structures as his “neural nets”). This is in contrast to the ‘isolinear chip-bound’ architecture of the Enterprise computer. Chips are complicated internally - presumably each one acts as a special module that is expert in some type of information processing task - but they must have a narrow array of input and output contacts, severely limiting the extent to which a chip can function as a unit in a recurrently connected network (a neuron in a brain is the opposite: internally it is simple, taking on just a few states like “firing” or “not firing”, but it makes tens of thousands of connections on both input and output sides, with other neurons). The computer on 1701-D seems, for all intents and purposes, like a huge motherboard with a ton of stuff plugged into it (we can get to the Intrepid class and its ‘bio-neural chips’ in just a bit).

Data, then, is conscious in virtue of his densely recurrently interconnected brain, which was exactly the intention of Dr Soong in constructing him – Soong didn’t want to create a simulation, he wanted to create a new being. I contrast Data at first with the Enterprise computer, which is clearly highly intelligent and capable of some degree of autonomy (as much as the captain will give it, if you believe Picard in 'Remember Me’). I won’t surmise anything about “ship cognition”, however. Now, if the ship’s computer walked around the ship in a humanoid body (a la EDI of the Mass Effect series), we might be more inclined to see a ghost in the machine, but because of the ship’s relatively compartmentalized ‘chip focused’ structure and it’s lack of a friendly face, I think it’s very easy to suppose that the computer is not conscious. But holographic programs running on that very same computer start to pull at our heartstrings - Moriarty, Minuet, but especially… the Doctor.

The Doctor is my favorite Voyager character (and Data is my favorite of TNG), because his nature is just so curious. Obviously the hologram “itself” is not conscious - it’s just a pattern of projected photons. The Doctor’s mind, such as it is, is in the ship’s medbay computer (or at times, we must assume, his mobile emitter) - he’s something of an instantiation of the famous ‘brain in a vat’ thought experiment, body in one place, mind in another. The Doctor himself admits that he is designed to simulate human behavior. The Voyager crew at first treats him impersonally, like a piece of technology - as though they do not believe he is “really there”, i.e. not conscious - but over time they warm to his character and he becomes something of an equal. I think, however, that the crew was ultimately mistaken as to the Doctor’s nature - he was autonomous, intelligent, and a fine simulation of human cognition and personality, but he was most likely not a conscious being (though he may have claimed that he was).

Over and over, we hear the Doctor refer to himself as a program, and he references movement of his program from one place to another; his program is labile and easily changed. This suggests that his mind, at any given moment, is not physically instantiated in a substrate. What I mean by this is that while a human mind (or Soong-type android mind) is immediately instantiated in a pattern of activity across trillions of synapses between physically-realized interconnected elements, the Doctor’s mind is not. His mind is a program stored in an array of memory buffers, cycling through a system of central processors – at any given moment, the Doctor’s mind just just those few bits that are flowing through a data bus between processor and memory (or input/output channel). The “rest of him”, so to speak, is inert, sitting in memory, waiting to flow through the processor. In other words, he is a simulation. Now, to be sure, in a lot of science fiction brains are treated as computers, as though they are programmable, downloadable or uploadable, but in general this is a very flawed perspective - brains and computers actually have very little in common. The Star Trek universe seems to recognize this, as I can’t think of any instances of outright abuse of this trope in a ST show. One important exception stands out: Ira Graves.

Ira Graves is a great cyberneticist, so let’s assume he knows his stuff (let’s forget about Maddox, who was a theoretically impoverished engineer). He believes that he can save his consciousness by moving it into Data’s brain. But Data’s brain is not a computer in any ordinary sense, as we detailed above: it’s a complex of interconnected elements made to emulate the physical structure of a human brain. (This is why his brain is such an incredible achievement: Data’s brain isn’t a miniaturized computer, it’s something unique and extraordinarily complex. This is why Lal couldn’t just be saved onto a disc for another attempt later on - Data impressed himself with her memories, but her consciousness died with her brain.) Anyways, Ira Graves somehow impresses his own brain structure into Data’s positronic brain - apparently killing himself in the process - and seems happy with the result (though he could be deluded - having lost his consciousness, but failing to recognize it). In the end, he relinquishes Data’s brain back to Data’s own mind (apparently suppressed but not sufficiently to oblieterate it), and downloads his knowledge into the Enterprise computer. Data believes, however, that Graves’ consciousness must have been lost in this maneuver, which is further support for the notion that a conscious mind cannot “run on a computer”: a human consciousness can exist in Data’s brain, but not on a network of isolinear chips.

The Doctor, in the end, is in the same situation. As a simulation of a human being, he has no inner life – although he is programmed at his core to behave as though he does. He will claim to be conscious because this makes his humanity, and thus his bedside manner, more effective and convincing. And he may autonomously believe that he is conscious – but, not being conscious, he could never know the difference, and so he cannot know if he’s making an error or not in this belief.

I think that here we can quickly bring up the bio-neural gel packs on Voyager. Aren’t they ‘brainlike’ in their constitution? If the Doctor’s program runs on this substrate, doesn’t that make him conscious? The answer is no – first, recall what Data had to say about neural function and biochemistry. Those aren’t the important factors – it’s the dense interconnectedness that instantiates an immediate conscious experience, and we have no reason to believe that the interconnection patterns of an array of bio-neural gel packs is fundamentally different from a network of isolinear chips. Bio-neural thingies are just supposed to be faster somehow, and implement ‘fuzzy logic’, but no one suggests they can serve as a substrate for conscious programs. And furthermore, the Doctor seems happy to move onto his mobile emitter, whose technology is mysterious, but certainly different from the gel packs. It seems that he is just a piece of software, and that he never really has any physical instantiation anywhere. In defense of his “sentience” (Voyager episode ‘Author, Author’), the Doctor’s crewmates only describe his behavioral capacities: he’s kind, he’s autonomous, he’s creative. No one offers any evidence that he actually possesses anything like phenomenal consciousness. (In the analogous scene in ‘Measure of a Man’, Picard at least waves his hand at the notion that, well, you can’t prove Data isn’tconscious, which I thought was pretty weak, but I guess it worked. I don’t know why they didn’t at least have a cyberneuroscientist or something testify.)

So that is my case: Data is conscious, and the Doctor is not. It’s a bit tragic, I think, to see the Doctor in this way – he’s an empty vessel, reacting to his situation and engendering real empathy in those he interacts with, but he has no pathos of his own. He becomes an ironically pathetic character – we feel for him, but he has no feelings. Data, meanwhile, in his misguided quest to become more human and gain access to emotional states (side note: emotion chip BLECH) is far more human, more real, than the most convincing holographic simulation can ever be.

IIT and Blade Runner 2049

Blade Runner 2049 is probably the best movie I've ever seen in a theater - definitely the best sci fi movie I've seen. The movie is a detective story about a replicant - an artificial human - who uncovers a mystery that has personal implications for himself, and broader implications for the dystopia that he lives in.

Are replicants conscious? It's hard to argue that they wouldn't be, and the movie doesn't seem to suggest they aren't. Instead, the movie focuses on memory and how your memories make you real or not - if your memories are false, you are a kind of false person in the world of 2049, and this is how people in the movie justify their enslavement of the replicants. The main theme of the movie is memory - are my memories real? If they're real, are they really mine, or someone else's? Does it really matter?

That stuff is all interesting, but like I said, consciousness is not the question with the replicants. It is the core question regarding one of the main characters: Joi the holographic girlfriend. We can speculate now on whether or not Joi is conscious. The movie is ambiguous about this, but there seems to be a subtext that she is not conscious, but that the main character (K), and we the audience, are supposed to believe that she is. And while there is this ambiguity, the resolution of the ambiguity is deeply meaningful to the story (just like the original Blade Runner, such a significant ambiguity is left unresolved).

First, to be clear on IIT terms: replicants are conscious because they are basically humans with human brains (and humans are clearly conscious) - what makes replicants different is that they are constructed as adults, with memories implanted (or not) to give them a more natural psychology. In the original Blade Runner, replicants like Roy Batty were assumed by their masters to be essentially psychopathic by nature, and the subsequent implantation of false memories was instituted to make them more psychologically healthy. But for a 'system in a state', the truth or falsity of memory is an extrinsic fact - from the IIT point of view (and probably any other modern theory of consciousness, or of the brain) it doesn't matter for the system itself. So replicants are conscious.

Joi, on the other hand, is not a human with a human brain. She's a holographic projection generated by a computer. The hologram of course is nothing but an image; what matters is in the computer. Computers as we know them cannot be conscious in any meaningful sense: the system is a very small set of very very fast switches, entirely feedforward at the most complex, finest grain, and extremely simple at coarser grains where we might see something like feedback or lateral connections. If computers in 2049 are like computers we know, Joi is not conscious - but computers might be very different. Joi has some kind of dedicated local unit, mounted on the wall in K's apartment - perhaps the computer in that unit is a neuromorphic system that replicates the connectivity structure of a human brain. But the picture of technology in the movie doesn't suggest this level of sophistication - I think that if we want to argue that Joi is conscious (in order to counter-argue) we need to weaken some assumptions.

Maybe Joi is conscious, but her consciousness is absolutely different from a human consciousness. That still requires some kind of neuromorphic computer, but it doesn't have to reflect the structure of the human brain, but there's a problem there that boils down to unreliability: if you want a simulation of a human being, you probably want something that's utterly controllable, like a performance - and where it's uncontrollable, it should still fulfill the simulation's desiderata. But consciousness is exactly uncontrollable - it's a closed locus of causal power (according to IIT) - so if your machine is conscious and you want it to simulate a human being, then its consciousness ought to resemble a human consciousness. Joi seems to do a very lifelike impression of a human being, so we have two choices - either she is conscious and her consciousness is specified by a neuromorphic computer that reproduces human neural connectivity, or she is an unconscious simulation.

As I said above, an artificial human brain (in the sense of an electronic device) seems beyond the technology of 2049; but even if it is in reach, it is hard to reconcile with the way Joi is quickly cut and copied over to her mobile emitter. First, it would mean that not only are there artificial human brains in 2049, but they are tiny enough to fit in the palm of your hand; second, it would mean that this brain can be constructed (or connected) in seconds, since remember that according to IIT it is a causally-interacting physical substrate that specifies consciousness - a computer program stored in memory is not causally-interacting in any important sense. I just don't think either of these is plausible in context.

So that leaves us with Joi the unconscious, but highly convincing, holographic girlfriend. Seeing Joi this way is easy when she first appears in the movie, but rather quickly it becomes clear that she is a dramatic and interesting character. Just like any other character in a movie or a play, it is then very difficult to imagine that she is not conscious. We know that the actor playing her is conscious, which makes it even more difficult. But if we try, we can see her as an entirely mechanical projection, like Siri or a chatbot, something that emulates humanity down to little details like evincing emotions like love and hope and excitement, and insistence on her own choices. But evincing emotion is not the same as feeling emotion - while there was an actor (Ana de Armas) that performed the character, there is nothing there on the screen while we watch the movie, and whether or not the actor was ever even real (or whether the performance is entirely artificial) doesn't matter to the fact that the performance on the screen is just a mechanical, unconscious projection. K the Blade Runner is in a similar situation: Joi is convincing, and maybe K himself cannot recognize that she is not intrinsically real, but she is nonetheless unreal.

This seems to me to contribute important meaning to the story, and it does resonate with some clues that are given out bit-by-bit, i.e. it seems the filmmakers probably also thought that Joi is not really real (not conscious). We see ads in the background of 2049 LA for Joi, touting that she is everything you want, and this is exactly what K seems to get. And K, who seems to despair at losing her (and losing other things), seems to recognize this (or remember it) when one of those ads reaches out to him and calls him by the same name that 'his' Joi had given to him. His copy of Joi wasn't even customized (it had all the 'factory settings'), i.e. not only is she unreal, she isn't even unique. So as a conscious being himself, K really is alone - his only companion is just a performance without an actor.

I think that, then, we're left with a hard question that aligns with the main theme of the story, which is (as I understood it) does it matter if my memories are mine, or if they are real? If none of my memories are real, am I real, do I matter? That theme gets a resolution: it is clear that K is real, he matters in important ways, and that his memories, real or not, nevertheless guide him significantly. The hard question is: does it matter if you are real? I mean that from the first person: I know I am real, but are you? Does it matter if you are or not, at least, does it matter to me? Well.. in a sense it is the same problem as the main theme - you are something I perceive, just as you might be something I remember. What matters, we might like to think, is whether or not I am real, whether or not I make my own choices of significance - and as pertains to you, whether or not you (real or not) have a significant role in my reality.

And, I think, K is left in a similar place with both versions of this problem; Joi had an effect on him, it seems clear to me, encouraging him and helping to destabilize him towards his ultimate fate, just as his memories did. But whereas (his) Joi is destroyed and lost to him, his memories - "all the best ones", at least - survive even when he dies, because they also belong to others. This happens to be an inversion of Roy Batty's famous observation that his memories will be lost with him.

Ok, enough!

IIT & Pacific Rim

I'm going to start posting short observations of how IIT would explain or be problematic for certain ideas in sci-fi movies or books.

To start: The film "Pacific Rim", a sci-fi action movie where the main characters are pilots controlling gigantic robots. The pilots control the robot through a direct brain-machine interface, but the job is apparently too much for one pilot so there are always at least two pilots. The two pilots have their minds joined by a "neural bridge" - basically an artificial corpus callosum. While joined, the pilots seem to have direct access to one another's experiences in a merged state called "the Drift" - it seems that their two consciousnesses become one.

This scenario is the predicted consequence, according to IIT, of sufficient causal linkage between two brains - at some point, the connection is sufficiently complex that the local maximum of integrated information is no longer within each pilot's brain, but now extends over both brains. What would be necessary to achieve this? The movie doesn't attempt to explain how the brain-machine interface works, but it must involve a very high-resolution, high-speed parallel system for both responding to and stimulating neurons in each pilot's brain.

One way of doing this would be cortical implants, where high-resolution electrode arrays are installed on the surface of each pilot's brain; this is at least plausible (if not possible) given existing technology. However, none of the pilots show signs of a brain implant, and the main character Mako Mori seems to become a pilot on pretty short notice, although she has apparently been training for a long time - maybe all trainees are implanted? A big commitment.

A more hand-wavy Star Trek kind of technology would involve some kind of transcranial magnetic field system that is powerful, precise, and fast enough to both stimulate individual neurons (current TMS systems certainly cannot do this) and measure their activity on a millisecond timescale (current fMRI systems absolutely cannot do this); however, the pilots simply wear helmets while piloting the robots (although Dr Newton, who almost certainly does not have any brain implants since he is not a trained pilot, does use some kind of transcranial setup to drift with a piece of monster brain), which I think makes a transcranial system very unlikely.

If I had to guess, wireless cortical implants are the only plausible means of establishing the Pacific Rim neural bridge, but some sort of transcranial system hidden in the pilots' helmets and based on some unimaginable technology is not excluded.

Verdict: Pacific Rim's "drift" is IIT Compatible

challenge

Irwin Borish versus Edwin Boring.

Go!

crastinus

Just need to get that 'idiot' post off the top. How's it going?

Lots of news!

Not going to tell you any of it, though. Wait till later.

-andrew

sports

The above is a 3D multidimensional scaling representation of the similarity of 14 popular sports. At least 3 dimensions are necessary to represent the similarity matrix; here I've rotated the plot so as to make the different points (sports) clear.

Euclidean distance between the sports is closely related to their similarity, but since this is a 2D projection of three coordinates, I also included edges (lines) between the sports as an extra cue to distance. The thickness and coloring of the edges is mapped to the similarity values (red/thin is "dissimilar", while blue/thick is "similar"). For each sport edges were drawn to the three nearest neighbors - sometimes those were very similar (e.g. the hockey triangle), sometimes very distant (baseball is very close to cricket, but it's next nearest neighbors are very far away: golf and ping pong).

Okay, so what constitutes "similarity" here? I made a list of 28 "sports properties" based entirely on my own subjective knowledge of the included sports. Undoubtedly a better list could be composed, but this is an okay first approximation. The set of properties works as a "present or not" binary value for each sport.

IIT and Sci-Fi

(Fake date on this post - wrote it in 2015, not sure of the date - back-posting it here)

This isn't a coherent post - just a list of some random thoughts having to do with IIT and science fiction scenarios that I'm familiar with. I can't claim to have any strength in the latter category: I'm not a big sci-fi reader (especially not in the last few years), and even the best sci-fi movies (I watch 20 times more movies than I read novels) don't tend to take you very far (excepting good time travel movies).

I just think that some of these examples (or others that you, dear reader, might think of) could help in illustrating some interesting IIT-based possibilities.

1. The book Foundation's Edge, by Isaac Asimov, ends on a question about a "supermind", a world where all life - human and otherwise - shares a single planetary consciousness. But it's something earlier in the book that has stuck with me; when the main character, Golan Trevize, is exiled from planet Terminus, he is given an advanced starship. Trevize is confused by the lack of traditional controls - instead, he learns that the ship is piloted by connecting a user's mind to the ship itself. In doing so, Trevize finds that he is now aware of not only the ship's control system, but his perceptual awareness now seems to include the space surrounding the ship: his consciousness now includes the ship:

"And as he and the computer held hands, their thinking merged and it no longer mattered whether his eyes were open or closed. Opening them did not improve his vision nor did closing them dim it. 

Either way, he saw the room with complete clarity—not just in the direction in which he was looking, but all around and above and below. 

He saw every room in the spaceship and he saw outside as well. The sun had risen and its brightness was dimmed in the morning mist, but he could look at it directly without being dazzled, for the computer automatically filtered the light waves. 

He felt the gentle wind and its temperature, and the sounds of the world about him. He detected the planet's magnetic field and the tiny electrical charges on the wall of the ship. 

He became aware of the controls of the ship, without even knowing what they were in detail. He knew only that if he wanted to lift the ship, or turn it, or accelerate it, or make use of any of its abilities, the process was the same as that of performing the analogous process to his body. He had but to use his will."

In IIT terms, the connection between his mind and the ship's systems has been made that allows states of the ship's electronics to causally constrain states of Trevize's brain - at the same time that states of his brain constrain those electronics. There is an irreducible global maximum in integrated information that extends outside Trevize's skull, and into the ship's computers.

Asimov doesn't elaborate on what it is "like" for the ship without a pilot - does it integrate information on its own, constituting a waiting ship's consciousness, ready always to link up with the pilot? If so, the link is like an artificial corpus callosum, joining two independent minds into one whole, and we can suppose that the ship persists as a conscious agent even in the absence of a pilot. Or maybe the ship is disintegrated without a human component - in a state something like slow-wave sleep, with systems online but carefully disconnected in such a way that without a pilot, there is no "ship" - just a collection of parts and mechanisms, like an ordinary machine.

2. Speaking of artificial corpus callosums, there is the excellent Pacific Rim to consider.

coming back online

I'm back!

What to report?

Hm..

You're back without a report?

I might need time to prepare one.

We're hoping for regular reports, you know.

Would you take a picture?

We don't usually publish pictures, but let's see what you have.

Ah, well...

No pictures?

I thought you could take mine.

I see.

Next time I'll have something.

Don't wait six months again, okay?

I'll try.

See you soon.

habitat

Made this last winter when I realized I was coming to Australia. Found it the other day and dressed it up a bit. From Yarra to Yahara, yep.

shouting into the wind

​I usually like io9, but this was pretty stupid, and yet it got a big response. It's a blurb on an essay, "On the Emptiness of Failed Replications", by Jason Mitchell, a social psychologist/neuroscientist, where he's criticizing some of the current discussion in the social sciences (and peripheral quarters). I've found other severely critical discussions of the essay, and none positive, although the negatives are (it seems) already entrenched in their opinions about how things *must* be done. Given my habitual uncertainty about everything, I think my take is fairly objective.

I haven't read most of the comments on the io9 article (just scanned the first page's worth), but it seems they are mostly agreeing with the negative post, and most of them don't discuss the essay *at all*. A lot of them are just bland 'social science is pseudoscience' stuff, but they're letting this blurb (assuming most of them didn't *read* the essay) feed their preconceptions. The comments on other blog posts are similar, though the quality of the posts themselves are generally better (if pedantic). I'm going to do some apologia here, since I'm avoiding working on a paper (or papers) of my own.

Mitchell's writing about the replication push, and explaining why, in basic philosophy-of-science terms (i.e. in terms of falsificationism - cf. previous post) replication as standard practice is not *scientific* practice. **Searching for null effects is meaningless**, he says, because there are vastly more ways to do an experiment wrong, and fail to detect an effect, than there are ways to do it properly. The scientific way to challenge a finding is not to try to reproduce it by following the steps in a methods section - it's by finding out how such a finding *could* arise, and explaining that in functional terms. The worst case is that the finding arises through fraud, and Mitchell goes pretty far in pointing out how a replication push can take the form of a witch-hunt.

Of course many science-minded folk are biased against the social sciences, in part because there have been a number of prominent frauds recently, but also because the methods are hard to discern. Social psychology, and most of experimental psychology at that, are different from the 'hard' sciences in important ways, but they are still scientific practices. It's not pseudoscience to say that you can measure a person's thoughts or perceptions or feelings or predispositions, although the measurements can be done well or done poorly.

There is one fundamental difference between the social sciences and the hard sciences (physics, chemistry, biology): in studying a human mind, you have to communicate with it, and every human mind is different - even a single human mind is different from day-to-day. Running a psychophysics or cognitive experiment effectively requires that you take this person, quickly figure out the contours of their personality (in a quick discussion or screening interview), and set their thought processes in such a way that you can *then do the experiment*. And furthermore, especially when it comes to in-depth studies like in real psychophysics, some people simply can't do the experiments; there are good psychophysics subjects and bad psychophysics subjects (in my experience, you might lump these into "people who are conscientious and have a high capacity for introspection", and "people who don't listen and who get bored by themselves"), and part of doing these experiments well is weeding out the bad ones. Usually it's pretty easy - a subject does some training blocks that produce crummy data - you try to explain to them how to improve their performance; they fail to improve, and you fire them. The foregoing steps are an art, the practice of communication and guidance and control, and some people are better at it than others. This is a relatively minor point in the essay, which is focused on replication, but it seems to have caught a lot of attention from the 'social science isn't science' folks.

I say all this as a social scientist, under general definitions. I'm a psychophysicist, a neuroscientist, and an experimental psychologist - most of my research is quantitative and computational and model-driven, with human beings (their behavior or their brains) as the source of my data. I completely recognize the situation that Mitchell is describing - I've had to explain these issues to juniors and colleagues many times over the years, though I don't know if I'm always convincing. I thought it was a great essay, and worth passing on. And, at the end of the day, I check the box 'psychologist', which puts me in the camp under attack, and so I feel I need to raise my tiny voice in defense of Dr Mitchell. Not that anyone comes here to read what I write, but I like to remember things, so.. meh.

The IIT is falsifiable

Sadly I have come to a period of procrastination - five straight months of work, and here it is. It's understandable; in coming up with some final analyses for what is sure to be one of the best papers I ever put together, I've allowed the dimensionality of my data to explode, at the same time that I'm trying to rewrite the code that produces that data so that it can be run on a supercomputer, and at the same time that I'm trying to generate a poster on the work with an eye to current developments. So, the paper has, for this week at least, ground to a halt, and I'm sitting here staring at pages of code and thousands of .mat files and making spreadsheets to try to force some organization on the process.

Okay, deep breath.

I'm doing some highly integrative neuroscience - I'm applying complex information-theoretic measures to human neural data, in such a way that behavioral data produced by those humans can guide my interpretations. It's complicated.

The theory I'm working under is called the Integrated Information Theory (IIT) of consciousness. Consciousness is a fraught area of science and philosophy - everyone has an opinion, even if they think they don't - and the people who know their opinions are really, really excited about them. Some of those people are kind of weird. But the IIT, I think, is actually a rather mundane theory - it's very abstract on the one hand, but it doesn't, by its nature, make any big metaphysical claims about existence, or try to tie itself to evolution or quantum physics. Rather, the IIT aims to be a theory of the neural correlates of consciousness (NCCs) - why is it that some parts of the brain correspond to conscious experience, while others don't? At this level, it competes with other theories of NCC, most notably the Global Neuronal Workspace theory (GNW), although they take opposite approaches - the GNW looks first at the brain, at the NCCs themselves, and tries to explain the specific neural processes that we can see, objectively, are producing consciousness. The IIT, on the other hand, looks first at conscious experience, and, from the inside-out, tries to explain what properties consciousness has by proposing an algorithmic or mathematical expression of that experience.

I know the IIT better than the GNW, and I prefer the approach of the IIT, but I think that eventually these two theories will meet in the middle, and we'll have a real, general theory of consciousness. Might not be for a while, but it will happen.

What the GNW gives you is a description of the brain's mechanics, and post-hoc philosophizing about how certain types of connections in the brain distribute information throughout a central network, and that this distribution is the promotion of something to the status of consciousness. This is based on seeing, objectively, that these things are correlated, so the GNW is a theory of the NCCs at its very root. To some eyes, this makes it the more plausible competitor, since it's based in objective reality.

The IIT, being based in subjective experience (by extracting 'axioms' about consciousness - it's integrated, it's informative, it's bounded, it has structure) is different from the GNW especially in that it isn't specifically based on any principles of neuroscience. It's a mathematical theory, a theory of information transmission in networks. It is, of course, expressed with the obvious intention of being applied to facts of neuroscience, and it seems to do well in this respect. Put simply (I'm obviously not trying to explain the IIT here), if you point the IIT at a brain, it should give you back a prediction (you know what I mean) of whether or not that brain is conscious, which parts are doing the job, and how the doing is organized. This last part is one thing that makes the IIT so interesting - it gives a way of describing the internal structure of what it claims is conscious experience.

So, this is what the IIT is intended for - to predict how consciousness arises in a brain. In this it is indeed falsifiable. An attack I've seen on the IIT from several sources - most recently some of the duller commenters on Scott Aaronson's much better attack - is that it isn't falsifiable, but this is clearly not correct. Tononi (the theorist behind the IIT) gives one clear example of how to falsify the theory: take two substances, both of which disrupt neural activity, but only one of which degrades the capacity of the brain to integrate information (in the specific terms of the theory). You could equalize the substances in other ways - make it so that one impairs attention, or some other manipulation that under a given theory should destroy consciousness - and then find out whether or not your (human) subject has lost consciousness. The IIT should predict exactly which interventions disrupt consciousness and which do not. In fact, it should predict which parts of consciousness are degraded - is vision lost, or hearing, or etc. If the theory fails, then it's wrong. So, the IIT is falsifiable.

But when the critics say it's unfalsifiable, they aren't thinking about the object of the theory - human (or animal) consciousness. They're thinking about panpsychism, and thus missing the whole point. It's true - the IIT predicts that systems that aren't brains can possess consciousness, and that it can be completely alien to human consciousness - no perception or cognition, for example. To many people this is 1) totally incoherent (this is Aaronson's criticism) and 2) unfalsifiable and thus the mark of a bad theory. But that a theory generates unfalsifiable statements is never grounds for dismissing the theory - thsi is basic logic (cf. Popper). What matters it that, in the realm where it's meant to explain something, the theory does its job. If you have two theories that explain equally well, but one generates incoherent, untestable predictions, then you can proceed with parsimony, but you have to have the alternative first. Occam's razor can't be used when you have no competition (unless you'd rather have no theory at all - and this is an ideological point rather than a scientific approach to a problem).

And anyways, even the panpsychism problem (taking the meaning of panpsychism very loosely, since IIT doesn't predict that everything is conscious - just that things that aren't brains can be conscious) isn't definitely untestable - the theory predicts that different conscious things are, in principle at least, linkable. So, if IIT says that a certain not-a-brain system is conscious, then the IIT adherent can simply plug his brain into the object, and if the theory predicts that the adherent's consciousness is linked to the not-a-brain system, then firsthand knowledge is the proper test. That's science fiction stuff there (the film Pacific Rim's drift technology was an implementation of the idea), but the IIT says it's technically possible. So, the ultimate test of the IIT might turn out to be technology - if every IIT-based device fails to produce predicted effects on consciousness, then the theory will be abandoned.

That's all I have for now. June is almost over. One more month in Australia, then home. Back to the grind...