Book Review: Fall (or, Dodge in Hell) by Neal Stephenson

 Not really a book review. More like, some thoughts on this book. It was long and not so straightforward as most books I read, so there was something to think about rather than just be awed (like with the Three Body books I finished last year - awesome books, huge ideas, but it was all there on the page - nothing to figure out, or reflect on, as far as I could tell).

I read this because of a review I read when it came out in 2019. There's an early section that I think got most of the attention of reviewers, where the characters go on a road trip through a post-truth America, which sounded most interesting to me. In the world of the book (or at least of this section - the story seems to span a century, and this is the only discussion of the world outside the immediate experiences of the main characters) the American hinterlands have devolved into a confused tribal culture where all belief systems are focused on the noise-filled descendant of the social-media internet we have today, and every belief is a false conspiracy theory of one sort or another. The city-classes rely on special tools, namely human editors, to keep their information accurate. That's a neat idea, and it barely has anything to do with the rest of the book!

I wasn't disappointed, though. Because while the whole back half of the book seems unrelated to this road trip, I realized after finishing the story that there was a serious connection to make. In the world of the road trip, it's pretty clear that the populace's disconnection from reality is a Bad Thing: the lives of people are brutal and confused and social progress is made impossible. It's suggested that this is because they are all exposed constantly to these noise-filled media channels where all content is false, generated by algorithms with varying hidden motives. The obvious implication is the what keeps the lives of the city people better is their connection to reality and true information - keeps them healthy, safe, and allows for progress.

The back half of the book isn't about the real world at all, though. It's about a digital afterlife where human consciousnesses* are uploaded after people die. This world begins when the first uploaded person (Dodge) finds himself exposed directly to world of chaos, that he learns to shape into meaningful things, thereby becoming the creator god of a new plane of existence. It's never said outright, but the chaos is various cloud-based (i.e. internet-based) computational activity, which means nothing at all to a human mind that is connected to it without an interface; by molding it into meaningful stuff (from Dodge's perspective), this computational activity is being replaced with new computational activity (from the outside-world perspective).

At first I thought that the book would get into an actual connection between this cloud-world of noise and the garbage-information world of the hinterland. But that connection never happens, and after a while I stopped expecting it and forgot about it. But after finishing the story I realized a different connection: the people of the digital afterlife are themselves all exposed to a false world - it's a computer game, basically - but it's ok, a Good Thing, and the ultimate meaning of the story is found there. Is it an inversion of the situation revealed in the road trip?

Sort of, I think. But also it's a comment on curation: the world Dodge creates is real in that everyone that inhabits it experiences it the same way. There are actual truths there, though they are ultimately shared psychological truths - at one point Dodge discovers that he can't change the world if other people are watching. 

This sets up a very different situation, but with obvious parallels to and deviations from the road trip world. In the real world, truth is essentially a physical thing: things exist physically, or events happen physically, and that's what makes them real, and what makes ideas about them true. People in the hinterland have beliefs that are largely false, because they are about things and events that don't exist or didn't happen. 

In the afterlife, there is no physical existence in the normal sense - physically, all the objects and places that people experience are actually processes running on vast banks of quantum computers. So strictly speaking, the things people believe in are all false, or at least not true. Not only that, but people in the afterlife have no knowledge of the 'real' world on the outside. Truth in the normal sense is impossible. Yet, because everyone agrees on this reality, we can think of beliefs in this world as having a kind of second-order truth value. Beliefs are at least meaningful and grounded, even if they are grounded in something (seems-to-be rocks and trees) that is not what it actually is (seems-to-be computers in buildings).

So then I wonder - is this a comment on the earlier part of the story? Because we can see the degenerated people of the American hinterland in the same way: their beliefs might be shared delusions or hallucinations, or vast conspiracy theories, but if they are meaningful and grounded, isn't that as good as what people have in the digital afterlife? Or, are the afterlife people in just as dire and meaningless a situation as the hinterlanders?

The book doesn't try to make this equation, I'm the one making it. The digital world is portrayed as wonderful for some, and nightmarish for many others, while the hinterland is depicted (indirectly) as more uniformly nightmarish. But again, maybe there are those in the hinterland that are living well, living their best lives despite their detachment from the larger realities. Is Dodge equivalent to a benevolent pirate king living out in the Nebraska wasteland?

At any rate, I was surprised at these connections, how they started to creep up on me.

*Re consciousness. I think that Stephenson is sufficiently vague about exactly how consciousnesses are simulated that I could suspend disbelief, or fill in the blanks myself. It sounds like the idea is that, with the connectome of a human brain, you just simulate the network on a massive parallel computer, giving each neuron its own computer processor. Maybe in some version of that, IIT consciousness might be possible (though I think probably not - but maybe)?

A bigger problem with the idea as he describes it is that a connectome could not be enough to simulate a brain, even if you had the connectome right down to every synapse, and even if (as we think is true) neurons connected with synapses are actually the right substrate of consciousness. That's because you don't know the rule for each neuron. A neuron has all these inputs, thousands of synapses on its dendrites, but what does it do when the neurons on the other side of those synapses are in whatever state? The connections don't tell you anything about that - you have to know something about how each neuron works, and what makes it harder is that there are so many kinds of neurons, so many kinds of synapses, and they might all follow different rules.

But those are details that could be filled in with enough imagination.

It was a great book, enjoyed it. 9/10. (minus 1 for being too long or too short - there's a bit more that could have been explained just about how the world worked, and the connections between the inside and outside of the afterlife, giving us a nice 1200 page book, or a part 1 and part 2 - or it could have been pared down to lessen expectations for those kinds of details. and the ending, considering how much we built up to it, was awfully abrubt, but good.)

***************************************************************************************************************************************************************************************************************************************************************************************

When I started grad school, one of the first things I did was to adopt a kitten that had been born earlier that summer in the apartment across the hall. Now, 18 years (!?!) later, here is a Eulogy for my Cat, in the form of a couple of cat-related anecdotes from my studies.

In my first year of school I was getting into the science of spatial vision - acuity, contrast, retina, V1, etc. I learned that a cat's visual acuity is like 5x worse than a human's. So, naturally, I figured that everything must look *blurry* to a cat.

See, I lived with this creature, and I would often think, how do things appear to her? I remember walking home late one night thinking about how stars appear to me as points - must they appear to a cat as blurry spots? And I realized that this couldn't be the case.

When I take off my glasses and see a point of light, it looks blurry in that I can see that what should be a point is actually smudged across a region of the visual field. And to see that smudge as a smudge, I must be seeing details within the region it fills.

If something appears as a point, that means that it has no apparent interior. So in fact, the acuity limit is telling us about the spatial resolution of appearance. Some visible thing that's smaller than the acuity limit must appear as a point. 

Without glasses, a point is imaged on my retina in a smudge that's a good fraction of a degree across. But my true visual acuity is in the range of 2 minutes of arc; so I see that smudge as an extended, 2d thing. My cat wouldn't see the smudge - she'd just see a point.

Now, years later I'm still making use of that insight, for example when I have argued that peripheral vision can and should look just as sharp as foveal vision - it's for the same reasons that a star should look like a point to my cat.

Another one, now about my perspective rather than the cat's. Some version of this is familiar to any of us: you see a dark shadow at night - is it the cat? or is it something else? There was one instance of this that I puzzled over for years.

I had a pair of shoes that I would sometimes leave by the wall in the hallway of my apartment in Boston - now we're in my first postdoc, 10 years ago. Time and again, I would see those shoes out of the corner of my eye and think, "Cat!" - then foveate and see, "shoes!'.

What fascinated me was that after the first time, I knew what was going on. That is, the high-level visual part of me knew they were shoes. But over and over, my early visual system was fooled - that brown shape of a certain size, on the floor, was most likely "Cat".

I started to collect these cases, where peripherally-seen X makes me think I'm seeing Y, even once I know X is really X. I posted an good case here a while back: (https://twitter.com/AndrewHaun3/status/1261891022625398784). But the prototype will always be Shoes->Cat.

Anyways, before I was so theoretically rigid as I am today, I used to wonder: when I thought the shoes were my Cat, did they actually *appear* as a Cat would appear? Did my mistaken recognition subtly reshape the spatial patterns so as to make them a better fit for "Cat"?

It does seem like this sort of thing can happen to some degree, e.g. with Ryota Kanai's "healing grid" illusion. But wholesale, at the object level? How could we find out? It doesn't matter, because I don't think that's what's happening here.

What I think is happening here, rather, is that I am experiencing a spatial form that is shoes-shaped, shoes-textured, etc. And it's decidedly not cat-shaped, cat-textured, etc - it's not clear cut pareidolia, as where I might see the Cat-shapedness of a Cat-shaped cloud. 

With the shoes, they're the right size and location for a Cat, and my super-sensitive Cat-recognizers are activated, and I experience recognition-of-Cat at the same time I experience a shoes-shaped, shoes-textured spatial gestalt. It's not a wholly congruent experience.

Even now, this week, when my Cat is no more, I've already made the mistake of thinking that a shadow in the hall was "Cat"; that a light *thump* in the other room was "Cat". Now, not only is it a mistake, it's an impossible mistake. But the brain does what it does.

***************************************************************************************************************************************************************************************************************************************************************************************

Procrastination

 I have reached peaks of procrastination that even I might once have thought were too great, too high. I've done it with the help of 'working from home', this monstrous situation that allows me to practice the piano, play video games, read nonsense, and any other disastrous activity, any time I like.

Back when I worked at the lab, in an office, I could only procrastinate for so long before I was cornered and had no choice but to do-the-thing. I couldn't play the piano or watch Netflix on a Tuesday afternoon in the lab. I might avoid work for a while by reading internet garbage, but eventually that runs out - it really does, it just takes a few days - and I have to do-the-thing.

But this...

Since the beginning of the pandemic, I've learned a Chopin nocturne - pretty well - laid groundwork for several Rachmaninoff preludes, memorized the 3rd movement of Beethoven's 'Moonlight' Sonata (biggest thing I've memorized in.. decades, probably?), and now I'm getting the hang of the awesome last movement of his first Sonata. I'm getting pretty good at improvising random lines over 9 chords! It's great! 

And it's worthless and stupid and dangerous, since I'm not a pianist and I'm not getting paid to play the piano. I have papers to write, finish, revise; experiments to plan; blah blah blah.

A point on interpreting inattentional blindness studies

On Inattentional Agnosia

In a recently published study, Cohen et al showed that, under very naturalistic conditions (viewing 3D natural scene videos in VR), observers often fail to notice that the entire periphery of the visual stimulus has been rendered colorless, i.e. completely desaturated. Cohen et al conclude that the visual periphery is far less colorful than one might have thought. They state this conclusion in several ways:

“these results demonstrate a surprising lack of awareness of peripheral color in everyday life”

-here qualifying the phenomenon as ‘lack of awareness of peripheral color’. Later, they say it less ambiguously: “If color perception in the real world is indeed as sparse as our findings suggest, the final question to consider is how this can be. Why does it intuitively feel like we see so much color when our data suggest we see so little?”.

So, Cohen et al believe that their data suggest we see very little color. This particular claim is logically absurd, however. I explain why in the following:

Cohen et al clearly believe that the phenomenon they present is a case of inattentional blindness. Inattentional blindness phenomena are frequently encountered in visual experience, and are not difficult to bring about in experimental scenarios. Typically, an observer is shown some stimuli connected to an explicit or implicit task; during the course of the task, some unexpected stimulus is inserted, and the observer may fail to notice. These failures to notice are often very retrospectively surprising, since once the observer knows what to look for (once they’ve been debriefed) it is easy to see the missed stimulus. Experimenters often conclude from these results – both the failures to notice and the retrospective surprise - that, in one way or another, the observers (and the rest of us) must see far less than they think they do. But this is not the kind of conclusion that Cohen et al are drawing.

The most famous example of an inattentional blindness phenomenon has got to be Simons and Chabris’ gorilla. In their experiment, observers watched a video of several people playing a ball-passing game. The players move around constantly, throwing the ball back and forth; the observer’s task is to count the number of times that certain players catch the ball. With no warning, halfway through the video, a person in a gorilla suit wanders into the middle of the ballgame, stops and waves at the camera, and then wanders back out of the frame – the ball game continues. Many observers do not notice the gorilla at all!

Simons and Chabris used this and similar results to advance a version of the “we see less than we think” argument. But what if we transplant Cohen et al’s conclusion to the gorilla experiment?

Here is a sentence in general terms that describes both studies:

“Observers viewed a complex stimulus that engaged their visual attention. After a while, a change was introduced to the stimulus that was retrospectively obvious. Many observers did not notice the change.”

Now, here is the reasonable, broad conclusion (a la Simons & Chabris):

“We must not notice as many things as we would expect based on what seems to be obviously noticeable.”

And here is the unreasonable, specific conclusion (a la Cohen et al):

“We must always be having the kinds of experiences evoked by the changed stimulus.”

In the Cohen et al study, they replaced a colorful scene with a colorless scene; observers didn’t notice; so, according to their reasoning, we must actually be having colorless experiences all the time (or more precisely, experiences of “so little” color). Otherwise, the reasoning seems to go, we would notice the change from colorful to colorless. We don’t notice it because it was colorless all along (they do include a caveat that maybe it’s the other way around, that even the grayscale scene evokes a colorful ‘filled-in’ experience, but that doesn’t seem to be their favored interpretation).

For the gorilla study, a gorilla was introduced incongruously into a ball game; observers didn’t notice; so, we must actually be having experiences of incongruous gorillas all the time (or, maybe more precisely, experiences of gorillas in ball games?). Otherwise, we would notice the change from a no-gorilla to an incongruous-gorilla scene. I don’t want to go on with this because it’s obviously absurd. But isn’t it the same logic as the color argument?

The absurdity comes in part from arguing from a complete lack of evidence: they are taking absence of evidence (failure to notice the change) to be evidence of absence (of color experiences). The experiments they are doing have no bearing, it seems to me, on whether or not their observers are actually experiencing color in their peripheral vision.

But more than this, the absurd conclusion comes from a lack of engagement with the important concepts at play. Color, most importantly. What does it mean to see color? That's for another time, I guess.

Before I finish here, an attempt at charity:

Perhaps the logic Cohen et al would derive from Simons & Chabris is somewhere in between the broad & reasonable, and the narrow and unreasonable:

"If we do not notice something, we are not experiencing it."

This is a strong claim, which I know that Cohen et al and many others would more-or-less endorse. But it does demand some engagement with some basic questions: if one's experience is colorless, what is it like? Is it like experience of a monochrome scene? Why are shades of gray excluded from 'color' status? What is special about 'chromatic hues'? Is there really less to seeing a monochrome scene than there is to seeing a color scene? Think about it: if you are seeing a spot as blue, that precludes your seeing it as any shade of gray, just as much as it precludes it from being yellow or red or whatever. Each part of the visual field always - it seems, at least - has some color in the broad sense.

In fact, Cohen et al did find that subjects always noticed if all color, in the broad sense, was removed from the periphery, i.e. if it were replaced with a flat gray field. Which would seem to defeat their basic conclusion that we are not seeing color, or much color. So, again, what is supposed to be so special about chromatic hues? Interesting questions, definitely.

A dialogue! (old but good)

I ask you to point your eyes up towards the clear sky and to tell me what you see. “An expanse of blue,” you say. I nod and say, “Ah, so simple. Blue is just one thing – your visual experience is so simple. Is that surprising?” 

This doesn’t sound right to you. You shake your head. “No, it’s not simple. There’s blue everywhere. Every location is blue; every part of the space is blue. It’s clearly not simple – it’s a vast structure of blue spots. I’m not even sure how to describe it to you.”

“Well,” I say, “you’re most likely confabulating this description. Through your life experience with using vision, you know that the sky is extended spatially, and that if you move your eyes around you’ll still see blue, and you know from moment-to-moment that the last thing you saw was what you’re seeing right now – simple blue – so you are illuded into claiming you see an expanse of blue. But you actually see no such thing.”

“How can you claim this?” you ask. “Why should you doubt what I tell you?”

I shake my head sadly. “Subjective reports are known for their fallibility. People often claim to have seen things that they could not have seen; they claim their experiences have qualities that they cannot have. But I’ll suspend my disbelief for a bit. Can you convince me?”

You seem a little annoyed, but you nod. “Perhaps.”

“Okay. How many parts are there to this blue expanse?” I ask. You don’t know. We go through some basic tests and it seems that you can’t really tell me about more than a handful of spots at once – yet you persist in claiming that the actual number of blue ‘spots’ is vast.

“Are they all there at once?” You seem to think that they are. “Could it be that the parts are there only when you look for them?”

“No,” you say, “it feels like a big, continuous expanse of blue. It’s not a little searchlight.”

I proceed. “But I’m asking you to convince me of that, not just to tell me again and again. As far as I can tell, you can only report the color of a few spots at a time – a big ‘sky-sized’ spot, or a few little ‘point-sized’ spots. But your momentary capacity seems to be extremely small – where is this huge expanse? And how does it make any sense that you should experience such enormous complexity, but be able to interact with only a vanishingly small portion of it?”

You seem unsettled: “Why,” you ask, “would I claim to see an expanse of spots when I only see a few at a time? What do I gain by confabulation?”

“But it’s a meaningful confabulation – you are unaware of the limits or boundaries between your momentary visual experience, your memories of recent experiences, and your expectations of what future experiences will be like. The reports you generate are more a confusion of these different processes, rather than a confabulation.” I concede a word, but little else.

“Well then,” you say slowly, “what does this confusion feel like? Might it feel like an expanse of blue? Or do you assume that only the perceptual process constitutes experience?”

I miss your point. “But we’re exactly talking about perceptual experience – of seeing blue – don’t try to shift the goal posts.”

“No, we aren’t talking strictly about perceptual experience, though I do think my experience is fairly categorized as perceptual rather than memorial or expectative. You’re the one who introduced other ‘processes’ into the conversation.”

“Well, this can’t work,” I say. “I can concede the process thing, but this doesn’t address the reportability issue at all, and it’s highly implausible, even worse than if you put everything in ‘perception’. Are you claiming that you experience all your memories or all your expectations, at once?”

“No no,” you dismiss the idea with a wave of your hand. “I was just asking – what do you think such confusion should feel like?”

“Like what you’re feeling now,” I say.

You roll your eyes. “Come on. Clearly the confusion you’re suggesting should feel very different from what I am feeling – or no, from what I claim to be feeling. Otherwise we wouldn’t be having this debate.”

“Well, I can’t say exactly. You are experiencing what you have access to, and you can report what you have access to; so your experience must be of a narrow set of blue spots. And you claim otherwise because whenever you check other spots, you immediately begin to experience them – so you mistakenly believe that they were there all along. Your experience isn’t what you think it is – and it isn’t what you claim.”

You seem perplexed. “Does that mean that unless I am queried about my experience, I am not under this illusion? I only become mistaken when asked to describe what I’m experiencing?”

“Maybe?”

You decide to change tack. “Okay. Can you tell me what substantive difference there is between this illusory or mistaken experience and an actual experience of a blue expanse?”

“Well, it would be a huge difference – the illusory experience is actually very limited and consists of very few parts, including the few blue spots and a particular set of expectations and memories that lead you to claim that you see an expanse of blue. The actual experience of a blue expanse would be just that – many many more spots, and no necessary memory or expectation aspects, though you’d probably also have those in addition.”

“I can’t help but think,” you say carefully, “that you’re doing something slippery here. You want to know why I claim to see a blue expanse, and your explanation has to do with these non-perceptual processes and how they seamlessly support my very limited perceptual process. And you reject my explanation for my claim – that I really am experiencing a blue expanse – because I can’t report the whole expanse to you. But can I report all my memories and expectations to you? Do you know how to collect such data?”

“I think that the fact you’re able to so quickly report on what you see at randomly cued locations suggests that those processes must be at work.”

“Surely they are, and I can tell you that I do indeed have experiences of memory and expectation. But I’m wondering why you think you must reject my explanation but are satisfied with your own.”

“Certainly I’m not satisfied – there is much to learn. We really understand perception etc rather poorly at this stage.”

You shake your head.

EHS

Falling asleep two nights ago, I realized I was hearing a terrible noise, a roaring, screaming cacophony; but then, when I realized it, I also realized I wasn't hearing anything at all. For a while, maybe tens of seconds or a few minutes, I had been lying there listening to this noise, with a vaguely uncomfortable feeling, mind drifting - falling asleep, in other words. On attending to it, I seem to have broken myself out of a hypnogogic auditory hallucination - the experience faded after a few seconds of my concentrating on the fact that, actually, I wasn't hearing anything at all, and as far as I know it didn't come back, and I fell asleep.

The next day I told Jonathan about it, and he reminded me of "Exploding head syndrome", which is usually described as a sharper, more acute noise ("snapping of the brain" was a term coined by a doctor in 1920), but I think what I experienced fits. Maybe it's happened before - probably, I think - and either I proceeded to sleep without the lucid break, or I've forgotten the details. It doesn't happen often, at least.

The experience was basically like auditory imagery, similar quality as hearing music or voices in your head; but whereas those kinds of experiences generally seem (to me) to be endogenous and even self-willed (even when it's a tune you can't get out of your head), this sound felt out of my control. It was frightening, but as I attended to it it was clearly a completely internal experience - less real than tinnitus, but in retrospect I think more than just a typical auditory image. I say this because now, when I try to imagine the sound, I can get a rough idea of it but I can't really experience it as really as I was during the episode; and when I imagine a piece of music, or a voice, these also seem less substantial.

At any rate, once I attended to the experience, and reassured myself that in fact I was hearing nothing and that it was a hallucination, the cacophony dissolved into the regular silence of auditory imagining. Maybe it was an aftereffect of the long holiday weekend? Exhaustion?

Parahypnic Hallucinosis?

Remember this episode:
http://xuexixs.blogspot.com/2018/08/blinking-plants.html

That was my bout with what I termed 'gardener's hallucinosis', where I spent all day pulling clear weed and etc from the gardens, and wound up with vivid blinking hallucinations. I likened it to the 'eyes open' geometric hallucinations I sometimes had back in my migraine days.

Well I've noticed a few times lately something similar happening under a specific circumstance. The circumstance is: I fall asleep in my daughter's room as I read her to sleep. I wake up a couple of hours later, stumble to the bathroom to brush my teeth, and I go get in my bed.

It doesn't happen every time, but sometimes - and last night very vividly - on waking in this way I have vivid and complex geometric hallucinations. Fine-grained, colorless - much of the content is just of very, very fine beads or dots, flickering and moving - but mixed into it are larger-scale features. Last night, the features were like a high-pass Kandinsky painting: discs and long, smoothly-curving lines, all moving and twisting around randomly, but no particular surface colors except for grayness, or darkness.

I could see it all fairly clearly with eyes opened, until I turned on the bathroom light and then the experience faded.

Maybe not coincidentally, I have been having minor headaches lately, and I think there was one yesterday. My brain must be in a state?

Two Things

Got two emails last night, one right after the other:

1. I get to give my 'color across the visual field' talk at ASSC 23. Gonna put my mouth where my money is! I wonder how they'll take it.

2. I was actually awarded a NEI travel award by the VSS! When's the last time I won something? Yeah, I don't know either!

Perfect Cadence

Just a few days ago, I realized something about my simple mind. It had been bubbling up for a few months, but suddenly it's crystallized:

Start your melody with the dominant tone on the upbeat, then down on the tonic ("5-1"), and I will love your melody and it will stick in my mind forever. Apparently.

I am a lifelong piano player, but I've only recently really started to get acquainted with the general classical piano repertoire. So while I've been playing everything by Erik Satie since I was a teenager, only in the last 5-6 years have I started to play a lot of Chopin, Scriabin, Grieg, etc. Only a 2-3 years ago did I first start to learn Chopin's etudes.

One of the etudes I played for a while, maybe the most beautiful one and one of the most famous, is the Op.10 no.3 in E-major. It starts on the 5, then to the 1:



That is a nice melody! Happy but a little sad, reflective, contemplative. Then this summer, I start looking for Schumann pieces to play, and of course I rediscover his "Traumerei" in F:



Very nice melody, easy to play, similar mood, meaning (in my mind) to the Chopin etude. And it also starts from the 5 to the 1. I notice this and think, how neat, two melodies I like that start the same way - and just a half-tone apart!

Then this summer, I start listening to Grieg's lyric pieces, and I get the book and they're all fun to play, mostly not too difficult. One of my favorites is one of the simplest, the "Watchman's song":

Now you see it again, again in E major: 5-1 to start the melody. And again, similar mood. Now I start to realize something funny is going on. I'm not just cherry-picking here: these are three of my favorite piano pieces to play or hum to myself in the hallways in the last year or so. I also realize that my favorite Grieg melody, "Solveig's Song" from Peer Gynt, starts the same way (5 up to 1 on the downbeat), though it is in A minor.

Then, well, I forget about it - and last week, not sure why, I'm thinking about it again. My favorite tune from Satie's Gnossienes: no 5, in G major.



Not just the first two notes - all four of these pieces (Chopin, Schumann, Grieg, Satie) have the same first three notes, and they're all in nearby keys (Emaj, Fmaj, Emaj, Gmaj).

Once I noticed the Gnossienne, I start finding the pattern everywhere. First, many other great classical melodies, though mostly in minor keys (the major ones apparently stand out to me), and I realize I can just sit and they're so easy to find in my brain:

Beethoven's "Marmotte"; Smetana's "Moldau"; Grieg's "Solveig's song"; Faure's Sicilienne, Scriabin's Prelude 14 (which I learned 10 years ago, one of the first of his that I learned); the 'song without words' from Holst's folk song Suite, which was always my favorite part of that tune when the band played it in high school. Those are all in minor keys, but notice this too: whereas the major key ones all go down from the tonic (to the major seven, each one), the minor key tunes all go up! And except for the Sicilienne, they all go up to the 2 (it goes up to the 3).

And what do you think has always been my favorite leitmotif from Star Wars? The "Force Theme", in C minor, which goes up from the tonic to the 2!

Then we can do folk songs and pop songs; I would have immediately have listed my childhood favorites as O Shenandoah (Major, but goes up from the tonic) and "My Grandfather's Clock" (Major, goes down to the 7 like the four initial examples). Everyone's favorite 'Simple Gifts' (the Shaker hymn used by Aaron Copland). My favorite Simon Garfunkel song: El Condor Pasa (Minor, down to the seven from the tonic); a pretty good Queen song, though not my favorite, is "Who wants to live forever", in a minor key going up to the 2 from the tonic.

My two favorite Tom Waits tunes: "Downtown train" (outside another yellow moon) and "If I have to go", though those are so similar it might be that the former is a rewrite of the latter. And what Mozart melody, that I would always have said was his most beautiful, starts in exactly the same way (same first 4 notes) as those Tom Waits melodies?

It doesn't start on the upbeat, but it's the theme from the second movement of his clarinet concerto:



***

Now, starting your melody by going from the dominant to the tonic doesn't guarantee I'll love it. I quickly thought of two examples I don't really care for - just two, while it was so easy to think of the two-dozen beloved ones above: I don't enjoy Schumann's "The Happy Farmer", maybe because my mother's a piano teacher and I've heard it way, way, way too many times in my childhood. And Wagner's wedding march ("Treulich geführt") is just clichéd to death. But then, speaking of Wagner, there is main theme of the Tannhauser overture, which I remember listening to over and over again, for its inspirational feeling, back 2-3 years ago. Easily my favorite Wagner theme, though I had not thought of it in a while (and I have never listened to the actual opera, so I don't know the source of the theme - wikipedia says it's from the "Pilgrim's Chorus").

In sum it seems that there is a key in my brain, shaped like "5-1", and it lets you right into my eternal musical memory.

Gardener's Hallucinosis?

Interesting experience yesterday, Sunday.

Spent ~5hrs outside in the gardens, pulling weeds. Had done the same for a few hours on Saturday. When I came inside for good, about 5 or 6, I started to hallucinate during blinks - when I blinked my eyes, especially when not prepared for it, I would see images of the plants I had been pulling all day. Sometimes very clear, seeing leaves with their serrations and textures, and tendrils curling around - the images were coherent and (mutedly) colorful, seemingly randomly selected but each was a recognizable one of the real plants I had seen, mostly members of the 2 or 3 most common weeds of the day.

Sometimes the images were strong enough to be distracting, making it hard to see - or to recognize - what was actually before my eyes. But I think they were only actually visible during the blinks. I managed over time to notice some properties of the images - I could hold my eyes closed after an effective blink. It was still unclear to what extent I was *really* seeing the fine details, or whether the actual images were coarser and just 'suggesting', as in normal visual imagery, the fine details. Holding my eyes shut, it seemed that the form of the afterimages or noise, in the eyes-shut darkness, guided the structure of the hallucination: spots of afterimage seemed to appear as leaves, streaks as stems or tendrils. But it was not so clear as to be certain of this.

The experience lasted until I went to bed, 6 or 7 hours later, but it had attenuated by then. I slept and remembered several dreams that had nothing at all to do with plants (one I remember, now, was that my lab seemed to be based in the house I grew up in, and some newcomers were using space in the den - dream-Giulio warned me not to give them to much space, or they'll think they can take more). When I woke up this morning, the phenomenon returned for another hour or so, but is gone now.

The phenomenon resembled, to me, the kinds of hypnogogic hallucinations some people have after long, repeated activity ('the Tetris effect'), but I can't find reports of this in normal waking experience (albeit during blinks). I described it to Giulio and others, the physiological explanations are kind of clear, but as to why it happened to me and why it isn't much more common, that's an open, strange question.

Another thing it reminded me of, was back in the migraine days, seeing geometric web patterns after waking, sometimes during blinks. Similar kind of dynamic, but I don't think those experiences ever lasted more than minutes, definitely not many hours.

IIT and Star Trek

[I originally posted this to a Star Trek forum because I am a Star Trek nerd but here it is for better posterity:]

"Complex systems can sometimes behave in ways that are entirely unpredictable. The Human brain for example, might be described in terms of cellular functions and neurochemical interactions, but that description does not explain human consciousness, a capacity that far exceeds simple neural functions. Consciousness is an emergent property.” - Lt.Cmdr. Data

A post the other day about strong AI in ST provoked me to think about one of my pet theories, there in the title: Data is conscious, the Doctor is not, and other cases can be inferred from there. Sorry that this is super long, but if you guys don't read it I don't know who will, and my procrastination needs an outlet.

First, some definitions. Consciousness is a famously misunderstood term, defined differently from many different perspectives; my perspective is that of a psychologist/neuroscientist (because that is what I am), and I would define consciousness to mean “subjective phenomenal experience”. That is, if X is conscious, then there is “something it is like to be X”.

There are several other properties that often get mixed up with consciousness as I have defined it. Three, in particular, are important for the current topic: cognition, intelligence, and autonomy. This is a bit involved, but it’s necessary to set the scene (just wait, we’ll get to Data and the Doctor eventually):

Cognition is a functional concept, i.e. it is a particular suite of things that an information processing system does, specifically it is the type of generalized information processing that an intelligent autonomous organism does. Thinking, perceiving, planning, etc, all fall under the broad rubric of “cognition”. Humans are considered to have complex cognition, and they are conscious, and those two things tend to be strongly associated (your cognition ‘goes away’ when you lose consciousness, and so long as you are conscious, you seem to usually be ‘cognizing’ about things). But it is well known that there is unconscious cognition (for example, you are completely unaware of how you plan your movements through a room, or how your visual system binds an object and presents it as against a background, how you understand language, or how you retrieve memories, etc) - and some theorists even argue that cognition is entirely unconscious, and we experience only the superficial perceptual qualities that are evoked by cognitive mechanisms (I am not sure about that). We might just summarize cognition as “animal-style information processing”, which is categorically different from “what it’s like to be an animal”.

Intelligence is another property that might get mixed up with consciousness; it is generally considered, rather crudely, as “how well” some information processing system handles a natural task. While cognition is a qualitative property, intelligence is more quantitative. If a system handles a ‘cognitive' task better, it is more intelligent, regardless of how it achieved the result. Conceiving of intelligence in this way, we understand why intelligence tests usually measure multiple factors: an agent might be intelligent (or unintelligent) in many different ways, depending on just what kinds of demands are being assessed. “Strong AI” is the term usually used to refer to a system that has a general kind of intelligence that is of a level comparable to human intelligence - it can do what a human mind can do, about as well (or better). No such thing exists in our time, but there is little doubt that such systems will eventually be constructed. Just like with cognition, there is an obvious association between consciousness and intelligence - your intelligence ‘goes away’ when you lose consciousness, etc. But it seems problematic to suppose that someone who is more intelligent is more conscious (does their experience consist of “more qualia”? What exactly does it have more of, then?), and more likely that they are simply better-able to do certain types of tasks. And it is clear, to me at least, that conscious experience is possible in the absence of intelligent behavior: I might just lie down and stare at the sky, meditating with a clear mind - I’m not “doing” anything at all, making my intelligence irrelevant, but I’m still conscious.

Autonomy is the third property that might get mixed up with consciousness. We see a creature moving around in the environment, navigating obstacles, making choices, and we are inclined to see it as having a sort of inner life - until we learn that, no, it was remote-controlled all along, and then that apparent inner life vanishes. If a system makes its own decisions, if it is autonomous, then it has, for human observers at least, an intrinsic animacy (this tendency is the ultimate basis of many human religious practices), and many would identify this with consciousness. But this is clearly just an observer bias: we humans are autonomous, and we assume that we are all conscious (I am; you are like me in basic ways, so I assume you are too), and so we conflate autonomy with consciousness. But, again, we can conceive of counter-examples - a patient with locked-in syndrome has no autonomy, but they retain their consciousness; and an airplane on autopilot has real (if limited) autonomy, but maybe it’s really just a complex Kalman filter in action, and why should a Kalman filter be conscious (i.e. “autonomy as consciousness” just results in an endless regression of burden-shifting - it doesn’t explain anything)?

To reiterate, consciousness is “something it’s like to be” something - there’s something-it’s-like-to-be me, for example, and likewise for you. We can turn this property around and query objects in nature, and then it gets hard, and we come to our current problem (i.e. Data and the Doctor). Is there something-it’s-like-to-be a rock? Certainly not. A cabbage? Probably not. Your digestive system? Maybe, but probably not. A cat? Almost certainly. Another human? Definitely. An autonomous, intelligent, android with human-style cognition? Hmmm… What if it’s a hologram? Hmmm….

That list I just gave (rock; cabbage; etc) was an intuition pump: most of us will agree that a rock, or a cabbage, has no such thing as phenomenal consciousness; most of us will agree that animals and other humanoids do have such a thing. What makes an animal different from a rock? The answer is obvious: animals have brains. Natural science makes clear that human consciousness (as well as intelligence, etc) relies on the brain. Does this mean that there’s something special about neurons, or synapses, or neurotransmitters? Probably not, or, at least there’s no reason to suppose that those are the magic factors (The 24th century would agree with this; see Data’s quote at the top of this essay). Instead, neuroscientists believe that consciousness is a consequence of “the way the brain is put together”, i.e. the way its components are interconnected. This interconnection allows for dynamically flexible information processing, which gives the overt properties we have listed, but it also somehow permits existence of a subjective point of view - the conscious experience. Rocks and cabbages have no such system of dynamical interconnections, so they’re clearly out. Brains seem to be special in this regard: they are big masses of complex dynamical interconnection, and so they are conscious.

What I’m describing here is, roughly, something called the “dynamic core hypothesis”, which leads into my favored theory of consciousness: “integrated information theory”. You can read about these here:http://www.scholarpedia.org/article/Models_of_consciousness The upshot of these theories is that consciousness arises in a system that is densely interconnected with itself. It is important to note here that computer systems do not have this property - a computer ultimately is largely a feed-forward system, with its feedback channels limited to long courses through its architecture, so that any particular component is strictly feed-foward. A brain, by contrast, is “feedback everywhere” - if a neuron gets inputs from some other neurons, then it is almost certainly sending inputs back their way, and this recurrent architecture seems implemented at just about every scale. It’s not until you get to sensorimotor channels (like the optic nerves, or the spinal cord) that you find mostly-feed-forward structures in the brain, which explains why consciousness doesn’t depend on the peripheral nervous system (it’s ‘inputs and outputs’). Anyways, this kind of densely interconnected structure is hypothesized to be the basis of conscious experience; the fact that the structure also ‘processes information’ means that such systems will also be intelligent, etc, but these capacities are orthogonal to the actual structure of the system’s implementation.

So, Data. Maybe Data isn’t conscious, but just gives a great impression of a conscious being: he’s autonomous, he’s intelligent, he has a sophisticated cognitive apparatus. Maybe there’s nothing “inside” - ultimately, he’s just a bunch of software running on a robotic computer platform. People treat him like he’s conscious (Maddox excepted) just because of his convincing appearance and behavior. But I don’t think it’s an illusion - I think Data is indeed conscious. 

Data’s “positronic brain” is, in a sense, a computer; it’s artificial and made from artificial materials, it’s rated in operations per second, it easily interfaces with other more familiar kinds of computers. But these are really superficial properties, and Data’s brain is different from a computer in the ways that really matter. It is specifically designed to mimic the structure of a human brain; there are numerous references throughout TNG that suggest that Data’s brain consisted critically of a massive network of interconnected fibers or filaments, intentionally comparable to the interconnected neurons of a biological brain (data often refers to these structures as his “neural nets”). This is in contrast to the ‘isolinear chip-bound’ architecture of the Enterprise computer. Chips are complicated internally - presumably each one acts as a special module that is expert in some type of information processing task - but they must have a narrow array of input and output contacts, severely limiting the extent to which a chip can function as a unit in a recurrently connected network (a neuron in a brain is the opposite: internally it is simple, taking on just a few states like “firing” or “not firing”, but it makes tens of thousands of connections on both input and output sides, with other neurons). The computer on 1701-D seems, for all intents and purposes, like a huge motherboard with a ton of stuff plugged into it (we can get to the Intrepid class and its ‘bio-neural chips’ in just a bit).

Data, then, is conscious in virtue of his densely recurrently interconnected brain, which was exactly the intention of Dr Soong in constructing him – Soong didn’t want to create a simulation, he wanted to create a new being. I contrast Data at first with the Enterprise computer, which is clearly highly intelligent and capable of some degree of autonomy (as much as the captain will give it, if you believe Picard in 'Remember Me’). I won’t surmise anything about “ship cognition”, however. Now, if the ship’s computer walked around the ship in a humanoid body (a la EDI of the Mass Effect series), we might be more inclined to see a ghost in the machine, but because of the ship’s relatively compartmentalized ‘chip focused’ structure and it’s lack of a friendly face, I think it’s very easy to suppose that the computer is not conscious. But holographic programs running on that very same computer start to pull at our heartstrings - Moriarty, Minuet, but especially… the Doctor.

The Doctor is my favorite Voyager character (and Data is my favorite of TNG), because his nature is just so curious. Obviously the hologram “itself” is not conscious - it’s just a pattern of projected photons. The Doctor’s mind, such as it is, is in the ship’s medbay computer (or at times, we must assume, his mobile emitter) - he’s something of an instantiation of the famous ‘brain in a vat’ thought experiment, body in one place, mind in another. The Doctor himself admits that he is designed to simulate human behavior. The Voyager crew at first treats him impersonally, like a piece of technology - as though they do not believe he is “really there”, i.e. not conscious - but over time they warm to his character and he becomes something of an equal. I think, however, that the crew was ultimately mistaken as to the Doctor’s nature - he was autonomous, intelligent, and a fine simulation of human cognition and personality, but he was most likely not a conscious being (though he may have claimed that he was).

Over and over, we hear the Doctor refer to himself as a program, and he references movement of his program from one place to another; his program is labile and easily changed. This suggests that his mind, at any given moment, is not physically instantiated in a substrate. What I mean by this is that while a human mind (or Soong-type android mind) is immediately instantiated in a pattern of activity across trillions of synapses between physically-realized interconnected elements, the Doctor’s mind is not. His mind is a program stored in an array of memory buffers, cycling through a system of central processors – at any given moment, the Doctor’s mind just just those few bits that are flowing through a data bus between processor and memory (or input/output channel). The “rest of him”, so to speak, is inert, sitting in memory, waiting to flow through the processor. In other words, he is a simulation. Now, to be sure, in a lot of science fiction brains are treated as computers, as though they are programmable, downloadable or uploadable, but in general this is a very flawed perspective - brains and computers actually have very little in common. The Star Trek universe seems to recognize this, as I can’t think of any instances of outright abuse of this trope in a ST show. One important exception stands out: Ira Graves.

Ira Graves is a great cyberneticist, so let’s assume he knows his stuff (let’s forget about Maddox, who was a theoretically impoverished engineer). He believes that he can save his consciousness by moving it into Data’s brain. But Data’s brain is not a computer in any ordinary sense, as we detailed above: it’s a complex of interconnected elements made to emulate the physical structure of a human brain. (This is why his brain is such an incredible achievement: Data’s brain isn’t a miniaturized computer, it’s something unique and extraordinarily complex. This is why Lal couldn’t just be saved onto a disc for another attempt later on - Data impressed himself with her memories, but her consciousness died with her brain.) Anyways, Ira Graves somehow impresses his own brain structure into Data’s positronic brain - apparently killing himself in the process - and seems happy with the result (though he could be deluded - having lost his consciousness, but failing to recognize it). In the end, he relinquishes Data’s brain back to Data’s own mind (apparently suppressed but not sufficiently to oblieterate it), and downloads his knowledge into the Enterprise computer. Data believes, however, that Graves’ consciousness must have been lost in this maneuver, which is further support for the notion that a conscious mind cannot “run on a computer”: a human consciousness can exist in Data’s brain, but not on a network of isolinear chips.

The Doctor, in the end, is in the same situation. As a simulation of a human being, he has no inner life – although he is programmed at his core to behave as though he does. He will claim to be conscious because this makes his humanity, and thus his bedside manner, more effective and convincing. And he may autonomously believe that he is conscious – but, not being conscious, he could never know the difference, and so he cannot know if he’s making an error or not in this belief.

I think that here we can quickly bring up the bio-neural gel packs on Voyager. Aren’t they ‘brainlike’ in their constitution? If the Doctor’s program runs on this substrate, doesn’t that make him conscious? The answer is no – first, recall what Data had to say about neural function and biochemistry. Those aren’t the important factors – it’s the dense interconnectedness that instantiates an immediate conscious experience, and we have no reason to believe that the interconnection patterns of an array of bio-neural gel packs is fundamentally different from a network of isolinear chips. Bio-neural thingies are just supposed to be faster somehow, and implement ‘fuzzy logic’, but no one suggests they can serve as a substrate for conscious programs. And furthermore, the Doctor seems happy to move onto his mobile emitter, whose technology is mysterious, but certainly different from the gel packs. It seems that he is just a piece of software, and that he never really has any physical instantiation anywhere. In defense of his “sentience” (Voyager episode ‘Author, Author’), the Doctor’s crewmates only describe his behavioral capacities: he’s kind, he’s autonomous, he’s creative. No one offers any evidence that he actually possesses anything like phenomenal consciousness. (In the analogous scene in ‘Measure of a Man’, Picard at least waves his hand at the notion that, well, you can’t prove Data isn’tconscious, which I thought was pretty weak, but I guess it worked. I don’t know why they didn’t at least have a cyberneuroscientist or something testify.)

So that is my case: Data is conscious, and the Doctor is not. It’s a bit tragic, I think, to see the Doctor in this way – he’s an empty vessel, reacting to his situation and engendering real empathy in those he interacts with, but he has no pathos of his own. He becomes an ironically pathetic character – we feel for him, but he has no feelings. Data, meanwhile, in his misguided quest to become more human and gain access to emotional states (side note: emotion chip BLECH) is far more human, more real, than the most convincing holographic simulation can ever be.

IIT and Blade Runner 2049

Blade Runner 2049 is probably the best movie I've ever seen in a theater - definitely the best sci fi movie I've seen. The movie is a detective story about a replicant - an artificial human - who uncovers a mystery that has personal implications for himself, and broader implications for the dystopia that he lives in.

Are replicants conscious? It's hard to argue that they wouldn't be, and the movie doesn't seem to suggest they aren't. Instead, the movie focuses on memory and how your memories make you real or not - if your memories are false, you are a kind of false person in the world of 2049, and this is how people in the movie justify their enslavement of the replicants. The main theme of the movie is memory - are my memories real? If they're real, are they really mine, or someone else's? Does it really matter?

That stuff is all interesting, but like I said, consciousness is not the question with the replicants. It is the core question regarding one of the main characters: Joi the holographic girlfriend. We can speculate now on whether or not Joi is conscious. The movie is ambiguous about this, but there seems to be a subtext that she is not conscious, but that the main character (K), and we the audience, are supposed to believe that she is. And while there is this ambiguity, the resolution of the ambiguity is deeply meaningful to the story (just like the original Blade Runner, such a significant ambiguity is left unresolved).

First, to be clear on IIT terms: replicants are conscious because they are basically humans with human brains (and humans are clearly conscious) - what makes replicants different is that they are constructed as adults, with memories implanted (or not) to give them a more natural psychology. In the original Blade Runner, replicants like Roy Batty were assumed by their masters to be essentially psychopathic by nature, and the subsequent implantation of false memories was instituted to make them more psychologically healthy. But for a 'system in a state', the truth or falsity of memory is an extrinsic fact - from the IIT point of view (and probably any other modern theory of consciousness, or of the brain) it doesn't matter for the system itself. So replicants are conscious.

Joi, on the other hand, is not a human with a human brain. She's a holographic projection generated by a computer. The hologram of course is nothing but an image; what matters is in the computer. Computers as we know them cannot be conscious in any meaningful sense: the system is a very small set of very very fast switches, entirely feedforward at the most complex, finest grain, and extremely simple at coarser grains where we might see something like feedback or lateral connections. If computers in 2049 are like computers we know, Joi is not conscious - but computers might be very different. Joi has some kind of dedicated local unit, mounted on the wall in K's apartment - perhaps the computer in that unit is a neuromorphic system that replicates the connectivity structure of a human brain. But the picture of technology in the movie doesn't suggest this level of sophistication - I think that if we want to argue that Joi is conscious (in order to counter-argue) we need to weaken some assumptions.

Maybe Joi is conscious, but her consciousness is absolutely different from a human consciousness. That still requires some kind of neuromorphic computer, but it doesn't have to reflect the structure of the human brain, but there's a problem there that boils down to unreliability: if you want a simulation of a human being, you probably want something that's utterly controllable, like a performance - and where it's uncontrollable, it should still fulfill the simulation's desiderata. But consciousness is exactly uncontrollable - it's a closed locus of causal power (according to IIT) - so if your machine is conscious and you want it to simulate a human being, then its consciousness ought to resemble a human consciousness. Joi seems to do a very lifelike impression of a human being, so we have two choices - either she is conscious and her consciousness is specified by a neuromorphic computer that reproduces human neural connectivity, or she is an unconscious simulation.

As I said above, an artificial human brain (in the sense of an electronic device) seems beyond the technology of 2049; but even if it is in reach, it is hard to reconcile with the way Joi is quickly cut and copied over to her mobile emitter. First, it would mean that not only are there artificial human brains in 2049, but they are tiny enough to fit in the palm of your hand; second, it would mean that this brain can be constructed (or connected) in seconds, since remember that according to IIT it is a causally-interacting physical substrate that specifies consciousness - a computer program stored in memory is not causally-interacting in any important sense. I just don't think either of these is plausible in context.

So that leaves us with Joi the unconscious, but highly convincing, holographic girlfriend. Seeing Joi this way is easy when she first appears in the movie, but rather quickly it becomes clear that she is a dramatic and interesting character. Just like any other character in a movie or a play, it is then very difficult to imagine that she is not conscious. We know that the actor playing her is conscious, which makes it even more difficult. But if we try, we can see her as an entirely mechanical projection, like Siri or a chatbot, something that emulates humanity down to little details like evincing emotions like love and hope and excitement, and insistence on her own choices. But evincing emotion is not the same as feeling emotion - while there was an actor (Ana de Armas) that performed the character, there is nothing there on the screen while we watch the movie, and whether or not the actor was ever even real (or whether the performance is entirely artificial) doesn't matter to the fact that the performance on the screen is just a mechanical, unconscious projection. K the Blade Runner is in a similar situation: Joi is convincing, and maybe K himself cannot recognize that she is not intrinsically real, but she is nonetheless unreal.

This seems to me to contribute important meaning to the story, and it does resonate with some clues that are given out bit-by-bit, i.e. it seems the filmmakers probably also thought that Joi is not really real (not conscious). We see ads in the background of 2049 LA for Joi, touting that she is everything you want, and this is exactly what K seems to get. And K, who seems to despair at losing her (and losing other things), seems to recognize this (or remember it) when one of those ads reaches out to him and calls him by the same name that 'his' Joi had given to him. His copy of Joi wasn't even customized (it had all the 'factory settings'), i.e. not only is she unreal, she isn't even unique. So as a conscious being himself, K really is alone - his only companion is just a performance without an actor.

I think that, then, we're left with a hard question that aligns with the main theme of the story, which is (as I understood it) does it matter if my memories are mine, or if they are real? If none of my memories are real, am I real, do I matter? That theme gets a resolution: it is clear that K is real, he matters in important ways, and that his memories, real or not, nevertheless guide him significantly. The hard question is: does it matter if you are real? I mean that from the first person: I know I am real, but are you? Does it matter if you are or not, at least, does it matter to me? Well.. in a sense it is the same problem as the main theme - you are something I perceive, just as you might be something I remember. What matters, we might like to think, is whether or not I am real, whether or not I make my own choices of significance - and as pertains to you, whether or not you (real or not) have a significant role in my reality.

And, I think, K is left in a similar place with both versions of this problem; Joi had an effect on him, it seems clear to me, encouraging him and helping to destabilize him towards his ultimate fate, just as his memories did. But whereas (his) Joi is destroyed and lost to him, his memories - "all the best ones", at least - survive even when he dies, because they also belong to others. This happens to be an inversion of Roy Batty's famous observation that his memories will be lost with him.

Ok, enough!

IIT & Pacific Rim

I'm going to start posting short observations of how IIT would explain or be problematic for certain ideas in sci-fi movies or books.

To start: The film "Pacific Rim", a sci-fi action movie where the main characters are pilots controlling gigantic robots. The pilots control the robot through a direct brain-machine interface, but the job is apparently too much for one pilot so there are always at least two pilots. The two pilots have their minds joined by a "neural bridge" - basically an artificial corpus callosum. While joined, the pilots seem to have direct access to one another's experiences in a merged state called "the Drift" - it seems that their two consciousnesses become one.

This scenario is the predicted consequence, according to IIT, of sufficient causal linkage between two brains - at some point, the connection is sufficiently complex that the local maximum of integrated information is no longer within each pilot's brain, but now extends over both brains. What would be necessary to achieve this? The movie doesn't attempt to explain how the brain-machine interface works, but it must involve a very high-resolution, high-speed parallel system for both responding to and stimulating neurons in each pilot's brain.

One way of doing this would be cortical implants, where high-resolution electrode arrays are installed on the surface of each pilot's brain; this is at least plausible (if not possible) given existing technology. However, none of the pilots show signs of a brain implant, and the main character Mako Mori seems to become a pilot on pretty short notice, although she has apparently been training for a long time - maybe all trainees are implanted? A big commitment.

A more hand-wavy Star Trek kind of technology would involve some kind of transcranial magnetic field system that is powerful, precise, and fast enough to both stimulate individual neurons (current TMS systems certainly cannot do this) and measure their activity on a millisecond timescale (current fMRI systems absolutely cannot do this); however, the pilots simply wear helmets while piloting the robots (although Dr Newton, who almost certainly does not have any brain implants since he is not a trained pilot, does use some kind of transcranial setup to drift with a piece of monster brain), which I think makes a transcranial system very unlikely.

If I had to guess, wireless cortical implants are the only plausible means of establishing the Pacific Rim neural bridge, but some sort of transcranial system hidden in the pilots' helmets and based on some unimaginable technology is not excluded.

Verdict: Pacific Rim's "drift" is IIT Compatible

challenge

Irwin Borish versus Edwin Boring.

Go!

crastinus

Just need to get that 'idiot' post off the top. How's it going?

Lots of news!

Not going to tell you any of it, though. Wait till later.

-andrew

sports

The above is a 3D multidimensional scaling representation of the similarity of 14 popular sports. At least 3 dimensions are necessary to represent the similarity matrix; here I've rotated the plot so as to make the different points (sports) clear.

Euclidean distance between the sports is closely related to their similarity, but since this is a 2D projection of three coordinates, I also included edges (lines) between the sports as an extra cue to distance. The thickness and coloring of the edges is mapped to the similarity values (red/thin is "dissimilar", while blue/thick is "similar"). For each sport edges were drawn to the three nearest neighbors - sometimes those were very similar (e.g. the hockey triangle), sometimes very distant (baseball is very close to cricket, but it's next nearest neighbors are very far away: golf and ping pong).

Okay, so what constitutes "similarity" here? I made a list of 28 "sports properties" based entirely on my own subjective knowledge of the included sports. Undoubtedly a better list could be composed, but this is an okay first approximation. The set of properties works as a "present or not" binary value for each sport.

IIT and Sci-Fi

(Fake date on this post - wrote it in 2015, not sure of the date - back-posting it here)

This isn't a coherent post - just a list of some random thoughts having to do with IIT and science fiction scenarios that I'm familiar with. I can't claim to have any strength in the latter category: I'm not a big sci-fi reader (especially not in the last few years), and even the best sci-fi movies (I watch 20 times more movies than I read novels) don't tend to take you very far (excepting good time travel movies).

I just think that some of these examples (or others that you, dear reader, might think of) could help in illustrating some interesting IIT-based possibilities.

1. The book Foundation's Edge, by Isaac Asimov, ends on a question about a "supermind", a world where all life - human and otherwise - shares a single planetary consciousness. But it's something earlier in the book that has stuck with me; when the main character, Golan Trevize, is exiled from planet Terminus, he is given an advanced starship. Trevize is confused by the lack of traditional controls - instead, he learns that the ship is piloted by connecting a user's mind to the ship itself. In doing so, Trevize finds that he is now aware of not only the ship's control system, but his perceptual awareness now seems to include the space surrounding the ship: his consciousness now includes the ship:

"And as he and the computer held hands, their thinking merged and it no longer mattered whether his eyes were open or closed. Opening them did not improve his vision nor did closing them dim it. 

Either way, he saw the room with complete clarity—not just in the direction in which he was looking, but all around and above and below. 

He saw every room in the spaceship and he saw outside as well. The sun had risen and its brightness was dimmed in the morning mist, but he could look at it directly without being dazzled, for the computer automatically filtered the light waves. 

He felt the gentle wind and its temperature, and the sounds of the world about him. He detected the planet's magnetic field and the tiny electrical charges on the wall of the ship. 

He became aware of the controls of the ship, without even knowing what they were in detail. He knew only that if he wanted to lift the ship, or turn it, or accelerate it, or make use of any of its abilities, the process was the same as that of performing the analogous process to his body. He had but to use his will."

In IIT terms, the connection between his mind and the ship's systems has been made that allows states of the ship's electronics to causally constrain states of Trevize's brain - at the same time that states of his brain constrain those electronics. There is an irreducible global maximum in integrated information that extends outside Trevize's skull, and into the ship's computers.

Asimov doesn't elaborate on what it is "like" for the ship without a pilot - does it integrate information on its own, constituting a waiting ship's consciousness, ready always to link up with the pilot? If so, the link is like an artificial corpus callosum, joining two independent minds into one whole, and we can suppose that the ship persists as a conscious agent even in the absence of a pilot. Or maybe the ship is disintegrated without a human component - in a state something like slow-wave sleep, with systems online but carefully disconnected in such a way that without a pilot, there is no "ship" - just a collection of parts and mechanisms, like an ordinary machine.

2. Speaking of artificial corpus callosums, there is the excellent Pacific Rim to consider.

coming back online

I'm back!

What to report?

Hm..

You're back without a report?

I might need time to prepare one.

We're hoping for regular reports, you know.

Would you take a picture?

We don't usually publish pictures, but let's see what you have.

Ah, well...

No pictures?

I thought you could take mine.

I see.

Next time I'll have something.

Don't wait six months again, okay?

I'll try.

See you soon.

habitat

Made this last winter when I realized I was coming to Australia. Found it the other day and dressed it up a bit. From Yarra to Yahara, yep.

shouting into the wind

​I usually like io9, but this was pretty stupid, and yet it got a big response. It's a blurb on an essay, "On the Emptiness of Failed Replications", by Jason Mitchell, a social psychologist/neuroscientist, where he's criticizing some of the current discussion in the social sciences (and peripheral quarters). I've found other severely critical discussions of the essay, and none positive, although the negatives are (it seems) already entrenched in their opinions about how things *must* be done. Given my habitual uncertainty about everything, I think my take is fairly objective.

I haven't read most of the comments on the io9 article (just scanned the first page's worth), but it seems they are mostly agreeing with the negative post, and most of them don't discuss the essay *at all*. A lot of them are just bland 'social science is pseudoscience' stuff, but they're letting this blurb (assuming most of them didn't *read* the essay) feed their preconceptions. The comments on other blog posts are similar, though the quality of the posts themselves are generally better (if pedantic). I'm going to do some apologia here, since I'm avoiding working on a paper (or papers) of my own.

Mitchell's writing about the replication push, and explaining why, in basic philosophy-of-science terms (i.e. in terms of falsificationism - cf. previous post) replication as standard practice is not *scientific* practice. **Searching for null effects is meaningless**, he says, because there are vastly more ways to do an experiment wrong, and fail to detect an effect, than there are ways to do it properly. The scientific way to challenge a finding is not to try to reproduce it by following the steps in a methods section - it's by finding out how such a finding *could* arise, and explaining that in functional terms. The worst case is that the finding arises through fraud, and Mitchell goes pretty far in pointing out how a replication push can take the form of a witch-hunt.

Of course many science-minded folk are biased against the social sciences, in part because there have been a number of prominent frauds recently, but also because the methods are hard to discern. Social psychology, and most of experimental psychology at that, are different from the 'hard' sciences in important ways, but they are still scientific practices. It's not pseudoscience to say that you can measure a person's thoughts or perceptions or feelings or predispositions, although the measurements can be done well or done poorly.

There is one fundamental difference between the social sciences and the hard sciences (physics, chemistry, biology): in studying a human mind, you have to communicate with it, and every human mind is different - even a single human mind is different from day-to-day. Running a psychophysics or cognitive experiment effectively requires that you take this person, quickly figure out the contours of their personality (in a quick discussion or screening interview), and set their thought processes in such a way that you can *then do the experiment*. And furthermore, especially when it comes to in-depth studies like in real psychophysics, some people simply can't do the experiments; there are good psychophysics subjects and bad psychophysics subjects (in my experience, you might lump these into "people who are conscientious and have a high capacity for introspection", and "people who don't listen and who get bored by themselves"), and part of doing these experiments well is weeding out the bad ones. Usually it's pretty easy - a subject does some training blocks that produce crummy data - you try to explain to them how to improve their performance; they fail to improve, and you fire them. The foregoing steps are an art, the practice of communication and guidance and control, and some people are better at it than others. This is a relatively minor point in the essay, which is focused on replication, but it seems to have caught a lot of attention from the 'social science isn't science' folks.

I say all this as a social scientist, under general definitions. I'm a psychophysicist, a neuroscientist, and an experimental psychologist - most of my research is quantitative and computational and model-driven, with human beings (their behavior or their brains) as the source of my data. I completely recognize the situation that Mitchell is describing - I've had to explain these issues to juniors and colleagues many times over the years, though I don't know if I'm always convincing. I thought it was a great essay, and worth passing on. And, at the end of the day, I check the box 'psychologist', which puts me in the camp under attack, and so I feel I need to raise my tiny voice in defense of Dr Mitchell. Not that anyone comes here to read what I write, but I like to remember things, so.. meh.