Monday, November 26, 2012

blur or no blur?

Some notes on the aftereffects of a paper revision I just submitted (not coincidentally linked to the rambling at the end of the previous entry):

The big problem I have left over after the last revision of the blur adapt paper is this: does it mean anything? I've wound up half convinced that while I have a good explanation for a complex and strange phenomenon, it may be seen as boiling down just to a measurement, by visual system proxy, of the stimuli themselves. That is, all the stuff about selectivity, slope changing, symmetry of adaptation, etc., might all just be a figment of the wholly unnatural way of blurring/sharpening images that we've used.

What's left? The method is good. There are also questions about the spatial selectivity of the phenomenon, and, most importantly I think, about its timecourse. If blur adaptation is something real and not just a spandrel interaction between contrast adaptation and strange stimuli, it doesn't make a lot of sense that it would manifest in everyone in the same way unless it did have some sort of perceptual utility. The utility that exists is a good question. Let's make a list:

1. Changes in fixation across depths. Most of the people who do these experiments are young and have good accommodation. Blur is one of the things that helps to drive accommodation, to the point where if everything is working correctly, within a few hundred (less?) milliseconds of changing fixation in depth, the image should be focused. So, blur adaptation would not be useful in this situation. Maybe it's useful when you're older, and for this reason it sits there, functional and in wait, for the lens to freeze up? Seems unlikely and implausible, but possible. When you get old, and look at different depths, the sharpness of the image will change, and it would be nice to have some dynamic means of clawing back whatever high s.f. contrasts might still be recoverable in a blurred image.

2. This begs the question of how much can be recovered from an image blurred locally. That is, the slope-change method is basically using an image-sized psf, which is what makes it so weird. Blur doesn't usually occur this way, instead it occurs by a spatially local psf applied to the image, like a gaussian filter. If an image is gaussian blurred, how much can it be sharpened?

3. Viewing through diffusive media, like gooey corneas or fog or rain, or muddy water. The latter phenomena, if I'm not mistaken, affect contrast at all frequencies, while stuff-in-the-eyes effects optical blur, i.e. more attenuation at high than at low frequencies. It would be nice to know, in detail, what types of blur or contrast reduction (it might be nice to reserve 'blur' for the familiar sense of high s.f. reduction) occur ecologically. We also have dark adaptation, where the image is sampled at a lower rate but is also noisier. The noise is effectively a physical part of the retinal image (photon, photochemical, neural), meaning that it's local like an optical defect and not diffusive like fog. Maybe blur adaptation is mostly good for night vision?

4. Television. CRTs. Maybe we're all adapted, long-term and dynamically, to blurred media. All captured and reproduced media are blurred. CRTs were worse than current technology, resulting in displayed images that were considerably blurrier than the transmitted images, which themselves were blurred on collection and analog transmission. Digital images are blurred on collection, although light field cameras seem to be getting around this, and digital displays are physically much less blurred. Maybe those of us who grew up watching CRT images, and accepting them as a special sort of normal, adapt more than the young people who are growing up with high-resolution LCD images?

5. Texture adaptation, i.e. adaptation to the local slope of the amplitude spectrum, i.e. exactly what is being manipulated in the experiments. This would be fine. Testing it would be a bit different; subjects would need to identify the grain or scale of a texture, something like that. I think that the materials perception people have done things like this. Anyways, this sort of adaptation makes sense. You might look at an object at a distance and barely be able to tell that its surface has a fine-grain texture, so a bit of local adaptation would allow you, after a few seconds, to see those small details. On the other hand, if you get in really close to the object so that the texture is loud and clear, and you can even see the texture of the elements of the larger texture, especially if there's a lot of light and the texture elements are opaque, this is effectively a much sharper texture than what you were seeing before, even within the same visual angle. The 1/f property of natural images is an average characteristic. Locally, images are lumpy in that objects represent discontinuities; textures on surfaces usually have a dominant scale, e.g. print on a page has a scale measured in points, and that will show up as a peak in the amplitude spectrum. So, texture adaptation, where the system wants to represent detail, seems like a plausible function for what we're calling blur adaptation. Maybe the system should work better somehow if images are classed in this way?

6. Parafoveal or 'off-attention' defocus. We almost always fixate things that are sharp, but if the fixated object is small, whatever is behind it will be blurred optically. Similar situation if the fixated object is viewed through an aperture, the aperture will be blurred. Whatever adaptation occurs in this situation must be passive, just contrast adaptation, as I can't imagine that there's much utility to the small gain in detail with adaptation to a gaussian blur.

For all of these situations, spatial selectivity makes sense but is not necessary. Even if you're viewing a scene through fog, nearby objects will be less fogged than faraway objects, but it all depends on where you're fixating; other object at different depths will be more or less fogged. At any rate, foveal or parafoveal adaptation is most important, as peripherally viewed details are, as far as I can understand, subordinate. If the process is spatially localized, as it should be if it is what it seems to be, then global adaptation is just a subset of all possible adaptation configurations. Temporal selectivity is more questionable. If the process is genuine, and not just broadband contrast adaptation (though this begs the question of what should the timecourse be for contrast adaptation), how fast should we expect it to be? If it's mostly used for long-term (minutes) activities (fixating muddy water, looking for fish; other veiling glare situations; gooey eyeball; accommodation failure), maybe it could stand to be slower, with a time constant measured in seconds, or tens of seconds. If it's mostly used for moment-to-moment changes in fixated structure, i.e. texture adaptation or depth (off-attention), it should be fast, with a time constant measured in hundreds of milliseconds.

Actually measuring the temporal properties of the adaptation might therefore help to some degree in understanding what the process is used for.

Sunday, November 25, 2012

reading about history

An idle essay on history, for the holiday!

In the past couple of years, a good portion of my recreational reading has been history, and some of that has been by ancient historians: Plutarch, Sima Qian, Livy. For the past few weeks, I've been alternating between two books, a collection of abridged Livy (from the Ad Urbe Condita) and Wittgenstein's Tractatus Logico-Philosophicus, which is a study of how language is connected, or can potentially be connected, to reality. I'm not claiming to fully understand the Wittgenstein, but there has been an interaction.

There are lots of reasons why reading history is enjoyable. The main reason, for me, is that it is so edifying: you are learning how the world came to be the way that it is, and you're also learning about certain constancies of the human condition, mistakes and actions and etc that have been repeated over and over again for thousands of years. Another reason is that it is entertaining in the same way that reading fiction is entertaining: there are heroes and villains, victories and tragedies, and all of it is ambiguous and complex, at least in hindsight.

What the TLP made me think about was this (although not in the confusing terms of propositions, pictures, facts, etc): the page one reads is a surface into which has been pressed different shapes. When one reads, one is feeling these shapes, and mentally reconstructing whatever it was that impressed them. When one reads fiction, the impressor is, supposedly, always secondhand, in that it is the mind of the author that has been impressed, and the author has reconstructed ideas based on those impression, recombined them into mental realities, and then created new impressions based on those mental realities in the page. One then uses those impressions to reconstruct the author's mental realities. Since these reconstructions are not based on physical reality, they constitute in the language of the TLP false facts (although, strictly, many of the components of these false facts must be true; a falsehood cannot be sensible if it is not seemingly possible, its possibility being dictated by the local truth of its parts).

When one reads history, then the intention is that physical reality is impressed into the page, and that when one reads history and reconstructs his own mental realities, these should be (or be close to being) true facts. This is the intention of the honest historian, but he must inevitably fail, because he cannot base all, or even most, of his impressions on physical realities. Historians gain their knowledge by reading what was written by others before them, and then they compile what they have read into narratives that can be understood holistically by others. The historian must judge what are true and false facts, and impress only the true facts. Since other writers may not have thought of themselves as historians, and may not have been intent on impressing true facts, these judgments will be difficult, and the historian will sometimes fail.

So, when reading history as a naive consumer of text like myself, one is in the interesting situation of feeling out these impressions and forming mental reconstructions of the impressors, which are actually impressions of reconstructions of impressors that are actually reconstructions themselves and et cetera. Some of the impressions are mostly true (with local falsehoods), and some must be mostly false (with local truths). It's like Indiana Jones and the Holy Grail, except there's no reward or punishment for deciding that one or another fact is true or false. In reading fiction, the decision is implicit in the definition, but in reading history, you get the sense of walking along the true side of a very fuzzy edge, a transition into the false side. This transition gets broader and broader the further back in time one goes.

This then gets back to another issue which I'd like to write about sometime: the ubiquity of blur. All systems for transmitting information lose local details before they lose fundamentals. High spatial frequencies are lost in image formation; high temporal frequencies are lost as sound travels through a medium; sharp edges on an object are worn down by friction over time; genetic mutations effect molecular changes in the phenotype; and the details of history - names, dates, the precise unfolding of events - are misremembered or, mostly, forgotten. These are details in the literary sense, but they seem exactly analogous to physical details: what happened in Caesar's final days? Did he go to the forum in spite of warnings? Was Brutus really his son? These sorts of details, the answers to these sorts of questions, are permanently forgotten, but we know the larger, deeper, important events: Caesar was murdered by a conspiracy of Senators.

Interestingly, in the same way that a knife might be sharpened, or faded images might be retouched, old stories about the past might be sharpened up with added details; doped with false facts, to bring them into narrative focus. Caesar was warned about the Ides of March; he saw Brutus and said, "You too, my son?" The doping could also be with irrelevant facts: this is what you could buy from a street vendor in those days, this is what the men and women of this station wore on their feet. This sharpening, false or irrelevant, is enjoyable in a special way when it comes from someone who was writing more than 2000 years ago, because it is more immediate: nothing (except for the translator) has touched these impressions since they were formed. It's like holding something very, very old in your hands.

Sunday, November 18, 2012

finally a post about those eye crank lines

has anyone ever tested basic visual psychophysics as a function of gaze direction? i don't think so. would it be interesting or important to do so? i think so.

1. when i crank my eyes out as far as i can, i see weird phosphene patterns around my foveae (below). nobody has given me a good explanation for what these phosphenes are, except that they are probably produced by some sort of tension or torsion on the optic nerve. this isn't much of an explanation, because the phosphenes are so local and fine that if it was torsion i would expect them to be everywhere. it could be the correct explanation, but then i need an explanation for why they aren't everywhere, or what is special about foveal optic nerve fibers etc etc in their placement in the optic nerve. the sort of thing i guess i could figure out from reading.
whatever the cause of this effect, it means that in the extreme, direction of gaze has an effect on low-level perception, i.e. i am seeing spatial phosphenes - which, really, look like band-pass patterns - and not hallucinating faces or whatever. so, it stands to reason that less extreme directions might also have effects that are more subtle.

anyways, i hope i am not tearing apart my optic nerves by doing this experiment. i try not to do it too often, but it's like thinking about reciting pi. when you think about reciting pi, you have to recite as many digits as you can remember. you can't stop. give me a second.

2. if e.g. contrast sensitivity is entirely determined by retinotopically coordinated visual mechanisms - i.e. retina, LGN, V1, striate cortex - direction of gaze shouldn't make any difference, because these areas don't know anything about direction of gaze. but visual areas in the parietal cortex do know about direction of gaze - areas like LIP and VIP combine input from the visual system, of such quality that it is used to plan eye movements, with proprioceptive, vestibular, motor, and other inputs.

it's implicit in the theory of psychophysics - the theory that physical stimuli are translatable into perceptual states, which are then behaviorally accessible - that the last stage of vision is motor, since no psychophysics can be done without motor responses. this is one reason why neuroimaging is not psychophysics.

so, if vision interacts with non-visual inputs, and if these same inputs mediate behavioral measurement of visual ability - i.e. psychophysics - then is it reasonable to suppose that direction of gaze should affect basic visual abilities? a good hypothetical mechanism for producing an effect would be the internal noise source. no one should suppose that the noise limiting performance is entirely visual, because this assumes that the rest of the system is deterministic, which it is not. since the rest of the system is not deterministic, the portion of the random variation that is contributed by the parietal cortex might well vary with the tonic motor state of the system; the part of the brain that is guiding or maintaining the motor aspects of the system, and mediating the responses of the system according to the experiment design, might be better adapted or learned in one gaze state than in others.

3. visual neglect. i guess this is a higher-level thing, but from what i've heard, it's independent of basic sensitivity; how could this have been confirmed? how can basic testing be carried out with the same quality in the neglect region as in the unaffected region? this sounds like something that's been tried over and over, and that i could go read about. a quick survey of some titles, abstracts, and a couple of the most relevant-sounding papers suggests that when such sensitivity has been measured, its in the non-neglect areas, but that the researchers are nonetheless looking for a connection. there's a paper where they suggest there's no difference in contrast sensitivity or s.f. discrimination between two groups of stroke patients, some with neglect symptoms, some without; that could mean that even a stroke big enough to cause neglect, while sparing early visual cortex, won't bother basic sensitivity, or that any serious enough stroke will impair sensitivity on basic tasks. hm...

Thursday, November 15, 2012

stack puzzle

Okay, I’ve been wondering for a while whether or not something is a valid question – a good question or a bad question. It is related to a few entries I’ve written here in the past year (esp. this and this), and to a paper that I’m about to get ready for submission.

The question: are the percepts contributed by different layers or modules of visual processing perceived as embedded within one another, or as layered in front of or behind one another?

Such percepts could include brightness, location and sharpness of an edge, its color, its boundary association; color and shape and texture of a face, its identity, its emotional valence, its association with concurrent speech sounds; scale of a texture, its orientation, its angle relative to the frontal plane, its stereoscopic properties.

All of these, and more, are separately computed properties of images as they are perceived, separate in that they are computed by different bits of neural machinery at different parts of the visual system hierarchy. Yet, they are all seen together, simultaneously, and the presence of one implies another. That is, to see an edge implies that it must have some contrast, some color, some orientation, some blur; but this implication is not trivial. That is, a mechanism that senses an edge does not need to signal contrast or color or orientation or scale; the decoder could simply interpret the responses of the mechanism as saying ‘there is an edge here’. To decode the orientation of an edge requires that many such mechanisms exist, each preferring different orientations, and that some subsequent mechanism exists which can discriminate the responses of one from another, i.e. the fact that the two properties are both discriminable (edge or no; orientation) means that there must be a hierarchy, or that there must be different mechanisms.

So, whenever something is seen, the seeing of the thing is the encoding of the thing by many, many different mechanisms, each of which has a special place in the visual system, a devoted job – discriminate orientation, discriminate luminance gradients, discriminate direction of motion, or color, etc.

So, although we know empirically and logically that there must be different mechanisms encoding these different properties, there is no direct perceptual evidence for such differences: the experience is simultaneous and whole. In other words, the different properties are bound together; this is the famous binding problem, and it is the fundamental problem of the study of perception, and of all study of subjective psychology or conscious experience.

This brings us to the question, reworded: how is the simultaneity arranged? From here, it is necessary to adopt a frame of reference to continue discussion, so I will adopt a spatial frame of reference, which I am sure is a severe error, and which is at the root of my attempts so far to understand this problem; it will be necessary to rework what comes below from different points of view, using different framing metaphors.

Say that the arrangement of the simultaneous elements of visual experience is analogous to a spatial arrangement. This is natural if we think of the visual system as a branching series of layers. As far as subjective experience goes, are ‘higher’ layers in front of or behind the ‘lower’ layers? Are they above or below? Do they interlock like... it is hard to think of a metaphor here. When do layers, as such, interlock so that they form a single variegated layer? D* suggested color printing as something similar, though this doesn’t quite satisfy me. I imagine a jigsaw puzzle where the solution is a solid block, and where every layer has the same extent as the solution but is mostly empty space. D* also mentioned layers of transparencies where on each layer a portion of the final image – which perhaps occludes lower parts – is printed; like the pages in the encyclopedia entry on the human body, where the skin, muscles, organs, bones, were printed on separate sheets.

But after some thought, I don't think these can work. An image as a metaphor for the perceptual image? A useful metaphor would have some explanatory degrees of freedom; one set of things that can be understood in one way, used to understand something different in a similar way. Where do we get by trying to understand one type of image as another type of image? Not very far, I think. The visual field is a sort of tensor: at every point in the field, multiple things are true at the same time, they are combined according to deterministic rules, and a unitary percept results. Trying to understand this problem in terms of a simpler type of image seems doomed to fail.

So, whether or not there is a convenient metaphor, I think that the idea of the question should be clear: how are the different components of the percept simultaneously present? A prominent part of psychophysics studies how different components interact: color and luminance contrast, or motion and orientation, but my understanding is that for the most part different components are independently encoded; i.e. nothing really affects the perceived orientation of an edge, except perhaps the orientations of other proximal (in space or time) edges.

Masking, i.e. making one thing harder to see by laying another thing in proximity to it, is also usually within-layer, i.e. motion-to-motion, or contrast-to-contrast. Here, I am revealing that my thinking is still stuck in the lowest levels: color, motion, contrast, orientation, are all encoded together, in overlapping ensembles. So, it may well be that a single mechanism can encode a feature with multiple perceptual elements.

Anyways, the reason why I wonder about these things is, lately, because of this study where I had subjects judge the contrast of photographic images and related these judgments to the contrasts of individual scales within the images. This is related to the bigger question because there is no obvious reason why the percept contrast of a complex, broadband image should correspond to the same percept contrast of a simple spatial pattern like a narrowband wavelet of one type or another. This is where we converge with what I have written a few months ago: the idea of doing psychophysics with simple stimuli is that a subject’s judgments can be correlated with the physical properties of the stimuli, which can be completely described because they are simple. When the stimuli are complex and natural, there is a hierarchy of physical properties for which the visual system is specifically designed, with its own hierarchy, to analyze. Simple stimuli target components of this system; complex stimuli activate the entire thing.

It is possible that when I ask you to identify the contrast – the luminance amplitude – of a Gabor patch, you are able to do so by looking, from your behavioral perch, at the response amplitude of a small number of neural mechanisms which are themselves stimulated directly by luminance gradients, which are exactly what I am controlling by controlling the contrast of the Gabor. It is not only possible, but this is the standard assumption in most contrast psychophysics (though I am suspicious that the Perceptual Template people have fuzzier ideas than this, I am not yet clear on their thinking – is the noisiness of a response also part of apparent magnitude?).

It is also possible that when I ask you to identify the contrast of a complex image, like a typical sort of image you look at every day (outside of spatial vision experiments), you are able to respond by doing the same thing: you pool together the responses of lots of neural mechanisms whose responses are determined by the amplitude of luminance gradients of matched shape. This is the assumption I set out to test in my experiment, that contrast is more or less the same, perceptually, whatever the stimulus is.

But, this does not need to be so. This assumption means that in judging the contrast of the complex image, you are able to ignore the responses of all the other mechanisms that are being stimulated by the image: mechanisms that respond to edges, texture gradients, trees, buildings, depth, occlusions, etc. Why should you be able to do this? Do these other responses not get in the way of ‘seeing’ those more basic responses? We know that responses later in the visual hierarchy are not so sensitive to the strength of a stimulus, rather they are sensitive to the spatial configuration of the stimulus; if you vary how much the configuration fits, you will vary the response of the neuron, but if you vary its contrast you will, across some threshold, turn the neuron on and off.

I don’t have a solution; the question is not answered by my experiment. I don’t doubt that you can see the luminance contrast of the elements in a complex scene, but I am not convinced that what you think is the contrast is entirely the contrast. In fact, we know for certain that it is not, because we have a plethora of lightness/brightness illusions.

No progress here, and I'm still not sure of the quality of the question. But, maybe this way of thinking can make for an interesting pitch at the outset of the introduction of the paper.

Thursday, November 08, 2012

meh

I keep noticing, lately, near- and far-peripheral flashes, phosphenes. There was one a few minutes ago, maybe 10deg below fovea, very obvious and yet hard to localize (it was almost as though it extended close to the fovea); I checked for a blindspot, found none. So, based on all these recent sparky things, I predict something happening in the next few days (especially since the BA paper is just about done, so I'm about to go through another relax-contract phase).

Monday, November 05, 2012

probability/likelihood

probability of an event occurring
likelihood of a condition existing

how often have i misused these words? unknown.

if i search this blog, i find one clear misuse of 'likelihood', in "a specific instantiation..." ("likelihood of a pass"); i use the term two other times, in the first post ever and in a later one that refers to that one. in those instances (likelihood that you are alive at age x) i think it's ambiguous, but i'll count those as accurate.

amazingly, there is only one post where i use the word 'probability' in the relevant context. and i can't tell if i'm using it right or not. i'm kind of disappointed in myself.

seems i've fallen off the HAZ-PJ wagon. writing is going well, though, which is good.