Some notes on the aftereffects of a paper revision I just submitted (not coincidentally linked to the rambling at the end of the previous entry):
The big problem I have left over after the last revision of the blur adapt paper is this: does it mean anything? I've wound up half convinced that while I have a good explanation for a complex and strange phenomenon, it may be seen as boiling down just to a measurement, by visual system proxy, of the stimuli themselves. That is, all the stuff about selectivity, slope changing, symmetry of adaptation, etc., might all just be a figment of the wholly unnatural way of blurring/sharpening images that we've used.
What's left? The method is good. There are also questions about the spatial selectivity of the phenomenon, and, most importantly I think, about its timecourse. If blur adaptation is something real and not just a spandrel interaction between contrast adaptation and strange stimuli, it doesn't make a lot of sense that it would manifest in everyone in the same way unless it did have some sort of perceptual utility. The utility that exists is a good question. Let's make a list:
1. Changes in fixation across depths. Most of the people who do these experiments are young and have good accommodation. Blur is one of the things that helps to drive accommodation, to the point where if everything is working correctly, within a few hundred (less?) milliseconds of changing fixation in depth, the image should be focused. So, blur adaptation would not be useful in this situation. Maybe it's useful when you're older, and for this reason it sits there, functional and in wait, for the lens to freeze up? Seems unlikely and implausible, but possible. When you get old, and look at different depths, the sharpness of the image will change, and it would be nice to have some dynamic means of clawing back whatever high s.f. contrasts might still be recoverable in a blurred image.
2. This begs the question of how much can be recovered from an image blurred locally. That is, the slope-change method is basically using an image-sized psf, which is what makes it so weird. Blur doesn't usually occur this way, instead it occurs by a spatially local psf applied to the image, like a gaussian filter. If an image is gaussian blurred, how much can it be sharpened?
3. Viewing through diffusive media, like gooey corneas or fog or rain, or muddy water. The latter phenomena, if I'm not mistaken, affect contrast at all frequencies, while stuff-in-the-eyes effects optical blur, i.e. more attenuation at high than at low frequencies. It would be nice to know, in detail, what types of blur or contrast reduction (it might be nice to reserve 'blur' for the familiar sense of high s.f. reduction) occur ecologically. We also have dark adaptation, where the image is sampled at a lower rate but is also noisier. The noise is effectively a physical part of the retinal image (photon, photochemical, neural), meaning that it's local like an optical defect and not diffusive like fog. Maybe blur adaptation is mostly good for night vision?
4. Television. CRTs. Maybe we're all adapted, long-term and dynamically, to blurred media. All captured and reproduced media are blurred. CRTs were worse than current technology, resulting in displayed images that were considerably blurrier than the transmitted images, which themselves were blurred on collection and analog transmission. Digital images are blurred on collection, although light field cameras seem to be getting around this, and digital displays are physically much less blurred. Maybe those of us who grew up watching CRT images, and accepting them as a special sort of normal, adapt more than the young people who are growing up with high-resolution LCD images?
5. Texture adaptation, i.e. adaptation to the local slope of the amplitude spectrum, i.e. exactly what is being manipulated in the experiments. This would be fine. Testing it would be a bit different; subjects would need to identify the grain or scale of a texture, something like that. I think that the materials perception people have done things like this. Anyways, this sort of adaptation makes sense. You might look at an object at a distance and barely be able to tell that its surface has a fine-grain texture, so a bit of local adaptation would allow you, after a few seconds, to see those small details. On the other hand, if you get in really close to the object so that the texture is loud and clear, and you can even see the texture of the elements of the larger texture, especially if there's a lot of light and the texture elements are opaque, this is effectively a much sharper texture than what you were seeing before, even within the same visual angle. The 1/f property of natural images is an average characteristic. Locally, images are lumpy in that objects represent discontinuities; textures on surfaces usually have a dominant scale, e.g. print on a page has a scale measured in points, and that will show up as a peak in the amplitude spectrum. So, texture adaptation, where the system wants to represent detail, seems like a plausible function for what we're calling blur adaptation. Maybe the system should work better somehow if images are classed in this way?
6. Parafoveal or 'off-attention' defocus. We almost always fixate things that are sharp, but if the fixated object is small, whatever is behind it will be blurred optically. Similar situation if the fixated object is viewed through an aperture, the aperture will be blurred. Whatever adaptation occurs in this situation must be passive, just contrast adaptation, as I can't imagine that there's much utility to the small gain in detail with adaptation to a gaussian blur.
For all of these situations, spatial selectivity makes sense but is not necessary. Even if you're viewing a scene through fog, nearby objects will be less fogged than faraway objects, but it all depends on where you're fixating; other object at different depths will be more or less fogged. At any rate, foveal or parafoveal adaptation is most important, as peripherally viewed details are, as far as I can understand, subordinate. If the process is spatially localized, as it should be if it is what it seems to be, then global adaptation is just a subset of all possible adaptation configurations. Temporal selectivity is more questionable. If the process is genuine, and not just broadband contrast adaptation (though this begs the question of what should the timecourse be for contrast adaptation), how fast should we expect it to be? If it's mostly used for long-term (minutes) activities (fixating muddy water, looking for fish; other veiling glare situations; gooey eyeball; accommodation failure), maybe it could stand to be slower, with a time constant measured in seconds, or tens of seconds. If it's mostly used for moment-to-moment changes in fixated structure, i.e. texture adaptation or depth (off-attention), it should be fast, with a time constant measured in hundreds of milliseconds.
Actually measuring the temporal properties of the adaptation might therefore help to some degree in understanding what the process is used for.
Monday, November 26, 2012
Sunday, November 25, 2012
reading about history
An idle essay on history, for the holiday!
In the past couple of years, a good portion of my recreational reading has been history, and some of that has been by ancient historians: Plutarch, Sima Qian, Livy. For the past few weeks, I've been alternating between two books, a collection of abridged Livy (from the Ad Urbe Condita) and Wittgenstein's Tractatus Logico-Philosophicus, which is a study of how language is connected, or can potentially be connected, to reality. I'm not claiming to fully understand the Wittgenstein, but there has been an interaction.
There are lots of reasons why reading history is enjoyable. The main reason, for me, is that it is so edifying: you are learning how the world came to be the way that it is, and you're also learning about certain constancies of the human condition, mistakes and actions and etc that have been repeated over and over again for thousands of years. Another reason is that it is entertaining in the same way that reading fiction is entertaining: there are heroes and villains, victories and tragedies, and all of it is ambiguous and complex, at least in hindsight.
What the TLP made me think about was this (although not in the confusing terms of propositions, pictures, facts, etc): the page one reads is a surface into which has been pressed different shapes. When one reads, one is feeling these shapes, and mentally reconstructing whatever it was that impressed them. When one reads fiction, the impressor is, supposedly, always secondhand, in that it is the mind of the author that has been impressed, and the author has reconstructed ideas based on those impression, recombined them into mental realities, and then created new impressions based on those mental realities in the page. One then uses those impressions to reconstruct the author's mental realities. Since these reconstructions are not based on physical reality, they constitute in the language of the TLP false facts (although, strictly, many of the components of these false facts must be true; a falsehood cannot be sensible if it is not seemingly possible, its possibility being dictated by the local truth of its parts).
When one reads history, then the intention is that physical reality is impressed into the page, and that when one reads history and reconstructs his own mental realities, these should be (or be close to being) true facts. This is the intention of the honest historian, but he must inevitably fail, because he cannot base all, or even most, of his impressions on physical realities. Historians gain their knowledge by reading what was written by others before them, and then they compile what they have read into narratives that can be understood holistically by others. The historian must judge what are true and false facts, and impress only the true facts. Since other writers may not have thought of themselves as historians, and may not have been intent on impressing true facts, these judgments will be difficult, and the historian will sometimes fail.
So, when reading history as a naive consumer of text like myself, one is in the interesting situation of feeling out these impressions and forming mental reconstructions of the impressors, which are actually impressions of reconstructions of impressors that are actually reconstructions themselves and et cetera. Some of the impressions are mostly true (with local falsehoods), and some must be mostly false (with local truths). It's like Indiana Jones and the Holy Grail, except there's no reward or punishment for deciding that one or another fact is true or false. In reading fiction, the decision is implicit in the definition, but in reading history, you get the sense of walking along the true side of a very fuzzy edge, a transition into the false side. This transition gets broader and broader the further back in time one goes.
This then gets back to another issue which I'd like to write about sometime: the ubiquity of blur. All systems for transmitting information lose local details before they lose fundamentals. High spatial frequencies are lost in image formation; high temporal frequencies are lost as sound travels through a medium; sharp edges on an object are worn down by friction over time; genetic mutations effect molecular changes in the phenotype; and the details of history - names, dates, the precise unfolding of events - are misremembered or, mostly, forgotten. These are details in the literary sense, but they seem exactly analogous to physical details: what happened in Caesar's final days? Did he go to the forum in spite of warnings? Was Brutus really his son? These sorts of details, the answers to these sorts of questions, are permanently forgotten, but we know the larger, deeper, important events: Caesar was murdered by a conspiracy of Senators.
Interestingly, in the same way that a knife might be sharpened, or faded images might be retouched, old stories about the past might be sharpened up with added details; doped with false facts, to bring them into narrative focus. Caesar was warned about the Ides of March; he saw Brutus and said, "You too, my son?" The doping could also be with irrelevant facts: this is what you could buy from a street vendor in those days, this is what the men and women of this station wore on their feet. This sharpening, false or irrelevant, is enjoyable in a special way when it comes from someone who was writing more than 2000 years ago, because it is more immediate: nothing (except for the translator) has touched these impressions since they were formed. It's like holding something very, very old in your hands.
In the past couple of years, a good portion of my recreational reading has been history, and some of that has been by ancient historians: Plutarch, Sima Qian, Livy. For the past few weeks, I've been alternating between two books, a collection of abridged Livy (from the Ad Urbe Condita) and Wittgenstein's Tractatus Logico-Philosophicus, which is a study of how language is connected, or can potentially be connected, to reality. I'm not claiming to fully understand the Wittgenstein, but there has been an interaction.
There are lots of reasons why reading history is enjoyable. The main reason, for me, is that it is so edifying: you are learning how the world came to be the way that it is, and you're also learning about certain constancies of the human condition, mistakes and actions and etc that have been repeated over and over again for thousands of years. Another reason is that it is entertaining in the same way that reading fiction is entertaining: there are heroes and villains, victories and tragedies, and all of it is ambiguous and complex, at least in hindsight.
What the TLP made me think about was this (although not in the confusing terms of propositions, pictures, facts, etc): the page one reads is a surface into which has been pressed different shapes. When one reads, one is feeling these shapes, and mentally reconstructing whatever it was that impressed them. When one reads fiction, the impressor is, supposedly, always secondhand, in that it is the mind of the author that has been impressed, and the author has reconstructed ideas based on those impression, recombined them into mental realities, and then created new impressions based on those mental realities in the page. One then uses those impressions to reconstruct the author's mental realities. Since these reconstructions are not based on physical reality, they constitute in the language of the TLP false facts (although, strictly, many of the components of these false facts must be true; a falsehood cannot be sensible if it is not seemingly possible, its possibility being dictated by the local truth of its parts).
When one reads history, then the intention is that physical reality is impressed into the page, and that when one reads history and reconstructs his own mental realities, these should be (or be close to being) true facts. This is the intention of the honest historian, but he must inevitably fail, because he cannot base all, or even most, of his impressions on physical realities. Historians gain their knowledge by reading what was written by others before them, and then they compile what they have read into narratives that can be understood holistically by others. The historian must judge what are true and false facts, and impress only the true facts. Since other writers may not have thought of themselves as historians, and may not have been intent on impressing true facts, these judgments will be difficult, and the historian will sometimes fail.
So, when reading history as a naive consumer of text like myself, one is in the interesting situation of feeling out these impressions and forming mental reconstructions of the impressors, which are actually impressions of reconstructions of impressors that are actually reconstructions themselves and et cetera. Some of the impressions are mostly true (with local falsehoods), and some must be mostly false (with local truths). It's like Indiana Jones and the Holy Grail, except there's no reward or punishment for deciding that one or another fact is true or false. In reading fiction, the decision is implicit in the definition, but in reading history, you get the sense of walking along the true side of a very fuzzy edge, a transition into the false side. This transition gets broader and broader the further back in time one goes.
This then gets back to another issue which I'd like to write about sometime: the ubiquity of blur. All systems for transmitting information lose local details before they lose fundamentals. High spatial frequencies are lost in image formation; high temporal frequencies are lost as sound travels through a medium; sharp edges on an object are worn down by friction over time; genetic mutations effect molecular changes in the phenotype; and the details of history - names, dates, the precise unfolding of events - are misremembered or, mostly, forgotten. These are details in the literary sense, but they seem exactly analogous to physical details: what happened in Caesar's final days? Did he go to the forum in spite of warnings? Was Brutus really his son? These sorts of details, the answers to these sorts of questions, are permanently forgotten, but we know the larger, deeper, important events: Caesar was murdered by a conspiracy of Senators.
Interestingly, in the same way that a knife might be sharpened, or faded images might be retouched, old stories about the past might be sharpened up with added details; doped with false facts, to bring them into narrative focus. Caesar was warned about the Ides of March; he saw Brutus and said, "You too, my son?" The doping could also be with irrelevant facts: this is what you could buy from a street vendor in those days, this is what the men and women of this station wore on their feet. This sharpening, false or irrelevant, is enjoyable in a special way when it comes from someone who was writing more than 2000 years ago, because it is more immediate: nothing (except for the translator) has touched these impressions since they were formed. It's like holding something very, very old in your hands.
Sunday, November 18, 2012
finally a post about those eye crank lines
has anyone ever tested basic visual psychophysics as a function of gaze direction? i don't think so. would it be interesting or important to do so? i think so.
1. when i crank my eyes out as far as i can, i see weird phosphene patterns around my foveae (below). nobody has given me a good explanation for what these phosphenes are, except that they are probably produced by some sort of tension or torsion on the optic nerve. this isn't much of an explanation, because the phosphenes are so local and fine that if it was torsion i would expect them to be everywhere. it could be the correct explanation, but then i need an explanation for why they aren't everywhere, or what is special about foveal optic nerve fibers etc etc in their placement in the optic nerve. the sort of thing i guess i could figure out from reading.
whatever the cause of this effect, it means that in the extreme, direction of gaze has an effect on low-level perception, i.e. i am seeing spatial phosphenes - which, really, look like band-pass patterns - and not hallucinating faces or whatever. so, it stands to reason that less extreme directions might also have effects that are more subtle.
anyways, i hope i am not tearing apart my optic nerves by doing this experiment. i try not to do it too often, but it's like thinking about reciting pi. when you think about reciting pi, you have to recite as many digits as you can remember. you can't stop. give me a second.
2. if e.g. contrast sensitivity is entirely determined by retinotopically coordinated visual mechanisms - i.e. retina, LGN, V1, striate cortex - direction of gaze shouldn't make any difference, because these areas don't know anything about direction of gaze. but visual areas in the parietal cortex do know about direction of gaze - areas like LIP and VIP combine input from the visual system, of such quality that it is used to plan eye movements, with proprioceptive, vestibular, motor, and other inputs.
it's implicit in the theory of psychophysics - the theory that physical stimuli are translatable into perceptual states, which are then behaviorally accessible - that the last stage of vision is motor, since no psychophysics can be done without motor responses. this is one reason why neuroimaging is not psychophysics.
so, if vision interacts with non-visual inputs, and if these same inputs mediate behavioral measurement of visual ability - i.e. psychophysics - then is it reasonable to suppose that direction of gaze should affect basic visual abilities? a good hypothetical mechanism for producing an effect would be the internal noise source. no one should suppose that the noise limiting performance is entirely visual, because this assumes that the rest of the system is deterministic, which it is not. since the rest of the system is not deterministic, the portion of the random variation that is contributed by the parietal cortex might well vary with the tonic motor state of the system; the part of the brain that is guiding or maintaining the motor aspects of the system, and mediating the responses of the system according to the experiment design, might be better adapted or learned in one gaze state than in others.
3. visual neglect. i guess this is a higher-level thing, but from what i've heard, it's independent of basic sensitivity; how could this have been confirmed? how can basic testing be carried out with the same quality in the neglect region as in the unaffected region? this sounds like something that's been tried over and over, and that i could go read about. a quick survey of some titles, abstracts, and a couple of the most relevant-sounding papers suggests that when such sensitivity has been measured, its in the non-neglect areas, but that the researchers are nonetheless looking for a connection. there's a paper where they suggest there's no difference in contrast sensitivity or s.f. discrimination between two groups of stroke patients, some with neglect symptoms, some without; that could mean that even a stroke big enough to cause neglect, while sparing early visual cortex, won't bother basic sensitivity, or that any serious enough stroke will impair sensitivity on basic tasks. hm...
1. when i crank my eyes out as far as i can, i see weird phosphene patterns around my foveae (below). nobody has given me a good explanation for what these phosphenes are, except that they are probably produced by some sort of tension or torsion on the optic nerve. this isn't much of an explanation, because the phosphenes are so local and fine that if it was torsion i would expect them to be everywhere. it could be the correct explanation, but then i need an explanation for why they aren't everywhere, or what is special about foveal optic nerve fibers etc etc in their placement in the optic nerve. the sort of thing i guess i could figure out from reading.
anyways, i hope i am not tearing apart my optic nerves by doing this experiment. i try not to do it too often, but it's like thinking about reciting pi. when you think about reciting pi, you have to recite as many digits as you can remember. you can't stop. give me a second.
2. if e.g. contrast sensitivity is entirely determined by retinotopically coordinated visual mechanisms - i.e. retina, LGN, V1, striate cortex - direction of gaze shouldn't make any difference, because these areas don't know anything about direction of gaze. but visual areas in the parietal cortex do know about direction of gaze - areas like LIP and VIP combine input from the visual system, of such quality that it is used to plan eye movements, with proprioceptive, vestibular, motor, and other inputs.
it's implicit in the theory of psychophysics - the theory that physical stimuli are translatable into perceptual states, which are then behaviorally accessible - that the last stage of vision is motor, since no psychophysics can be done without motor responses. this is one reason why neuroimaging is not psychophysics.
so, if vision interacts with non-visual inputs, and if these same inputs mediate behavioral measurement of visual ability - i.e. psychophysics - then is it reasonable to suppose that direction of gaze should affect basic visual abilities? a good hypothetical mechanism for producing an effect would be the internal noise source. no one should suppose that the noise limiting performance is entirely visual, because this assumes that the rest of the system is deterministic, which it is not. since the rest of the system is not deterministic, the portion of the random variation that is contributed by the parietal cortex might well vary with the tonic motor state of the system; the part of the brain that is guiding or maintaining the motor aspects of the system, and mediating the responses of the system according to the experiment design, might be better adapted or learned in one gaze state than in others.
3. visual neglect. i guess this is a higher-level thing, but from what i've heard, it's independent of basic sensitivity; how could this have been confirmed? how can basic testing be carried out with the same quality in the neglect region as in the unaffected region? this sounds like something that's been tried over and over, and that i could go read about. a quick survey of some titles, abstracts, and a couple of the most relevant-sounding papers suggests that when such sensitivity has been measured, its in the non-neglect areas, but that the researchers are nonetheless looking for a connection. there's a paper where they suggest there's no difference in contrast sensitivity or s.f. discrimination between two groups of stroke patients, some with neglect symptoms, some without; that could mean that even a stroke big enough to cause neglect, while sparing early visual cortex, won't bother basic sensitivity, or that any serious enough stroke will impair sensitivity on basic tasks. hm...
Thursday, November 15, 2012
stack puzzle
Okay, I’ve been wondering for a while whether or not something is a valid question – a good question or a bad question. It is related to a few entries I’ve written here in the past year (esp. this and this), and to a paper that I’m about to get ready for submission.
The question: are the percepts contributed by different layers or modules of visual processing perceived as embedded within one another, or as layered in front of or behind one another?
Such percepts could include brightness, location and sharpness of an edge, its color, its boundary association; color and shape and texture of a face, its identity, its emotional valence, its association with concurrent speech sounds; scale of a texture, its orientation, its angle relative to the frontal plane, its stereoscopic properties.
All of these, and more, are separately computed properties of images as they are perceived, separate in that they are computed by different bits of neural machinery at different parts of the visual system hierarchy. Yet, they are all seen together, simultaneously, and the presence of one implies another. That is, to see an edge implies that it must have some contrast, some color, some orientation, some blur; but this implication is not trivial. That is, a mechanism that senses an edge does not need to signal contrast or color or orientation or scale; the decoder could simply interpret the responses of the mechanism as saying ‘there is an edge here’. To decode the orientation of an edge requires that many such mechanisms exist, each preferring different orientations, and that some subsequent mechanism exists which can discriminate the responses of one from another, i.e. the fact that the two properties are both discriminable (edge or no; orientation) means that there must be a hierarchy, or that there must be different mechanisms.
So, whenever something is seen, the seeing of the thing is the encoding of the thing by many, many different mechanisms, each of which has a special place in the visual system, a devoted job – discriminate orientation, discriminate luminance gradients, discriminate direction of motion, or color, etc.
So, although we know empirically and logically that there must be different mechanisms encoding these different properties, there is no direct perceptual evidence for such differences: the experience is simultaneous and whole. In other words, the different properties are bound together; this is the famous binding problem, and it is the fundamental problem of the study of perception, and of all study of subjective psychology or conscious experience.
This brings us to the question, reworded: how is the simultaneity arranged? From here, it is necessary to adopt a frame of reference to continue discussion, so I will adopt a spatial frame of reference, which I am sure is a severe error, and which is at the root of my attempts so far to understand this problem; it will be necessary to rework what comes below from different points of view, using different framing metaphors.
Say that the arrangement of the simultaneous elements of visual experience is analogous to a spatial arrangement. This is natural if we think of the visual system as a branching series of layers. As far as subjective experience goes, are ‘higher’ layers in front of or behind the ‘lower’ layers? Are they above or below? Do they interlock like... it is hard to think of a metaphor here. When do layers, as such, interlock so that they form a single variegated layer? D* suggested color printing as something similar, though this doesn’t quite satisfy me. I imagine a jigsaw puzzle where the solution is a solid block, and where every layer has the same extent as the solution but is mostly empty space. D* also mentioned layers of transparencies where on each layer a portion of the final image – which perhaps occludes lower parts – is printed; like the pages in the encyclopedia entry on the human body, where the skin, muscles, organs, bones, were printed on separate sheets.
But after some thought, I don't think these can work. An image as a metaphor for the perceptual image? A useful metaphor would have some explanatory degrees of freedom; one set of things that can be understood in one way, used to understand something different in a similar way. Where do we get by trying to understand one type of image as another type of image? Not very far, I think. The visual field is a sort of tensor: at every point in the field, multiple things are true at the same time, they are combined according to deterministic rules, and a unitary percept results. Trying to understand this problem in terms of a simpler type of image seems doomed to fail.
So, whether or not there is a convenient metaphor, I think that the idea of the question should be clear: how are the different components of the percept simultaneously present? A prominent part of psychophysics studies how different components interact: color and luminance contrast, or motion and orientation, but my understanding is that for the most part different components are independently encoded; i.e. nothing really affects the perceived orientation of an edge, except perhaps the orientations of other proximal (in space or time) edges.
Masking, i.e. making one thing harder to see by laying another thing in proximity to it, is also usually within-layer, i.e. motion-to-motion, or contrast-to-contrast. Here, I am revealing that my thinking is still stuck in the lowest levels: color, motion, contrast, orientation, are all encoded together, in overlapping ensembles. So, it may well be that a single mechanism can encode a feature with multiple perceptual elements.
Anyways, the reason why I wonder about these things is, lately, because of this study where I had subjects judge the contrast of photographic images and related these judgments to the contrasts of individual scales within the images. This is related to the bigger question because there is no obvious reason why the percept contrast of a complex, broadband image should correspond to the same percept contrast of a simple spatial pattern like a narrowband wavelet of one type or another. This is where we converge with what I have written a few months ago: the idea of doing psychophysics with simple stimuli is that a subject’s judgments can be correlated with the physical properties of the stimuli, which can be completely described because they are simple. When the stimuli are complex and natural, there is a hierarchy of physical properties for which the visual system is specifically designed, with its own hierarchy, to analyze. Simple stimuli target components of this system; complex stimuli activate the entire thing.
It is possible that when I ask you to identify the contrast – the luminance amplitude – of a Gabor patch, you are able to do so by looking, from your behavioral perch, at the response amplitude of a small number of neural mechanisms which are themselves stimulated directly by luminance gradients, which are exactly what I am controlling by controlling the contrast of the Gabor. It is not only possible, but this is the standard assumption in most contrast psychophysics (though I am suspicious that the Perceptual Template people have fuzzier ideas than this, I am not yet clear on their thinking – is the noisiness of a response also part of apparent magnitude?).
It is also possible that when I ask you to identify the contrast of a complex image, like a typical sort of image you look at every day (outside of spatial vision experiments), you are able to respond by doing the same thing: you pool together the responses of lots of neural mechanisms whose responses are determined by the amplitude of luminance gradients of matched shape. This is the assumption I set out to test in my experiment, that contrast is more or less the same, perceptually, whatever the stimulus is.
But, this does not need to be so. This assumption means that in judging the contrast of the complex image, you are able to ignore the responses of all the other mechanisms that are being stimulated by the image: mechanisms that respond to edges, texture gradients, trees, buildings, depth, occlusions, etc. Why should you be able to do this? Do these other responses not get in the way of ‘seeing’ those more basic responses? We know that responses later in the visual hierarchy are not so sensitive to the strength of a stimulus, rather they are sensitive to the spatial configuration of the stimulus; if you vary how much the configuration fits, you will vary the response of the neuron, but if you vary its contrast you will, across some threshold, turn the neuron on and off.
I don’t have a solution; the question is not answered by my experiment. I don’t doubt that you can see the luminance contrast of the elements in a complex scene, but I am not convinced that what you think is the contrast is entirely the contrast. In fact, we know for certain that it is not, because we have a plethora of lightness/brightness illusions.
No progress here, and I'm still not sure of the quality of the question. But, maybe this way of thinking can make for an interesting pitch at the outset of the introduction of the paper.
The question: are the percepts contributed by different layers or modules of visual processing perceived as embedded within one another, or as layered in front of or behind one another?
Such percepts could include brightness, location and sharpness of an edge, its color, its boundary association; color and shape and texture of a face, its identity, its emotional valence, its association with concurrent speech sounds; scale of a texture, its orientation, its angle relative to the frontal plane, its stereoscopic properties.
All of these, and more, are separately computed properties of images as they are perceived, separate in that they are computed by different bits of neural machinery at different parts of the visual system hierarchy. Yet, they are all seen together, simultaneously, and the presence of one implies another. That is, to see an edge implies that it must have some contrast, some color, some orientation, some blur; but this implication is not trivial. That is, a mechanism that senses an edge does not need to signal contrast or color or orientation or scale; the decoder could simply interpret the responses of the mechanism as saying ‘there is an edge here’. To decode the orientation of an edge requires that many such mechanisms exist, each preferring different orientations, and that some subsequent mechanism exists which can discriminate the responses of one from another, i.e. the fact that the two properties are both discriminable (edge or no; orientation) means that there must be a hierarchy, or that there must be different mechanisms.
So, whenever something is seen, the seeing of the thing is the encoding of the thing by many, many different mechanisms, each of which has a special place in the visual system, a devoted job – discriminate orientation, discriminate luminance gradients, discriminate direction of motion, or color, etc.
So, although we know empirically and logically that there must be different mechanisms encoding these different properties, there is no direct perceptual evidence for such differences: the experience is simultaneous and whole. In other words, the different properties are bound together; this is the famous binding problem, and it is the fundamental problem of the study of perception, and of all study of subjective psychology or conscious experience.
This brings us to the question, reworded: how is the simultaneity arranged? From here, it is necessary to adopt a frame of reference to continue discussion, so I will adopt a spatial frame of reference, which I am sure is a severe error, and which is at the root of my attempts so far to understand this problem; it will be necessary to rework what comes below from different points of view, using different framing metaphors.
Say that the arrangement of the simultaneous elements of visual experience is analogous to a spatial arrangement. This is natural if we think of the visual system as a branching series of layers. As far as subjective experience goes, are ‘higher’ layers in front of or behind the ‘lower’ layers? Are they above or below? Do they interlock like... it is hard to think of a metaphor here. When do layers, as such, interlock so that they form a single variegated layer? D* suggested color printing as something similar, though this doesn’t quite satisfy me. I imagine a jigsaw puzzle where the solution is a solid block, and where every layer has the same extent as the solution but is mostly empty space. D* also mentioned layers of transparencies where on each layer a portion of the final image – which perhaps occludes lower parts – is printed; like the pages in the encyclopedia entry on the human body, where the skin, muscles, organs, bones, were printed on separate sheets.
But after some thought, I don't think these can work. An image as a metaphor for the perceptual image? A useful metaphor would have some explanatory degrees of freedom; one set of things that can be understood in one way, used to understand something different in a similar way. Where do we get by trying to understand one type of image as another type of image? Not very far, I think. The visual field is a sort of tensor: at every point in the field, multiple things are true at the same time, they are combined according to deterministic rules, and a unitary percept results. Trying to understand this problem in terms of a simpler type of image seems doomed to fail.
So, whether or not there is a convenient metaphor, I think that the idea of the question should be clear: how are the different components of the percept simultaneously present? A prominent part of psychophysics studies how different components interact: color and luminance contrast, or motion and orientation, but my understanding is that for the most part different components are independently encoded; i.e. nothing really affects the perceived orientation of an edge, except perhaps the orientations of other proximal (in space or time) edges.
Masking, i.e. making one thing harder to see by laying another thing in proximity to it, is also usually within-layer, i.e. motion-to-motion, or contrast-to-contrast. Here, I am revealing that my thinking is still stuck in the lowest levels: color, motion, contrast, orientation, are all encoded together, in overlapping ensembles. So, it may well be that a single mechanism can encode a feature with multiple perceptual elements.
Anyways, the reason why I wonder about these things is, lately, because of this study where I had subjects judge the contrast of photographic images and related these judgments to the contrasts of individual scales within the images. This is related to the bigger question because there is no obvious reason why the percept contrast of a complex, broadband image should correspond to the same percept contrast of a simple spatial pattern like a narrowband wavelet of one type or another. This is where we converge with what I have written a few months ago: the idea of doing psychophysics with simple stimuli is that a subject’s judgments can be correlated with the physical properties of the stimuli, which can be completely described because they are simple. When the stimuli are complex and natural, there is a hierarchy of physical properties for which the visual system is specifically designed, with its own hierarchy, to analyze. Simple stimuli target components of this system; complex stimuli activate the entire thing.
It is possible that when I ask you to identify the contrast – the luminance amplitude – of a Gabor patch, you are able to do so by looking, from your behavioral perch, at the response amplitude of a small number of neural mechanisms which are themselves stimulated directly by luminance gradients, which are exactly what I am controlling by controlling the contrast of the Gabor. It is not only possible, but this is the standard assumption in most contrast psychophysics (though I am suspicious that the Perceptual Template people have fuzzier ideas than this, I am not yet clear on their thinking – is the noisiness of a response also part of apparent magnitude?).
It is also possible that when I ask you to identify the contrast of a complex image, like a typical sort of image you look at every day (outside of spatial vision experiments), you are able to respond by doing the same thing: you pool together the responses of lots of neural mechanisms whose responses are determined by the amplitude of luminance gradients of matched shape. This is the assumption I set out to test in my experiment, that contrast is more or less the same, perceptually, whatever the stimulus is.
But, this does not need to be so. This assumption means that in judging the contrast of the complex image, you are able to ignore the responses of all the other mechanisms that are being stimulated by the image: mechanisms that respond to edges, texture gradients, trees, buildings, depth, occlusions, etc. Why should you be able to do this? Do these other responses not get in the way of ‘seeing’ those more basic responses? We know that responses later in the visual hierarchy are not so sensitive to the strength of a stimulus, rather they are sensitive to the spatial configuration of the stimulus; if you vary how much the configuration fits, you will vary the response of the neuron, but if you vary its contrast you will, across some threshold, turn the neuron on and off.
I don’t have a solution; the question is not answered by my experiment. I don’t doubt that you can see the luminance contrast of the elements in a complex scene, but I am not convinced that what you think is the contrast is entirely the contrast. In fact, we know for certain that it is not, because we have a plethora of lightness/brightness illusions.
No progress here, and I'm still not sure of the quality of the question. But, maybe this way of thinking can make for an interesting pitch at the outset of the introduction of the paper.
Thursday, November 08, 2012
meh
I keep noticing, lately, near- and far-peripheral flashes, phosphenes. There was one a few minutes ago, maybe 10deg below fovea, very obvious and yet hard to localize (it was almost as though it extended close to the fovea); I checked for a blindspot, found none. So, based on all these recent sparky things, I predict something happening in the next few days (especially since the BA paper is just about done, so I'm about to go through another relax-contract phase).
Monday, November 05, 2012
probability/likelihood
probability of an event occurring
likelihood of a condition existing
how often have i misused these words? unknown.
if i search this blog, i find one clear misuse of 'likelihood', in "a specific instantiation..." ("likelihood of a pass"); i use the term two other times, in the first post ever and in a later one that refers to that one. in those instances (likelihood that you are alive at age x) i think it's ambiguous, but i'll count those as accurate.
amazingly, there is only one post where i use the word 'probability' in the relevant context. and i can't tell if i'm using it right or not. i'm kind of disappointed in myself.
seems i've fallen off the HAZ-PJ wagon. writing is going well, though, which is good.
likelihood of a condition existing
how often have i misused these words? unknown.
if i search this blog, i find one clear misuse of 'likelihood', in "a specific instantiation..." ("likelihood of a pass"); i use the term two other times, in the first post ever and in a later one that refers to that one. in those instances (likelihood that you are alive at age x) i think it's ambiguous, but i'll count those as accurate.
amazingly, there is only one post where i use the word 'probability' in the relevant context. and i can't tell if i'm using it right or not. i'm kind of disappointed in myself.
seems i've fallen off the HAZ-PJ wagon. writing is going well, though, which is good.
Friday, October 26, 2012
government as a design problem
trying to work on a paper revision due sooner and sooner, but i keep thinking about politics. of course, it is The Time to think about politics, but i wish i could escape it.
anyways, here's what i've been thinking, a tiny idea:
from the institution of a new state or government, for a time, it is reasonable to expect the government to grow and acquire new features. this is just because upon its institution, the government must be incomplete or flawed. virtually nothing complex can approach perfection, especially in its first design.
however, at some point, we might consider the institution - or, and here's the real idea, a given version of the institution - to be complete. that is, we have this complex structure, with many parts and many layers and many functions, and it is intended to accomplish many things under particular constraints. presumably, changes made to this structure over time are intended to fulfill these intentions. we could think of this as efficiency, i.e., how much of what the system is meant to do is it actually doing? this is a funny idea, since it implies that if the system exceeds its mandate, it is being overly efficient. i will get back to this in a moment.
the idea is that a version of the institution can be considered complete, in that a time will come when it's clear to everyone that no changes, or only basic maintenance changes, are necessary to meet the objectives of the system; or, it might be decided that the objectives are outdated, and that new objectives have arisen, and that a new system needs to be designed to replace the old one. have we ever reached that point with the american federal government? i think maybe we have, and it was a long time ago: pre-civil war, really. in the 1850s, the federal government wasn't really creating many new responsibilities for itself, and was instead preoccupied with its intended functions of maintaining relations between the states, applying tariffs in international trade, occupying new territories that would eventually become states, etc. i think this is the tail end of what historians refer to as the "second party system": FED2.0. FED2.0 was rolled out in the 1830s, had some successes early on, and then crashed and burned.
it was around the time of the civil war that the government basically went through a big redesign, acquiring new responsibilities which then required new features to be fulfilled. this was the "third party system" that lasted until the 1890s, when it was replaced with FED4.0, which lasted until the great depression. versions 3 and 4, i think, are not really considered to be very good versions (and probably could be collapsed into subversions of FED3), while a lot of people are clearly very nostalgic for versions 1.0 and maybe 2.0 (and might see those as subversions of FED1).
in the 1930s, the government went through a huge redesign: FED5.0; the end of the 1960s saw a big advance on this (FED5.1), and now we're probably at version 5.3 or 5.4. version 5 is the longest-lived political system that the US has had (or similar with FED3/4). clearly, i think, it's time for a redesign. at this point, the two parties are just concerned with adding, subtracting, or modifying features, with a strong tendency towards addition (the 'ratchet effect' or 'featuritis'). i think that a lot of people thought that with o* and the d*s, after the 2008 election, we would be moving on to a new version 6; a lot of people thought that in 2004 with b* and the r*s. neither succeeded; i don't think that either really succeeded in moving a new subversion, either: we're stuck in beta, at 5.3.2 or something like that.
so, back to 'excess efficiency'. what is that? it's not what it sounds like. when a system isn't quite fulfilling its promised aims, if it wants to preserve itself (consider that institutions don't want to die), it might throw up new proxy aims. it can them give the illusion of accomplishment or fulfillment by moving to meet those new aims, thus obscuring the fact that the old aims aren't exactly complete; or, that they're no longer valid, and that the system thus is working to fulfill aims that no longer exist. i.e., excess efficiency is a sign that a system is desperate and needs to be replaced.
anyways, here's what i've been thinking, a tiny idea:
from the institution of a new state or government, for a time, it is reasonable to expect the government to grow and acquire new features. this is just because upon its institution, the government must be incomplete or flawed. virtually nothing complex can approach perfection, especially in its first design.
however, at some point, we might consider the institution - or, and here's the real idea, a given version of the institution - to be complete. that is, we have this complex structure, with many parts and many layers and many functions, and it is intended to accomplish many things under particular constraints. presumably, changes made to this structure over time are intended to fulfill these intentions. we could think of this as efficiency, i.e., how much of what the system is meant to do is it actually doing? this is a funny idea, since it implies that if the system exceeds its mandate, it is being overly efficient. i will get back to this in a moment.
the idea is that a version of the institution can be considered complete, in that a time will come when it's clear to everyone that no changes, or only basic maintenance changes, are necessary to meet the objectives of the system; or, it might be decided that the objectives are outdated, and that new objectives have arisen, and that a new system needs to be designed to replace the old one. have we ever reached that point with the american federal government? i think maybe we have, and it was a long time ago: pre-civil war, really. in the 1850s, the federal government wasn't really creating many new responsibilities for itself, and was instead preoccupied with its intended functions of maintaining relations between the states, applying tariffs in international trade, occupying new territories that would eventually become states, etc. i think this is the tail end of what historians refer to as the "second party system": FED2.0. FED2.0 was rolled out in the 1830s, had some successes early on, and then crashed and burned.
it was around the time of the civil war that the government basically went through a big redesign, acquiring new responsibilities which then required new features to be fulfilled. this was the "third party system" that lasted until the 1890s, when it was replaced with FED4.0, which lasted until the great depression. versions 3 and 4, i think, are not really considered to be very good versions (and probably could be collapsed into subversions of FED3), while a lot of people are clearly very nostalgic for versions 1.0 and maybe 2.0 (and might see those as subversions of FED1).
in the 1930s, the government went through a huge redesign: FED5.0; the end of the 1960s saw a big advance on this (FED5.1), and now we're probably at version 5.3 or 5.4. version 5 is the longest-lived political system that the US has had (or similar with FED3/4). clearly, i think, it's time for a redesign. at this point, the two parties are just concerned with adding, subtracting, or modifying features, with a strong tendency towards addition (the 'ratchet effect' or 'featuritis'). i think that a lot of people thought that with o* and the d*s, after the 2008 election, we would be moving on to a new version 6; a lot of people thought that in 2004 with b* and the r*s. neither succeeded; i don't think that either really succeeded in moving a new subversion, either: we're stuck in beta, at 5.3.2 or something like that.
so, back to 'excess efficiency'. what is that? it's not what it sounds like. when a system isn't quite fulfilling its promised aims, if it wants to preserve itself (consider that institutions don't want to die), it might throw up new proxy aims. it can them give the illusion of accomplishment or fulfillment by moving to meet those new aims, thus obscuring the fact that the old aims aren't exactly complete; or, that they're no longer valid, and that the system thus is working to fulfill aims that no longer exist. i.e., excess efficiency is a sign that a system is desperate and needs to be replaced.
Monday, October 22, 2012
taxes and politics
a friend posted this link on facebook, along with a quote to the effect of, "raising taxes to pay for investments in the middle class creates jobs". i resisted posting a response there because i don't like to argue about these things and would rather keep my opinions to myself, and because (relatedly) i am afraid to affect others' opinions of me in ways that i don't have close control over. so i thought i'd post a response here, where no one can read it:
***
while i agree generally with sentiment that says the rich should be taxed relatively more, the idea that this is then turned around by the federal government into "investment in the middle class" is not obvious to me. i think that very little of taxation, at least in a developed country like the US, translates directly into economic growth. in fact i think it tends to be the opposite, and i agree more with the idea that raising taxes tends to suppress growth.
a large portion of government spending put in place by the democratic party (which is presumably the favorable political dimension for the approving audience of this talk) is in the form of political favors to constituencies that have an insignificant impact on the economy (poor, elderly); institution of new bureaucracies which have to be funded at the same time that their mandate is usually to impose some form of *restriction* on certain types of economic activity; increased funding of dysfunctional programs without improving function ("education"); and, you can be sure that the more the US government collects in taxes, the more it will spend on the military, or on foreign aid, etc.
interesting case in point relevant to our livelihoods: significant government spending goes into biomedical research, which winds up making people live longer at the same time that it makes all forms of healthcare more expensive (MRI for everybody! one-of-a-kind cancer drug for my grampa! YOU CAN LIVE FOREVER NOW). i guess this creates jobs in the hospital/rest home industries.
*not that any of these things are wrong per se*, but it's not clear how any of this works as investment in the consumer class that generates net jobs. i think the government of a developed country actually has very little capacity to "create jobs", except in managing trade policy and maintaining transportation infrastructure. i.e. i think this guy's argument is pretty arguable.
***
while i agree generally with sentiment that says the rich should be taxed relatively more, the idea that this is then turned around by the federal government into "investment in the middle class" is not obvious to me. i think that very little of taxation, at least in a developed country like the US, translates directly into economic growth. in fact i think it tends to be the opposite, and i agree more with the idea that raising taxes tends to suppress growth.
a large portion of government spending put in place by the democratic party (which is presumably the favorable political dimension for the approving audience of this talk) is in the form of political favors to constituencies that have an insignificant impact on the economy (poor, elderly); institution of new bureaucracies which have to be funded at the same time that their mandate is usually to impose some form of *restriction* on certain types of economic activity; increased funding of dysfunctional programs without improving function ("education"); and, you can be sure that the more the US government collects in taxes, the more it will spend on the military, or on foreign aid, etc.
interesting case in point relevant to our livelihoods: significant government spending goes into biomedical research, which winds up making people live longer at the same time that it makes all forms of healthcare more expensive (MRI for everybody! one-of-a-kind cancer drug for my grampa! YOU CAN LIVE FOREVER NOW). i guess this creates jobs in the hospital/rest home industries.
*not that any of these things are wrong per se*, but it's not clear how any of this works as investment in the consumer class that generates net jobs. i think the government of a developed country actually has very little capacity to "create jobs", except in managing trade policy and maintaining transportation infrastructure. i.e. i think this guy's argument is pretty arguable.
Sunday, October 14, 2012
lull
finally working on that blur adaptation paper. not much else to think about or report.
***
last saturday (the 6th) woke up with a headache, fairly painful: i'd rate the usual ones at 2/10, and this was a 4/10. today, i wake up with one even worse: i'd put this at 6/10 (assuming it can get much worse). this is awful - it's as bad as that night tukrong punched me in the head 10 times. what is going on with my brain?
***
last saturday (the 6th) woke up with a headache, fairly painful: i'd rate the usual ones at 2/10, and this was a 4/10. today, i wake up with one even worse: i'd put this at 6/10 (assuming it can get much worse). this is awful - it's as bad as that night tukrong punched me in the head 10 times. what is going on with my brain?
Friday, October 05, 2012
Wednesday, October 03, 2012
inevitable negativity
i am now (technically) a faculty member at harvard medical school ("instructor": about as junior junior junior as possible)! wow! and, (technically), no longer a postdoctoral fellow: i am now a "senior scientific associate". all so i can apply for a grant with a less than a 1% chance of getting funded (optimistic i am). so, something there. beautiful appearance of progress.
also, couple of papers accepted; probably will have a paper in PLoS-one, which is nice, but i'm third of four authors, so..
***
came up with this on the train, coming home last friday night (9-28-12)
on Cambridge Street
put away
your umbrellas
or they'll wind up
cast aside
in tattered heaps
abandoned
on Cambridge Street
thrust your head
into the wind
and bear the rain
it's autumn
it's not cold yet
be thankful
Sunday, September 30, 2012
中秋快乐!
quick notes for the end of september:
week 1 of bring-your-laptop-to-work was a success; worked steadily in the lab every day, and came home each night to do particular jobs by hand, with pen and paper. extremely effective. laptop came back home friday night; going to continue this for the foreseeable future. should make the next MS revision and the following MS submission much easier.
headache last night, gradual onset; eventually focused pain above right eye socket; photophobia; went to bed, closed eyes, weird eigenlicht flicker, maybe 40-50Hz; what is that? slight headache remnant now, indistinct.
recent weirdness with reading text, usually notice in the morning; right now, left of fixation feels scotoma-like, but i can see there..
**
also, a story: when i sit at the kitchen table, in the chair by the window, i have a view of the pantry area, with the fridge and the back door. my leather sandals are wedged between the fridge and the wall, by the door, so i can wear them outside when i go to throw trash out.
i regularly mistake the sandals, peripherally, for Olive the Cat, sitting by the back door, wanting to go out. then i foveate them, and see that they are my sandals. this has happened repeatedly, maybe dozens of times: deja trompé!
week 1 of bring-your-laptop-to-work was a success; worked steadily in the lab every day, and came home each night to do particular jobs by hand, with pen and paper. extremely effective. laptop came back home friday night; going to continue this for the foreseeable future. should make the next MS revision and the following MS submission much easier.
headache last night, gradual onset; eventually focused pain above right eye socket; photophobia; went to bed, closed eyes, weird eigenlicht flicker, maybe 40-50Hz; what is that? slight headache remnant now, indistinct.
recent weirdness with reading text, usually notice in the morning; right now, left of fixation feels scotoma-like, but i can see there..
**
also, a story: when i sit at the kitchen table, in the chair by the window, i have a view of the pantry area, with the fridge and the back door. my leather sandals are wedged between the fridge and the wall, by the door, so i can wear them outside when i go to throw trash out.
i regularly mistake the sandals, peripherally, for Olive the Cat, sitting by the back door, wanting to go out. then i foveate them, and see that they are my sandals. this has happened repeatedly, maybe dozens of times: deja trompé!
Tuesday, September 25, 2012
why do i keep writing poems
Batten down the hatches!
In this electric squall
Or else we'll be sent to the deep -
The web will drown us all.
So home I'll go! To printed word,
With pen and paper work.
No opportunity to drift
Through forums or to lurk
In hiding from my calling,
I'll forge ideas by thought
And stare down syntax, words reform
To make all logic-wrought.
So batten down the hatches!
And keep the ship afloat
For though I'll try to steer us,
The net may wreck this boat.
In this electric squall
Or else we'll be sent to the deep -
The web will drown us all.
So home I'll go! To printed word,
With pen and paper work.
No opportunity to drift
Through forums or to lurk
In hiding from my calling,
I'll forge ideas by thought
And stare down syntax, words reform
To make all logic-wrought.
So batten down the hatches!
And keep the ship afloat
For though I'll try to steer us,
The net may wreck this boat.
Friday, September 21, 2012
grant, presentation, paper, model
Been trying to skip between several jobs: grant proposal with a looming deadline, modeling experiments for a paper revision with a looming deadline, looming conference presentation... well, the conference is over, and the grant is coming along, though I still do not believe I will make it.
The paper.. okay, another paper: poked an editor yesterday, and he came back with a 'minor revision' request, which I fulfilled by late afternoon today. So, finally, we have a journal article - in a 1.0 impact factor journal - to show for a 3 year postdoc. Sigh. Another in revision, in a better journal, but that's the big problem: I'm doing all these model tests, but I can't get any real momentum because I keep flipping back to the grant. Sigh. I keep complaining about the same thing. Need to set a deadline - 3 more years? - after which if I'm still making the same complaint, something needs to change.
Let's talk about the model stuff. I've talked about it already in the past few posts: in the original paper, I proposed a modification to an existing model, a minor modification, which was able to closely fit our data, but which was a bit complexified, and difficult to explain exactly why it worked as well as it did, and also unable to show how varying its parameters explained the variance in our data, etc. So, it "worked", but that's about all it did. It didn't explain much.
The existing model we call the "simple model". The simple model is indeed simple. It's so simple that it's almost meaningless, which is what frustrates me. Of course it's not that simple; you can interpret its components in very simplified, but real, visual system terms. And, it basically can describe our data, even when I complexify it just a bit to handle the extra complexity of our stimuli. And this complexification is fine, because it works best if I remove an odd hand-waving component that the original author had found it necessary to include to explain his data. Only... it doesn't quite work. The matching functions that make up the main set of data have slopes that are different in a pattern that is replicated by the simple model, but overall the model slopes are too shallow. I spent last week trying to find a dimension of the model that I could vary in order to shift the slopes up and down without destroying other aspects of its performance.. no dice.. fail fail fail.
So, I'm thinking that I can present a 'near miss': the model gets a lot of things right, and it fails to get everything right for reasons that I haven't thought hard enough about just yet. I really need to sit some afternoon and really think it out. Why, for the normal adaptor, is the matching function slope steeper than the identity line, but never steep enough? What is missing? Is it really the curvature of the CSF? How do I prove it?
Now, out of some horrible masochistic urge, I'm running the big image-based version of the "simple model". This version doesn't collapse the input and adaptation terms into single vectors until the 'blur decoding' stage. It seems like, really, some version of this has to work, but it hasn't come close yet. Looking at it now, though, I see that I did some strange things that are kind of hard to explain... Gonna give it another chance overnight.
The paper.. okay, another paper: poked an editor yesterday, and he came back with a 'minor revision' request, which I fulfilled by late afternoon today. So, finally, we have a journal article - in a 1.0 impact factor journal - to show for a 3 year postdoc. Sigh. Another in revision, in a better journal, but that's the big problem: I'm doing all these model tests, but I can't get any real momentum because I keep flipping back to the grant. Sigh. I keep complaining about the same thing. Need to set a deadline - 3 more years? - after which if I'm still making the same complaint, something needs to change.
Let's talk about the model stuff. I've talked about it already in the past few posts: in the original paper, I proposed a modification to an existing model, a minor modification, which was able to closely fit our data, but which was a bit complexified, and difficult to explain exactly why it worked as well as it did, and also unable to show how varying its parameters explained the variance in our data, etc. So, it "worked", but that's about all it did. It didn't explain much.
The existing model we call the "simple model". The simple model is indeed simple. It's so simple that it's almost meaningless, which is what frustrates me. Of course it's not that simple; you can interpret its components in very simplified, but real, visual system terms. And, it basically can describe our data, even when I complexify it just a bit to handle the extra complexity of our stimuli. And this complexification is fine, because it works best if I remove an odd hand-waving component that the original author had found it necessary to include to explain his data. Only... it doesn't quite work. The matching functions that make up the main set of data have slopes that are different in a pattern that is replicated by the simple model, but overall the model slopes are too shallow. I spent last week trying to find a dimension of the model that I could vary in order to shift the slopes up and down without destroying other aspects of its performance.. no dice.. fail fail fail.
So, I'm thinking that I can present a 'near miss': the model gets a lot of things right, and it fails to get everything right for reasons that I haven't thought hard enough about just yet. I really need to sit some afternoon and really think it out. Why, for the normal adaptor, is the matching function slope steeper than the identity line, but never steep enough? What is missing? Is it really the curvature of the CSF? How do I prove it?
Now, out of some horrible masochistic urge, I'm running the big image-based version of the "simple model". This version doesn't collapse the input and adaptation terms into single vectors until the 'blur decoding' stage. It seems like, really, some version of this has to work, but it hasn't come close yet. Looking at it now, though, I see that I did some strange things that are kind of hard to explain... Gonna give it another chance overnight.
Sunday, September 16, 2012
too far, too far
one hundred words
in haiku form
while waiting for
my flight on a
sunday evening
in september:
Rochester airport
September Sunday evening
me and three women
80s pop radio
electric piano solo
fluorescent lighting
now another man
sneakers, backwards baseball cap
the sun is setting
PA announcement
the guy's voice croaks like Stallone
a fine disco beat
two smartphones, a book
two pair boots, one pair flip flops
not a conjunction
what will our plane be?
CRJ, Boeing, Airbus?
another man comes
three women, three men
the humans are trickling in
going to Boston
the sun sets slowly
slower than it usually does
suspicious liquids
dinner of junk food
reflection of ceiling lights
in my laptop screen
in haiku form
while waiting for
my flight on a
sunday evening
in september:
Rochester airport
September Sunday evening
me and three women
80s pop radio
electric piano solo
fluorescent lighting
now another man
sneakers, backwards baseball cap
the sun is setting
PA announcement
the guy's voice croaks like Stallone
a fine disco beat
two smartphones, a book
two pair boots, one pair flip flops
not a conjunction
what will our plane be?
CRJ, Boeing, Airbus?
another man comes
three women, three men
the humans are trickling in
going to Boston
the sun sets slowly
slower than it usually does
suspicious liquids
dinner of junk food
reflection of ceiling lights
in my laptop screen
Saturday, September 15, 2012
morning aura
in rochester for the OSA vision meeting.
woke up this morning about 6:30ish, with terry yelling at me to wake up. went to take a shower, and while there, realized i couldn't see my fingertips as i was washing, grabbing soap, etc. got out of the shower, got dressed, left the room and went to the lobby. got there a little after seven, and the scotoma was well into the periphery, flickering etc. it was just like the last three: left field, straight right-left through the upper field and arcing downward into the lower field.
the last one i managed to record, back in june, had a time gap between the foveal scotoma and the peripheral arcs, which i had post hoc explained as me needing to recalibrate some part of the perimeter or something. but during the last two, i noticed that the scotoma actually does seem to disappear between the foveal appearance and the peripheral arcs. wonder what is going on with that...
didn't notice any peripheral rough spot this time, but it was so early that i might just have been too dazed.. these morning scotomas that i've experienced - the last one a few weeks ago, and the one last year in the winter - they seem to have started just as i awoke. may be coincidental, since it's just a sample size of three, but i haven't had one start a half hour after waking, and i haven't woken up halfway through one (though the first time, i think i lay there with my eyes closed for the first 10 minutes or so). might be interesting to look up what sort of neurochemical changes occur in cortex, esp. visual cortex, during waking.
headache was ok, took some tylenol this time. nauseated all day long.
**
yesterday (or maybe thursday night, not sure), i remember feeling suspicious that something might be about to happen: i had the thought, i should keep track of these suspicions, to see if they're actually correlated. it is possible that i am suspicious very frequently, and just notice the coincidences..
woke up this morning about 6:30ish, with terry yelling at me to wake up. went to take a shower, and while there, realized i couldn't see my fingertips as i was washing, grabbing soap, etc. got out of the shower, got dressed, left the room and went to the lobby. got there a little after seven, and the scotoma was well into the periphery, flickering etc. it was just like the last three: left field, straight right-left through the upper field and arcing downward into the lower field.
the last one i managed to record, back in june, had a time gap between the foveal scotoma and the peripheral arcs, which i had post hoc explained as me needing to recalibrate some part of the perimeter or something. but during the last two, i noticed that the scotoma actually does seem to disappear between the foveal appearance and the peripheral arcs. wonder what is going on with that...
didn't notice any peripheral rough spot this time, but it was so early that i might just have been too dazed.. these morning scotomas that i've experienced - the last one a few weeks ago, and the one last year in the winter - they seem to have started just as i awoke. may be coincidental, since it's just a sample size of three, but i haven't had one start a half hour after waking, and i haven't woken up halfway through one (though the first time, i think i lay there with my eyes closed for the first 10 minutes or so). might be interesting to look up what sort of neurochemical changes occur in cortex, esp. visual cortex, during waking.
headache was ok, took some tylenol this time. nauseated all day long.
**
yesterday (or maybe thursday night, not sure), i remember feeling suspicious that something might be about to happen: i had the thought, i should keep track of these suspicions, to see if they're actually correlated. it is possible that i am suspicious very frequently, and just notice the coincidences..
Thursday, September 13, 2012
Deja Trompé
When I was in graduate school, I lived in Old Louisville, and walked, most days, down 3rd street to campus. Whenever I crossed the big road separating the neighborhood from campus, Cardinal Avenue, at a certain spot, I would see something up in my right peripheral visual field, and think, "Starlings!"
It was never starlings. It was always the tattered insulation hanging off a bunch of power lines strung over Cardinal. I remember this because even though I learned, pretty quickly, that I wasn't seeing starlings in that instant whenever it occurred, the fastest part of me - whatever part just automatically identifies salient stuff in the vast periphery - always thought that I was.
"Starlings!"
It not that I was hallucinating starlings. A bunch of speckly black stuff fluttering against the sky kind of looks like birds, even when you know it isn't. You can't blame me. I don't blame my visual system. It's an honest mistake. The interesting thing is that I kept making it, over and over again, with apparently no control over it. An inconsequential and incessant perceptual mistake.
I've noticed similar situations over the years, but right now I can't remember the others. I should start making a list. I bring this up because recently someone cleaned out the shared kitchen on this side of the institute, and because I always turn the lights out in the same kitchen.
I think that, because I always turn out the light when I leave the little kitchen, other people have started following my example, and now, often, when I go to the kitchen to get hot water for my tea, the light is already out. This makes me happy. It's happened very gradually. Change is slow, usually.
Usually. Recently, the development office got a new temp who is apparently a complete OCD clean freak. It's great. She cleaned this kitchen and the other one. She put up little signs everywhere telling people not to be such pigs. I love her.
Anyways, now, when I go into the little kitchen to get my water, I stand at the dispenser, watching it to make sure my hand doesn't stray and I don't get scalded, and the microwave with its little sign sits down in my lower left field. Often, lately, the light is out when I get there. I leave it that way, because there's enough light trickling in from the hallway. Every time I am in this configuration, with the light out, it looks for all the world that there is light coming out of the microwave window.
This happens over and over again. It's very robust; I can stand there and look straight at the microwave and its little paper sign, and that's what I see; then I look away, and the sign becomes an emission of lamp light from within the microwave. I can turn the mistake on and off by moving my eyes back and forth.
Again, I don't blame my visual system. It's doing the best it can. I've seen so many microwaves, and when they're cooking, they usually have little lamps inside, so you can see your whatever rotating on the little turntable. If the room is dark, the image is basically of a luminous rectangle in the front door of a microwave. Not many microwaves that I have known have worn little paper signs on their doors. To their disgusting, disgusting peril.
There must be a name for this, but I can't find it. So for now I'm going to invent a term: deja trompé, "fooled again". Deja as in deja vu, "again seen"; trompé as in trompé l'oeil, "deceives the eye". Seems like the right flavor for this sort of thing. I'll start keeping track of these, however rare they are. I'll inaugurate the list with a new entry label.
BACK TO WORK
It was never starlings. It was always the tattered insulation hanging off a bunch of power lines strung over Cardinal. I remember this because even though I learned, pretty quickly, that I wasn't seeing starlings in that instant whenever it occurred, the fastest part of me - whatever part just automatically identifies salient stuff in the vast periphery - always thought that I was.
"Starlings!"
It not that I was hallucinating starlings. A bunch of speckly black stuff fluttering against the sky kind of looks like birds, even when you know it isn't. You can't blame me. I don't blame my visual system. It's an honest mistake. The interesting thing is that I kept making it, over and over again, with apparently no control over it. An inconsequential and incessant perceptual mistake.
I've noticed similar situations over the years, but right now I can't remember the others. I should start making a list. I bring this up because recently someone cleaned out the shared kitchen on this side of the institute, and because I always turn the lights out in the same kitchen.
I think that, because I always turn out the light when I leave the little kitchen, other people have started following my example, and now, often, when I go to the kitchen to get hot water for my tea, the light is already out. This makes me happy. It's happened very gradually. Change is slow, usually.
Usually. Recently, the development office got a new temp who is apparently a complete OCD clean freak. It's great. She cleaned this kitchen and the other one. She put up little signs everywhere telling people not to be such pigs. I love her.
Anyways, now, when I go into the little kitchen to get my water, I stand at the dispenser, watching it to make sure my hand doesn't stray and I don't get scalded, and the microwave with its little sign sits down in my lower left field. Often, lately, the light is out when I get there. I leave it that way, because there's enough light trickling in from the hallway. Every time I am in this configuration, with the light out, it looks for all the world that there is light coming out of the microwave window.
This happens over and over again. It's very robust; I can stand there and look straight at the microwave and its little paper sign, and that's what I see; then I look away, and the sign becomes an emission of lamp light from within the microwave. I can turn the mistake on and off by moving my eyes back and forth.
Again, I don't blame my visual system. It's doing the best it can. I've seen so many microwaves, and when they're cooking, they usually have little lamps inside, so you can see your whatever rotating on the little turntable. If the room is dark, the image is basically of a luminous rectangle in the front door of a microwave. Not many microwaves that I have known have worn little paper signs on their doors. To their disgusting, disgusting peril.
There must be a name for this, but I can't find it. So for now I'm going to invent a term: deja trompé, "fooled again". Deja as in deja vu, "again seen"; trompé as in trompé l'oeil, "deceives the eye". Seems like the right flavor for this sort of thing. I'll start keeping track of these, however rare they are. I'll inaugurate the list with a new entry label.
BACK TO WORK
Wednesday, September 12, 2012
adaptomatic
Trying to figure out how to proceed with this adaptation paper, and I retreat here.
Minor problem is the rewrite: this will get done, not too worried about it. May be the last thing that gets done, since the major problem needs to be solved materially first.
Major problem is the modeling. The original paper details a complexified version of the model proposed by the authors of a paper that our paper basically replicates, accidentally. We were scooped, and so I thought that to novelify our paper, I would take their model and try to push it a little further, and do some extra analysis of it.
What I didn't do was what I should have done, which was to also test the simple model and show that it is somehow inadequate, and that complexification is therefore justified or necessary. I am actually ambivalent about this. My main idea was that we should take a model which has generalizable features and use it to explain the data; but, it's true that the more sophisticated version can't really be credited with achieving anything unless the simple one can also be shown to fail.
So the problem is that I have to do a lot of testing of the simple model. So, I decided that I would scrap the section that was already in the paper and replace it with an evaluation of the simple model, but make up for the lack of 'advance' by employing the simple model in a more realistic simulation of the actual experiments. This is what I've been trying to do, and basically failing at, for several weeks now.
The first idea was to use the simplest form of the model, but the most complete form of the stimuli: videos, played frame by frame and decomposed into the relevant stimulus bands, adaptation developing according to a simple differential equation with the same dimensions as the stimulus. This didn't work. Or, it almost worked. The problem is that adaptation just won't build up in the high frequency channels, unless it's way overpowered, which is against any bit of evidence I can think about. If high frequency adaptation were so strong, everything would be blurry all the time. I think it should be the weakest, or the slipperiest.
Soon after that, I gave up and retreated to the 'global sum' model, where instead of using 2d inputs, I use 0d inputs - i.e. the stimulus is treated as a scalar. I get the scalars from the real stimuli, and the same dynamic simulation is run. It's tons faster, of course, which makes it easier to play around with. I figured I would have found a solution by now.
See, it's so close. It's easy to get a solution, by adjusting the time constants, how they vary with frequency, and the masking strength, and get a set of simulated matching functions that look a lot like the human data. But I figure this is uninteresting. I have a set of data for 10 subjects, and they seem to vary in particular ways - but I can't get the simulated data to vary in the same way. If I can't do that, what is the point of the variability data?
Also, last night I spent some time looking closely at the statistics of the original test videos. There's something suspicious about them. Not wrong - I don't doubt that the slope change that was imposed was imposed correctly. But the way contrast changes with frequency and slope is not linear - it flattens out, at different frequencies, at the extreme slope changes. In the middle range, around zero, all contrasts change. Suspiciously like the gain peak, which I'm wondering isn't somehow an artifact of this sort of image manipulation.
I don't expect to figure that last bit out before the revision is done. But, I'm thinking it might be a good idea to play down the gain peak business, since I might wind up figuring out that e.g. adaptation is much more linear than it appears, and that the apparent flattening out is really an artifact of the procedure. I don't think I'll find that, but - did I mention I'm going to write a model-only paper after this one? - seems a good idea not to go too far out on a limb when there are doubts.
I have a nagging feeling that I gave up too soon on the image-based model...
Minor problem is the rewrite: this will get done, not too worried about it. May be the last thing that gets done, since the major problem needs to be solved materially first.
Major problem is the modeling. The original paper details a complexified version of the model proposed by the authors of a paper that our paper basically replicates, accidentally. We were scooped, and so I thought that to novelify our paper, I would take their model and try to push it a little further, and do some extra analysis of it.
What I didn't do was what I should have done, which was to also test the simple model and show that it is somehow inadequate, and that complexification is therefore justified or necessary. I am actually ambivalent about this. My main idea was that we should take a model which has generalizable features and use it to explain the data; but, it's true that the more sophisticated version can't really be credited with achieving anything unless the simple one can also be shown to fail.
So the problem is that I have to do a lot of testing of the simple model. So, I decided that I would scrap the section that was already in the paper and replace it with an evaluation of the simple model, but make up for the lack of 'advance' by employing the simple model in a more realistic simulation of the actual experiments. This is what I've been trying to do, and basically failing at, for several weeks now.
The first idea was to use the simplest form of the model, but the most complete form of the stimuli: videos, played frame by frame and decomposed into the relevant stimulus bands, adaptation developing according to a simple differential equation with the same dimensions as the stimulus. This didn't work. Or, it almost worked. The problem is that adaptation just won't build up in the high frequency channels, unless it's way overpowered, which is against any bit of evidence I can think about. If high frequency adaptation were so strong, everything would be blurry all the time. I think it should be the weakest, or the slipperiest.
Soon after that, I gave up and retreated to the 'global sum' model, where instead of using 2d inputs, I use 0d inputs - i.e. the stimulus is treated as a scalar. I get the scalars from the real stimuli, and the same dynamic simulation is run. It's tons faster, of course, which makes it easier to play around with. I figured I would have found a solution by now.
See, it's so close. It's easy to get a solution, by adjusting the time constants, how they vary with frequency, and the masking strength, and get a set of simulated matching functions that look a lot like the human data. But I figure this is uninteresting. I have a set of data for 10 subjects, and they seem to vary in particular ways - but I can't get the simulated data to vary in the same way. If I can't do that, what is the point of the variability data?
Also, last night I spent some time looking closely at the statistics of the original test videos. There's something suspicious about them. Not wrong - I don't doubt that the slope change that was imposed was imposed correctly. But the way contrast changes with frequency and slope is not linear - it flattens out, at different frequencies, at the extreme slope changes. In the middle range, around zero, all contrasts change. Suspiciously like the gain peak, which I'm wondering isn't somehow an artifact of this sort of image manipulation.
I don't expect to figure that last bit out before the revision is done. But, I'm thinking it might be a good idea to play down the gain peak business, since I might wind up figuring out that e.g. adaptation is much more linear than it appears, and that the apparent flattening out is really an artifact of the procedure. I don't think I'll find that, but - did I mention I'm going to write a model-only paper after this one? - seems a good idea not to go too far out on a limb when there are doubts.
I have a nagging feeling that I gave up too soon on the image-based model...
Friday, September 07, 2012
talk: 97%
did a dry run today for my FVM talk. i think it went well, but there was a good amount of feedback. (incidentally, earlier this week i came to the lab, and passed my preceptor e* talking with a familiar old guy in the hall; a few minutes later, e* brings the guy to my office and asks me to show him my work. the old guy was l.s., one of the elder statesmen of european psychophysics. turns out he had been a postdoc at the instutute more than 40 years ago, and was in town, and had just dropped in to see old friends.. i took him through my presentation at quarter speed, and he was very enthusiastic. made some suggestions about controlling for the 'knowledge' aspect of my stimuli and experiment design. took notes. had a good talk with him, he seems to know my grad school mentor well, knows all his students. so i didn't go to ECVP this week, but i got to spend a morning with one of its founders...)
anyways, the dry run: p* was the only one, as i guess i expected, to make real comments on the substance of the talk. he had two points/questions:
1. what happens if the two images are different, i.e. if they have different phase spectra? i have not tried to do this experiment, or to predict the result. i guess that technically, the model that i am evaluating would make clear predictions in such an experiment, and the perceptual process i am claiming to occur would be equally applicable. but, really, i am tacitly assuming that the similarity of the two images is tamping down noise that would otherwise be there, somehow in the spatial summation, that isn't actually reflected in the model but that would be there for the humans. but, it might work just fine. i should really try it out, just to see what happens... (*edit: i tested it in the afternoon, and the result is exactly the same. experiment is harder, and the normalization is wacky, but seems clear it works...)
2. don't the weighting functions look just like CSFs? isn't this what would happen if perceived contrasts were just CSF-weighted image contrasts? yeah, sure, but there's no reason to believe that this is how perceived contrast is computed. the flat-GC model is close to this. i wonder if i shouldn't just show a family of flat-GC models instead of a single one, with one of them having 0-weighted GC...
the other main criticism was of the slide with all the equations. this is the main thing i think i need to address. i need to remake that slide so it more naturally presents the system of processes that the equations represent. some sort of flow or wiring diagram, showing how the equations are nested...
also need to modify the explanation of the contrast randomization; not add information, but make clearer that the two contrast weighting vectors are indeed random and (basically) independent.
Monday, September 03, 2012
two out of three ain't enough
okay, so, really, i spent the labor day weekend watching youtube videos, looking at funny gifs, reading the news, and other random things, while running half-baked model simulations for the blur adaptation revision.
first thing i did was to run the video-based model through the experiment on the same three adaptation levels used in the original experiment. it worked at an operational level, i.e. it matched sharper things with sharper things and blurrier things with blurrier things, and the effects of the adaptors were correctly ordered - it didn't do anything crazy. on an empirical level, though, it was wrong.
for the original subjects, and most of the replication subjects, the perceived normal after blank adaptation should be matched to a slightly sharpened normal-video-adapted test; the simulation did the opposite. not a huge problem, but like i said, against the trend.
bigger problem is that the simulation failed to get the 'gain' peak for the normal adaptation condition; instead, gain just increased with sharpness of the adaptor. now i'm rerunning the simulation with some basic changes (adding white noise to the spatial inputs, which i don't think will work - might make it worse by increasing the effective sharpness of all inputs - but might have something of a CSF effect; and windowing the edges, which i should have done from the start).
one funny thing: even though the gain for the sharp adaptor is too high (being higher than for the normal adaptor), the gains for the normal and blurred adaptors are *exactly* the same as the means for the original three subjects: enough to make me think i was doing something horribly weirdly wrong in the spreadsheet, but there it is:
weird, but too good to be true. undoubtedly, every change to the model will change all of the simulation measurements, and the sim is definitely as noisy as the humans - even the same one run again would not get the same values.
first thing i did was to run the video-based model through the experiment on the same three adaptation levels used in the original experiment. it worked at an operational level, i.e. it matched sharper things with sharper things and blurrier things with blurrier things, and the effects of the adaptors were correctly ordered - it didn't do anything crazy. on an empirical level, though, it was wrong.
for the original subjects, and most of the replication subjects, the perceived normal after blank adaptation should be matched to a slightly sharpened normal-video-adapted test; the simulation did the opposite. not a huge problem, but like i said, against the trend.
bigger problem is that the simulation failed to get the 'gain' peak for the normal adaptation condition; instead, gain just increased with sharpness of the adaptor. now i'm rerunning the simulation with some basic changes (adding white noise to the spatial inputs, which i don't think will work - might make it worse by increasing the effective sharpness of all inputs - but might have something of a CSF effect; and windowing the edges, which i should have done from the start).
one funny thing: even though the gain for the sharp adaptor is too high (being higher than for the normal adaptor), the gains for the normal and blurred adaptors are *exactly* the same as the means for the original three subjects: enough to make me think i was doing something horribly weirdly wrong in the spreadsheet, but there it is:
weird, but too good to be true. undoubtedly, every change to the model will change all of the simulation measurements, and the sim is definitely as noisy as the humans - even the same one run again would not get the same values.
Subscribe to:
Posts (Atom)