Trying to figure out how to proceed with this adaptation paper, and I retreat here.
Minor problem is the rewrite: this will get done, not too worried about it. May be the last thing that gets done, since the major problem needs to be solved materially first.
Major problem is the modeling. The original paper details a complexified version of the model proposed by the authors of a paper that our paper basically replicates, accidentally. We were scooped, and so I thought that to novelify our paper, I would take their model and try to push it a little further, and do some extra analysis of it.
What I didn't do was what I should have done, which was to also test the simple model and show that it is somehow inadequate, and that complexification is therefore justified or necessary. I am actually ambivalent about this. My main idea was that we should take a model which has generalizable features and use it to explain the data; but, it's true that the more sophisticated version can't really be credited with achieving anything unless the simple one can also be shown to fail.
So the problem is that I have to do a lot of testing of the simple model. So, I decided that I would scrap the section that was already in the paper and replace it with an evaluation of the simple model, but make up for the lack of 'advance' by employing the simple model in a more realistic simulation of the actual experiments. This is what I've been trying to do, and basically failing at, for several weeks now.
The first idea was to use the simplest form of the model, but the most complete form of the stimuli: videos, played frame by frame and decomposed into the relevant stimulus bands, adaptation developing according to a simple differential equation with the same dimensions as the stimulus. This didn't work. Or, it almost worked. The problem is that adaptation just won't build up in the high frequency channels, unless it's way overpowered, which is against any bit of evidence I can think about. If high frequency adaptation were so strong, everything would be blurry all the time. I think it should be the weakest, or the slipperiest.
Soon after that, I gave up and retreated to the 'global sum' model, where instead of using 2d inputs, I use 0d inputs - i.e. the stimulus is treated as a scalar. I get the scalars from the real stimuli, and the same dynamic simulation is run. It's tons faster, of course, which makes it easier to play around with. I figured I would have found a solution by now.
See, it's so close. It's easy to get a solution, by adjusting the time constants, how they vary with frequency, and the masking strength, and get a set of simulated matching functions that look a lot like the human data. But I figure this is uninteresting. I have a set of data for 10 subjects, and they seem to vary in particular ways - but I can't get the simulated data to vary in the same way. If I can't do that, what is the point of the variability data?
Also, last night I spent some time looking closely at the statistics of the original test videos. There's something suspicious about them. Not wrong - I don't doubt that the slope change that was imposed was imposed correctly. But the way contrast changes with frequency and slope is not linear - it flattens out, at different frequencies, at the extreme slope changes. In the middle range, around zero, all contrasts change. Suspiciously like the gain peak, which I'm wondering isn't somehow an artifact of this sort of image manipulation.
I don't expect to figure that last bit out before the revision is done. But, I'm thinking it might be a good idea to play down the gain peak business, since I might wind up figuring out that e.g. adaptation is much more linear than it appears, and that the apparent flattening out is really an artifact of the procedure. I don't think I'll find that, but - did I mention I'm going to write a model-only paper after this one? - seems a good idea not to go too far out on a limb when there are doubts.
I have a nagging feeling that I gave up too soon on the image-based model...
Wednesday, September 12, 2012
Friday, September 07, 2012
talk: 97%
did a dry run today for my FVM talk. i think it went well, but there was a good amount of feedback. (incidentally, earlier this week i came to the lab, and passed my preceptor e* talking with a familiar old guy in the hall; a few minutes later, e* brings the guy to my office and asks me to show him my work. the old guy was l.s., one of the elder statesmen of european psychophysics. turns out he had been a postdoc at the instutute more than 40 years ago, and was in town, and had just dropped in to see old friends.. i took him through my presentation at quarter speed, and he was very enthusiastic. made some suggestions about controlling for the 'knowledge' aspect of my stimuli and experiment design. took notes. had a good talk with him, he seems to know my grad school mentor well, knows all his students. so i didn't go to ECVP this week, but i got to spend a morning with one of its founders...)
anyways, the dry run: p* was the only one, as i guess i expected, to make real comments on the substance of the talk. he had two points/questions:
1. what happens if the two images are different, i.e. if they have different phase spectra? i have not tried to do this experiment, or to predict the result. i guess that technically, the model that i am evaluating would make clear predictions in such an experiment, and the perceptual process i am claiming to occur would be equally applicable. but, really, i am tacitly assuming that the similarity of the two images is tamping down noise that would otherwise be there, somehow in the spatial summation, that isn't actually reflected in the model but that would be there for the humans. but, it might work just fine. i should really try it out, just to see what happens... (*edit: i tested it in the afternoon, and the result is exactly the same. experiment is harder, and the normalization is wacky, but seems clear it works...)
2. don't the weighting functions look just like CSFs? isn't this what would happen if perceived contrasts were just CSF-weighted image contrasts? yeah, sure, but there's no reason to believe that this is how perceived contrast is computed. the flat-GC model is close to this. i wonder if i shouldn't just show a family of flat-GC models instead of a single one, with one of them having 0-weighted GC...
the other main criticism was of the slide with all the equations. this is the main thing i think i need to address. i need to remake that slide so it more naturally presents the system of processes that the equations represent. some sort of flow or wiring diagram, showing how the equations are nested...
also need to modify the explanation of the contrast randomization; not add information, but make clearer that the two contrast weighting vectors are indeed random and (basically) independent.
Monday, September 03, 2012
two out of three ain't enough
okay, so, really, i spent the labor day weekend watching youtube videos, looking at funny gifs, reading the news, and other random things, while running half-baked model simulations for the blur adaptation revision.
first thing i did was to run the video-based model through the experiment on the same three adaptation levels used in the original experiment. it worked at an operational level, i.e. it matched sharper things with sharper things and blurrier things with blurrier things, and the effects of the adaptors were correctly ordered - it didn't do anything crazy. on an empirical level, though, it was wrong.
for the original subjects, and most of the replication subjects, the perceived normal after blank adaptation should be matched to a slightly sharpened normal-video-adapted test; the simulation did the opposite. not a huge problem, but like i said, against the trend.
bigger problem is that the simulation failed to get the 'gain' peak for the normal adaptation condition; instead, gain just increased with sharpness of the adaptor. now i'm rerunning the simulation with some basic changes (adding white noise to the spatial inputs, which i don't think will work - might make it worse by increasing the effective sharpness of all inputs - but might have something of a CSF effect; and windowing the edges, which i should have done from the start).
one funny thing: even though the gain for the sharp adaptor is too high (being higher than for the normal adaptor), the gains for the normal and blurred adaptors are *exactly* the same as the means for the original three subjects: enough to make me think i was doing something horribly weirdly wrong in the spreadsheet, but there it is:

weird, but too good to be true. undoubtedly, every change to the model will change all of the simulation measurements, and the sim is definitely as noisy as the humans - even the same one run again would not get the same values.
first thing i did was to run the video-based model through the experiment on the same three adaptation levels used in the original experiment. it worked at an operational level, i.e. it matched sharper things with sharper things and blurrier things with blurrier things, and the effects of the adaptors were correctly ordered - it didn't do anything crazy. on an empirical level, though, it was wrong.
for the original subjects, and most of the replication subjects, the perceived normal after blank adaptation should be matched to a slightly sharpened normal-video-adapted test; the simulation did the opposite. not a huge problem, but like i said, against the trend.
bigger problem is that the simulation failed to get the 'gain' peak for the normal adaptation condition; instead, gain just increased with sharpness of the adaptor. now i'm rerunning the simulation with some basic changes (adding white noise to the spatial inputs, which i don't think will work - might make it worse by increasing the effective sharpness of all inputs - but might have something of a CSF effect; and windowing the edges, which i should have done from the start).
one funny thing: even though the gain for the sharp adaptor is too high (being higher than for the normal adaptor), the gains for the normal and blurred adaptors are *exactly* the same as the means for the original three subjects: enough to make me think i was doing something horribly weirdly wrong in the spreadsheet, but there it is:
weird, but too good to be true. undoubtedly, every change to the model will change all of the simulation measurements, and the sim is definitely as noisy as the humans - even the same one run again would not get the same values.
Sunday, September 02, 2012
random
I seem to have gotten into treating this thing as a migraine journal, so here: headache last night (Saturday). Strange one, came on slowly, from mid-afternoon, increased gradually until 10 or so, when it was actually pretty irritating. May be something else. It's kind of still here, vaguely. Front of the head, above-behind the eyes, but something about it is different. Dunno.
As for work, I should have done more this weekend. I have 3 current main foci: FVM presentation, blur adaptation revision, and R01 application.
The presentation is >90% done. I'm leaving it for a few days.
The blur adapt revision is 0% done. I'm trying to figure out what "simple" model to replace the section in the paper with. If I can't get it to work by the end of the week, I think I'll have to stick with the original "complicated" model, and *add* material (thus making it *more* complicated) to explain why the simple version can't be easily adapted to work. What this entails is about an hour of programming and 24 hours of running the simulations/measurements so I can see the results and decide on what isn't working and make changes and repeat the process. In the meantime, I do nothing productive. So:
R01 application is... well... I don't want to do it. It's futile, but it's my job. Will start soon. Should have started this weekend.
As for work, I should have done more this weekend. I have 3 current main foci: FVM presentation, blur adaptation revision, and R01 application.
The presentation is >90% done. I'm leaving it for a few days.
The blur adapt revision is 0% done. I'm trying to figure out what "simple" model to replace the section in the paper with. If I can't get it to work by the end of the week, I think I'll have to stick with the original "complicated" model, and *add* material (thus making it *more* complicated) to explain why the simple version can't be easily adapted to work. What this entails is about an hour of programming and 24 hours of running the simulations/measurements so I can see the results and decide on what isn't working and make changes and repeat the process. In the meantime, I do nothing productive. So:
R01 application is... well... I don't want to do it. It's futile, but it's my job. Will start soon. Should have started this weekend.
Subscribe to:
Comments (Atom)