VR06Paper

Purpose of the Paper

?

Things we can show

And how we’re going to justify each claim.

• Among the measures we took there is no (very little?) statistically significant difference between Real, Cowl, and VRWalk.
  • ?

• Video game experience correlates with performance (exposure, consistency).
  • Comparing performance metrics in joystick cond with gaming-hours and joystick-hours.

• Real5 is faster than Real0 (measure of learning)
  • Also: Cowls are faster when they come at the end.
  • Compare peakvel_vel and vel_mean between conditions Real0, Real5, Cowl-earlier, and Cowl-later. This means we need to know which subjets had early/late Cowls.

• Good WIPers look like real-walking once the wip-data is filtered.
  • For each subject Look at the filtered WIP plots and compare that to the corresponding real plots (both 0 and 5). Note which ones are similar and which ones are really different.
  • Show this numerically by pointwise difference between velocity profiles. (kind of goes along with PCA).

Measures

• Questionaire data
  • hours of video game experience
  • gender
  • how people rated the conditions

• Computed numerical measures that are clear indicators of performance.
  • exposure metric
  • consistency of stopping point (stdev. of 12 points in XY-plane)
  • Number of corrective gestures.

• Computed numerical measures that have no obvious meaning.
  • dependent vars used in virte3 (peak/mean velocity, etc)
  • amt of curvature
  • smoothness? (not sure exactly how to measure it, though)
  • location of mean stopping point, relative to that of the REAL condition.

Identifying patterns in locomotion behavior

• Look at 2D overhead plots
• Look at forward-velocity profile and normal-acceleration profile
• Identify real-walking strategies and show that conditions {0,1,2,5} all fit into the same set of categories.
• Identify joystick and WIP strategies. (use un-filtered data)

PCA comparison of paths

Use PCA to do a point-wise comparison of paths, based on their forward-velocity profiles. This is one way of showing that real/cowl/vrwalk are alike and joystick/wip are much different.

What pairwise comparisons of conditions do we want to do?

Should we do all 6-choose-2 combinations, even if we only choose to look at a small subset of this data? Or would that be cumbersome? If i had to pick a sub-set, I’d do this:

• Compare Real(0) and Real(5) to eachother. Correlate {0,5}
• Use Real(5) as our real-world baseline to which all the other conditions, (1)-(4), will be compared. Correlate {1,5},{2,5},{3,5},{4,5}
• VRWalk(2) is a baseline to which the other VR cases are compared. Correlate {2,3},{2,4}
• To get at FOV issues, compare Cowl(1) to Real(5) and VRWalk(2). Correlate {1,5} and {1,2}

See DependentVariablesVirte4

To-Do List

These are mostly for Jeff.

• Point-wise Mean of the velocity curves. (comes with PCA)

• Stopping point consistency
  • Also do other performance measures like number of corrective gestures and smoothness.

These are mostly for someone besides Jeff

• Read through the comments and look for themes. (quoted from Mary)
  • Something else that needs to be done is to read through the comments looking for themes, categories of responses.
  • The way social scientists would do this is have two people independently review for themes/categories. Then those two people talk about the categories, agree to a set, do it again with the combined set, then have a third person look through it to see if they spot anything the other two didn’t. Then, you can start thinking about what it means.
  • How about one RA and Sarah to do the initial reviews, then someone who’s not in the critical path do the last look (perhaps Dr. Brooks would like to do this as a contribution to the work?)
  • I’ve printed them out and I’m sure I won’t be able to keep myself from reading them over,
  • Multiple colors of high lighters (real or electronic) are a good way to attack this.

• Figure out which cowls came 2nd and which cowls came 2nd-last. This will be useful for trying to quantify the speeding-up trianing effects that we’ve been seeing betwen early conditions and later conditions.

• Identify (by eye) the people who are good and bad at joystick and wip. I guess that comes along with the general task of identifying and categorizing the joystick strategies. I recall that we made comments on the experiment notes about who we thought were doing well or poorly; we should look at these comments.

Discussion Thread:

Email from Jeff to Mary and ChrisV

Ugh. The post-question data is uglier than i anticipated. many of the form elements were filled out incorrectly or not at all.
The relative-ranking questions (1–8) in part II may not be very useful.

It’s the part III responses (9,11,13) that i think will be most helpful. This is where people rated the 3 VR interfaces on an absolute scale, from 1 to 7. These are in columns AD-AF, AG-AI, and AJ-AL of the spreadsheet. We’ll want to look for a correlation between how people rated the 3 VR conditions and how they actually performed (i.e. exposure, stopping-point consistency, and all the other computed measures).
Since they’re rating these on an absolute scale (and the range of values that people choose will vary from person to person) some sort of normalization will be needed in order to put all the responses on comparable scales. I don’t know how survey designers deal with that problem, but i can try to take a stab at it:
For each question (general performance; move to where you wanna go; stop where you wanna stop) there are 3 items (VRWalk, Joystick, WIP). So we have a set of 3 values on a scale of 1 to 7. We can take the mean of the 3 values and subtract that out, which will give a common origin to the scales.
Then there remains some amount of variation in the magnitude of the responses (for instance, some people used the full spectrum, 1 to 7, and other people used just a small portion of it (2,3,4)). It seems to me that we don’t want to eliminate this kind of variation, since it gives some indication of whether people felt that there was a huge difference or a minor difference between the conditions.
Does this sound right? The only thing we’d do is re-center the 3 scales around the mean for each subject for each question?
So, for example, if the responses to “move where you wished to go” were {1,5,6} for {vrwalk,joy,wip} for a particular subject, we’d subtract out the mean (4), and get {-3,+1,+2}.

For the pre-questionnaire, the gender data is straight-forward. The video-game hours data is slightly less so. We ask the following:

• Over the past two years, what is the most you have played video games in a single week? Select one:
  • Less than 1 hour
  • 1–2 hours
  • 2–5 hours
  • 5–10 hours
  • 10–20 hours
  • 20–40 hours
  • 40 or more hours
    
• In what percent of your computer game-playing do you use a gamepad or joystick to move around in the environment? Select one:
  • 0% (Never) 20% 40% 60% 80% 100% (Always)

So there’s the counfounding factors of gaming experience and amount of joystick usage within the gaming experience. Are we interested in separating the gamers from the rest or separating the joystick-gamers from the rest? This will make a big difference. Some people played 40+ hours but have 0% joystick; others play less than 5 hours but have 100% joystick.
It seems like a reasonable way to estimate joystick experience specifically would be to multiply the number of hours by the joystick percentage. We may want to do our correllations with two different measures: one of game-hours times joystick%, another of just game-hours alone.

Also, I didn’t realize that we were asking about MAX video game hours per week. This data is somewhat useful, but I think next time we should ask for average hours instead. What we’re really after is the total amount of video game experience, of which the average is a better indicator than the max.

Chris,
Mary and I are working on generating as much information as we can, but it would be helpful if we had your suggestions on exactly which bits of information will be most relevent to the paper. It’s also helpful to know which bits of information are NOT worth looking in to. For instance, it’s probably not helpful to look for a correlation between the gaming-hours-with-joystick measure and the exposure values for VRWalk (though, of course, we do want to compare gaming-hours-with-joystick to the exposure values for Joystick).

So while you’re writing the outline, as you think of questions that you’d like to “ask” of the dataset, please pass them along to us or make a list at the bottom of the wiki page.
https://wwwx.cs.unc.edu/~feasel/wiki/VR06Paper
Doing this will really help to focus the analysis.

—Jeff

Email from Mary on status of Statistics, Sat 1630.

1. Writing up Stats
   I’m assuming that I’ll write up the PROCESS of the statistics work, but not the analysis of the output.

1. For the ANOVAs. The program is at:
   ~eve\common\project\virte\analysis_2005\SAS_2005\V4_2005\Programs\V4-mcw4.sas

—I’ve got the macros and the input and output all working properly.
—I need to visit Chris Wiesen one more time on Monday to verify that I have written
the contrast and estimate statements correctly for the V4 data.

## WHAT I NEED TO KNOW:
##* A. which dependent variables (dv) are we going to do analysis on?
##* B. what pairwise comparisons of conditions do we want to do?

## WHAT I’LL NEED FROM JEFF/MATLAB:
##* per path data for the DVs we want to analyze including (as we worked on Friday)
the trial number and the barrier number for each data record.

WHEN we have A. above, there will need to be lines added to the code to call the main macro once for each dv and line that will add the data for those calls to the macro to the final output data files.

This will likely take about an hour, then we can look at the data and try to figure out whether we can collapse trials/barriers to simplify our analysis.

It would be great to do this Monday afternoon.

1. For the INTERCLASS CORRELATIONS
   ~eve\common\project\virte\analysis_2005\SAS_2005\InterClassCorrelation

I need to get Chris W. to write me a macro that will subset the input data into data to input to the icc. (There is such a program for V3 (icc_V3Setup) in the V3_2005/Program folder)

The .doc file shows the input data file format: three columns
subject

In the data in the .doc file, they are peakdecel_acc for conditions 2 and 4

For V3, the ICC runs on data for one corner or wall and one dv. Since we haven’t shown yet that we can combine data from more than one wall/corner, that’s the way it has to be so far.

So, for V4, it will be one trial/barrier until we show we can combine subsets based on the outcome of the ANOVAs. With the level of granularity of the current program, there will be a zillion of them
(24 subjects x 3 trials x 4 barriers) = 288, then X N for the number of pairs we want to compare.

I NEED TO KNOW:
—what things do we want to run correlations on?

I NEED: file of data (excel or cvs) from the forms for any items we want to run comparisons on.

The icc’s will explode into an unmanageable task if we don’t think carefully about what we want to test before we start.