TUI Comparison Analysis and Results
- 1 Introduction
- 2 Impact of both device and display on performance
- 3 Questionnaire
- 4 Hypohteses testing for MT
- 5 Hypohteses testing for error rate
- 6 Hypohteses testing for SUS
- 7 Investigating the learning of each combination
1 Introduction
1.1 Loading necessary libraries
library('sciplot')
library('PerformanceAnalytics')
library('car')
library('lattice')
library('latticeExtra')
library(boot)1.2 Utilities functions
Geometric mean
geomMean <- function(x) {
return(10^mean(log10(x), na.rm=TRUE))
}Bootstrapped confidence interval: Returns the point estimate and confidence interval in an array of length 3 (this convenience function is borrowed from Jansen/Dragicevic)
samplemean <- function(x, d) {
return(geomMean(x[d]))
}
bootstrapCI <- function(datapoints) {
datapoints <- datapoints[!is.na(datapoints)]
# Compute the point estimate
pointEstimate <- samplemean(datapoints)
# Make the rest of the code deterministic
# if (deterministic) set.seed(0)
# Generate bootstrap replicates
b <- boot(datapoints, samplemean, R = 10000, parallel="multicore")
# Compute interval
ci <- boot.ci(b, type = "bca")
# Return the point estimate and CI bounds
# You can print the ci object for more info and debug
lowerBound <- ci$bca[4]
upperBound <- ci$bca[5]
return(c(pointEstimate, lowerBound, upperBound))
}Utility to retrieve only the lower and upperBound to be used with lineplot.CI
ci <- function(datapoints){
return(c(bootstrapCI(datapoints)[2], bootstrapCI(datapoints)[3]))
}1.3 Loading raw data
To perform the analysis yourself, download the raw data logs from the raw data folder
data <- data.frame()
p <- "../data/data_xp_3/logs/"
files <- list.files(path=p, pattern="*.csv")
for(file in files)
{
df <- read.csv(paste(p, file, sep='/'), sep=";")
data <- rbind(data, df)
}Renaming column for better readability
# rename column in dataframe to match Benjamin's pilot experiment's naming
names(data)[names(data)=="success"] <- "score"
names(data)[names(data)=="X..step"] <- "stage"
# rename factor values to more easily make the difference between device and display values
levels(data$graphic)[levels(data$graphic)=="slider"] <- "line"
levels(data$graphic)[levels(data$graphic)=="circular"] <- "circle"1.4 Checking the raw data and removing the outliers found
1.4.1 Basic checking of data
Checking number of participants is as expected (to find error in namings)
length(unique(data$name)) == 9## [1] TRUEChecking number of devices
length(unique(data$device)) == 3## [1] TRUEChecking number of displays
length(unique(data$graphic)) == 2## [1] TRUEChecking number of pointing tasks
length(unique(data$stage)) == 27## [1] TRUE1.4.2 Plot raw trajectories to find outliers
From the trajectories generated by a python script, we did not find any outliers (no trajectories were abnormally starting late, or showed problem with the tracking like it happenned in previous version of this experiment)
1.5 Formatting the data and computing variables
From the clean data, we compute index of difficulty, error and movement time
# convert timestamp to machine processable format
t <- strptime(data$datetime,"%Y-%m-%d %H:%M:%OS")
# precise calculation with 6 digits for seconds
op <- options(digits.secs=6)
# convert time to duration -- since beginning of the device's block
time <- t - t[1]
data$time <- time
# vector "endtarget" contains all lines' indexes where participants successfully acquired the target
endtarget <- which(data$score == 1)
# vector "starttarget" contains same lines' indexes but with an offset:
# first target starts at 1
# then the following starts of targets correspond to
# the line following (+1) the end of the a target (endtarget[1:(length(endtarget)-1)])
starttarget <- c(1,
endtarget[1:(length(endtarget)-1)]+1)
# calculation of indexes of difficulty
id <- log2(data$distance[endtarget]/data$targetsize[endtarget] + 1)
# number of errors
error<-c()
# in order to count the number of validation,
# we change 0=no motion for 1=validation
# and 1=motion to 0=no validation
data$motion <- (data$motion + 1) %% 2
# and change the labelling of column
names(data)[names(data)=="motion"] <- "validation"
# number of errors is (number of validation(s) - 1) for each target aquisition task
for (i in 1:(length(endtarget))){
error[i] <- sum(data$validation[starttarget[i]:endtarget[i]]) - 1
}
#duration to acquire a target (= movement time)
duration <- time[endtarget] - time[starttarget]1.6 Building main working dataframe
We build now the main working data frame linking, for each participant:
- Independant variables:
- Device,
- Display,
- Fitt’s index of difficulty, resulting from independant variables: Width of target and Distance to target.
- Dependant variables:
- Movement time,
- Error rate.
dataFrame <- data.frame(participant=data$name[endtarget],
device=data$device[endtarget],
display=data$graphic[endtarget],
id=id,
d=data$distance[endtarget],
w=data$targetsize[endtarget],
time=duration,
error=error)1.6.1 Checking the aggregated data
# check on the aggregated data frame
# checking number of trials per device X display
table(dataFrame$device, dataFrame$display)##
## circle line
## knob 243 243
## slider 243 243
## wheel 243 243Now we check the resulting distribution of trials on our distance and width independant variables.
# checking number of logs per target's width and distance (raw number and visually)
table(dataFrame$d, dataFrame$w)##
## 0.0125 0.025 0.05
## 0.1 162 162 162
## 0.2 162 162 162
## 0.4 162 162 1621.7 Aggregate data per participant
We aggregate data accross participant, as we are interested in how an average user can perform, not how efficient a single try can be.
Note that for movement time, we use the geometric mean as this was shown in [Sauro, 2010] that this is the best indicator for movement time.
dataFrameTimeAggregate <- aggregate(dataFrame$time,
by=list(dataFrame$participant,
dataFrame$device,
dataFrame$display,
dataFrame$id),
FUN=mean.geometric)
colnames(dataFrameTimeAggregate) <- c("participant", "device", "display", "id", "time")
dataFrameErrorAggregate <- aggregate(dataFrame$error,
by=list(dataFrame$participant,
dataFrame$device,
dataFrame$display,
dataFrame$id),
FUN=mean)
colnames(dataFrameErrorAggregate) <- c("participant", "device", "display", "id", "error")2 Impact of both device and display on performance
2.1 Movement time
We examine here the impact of both the motor and visual task (and their interaction) on movement time to acquire the targets.
rmt <- lineplot.CI(x.factor=dataFrameTimeAggregate$device,
response=dataFrameTimeAggregate$time,
group=dataFrameTimeAggregate$display,
err.width=0,
#type="p",
legend=TRUE,
main="Movement time per device and display",
ci.fun=ci,
col=c('red','blue'),
xlab="device",
ylab="Movement time (s)",
ylim=range(0,2.2))
rmt$vals## circle line
## knob 1.491642 1.620374
## slider 1.579207 1.324913
## wheel 1.907862 1.663334rmt$CI['knob', 'circle']## [[1]]
## [1] 1.386273 1.548397rmt$CI['knob', 'line']## [[1]]
## [1] 1.461469 1.677741rmt$CI['slider', 'circle']## [[1]]
## [1] 1.468102 1.643580rmt$CI['slider', 'line']## [[1]]
## [1] 1.237654 1.373867rmt$CI['wheel', 'circle']## [[1]]
## [1] 1.690220 1.997041rmt$CI['wheel', 'line']## [[1]]
## [1] 1.508842 1.728470Computing the gain in MT between all combinations
# gain in seconds
g <- rmt$vals['knob', 'circle'] - rmt$vals['slider', 'circle']
g## [1] -0.08756498# gain in %
g*100/rmt$vals['slider', 'circle']## [1] -5.544872# gain in seconds
g <- rmt$vals['knob', 'circle'] - rmt$vals['wheel', 'circle']
g## [1] -0.4162208# gain in %
g*100/rmt$vals['wheel', 'circle']## [1] -21.81608# gain in seconds
g <- rmt$vals['slider', 'line'] - rmt$vals['knob', 'line']
g## [1] -0.2954602# gain in %
g*100/rmt$vals['knob', 'line']## [1] -18.23408# gain in seconds
g <- rmt$vals['slider', 'line'] - rmt$vals['wheel', 'line']
g## [1] -0.3384202# gain in %
g*100/rmt$vals['wheel', 'line']## [1] -20.3459# gain in seconds
g <- rmt$vals['slider', 'circle'] - rmt$vals['wheel', 'circle']
g## [1] -0.3286558# gain in %
g*100/rmt$vals['wheel', 'circle']## [1] -17.22639# gain in seconds
g <- rmt$vals['knob', 'line'] - rmt$vals['wheel', 'line']
g## [1] -0.04295999# gain in %
g*100/rmt$vals['wheel', 'line']## [1] -2.582765# gain in seconds
g <- rmt$vals['knob', 'circle'] - rmt$vals['knob', 'line']
g## [1] -0.1287321# gain in %
g*100/rmt$vals['knob', 'line']## [1] -7.944592# gain in seconds
g <- rmt$vals['slider', 'line'] - rmt$vals['slider', 'circle']
g## [1] -0.2542931# gain in %
g*100/rmt$vals['slider', 'circle']## [1] -16.10259# gain in seconds
g <- rmt$vals['wheel', 'line'] - rmt$vals['wheel', 'circle']
g## [1] -0.2445287# gain in %
g*100/rmt$vals['wheel', 'circle']## [1] -12.816892.2 Error rate
We examine here the impact of both the motor and visual task (and their interaction) on the error rate / number of errors, to acquire the targets.
re <- lineplot.CI(x.factor=dataFrameErrorAggregate$device,
response=dataFrameErrorAggregate$error,
group=dataFrameErrorAggregate$display,
err.width=0,
#type="p",
legend=TRUE,
main="Error rate per device and display",
ci.fun=function(x) {c(mean(x) - 1.96*se(x), mean(x) + 1.96*se(x))},
#ci.fun=ci, #TODO the bootstrapping of the CI returns an error that I could not solve
col=c('red','blue'),
xlab="device",
ylab="error rate")
re$vals## circle line
## knob 0.10617284 0.06666667
## slider 0.09876543 0.07777778
## wheel 0.12839506 0.06296296re$CI['knob', 'circle']## [[1]]
## [1] 0.05235651 0.15998917re$CI['knob', 'line']## [[1]]
## [1] 0.02711748 0.10621585re$CI['slider', 'circle']## [[1]]
## [1] 0.04297499 0.15455588re$CI['slider', 'line']## [[1]]
## [1] 0.03463665 0.12091890re$CI['wheel', 'circle']## [[1]]
## [1] 0.06887253 0.18791759re$CI['wheel', 'line']## [[1]]
## [1] 0.03347571 0.092450213 Questionnaire
Loading questionnaire data
dataQuestionnaire <- read.csv("../data/data_xp_3/questionnaire/Tangible Interaction Comparison (Responses) - Form Responses 1.csv", header=TRUE, sep=',')Renaming column for better readability
# rename column in dataframe
names(dataQuestionnaire)[names(dataQuestionnaire)=="ID (ne pas remplir)"] <- "name"
names(dataQuestionnaire)[names(dataQuestionnaire)=="Date.de.naissance"] <- "Birthday"
colnames(dataQuestionnaire)[103] <- "Gender"
levels(dataQuestionnaire$Gender)[levels(dataQuestionnaire$Gender)=="un homme"] <- "male"
levels(dataQuestionnaire$Gender)[levels(dataQuestionnaire$Gender)=="une femme"] <- "female"3.1 Gender
table(dataQuestionnaire$Gender)##
## male female
## 4 53.2 Age
Get age of participant from date of birth and timestamp
#Change Age to a number format from the text format of the birthday
dataQuestionnaire$Birthday <- strptime(dataQuestionnaire$Birthday,"%m/%d/%Y")
dataQuestionnaire$Timestamp <- strptime(dataQuestionnaire$Timestamp, "%m/%d/%Y %H:%M:%S")
age = function(from, to) {
from_lt = as.POSIXlt(from)
to_lt = as.POSIXlt(to)
age = to_lt$year - from_lt$year
ifelse(to_lt$mon < from_lt$mon |
(to_lt$mon == from_lt$mon & to_lt$mday < from_lt$mday),
age - 1, age)
}
dataQuestionnaire$Age <- age(dataQuestionnaire$Birthday, dataQuestionnaire$Timestamp)
min(dataQuestionnaire$Age)## [1] 22max(dataQuestionnaire$Age)## [1] 38mean(dataQuestionnaire$Age)## [1] 31sd(dataQuestionnaire$Age)## [1] 4.8733973.3 Explanation about how we introduce the info about the ordering of the conditions in the questionnaire
Embbed the info about the device and display used in the questionnaire. To do this, we use the the ordering of devices (a latin square – with confusion between p7 and p8 during the experiment) :
| Participant | Technique 1 | Technique 2 | Technique 3 |
|---|---|---|---|
| P1 | slider | knob | wheel |
| P2 | knob | wheel | slider |
| P3 | wheel | slider | knob |
| P4 | slider | knob | wheel |
| P5 | knob | wheel | slider |
| P6 | wheel | slider | knob |
| P7 | knob | wheel | slider |
| P8 | slider | knob | wheel |
| P9 | wheel | slider | knob |
#p1 slider
dataQuestionnaire[1, 2:28]## I.think.that.I.would.like.to.use.this.system.frequently
## 1 4
## I.found.the.system.unnecessarily.complex
## 1 1
## I.thought.the.system.was.easy.to.use
## 1 4
## I.think.that.I.would.need.the.support.of.a.technical.person.to.be.able.to.use.this.system
## 1 1
## I.found.the.various.functions.in.this.system.were.well.integrated
## 1 4
## I.thought.there.was.too.much.inconsistency.in.this.system
## 1 2
## I.would.imagine.that.most.people.would.learn.to.use.this.system.very.quickly
## 1 5
## I.found.the.system.very.cumbersome.to.use
## 1 1
## I.felt.very.confident.using.the.system
## 1 4
## I.needed.to.learn.a.lot.of.things.before.I.could.get.going.with.this.system
## 1 2
## I.think.that.I.would.like.to.use.this.system.frequently.1
## 1 4
## I.found.the.system.unnecessarily.complex.1
## 1 1
## I.thought.the.system.was.easy.to.use.1
## 1 5
## I.think.that.I.would.need.the.support.of.a.technical.person.to.be.able.to.use.this.system.1
## 1 1
## I.found.the.various.functions.in.this.system.were.well.integrated.1
## 1 4
## I.thought.there.was.too.much.inconsistency.in.this.system.1
## 1 1
## I.would.imagine.that.most.people.would.learn.to.use.this.system.very.quickly.1
## 1 5
## I.found.the.system.very.cumbersome.to.use.1
## 1 1
## I.felt.very.confident.using.the.system.1
## 1 5
## I.needed.to.learn.a.lot.of.things.before.I.could.get.going.with.this.system.1
## 1 1
## Classez.les.2.représentations.visuelles.par.ordre.de.préférence...Curseur.Circulaire..CC..
## 1 2. Moins préféré.e
## Classez.les.2.représentations.visuelles.par.ordre.de.préférence...Curseur.Vertical..CV..
## 1 1. Plus préféré.e
## Classez.les.2.représentations.visuelles.de.la.plus.rapide.à.la.moins.rapide...Curseur.Circulaire..CC..
## 1 2. Moins rapide
## Classez.les.2.représentations.visuelles.de.la.plus.rapide.à.la.moins.rapide...Curseur.Vertical..CV..
## 1 1. Plus rapide
## Classez.les.2.représentations.visuelles.de.la.plus.précise.à.la.moins.précise...Curseur.Circulaire..CC..
## 1 2. Moins précise
## Classez.les.2.représentations.visuelles.de.la.plus.précise.à.la.moins.précise...Curseur.Vertical..CV..
## 1 1. Plus précise
## Commentaires
## 1#p1 knob
dataQuestionnaire[1, 29:55]## I.think.that.I.would.like.to.use.this.system.frequently.2
## 1 4
## I.found.the.system.unnecessarily.complex.2
## 1 1
## I.thought.the.system.was.easy.to.use.2
## 1 4
## I.think.that.I.would.need.the.support.of.a.technical.person.to.be.able.to.use.this.system.2
## 1 1
## I.found.the.various.functions.in.this.system.were.well.integrated.2
## 1 3
## I.thought.there.was.too.much.inconsistency.in.this.system.2
## 1 1
## I.would.imagine.that.most.people.would.learn.to.use.this.system.very.quickly.2
## 1 5
## I.found.the.system.very.cumbersome.to.use.2
## 1 1
## I.felt.very.confident.using.the.system.2
## 1 4
## I.needed.to.learn.a.lot.of.things.before.I.could.get.going.with.this.system.2
## 1 1
## I.think.that.I.would.like.to.use.this.system.frequently.3
## 1 4
## I.found.the.system.unnecessarily.complex.3
## 1 1
## I.thought.the.system.was.easy.to.use.3
## 1 5
## I.think.that.I.would.need.the.support.of.a.technical.person.to.be.able.to.use.this.system.3
## 1 1
## I.found.the.various.functions.in.this.system.were.well.integrated.3
## 1 3
## I.thought.there.was.too.much.inconsistency.in.this.system.3
## 1 1
## I.would.imagine.that.most.people.would.learn.to.use.this.system.very.quickly.3
## 1 5
## I.found.the.system.very.cumbersome.to.use.3
## 1 1
## I.felt.very.confident.using.the.system.3
## 1 5
## I.needed.to.learn.a.lot.of.things.before.I.could.get.going.with.this.system.3
## 1 1
## Classez.les.2.représentations.visuelles.par.ordre.de.préférence...Curseur.Circulaire..CC...1
## 1 1. Plus préféré.e
## Classez.les.2.représentations.visuelles.par.ordre.de.préférence...Curseur.Vertical..CV...1
## 1 2. Moins préféré.e
## Classez.les.2.représentations.visuelles.de.la.plus.rapide.à.la.moins.rapide...Curseur.Circulaire..CC...1
## 1 1. Plus rapide
## Classez.les.2.représentations.visuelles.de.la.plus.rapide.à.la.moins.rapide...Curseur.Vertical..CV...1
## 1 2. Moins rapide
## Classez.les.2.représentations.visuelles.de.la.plus.précise.à.la.moins.précise...Curseur.Circulaire..CC...1
## 1 2. Moins précise
## Classez.les.2.représentations.visuelles.de.la.plus.précise.à.la.moins.précise...Curseur.Vertical..CV...1
## 1 1. Plus précise
## Commentaires.1
## 1#p1 wheel
dataQuestionnaire[1, 56:82]## I.think.that.I.would.like.to.use.this.system.frequently.4
## 1 3
## I.found.the.system.unnecessarily.complex.4
## 1 1
## I.thought.the.system.was.easy.to.use.4
## 1 4
## I.think.that.I.would.need.the.support.of.a.technical.person.to.be.able.to.use.this.system.4
## 1 1
## I.found.the.various.functions.in.this.system.were.well.integrated.4
## 1 3
## I.thought.there.was.too.much.inconsistency.in.this.system.4
## 1 1
## I.would.imagine.that.most.people.would.learn.to.use.this.system.very.quickly.4
## 1 5
## I.found.the.system.very.cumbersome.to.use.4
## 1 1
## I.felt.very.confident.using.the.system.4
## 1 5
## I.needed.to.learn.a.lot.of.things.before.I.could.get.going.with.this.system.4
## 1 1
## I.think.that.I.would.like.to.use.this.system.frequently.5
## 1 4
## I.found.the.system.unnecessarily.complex.5
## 1 1
## I.thought.the.system.was.easy.to.use.5
## 1 5
## I.think.that.I.would.need.the.support.of.a.technical.person.to.be.able.to.use.this.system.5
## 1 1
## I.found.the.various.functions.in.this.system.were.well.integrated.5
## 1 3
## I.thought.there.was.too.much.inconsistency.in.this.system.5
## 1 2
## I.would.imagine.that.most.people.would.learn.to.use.this.system.very.quickly.5
## 1 5
## I.found.the.system.very.cumbersome.to.use.5
## 1 1
## I.felt.very.confident.using.the.system.5
## 1 4
## I.needed.to.learn.a.lot.of.things.before.I.could.get.going.with.this.system.5
## 1 1
## Classez.les.2.représentations.visuelles.par.ordre.de.préférence...Curseur.Circulaire..CC...2
## 1 2. Moins préféré.e
## Classez.les.2.représentations.visuelles.par.ordre.de.préférence...Curseur.Vertical..CV...2
## 1 1. Plus préféré.e
## Classez.les.2.représentations.visuelles.de.la.plus.rapide.à.la.moins.rapide...Curseur.Circulaire..CC...2
## 1 2. Moins rapide
## Classez.les.2.représentations.visuelles.de.la.plus.rapide.à.la.moins.rapide...Curseur.Vertical..CV...2
## 1 1. Plus rapide
## Classez.les.2.représentations.visuelles.de.la.plus.précise.à.la.moins.précise...Curseur.Circulaire..CC...2
## 1 2. Moins précise
## Classez.les.2.représentations.visuelles.de.la.plus.précise.à.la.moins.précise...Curseur.Vertical..CV...2
## 1 1. Plus précise
## Commentaires.2
## 1#p2 knob
dataQuestionnaire[2, 2:28]## I.think.that.I.would.like.to.use.this.system.frequently
## 2 4
## I.found.the.system.unnecessarily.complex
## 2 1
## I.thought.the.system.was.easy.to.use
## 2 1
## I.think.that.I.would.need.the.support.of.a.technical.person.to.be.able.to.use.this.system
## 2 1
## I.found.the.various.functions.in.this.system.were.well.integrated
## 2 3
## I.thought.there.was.too.much.inconsistency.in.this.system
## 2 1
## I.would.imagine.that.most.people.would.learn.to.use.this.system.very.quickly
## 2 5
## I.found.the.system.very.cumbersome.to.use
## 2 2
## I.felt.very.confident.using.the.system
## 2 5
## I.needed.to.learn.a.lot.of.things.before.I.could.get.going.with.this.system
## 2 1
## I.think.that.I.would.like.to.use.this.system.frequently.1
## 2 5
## I.found.the.system.unnecessarily.complex.1
## 2 1
## I.thought.the.system.was.easy.to.use.1
## 2 5
## I.think.that.I.would.need.the.support.of.a.technical.person.to.be.able.to.use.this.system.1
## 2 1
## I.found.the.various.functions.in.this.system.were.well.integrated.1
## 2 3
## I.thought.there.was.too.much.inconsistency.in.this.system.1
## 2 1
## I.would.imagine.that.most.people.would.learn.to.use.this.system.very.quickly.1
## 2 5
## I.found.the.system.very.cumbersome.to.use.1
## 2 2
## I.felt.very.confident.using.the.system.1
## 2 4
## I.needed.to.learn.a.lot.of.things.before.I.could.get.going.with.this.system.1
## 2 1
## Classez.les.2.représentations.visuelles.par.ordre.de.préférence...Curseur.Circulaire..CC..
## 2 1. Plus préféré.e
## Classez.les.2.représentations.visuelles.par.ordre.de.préférence...Curseur.Vertical..CV..
## 2 2. Moins préféré.e
## Classez.les.2.représentations.visuelles.de.la.plus.rapide.à.la.moins.rapide...Curseur.Circulaire..CC..
## 2 1. Plus rapide
## Classez.les.2.représentations.visuelles.de.la.plus.rapide.à.la.moins.rapide...Curseur.Vertical..CV..
## 2 2. Moins rapide
## Classez.les.2.représentations.visuelles.de.la.plus.précise.à.la.moins.précise...Curseur.Circulaire..CC..
## 2 1. Plus précise
## Classez.les.2.représentations.visuelles.de.la.plus.précise.à.la.moins.précise...Curseur.Vertical..CV..
## 2 2. Moins précise
## Commentaires
## 2 CC : moins d'erreurs d'orientation; circulaire plus intuitif pour la direction dans laquelle tourner. Avec CV, j,ai fait des erreurs en voulant aller vers le haut ou le bas mais en commencant par tourner dans le mauvais sens.#p2 wheel
dataQuestionnaire[2, 29:55]## I.think.that.I.would.like.to.use.this.system.frequently.2
## 2 3
## I.found.the.system.unnecessarily.complex.2
## 2 1
## I.thought.the.system.was.easy.to.use.2
## 2 4
## I.think.that.I.would.need.the.support.of.a.technical.person.to.be.able.to.use.this.system.2
## 2 1
## I.found.the.various.functions.in.this.system.were.well.integrated.2
## 2 3
## I.thought.there.was.too.much.inconsistency.in.this.system.2
## 2 2
## I.would.imagine.that.most.people.would.learn.to.use.this.system.very.quickly.2
## 2 4
## I.found.the.system.very.cumbersome.to.use.2
## 2 2
## I.felt.very.confident.using.the.system.2
## 2 4
## I.needed.to.learn.a.lot.of.things.before.I.could.get.going.with.this.system.2
## 2 2
## I.think.that.I.would.like.to.use.this.system.frequently.3
## 2 2
## I.found.the.system.unnecessarily.complex.3
## 2 2
## I.thought.the.system.was.easy.to.use.3
## 2 3
## I.think.that.I.would.need.the.support.of.a.technical.person.to.be.able.to.use.this.system.3
## 2 1
## I.found.the.various.functions.in.this.system.were.well.integrated.3
## 2 3
## I.thought.there.was.too.much.inconsistency.in.this.system.3
## 2 2
## I.would.imagine.that.most.people.would.learn.to.use.this.system.very.quickly.3
## 2 3
## I.found.the.system.very.cumbersome.to.use.3
## 2 3
## I.felt.very.confident.using.the.system.3
## 2 3
## I.needed.to.learn.a.lot.of.things.before.I.could.get.going.with.this.system.3
## 2 2
## Classez.les.2.représentations.visuelles.par.ordre.de.préférence...Curseur.Circulaire..CC...1
## 2 2. Moins préféré.e
## Classez.les.2.représentations.visuelles.par.ordre.de.préférence...Curseur.Vertical..CV...1
## 2 1. Plus préféré.e
## Classez.les.2.représentations.visuelles.de.la.plus.rapide.à.la.moins.rapide...Curseur.Circulaire..CC...1
## 2 2. Moins rapide
## Classez.les.2.représentations.visuelles.de.la.plus.rapide.à.la.moins.rapide...Curseur.Vertical..CV...1
## 2 1. Plus rapide
## Classez.les.2.représentations.visuelles.de.la.plus.précise.à.la.moins.précise...Curseur.Circulaire..CC...1
## 2 2. Moins précise
## Classez.les.2.représentations.visuelles.de.la.plus.précise.à.la.moins.précise...Curseur.Vertical..CV...1
## 2 1. Plus précise
## Commentaires.1
## 2 CC ; ewrreurs faites en partant dans la mauvaise direction, ce aui n;arrivait pas avec CV. Mais precision me semble qssez similaire, juste le temps est plus important avec CC en cas de faux depart.#p2 slider
dataQuestionnaire[2, 56:82]## I.think.that.I.would.like.to.use.this.system.frequently.4
## 2 4
## I.found.the.system.unnecessarily.complex.4
## 2 2
## I.thought.the.system.was.easy.to.use.4
## 2 4
## I.think.that.I.would.need.the.support.of.a.technical.person.to.be.able.to.use.this.system.4
## 2 1
## I.found.the.various.functions.in.this.system.were.well.integrated.4
## 2 3
## I.thought.there.was.too.much.inconsistency.in.this.system.4
## 2 2
## I.would.imagine.that.most.people.would.learn.to.use.this.system.very.quickly.4
## 2 4
## I.found.the.system.very.cumbersome.to.use.4
## 2 2
## I.felt.very.confident.using.the.system.4
## 2 4
## I.needed.to.learn.a.lot.of.things.before.I.could.get.going.with.this.system.4
## 2 2
## I.think.that.I.would.like.to.use.this.system.frequently.5
## 2 4
## I.found.the.system.unnecessarily.complex.5
## 2 1
## I.thought.the.system.was.easy.to.use.5
## 2 5
## I.think.that.I.would.need.the.support.of.a.technical.person.to.be.able.to.use.this.system.5
## 2 1
## I.found.the.various.functions.in.this.system.were.well.integrated.5
## 2 3
## I.thought.there.was.too.much.inconsistency.in.this.system.5
## 2 2
## I.would.imagine.that.most.people.would.learn.to.use.this.system.very.quickly.5
## 2 5
## I.found.the.system.very.cumbersome.to.use.5
## 2 2
## I.felt.very.confident.using.the.system.5
## 2 4
## I.needed.to.learn.a.lot.of.things.before.I.could.get.going.with.this.system.5
## 2 2
## Classez.les.2.représentations.visuelles.par.ordre.de.préférence...Curseur.Circulaire..CC...2
## 2 2. Moins préféré.e
## Classez.les.2.représentations.visuelles.par.ordre.de.préférence...Curseur.Vertical..CV...2
## 2 1. Plus préféré.e
## Classez.les.2.représentations.visuelles.de.la.plus.rapide.à.la.moins.rapide...Curseur.Circulaire..CC...2
## 2 2. Moins rapide
## Classez.les.2.représentations.visuelles.de.la.plus.rapide.à.la.moins.rapide...Curseur.Vertical..CV...2
## 2 1. Plus rapide
## Classez.les.2.représentations.visuelles.de.la.plus.précise.à.la.moins.précise...Curseur.Circulaire..CC...2
## 2 2. Moins précise
## Classez.les.2.représentations.visuelles.de.la.plus.précise.à.la.moins.précise...Curseur.Vertical..CV...2
## 2 1. Plus précise
## Commentaires.2
## 2 meme chose qu'avant, mais moins marque : moins de faux departs, meme s'il y a encore avec CC.#etc.The ordering of the display was randomized, but logged in the data.
d <- data[which(data$name == 'p1' & data$device == 'slider'),]
unique(d$graphic)## [1] circle line
## Levels: circle lined <- data[which(data$name == 'p1' & data$device == 'knob'),]
unique(d$graphic)## [1] circle line
## Levels: circle lined <- data[which(data$name == 'p1' & data$device == 'wheel'),]
unique(d$graphic)## [1] line circle
## Levels: circle lined <- data[which(data$name == 'p2' & data$device == 'knob'),]
unique(d$graphic)## [1] line circle
## Levels: circle lined <- data[which(data$name == 'p2' & data$device == 'wheel'),]
unique(d$graphic)## [1] line circle
## Levels: circle lined <- data[which(data$name == 'p2' & data$device == 'slider'),]
unique(d$graphic)## [1] circle line
## Levels: circle line#etc.3.4 SUS scores
3.4.1 Building the data from SUS questions as a separate data frame
#each participant did 6 combinations of device x display, and answered 10 questions of SUS for each combination
participant <- c(rep("p1", 60),
rep("p2", 60),
rep("p3", 60),
rep("p4", 60),
rep("p5", 60),
rep("p6", 60),
rep("p7", 60),
rep("p8", 60),
rep("p9", 60))
# each participant did twice the SUS questionnaire (10 questions) for each device
d <- c(rep("knob", 20), rep("wheel", 20), rep("slider", 20))
device <- rep(d, 9)
# each participant did three times the SUS questionnaire (10 questions) for each display
d <- c(rep("line", 10), rep("circle", 10))
d <- c(rep(d, 3))
display <- rep(d, 9)
#each participant answered each question of the SUS questionnaire 6 times
d <- c("I.think.that.I.would.like.to.use.this.system.frequently",
"I.found.the.system.unnecessarily.complex",
"I.thought.the.system.was.easy.to.use",
"I.think.that.I.would.need.the.support.of.a.technical.person.to.be.able.to.use.this.system",
"I.found.the.various.functions.in.this.system.were.well.integrated",
"I.thought.there.was.too.much.inconsistency.in.this.system",
"I.would.imagine.that.most.people.would.learn.to.use.this.system.very.quickly",
"I.found.the.system.very.cumbersome.to.use",
"I.felt.very.confident.using.the.system",
"I.needed.to.learn.a.lot.of.things.before.I.could.get.going.with.this.system")
d <- rep(d, 6)
Q <- rep(d, 9)
rating <- c(NA)
dataSUS <- data.frame(participant, device, display, Q, rating)
# P1 ordering was: slider | knob | wheel |
# p1 started with slider and the first display (logged) is d$graphic[1]
d <- data[which(data$name == 'p1' & data$device == 'slider'),]
dataSUS[which(dataSUS$participant == 'p1' &
dataSUS$device == 'slider' &
dataSUS$display == unique(d$graphic)[1]),]$rating <- t(dataQuestionnaire[1, 2:11])
dataSUS[which(dataSUS$participant == 'p1' &
dataSUS$device == 'slider' &
dataSUS$display == unique(d$graphic)[2]),]$rating <- t(dataQuestionnaire[1, 12:21])
d <- data[which(data$name == 'p1' & data$device == 'knob'),]
dataSUS[which(dataSUS$participant == 'p1' &
dataSUS$device == 'knob' &
dataSUS$display == unique(d$graphic)[1]),]$rating <- t(dataQuestionnaire[1, 29:38])
dataSUS[which(dataSUS$participant == 'p1' &
dataSUS$device == 'knob' &
dataSUS$display == unique(d$graphic)[2]),]$rating <- t(dataQuestionnaire[1, 39:48])
d <- data[which(data$name == 'p1' & data$device == 'wheel'),]
dataSUS[which(dataSUS$participant == 'p1' &
dataSUS$device == 'wheel' &
dataSUS$display == unique(d$graphic)[1]),]$rating <- t(dataQuestionnaire[1, 56:65])
dataSUS[which(dataSUS$participant == 'p1' &
dataSUS$device == 'wheel' &
dataSUS$display == unique(d$graphic)[2]),]$rating <- t(dataQuestionnaire[1, 66:75])
# P2 ordering was: knob | wheel | slider |
d <- data[which(data$name == 'p2' & data$device == 'knob'),]
dataSUS[which(dataSUS$participant == 'p2' &
dataSUS$device == 'knob' &
dataSUS$display == unique(d$graphic)[1]),]$rating <- t(dataQuestionnaire[2, 2:11])
dataSUS[which(dataSUS$participant == 'p2' &
dataSUS$device == 'knob' &
dataSUS$display == unique(d$graphic)[2]),]$rating <- t(dataQuestionnaire[2, 12:21])
d <- data[which(data$name == 'p2' & data$device == 'wheel'),]
dataSUS[which(dataSUS$participant == 'p2' &
dataSUS$device == 'wheel' &
dataSUS$display == unique(d$graphic)[1]),]$rating <- t(dataQuestionnaire[2, 29:38])
dataSUS[which(dataSUS$participant == 'p2' &
dataSUS$device == 'wheel' &
dataSUS$display == unique(d$graphic)[2]),]$rating <- t(dataQuestionnaire[2, 39:48])
d <- data[which(data$name == 'p2' & data$device == 'slider'),]
dataSUS[which(dataSUS$participant == 'p2' &
dataSUS$device == 'slider' &
dataSUS$display == unique(d$graphic)[1]),]$rating <- t(dataQuestionnaire[2, 56:65])
dataSUS[which(dataSUS$participant == 'p2' &
dataSUS$device == 'slider' &
dataSUS$display == unique(d$graphic)[2]),]$rating <- t(dataQuestionnaire[2, 66:75])
# P3 | wheel | slider | knob |
d <- data[which(data$name == 'p3' & data$device == 'wheel'),]
dataSUS[which(dataSUS$participant == 'p3' &
dataSUS$device == 'wheel' &
dataSUS$display == unique(d$graphic)[1]),]$rating <- t(dataQuestionnaire[3, 2:11])
dataSUS[which(dataSUS$participant == 'p3' &
dataSUS$device == 'wheel' &
dataSUS$display == unique(d$graphic)[2]),]$rating <- t(dataQuestionnaire[3, 12:21])
d <- data[which(data$name == 'p3' & data$device == 'slider'),]
dataSUS[which(dataSUS$participant == 'p3' &
dataSUS$device == 'slider' &
dataSUS$display == unique(d$graphic)[1]),]$rating <- t(dataQuestionnaire[3, 29:38])
dataSUS[which(dataSUS$participant == 'p3' &
dataSUS$device == 'slider' &
dataSUS$display == unique(d$graphic)[2]),]$rating <- t(dataQuestionnaire[3, 39:48])
d <- data[which(data$name == 'p3' & data$device == 'knob'),]
dataSUS[which(dataSUS$participant == 'p3' &
dataSUS$device == 'knob' &
dataSUS$display == unique(d$graphic)[1]),]$rating <- t(dataQuestionnaire[3, 56:65])
dataSUS[which(dataSUS$participant == 'p3' &
dataSUS$device == 'knob' &
dataSUS$display == unique(d$graphic)[2]),]$rating <- t(dataQuestionnaire[3, 66:75])
# P4 | slider | knob | wheel |
d <- data[which(data$name == 'p4' & data$device == 'slider'),]
dataSUS[which(dataSUS$participant == 'p4' &
dataSUS$device == 'slider' &
dataSUS$display == unique(d$graphic)[1]),]$rating <- t(dataQuestionnaire[4, 2:11])
dataSUS[which(dataSUS$participant == 'p4' &
dataSUS$device == 'slider' &
dataSUS$display == unique(d$graphic)[2]),]$rating <- t(dataQuestionnaire[4, 12:21])
d <- data[which(data$name == 'p4' & data$device == 'knob'),]
dataSUS[which(dataSUS$participant == 'p4' &
dataSUS$device == 'knob' &
dataSUS$display == unique(d$graphic)[1]),]$rating <- t(dataQuestionnaire[4, 29:38])
dataSUS[which(dataSUS$participant == 'p4' &
dataSUS$device == 'knob' &
dataSUS$display == unique(d$graphic)[2]),]$rating <- t(dataQuestionnaire[4, 39:48])
d <- data[which(data$name == 'p4' & data$device == 'wheel'),]
dataSUS[which(dataSUS$participant == 'p4' &
dataSUS$device == 'wheel' &
dataSUS$display == unique(d$graphic)[1]),]$rating <- t(dataQuestionnaire[4, 56:65])
dataSUS[which(dataSUS$participant == 'p4' &
dataSUS$device == 'wheel' &
dataSUS$display == unique(d$graphic)[2]),]$rating <- t(dataQuestionnaire[4, 66:75])
# P5 | knob | wheel | slider |
d <- data[which(data$name == 'p5' & data$device == 'knob'),]
dataSUS[which(dataSUS$participant == 'p5' &
dataSUS$device == 'knob' &
dataSUS$display == unique(d$graphic)[1]),]$rating <- t(dataQuestionnaire[5, 2:11])
dataSUS[which(dataSUS$participant == 'p5' &
dataSUS$device == 'knob' &
dataSUS$display == unique(d$graphic)[2]),]$rating <- t(dataQuestionnaire[5, 12:21])
d <- data[which(data$name == 'p5' & data$device == 'wheel'),]
dataSUS[which(dataSUS$participant == 'p5' &
dataSUS$device == 'wheel' &
dataSUS$display == unique(d$graphic)[1]),]$rating <- t(dataQuestionnaire[5, 29:38])
dataSUS[which(dataSUS$participant == 'p5' &
dataSUS$device == 'wheel' &
dataSUS$display == unique(d$graphic)[2]),]$rating <- t(dataQuestionnaire[5, 39:48])
d <- data[which(data$name == 'p5' & data$device == 'slider'),]
dataSUS[which(dataSUS$participant == 'p5' &
dataSUS$device == 'slider' &
dataSUS$display == unique(d$graphic)[1]),]$rating <- t(dataQuestionnaire[5, 56:65])
dataSUS[which(dataSUS$participant == 'p5' &
dataSUS$device == 'slider' &
dataSUS$display == unique(d$graphic)[2]),]$rating <- t(dataQuestionnaire[5, 66:75])
# P6 | wheel | slider | knob |
d <- data[which(data$name == 'p6' & data$device == 'wheel'),]
dataSUS[which(dataSUS$participant == 'p6' &
dataSUS$device == 'wheel' &
dataSUS$display == unique(d$graphic)[1]),]$rating <- t(dataQuestionnaire[6, 2:11])
dataSUS[which(dataSUS$participant == 'p6' &
dataSUS$device == 'wheel' &
dataSUS$display == unique(d$graphic)[2]),]$rating <- t(dataQuestionnaire[6, 12:21])
d <- data[which(data$name == 'p6' & data$device == 'slider'),]
dataSUS[which(dataSUS$participant == 'p6' &
dataSUS$device == 'slider' &
dataSUS$display == unique(d$graphic)[1]),]$rating <- t(dataQuestionnaire[6, 29:38])
dataSUS[which(dataSUS$participant == 'p6' &
dataSUS$device == 'slider' &
dataSUS$display == unique(d$graphic)[2]),]$rating <- t(dataQuestionnaire[6, 39:48])
d <- data[which(data$name == 'p6' & data$device == 'knob'),]
dataSUS[which(dataSUS$participant == 'p6' &
dataSUS$device == 'knob' &
dataSUS$display == unique(d$graphic)[1]),]$rating <- t(dataQuestionnaire[6, 56:65])
dataSUS[which(dataSUS$participant == 'p6' &
dataSUS$device == 'knob' &
dataSUS$display == unique(d$graphic)[2]),]$rating <- t(dataQuestionnaire[6, 66:75])
# P7 | knob | wheel | slider |
d <- data[which(data$name == 'p7' & data$device == 'knob'),]
dataSUS[which(dataSUS$participant == 'p7' &
dataSUS$device == 'knob' &
dataSUS$display == unique(d$graphic)[1]),]$rating <- t(dataQuestionnaire[7, 2:11])
dataSUS[which(dataSUS$participant == 'p7' &
dataSUS$device == 'knob' &
dataSUS$display == unique(d$graphic)[2]),]$rating <- t(dataQuestionnaire[7, 12:21])
d <- data[which(data$name == 'p7' & data$device == 'wheel'),]
dataSUS[which(dataSUS$participant == 'p7' &
dataSUS$device == 'wheel' &
dataSUS$display == unique(d$graphic)[1]),]$rating <- t(dataQuestionnaire[7, 29:38])
dataSUS[which(dataSUS$participant == 'p7' &
dataSUS$device == 'wheel' &
dataSUS$display == unique(d$graphic)[2]),]$rating <- t(dataQuestionnaire[7, 39:48])
d <- data[which(data$name == 'p7' & data$device == 'slider'),]
dataSUS[which(dataSUS$participant == 'p7' &
dataSUS$device == 'slider' &
dataSUS$display == unique(d$graphic)[1]),]$rating <- t(dataQuestionnaire[7, 56:65])
dataSUS[which(dataSUS$participant == 'p7' &
dataSUS$device == 'slider' &
dataSUS$display == unique(d$graphic)[2]),]$rating <- t(dataQuestionnaire[7, 66:75])
# P8 | slider | knob | wheel |
d <- data[which(data$name == 'p8' & data$device == 'slider'),]
dataSUS[which(dataSUS$participant == 'p8' &
dataSUS$device == 'slider' &
dataSUS$display == unique(d$graphic)[1]),]$rating <- t(dataQuestionnaire[8, 2:11])
dataSUS[which(dataSUS$participant == 'p8' &
dataSUS$device == 'slider' &
dataSUS$display == unique(d$graphic)[2]),]$rating <- t(dataQuestionnaire[8, 12:21])
d <- data[which(data$name == 'p8' & data$device == 'knob'),]
dataSUS[which(dataSUS$participant == 'p8' &
dataSUS$device == 'knob' &
dataSUS$display == unique(d$graphic)[1]),]$rating <- t(dataQuestionnaire[8, 29:38])
dataSUS[which(dataSUS$participant == 'p8' &
dataSUS$device == 'knob' &
dataSUS$display == unique(d$graphic)[2]),]$rating <- t(dataQuestionnaire[8, 39:48])
d <- data[which(data$name == 'p8' & data$device == 'wheel'),]
dataSUS[which(dataSUS$participant == 'p8' &
dataSUS$device == 'wheel' &
dataSUS$display == unique(d$graphic)[1]),]$rating <- t(dataQuestionnaire[8, 56:65])
dataSUS[which(dataSUS$participant == 'p8' &
dataSUS$device == 'wheel' &
dataSUS$display == unique(d$graphic)[2]),]$rating <- t(dataQuestionnaire[8, 66:75])
# P9 | wheel | slider | knob |
d <- data[which(data$name == 'p9' & data$device == 'wheel'),]
dataSUS[which(dataSUS$participant == 'p9' &
dataSUS$device == 'wheel' &
dataSUS$display == unique(d$graphic)[1]),]$rating <- t(dataQuestionnaire[9, 2:11])
dataSUS[which(dataSUS$participant == 'p9' &
dataSUS$device == 'wheel' &
dataSUS$display == unique(d$graphic)[2]),]$rating <- t(dataQuestionnaire[9, 12:21])
d <- data[which(data$name == 'p9' & data$device == 'slider'),]
dataSUS[which(dataSUS$participant == 'p9' &
dataSUS$device == 'slider' &
dataSUS$display == unique(d$graphic)[1]),]$rating <- t(dataQuestionnaire[9, 29:38])
dataSUS[which(dataSUS$participant == 'p9' &
dataSUS$device == 'slider' &
dataSUS$display == unique(d$graphic)[2]),]$rating <- t(dataQuestionnaire[9, 39:48])
d <- data[which(data$name == 'p9' & data$device == 'knob'),]
dataSUS[which(dataSUS$participant == 'p9' &
dataSUS$device == 'knob' &
dataSUS$display == unique(d$graphic)[1]),]$rating <- t(dataQuestionnaire[9, 56:65])
dataSUS[which(dataSUS$participant == 'p9' &
dataSUS$device == 'knob' &
dataSUS$display == unique(d$graphic)[2]),]$rating <- t(dataQuestionnaire[9, 66:75])3.4.2 Now computing the SUS scores for each combination of devices and displays
3.4.2.1 First, making a data frame to embed the SUS scores
#each participant did 6 combinations of device x display
participant <- c(rep("p1", 6),
rep("p2", 6),
rep("p3", 6),
rep("p4", 6),
rep("p5", 6),
rep("p6", 6),
rep("p7", 6),
rep("p8", 6),
rep("p9", 6))
# each participant did twice the SUS questionnaire for each device
d <- c(rep("knob", 2), rep("wheel", 2), rep("slider", 2))
device <- rep(d, 9)
# each participant did three times the SUS questionnaire for each display
d <- c("line", "circle")
d <- c(rep(d, 3))
display <- rep(d, 9)
score <- c(NA)
dataSUSscores <- data.frame(participant, device, display, score)3.4.2.2 Second, calcuating the scores for each participant x device x display
dataSUS$rating[which(dataSUS$Q == 'I.think.that.I.would.like.to.use.this.system.frequently' |
dataSUS$Q == 'I.thought.the.system.was.easy.to.use' |
dataSUS$Q == 'I.found.the.various.functions.in.this.system.were.well.integrated' |
dataSUS$Q == 'I.would.imagine.that.most.people.would.learn.to.use.this.system.very.quickly' |
dataSUS$Q == 'I.felt.very.confident.using.the.system')] <- dataSUS$rating[which(dataSUS$Q == 'I.think.that.I.would.like.to.use.this.system.frequently' |
dataSUS$Q == 'I.thought.the.system.was.easy.to.use' |
dataSUS$Q == 'I.found.the.various.functions.in.this.system.were.well.integrated' |
dataSUS$Q == 'I.would.imagine.that.most.people.would.learn.to.use.this.system.very.quickly' |
dataSUS$Q == 'I.felt.very.confident.using.the.system')] - 1
dataSUS$rating[which(dataSUS$Q == 'I.found.the.system.unnecessarily.complex' |
dataSUS$Q == 'I.think.that.I.would.need.the.support.of.a.technical.person.to.be.able.to.use.this.system' |
dataSUS$Q == 'I.thought.there.was.too.much.inconsistency.in.this.system' |
dataSUS$Q == 'I.found.the.system.very.cumbersome.to.use' |
dataSUS$Q == 'I.needed.to.learn.a.lot.of.things.before.I.could.get.going.with.this.system')] <- 5 - dataSUS$rating[which(dataSUS$Q == 'I.found.the.system.unnecessarily.complex' |
dataSUS$Q == 'I.think.that.I.would.need.the.support.of.a.technical.person.to.be.able.to.use.this.system' |
dataSUS$Q == 'I.thought.there.was.too.much.inconsistency.in.this.system' |
dataSUS$Q == 'I.found.the.system.very.cumbersome.to.use' |
dataSUS$Q == 'I.needed.to.learn.a.lot.of.things.before.I.could.get.going.with.this.system')]
dataSUSscores <- aggregate(dataSUS$rating,
by=list(dataSUS$participant,
dataSUS$device,
dataSUS$display),
FUN=sum)
colnames(dataSUSscores) <- c("participant", "device", "display", "score")
dataSUSscores$score <- dataSUSscores$score * 2.53.4.2.3 Now display data
rsus <- lineplot.CI(x.factor=dataSUSscores$device,
response=dataSUSscores$score,
group=dataSUSscores$display,
err.width=0,
#type="p",
legend=TRUE,
main="SUS score per device and display",
ci.fun=ci,
col=c('red','blue'),
xlab="device",
ylab="SUS score",
ylim=range(0,100))
rsus$vals## circle line
## knob 91.38889 85.55556
## slider 81.94444 91.94444
## wheel 76.38889 87.77778rsus$CI['knob', 'circle']## [[1]]
## [1] 87.18651 95.63788rsus$CI['knob', 'line']## [[1]]
## [1] 80.68385 89.67696rsus$CI['slider', 'circle']## [[1]]
## [1] 67.70085 88.11861rsus$CI['slider', 'line']## [[1]]
## [1] 86.75497 96.04303rsus$CI['wheel', 'circle']## [[1]]
## [1] 52.84410 84.44491rsus$CI['wheel', 'line']## [[1]]
## [1] 79.64831 92.451323.6 General rankings
General ranking are in dataQuestionnaire[, 83:100]
Rename columns for better readability
colnames(dataQuestionnaire)[83] <- "general.preference.circle.knob"Rename all the values as number for easier description of the data, e.g., “1. Préféré” becomes ‘1’
levels(dataQuestionnaire$general.preference.circle.knob)[grepl('1', levels(dataQuestionnaire$general.preference.circle.knob))] <- "1"
levels(dataQuestionnaire$general.preference.circle.knob)[grepl('6', levels(dataQuestionnaire$general.preference.circle.knob))] <- "6"Do the same two steps fo the rest of the data
colnames(dataQuestionnaire)[84] <- "general.preference.line.knob"
levels(dataQuestionnaire$general.preference.line.knob)[grepl('1', levels(dataQuestionnaire$general.preference.line.knob))] <- "1"
levels(dataQuestionnaire$general.preference.line.knob)[grepl('6', levels(dataQuestionnaire$general.preference.line.knob))] <- "6"
colnames(dataQuestionnaire)[85] <- "general.preference.circle.slider"
levels(dataQuestionnaire$general.preference.circle.slider)[grepl('1', levels(dataQuestionnaire$general.preference.circle.slider))] <- "1"
levels(dataQuestionnaire$general.preference.circle.slider)[grepl('6', levels(dataQuestionnaire$general.preference.circle.slider))] <- "6"
colnames(dataQuestionnaire)[86] <- "general.preference.line.slider"
levels(dataQuestionnaire$general.preference.line.slider)[grepl('1', levels(dataQuestionnaire$general.preference.line.slider))] <- "1"
levels(dataQuestionnaire$general.preference.line.slider)[grepl('6', levels(dataQuestionnaire$general.preference.line.slider))] <- "6"
colnames(dataQuestionnaire)[87] <- "general.preference.circle.wheel"
levels(dataQuestionnaire$general.preference.circle.wheel)[grepl('1', levels(dataQuestionnaire$general.preference.circle.wheel))] <- "1"
levels(dataQuestionnaire$general.preference.circle.wheel)[grepl('6', levels(dataQuestionnaire$general.preference.circle.wheel))] <- "6"
colnames(dataQuestionnaire)[88] <- "general.preference.line.wheel"
levels(dataQuestionnaire$general.preference.line.wheel)[grepl('1', levels(dataQuestionnaire$general.preference.line.wheel))] <- "1"
levels(dataQuestionnaire$general.preference.line.wheel)[grepl('6', levels(dataQuestionnaire$general.preference.line.wheel))] <- "6"
colnames(dataQuestionnaire)[89] <- "general.rapidity.circle.knob"
levels(dataQuestionnaire$general.rapidity.circle.knob)[grepl('1', levels(dataQuestionnaire$general.rapidity.circle.knob))] <- "1"
levels(dataQuestionnaire$general.rapidity.circle.knob)[grepl('6', levels(dataQuestionnaire$general.rapidity.circle.knob))] <- "6"
colnames(dataQuestionnaire)[90] <- "general.rapidity.line.knob"
levels(dataQuestionnaire$general.rapidity.line.knob)[grepl('1', levels(dataQuestionnaire$general.rapidity.line.knob))] <- "1"
levels(dataQuestionnaire$general.rapidity.line.knob)[grepl('6', levels(dataQuestionnaire$general.rapidity.line.knob))] <- "6"
colnames(dataQuestionnaire)[91] <- "general.rapidity.circle.slider"
levels(dataQuestionnaire$general.rapidity.circle.slider)[grepl('1', levels(dataQuestionnaire$general.rapidity.circle.slider))] <- "1"
levels(dataQuestionnaire$general.rapidity.circle.slider)[grepl('6', levels(dataQuestionnaire$general.rapidity.circle.slider))] <- "6"
colnames(dataQuestionnaire)[92] <- "general.rapidity.line.slider"
levels(dataQuestionnaire$general.rapidity.line.slider)[grepl('1', levels(dataQuestionnaire$general.rapidity.line.slider))] <- "1"
levels(dataQuestionnaire$general.rapidity.line.slider)[grepl('6', levels(dataQuestionnaire$general.rapidity.line.slider))] <- "6"
colnames(dataQuestionnaire)[93] <- "general.rapidity.circle.wheel"
levels(dataQuestionnaire$general.rapidity.circle.wheel)[grepl('1', levels(dataQuestionnaire$general.rapidity.circle.wheel))] <- "1"
levels(dataQuestionnaire$general.rapidity.circle.wheel)[grepl('6', levels(dataQuestionnaire$general.rapidity.circle.wheel))] <- "6"
colnames(dataQuestionnaire)[94] <- "general.rapidity.line.wheel"
levels(dataQuestionnaire$general.rapidity.line.wheel)[grepl('1', levels(dataQuestionnaire$general.rapidity.line.wheel))] <- "1"
levels(dataQuestionnaire$general.rapidity.line.wheel)[grepl('6', levels(dataQuestionnaire$general.rapidity.line.wheel))] <- "6"
colnames(dataQuestionnaire)[95] <- "general.accuracy.circle.knob"
levels(dataQuestionnaire$general.accuracy.circle.knob)[grepl('1', levels(dataQuestionnaire$general.accuracy.circle.knob))] <- "1"
levels(dataQuestionnaire$general.accuracy.circle.knob)[grepl('6', levels(dataQuestionnaire$general.accuracy.circle.knob))] <- "6"
colnames(dataQuestionnaire)[96] <- "general.accuracy.line.knob"
levels(dataQuestionnaire$general.accuracy.line.knob)[grepl('1', levels(dataQuestionnaire$general.accuracy.line.knob))] <- "1"
levels(dataQuestionnaire$general.accuracy.line.knob)[grepl('6', levels(dataQuestionnaire$general.accuracy.line.knob))] <- "6"
colnames(dataQuestionnaire)[97] <- "general.accuracy.circle.slider"
levels(dataQuestionnaire$general.accuracy.circle.slider)[grepl('1', levels(dataQuestionnaire$general.accuracy.circle.slider))] <- "1"
levels(dataQuestionnaire$general.accuracy.circle.slider)[grepl('6', levels(dataQuestionnaire$general.accuracy.circle.slider))] <- "6"
colnames(dataQuestionnaire)[98] <- "general.accuracy.line.slider"
levels(dataQuestionnaire$general.accuracy.line.slider)[grepl('1', levels(dataQuestionnaire$general.accuracy.line.slider))] <- "1"
levels(dataQuestionnaire$general.accuracy.line.slider)[grepl('6', levels(dataQuestionnaire$general.accuracy.line.slider))] <- "6"
colnames(dataQuestionnaire)[99] <- "general.accuracy.circle.wheel"
levels(dataQuestionnaire$general.accuracy.circle.wheel)[grepl('1', levels(dataQuestionnaire$general.accuracy.circle.wheel))] <- "1"
levels(dataQuestionnaire$general.accuracy.circle.wheel)[grepl('6', levels(dataQuestionnaire$general.accuracy.circle.wheel))] <- "6"
colnames(dataQuestionnaire)[100] <- "general.accuracy.line.wheel"
levels(dataQuestionnaire$general.accuracy.line.wheel)[grepl('1', levels(dataQuestionnaire$general.accuracy.line.wheel))] <- "1"
levels(dataQuestionnaire$general.accuracy.line.wheel)[grepl('6', levels(dataQuestionnaire$general.accuracy.line.wheel))] <- "6"3.6.1 Utilities
Building data frame
buildGeneralRankingsDataFrame <- function(var) {
device <- c("knob", "knob", "wheel", "wheel", "slider", "slider")
display <- c("line", "circle", "line", "circle", "line", "circle")
nbOfFirst <- c(0)
nbOfSecond <- c(0)
nbOfThird <- c(0)
nbOfFourth <- c(0)
nbOfFifth <- c(0)
nbOfLast <- c(0)
df <- data.frame(device,
display,
nbOfFirst,
nbOfSecond,
nbOfThird,
nbOfFourth,
nbOfFifth,
nbOfLast)
df$nbOfFirst[which(df$device == "knob" & df$display == "line")] <-
sum(dataQuestionnaire[[paste(paste("general", var, sep = "."), "line.knob", sep = ".")]] == 1)
df$nbOfFirst[which(df$device == "knob" & df$display == "circle")] <-
sum(dataQuestionnaire[[paste(paste("general", var, sep = "."), "circle.knob", sep = ".")]] == 1)
df$nbOfSecond[which(df$device == "knob" & df$display == "line")] <-
sum(dataQuestionnaire[[paste(paste("general", var, sep = "."), "line.knob", sep = ".")]] == 2)
df$nbOfSecond[which(df$device == "knob" & df$display == "circle")] <-
sum(dataQuestionnaire[[paste(paste("general", var, sep = "."), "circle.knob", sep = ".")]] == 2)
df$nbOfThird[which(df$device == "knob" & df$display == "line")] <-
sum(dataQuestionnaire[[paste(paste("general", var, sep = "."), "line.knob", sep = ".")]] == 3)
df$nbOfThird[which(df$device == "knob" & df$display == "circle")] <-
sum(dataQuestionnaire[[paste(paste("general", var, sep = "."), "circle.knob", sep = ".")]] == 3)
df$nbOfFourth[which(df$device == "knob" & df$display == "line")] <-
sum(dataQuestionnaire[[paste(paste("general", var, sep = "."), "line.knob", sep = ".")]] == 4)
df$nbOfFourth[which(df$device == "knob" & df$display == "circle")] <-
sum(dataQuestionnaire[[paste(paste("general", var, sep = "."), "circle.knob", sep = ".")]] == 4)
df$nbOfFifth[which(df$device == "knob" & df$display == "line")] <-
sum(dataQuestionnaire[[paste(paste("general", var, sep = "."), "line.knob", sep = ".")]] == 5)
df$nbOfFifth[which(df$device == "knob" & df$display == "circle")] <-
sum(dataQuestionnaire[[paste(paste("general", var, sep = "."), "circle.knob", sep = ".")]] == 5)
df$nbOfLast[which(df$device == "knob" & df$display == "line")] <-
sum(dataQuestionnaire[[paste(paste("general", var, sep = "."), "line.knob", sep = ".")]] == 6)
df$nbOfLast[which(df$device == "knob" & df$display == "circle")] <-
sum(dataQuestionnaire[[paste(paste("general", var, sep = "."), "circle.knob", sep = ".")]] == 6)
df$nbOfFirst[which(df$device == "wheel" & df$display == "line")] <-
sum(dataQuestionnaire[[paste(paste("general", var, sep = "."), "line.wheel", sep = ".")]] == 1)
df$nbOfFirst[which(df$device == "wheel" & df$display == "circle")] <-
sum(dataQuestionnaire[[paste(paste("general", var, sep = "."), "circle.wheel", sep = ".")]] == 1)
df$nbOfSecond[which(df$device == "wheel" & df$display == "line")] <-
sum(dataQuestionnaire[[paste(paste("general", var, sep = "."), "line.wheel", sep = ".")]] == 2)
df$nbOfSecond[which(df$device == "wheel" & df$display == "circle")] <-
sum(dataQuestionnaire[[paste(paste("general", var, sep = "."), "circle.wheel", sep = ".")]] == 2)
df$nbOfThird[which(df$device == "wheel" & df$display == "line")] <-
sum(dataQuestionnaire[[paste(paste("general", var, sep = "."), "line.wheel", sep = ".")]] == 3)
df$nbOfThird[which(df$device == "wheel" & df$display == "circle")] <-
sum(dataQuestionnaire[[paste(paste("general", var, sep = "."), "circle.wheel", sep = ".")]] == 3)
df$nbOfFourth[which(df$device == "wheel" & df$display == "line")] <-
sum(dataQuestionnaire[[paste(paste("general", var, sep = "."), "line.wheel", sep = ".")]] == 4)
df$nbOfFourth[which(df$device == "wheel" & df$display == "circle")] <-
sum(dataQuestionnaire[[paste(paste("general", var, sep = "."), "circle.wheel", sep = ".")]] == 4)
df$nbOfFifth[which(df$device == "wheel" & df$display == "line")] <-
sum(dataQuestionnaire[[paste(paste("general", var, sep = "."), "line.wheel", sep = ".")]] == 5)
df$nbOfFifth[which(df$device == "wheel" & df$display == "circle")] <-
sum(dataQuestionnaire[[paste(paste("general", var, sep = "."), "circle.wheel", sep = ".")]] == 5)
df$nbOfLast[which(df$device == "wheel" & df$display == "line")] <-
sum(dataQuestionnaire[[paste(paste("general", var, sep = "."), "line.wheel", sep = ".")]] == 6)
df$nbOfLast[which(df$device == "wheel" & df$display == "circle")] <-
sum(dataQuestionnaire[[paste(paste("general", var, sep = "."), "circle.wheel", sep = ".")]] == 6)
df$nbOfFirst[which(df$device == "slider" & df$display == "line")] <-
sum(dataQuestionnaire[[paste(paste("general", var, sep = "."), "line.slider", sep = ".")]] == 1)
df$nbOfFirst[which(df$device == "slider" & df$display == "circle")] <-
sum(dataQuestionnaire[[paste(paste("general", var, sep = "."), "circle.slider", sep = ".")]] == 1)
df$nbOfSecond[which(df$device == "slider" & df$display == "line")] <-
sum(dataQuestionnaire[[paste(paste("general", var, sep = "."), "line.slider", sep = ".")]] == 2)
df$nbOfSecond[which(df$device == "slider" & df$display == "circle")] <-
sum(dataQuestionnaire[[paste(paste("general", var, sep = "."), "circle.slider", sep = ".")]] == 2)
df$nbOfThird[which(df$device == "slider" & df$display == "line")] <-
sum(dataQuestionnaire[[paste(paste("general", var, sep = "."), "line.slider", sep = ".")]] == 3)
df$nbOfThird[which(df$device == "slider" & df$display == "circle")] <-
sum(dataQuestionnaire[[paste(paste("general", var, sep = "."), "circle.slider", sep = ".")]] == 3)
df$nbOfFourth[which(df$device == "slider" & df$display == "line")] <-
sum(dataQuestionnaire[[paste(paste("general", var, sep = "."), "line.slider", sep = ".")]] == 4)
df$nbOfFourth[which(df$device == "slider" & df$display == "circle")] <-
sum(dataQuestionnaire[[paste(paste("general", var, sep = "."), "circle.slider", sep = ".")]] == 4)
df$nbOfFifth[which(df$device == "slider" & df$display == "line")] <-
sum(dataQuestionnaire[[paste(paste("general", var, sep = "."), "line.slider", sep = ".")]] == 5)
df$nbOfFifth[which(df$device == "slider" & df$display == "circle")] <-
sum(dataQuestionnaire[[paste(paste("general", var, sep = "."), "circle.slider", sep = ".")]] == 5)
df$nbOfLast[which(df$device == "slider" & df$display == "line")] <-
sum(dataQuestionnaire[[paste(paste("general", var, sep = "."), "line.slider", sep = ".")]] == 6)
df$nbOfLast[which(df$device == "slider" & df$display == "circle")] <-
sum(dataQuestionnaire[[paste(paste("general", var, sep = "."), "circle.slider", sep = ".")]] == 6)
df$nbForEachRank <- df[3:8]
return(df)
}Displaying data
displayGeneralRankings <- function(df, title) {
# matrix of 3 X 2 graphs
par(mfrow=c(3,2))
# graph wheel line
subdata <- df[which(df$device == "wheel" &
df$display == "line"),]
row.names(subdata$nbForEachRank) <- paste(subdata$device, subdata$display, sep = " + ")
barplot(t(as.matrix(subdata$nbForEachRank)),
ylim = range(0,9),
col = heat.colors(6))
# graph wheel circle
subdata <- df[which(df$device == "wheel" &
df$display == "circle"),]
row.names(subdata$nbForEachRank) <- paste(subdata$device, subdata$display, sep = " + ")
barplot(t(as.matrix(subdata$nbForEachRank)),
ylim = range(0,9),
col = heat.colors(6))
# graph knob line
subdata <- df[which(df$device == "knob" &
df$display == "line"),]
row.names(subdata$nbForEachRank) <- paste(subdata$device, subdata$display, sep = " + ")
barplot(t(as.matrix(subdata$nbForEachRank)),
ylim = range(0,9),
col = heat.colors(6))
# graph knob circle
subdata <- df[which(df$device == "knob" &
df$display == "circle"),]
row.names(subdata$nbForEachRank) <- paste(subdata$device, subdata$display, sep = " + ")
barplot(t(as.matrix(subdata$nbForEachRank)),
ylim = range(0,9),
col = heat.colors(6))
# graph slider line
subdata <- df[which(df$device == "slider" &
df$display == "line"),]
row.names(subdata$nbForEachRank) <- paste(subdata$device, subdata$display, sep = " + ")
barplot(t(as.matrix(subdata$nbForEachRank)),
ylim = range(0,9),
col = heat.colors(6))
# graph slider circle
subdata <- df[which(df$device == "slider" &
df$display == "circle"),]
row.names(subdata$nbForEachRank) <- paste(subdata$device, subdata$display, sep = " + ")
barplot(t(as.matrix(subdata$nbForEachRank)),
ylim = range(0,9),
col = heat.colors(6))
# General legend and title
legend("topright",fill = heat.colors(6), legend = c("Most", "Second", "Third", "Fourth", "Fifth", "Least"))
title(paste("General " , paste(title, " by device and display"), sep=""), outer=TRUE)
}3.6.2 General preference per device and display
dataGeneralPreference <- buildGeneralRankingsDataFrame("preference")
displayGeneralRankings(dataGeneralPreference, "preference")
3.6.3 General perceived rapidity per device and display
dataGeneralRapidity <- buildGeneralRankingsDataFrame("rapidity")
displayGeneralRankings(dataGeneralRapidity, "rapidity")
3.6.4 General perceived accuracy per device and display
dataGeneralAccuracy <- buildGeneralRankingsDataFrame("accuracy")
displayGeneralRankings(dataGeneralAccuracy, "accuracy")
4 Hypohteses testing for MT
4.1 Loading libraries
library(ARTool)
library(sciplot)
library(ggplot2)##
## Attaching package: 'ggplot2'## The following object is masked from 'package:latticeExtra':
##
## layerlibrary(reshape)
library(estimability)
library(car)
library(lsmeans)
library(phia)
library(mvtnorm)
library(survival)##
## Attaching package: 'survival'## The following object is masked from 'package:boot':
##
## amllibrary(TH.data)## Loading required package: MASS##
## Attaching package: 'TH.data'## The following object is masked from 'package:MASS':
##
## geyserlibrary(MASS)
library(multcomp)
library(dplyr) #data_frame, %>%, filter, summarise, group_by##
## Attaching package: 'dplyr'## The following object is masked from 'package:MASS':
##
## select## The following object is masked from 'package:reshape':
##
## rename## The following object is masked from 'package:car':
##
## recode## The following objects are masked from 'package:xts':
##
## first, last## The following objects are masked from 'package:stats':
##
## filter, lag## The following objects are masked from 'package:base':
##
## intersect, setdiff, setequal, unionlibrary(tidyr) #spread##
## Attaching package: 'tidyr'## The following objects are masked from 'package:reshape':
##
## expand, smiths4.2 Normality test
Preview of data distribution
hist(dataFrameTimeAggregate$time, breaks=seq(0,max(dataFrameTimeAggregate$time)+1,0.5))
qqnorm(dataFrameTimeAggregate$time)
qqline(dataFrameTimeAggregate$time)
We test the normality of the data
shapiro.test(dataFrameTimeAggregate$time)##
## Shapiro-Wilk normality test
##
## data: dataFrameTimeAggregate$time
## W = 0.90288, p-value = 3.495e-12We can’t consider the data to follow a normal distribution, so ANOVA is not possible.
4.3 Anova with ART
Without normality, we cannot directly perform ANOVA on the data. Instead, we try to perform an Aligned Rank Transformation of data before performing ANOVA: http://depts.washington.edu/madlab/proj/art/index.html
m <- art(time ~ display * device * as.factor(id) + (1|participant), data=dataFrameTimeAggregate)Then check that ART is appropriate
summary(m)## Aligned Rank Transform of Factorial Model
##
## Call:
## art(formula = time ~ display * device * as.factor(id) + (1 |
## participant), data = dataFrameTimeAggregate)
##
## Column sums of aligned responses (should all be ~0):
## display device
## 0 0
## as.factor(id) display:device
## 0 0
## display:as.factor(id) device:as.factor(id)
## 0 0
## display:device:as.factor(id)
## 0
##
## F values of ANOVAs on aligned responses not of interest (should all be ~0):
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 0 0 0 0 0 0The last values are ~0 so ART is an appropriate method. We double-check with a more detailed way of checking that ART is appropriate
anova(m, response="aligned")## Analysis of Variance of Aligned Rank Transformed Data
##
## Table Type: Analysis of Deviance Table (Type III Wald F tests with Kenward-Roger df)
## Model: Mixed Effects (lmer)
## Response: aligned(time)
##
## Aligned By F Df Df.res Pr(>F)
## 1 device display 3.3172e-31 2 232 1
## 2 as.factor(id) display 3.3750e-31 4 232 1
## 3 display:device display 1.1417e-30 2 232 1
## 4 display:as.factor(id) display 8.6787e-32 4 232 1
## 5 device:as.factor(id) display 9.1030e-31 8 232 1
## 6 display:device:as.factor(id) display 4.1850e-31 8 232 1
## 7 display device 7.7144e-33 1 232 1
## 8 as.factor(id) device 4.9372e-31 4 232 1
## 9 display:device device 1.3037e-30 2 232 1
## 10 display:as.factor(id) device 1.2343e-31 4 232 1
## 11 device:as.factor(id) device 1.5390e-30 8 232 1
## 12 display:device:as.factor(id) device 3.5100e-31 8 232 1
## 13 display as.fctr(d) 2.7772e-31 1 232 1
## 14 device as.fctr(d) 2.7772e-31 2 232 1
## 15 display:device as.fctr(d) 1.1417e-30 2 232 1
## 16 display:as.factor(id) as.fctr(d) 2.7772e-31 4 232 1
## 17 device:as.factor(id) as.fctr(d) 9.9901e-31 8 232 1
## 18 display:device:as.factor(id) as.fctr(d) 4.3200e-31 8 232 1
## 19 display disply:dvc 1.9286e-33 1 232 1
## 20 device disply:dvc 3.4078e-30 2 232 1
## 21 as.factor(id) disply:dvc 4.3393e-31 4 232 1
## 22 display:as.factor(id) disply:dvc 1.3693e-31 4 232 1
## 23 device:as.factor(id) disply:dvc 8.4183e-31 8 232 1
## 24 display:device:as.factor(id) disply:dvc 3.5968e-31 8 232 1
## 25 display dsply:s.() 5.6238e-30 1 232 1
## 26 device dsply:s.() 3.3172e-31 2 232 1
## 27 as.factor(id) dsply:s.() 2.6885e-30 4 232 1
## 28 display:device dsply:s.() 1.1417e-30 2 232 1
## 29 device:as.factor(id) dsply:s.() 9.1030e-31 8 232 1
## 30 display:device:as.factor(id) dsply:s.() 4.1850e-31 8 232 1
## 31 display dvc:s.fc() 1.9286e-33 1 232 1
## 32 device dvc:s.fc() 5.2651e-31 2 232 1
## 33 as.factor(id) dvc:s.fc() 1.9257e-30 4 232 1
## 34 display:device dvc:s.fc() 1.1417e-30 2 232 1
## 35 display:as.factor(id) dvc:s.fc() 1.0318e-31 4 232 1
## 36 display:device:as.factor(id) dvc:s.fc() 4.1658e-31 8 232 1
## 37 display dsply::.() 5.5736e-31 1 232 1
## 38 device dsply::.() 1.9980e-30 2 232 1
## 39 as.factor(id) dsply::.() 4.6768e-31 4 232 1
## 40 display:device dsply::.() 1.8649e-30 2 232 1
## 41 display:as.factor(id) dsply::.() 1.4397e-30 4 232 1
## 42 device:as.factor(id) dsply::.() 1.6219e-30 8 232 1
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1The last column is ~1 so ART is confirmed to be appropriate.
We now perform Anova on the resulting ART table.
anova(m)## Analysis of Variance of Aligned Rank Transformed Data
##
## Table Type: Analysis of Deviance Table (Type III Wald F tests with Kenward-Roger df)
## Model: Mixed Effects (lmer)
## Response: art(time)
##
## F Df Df.res Pr(>F)
## 1 display 9.00819 1 232 0.0029815 **
## 2 device 30.42125 2 232 1.8495e-12 ***
## 3 as.factor(id) 30.37782 4 232 < 2.22e-16 ***
## 4 display:device 13.23202 2 232 3.6157e-06 ***
## 5 display:as.factor(id) 1.10402 4 232 0.3553812
## 6 device:as.factor(id) 0.50473 8 232 0.8521647
## 7 display:device:as.factor(id) 0.21532 8 232 0.9880101
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1We find a significant effect of device, display, id, and of the interaction between device and display, on the Movement Time.
4.4 Post-hoc analysis
Interactions between device and display
testInteractions(artlm(m, "display:device"), pairwise=c("display","device"))## Chisq Test:
## P-value adjustment method: holm
## Value Df Chisq Pr(>Chisq)
## circle-line : knob-slider -98.822 1 24.6216 2.093e-06 ***
## circle-line : knob-wheel -72.822 1 13.3701 0.0005113 ***
## circle-line : slider-wheel 26.000 1 1.7043 0.1917225
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1Another option (because the difference of difference is not really what we are looking for): Post-hoc cross-factor pairwise comparisons using Wilcoxon signed-rank tests (non-parametric tests to be used when subjects were in both of the compared conditions in each case)
knob_vs_slider_in_circle = wilcox.test(dataFrameTimeAggregate[dataFrameTimeAggregate$device == "knob" & dataFrameTimeAggregate$display == "circle",]$time,
dataFrameTimeAggregate[dataFrameTimeAggregate$device == "slider" & dataFrameTimeAggregate$display == "circle",]$time,
paired=TRUE)
knob_vs_slider_in_line = wilcox.test(dataFrameTimeAggregate[dataFrameTimeAggregate$device == "knob" & dataFrameTimeAggregate$display == "line",]$time,
dataFrameTimeAggregate[dataFrameTimeAggregate$device == "slider" & dataFrameTimeAggregate$display == "line",]$time,
paired=TRUE)
knob_vs_wheel_in_circle = wilcox.test(dataFrameTimeAggregate[dataFrameTimeAggregate$device == "knob" & dataFrameTimeAggregate$display == "circle",]$time,
dataFrameTimeAggregate[dataFrameTimeAggregate$device == "wheel" & dataFrameTimeAggregate$display == "circle",]$time,
paired=TRUE)
knob_vs_wheel_in_line = wilcox.test(dataFrameTimeAggregate[dataFrameTimeAggregate$device == "knob" & dataFrameTimeAggregate$display == "line",]$time,
dataFrameTimeAggregate[dataFrameTimeAggregate$device == "wheel" & dataFrameTimeAggregate$display == "line",]$time,
paired=TRUE)
slider_vs_wheel_in_circle = wilcox.test(dataFrameTimeAggregate[dataFrameTimeAggregate$device == "slider" & dataFrameTimeAggregate$display == "circle",]$time,
dataFrameTimeAggregate[dataFrameTimeAggregate$device == "wheel" & dataFrameTimeAggregate$display == "circle",]$time,
paired=TRUE)
slider_vs_wheel_in_line = wilcox.test(dataFrameTimeAggregate[dataFrameTimeAggregate$device == "slider" & dataFrameTimeAggregate$display == "line",]$time,
dataFrameTimeAggregate[dataFrameTimeAggregate$device == "wheel" & dataFrameTimeAggregate$display == "line",]$time,
paired=TRUE)
circle_vs_line_in_knob = wilcox.test(dataFrameTimeAggregate[dataFrameTimeAggregate$display == "circle" & dataFrameTimeAggregate$device == "knob",]$time,
dataFrameTimeAggregate[dataFrameTimeAggregate$display == "line" & dataFrameTimeAggregate$device == "knob",]$time,
paired=TRUE)
circle_vs_line_in_slider = wilcox.test(dataFrameTimeAggregate[dataFrameTimeAggregate$display == "circle" & dataFrameTimeAggregate$device == "slider",]$time,
dataFrameTimeAggregate[dataFrameTimeAggregate$display == "line" & dataFrameTimeAggregate$device == "slider",]$time,
paired=TRUE)
circle_vs_line_in_wheel = wilcox.test(dataFrameTimeAggregate[dataFrameTimeAggregate$display == "circle" & dataFrameTimeAggregate$device == "wheel",]$time,
dataFrameTimeAggregate[dataFrameTimeAggregate$display == "line" & dataFrameTimeAggregate$device == "wheel",]$time,
paired=TRUE)
# correct for multiple comparisons using Holm's sequential Bonferroni procedure (Holm 1979)
p.adjust(c(knob_vs_slider_in_circle$p.value,
knob_vs_slider_in_line$p.value,
knob_vs_wheel_in_circle$p.value,
knob_vs_wheel_in_line$p.value,
slider_vs_wheel_in_circle$p.value,
slider_vs_wheel_in_line$p.value,
circle_vs_line_in_knob$p.value,
circle_vs_line_in_slider$p.value,
circle_vs_line_in_wheel$p.value), method="holm")## [1] 6.206678e-02 5.422194e-06 9.520836e-06 6.465870e-01 2.977515e-03
## [6] 2.278344e-07 2.463736e-02 5.848035e-05 6.947360e-035 Hypohteses testing for error rate
5.1 Normality test
Preview of data distribution
hist(dataFrameErrorAggregate$error, breaks=seq(0,max(dataFrameErrorAggregate$error)+1,0.5))
qqnorm(dataFrameErrorAggregate$error)
qqline(dataFrameErrorAggregate$error)
We test the normality of the data
shapiro.test(dataFrameErrorAggregate$error)##
## Shapiro-Wilk normality test
##
## data: dataFrameErrorAggregate$error
## W = 0.61001, p-value < 2.2e-16We can’t consider the data to follow a normal distribution, so ANOVA is not possible.
5.2 Anova with ART
Without normality, we cannot directly perform ANOVA on the data. Instead, we try to perform an Aligned Rank Transformation of data before performing ANOVA: http://depts.washington.edu/madlab/proj/art/index.html
m <- art(error ~ display * device * as.factor(id) + (1|participant), data=dataFrameErrorAggregate)Then check that ART is appropriate
summary(m)## Aligned Rank Transform of Factorial Model
##
## Call:
## art(formula = error ~ display * device * as.factor(id) + (1 |
## participant), data = dataFrameErrorAggregate)
##
## Column sums of aligned responses (should all be ~0):
## display device
## 0 0
## as.factor(id) display:device
## 0 0
## display:as.factor(id) device:as.factor(id)
## 0 0
## display:device:as.factor(id)
## 0
##
## F values of ANOVAs on aligned responses not of interest (should all be ~0):
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 0 0 0 0 0 0The last values are ~0 so ART is an appropriate method. We double-check with a more detailed way of checking that ART is appropriate
anova(m, response="aligned")## Analysis of Variance of Aligned Rank Transformed Data
##
## Table Type: Analysis of Deviance Table (Type III Wald F tests with Kenward-Roger df)
## Model: Mixed Effects (lmer)
## Response: aligned(error)
##
## Aligned By F Df Df.res Pr(>F)
## 1 device display 1.3749e-32 2 232 1
## 2 as.factor(id) display 1.9304e-31 4 232 1
## 3 display:device display 5.4997e-34 2 232 1
## 4 display:as.factor(id) display 4.6748e-32 4 232 1
## 5 device:as.factor(id) display 1.7049e-32 8 232 1
## 6 display:device:as.factor(id) display 3.9598e-32 8 232 1
## 7 display device 6.7372e-33 1 232 1
## 8 as.factor(id) device 3.4373e-33 4 232 1
## 9 display:device device 1.5262e-32 2 232 1
## 10 display:as.factor(id) device 2.7636e-32 4 232 1
## 11 device:as.factor(id) device 2.2136e-32 8 232 1
## 12 display:device:as.factor(id) device 9.8995e-33 8 232 1
## 13 display as.fctr(d) 6.6547e-32 1 232 1
## 14 device as.fctr(d) 2.1449e-32 2 232 1
## 15 display:device as.fctr(d) 2.1449e-32 2 232 1
## 16 display:as.factor(id) as.fctr(d) 2.8654e-31 4 232 1
## 17 device:as.factor(id) as.fctr(d) 2.1449e-32 8 232 1
## 18 display:device:as.factor(id) as.fctr(d) 2.9699e-32 8 232 1
## 19 display disply:dvc 2.0913e-31 1 232 1
## 20 device disply:dvc 9.9408e-32 2 232 1
## 21 as.factor(id) disply:dvc 1.2237e-32 4 232 1
## 22 display:as.factor(id) disply:dvc 1.4987e-32 4 232 1
## 23 device:as.factor(id) disply:dvc 1.6774e-32 8 232 1
## 24 display:device:as.factor(id) disply:dvc 1.1962e-32 8 232 1
## 25 display dsply:s.() 4.0093e-31 1 232 1
## 26 device dsply:s.() 3.3548e-32 2 232 1
## 27 as.factor(id) dsply:s.() 1.1054e-31 4 232 1
## 28 display:device dsply:s.() 2.0349e-32 2 232 1
## 29 device:as.factor(id) dsply:s.() 3.1898e-32 8 232 1
## 30 display:device:as.factor(id) dsply:s.() 2.4749e-32 8 232 1
## 31 display dvc:s.fc() 3.4373e-33 1 232 1
## 32 device dvc:s.fc() 1.1178e-31 2 232 1
## 33 as.factor(id) dvc:s.fc() 3.0386e-32 4 232 1
## 34 display:device dvc:s.fc() 2.7636e-32 2 232 1
## 35 display:as.factor(id) dvc:s.fc() 1.2374e-33 4 232 1
## 36 display:device:as.factor(id) dvc:s.fc() 5.3622e-33 8 232 1
## 37 display dsply::.() 3.1679e-31 1 232 1
## 38 device dsply::.() 2.8599e-32 2 232 1
## 39 as.factor(id) dsply::.() 3.5198e-32 4 232 1
## 40 display:device dsply::.() 1.2264e-31 2 232 1
## 41 display:as.factor(id) dsply::.() 7.9196e-32 4 232 1
## 42 device:as.factor(id) dsply::.() 2.2549e-32 8 232 1
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1The last column is ~1 so ART is confirmed to be appropriate.
We now perform Anova on the resulting ART table.
anova(m)## Analysis of Variance of Aligned Rank Transformed Data
##
## Table Type: Analysis of Deviance Table (Type III Wald F tests with Kenward-Roger df)
## Model: Mixed Effects (lmer)
## Response: art(error)
##
## F Df Df.res Pr(>F)
## 1 display 7.94094 1 232 0.00524985 **
## 2 device 0.94764 2 232 0.38914897
## 3 as.factor(id) 4.96840 4 232 0.00073452 ***
## 4 display:device 0.43277 2 232 0.64923521
## 5 display:as.factor(id) 2.25449 4 232 0.06403810 .
## 6 device:as.factor(id) 1.38178 8 232 0.20526082
## 7 display:device:as.factor(id) 1.60942 8 232 0.12291280
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1We find a significant effect of display and id on the error rate.
5.2.1 Post-hoc analysis
Same procedure as for movement time
knob_vs_slider_in_circle = wilcox.test(dataFrameErrorAggregate[dataFrameErrorAggregate$device == "knob" & dataFrameErrorAggregate$display == "circle",]$error,
dataFrameErrorAggregate[dataFrameErrorAggregate$device == "slider" & dataFrameErrorAggregate$display == "circle",]$error,
paired=TRUE)
knob_vs_slider_in_line = wilcox.test(dataFrameErrorAggregate[dataFrameErrorAggregate$device == "knob" & dataFrameErrorAggregate$display == "line",]$error,
dataFrameErrorAggregate[dataFrameErrorAggregate$device == "slider" & dataFrameErrorAggregate$display == "line",]$error,
paired=TRUE)
knob_vs_wheel_in_circle = wilcox.test(dataFrameErrorAggregate[dataFrameErrorAggregate$device == "knob" & dataFrameErrorAggregate$display == "circle",]$error,
dataFrameErrorAggregate[dataFrameErrorAggregate$device == "wheel" & dataFrameErrorAggregate$display == "circle",]$error,
paired=TRUE)
knob_vs_wheel_in_line = wilcox.test(dataFrameErrorAggregate[dataFrameErrorAggregate$device == "knob" & dataFrameErrorAggregate$display == "line",]$error,
dataFrameErrorAggregate[dataFrameErrorAggregate$device == "wheel" & dataFrameErrorAggregate$display == "line",]$error,
paired=TRUE)
slider_vs_wheel_in_circle = wilcox.test(dataFrameErrorAggregate[dataFrameErrorAggregate$device == "slider" & dataFrameErrorAggregate$display == "circle",]$error,
dataFrameErrorAggregate[dataFrameErrorAggregate$device == "wheel" & dataFrameErrorAggregate$display == "circle",]$error,
paired=TRUE)
slider_vs_wheel_in_line = wilcox.test(dataFrameErrorAggregate[dataFrameErrorAggregate$device == "slider" & dataFrameErrorAggregate$display == "line",]$error,
dataFrameErrorAggregate[dataFrameErrorAggregate$device == "wheel" & dataFrameErrorAggregate$display == "line",]$error,
paired=TRUE)
circle_vs_line_in_knob = wilcox.test(dataFrameErrorAggregate[dataFrameErrorAggregate$display == "circle" & dataFrameErrorAggregate$device == "knob",]$error,
dataFrameErrorAggregate[dataFrameErrorAggregate$display == "line" & dataFrameErrorAggregate$device == "knob",]$error,
paired=TRUE)
circle_vs_line_in_slider = wilcox.test(dataFrameErrorAggregate[dataFrameErrorAggregate$display == "circle" & dataFrameErrorAggregate$device == "slider",]$error,
dataFrameErrorAggregate[dataFrameErrorAggregate$display == "line" & dataFrameErrorAggregate$device == "slider",]$error,
paired=TRUE)
circle_vs_line_in_wheel = wilcox.test(dataFrameErrorAggregate[dataFrameErrorAggregate$display == "circle" & dataFrameErrorAggregate$device == "wheel",]$error,
dataFrameErrorAggregate[dataFrameErrorAggregate$display == "line" & dataFrameErrorAggregate$device == "wheel",]$error,
paired=TRUE)
# correct for multiple comparisons using Holm's sequential Bonferroni procedure (Holm 1979)
p.adjust(c(knob_vs_slider_in_circle$p.value,
knob_vs_slider_in_line$p.value,
knob_vs_wheel_in_circle$p.value,
knob_vs_wheel_in_line$p.value,
slider_vs_wheel_in_circle$p.value,
slider_vs_wheel_in_line$p.value,
circle_vs_line_in_knob$p.value,
circle_vs_line_in_slider$p.value,
circle_vs_line_in_wheel$p.value), method="holm")## [1] 1.0000000 1.0000000 1.0000000 1.0000000 1.0000000 1.0000000 1.0000000
## [8] 1.0000000 0.22102316 Hypohteses testing for SUS
6.1 Normality test
Preview of data distribution
hist(dataSUSscores$score, breaks=seq(0,max(dataSUSscores$score)+1,0.5))
qqnorm(dataSUSscores$score)
qqline(dataSUSscores$score)
We test the normality of the data
shapiro.test(dataSUSscores$score)##
## Shapiro-Wilk normality test
##
## data: dataSUSscores$score
## W = 0.83678, p-value = 3.454e-06We can’t consider the data to follow a normal distribution, so ANOVA is not possible.
6.2 Anova with ART
Without normality, we cannot directly perform ANOVA on the data. Instead, we try to perform an Aligned Rank Transformation of data before performing ANOVA: http://depts.washington.edu/madlab/proj/art/index.html
m <- art(score ~ display * device + (1|participant), data=dataSUSscores)Then check that ART is appropriate
summary(m)## Aligned Rank Transform of Factorial Model
##
## Call:
## art(formula = score ~ display * device + (1 | participant), data = dataSUSscores)
##
## Column sums of aligned responses (should all be ~0):
## display device display:device
## 0 0 0
##
## F values of ANOVAs on aligned responses not of interest (should all be ~0):
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 0 0 0 0 0 0The last values are ~0 so ART is an appropriate method. We double-check with a more detailed way of checking that ART is appropriate
anova(m, response="aligned")## Analysis of Variance of Aligned Rank Transformed Data
##
## Table Type: Analysis of Deviance Table (Type III Wald F tests with Kenward-Roger df)
## Model: Mixed Effects (lmer)
## Response: aligned(score)
##
## Aligned By F Df Df.res Pr(>F)
## 1 device display 1.0093e-30 2 40 1
## 2 display:device display 9.1317e-31 2 40 1
## 3 display device 4.8061e-32 1 40 1
## 4 display:device device 9.1317e-31 2 40 1
## 5 display disply:dvc 1.3890e-29 1 40 1
## 6 device disply:dvc 7.4976e-30 2 40 1
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1The last column is ~1 so ART is confirmed to be appropriate.
We now perform Anova on the resulting ART table.
anova(m)## Analysis of Variance of Aligned Rank Transformed Data
##
## Table Type: Analysis of Deviance Table (Type III Wald F tests with Kenward-Roger df)
## Model: Mixed Effects (lmer)
## Response: art(score)
##
## F Df Df.res Pr(>F)
## 1 display 3.3751 1 40 0.073628 .
## 2 device 1.3646 2 40 0.267123
## 3 display:device 5.1449 2 40 0.010271 *
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1We find a significant effect of the interaction between display and device on the SUS score.
6.3 Post-hoc analysis
Same procedure as for movement time
knob_vs_slider_in_circle = wilcox.test(dataSUSscores[dataSUSscores$device == "knob" & dataSUSscores$display == "circle",]$score,
dataSUSscores[dataSUSscores$device == "slider" & dataSUSscores$display == "circle",]$score,
paired=TRUE)
knob_vs_slider_in_line = wilcox.test(dataSUSscores[dataSUSscores$device == "knob" & dataSUSscores$display == "line",]$score,
dataSUSscores[dataSUSscores$device == "slider" & dataSUSscores$display == "line",]$score,
paired=TRUE)
knob_vs_wheel_in_circle = wilcox.test(dataSUSscores[dataSUSscores$device == "knob" & dataSUSscores$display == "circle",]$score,
dataSUSscores[dataSUSscores$device == "wheel" & dataSUSscores$display == "circle",]$score,
paired=TRUE)
knob_vs_wheel_in_line = wilcox.test(dataSUSscores[dataSUSscores$device == "knob" & dataSUSscores$display == "line",]$score,
dataSUSscores[dataSUSscores$device == "wheel" & dataSUSscores$display == "line",]$score,
paired=TRUE)
slider_vs_wheel_in_circle = wilcox.test(dataSUSscores[dataSUSscores$device == "slider" & dataSUSscores$display == "circle",]$score,
dataSUSscores[dataSUSscores$device == "wheel" & dataSUSscores$display == "circle",]$score,
paired=TRUE)
slider_vs_wheel_in_line = wilcox.test(dataSUSscores[dataSUSscores$device == "slider" & dataSUSscores$display == "line",]$score,
dataSUSscores[dataSUSscores$device == "wheel" & dataSUSscores$display == "line",]$score,
paired=TRUE)
circle_vs_line_in_knob = wilcox.test(dataSUSscores[dataSUSscores$display == "circle" & dataSUSscores$device == "knob",]$score,
dataSUSscores[dataSUSscores$display == "line" & dataSUSscores$device == "knob",]$score,
paired=TRUE)
circle_vs_line_in_slider = wilcox.test(dataSUSscores[dataSUSscores$display == "circle" & dataSUSscores$device == "slider",]$score,
dataSUSscores[dataSUSscores$display == "line" & dataSUSscores$device == "slider",]$score,
paired=TRUE)
circle_vs_line_in_wheel = wilcox.test(dataSUSscores[dataSUSscores$display == "circle" & dataSUSscores$device == "wheel",]$score,
dataSUSscores[dataSUSscores$display == "line" & dataSUSscores$device == "wheel",]$score,
paired=TRUE)
# correct for multiple comparisons using Holm's sequential Bonferroni procedure (Holm 1979)
p.adjust(c(knob_vs_slider_in_circle$p.value,
knob_vs_slider_in_line$p.value,
knob_vs_wheel_in_circle$p.value,
knob_vs_wheel_in_line$p.value,
slider_vs_wheel_in_circle$p.value,
slider_vs_wheel_in_line$p.value,
circle_vs_line_in_knob$p.value,
circle_vs_line_in_slider$p.value,
circle_vs_line_in_wheel$p.value), method="holm")## [1] 0.5463177 0.2205788 0.3665579 0.6108130 0.5463177 0.5463177 0.2211673
## [8] 0.2829554 0.20028717 Investigating the learning of each combination
d = dataFrame[which(dataFrame$device == 'slider' & dataFrame$display == 'circle'),]
plot(d$time)
d = dataFrame[which(dataFrame$device == 'knob' & dataFrame$display == 'line'),]
plot(d$time)
d = dataFrame[which(dataFrame$device == 'slider' & dataFrame$display == 'line'),]
plot(d$time)
d = dataFrame[which(dataFrame$device == 'knob' & dataFrame$display == 'circle'),]
plot(d$time)
d = dataFrame[which(dataFrame$device == 'wheel' & dataFrame$display == 'line'),]
plot(d$time)
d = dataFrame[which(dataFrame$device == 'wheel' & dataFrame$display == 'circle'),]
plot(d$time)
3.5 Comments for each devices
After reading the comments, we can see that some participants refered to the previous comments that they made: in p2/slider (-> wheel was just before), p7/slider (-> wheel was just before too). We therefore add the previous comment by hand in the data frame in order for the comment to be understandable.