Longitudinal experiments
VERA provides support for running longitudinal experiments, or otherwise an experiment where a single participant may take the experiment multiple times / have multiple experiment sessions. Below is a guide on how to set up and run a longitudinal experiment in VERA.
What is a VERA longitudinal experiment?
In VERA, by default, a “normal” experiment represents an experiment where every participant may only have a single experiment session - they complete the experiment once, and may not take it again (their entry is blocked after they have completed the experiment).
A longitudinal experiment is an experiment where a single participant may have multiple sessions, and may take the experiment multiple times. The most common use case for a longitudinal experiment is a study where a participant repeatedly takes the experiment multiple times over a period of time; but, a VERA longitudinal experiment can be used for any experiment where it is important for a single participant to have multiple experimental sessions or take the experiment multiple times.
A VERA longitudinal experiment allows the following behaviors: - A single participant ID may be associated with multiple experiment sessions - Data from multiple sessions may be associated with a single participant ID - Participants may take the experiment multiple times and are not blocked from entry upon completion of the experiment
Warning
When running a longitudinal experiment, you will be responsible for scheduling the participant sessions and ensuring participants know when to start their next session. VERA does not yet provide any calendar, scheduling, or reminder system for longitudinal experiments. Unless you set a session spacing rule (see Limiting the time between sessions), participants are able to start their next session at any time, meaning you must ensure participants know when they are supposed to start their next session.
You can use your own email list, a spreadsheet, or some other method to ensure participants know when they are supposed to start their next session.
Setting up a longitudinal experiment
On the experiment setup page for your experiment, go to the “Experiment Flow” category. At the top of the graphical experiment flow editor, you will see a box for your experiment type. If you have not set an experiment type yet, it will say “Experiment type not set”. Click on this box to select your experiment type. The options listed will be as follows:
Standard experiment: Each participant completes the experiment in a single session. After their session ends, they cannot start another (i.e. one session per participant).
Longitudinal experiment: A single participant may have multiple sessions, and may take the experiment multiple times. You will be responsible for scheduling the sessions and ensuring participants know when to start their next session.
For a longitudinal experiment, you will need to select the “Longitudinal experiment” option. Click “Save Changes” to save your experiment setup.
Limiting the time between sessions
By default a participant can start their next session the moment the previous one ends. If your protocol requires a washout period, a fixed interval between visits, or a deadline for each follow-up, set a session spacing rule on the first Activate step alongside distribution settings (Activate -> Session spacing). Session spacing only appears for longitudinal experiments and offers three independent controls:
Maximum sessions per participant: after a participant finishes this many sessions, they cannot start another. Their study card tells them they have completed the final session. Leave empty for no limit.
Minimum wait between sessions: a participant cannot start their next session until this much time has passed since their previous session ended. Before then, their study card shows the date their next session opens and the start button is unavailable.
Maximum window to start the next session: once this much time has passed since their previous session ended, the next session closes. The participant is told to contact you rather than being able to start on their own.
Each bound is entered as a whole number of hours or days, up to one year. Leave a field empty (or enter 0) for no limit on that side. Both bounds are measured from the end of the participant’s previous session; a session that was never closed falls back to its start time.
Note
Session spacing restricts the ways a participant starts their own next session: the VERA participant portal, and remote / Prolific links. Those are the routes where a participant decides when to come back, so they are the ones the rule needs to hold.
When you start a session yourself from Participant Management, VERA asks you to confirm before starting one that is not due yet, and then starts it — you can always bring a participant in off-schedule when your protocol allows it.
Changing the rule applies immediately to every enrolled participant and is measured against their last completed session, so tightening a minimum wait can close a window that was previously open.
Warning
Sessions started from Unity with VERASessionManager.ManualStartParticipantSessionFromId() are not spaced, so a direct desktop or
Android build can begin a visit outside the window. Unity treats a rejected request as a temporary network problem and keeps recording
locally so a connection drop cannot ruin a session in progress, which means it would not stop for a spacing rule either.
If your protocol depends on the spacing being enforced, distribute the study through the participant portal or a remote link rather than a direct build, or keep using the scheduling process you would have used anyway.
Support for direct builds is planned and needs a coordinated change on the Unity side; the rule you configure here will not need to change when it arrives.
Running a longitudinal experiment
Once you have set up your longitudinal experiment, you can start running it. This will look different depending on how your experiment is configured:
Utilizing the VERA web interface / WebXR browser
If your participants are using the VERA web interface / WebXR browser (i.e. participants are accessing the experiment via a VERA web link rather than a direct desktop build, Android APK build, Unity editor, etc.), participants who complete the experiment will be able to start a new session at any time, unless you have set a session spacing rule (see Limiting the time between sessions).
Once the experiment is completed by a participant, they will see a completion screen which tells them that their session has been successfully completed, and to come back later when they are scheduled to start their next session. They can then click a button to start their next session, which will start the experiment from the beginning and record data for a new session. If a session spacing rule is set, the completion screen shows the date their next session opens (or that its window has closed) in place of that button.
That’s it! After you make sure participants know when they are supposed to start their next session, you can let the experiment run and participants will be able to start their next session at any time.
Utilizing a direct desktop build, Android APK build, Unity editor, etc.
If your participants are using a direct desktop build, Android APK build, Unity editor, etc. (i.e. participants are accessing the experiment directly and not via the VERA web interface / WebXR browser), you will need to manually link your participant sessions to a participant ID from Unity.
To do so, implement the following steps: 1. In the Unity VERA Settings window (menu bar -> VERA -> Settings), under the “Data Recording” section, set the “Session Start Behavior” to “Manual”. This will ensure participant sessions will not start automatically and create a new participant entry before we can link it to the correct longitudinal ID. 2. In a C# script, use VERASessionManager.ManualStartParticipantSessionFromId(int participantId) to start a new participant session. Provide the exact longitudinal ID for the session you want to start to ensure the new session is linked to the correct ID. If the ID does not already exist, it will create a new participant under that specific ID, and then start a new session for that participant; if the ID does already exist, it will simply start a new session under that specific ID.
The only step left is to ensure the proper ID is being passed to the ManualStartParticipantSessionFromId method. You may do so in whatever way you see fit. For example, you could provide each participant a unique ID via email or in-person, then prompt them for their ID when they start the experiment to ensure the proper ID is being passed to the ManualStartParticipantSessionFromId method.
Once participants start running the experiment, you will see their data show up on the VERA web portal as usual. Each participant can have multiple sessions, each session marked with its own unique ID (for example, “S1”, “S2”, “S3”, etc.).
Instead of approving/rejecting each participant entry, you will instead need to approve/reject each session entry. So, if a single participant has 3 sessions, you will need to approve/reject 3 session entries.
If you ever need to access a participant’s longitudinal session number from Unity, you can do so with the VERASessionManager.sessionNumber. This will return current session (such as 1, 2, 3, etc.) for the current participant.
Questionnaires across sessions
Pre-VR and post-VR questionnaires run on every session unless you choose otherwise. On the experiment’s Surveys & Questionnaires page, each pre-VR and post-VR card has a Show on control. The default is all sessions. You can instead limit a questionnaire to:
the first session only
the last session only (enter how many sessions each participant will complete; the questionnaire runs on that session)
odd sessions only, or even sessions only
every N sessions, starting at a session you choose
specific session numbers, such as session 2 or sessions 1, 3, and 5
a range of sessions, such as sessions 2 through 4
Mid-VR questionnaires are not affected. They can still be triggered from Unity on any session. Standard (non-longitudinal) experiments ignore these settings and always show every pre- and post-VR questionnaire.
Memory between sessions
It may be useful to remember some participant information between sessions. For example, you may want to remember a participant’s group assignment, demographic information, or survey responses between sessions.
VERA provides a lightweight key-value storage system for memory between sessions. During a session, you can store and retrieve string-based key-value pairs for a participant using the following VERASessionManager functions:
VERASessionManager.SetMemoryKey(string keyName, string keyValue): Store a string-based key-value pair for a participant. keyName represents the name of the key to store, and keyValue represents the value to store associated with that key. Optionally, you can utilize the onSuccess and onFailure parameters to handle the success or failure of the operation.
VERASessionManager.GetMemoryKey(string keyName, Action<string> onSuccess): Retrieve a string-based key-value pair for a participant. keyName represents the name of the key to retrieve the value for. The result of the operation will be passed to the onSuccess callback, providing the value of the key if it exists, or null if it does not exist. Optionally, you can use the onFailure callback to handle possible failure of the operation.
Note that the memory between sessions is saved for the participant as a whole - it will remain stored between sessions, and changing the value of a key will change it across all sessions, not just the current session. Also note that there is a 2MB limit on the total size of a single memory value, and a 128 character limit on each key name.
Memory is mostly designed to be used to programmatically store and retrieve information between sessions, but if you would like to preview or download the memory for a participant, VERA also provides this memory directly as a file type on the participant’s results page. For each participant session, you will see a “MemoryFields.json” file that contains the participant’s memory as a JSON object (note again that this is the total memory for the participant as a whole, not just for the session you are viewing or downloading).
For values shared by every participant in the experiment, such as a running condition count, see Experiment memory.