Getting started

On this page you will build a small but complete eye-tracking experiment, one piece at a time. The participant will:

1. turn on their webcam and line up their face,
2. watch the screen switch between black and white for fifteen seconds,
3. look at a series of dots, first to calibrate the tracker and then to check it,
4. see two pictures side by side, look at the one they prefer, and press a key.

At the end you will have, for that last trial, a list of where on the screen the participant was looking, one estimate per camera frame (typically up to 30 a second), and which picture each look landed on.

What you need: Chrome or Edge on a laptop or desktop, a webcam, and a way to open the page over https:// or from localhost (browsers only allow camera access on secure pages). Other browsers work too, usually more slowly; see Browser compatibility.

If you would like to see the result before building it, try the live demo. It runs the same setup steps in your browser and then shows you where you looked on a painting.

How the pieces fit together

saccade.js adds two kinds of things to a jsPsych experiment:

• Setup trials. A handful of plugins, each doing one job: start the camera, measure timing, calibrate, check accuracy. You put them near the start of your timeline, like consent or instructions.
• Recording. The saccade.js extension records gaze during any trial you attach it to. Your task trials can use any jsPsych plugin you like; adding one line to a trial turns recording on for that trial.

Build the experiment

1. Load the packages

   Each part of saccade.js is its own small package. Load them in your page's <head>, alongside jsPsych:

   <!doctype html>
   <html>
     <head>
       <script src="https://unpkg.com/jspsych@8"></script>
       <script src="https://unpkg.com/@jspsych/plugin-html-keyboard-response@2"></script>
       <script src="https://unpkg.com/@saccadejs/core@0.1.0"></script>
       <script src="https://unpkg.com/@saccadejs/extension@0.1.0"></script>
       <script src="https://unpkg.com/@saccadejs/plugin-preview@0.1.0"></script>
       <script src="https://unpkg.com/@saccadejs/plugin-time-sync@0.1.0"></script>
       <script src="https://unpkg.com/@saccadejs/plugin-calibrate@0.1.0"></script>
       <script src="https://unpkg.com/@saccadejs/plugin-validate@0.1.0"></script>
       <link rel="stylesheet" href="https://unpkg.com/jspsych@8/css/jspsych.css" />
     </head>
     <body></body>
   </html>

   Two things to notice:

   • @saccadejs/core comes first. The other packages depend on it.
   • Every URL names an exact version (@0.1.0). Keep it that way. If you wrote @saccadejs/core with no version, participants who arrive after a new release would get a different tracker from those who came before, and you would end up with two datasets mixed together.

2. Register the extension

   Tell jsPsych about the extension when you start it:

   const jsPsych = initJsPsych({
     extensions: [{ type: jsPsychExtensionSaccade }],
     on_finish: () => jsPsych.data.displayData(),
   });

   This only makes eye tracking available. It does not turn on the camera yet; that happens in the next step, when the participant can see what is being asked of them.

3. Start the camera

   const timeline = [{ type: jsPsychSaccadePreview }];

   This trial asks the participant for permission to use the camera, then downloads the eye-tracking model (about 20 MB) while showing a progress bar. Once it is running, the participant sees:

   • their own camera image, mirrored like a video call,
   • a close-up strip of their eyes, which is the only part of the image the tracker uses,
   • an indicator that turns on when their face is found.

   The Continue button stays disabled until a face is found. Put this trial early. Every saccade.js trial after it expects the camera to be running.

4. Measure the timing delay

   timeline.push({ type: jsPsychSaccadeTimeSync });

   There is a short delay between the moment something appears on the screen and the moment the camera captures a picture of the participant reacting to it. The delay differs from one computer to the next, and the browser has no way to report it.

   This trial measures it. For fifteen seconds the screen switches between black and white at random moments while the camera watches the screen. Comparing when the screen changed with when the camera saw it gives the delay, and saccade.js then subtracts it from every gaze timestamp recorded afterwards.

   Do you need it? If your question is where people looked (which picture, which word), you can skip this trial. If it is when they looked (how fast they moved their eyes to a target), keep it. Timing and synchrony explains the details.

5. Calibrate, then check the calibration

   timeline.push({ type: jsPsychSaccadeCalibrate }, { type: jsPsychSaccadeValidate });

   Calibration teaches the tracker how this particular person's eyes look when they are looking at different parts of the screen. The participant follows a dot to thirteen positions, which takes about twenty seconds. Until calibration finishes, the tracker cannot estimate gaze at all.

   Validation then checks how well calibration worked. The participant looks at nine more dots, eight of them in positions calibration did not use, and saccade.js measures how far its estimates land from each dot. The result is saved as median_error_viewport: the typical error, measured as a fraction of the window's diagonal so that it compares across screens. For example, 0.08 means the estimates typically landed about 8% of the diagonal away from the dot, which is roughly 150 pixels on a 1,600 × 900 window.

   Report this number in your paper. Decide before you collect data how large an error is too large for your design, and exclude participants above it. The saccade-validate page shows how to give a participant a second attempt at calibration when the first one goes badly.

6. Record gaze during a trial

   Now the task itself. Any jsPsych trial can record gaze, whatever plugin it uses. Add the extension to it with the extensions parameter:

   timeline.push({
     type: jsPsychHtmlKeyboardResponse,
     stimulus: `
       <div style="display:flex; gap:20vw; justify-content:center">
         <img id="left" src="left.png" width="300" />
         <img id="right" src="right.png" width="300" />
       </div>
       <p>Look at whichever picture you prefer, then press any key.</p>`,
     extensions: [{ type: jsPsychExtensionSaccade, params: { targets: ["#left", "#right"] } }],
   });
   
   jsPsych.run(timeline);

   The targets setting is optional but useful. Each participant's window is a different size, so the pictures end up in different places on different screens. targets records exactly where #left and #right were for this participant, so you can tell which picture they looked at without having to work out the layout later.

The whole file

Here is everything above in one page:

<!doctype html>
<html>
  <head>
    <script src="https://unpkg.com/jspsych@8"></script>
    <script src="https://unpkg.com/@jspsych/plugin-html-keyboard-response@2"></script>
    <script src="https://unpkg.com/@saccadejs/core@0.1.0"></script>
    <script src="https://unpkg.com/@saccadejs/extension@0.1.0"></script>
    <script src="https://unpkg.com/@saccadejs/plugin-preview@0.1.0"></script>
    <script src="https://unpkg.com/@saccadejs/plugin-time-sync@0.1.0"></script>
    <script src="https://unpkg.com/@saccadejs/plugin-calibrate@0.1.0"></script>
    <script src="https://unpkg.com/@saccadejs/plugin-validate@0.1.0"></script>
    <link rel="stylesheet" href="https://unpkg.com/jspsych@8/css/jspsych.css" />
  </head>
  <body></body>
  <script>
    const jsPsych = initJsPsych({
      extensions: [{ type: jsPsychExtensionSaccade }],
      on_finish: () => jsPsych.data.displayData(),
    });

    const timeline = [
      // Ask for the camera, download the model, and let the participant line up their face.
      { type: jsPsychSaccadePreview },
      // Measure this computer's screen-to-camera delay, so later timestamps can be corrected.
      { type: jsPsychSaccadeTimeSync },
      // Calibrate on 13 dots, then check accuracy on 9 different dots.
      { type: jsPsychSaccadeCalibrate },
      { type: jsPsychSaccadeValidate },
      // The task. The `extensions` line is what turns gaze recording on for this trial.
      {
        type: jsPsychHtmlKeyboardResponse,
        stimulus: `
          <div style="display:flex; gap:20vw; justify-content:center">
            <img id="left" src="left.png" width="300" />
            <img id="right" src="right.png" width="300" />
          </div>
          <p>Look at whichever picture you prefer, then press any key.</p>`,
        extensions: [
          { type: jsPsychExtensionSaccade, params: { targets: ["#left", "#right"] } },
        ],
      },
    ];

    jsPsych.run(timeline);
  </script>
</html>

Open it in Chrome or Edge over https:// or from localhost. Opening the file directly (a file:// address) will not work, because the browser will not allow the camera. To use localhost, run a local web server in the folder. If you have Node.js installed, npx serve is one way. The first run downloads the model; after that the browser keeps a copy.

What the data looks like

Every trial with the extension attached gets three extra fields in its data:

{
  "rt": 3184,
  "saccade_data": [
    { "x": 412, "y": 388, "t": 51 },
    { "x": 418, "y": 381, "t": 85 },
    { "x": 903, "y": 402, "t": 118 }
    // ...one row for each camera frame in which a face was found
  ],
  "saccade_targets": {
    "#left":  { "x": 260, "y": 300, "width": 300, "height": 200,
                "top": 300, "bottom": 500, "left": 260, "right": 560 },
    "#right": { "x": 900, "y": 300, "width": 300, "height": 200,
                "top": 300, "bottom": 500, "left": 900, "right": 1200 }
  },
  "saccade_timing": {
    "offset_ms": 78, "corrected": true, "clock": "captureTime",
    "dropped_frames": 1, "fps": 29.6, "smoothing_frames": 1, "trial_start": 48213.7
  }
}

saccade_data is the gaze itself. Each row is one estimate:

• x and y are the position on the page, in pixels, measured from the top-left corner of the participant's browser window. (The visible area of the window is called the viewport, and you will see that word throughout these docs.)
• t is when the participant was looking there, in milliseconds since the trial began. It is on the same clock as jsPsych's rt, so a row with t: 3000 is 184 ms before the key press above. The screen-to-camera delay measured in step 4 has already been subtracted.

There is one row per camera frame in which the tracker found a face. A gap in t means it did not find one: the participant blinked, looked away, or moved out of the camera's view.

saccade_targets records where each element you named in targets was on this participant's screen, in the same pixel coordinates as x and y. In the example, the left picture spans from left: 260 to right: 560 horizontally, so the first two gaze rows (x of 412 and 418) are on it and the third (x: 903) is on the right picture.

saccade_timing describes how the timestamps were produced. You can ignore it to start with; Timing and synchrony explains each field.

Here is how to count how many gaze samples landed on each picture:

const data = jsPsych.data.get().filter({ trial_type: "html-keyboard-response" }).values()[0];

// Is sample s inside rectangle r?
const inBox = (s, r) => s.x >= r.left && s.x <= r.right && s.y >= r.top && s.y <= r.bottom;

const left = data.saccade_data.filter((s) => inBox(s, data.saccade_targets["#left"])).length;
const right = data.saccade_data.filter((s) => inBox(s, data.saccade_targets["#right"])).length;

Installing with npm instead

If your experiment is built with npm and a bundler (for example, with jsPsych Builder or Vite), install the packages instead of using <script> tags:

npm install jspsych @saccadejs/extension @saccadejs/plugin-preview \
  @saccadejs/plugin-time-sync @saccadejs/plugin-calibrate @saccadejs/plugin-validate

import { initJsPsych } from "jspsych";
import jsPsychExtensionSaccade from "@saccadejs/extension";
import jsPsychSaccadePreview from "@saccadejs/plugin-preview";
import jsPsychSaccadeTimeSync from "@saccadejs/plugin-time-sync";
import jsPsychSaccadeCalibrate from "@saccadejs/plugin-calibrate";
import jsPsychSaccadeValidate from "@saccadejs/plugin-validate";

The timeline is exactly the same. @saccadejs/core is installed automatically as a dependency of the other packages; you only need to install it yourself if you use the core API directly.

Using saccade.js without jsPsych

The core library does not need jsPsych. Underneath, the plugins do the same four things you just saw: start the tracker, measure the delay, calibrate, and then read gaze estimates as they arrive. Without jsPsych, you do those yourself and draw your own calibration dots:

import { SaccadeTracker, runCalibration, defaultGrid13, runLoopback } from "@saccadejs/core";

// 1. Start the camera and load the model.
const tracker = new SaccadeTracker();
await tracker.init();
document.body.appendChild(tracker.video);
tracker.start();

// 2. Measure the screen-to-camera delay.
const { lagMs, verdict } = await runLoopback(tracker);

// 3. Calibrate on 13 points. You supply the function that draws each dot.
await runCalibration(tracker, defaultGrid13(), { settleMs: 1000, captureMs: 500 }, {
  showTarget: (target, phase) => drawYourOwnTarget(target, phase),
});
tracker.fitCalibration();

// 4. Receive an estimate for every camera frame.
tracker.onFrame((frame) => {
  if (!frame.gaze) return; // no face in this frame
  const t = (frame.time.meanCapture ?? frame.time.capture) - (verdict === "OK" ? lagMs : 0);
  console.log(frame.gaze.x, frame.gaze.y, t);
});

One difference to watch for: here frame.gaze.x and frame.gaze.y are fractions of the window (0 is the left or top edge, 1 is the right or bottom edge), not pixels. Multiply by window.innerWidth and window.innerHeight to get pixels. The core API page has the details.

Before you collect real data

This page gets an experiment running. Before you put one in front of participants, read these:

• Hosting the assets. By default the model downloads from a public server. For a real study, serve it from your own, so an outage elsewhere cannot stop your study.
• saccade-performance. A five-second check of how fast the tracker runs on the participant's computer, so you can turn away computers too slow for your study before they reach calibration.
• Timing and synchrony. What t means exactly, how precise it is, and how to line it up with events of your own, such as a sound.
• Reference. Every setting and every data field.