Real-time BCI: why an offline accuracy is not a working BCI

An offline decoder classifies neatly cut trials with known onsets. A live BCI gets a continuous stream, has no idea when the user starts, and pays for every false alarm.

Updated

Most BCI papers report offline accuracy: epochs cut around known cues, classified after the fact. A real-time (online) BCI has to decide continuously, from a sliding window, with a latency budget, while the user reacts to its feedback. Closed-loop feedback changes the user's brain activity too, which is part of how users learn to control a BCI.

The gap between the two is large and predictable. The useful step in between is pseudo-online evaluation: replay a recorded session as if it were live and measure what a user would have experienced.

Metrics that matter online

BCILattice's Pseudo-Online Evaluator block reports false alarms per minute, delay and bit rate from recorded data, which is a far better predictor of live usefulness than an epoch-level accuracy. The Time-Resolved Decoder shows when in the trial the classes become separable, which sets the earliest a live decision can be made.

MetricWhy
False alarms per minuteIn self-paced BCIs, the decoder must stay quiet when the user is not issuing a command.
Detection delayHow long after the user's intent the command arrives.
Information transfer rateBits per minute, combining accuracy, number of choices and selection time.
Calibration timeHow much data a new session needs before it works.

Streaming and deployment: current status

  • Lab Streaming Layer (beta)

    BCILattice can receive an LSL stream to preview, capture and analyse in the same session. It is in beta and not yet validated against physical acquisition hardware.

  • On-device (not yet released)

    BCINexus Edge, for running trained models on boards such as a Raspberry Pi or Jetson with a set loop rate, is in internal testing and not yet released.

  • Today, for closed-loop

    Use a real-time framework such as OpenViBE or BCI2000 for the live loop, and BCILattice for the offline development and evaluation around it.

Designing a decoder to survive going live

Train on sliding windows rather than perfectly aligned epochs, evaluate across sessions, and plan for drift with adaptation. Domain adaptation blocks such as Session Transfer and Euclidean Alignment address between-session shift. Small models (EEGNet and its edge variants) keep latency low.

Questions

What is the difference between online and offline BCI?

An offline BCI analyses recorded data after the fact, usually with trials cut around known events. An online (real-time) BCI decodes a continuous stream as it arrives and gives feedback within a latency budget.

What is pseudo-online evaluation?

Replaying a recorded session as a continuous stream through a decoder, as if it were live, to measure false alarms, delays and information transfer rate before testing with users.

What is Lab Streaming Layer (LSL)?

LSL is an open system for streaming time-synchronised data such as EEG, markers and other sensors over a network, widely used to connect acquisition hardware to BCI software.

Can BCILattice run a closed-loop BCI?

Not yet as a validated system. LSL streaming is in beta and on-device deployment is in internal testing. BCILattice covers offline development and pseudo-online evaluation today.

Read the reference

Related

Run a study end to end, on your own machine

The free plan covers two studies, the full analysis suite, and 118 of the 119 ML blocks.