ANNI

Overview

ANNI (Auditory Nerve Noticeability Index) measures how noticeable the difference between a reference and a test recording is. It simulates how the auditory nerve responds to each one and compares the two responses. The score is in d′ units: 0 means no difference, and 1 is a just-noticeable difference.

We built it for changes near the threshold of hearing, such as audio watermarks, where PESQ and ViSQOL miss what listeners hear. Goodbye, PESQ. Meet ANNI. tells the story and explains how it works.

ANNI is a research prototype (v0.1). Its scale is calibrated on classic laboratory listening experiments. Listening tests on real-world audio have not confirmed it yet.

Open source under the MIT license: GitHub · PyPI

Installation

ANNI needs Python 3.9 or newer:

python3 -m venv .venv
source .venv/bin/activate
pip install deepmark-anni

It pulls in PyTorch. On Linux the default PyTorch build includes CUDA, about 3 GB to download. On a machine without an NVIDIA GPU, install the CPU build first, then ANNI:

pip install torch --index-url https://download.pytorch.org/whl/cpu

Usage

# pip install soundfile, or use any audio loader
import soundfile as sf
from anni import ANNI

reference, sr = sf.read("original.wav")
test, _ = sf.read("watermarked.wav")

anni = ANNI()
print(anni(reference, test, sr=sr))
# e.g. {'D_peak': 1.49, 'D_med': 0.81}

D_peak: how noticeable the difference is at its most noticeable 200 ms moment.
D_med: how noticeable it is at a typical moment (the median over 200 ms windows).

Lower is better. To compare two processed versions, score both against the same reference.

Inputs

Two mono signals at the same sample rate. ANNI resamples them to 44.1 kHz.
The reference plays at 65 dB SPL, about the level of conversation; change it with ANNI(level=...). Both signals get the same gain, so a level change counts as a difference.
A constant delay of up to ±10 ms is removed. If the lengths differ, the longer signal is cut.
At least 0.5 s long. A 5 s pair takes about 1 s and 0.9 GB on a laptop CPU; on an NVIDIA GPU, use ANNI(device="cuda").
ANNI warns when it changes the inputs, and when the two signals correlate weakly, for example after a speech codec. The scores may then not be meaningful.

Evaluation

On ODAQ, a dataset of music, film sound and speech over music rated by 26 listeners, D_med ranks the versions like the listeners do, with a Spearman correlation of 0.85. That is on par with Zimtohrli (0.83) and above ViSQOL (0.65) and PESQ (0.42). On degraded speech (packet loss, clipping, noise), PESQ and ViSQOL follow listeners better.

To hear the difference, the repository has examples of small changes that PESQ and ViSQOL miss, with audio and scores.

Limitations

It compares the signals moment by moment, so it does not suit audio that was time-stretched, pitch-shifted or sent through a speech codec.
It does not hear energy above 16 kHz or below 80 Hz, very fast fluctuations or fine timing.
Mono only, and it models normal hearing.

Citation

The auditory-nerve model is adapted from Mark R. Saddler's auditory_machine_learning. If you use ANNI, please also cite:

Saddler, M. R., & McDermott, J. H. (2024). Models optimized for real-world tasks reveal the task-dependent necessity of precise temporal coding in hearing. Nature Communications. doi:10.1038/s41467-024-54700-5