Introducing nmf_audio_benchmark (https://github.com/ax-le/nmf_audio_benchmark/), an open-source toolbox for benchmarking NMF algorithms and variants on audio tasks. This toolbox was designed because new low-rank factorization models are still being developed (in particular under new constraints or objective functions), but testing them under real conditions with audio data is not easy. In that spirit, this toolbox was primarily created to: - Provide a standardized framework for evaluating NMF-based audio processing techniques. - Offer a collection of audio datasets and pre-processing - Include a set of baseline NMF algorithms for comparison. - Enable easy integration of new NMF models for benchmarking. This toolbox is primarily designed for people developing new low-rank factorization models. Hence, it should be easy to add new NMF algorithms.