Audio signal classification using joint time-frequency scattering

Audio signals exhibit complex time-frequency structures that are usually inadequately represented using traditional descriptors for audio classification, which treat the temporal and frequential domains separately. We introduce the joint time-frequency scattering transform, which decomposes a wavelet scalogram in time and log-frequency using a two-dimensional wavelet transform. The resulting descriptor is shown to successfully characterize sophisticated time-frequency structures such as time-varying filters and frequency-modulated excitations. The power of the representation is also illustrated through state-of-the-art classification results on a phone segment classification task.