Vorbis audio format is suitable for effective speech compression (note that this will also downmix it from stereo to mono).

oggenc (command line tool) parameters for speech compression with best size/quality ratio:

smallest file, lower quality:
-q 0 --resample 22050 -S 1 --downmix --advanced-encode-option lowpass_frequency=6

small file, better quality:
-q 1 --resample 22050 -S 1 --downmix --advanced-encode-option lowpass_frequency=8

oggenc is part of package vorbis-tools.

Sidenote: Opus audio format is generally more effective for both music and speech, but player support is a bit worse compared to Vorbis.