Language and Speech Models

Tests of MobileBert and streamable Keyword Spotting models.

IREE has three backend targets: vmla , llvm-ir and vulkan-spirv. We also test TFLite in our infrastructure for benchmarking purposes.

Last Updated: 2020/12/8

End to end test of MobileBert on SQuAD

target	tflite	vmla	vulkan-spirv
mobile_bert_squad_test	✓	✓	✓

End to end tests of Keyword Spotting Streaming models

target	tflite	vmla	vulkan-spirv
att_mh_rnn	✓	✗	✗
att_rnn	✓	✗	✗
cnn	✓	✓	✓
cnn_stride	✓	✓	✓
crnn	✓	✗	✓
dnn	✓	✓	✓
ds_cnn	✓	✓	✓
ds_tc_resnet	✓	✓	✓
gru	✓	✗	✓
inception	✓	✓	✓
inception_resnet	✓	✓	✓
lstm	✓	✗	✓
mobilenet	✓	✓	✓
mobilenet_v2	✓	✓	✓
svdf	✓	✓	✓
svdf_resnet	✓	✓	✓
tc_resnet	✓	✓	✓
xception	✓	✓	✓

End to end tests of Keyword Spotting Streaming models in internal streaming mode

target	tflite	vmla	vulkan-spirv
cnn	✗	✓	✓
cnn_stride	✗	✓	✓
crnn	✗	✓	✓
dnn	✗	✓	✓
ds_tc_resnet	✗	✓	✓
gru	✗	✓	✓
lstm	✗	✓	✓
svdf	✗	✓	✓