commit | a43730e76b760c9ded726de9b659161645a728d9 | [log] [tgz] |
---|---|---|
author | Lun Dong <lundong@google.com> | Wed Jun 22 21:52:14 2022 -0700 |
committer | Lun Dong <lundong@google.com> | Mon Jun 27 13:37:36 2022 -0700 |
tree | 1e3bbcdccb67ee1580eafbd10ef35369db3b4b51 | |
parent | ffe0c897c8322143af495417c39af4717ed4cdd4 [diff] |
Add support for audio pre-processing (MFCC extraction) This is the complete code change for adding audio pre-processing block (MFCC feature extraction). The code was re-written in C based on http://google3/audio/dsp/mfcc/mfcc_mel.py, with necessary simplifications. We also integrated audio pre-processing into daredevil, to allow for end-to-end simulations with .wav audio input. Numerical correctness has been validated: extracted MFCC matches with the reference (golden_whistle_spectrogram) and returned output is correct (Whistling). Unit tests passed. Profiling result: https://docs.google.com/document/d/1vAgRgZeVyOwIniiIp8RrmEvg3CB_p0PFgB2X9NLN4Fo The pre-processing block adds a relatively constant number of extra instructions. So the percentage depends on the computational complexity of audio models (for daredevil: 65%) Change-Id: I662914e68170e5e28384960ad345c3366f9fb627
This project contains ML model codegen and execution libraries/examples. The artifacts built from this project is targeted for RISC-V 32-bit baremetal machine (Springbok).
If you get this project from Project Shodan manifest, you are all set. If not, you need to have following projects as well to build the project successfully.
<dir>/sw/vec
<dir>/toolchain/iree
<dir>/ml/ml-models-public
This project needs to be at <dir>/sw/vec_iree
If you are running from Project Shodan repo, you can build the artifacts with m iree
. If you download this project alone, you need to download/build IREE's host compiler, then at the top level directory
cmake -B <output dir> -G Ninja \ -DCMAKE_TOOLCHAIN_FILE="$(realpath sw/vec_iree/cmake/riscv_iree.cmake)" \ -DCMAKE_BUILD_TYPE=MinSizeRel \ -DIREE_HOST_BINARY_ROOT="$(IREE_COMPILER_DIR)/install" \ -DRISCV_TOOLCHAIN_ROOT=$(TOOLCHAINRV32_PATH) \ <dir>/sw/vec_iree cmake --build <output dir> --target all
You can run QEMU or RENODE emulation with the built executables. In Shodan repo, qemu_sim_springbok
for QEMU emulation, and sim_springbok
for Renode simulation. Out-of-repo emulation is possible but require more steps.
In Shodan, this project utilizes LLVM lit
and FileCheck
to test the ML executable performance. The tests are defined in the *_test.txt files under samples
. To run the test, at the Shodan top level directory
lit --path <Filecheck dir> -a sw/vec_iree/samples
Filecheck in Debian testing is under /usr/lib/llvm-11/bin
Add -D FEATURES=internal
to enable the internal model tests. Add -D RUNNER=qemu
to enable the qemu tests.