Fixes for generic benchmark wrt corstone-300 (#2581) @tensorflow/micro Update README to use cortex-m4 (it has instruction cycle timer) Update README sample output to reflect cortex-m4 Update test script to not use Ethos-U co-processor (instruction cycle timer non-functional in simulator) when running benchmark. bug=fixes #2580
diff --git a/tensorflow/lite/micro/tools/benchmarking/README.md b/tensorflow/lite/micro/tools/benchmarking/README.md index 45f52b5..c203820 100644 --- a/tensorflow/lite/micro/tools/benchmarking/README.md +++ b/tensorflow/lite/micro/tools/benchmarking/README.md
@@ -46,7 +46,7 @@ ## Build and run for Cortex-M using Corstone 300 simulator Build and run with model compiled into tool: ``` -make -f tensorflow/lite/micro/tools/make/Makefile TARGET=cortex_m_corstone_300 TARGET_ARCH=cortex-m0 OPTIMIZED_KERNEL_DIR=cmsis_nn BUILD_TYPE=default run_tflm_benchmark -j$(nproc) GENERIC_BENCHMARK_MODEL_PATH=tensorflow/lite/micro/models/person_detect.tflite GENERIC_BENCHMARK_ARENA_SIZE=`expr 150 \* 1024` +make -f tensorflow/lite/micro/tools/make/Makefile TARGET=cortex_m_corstone_300 TARGET_ARCH=cortex-m4 OPTIMIZED_KERNEL_DIR=cmsis_nn BUILD_TYPE=default run_tflm_benchmark -j$(nproc) GENERIC_BENCHMARK_MODEL_PATH=tensorflow/lite/micro/models/person_detect.tflite GENERIC_BENCHMARK_ARENA_SIZE=`expr 150 \* 1024` ``` ## Build and run using Bazel @@ -66,20 +66,14 @@ -------------------- Compiled on: -Thu Mar 7 04:59:13 AM PST 2024 +Fri May 17 03:36:59 PM PDT 2024 -------------------- -Git SHA: 27b1f546cec03c87deaf2ff94c830f9cbd0f2e69 +Git SHA: a4390a1d73edf5a8d3affa1da60e1eba88e0cb13 Git status: On branch main Your branch is up to date with 'origin/main'. - -Untracked files: - (use "git add <file>..." to include in what will be committed) - MODULE.bazel - MODULE.bazel.lock -nothing added to commit but untracked files present (use "git add" to track) -------------------- C compiler: tensorflow/lite/micro/tools/make/downloads/gcc_embedded/bin/arm-none-eabi-gcc Version: @@ -93,9 +87,9 @@ -Wimplicit-function-declaration -std=c11 -Werror -fno-unwind-tables -ffunction-sections -fdata-sections -fmessage-length=0 -DTF_LITE_STATIC_MEMORY -DTF_LITE_DISABLE_X86_NEON --DCMSIS_NN -DKERNELS_OPTIMIZED_FOR_SPEED -mcpu=cortex-m0 -mfpu=auto +-DCMSIS_NN -DKERNELS_OPTIMIZED_FOR_SPEED -mcpu=cortex-m4+nofp -mfpu=auto -DTF_LITE_MCU_DEBUG_LOG -mthumb -mfloat-abi=soft -funsigned-char -mlittle-endian --fomit-frame-pointer -MD -DARMCM0 +-fomit-frame-pointer -MD -DARMCM4 C++ compiler: tensorflow/lite/micro/tools/make/downloads/gcc_embedded/bin/arm-none-eabi-g++ Version: @@ -112,31 +106,31 @@ -DTF_LITE_STATIC_MEMORY -DTF_LITE_DISABLE_X86_NEON -Wsign-compare -Wdouble-promotion -Wunused-variable -Wunused-function -Wswitch -Wvla -Wall -Wextra -Wmissing-field-initializers -Wstrict-aliasing -Wno-unused-parameter -DCMSIS_NN --DKERNELS_OPTIMIZED_FOR_SPEED -mcpu=cortex-m0 -mfpu=auto -DTF_LITE_MCU_DEBUG_LOG -mthumb --mfloat-abi=soft -funsigned-char -mlittle-endian -fomit-frame-pointer -MD -DARMCM0 --DCMSIS_DEVICE_ARM_CORTEX_M_XX_HEADER_FILE="ARMCM0.h" +-DKERNELS_OPTIMIZED_FOR_SPEED -mcpu=cortex-m4+nofp -mfpu=auto -DTF_LITE_MCU_DEBUG_LOG +-mthumb -mfloat-abi=soft -funsigned-char -mlittle-endian -fomit-frame-pointer -MD +-DARMCM4 -DCMSIS_DEVICE_ARM_CORTEX_M_XX_HEADER_FILE="ARMCM4.h" -DGENERIC_BENCHMARK_USING_BUILTIN_MODEL -DGENERIC_BENCHMARK_MODEL_HEADER_PATH="tensorflow/lite/micro/models/person_detect_model_da ta.h" -DGENERIC_BENCHMARK_MODEL_NAME=person_detect --DGENERIC_BENCHMARK_TENSOR_ARENA_SIZE=153600 -DGENERIC_BENCHMARK_SHOW_META_DATA +-DGENERIC_BENCHMARK_TENSOR_ARENA_SIZE=153600 Optimization: kernel= -O2 core= -Os third-party-kernel= -O2 -------------------- Target information: TARGET=cortex_m_corstone_300 -TARGET_ARCH=cortex-m0 +TARGET_ARCH=cortex-m4 OPTIMIZATION=cmsis_nn BUILD_TYPE=default -------------------- NN library download URLs: -http://github.com/ARM-software/CMSIS-NN/archive/8492d82a1a81651977c5f5128492b4a0f0cf6715.z +http://github.com/ARM-software/CMSIS-NN/archive/01dee38e6d6bfbbf202f0cd425bbea1731747d51.z ip NN library MD5 checksums: -2cb03e4f044b78af6751009cd53247a8 +f20be93ededf42bb704c19f699a24313 -------------------- Model SHA1: @@ -241,13 +235,13 @@ T#25(MobilenetV1/Conv2d_9_depthwise/depthwise_weights/read) shape:[1, 3, 3, 128], type:INT8 RO 1152 bytes, buffer: 80, data:[^, ., ~, ., ., ...] T#26(MobilenetV1/Conv2d_9_pointwise/weights/read) shape:[128, 1, 1, 128], type:INT8 RO -16384 bytes, buffer: 16, data:[., ., , ., , ...] +16384 bytes, buffer: 16, data:[., ., , ., %, ...] T#27(MobilenetV1/Logits/AvgPool_1a/AvgPool) shape:[1, 1, 1, 256], type:INT8 T#28(MobilenetV1/Logits/Conv2d_1c_1x1/BiasAdd) shape:[1, 1, 1, 2], type:INT8 T#29(MobilenetV1/Logits/Conv2d_1c_1x1/Conv2D_bias) shape:[2], type:INT32 RO 8 bytes, buffer: 2, data:[16267, -17079] T#30(MobilenetV1/Logits/Conv2d_1c_1x1/weights/read) shape:[2, 1, 1, 256], type:INT8 RO -512 bytes, buffer: 3, data:[., , ., ., ., ...] +512 bytes, buffer: 3, data:[., %, ., ., ., ...] T#31(MobilenetV1/Logits/SpatialSqueeze) shape:[1, 2], type:INT8 T#32(MobilenetV1/Logits/SpatialSqueeze_shape) shape:[2], type:INT32 RO 8 bytes, buffer: 1, data:[1, 2] @@ -336,56 +330,56 @@ T#88(input) shape:[1, 96, 96, 1], type:INT8 --------------------------------------------------------------- Model size: 300568 bytes - Non-data buffer size: 81640 bytes (27.16 ) - Total data buffer size: 218928 bytes (72.84 ) - (Zero value buffers): 0 bytes (00.00 ) + Non-data buffer size: 81640 bytes (27.16 %) + Total data buffer size: 218928 bytes (72.84 %) + (Zero value buffers): 0 bytes (00.00 %) * Buffers of TFLite model are mostly used for constant tensors. And zero value buffers are buffers filled with zeros. Non-data buffers area are used to store operators, subgraphs and etc. You can find more details from https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/schema/schema.fbs -------------------- -TfliteGetModel took 0 ticks (0 ms). +TfliteGetModel took 4 ticks (0 ms). -DEPTHWISE_CONV_2D took 0 ticks (0 ms). -DEPTHWISE_CONV_2D took 0 ticks (0 ms). -CONV_2D took 0 ticks (0 ms). -DEPTHWISE_CONV_2D took 0 ticks (0 ms). -CONV_2D took 0 ticks (0 ms). -DEPTHWISE_CONV_2D took 0 ticks (0 ms). -CONV_2D took 0 ticks (0 ms). -DEPTHWISE_CONV_2D took 0 ticks (0 ms). -CONV_2D took 0 ticks (0 ms). -DEPTHWISE_CONV_2D took 0 ticks (0 ms). -CONV_2D took 0 ticks (0 ms). -DEPTHWISE_CONV_2D took 0 ticks (0 ms). -CONV_2D took 0 ticks (0 ms). -DEPTHWISE_CONV_2D took 0 ticks (0 ms). -CONV_2D took 0 ticks (0 ms). -DEPTHWISE_CONV_2D took 0 ticks (0 ms). -CONV_2D took 0 ticks (0 ms). -DEPTHWISE_CONV_2D took 0 ticks (0 ms). -CONV_2D took 0 ticks (0 ms). -DEPTHWISE_CONV_2D took 0 ticks (0 ms). -CONV_2D took 0 ticks (0 ms). -DEPTHWISE_CONV_2D took 0 ticks (0 ms). -CONV_2D took 0 ticks (0 ms). -DEPTHWISE_CONV_2D took 0 ticks (0 ms). -CONV_2D took 0 ticks (0 ms). -DEPTHWISE_CONV_2D took 0 ticks (0 ms). -CONV_2D took 0 ticks (0 ms). -AVERAGE_POOL_2D took 0 ticks (0 ms). -CONV_2D took 0 ticks (0 ms). -RESHAPE took 0 ticks (0 ms). -SOFTMAX took 0 ticks (0 ms). +DEPTHWISE_CONV_2D took 224622 ticks (8 ms). +DEPTHWISE_CONV_2D took 175917 ticks (7 ms). +CONV_2D took 249560 ticks (9 ms). +DEPTHWISE_CONV_2D took 84958 ticks (3 ms). +CONV_2D took 145817 ticks (5 ms). +DEPTHWISE_CONV_2D took 164915 ticks (6 ms). +CONV_2D took 197283 ticks (7 ms). +DEPTHWISE_CONV_2D took 41304 ticks (1 ms). +CONV_2D took 99472 ticks (3 ms). +DEPTHWISE_CONV_2D took 79969 ticks (3 ms). +CONV_2D took 151505 ticks (6 ms). +DEPTHWISE_CONV_2D took 20053 ticks (0 ms). +CONV_2D took 78521 ticks (3 ms). +DEPTHWISE_CONV_2D took 38127 ticks (1 ms). +CONV_2D took 132862 ticks (5 ms). +DEPTHWISE_CONV_2D took 38127 ticks (1 ms). +CONV_2D took 132865 ticks (5 ms). +DEPTHWISE_CONV_2D took 38127 ticks (1 ms). +CONV_2D took 132859 ticks (5 ms). +DEPTHWISE_CONV_2D took 38127 ticks (1 ms). +CONV_2D took 132851 ticks (5 ms). +DEPTHWISE_CONV_2D took 38127 ticks (1 ms). +CONV_2D took 132853 ticks (5 ms). +DEPTHWISE_CONV_2D took 9585 ticks (0 ms). +CONV_2D took 78470 ticks (3 ms). +DEPTHWISE_CONV_2D took 17473 ticks (0 ms). +CONV_2D took 143615 ticks (5 ms). +AVERAGE_POOL_2D took 2229 ticks (0 ms). +CONV_2D took 386 ticks (0 ms). +RESHAPE took 28 ticks (0 ms). +SOFTMAX took 163 ticks (0 ms). "Unique Tag","Total ticks across all events with that tag." -DEPTHWISE_CONV_2D, 0 -CONV_2D, 0 -AVERAGE_POOL_2D, 0 -RESHAPE, 0 -SOFTMAX, 0 -"total number of ticks", 0 +DEPTHWISE_CONV_2D, 1009431 +CONV_2D, 1808919 +AVERAGE_POOL_2D, 2229 +RESHAPE, 28 +SOFTMAX, 163 +"total number of ticks", 2820770 [[ Table ]]: Arena Arena Bytes % Arena @@ -398,7 +392,7 @@ Eval tensor data | 0 | 1068 | 1068 | 89 | 1.26 Persistent tensor data | 1 | 64 | 64 | 2 | 0.08 Persistent quantization data | 2 | 40 | 40 | 4 | 0.05 - Persistent buffer data | 3 | 25876 | 25704 | 90 | 30.65 + Persistent buffer data | 3 | 25872 | 25704 | 90 | 30.64 Tensor variable buffer data | 4 | 0 | 0 | 0 | 0.00 Node and registration array | 5 | 992 | 992 | 31 | 1.17 Operation data | 6 | 0 | 0 | 0 | 0.00 @@ -409,11 +403,12 @@ [warning ][main@0][01 ns] Simulation stopped by user --- FVP_MPS3_Corstone_SSE_300 statistics: ------------------------------------- -Simulated time : 5.474678s -User time : 1.609473s -System time : 0.123380s -Wall time : 1.892705s -Performance index : 2.89 -FVP_MPS3_Corstone_SSE_300.cpu0 : 78.98 MIPS ( 136866941 Inst) +Simulated time : 2.879993s +User time : 2.027100s +System time : 0.135914s +Wall time : 2.663214s +Performance index : 1.08 +cpu0 : 27.03 MIPS ( 71999848 Inst) +Memory highwater mark : 0x11919000 bytes ( 0.275 GB ) ------------------------------------------------------------------------------- ```
diff --git a/tensorflow/lite/micro/tools/ci_build/test_cortex_m_corstone_300.sh b/tensorflow/lite/micro/tools/ci_build/test_cortex_m_corstone_300.sh index a5d02db..39e8449 100755 --- a/tensorflow/lite/micro/tools/ci_build/test_cortex_m_corstone_300.sh +++ b/tensorflow/lite/micro/tools/ci_build/test_cortex_m_corstone_300.sh
@@ -43,9 +43,9 @@ readable_run make -j$(nproc) -f tensorflow/lite/micro/tools/make/Makefile CO_PROCESSOR=ethos_u OPTIMIZED_KERNEL_DIR=${OPTIMIZED_KERNEL_DIR} TARGET=${TARGET} TARGET_ARCH=${TARGET_ARCH} TOOLCHAIN=${TOOLCHAIN} build readable_run make -f tensorflow/lite/micro/tools/make/Makefile CO_PROCESSOR=ethos_u OPTIMIZED_KERNEL_DIR=${OPTIMIZED_KERNEL_DIR} TARGET=${TARGET} TARGET_ARCH=${TARGET_ARCH} TOOLCHAIN=${TOOLCHAIN} test -# run generic benchmark +# Run generic benchmark. +# Instruction cycle counter not functional in simulator for ethos-u co-processor. readable_run make -j$(nproc) -f tensorflow/lite/micro/tools/make/Makefile \ - CO_PROCESSOR=ethos_u \ OPTIMIZED_KERNEL_DIR=${OPTIMIZED_KERNEL_DIR} \ TARGET=${TARGET} \ TARGET_ARCH=${TARGET_ARCH} \