Integrate RISP4ML into Shodan ML Toolchain using Fssd model This CL is Part 2 of the comprehensive 3-part changes of integrating RISP4ML into Shodan ML toolchain. This change focuses on integrating RISP4ML using the FSSD model as an example. I have verifed that the output@RISP4ML running on Qemu/Renode completely matches with the output@RISPML running on google3. Details of the 3-part changes: (Part 1) Completely re-wrote RISP4ML in plain C. - https://spacebeaker-review.googlesource.com/c/shodan/sw/vec-iree/+/17180 - The original RISP4ML on google3 was written in C++. The memory usage with C++ RISP4ML was significantly larger. Thus, I have completely re-written RISP4ML in plain C. The memory increase with the plain-C RISP4ML is much smaller. - Made the input/output shape to be configurable arguments instead of constants. - Made low-level optimizations and simplifications. (Part 2) Integrated plain-C RISP4ML into Shodan ML toolchain (this CL) - Use fssd_25_8bit_v2 as an example to load the binary (raw bayer) file and go through RISP4ML toolchain, and feed the RISP4ML output into IREE flow. - The fssd_25_8bit_v2 example is up and running, and the correctness of output has been verified (compare output@plain-C risp4ml against output@google3). (Part 3) Added unit tests for plain-C RISP4ML - Added unit tests based on pw_unit_test - Build unit tests in iree_cc_binary - Run and check unit tests via lit_test Change-Id: I90eb31cb4b5cecfee79ad0952025ad97e12b9b52
diff --git a/samples/quant_model/CMakeLists.txt b/samples/quant_model/CMakeLists.txt index c0ed7c3..e3e70ff 100644 --- a/samples/quant_model/CMakeLists.txt +++ b/samples/quant_model/CMakeLists.txt
@@ -159,6 +159,20 @@ QUANT ) +if(${BUILD_INTERNAL_MODELS}) + +iree_model_input( + NAME + fssd_quant_input + SHAPE + "1, 480, 640, 1" + SRC + "$ENV{ROOTDIR}/sw/vec_iree/samples/risp4ml/test_data/faces_480x640_uint8_numpy_bayer.bin" + QUANT +) + +endif(${BUILD_INTERNAL_MODELS}) + #------------------------------------------------------------------------------- # Binaries to execute the MLIR bytecode modules #------------------------------------------------------------------------------- @@ -334,9 +348,11 @@ SRCS "fssd_25_8bit_v2.c" DEPS + ::fssd_quant_input_c ::fssd_25_8bit_v2_bytecode_module_static ::fssd_25_8bit_v2_bytecode_module_static_c iree::vm::bytecode_module + samples::risp4ml::pipeline::pipeline samples::util::util LINKOPTS "LINKER:--defsym=__stack_size__=100k" @@ -348,8 +364,10 @@ SRCS "fssd_25_8bit_v2.c" DEPS + ::fssd_quant_input_c ::fssd_25_8bit_v2_c_module_static_c ::fssd_25_8bit_v2_c_module_static_emitc + samples::risp4ml::pipeline::pipeline samples::util::util LINKOPTS "LINKER:--defsym=__stack_size__=100k"
diff --git a/samples/quant_model/fssd_25_8bit_v2.c b/samples/quant_model/fssd_25_8bit_v2.c index 7a272da..0ba243e 100644 --- a/samples/quant_model/fssd_25_8bit_v2.c +++ b/samples/quant_model/fssd_25_8bit_v2.c
@@ -6,9 +6,11 @@ #include "iree/base/api.h" #include "iree/hal/api.h" +#include "samples/risp4ml/pipeline/pipeline.h" #include "samples/util/util.h" // Compiled module embedded here to avoid file IO: +#include "samples/quant_model/fssd_quant_input_c.h" #if !defined(BUILD_EMITC) #include "samples/quant_model/fssd_25_8bit_v2_bytecode_module_static.h" #include "samples/quant_model/fssd_25_8bit_v2_bytecode_module_static_c.h" @@ -60,13 +62,13 @@ iree_status_t load_input_data(const MlModel *model, void **buffer, iree_byte_span_t **byte_span) { iree_status_t result = alloc_input_buffer(model, buffer); - // Populate initial value - srand(11111111); - if (iree_status_is_ok(result)) { - for (int i = 0; i < model->input_length[0]; ++i) { - ((uint8_t *)*buffer)[i] = (uint8_t)rand(); - } - } + + ImageU8 input = { + .width = 640, .height = 480, .num_channels = 1, .data = fssd_quant_input}; + ImageU8 output = { + .width = 320, .height = 320, .num_channels = 3, .data = buffer[0]}; + isp_pipeline(&input, &output); + byte_span[0] = malloc(sizeof(iree_byte_span_t)); *byte_span[0] = iree_make_byte_span( buffer[0], model->input_size_bytes[0] * model->input_length[0]);