Adding IREE parameter archive format and tooling support. (#15670)
The new format allows us to store parameters for both inference and
training observing the requirements for both efficient CPU and
GPU/accelerator execution. We can also support additional storage types
such as splats allowing for stripped parameter files that work with
programs compiled assuming real parameters. The format has a provision
for referencing external file ranges but support for reading such files
is TBD.
The format can be used like gguf/safetensors and is supported by the
tooling in the same way. Additionally a new `iree-convert-parameters`
tool is added to convert any format supported for loading (gguf,
safetensors, and irpa itself) into irpa files with some control over
which parameters are included, renaming of parameters, and stripping
parameters and replacing them with splat values. This should make it
easy to take any gguf/safetensors file and quickly create stripped
variants for easy reproducers/CI benchmarking without needing to ship
around the original files. The `iree-create-parameters` tool can be used
to create empty archives that are ready for initialization from a
program that mutates them or to create parameter archives with named
parameters when there is no source gguf/safetensors file.
All of this is still using memory-mapped files; this limits our
parameter file sizes on 32-bit systems but I suspect no one is going to
run this tool for large models on 32-bit systems. In the future we can
make the conversion tool use the HAL and schedule out optimized file
I/O. For now we just copy parameters via a normal read/write loop and
it's fastish-enough (pretty much I/O bound, with less optimal reads
because of memory mapping). For me with a cold cache it takes ~1min to
rewrite a 25GB file and 25sec with a hot cache.
This initial commit has the IRPA builder using the new iree_io_stream_t
but switching all format parsers to use it is deferred to future
changes.
Progress on #15521.
diff --git a/tools/CMakeLists.txt b/tools/CMakeLists.txt
index 65608e3..84c1a85 100644
--- a/tools/CMakeLists.txt
+++ b/tools/CMakeLists.txt
@@ -111,6 +111,22 @@
iree_cc_binary(
NAME
+ iree-convert-parameters
+ SRCS
+ "iree-convert-parameters-main.c"
+ DEPS
+ iree::base
+ iree::base::internal::file_io
+ iree::base::internal::flags
+ iree::hal
+ iree::io::formats::irpa
+ iree::io::parameter_index
+ iree::io::scope_map
+ iree::tooling::parameter_util
+)
+
+iree_cc_binary(
+ NAME
iree-cpuinfo
SRCS
"iree-cpuinfo.c"
@@ -122,6 +138,22 @@
iree_cc_binary(
NAME
+ iree-create-parameters
+ SRCS
+ "iree-create-parameters-main.c"
+ DEPS
+ iree::base
+ iree::base::internal::file_io
+ iree::base::internal::flags
+ iree::hal
+ iree::io::formats::irpa
+ iree::io::parameter_index
+ iree::io::scope_map
+ iree::io::stream
+)
+
+iree_cc_binary(
+ NAME
iree-dump-instruments
SRCS
"iree-dump-instruments-main.c"
