tree 0cfcfa42b8aa11ea2b4884f9db10a93c93c45587
parent 57b9239e0d4988dc440cbe794f689003714e2bda
author Stella Laurenzo <stellaraccident@gmail.com> 1691735842 -0700
committer GitHub <noreply@github.com> 1691735842 -0700
gpgsig -----BEGIN PGP SIGNATURE-----
 
 wsBcBAABCAAQBQJk1dciCRBK7hj4Ov3rIwAAYAUIAKcHKbIRFcimxdhQZkaBZ3Mk
 d3CMJAhfIOg2taTc69VJQJwLgbpgZToMo2qKBvnWp7JaejLxVe9saL35NNpr4kp7
 NI2t0JJ0VEeJkPs6syRMtocoGwnAyHu7SP2RBbGUfnGtoL8xX+aYMOB/dcZY7BCR
 nneY4ki4xb5sSzdEB65GsezlcnrkRp8ziheQZrlpESf6KgLtb99+J2vq6oWz00Wk
 N5AUbnrSyL/FB/qLnlY1c0DKvZOoAc77aptYstG7EEy/+zt1/eJ7DNNMVRwFCd/y
 J++ed8U1K8o5YhUUhK8ILhsQ2QTJy+rtRm3zhVEwkn/EN8lH3c16V8mgrogL0iE=
 =tJRI
 -----END PGP SIGNATURE-----
 

Add a CLI `iree-ir-tool` with a command to strip data. (#14636)

Been meaning to do this for a while in order to have a place to stash
more power-user style things that core developers typically use iree-opt
for.

This one adds a `strip-data` sub-command which uses the passes from
#14627 to systematically replace tensor constants with synthetic values.
With an ASAN/asserts build, this was able to strip a 7GiB int4 vicuna
MLIR file in ~5s and a 23GiB h2ogpt model in about 30s (the latter has
some other characteristics which make it more expensive to load as well
as being bigger). Results were a 2.6MiB and 1.4MiB MLIR file
respectively, consisting just of program IR and annotations for
synthetic data.

Getting the opt pipeline right for arbitrary input is a bit tricky, so I
decided we should just armor this into a tool

From installed packages, this can be used as:

```
iree-ir-tool strip-data input.mlir -o output.mlir
```

From a build tree with Python setup:

```
python -m iree.compiler.tools.ir_tool strip-data input.mlir -o output.mlir
```

Required adding some additional compiler APIs:

* `ireeCompilerInvocationRunPassPipeline` to run an arbitrary textual
pass pipeline on an invocation.
* `ireeCompilerInvocationOutputIRBytecode` to emit bytecode from an
invocation.