| commit | 3bb67d3972cce3bbec3603b3387647d0e1a5ebab | [log] [tgz] |
|---|---|---|
| author | Ben Vanik <ben.vanik@gmail.com> | Mon Jan 09 17:52:37 2023 -0800 |
| committer | GitHub <noreply@github.com> | Tue Jan 10 01:52:37 2023 +0000 |
| tree | bb238134cf3d034a70fb8d8c76b4d2f3627e6be2 | |
| parent | d222387ce55dbacde67d84d1801c11c3cdb66acd [diff] |
Reworking task API to provide iree_task_executors_create_from_flags. (#11614) This allows for multiple executors to be created from flags that can then be passed into the local-task creation routines. Topology queries are slightly extended to allow for specifying which node the physical cores are selected from when initializing with that mode but otherwise the programmatic control by users not using flags remains the same. The new `--task_topology_nodes=` flag can be used to control which NUMA nodes get executors. By default (or if `current` is specified) the node is inherited from the calling thread but `all` can be specified to create one executor per NUMA node. In addition a comma-separated list of node IDs can be passed to subset the nodes (`0,1,4`). The inheritance is kind of sketchy as the node is queried at driver creation time (usually just during startup) but should play well with numactl launches of the process. NOTE: because the `--task_topology_group_count` flag does no thread affinity placement it is not directly useful with multiple NUMA nodes as each worker may run on any processor. Always use `--task_topology_max_group_count` instead. I've made this an error for now but in the future we may want to allow some mixed modes. To make debugging the topology detection easier the `--dump_task_topologies` flag now prints all configured topologies to stdout. Example output: https://gist.github.com/benvanik/35611e9a343b9f0093187e83cfe6db67 Some TODOs were added for where future work is required around binding host allocations to NUMA nodes. This is inspired by the work of @aviator19941 in #11371 but switched around to create the multiple executors based on the flags, use the existing `iree_thread_affinity_t` to hold the node identifier, and make it possible for programmatic control over the topology node pinning. That PR had a way to filter the nodes a particular device would use based on URI parameters that is not yet in here as we'd need to rework the executor ownership rules on drivers. Closes #11371.
IREE (Intermediate Representation Execution Environment, pronounced as “eerie”) is an MLIR-based end-to-end compiler and runtime that lowers Machine Learning (ML) models to a unified IR that scales up to meet the needs of the datacenter and down to satisfy the constraints and special considerations of mobile and edge deployments.
See our website for project details, user guides, and instructions on building from source.
IREE is still in its early phase. We have settled down on the overarching infrastructure and are actively improving various software components as well as project logistics. It is still quite far from ready for everyday use and is made available without any support at the moment. With that said, we welcome any kind of feedback on any communication channels!
See our website for more information.
IREE is licensed under the terms of the Apache 2.0 License with LLVM Exceptions. See LICENSE for more information.