commit b14aaf486b4408b8bbbd6454b0247b105849a6e4
author bjacob <benoitjacob@google.com> Mon Jan 30 12:05:49 2023 -0500
committer GitHub <noreply@github.com> Mon Jan 30 12:05:49 2023 -0500
tree 08f9c733269bfae1a4df1e2f4ab225716f2a97f5
parent 5f6f9892b5fe0c4b228e4e659e6205056d580c42

pack microkernel padding improvements (#11987)

- More correct: the old code was incorrect for really large padding
(test expanded).
- Faster: padding is handled with an intermediate buffer, which allows
padded tiles to still be packed by the fast target-specific tile func,
as opposed to a super slow generic fallback. Moreover, we optimize the
common case where the padding byte pattern is a single-byte pattern so
we can use memset.

Even with that, padding is still surprisingly slow. This means that
compiler-level ways to remove padding overhead (pad fusions) are
important in a way that is not diminished by the ability to call
microkernels. @MaheshRavishankar @hanhanW

Note: in ruy and XNNPACK, padding is handled by having separate source
pointers for N strided rows being packed (so padding rows can just have
their pointer point at some zeros). This does not scale well to the
degree of generality and runtime-flexibility that we are aiming for here
(this essentially requires knowing some tile dimensions at compile time,
and being able to design the whole system around a restricted range of
values for these tile sizes). While not the fastest, the
intermediate-buffer approach taken here is not that slow in principle
(that it happens to be less fast than I hoped seems to be an accident of
how it's compiled by Clang, but I haven't been able to dig to the bottom
of that), and at least it makes for smaller code and insulates the
target-optimized kernels from any padding consideration.

At first I thought that in large-stride cases, L1 cache aliasing would
be a major issue, so I thought that the temporary buffer would be useful
to deal with that. But it almost never was a net win in benchmarks. So
the PR evolved towards its current state where the temp buffer is only
used to deal with padding. But at least it's still easy to revisit that
later.