[Metal] Fix fill_buffer_1byte edge handling for unaligned offsets (#24639)
fill_buffer_1byte silently corrupted data for any buffer_offset % 4 in
{1, 3}, and corrupted neighboring bytes for fills contained within a
single 4-byte word. Two independent defects, both in the edge handling:
1. left_mask selected the wrong byte positions. It was computed as ~((1
<< (8 * (4 - left_byte_count))) - 1), selecting byte positions [4-k, 4)
of the left word. But the left edge must overwrite the high (4-k) bytes,
i.e. positions [k, 4), so the mask must be ~((1 << (8 * k)) - 1). The
stray "4 -" is the asymmetry with the correct right_mask (which uses
right_byte_count directly). For k in {1, 3} the wrong mask is a
symmetric 2-byte misplacement (|4-2k| == 2):
- offset % 4 == 1: under-fill -- the leading 2 in-region bytes are left
stale.
- offset % 4 == 3: underrun -- the 2 bytes immediately preceding the
region are clobbered.
2. When the entire fill range fits within one 32-bit word, the left and
right read-modify-write blocks both attacked that same word with masks
assuming a complete edge, writing pattern bytes outside [offset, offset
+ length). Reachable via short 1/2-byte-pattern fills (duplicated to 4
bytes on the host) whose length lands within one word.
Beyond the k == 0 case (handled by fill_buffer_4byte), the kernel was
correct only for offset % 4 == 2 with length >= 2.
Fix (kernel-only):
- Correct left_mask: use left_byte_count instead of (4 -
left_byte_count).
- Add a single-word guard: when left_start == right_start, perform one
read-modify-write over bytes [left_byte_count, right_byte_count) using
mask = left_mask & right_mask; otherwise keep the existing separate
left/right blocks. This also makes length == 0 a safe no-op via mask
collapse, where the original corrupted up to 4 neighbor bytes.
No host-side change: the middle word range [middle_start, right_start)
and thus the thread-count requirement are unchanged, and the existing
dispatch sizing in builtin_executables.m already over-dispatches safely.
Verified:
CTS/CommandBufferFillBufferTest.FillSizeAlignmentAndPatternClasses/metal
and CTS/QueueTransferTest.FillSizeAlignmentAndPatternClasses/metal now
pass (previously failed with a 2-byte under-fill at offset%4==1 and a
2-byte underrun at offset%4==3); all *Fill* cases pass (22/22 in
command_buffer_tests, 8/8 in queue_tests).
Signed-off-by: Alex Vasile <48962821+Alex-Vasile@users.noreply.github.com>diff --git a/runtime/src/iree/hal/drivers/metal/builtin/fill_buffer_generic.metal b/runtime/src/iree/hal/drivers/metal/builtin/fill_buffer_generic.metal
index 281bd05..09dafec 100644
--- a/runtime/src/iree/hal/drivers/metal/builtin/fill_buffer_generic.metal
+++ b/runtime/src/iree/hal/drivers/metal/builtin/fill_buffer_generic.metal
@@ -66,7 +66,7 @@
// *little endian*, left bytes will replace high bits of the leftmost touched
// 32-bit scalar, while right bytes will replace low bits of the rightmost
// touched 32-bit scalar.
- uint32_t left_mask = ~((uint64_t(1) << (8 * (4 - left_byte_count))) - 1);
+ uint32_t left_mask = ~((uint64_t(1) << (8 * left_byte_count)) - 1);
uint32_t right_mask = (uint64_t(1) << (8 * right_byte_count)) - 1;
// Indexing start points in |buffer| for the three parts.
@@ -79,13 +79,24 @@
buffer[middle_start + id] = middle_pattern;
}
- if (left_byte_count != 0 && id == 0) { // Left bytes
- uint32_t old = buffer[left_start];
- buffer[left_start] = (old & (~left_mask)) | (middle_pattern & left_mask);
- }
-
- if (right_byte_count != 0 && id == 0) { // Right bytes
- uint32_t old = buffer[right_start];
- buffer[right_start] = (old & (~right_mask)) | (middle_pattern & right_mask);
+ if (id == 0) {
+ if (left_start == right_start) {
+ // The entire fill range fits within a single 32-bit scalar; read-modify-
+ // write only the bytes in [left_byte_count, right_byte_count).
+ uint32_t mask = left_mask & right_mask;
+ uint32_t old = buffer[left_start];
+ buffer[left_start] = (old & (~mask)) | (middle_pattern & mask);
+ } else {
+ if (left_byte_count != 0) { // Left bytes
+ uint32_t old = buffer[left_start];
+ buffer[left_start] =
+ (old & (~left_mask)) | (middle_pattern & left_mask);
+ }
+ if (right_byte_count != 0) { // Right bytes
+ uint32_t old = buffer[right_start];
+ buffer[right_start] =
+ (old & (~right_mask)) | (middle_pattern & right_mask);
+ }
+ }
}
}