Was not keeping offset in sync with actual_offset and HLSL could trigger spurious realignments due to the straddle check.