dispelledx86-64 SIMD and Performance Fundamentals
SIMD means single instruction, multiple data: one instruction operates on several independent elements packed in a vector register. It is an optimization technique for appropriate data-parallel work, not a promise that every loop becomes faster. This lesson discusses SIMD extensions available on x86-64 processors (also called AMD64 or Intel 64). Microsoft often calls the same 64-bit x86 platform “x64”; that shorthand should not obscure the distinction between the base x86-64 ISA and optional SSE2, AVX, and AVX2 feature sets.
Feature levels and register views
| Feature | Register/instruction width | Typical packed data examples | Availability rule |
|---|---|---|---|
| SSE2 | 128-bit XMM | integers; double-precision floating point | Required by the x86-64 architectural baseline, though OS/toolchain policy still matters. |
| AVX | 256-bit YMM for AVX operations | packed floating point; non-destructive three-operand forms | Optional; CPU and operating system support are both required. |
| AVX2 | 256-bit YMM | adds broad integer vector operations | Optional extension beyond AVX. |
XMM, YMM, and ZMM names are views of vector register state, not three unrelated sets. An AVX-capable processor has YMM registers whose low 128 bits correspond to XMM registers. AVX’s VEX encoding enables three operands, so a destination need not overwrite one input. AVX2 extends integer operations to 256 bits; it does not add double-width floating point beyond AVX’s floating-point capability. This lesson deliberately stops there: later extensions have additional feature, state-management, and frequency considerations.
On 64-bit x86, SSE2 is part of the baseline specified for long mode, which lets compilers use XMM registers for ordinary scalar floating-point code. That fact does not allow a program to assume AVX or AVX2. Shipping a binary that executes an unsupported optional instruction causes an illegal-instruction failure. Choose a conservative baseline, compile specialized variants, and dispatch only after legitimate feature detection.
Detection includes operating-system state
The processor’s CPUID instruction reports architectural feature bits. For AVX, hardware support alone is insufficient: the operating system must save and restore extended vector state. CPUID’s OSXSAVE bit indicates OS support for XSAVE management, and XGETBV can then reveal whether XMM and YMM state are enabled in XCR0. AVX2 is reported in a separate CPUID feature leaf and also depends on the AVX state prerequisites. Compilers and system libraries encapsulate these details; use their supported facilities rather than scattering ad hoc inline assembly through an application.
GCC and Clang provide __builtin_cpu_supports("avx2") on supported x86 targets; consult their documentation for initialization and portability constraints. MSVC documents __cpuidex and _xgetbv intrinsics. A robust application can use a baseline implementation by default and select a separately compiled AVX2 implementation only when its detection routine says it is safe. Keep the dispatch code itself within the baseline, and test both paths. Compiler function multiversioning can help, but inspect the generated target clones and dispatch policy before relying on it.
Scalar and vector work
Scalar code processes one element per operation. Vector code packs, for example, four 32-bit integers into an XMM register or eight into a YMM register. Consider an owned function that adds two arrays of float into an output array. A scalar loop conceptually loads one value from each input, adds them, and stores one result. An AVX loop can load eight floats, perform one packed add, and store eight results. The vector body needs a remainder path for elements after the largest vector-sized prefix unless the loop bounds guarantee a multiple.
; conceptual AVX-style vector body, eight float elements
vmovups ymm0, [rsi + rax*4] ; input a[i..i+7], unaligned-safe form
vaddps ymm0, ymm0, [rdx + rax*4]
vmovups [rdi + rax*4], ymm0 ; output[i..i+7]
; increment i by 8, then handle the tail safely
This annotation explains data flow, not a complete callable function. Real compiler output must also satisfy the relevant ABI, prove aliasing and bounds facts, choose loop versioning, and arrange the tail. Do not copy partial snippets into a program. Prefer clear C or C++ first, then inspect whether auto-vectorization occurred; use documented intrinsics only when measurement demonstrates a need and the resulting code remains understandable.
Alignment and correctness
Alignment is the address’s divisibility by a power of two. A 32-byte-aligned address is divisible by 32. Some vector instructions require aligned operands, while others explicitly accept unaligned addresses; instruction choice, generation, and memory placement determine the actual requirement. “Unaligned” does not mean invalid: ordinary allocations often satisfy enough alignment for their element type but not necessarily 16- or 32-byte alignment at every offset. A valid unaligned access can still be slower in some placements, especially if it crosses cache-line or page boundaries.
Do not promise alignment to the compiler unless the allocation and every derived pointer truly meet it. A wrong alignment assumption can produce a faulting aligned load or undefined behavior at the language level. Similarly, a vectorized loop must preserve the scalar algorithm’s bounds, aliasing, rounding, overflow, NaN, and exception behavior. Floating-point reassociation may change numerical results; fast-math options make explicit tradeoffs and should be reviewed, not applied merely to encourage vectorization. For overlapping ranges, use an algorithm/API with defined overlap behavior rather than assuming independent pointers.
Reading compiler vectorization reports
Ask the compiler for a report as well as assembly. GCC has options such as -fopt-info-vec; Clang offers optimization remarks such as -Rpass=loop-vectorize and missed-optimization remarks. Compile an owned loop with a fixed target baseline, then with -mavx2 only for an AVX2-specific build. The report may say a loop was not vectorized because of possible aliasing, unknown trip count, a function call, or strict floating semantics. Treat that as a design question: can the source express independent ranges or a safe precondition? Do not add an incorrect restrict, alignment assertion, or unsafe cast just to silence the report.
Disassembly can confirm broad facts: packed operations often end in ps, pd, or integer vector mnemonics; VEX-encoded instructions commonly begin with v; XMM versus YMM operands reveal 128- versus 256-bit operations. It cannot by itself prove a speedup. A scalar cleanup loop and runtime checks may be the right result for correctness.
Performance is a measurement problem
Throughput is how much work can begin or complete per unit time; latency is how long a dependent result takes. A vector instruction can improve throughput while a loop remains limited by memory bandwidth, cache misses, branches, dependency chains, stores, or another bottleneck. Fewer instructions is not automatically fewer cycles, and wider vectors do not guarantee a proportional speedup. Data layout and locality can matter more than replacing one arithmetic instruction.
Modern processors also vary frequency with power, temperature, and instruction mix. Sustained wide-vector workloads can run at a different frequency or trigger power-management behavior on some implementations. Therefore “AVX2 is twice as fast as SSE” is not a safe general claim. Measure the complete workload on representative hardware, under a documented power policy, after warm-up, and report distributions across repeated runs. Compare equivalent outputs, pin down input sizes and alignment, prevent the compiler from optimizing away the result, and separate setup/allocation time from the kernel if the question is kernel throughput.
Use trusted profilers and counters available on your owned system, but interpret them cautiously: wall-clock time includes scheduling noise; cycle counters can vary with frequency; virtual machines can alter observations. Benchmark a scalar baseline, an auto-vectorized version, and a specialized version only if all are correct. Keep tests that include zero lengths, non-multiples of vector width, deliberately unaligned valid slices, negative values for integer code, and floating-point edge cases relevant to the algorithm.
Mixing SSE and AVX responsibly
Code that mixes legacy SSE instructions with 256-bit AVX instructions can incur transition costs on some processors when upper YMM state is live. Compilers commonly insert vzeroupper at appropriate boundaries in AVX-generated code. This is a compiler and microarchitecture concern, not a cue to sprinkle the instruction into unrelated code. Compile modules with compatible target settings, follow compiler intrinsics guidance, and inspect boundaries when a profiler identifies a real issue.
Safe exercises
- Write an owned elementwise integer-add loop with a scalar reference implementation. Test all output elements before timing anything.
- Enable a vectorization report and explain one accepted or missed loop decision in terms of source semantics.
- Compile a baseline and an AVX2 variant. On a machine where detection permits it, dispatch between them and verify identical results for lengths 0 through 33.
- Benchmark several lengths, including tails and cache-unfriendly sizes. Report median and spread, not one best run, and note that frequency behavior may affect conclusions.
For register names, extension rules, and address expressions, revisit Registers, Addressing, and Position-Independent Code. For compiler flags, symbols, and reliable inspection, use Reading x86-64 Compiler Output and Object Files. ABI details in System V and Windows ABI Practice remain relevant when vector functions cross a call boundary.