dispelledx86 32-bit Stack Frames and Calling Conventions
A calling convention is a contract between separately compiled pieces of a program. It specifies argument locations, result locations, stack ownership, preserved registers, alignment, and details such as structure returns. IA-32 has several conventions, so “x86 calling convention” is not enough information. This article uses common 32-bit C conventions as teaching models; always follow the ABI and compiler documentation for the operating system, object format, language, and compiler options actually selected.
The stack model
On conventional 32-bit x86, the stack grows toward lower addresses. ESP points at its current top. A push subtracts four bytes then stores a 32-bit value; pop loads then adds four bytes. A near call pushes a return address and transfers control; ret obtains that address from the stack. Those effects explain a simple frame, but an optimizer can omit it, reserve all locals at once, inline a function, or keep values in registers. Read a frame diagram as an ABI aid, not a fixed signature for locating functions.
; pedagogical 32-bit frame, Intel syntax
sum_two:
push ebp
mov ebp, esp
mov eax, [ebp+8] ; first stack argument in this cdecl example
add eax, [ebp+12] ; second stack argument
pop ebp
ret
Immediately after entry, [esp] is the return address. After the usual push ebp; mov ebp, esp, [ebp] is the caller’s saved EBP, [ebp+4] is the return address, and the first stack argument is at [ebp+8]. A local can be made by subtracting from ESP and addressed at a negative EBP offset. The example has no locals, so it does not allocate space. It also assumes the caller and callee have agreed on cdecl; it is not a universal recipe.
Who cleans arguments?
| Convention | Arguments | Stack cleanup | Use and caveat |
|---|---|---|---|
| cdecl | Usually right-to-left on stack | caller | Common 32-bit C convention; supports variable arguments because caller knows the count. |
| stdcall | Usually stack arguments | callee | Historically common for 32-bit Windows APIs; exact decoration/toolchain behavior varies. |
| fastcall | Some early integer arguments in registers | varies by compiler convention | There is no single portable “fastcall”; name the compiler’s documented variant. |
For a two-argument cdecl call, a caller may push the second value, push the first, execute call sum_two, then add eight to ESP after return. A stdcall callee may instead return with ret 8, which removes its two stack arguments in addition to the return address. Mixing these contracts corrupts the stack pointer even if the first call appears to work. Variadic functions are a practical reason cdecl makes the caller reclaim space.
“fastcall” needs especially careful qualification. Microsoft 32-bit __fastcall commonly places the first eligible arguments in ECX and EDX, while GCC attributes and Borland-family conventions have their own rules. Register use, aggregate passing, naming decoration, and fallback to stack arguments are not interchangeable. A header, compiler attribute, and target ABI are more authoritative than a mnemonic convention label.
Volatile and preserved registers
In common i386 System V and 32-bit Microsoft-style conventions, EAX, ECX, and EDX are caller-saved (volatile): a caller that needs one after a call must save it. EBX, ESI, EDI, and EBP are normally callee-saved: a callee that changes one must restore its incoming value before returning. ESP must of course be restored to the contractually expected position. This shared pattern is useful, but special functions, compiler switches, position-independent code, and non-C language runtimes can add rules.
| Value | Typical responsibility | Reason |
|---|---|---|
| EAX | callee supplies scalar integer result; caller treats prior value as lost | Efficient short result path. |
| ECX, EDX | caller saves if live across call | Scratch registers and possible fastcall argument registers. |
| EBX, ESI, EDI, EBP | callee saves before use and restores | Caller can keep long-lived values there. |
| EFLAGS | usually caller-clobbered | Arithmetic and comparisons naturally change flags. |
A correct callee that uses ESI could push it in its prologue and pop it in reverse order before ret. Saving a register “just in case” is not a substitute for understanding the interface, and failure paths must restore the same state too. When writing inline assembly, tell the compiler every input, output, and clobber; manually preserving a register behind the compiler’s back does not describe memory effects or condition-code changes.
Alignment, locals, and unwinding
Stack alignment is an ABI rule, not a cosmetic preference. SSE instructions and compiler-generated code may require or prefer 16-byte alignment at particular points. The exact i386 requirement is platform and compiler dependent, so hand-written code must consult its target ABI and the compiler’s calling-convention documentation. A mismatched assumption can break a library routine far from the call site.
Frame pointers make debugging and simple unwinding easier, but -fomit-frame-pointer can free EBP. Debuggers and exception systems can instead rely on DWARF call-frame information or platform-specific unwind metadata. Therefore a chain of saved EBP values is neither a security boundary nor a reliable way to infer all callers. Preserve correct metadata when an assembler supports it, and use compiler-generated assembly as a reference for your exact target.
Integer, x87, and SSE interfaces
The integer calling rules do not fully describe floating-point or vector code. IA-32 began with the x87 floating-point unit, whose ST(0) stack register is traditionally used for scalar floating return values in common 32-bit ABIs. SSE later offered XMM registers and scalar/vector instructions. Compilers may choose x87 or SSE based on target flags, and a function’s prototype controls how floating and aggregate values are passed. Do not infer an interface merely because a disassembly happens to use an XMM register. State initialization, control words, alignment, and register preservation must all match the documented ABI.
Object-only example
The following NASM source is deliberately an ELF32 object, not a complete program. It has no entry point, runtime, or system call; assembling it therefore does not require a 32-bit libc or multilib runtime.
; add_cdecl.asm
BITS 32
section .text
global add_cdecl
add_cdecl:
push ebp
mov ebp, esp
mov eax, [ebp+8]
add eax, [ebp+12]
pop ebp
ret
nasm -f elf32 add_cdecl.asm -o add_cdecl.o
readelf -h -s add_cdecl.o
objdump -dr -Mintel add_cdecl.o
Use the inspection output to identify the ELF32 class and exported symbol; do not turn this lesson into a binary launcher. Linking requires a matching i386 linker setup and, when calling C, compatible startup files and libraries. Those resources are deliberately outside this object-only example.
Boundaries and safe practice
A function contract also includes types. A byte, a signed 32-bit integer, a pointer, a structure, and a variadic argument list are not interchangeable merely because the machine moves words. Signed extension, hidden structure-return pointers, and language-specific exception handling are common sources of mistakes. Export an explicit prototype, compile a tiny reference implementation for the intended target, and compare symbol and relocation information before integrating assembly into a program you own.
- Draw the cdecl frame for a function with two arguments and one four-byte local, labeling offsets from EBP.
- Modify the object-only example to preserve ESI while using it for a temporary. Explain why this is required under the stated common convention.
- Find your compiler’s documentation for its 32-bit fastcall attribute. Record its target and version before comparing it with another compiler’s rule.
Reviewing an interface
Before linking hand-written assembly with another language, make a compact interface note: target triple and object format; symbol spelling and visibility; convention and prototype; argument widths and signedness; result location; stack alignment at the call boundary; registers and flags clobbered; and floating-point/vector assumptions. Build a reference object from a tiny implementation under the same compiler options and inspect its symbols, relocations, and generated assembly. This is safer than relying on a web snippet whose operating system or compiler is unstated.
A stack frame is private implementation state after entry, while the call boundary is public contract. For example, one compiler may spill an argument into a local slot and another may never materialize that slot. Both can satisfy the same cdecl interface. Conversely, an apparently tidy prologue can still be wrong if it fails to restore ESI or returns with a different ESP. Test only programs and inputs you are authorized to use, and preserve debug/unwind data when the selected toolchain requires it.
For register and encoding background, continue with x86 32-bit registers, modes, and encoding. The overview of x86 calling conventions separates these IA-32 rules from x86-64 references and explains why architecture names alone never define a callable interface.