Disassembly Debugging and Responsible Reverse Engineering

Disassembly debugging answers concrete questions about binaries you own or are explicitly authorized to assess: which instructions ran, which ABI rule applies, why a source breakpoint moved, or where a crash occurred. It is not permission to inspect, alter, bypass, or redistribute someone else’s software. Start from authorization, preserve evidence, and use a small reproducible binary whenever learning. A debugger is powerful because it can execute code; do not use it as a way to “safely” open an unknown program.

Scope, safety, and a lab you can reproduce

Written permission should identify the binary, systems, actions allowed, time window, and reporting path. Software licenses, contracts, privacy law, anti-circumvention rules, and local law can impose additional limits. A purchased copy, a public download, or technical curiosity is not automatically authorization. If scope is unclear, stop and obtain clarification. Keep notes of commands, hashes, build flags, debugger version, and observations; they make an engineering result reviewable without turning it into a guide for bypassing protections.

Never run an unknown or suspected-malicious sample on a workstation, production host, or personal account. Do not upload confidential binaries to public analysis services. Professional malware-response workflows use formally approved, isolated environments, snapshots, controlled networking, monitoring, and trained personnel; those controls reduce risk but are not an invitation to experiment. This lesson does not teach evasion, persistence, credential access, or bypassing security controls. For instruction-level practice, compile this harmless file yourself:

/* tiny.c: owned, deterministic enough to inspect */
int twice_if_positive(int x) {
    if (x > 0) return x * 2;
    return 0;
}
int main(void) { return twice_if_positive(3) == 6 ? 0 : 1; }

Build a debug artifact and a separate optimized artifact: cc -g -O0 -fno-omit-frame-pointer tiny.c -o tiny-debug and cc -g -O2 tiny.c -o tiny-O2. On supported targets, clang or gcc may replace cc. The first build makes an initial debugging exercise easier; it is not a claim that production builds should disable all hardening. Inspect an object without running it using cc -c -g -O2 tiny.c -o tiny.o.

Start with file facts, not guesses

A disassembler needs the correct architecture, mode, byte order, and entry context. First identify the artifact: file tiny-debug gives a compact classification; on ELF, readelf -h tiny-debug identifies class (32/64-bit), endianness, machine, and file type. readelf -l shows program headers/segments, readelf -S shows sections, and readelf -d displays dynamic dependencies. Use readelf -r for relocations and readelf --debug-dump=decodedline for decoded source line information where present.

Tool	Good first use	Important limitation
`objdump`	`objdump -drwC -S tiny.o`; decode, relocations, names, available source	Needs target support and trustworthy metadata
`llvm-objdump`	`llvm-objdump -dr --source tiny.o`	Option spelling and supported formats differ by version
`readelf`	ELF headers, sections, relocations, DWARF	ELF only; it is not a PE/Mach-O inspector
`nm`	`nm -nC tiny-debug` for ordered, demangled symbols	Stripping, inlining, and visibility can remove useful names
GDB	Run an owned program under controlled inputs and inspect state	Executing a binary has the binary’s effects

On GNU binutils, objdump -d disassembles sections marked executable; -D tries all sections and can decode data as misleading instructions. Prefer -d and narrow the question with --disassemble=twice_if_positive. -r retains relocation annotations in an object file, -C demangles C++ names, and -S interleaves source only if debug source mapping and source files are available. LLVM’s llvm-objdump is a valuable second decoder, not an oracle. Compare disagreements against architecture manuals, relocations, and the actual target.

Objects, executables, symbols, and stripped files

A relocatable object has unresolved references that the linker will resolve. A linked executable or shared library can add startup code, linker-generated stubs, unwind tables, and dynamic-loader machinery. Static linking includes selected library object code in the output. Dynamic linking leaves dependencies for the system loader and commonly uses import tables, GOT/PLT-style mechanisms, or platform equivalents. Therefore the same source call can look like a relocation in tiny.o, a local direct call in one executable, or a linkage stub in another. Read the surrounding metadata before assigning intent to indirection.

ELF is common on Linux and records sections, program headers, symbol tables, relocations, and often DWARF debug sections. PE/COFF is standard on Windows; import/export tables and PDB debug files are central there, and tools such as Visual Studio’s debugger, dumpbin, and LLVM tooling may be appropriate. Mach-O is used on Apple platforms; it has load commands, segments/sections, dynamic libraries, and commonly external dSYM debug information, inspected with platform tools such as otool and nm. Similar concepts do not mean records or command options are interchangeable.

A stripped executable has had some symbol or debug information removed. It can still contain dynamic symbols needed for loading, unwind information, literals, relocation-related data, or exported names; it might also contain almost none of the convenient names. Stripping does not make a binary unreadable, but it lowers confidence in source-level labels. Do not invent function names or source lines from patterns. Report address ranges, observed control flow, format facts, and uncertainty. When authorized, ask the owner for the exact build, separate debug package/PDB/dSYM, map file, and matching source rather than attempting to reconstruct more than the evidence supports.

GDB fundamentals on the owned lab

Launch GDB with gdb --args ./tiny-debug. help and help command document the installed version; show architecture confirms GDB’s selected architecture. Set a source breakpoint with break twice_if_positive or break tiny.c:2, examine it with info breakpoints, then type run. A breakpoint stops before the selected location executes. Continue with continue. Remove a breakpoint only by its number after checking it: delete 1.

GDB command	Purpose in this lab	Reading caution
`list`, `bt`	Show nearby source and backtrace	Optimized/inlined frames may be incomplete or surprising
`next`, `step`	Advance by source line, over/into calls	Source lines may cover ranges, not individual instructions
`stepi` (`si`), `nexti` (`ni`)	Advance one instruction, into/over calls	Instruction stepping executes code; use only owned safe inputs
`disassemble /m twice_if_positive`	Show mixed source/instructions when mappings exist	Mappings are approximate under optimization
`info registers`, `p/x $pc`	Display registers and program counter in hex	Register names and ABI vary by architecture
`x/8i $pc`, `x/16xb ADDRESS`	Examine upcoming instructions or bytes	Only examine valid addresses in the permitted process
`info symbol ADDRESS`	Relate an address to a known symbol	Nearest symbol is not proof of exact source intent

Use display/i $pc to show the next instruction at each stop and undisplay when finished. info registers shows general state; info all-registers can be much noisier. p $rax is meaningful on x86-64, while ARM64 commonly uses $x0–$x30, $sp, and $pc; use info registers to see the target’s accepted names. x means “examine memory,” not “execute”: formats include x/4xw ADDRESS for four hex words and x/s ADDRESS for a C string. Read memory first. Arbitrarily writing registers or memory changes the experiment and can invalidate evidence.

Source stepping versus instruction stepping

Debug info maps address ranges to source locations; it does not impose a one-instruction-per-line relationship. At -O0, setup instructions can share a line, and a condition can involve loads, comparisons, and branches. At -O2, constant folding removes lines, registers replace variables, code motion changes apparent order, and common tails merge. A compiler can use branchless conditional instructions, inline a callee, or split one source statement across hot and cold blocks. next may appear to skip a line, repeat one, or land in a line that was not textually “next.” That is usually a mapping and optimization effect, not automatically a debugger failure.

Inlining makes this especially visible. The optimizer can replace a call with the callee’s operations at the caller site; no physical call instruction or standalone callee body need remain. DWARF can describe inline call chains, so bt may show logical inline frames, while raw disassembly shows only one contiguous instruction sequence. Tail calls can also reuse a caller’s return path. Frame-pointer omission and aggressive optimization can make stack unwinding less precise; modern unwind metadata often helps, but a perfect backtrace is never guaranteed after memory corruption.

Compare tiny-debug and tiny-O2 deliberately. In the optimized build, twice_if_positive may be inlined into main or simplified because its input is a constant. To force a meaningful function comparison without claiming normal optimization behavior, make a separate variant whose input comes from a defined external interface, then document the change. Do not use undefined behavior or opaque tricks as a benchmark. Build configuration is part of the result.

Breakpoints, symbols, and disassembly

A symbolic breakpoint is convenient when symbols and debug information exist. An address breakpoint may be appropriate in an authorized low-level investigation, but ASLR, position-independent executables, shared-library loading, and different builds make hard-coded addresses fragile. Prefer a symbol, source location, or a runtime-resolved address you document. Hardware watchpoints can stop on a permitted data location, but they are limited resources and platform dependent. Conditional breakpoints evaluate debugger expressions and may affect timing; use them sparingly and record them.

disassemble twice_if_positive prints the named range; disassemble /r includes raw bytes, and disassemble /m requests mixed source. info functions twice searches known function names. On ELF, info sharedlibrary describes loaded shared libraries. These are observations, not authorization to inspect unrelated processes. A named symbol may denote a PLT entry, thunk, alias, or compiler-generated routine rather than the source function expected. Relocations and dynamic-symbol tables help distinguish those cases.

Crashes and core files

A core dump is a snapshot the operating system may write when a process crashes; it can contain memory, register state, arguments, secrets, and proprietary data. Treat it as sensitive. Collect and retain it only under policy, store it accessibly only to authorized staff, and pair it with the exact executable and compatible debug information. On systems configured to produce a conventional core, open it without rerunning the program: gdb ./tiny-debug core. Then begin with bt, info threads, thread apply all bt, frame N, info registers, and a small disassembly around $pc.

A crash address alone does not prove root cause. The instruction that faults may merely be where earlier memory corruption became visible. Optimized code, omitted frame pointers, stripped symbols, mismatched sources, and overwritten stacks reduce confidence. Preserve the original core; analyze a copy if policy permits; record the executable build ID/hash and loaded library versions. Do not “fix” a production core by modifying it or guessing at source from a nearest line table entry.

Responsible workflow checklist

Stage	Check
Authorize	Written scope, ownership/permission, legal and contractual constraints, reporting contact
Contain	Owned tiny lab or approved professional isolation; never run unknown samples casually
Identify	Format, architecture, ABI, hashes/build IDs, debug companion files, link model
Inspect	Use target-aware `objdump`/`llvm-objdump`, symbols and relocations before interpretations
Debug	Use GDB breakpoints and instruction stepping only on permitted binaries/inputs; preserve originals
Report	Facts, commands, version/flags, address ranges, confidence and limitations; protect sensitive artifacts

For an introductory command-line inspection workflow, cross-reference Code: Disassembling C code on the Command Line. ARM64 readers can continue with Debugging Assembly code for Arm64 with GDB and the ARM64 GDB branch tutorial. Apply their mechanics only within authorized scope.