The 286: Protected Mode and the Limits of DOS

Intel’s 80286, introduced in 1982 and used by IBM in the 1984 PC/AT, was a major change hidden beneath a familiar 8086-compatible surface. It retained real mode for existing software but added protected mode: hardware-managed segment descriptors, privilege checks, task-related state, and a much larger address space. The chip did not by itself replace DOS. Operating systems, drivers, applications, and the installed base of real-mode programs determined how much of its new machinery people could use.

From segment values to descriptors

In real mode, a segment register supplies the value used in the familiar segment-times-sixteen address calculation. In protected mode, the visible segment value is a selector. It chooses a descriptor from a global or local descriptor table. The descriptor supplies a base address, a limit, a type, and access rights. Intel’s architecture can then check that an offset stays within the segment limit and that the current privilege level is permitted to use the segment. A bad selector, limit violation, or access-rights violation raises an exception instead of silently reaching an arbitrary address.

The 286 has four privilege levels, conventionally called rings 0 through 3. Ring 0 is normally used by the kernel; ring 3 is normally used by applications. Gates and controlled transitions allow selected services to cross boundaries. Rings are mechanisms, not a complete security policy: a sound protected operating system still needs correct kernel code, validation, drivers, and administration. The processor also includes descriptor types for code, data, system structures, and tasks.

Mode or facilityWhat it providesWhat it does not provide alone
Real mode8086-style addressing and startup compatibilityPer-process protection or access beyond the one-megabyte map.
Protected modeDescriptors, limits, privilege checks, up to 16 MiB physical addressing386-style paging or an easy path back to real mode.
Privilege ringsHardware distinction between trusted and less-trusted codeCorrectly designed operating-system services.
286 task facilitiesArchitectural task-state supportA requirement that every OS use hardware task switching.

Why DOS did not simply move upward

PC DOS and MS-DOS were designed around real mode and largely gave one program direct control of the machine. Many applications assumed conventional memory, BIOS and DOS interrupts, direct video access, and hardware-specific timing. The 286’s protected mode made those assumptions harder to preserve, particularly when applications and device drivers expected to manipulate segment registers or hardware freely. Protected-mode operating systems existed, including Xenix and later OS/2, but they needed applications, drivers, and installation support suited to their model.

The 286 can address 16 MiB of physical memory in protected mode, far beyond the 8086’s one-megabyte physical space. This is not the same as a flat 16 MiB pointer model. Offsets remain sixteen bits and descriptors establish bases and limits; programs can use multiple segments. It is consequently misleading to call the 286 simply a “16 MB DOS machine.” DOS’s usual real-mode execution environment retained its own address-map and compatibility constraints.

The awkward return

On the original 286 architecture, software can set the protected-mode enable bit to enter protected mode, but cannot clear it by an ordinary instruction to resume real mode. A reset returns the processor to real mode. AT-compatible machines exploited reset circuitry and carefully preserved state to make a controlled transition; later systems and extenders developed techniques around this limitation. This was workable engineering, but it was not the clean, routine mode switching offered by the 80386. The detail mattered to developers attempting to combine a DOS environment with protected-memory services.

Memory managers and DOS extenders were responses to an application market that could not abandon DOS overnight. An extender typically loaded a protected-mode program while providing a DOS-compatible interface for file and console services. It did not make every ordinary DOS program protected-mode safe. DPMI, specified later for 386-era systems, standardized useful services for DOS programs using protected mode; it should not be retroactively treated as an original 286 feature. Expanded memory (EMS), extended memory (XMS), and hardware-specific banking schemes also addressed different bottlenecks and are not interchangeable terms.

AT systems in context

The PC/AT paired the 286 with a 16-bit expansion bus now commonly called AT-class ISA. Its higher-capacity disks, 1.2 MB floppy drive, and more capable system board made it attractive for business work. Yet “AT compatible” described an ecosystem with variations in BIOS behavior, controllers, clock rates, and expansion cards. The IBM history provides the surrounding system story. A 286 board could be technically capable while a particular program remained tied to a display adapter, memory manager, or DOS version.

Protected mode anticipated the separation of kernel and application that became routine in later PC operating systems. It also demonstrates a recurring x86 pattern: newer facilities are introduced alongside mechanisms for existing software. The cost is complexity, but the benefit is migration rather than abrupt replacement. The 386 altered the balance by adding 32-bit addressing, paging, and virtual-8086 mode.

A descriptor example

Suppose an operating system installs a data descriptor whose base is 00200000h, whose limit covers 64 KiB, and whose privilege policy permits an application at ring 3 to read and write it. A selector loaded into DS identifies that descriptor. An instruction accessing offset 0100h reaches linear address 00200100h after the processor checks the selector and limit. An attempt at offset 10000h is outside a 64 KiB segment and faults rather than continuing into the next region. If the descriptor is marked read-only, a write faults even when its numerical address would otherwise be valid.

This example is deliberately simpler than a complete operating system. The descriptor tables themselves are privileged data structures; selectors include a table indicator and requested privilege level; and code transfers can involve conforming rules, call gates, or interrupt gates. A process cannot protect itself merely by choosing a different selector. The processor compares privilege information and descriptor attributes. Equally, descriptors do not identify files or users. They are low-level memory and control-transfer objects on which an operating system can build higher-level policy.

The 286 distinguishes logical addresses, formed from selector and offset, from physical addresses obtained after descriptor translation. Unlike a 386, it has no paging stage between a process’s linear view and physical memory. That makes a useful generation comparison. Segmentation can give separate bounded regions and privilege checks, but paging makes fixed-size mappings, shared pages, copy-on-write strategies, and demand-paged virtual memory much more convenient. It is possible to design useful protected software without paging; it is inaccurate to assume every 286 protected-mode program offered the virtual-memory behavior familiar from later systems.

Switching environments in practice

Entering protected mode requires more than setting one control bit. System software prepares a Global Descriptor Table, loads its base and limit through the GDTR, establishes suitable descriptors, sets the PE bit in CR0, and performs a control transfer that reloads CS under protected-mode rules. It also reloads the other segment registers with valid selectors and establishes an interrupt descriptor table before relying on interrupts. A partial transition is dangerous: a real-mode interrupt vector is not a protected-mode gate, and old segment values are not automatically meaningful selectors.

The reset requirement for returning to real mode made mixed-mode designs awkward. The keyboard controller on many AT systems could assert a CPU reset, and software arranged a warm-reset path that restored necessary state. This technique belongs to platform engineering, not an elegant processor instruction. It could vary with firmware and clone hardware. A developer should not infer that any 286 board will behave like a particular AT, nor that reset-based switching preserves every peripheral state without careful code.

Some period software chose a less ambitious route: remain in real mode and use extended-memory services to copy data above 1 MiB, then copy it back before execution. Others used EMS boards or emulators that exposed banked 16 KiB pages through a window in the address map. XMS normally refers to a software interface for extended memory, whereas EMS refers to a bank-switching convention; both names identify ecosystems as much as raw RAM. They solved useful problems for DOS applications, but neither supplied the 286’s descriptor isolation to ordinary real-mode code.

DOS extenders, OS/2, and application choices

A DOS extender commonly provides a small real-mode loader and a protected-mode runtime. A compiler targets the extender’s API, and the application receives a larger protected address space while the extender marshals selected DOS calls. This can work well for games, CAD, databases, and technical programs written for that environment. It is not a transparent upgrade for arbitrary binaries: programs that install low-level interrupt handlers, rely on conventional-memory layouts, or access devices directly may require special handling. Extenders also differ in memory models and API contracts.

IBM and Microsoft’s early OS/2 effort shows another approach. OS/2 was designed to use 286 protected mode and offer a more structured multitasking environment, while its early versions also supplied a compatibility box for DOS applications. Later 32-bit versions changed the technical base. Its commercial history is not evidence that the 286 was incapable; rather, it shows that applications, drivers, pricing, and migration all determine adoption. A processor feature becomes broadly useful only when surrounding software makes it practical.

Rings deserve a final caution. DOS generally runs applications with extensive machine authority because it was not designed as a protected, multiuser system. A 286 operating system can reserve ring 0 for its kernel, but a driver with ring-0 privileges can still damage the system. Modern readers should not project contemporary threat models onto every 1980s PC. The mechanism established an important foundation, while robust isolation remained an operating-system engineering problem.

Sources and further reading