Foundations of x86-64 Instruction Encoding - Volume 1

About this book

Foundations of x86-64 Instruction Encoding — Volume 1 is a practical technical reference for programmers and engineers who want to understand how x86-64 machine instructions are actually constructed at the byte level. Rather than presenting instruction encoding as a collection of isolated rules, the book develops a systematic method that can be reused when building assemblers, JIT compilers, disassemblers, binary analysis tools, virtual machines, and other low-level software.

The volume focuses on the fundamental structures behind classic x86-64 instruction encoding, including instruction forms, opcodes, prefixes, ModR/M, SIB, displacement fields, immediate values, operand-size rules, address-size behavior, register selection, and REX prefixes. The goal is not simply to show what the final machine code looks like, but to explain how each field is derived and why it appears in a specific position within the instruction stream.

A central theme of the book is a repeatable implementation workflow:

select a legal instruction form → resolve operand and encoding fields → emit bytes in architectural order → independently verify the result.

This approach makes the material especially useful for developers who intend to implement their own encoder rather than merely read machine code produced by existing assemblers.

Through carefully structured explanations and encoding examples, readers learn how to manually translate common x86-64 instructions into their binary representations and how to convert that knowledge into a compact software architecture for an assembler or encoder. Particular attention is given to the legacy and REX encoding model, which remains the essential foundation for understanding later encoding systems such as VEX, EVEX, AVX, AVX2, AVX-512, and newer extensions.

The book also emphasizes verification. Correct instruction encoding should never depend solely on visual inspection or assumptions. Encoded bytes should be checked independently using trusted assemblers, disassemblers, or dedicated encoding libraries. This discipline is important because x86-64 contains many overlapping instruction forms, special register cases, addressing rules, and historical compatibility constraints.

The reference basis for this volume includes the 2026 revisions of the Intel 64 and IA-32 Software Developer’s Manual, the AMD64 Architecture Programmer’s Manual Volumes 1–3, NASM 3.02 documentation, and the Intel XED encoder/decoder reference.

By the end of Volume 1, the reader should be able to hand-encode a substantial set of common x86-64 instructions, understand the binary structure of their operands and addressing modes, and implement a small but reliable legacy/REX instruction encoder core.

This volume therefore serves both as an educational introduction to x86-64 machine-code construction and as the architectural foundation for more advanced assembler and binary-tool development.