Gain insights into using GDB for debugging in Linux. Explore disassembly, memory investigation, and register updates. Learn about pointers, stack, and function parameters in x64 architecture.

gdb.tar.gz

C2-LA

C4-L2

C4-L3

C4-L4

C4-L5

C6-L3

C8-L1

C9-L3

C10-L2

C10-L3

C11-L2

C13-L1

test

GNU Debugger (GDB) is used for debugging C/C++ programs in LINUX/UNIX environments. It is a good tool to investigate what is happening inside a program, and how the contents inside the memory are changed with the execution of the program.

The main focus of the course is the disassembly of the program, where we'll use simple operations in C. With the help of GDB, we'll examine the contents of the registers and memory. We'll also learn how they are changed while executing basic operations. We’ll then explore the use of pointers, stack, and function parameters, and analyze how different registers update their values. With the help of disassembly output, we’ll learn how a simple C program can be reconstructed. The disassembly output is important for debugging and core dump analysis. We’ll use assembly language for x64 architecture.

By the end of this course, you should be able to debug the programs and memory contents at assembly level when a program executes or probe into the problem when a program crashes.

Debugging, Disassembly & Reversing in Linux for x64 Architecture

# Stack usage
In addition to storage for the return addresses of `CALL` instructions, a stack is used to **pass additional parameters to functions** and **store local variables**. The stack is also used to save and restore values held in registers when we want to preserve them during some computation or across function calls. For example, suppose we want to do multiplication, but we have other valuable data in registers `%RAX` and `%RDX`. The multiplication result will overwrite `%RAX` and `%RDX` values, and we temporarily put their values on the stack:

mov $10, %rax
mov $20, %rdx
...
...
...               # now we want to preserve %RAX and %RDX
push %rax
push %rdx
imul %rdx         # %RDX and %RAX contain the result of %RAX*%RDX
mov %rax, result
pop %rdx          # pop in reverse order
pop %rax          # stack is LIFO

# Register review

So far, we've encountered these general-purpose registers:

* `%RAX` and `%RBX`, which contain the function return value and the bottom of a multiplication result.
* `%RCX`, a loop counter.
* `%RDX`, which contains the higher part of a multiplication result (if it exceeds the maximum $64$-bit value).
* `%RIP`, or Instruction Pointer register, which points to the next instruction to be executed.
* `%RSP`, or Stack Pointer register, which points to the top of the stack.

Next, we’ll look at another important register on Linux platforms, `%RBP`, also called **Base Pointer register** or **Stack Frame Pointer register**.

Learn about frame pointers and local variables using stack.

Introduction to the Course

Memory, Registers, and Simple Arithmetic

Code Optimization

Number Representations

Pointers

Bytes, Words, Double, and Quad Words

Pointers to Memory

Logical Instructions and RIP

Reconstructing a Program with Pointers

Memory and Stacks

Frame Pointer and Local Variables

Function Parameters

More Instructions

Function Pointer Parameters

Summary of Code Disassembly Patterns

x64 Debugging, Disassembly & Reversing Exam

Stacks in Frame Pointers and Local Variables

Stack usage