1. 前言

限于作者能力水平，本文可能存在谬误，因此而给读者带来的损失，作者不做任何承诺。

2. 原理

Linux 内核划出 32TB 虚拟地址区间作为分配内存的 shadow 区域，以实现 KASAN 功能。

KASAN 基本原理，首先是通过 1/8 虚拟内存（上图中的 32TB kasan 区），也称 shadow memory，对系统虚拟地址空间进行标记：每连续的 8 字节，对应一个 shadow memory 字节，如果该 shadow memory 字节值为 0，则表示连续的 8 字节可以访问，shadow memory 字节值 1~7 则表示有 1~7 字节可以访问，如下图所示（图片借自博客 https://blog.csdn.net/gy794627991/article/details/132139699）：

然后通过编译器选项 -fsanitize=kernel-address（这对编译器的版本有要求），指示编译器在所有读写访问处插入 __asan_loadN 或 __asan_storeN 调用，然后内核实现这些接口，检查对应读写地址的 shadow memory 字节，看是否可以访问，不能访问则报 BUG。

值得一提的是，编译器插入 __asan_{load,store}N 是基于默认的 CONFIG_KASAN_OUTLINE=y 配置，Linux 内核提供 CONFIG_KASAN_INLINE=y 配置项，此时编译器不会插入 __asan_{load,store}N 调用，而是直接插入内存 shadow 标记检测代码。本文讨论的是基于默认的 CONFIG_KASAN_OUTLINE=y 配置的情形。

3. 实现

本文以 ARM64 + Linux 4.14.111 分析 Linux 内核的 KASAN 功能。通过配置项 CONFIG_KASAN=y 开启 Linux 内核的 KASAN 功能。

ARM64 架构下，对 KASAN 的具体实现，本文主要讨论 2 个核心要点：

• KASAN shadow 内存的分配
• KASAN 对内存分配时标记和访问时检测

3.1 KASAN shadow 内存的分配

在 Linux 内核启动阶段，首先通过 kasan_early_init() 将整个 KASAN shadow 内存虚拟地址区间 [KASAN_SHADOW_START,KASAN_SHADOW_END] 映射到同一页面 kasan_zero_page[PAGE_SIZE]，然后在 kasan_init() 为支持 KASAN 功能虚拟地址区间的分配 shadow 物理内存：

void __init kasan_init(void)
{
	u64 kimg_shadow_start, kimg_shadow_end;
	u64 mod_shadow_start, mod_shadow_end;
	struct memblock_region *reg;
	int i;

	/* 计算 内核镜像区间 的 kasan shadow memory 的范围 */
	kimg_shadow_start = (u64)kasan_mem_to_shadow(_text);
	kimg_shadow_end = (u64)kasan_mem_to_shadow(_end);

	/* 计算 module 区间 的 kasan shadow memory 的范围 */
	mod_shadow_start = (u64)kasan_mem_to_shadow((void *)MODULES_VADDR);
	mod_shadow_end = (u64)kasan_mem_to_shadow((void *)MODULES_END);

	/*
	 * We are going to perform proper setup of shadow memory.
	 * At first we should unmap early shadow (clear_pgds() call bellow).
	 * However, instrumented code couldn't execute without shadow memory.
	 * tmp_pg_dir used to keep early shadow mapped until full shadow
	 * setup will be finished.
	 */
	/*
	 * 接下来要配置 内核 PGD 页表 swapper_pg_dir[] 中的 kasan shadow memory 
	 * 部分, 先切换到临时 PGD 页表 tmp_pg_dir[], 然后再对 swapper_pg_dir[] 
	 * 进行配置, 配置完成后再切回到 swapper_pg_dir[].
	 */
	memcpy(tmp_pg_dir, swapper_pg_dir, sizeof(tmp_pg_dir));
	dsb(ishst);
	cpu_replace_ttbr1(lm_alias(tmp_pg_dir));

	/* 清除 PGD 页表 swapper_pg_dir[] 中 kasan shadow memory 的页表项 */
	clear_pgds(KASAN_SHADOW_START, KASAN_SHADOW_END);

	/*
	 * 建立 内核镜像区间 的 kasan memory 区间的 PGD,PUD,PMD 这 3 级页表映射,
	 * 并为 kasan memory 区间分配 物理内存.
	 */
	vmemmap_populate(kimg_shadow_start, kimg_shadow_end,
			 pfn_to_nid(virt_to_pfn(lm_alias(_text))));

	/*
	 * vmemmap_populate() has populated the shadow region that covers the
	 * kernel image with SWAPPER_BLOCK_SIZE mappings, so we have to round
	 * the start and end addresses to SWAPPER_BLOCK_SIZE as well, to prevent
	 * kasan_populate_zero_shadow() from replacing the page table entries
	 * (PMD or PTE) at the edges of the shadow region for the kernel
	 * image.
	 */
	kimg_shadow_start = round_down(kimg_shadow_start, SWAPPER_BLOCK_SIZE);
	kimg_shadow_end = round_up(kimg_shadow_end, SWAPPER_BLOCK_SIZE);

	/*
	 * 映射 多个 不使用 kasan 功能的 kasan shadow 子区间 到 kasan_zero_page[]:
	 * 即 这些子区间 不使用 kasan 功能.
	 */
	kasan_populate_zero_shadow((void *)KASAN_SHADOW_START,
				   (void *)mod_shadow_start); /* 整个 kasan shadow 内存区间自身禁用 kasan 功能 */
	kasan_populate_zero_shadow((void *)kimg_shadow_end, /* [fixmap + PCI I/O, vmemmap] 区间 禁用 kasan 功能 */
				   kasan_mem_to_shadow((void *)PAGE_OFFSET));

	if (kimg_shadow_start > mod_shadow_end)
		kasan_populate_zero_shadow((void *)mod_shadow_end,
					   (void *)kimg_shadow_start);

	/* 建立 RAM 线性映射区间的 kasan shadow 子区间 页表映射, 并分配物理内存 */
	for_each_memblock(memory, reg) {
		void *start = (void *)__phys_to_virt(reg->base);
		void *end = (void *)__phys_to_virt(reg->base + reg->size);

		if (start >= end)
			break;

		vmemmap_populate((unsigned long)kasan_mem_to_shadow(start),
				(unsigned long)kasan_mem_to_shadow(end),
				pfn_to_nid(virt_to_pfn(start)));
	}

	/*
	 * KAsan may reuse the contents of kasan_zero_pte directly, so we
	 * should make sure that it maps the zero page read-only.
	 */
	/* 所有的 kasan 目前都映射到 kasan_zero_page[] 页面, 且以 只读 方式映射 */
	for (i = 0; i < PTRS_PER_PTE; i++)
		set_pte(&kasan_zero_pte[i],
			pfn_pte(sym_to_pfn(kasan_zero_page), PAGE_KERNEL_RO));

	memset(kasan_zero_page, 0, PAGE_SIZE);
	/*
	 * 已经配置好了 swapper_pg_dir[] 内核页表 的 kasan shadow 区间
	 * 映射(全部映射到了 kasan_zero_page[] 只读页面), 切换回 swapper_pg_dir[] 
	 * 内核页表.
	 */
	cpu_replace_ttbr1(lm_alias(swapper_pg_dir));

	/* At this point kasan is fully initialized. Enable error messages */
	init_task.kasan_depth = 0;
	pr_info("KernelAddressSanitizer initialized\n");
}

3.2 KASAN 对内存分配时标记和访问时检测

KASAN 支持对以下几种内存进行标记和访问检测：

• 全局变量
• buddy 页面
• slab 空间
• 栈空间

3.2.1 全局变量的标记和检测

3.2.1.1 全局变量的标记

Linux 内核开启 KASAN 功能后，编译器会为全局变量生成一段初始化代码，类似如下的片段：

可以看到这段代码调用了 __asan_register_globals() 函数，然后在下面路径中被调用：

kernel_init_freeable()
	do_basic_setup()
		do_ctors()
			_sub_I_65535_1()
				__asan_register_globals()

那么 __asan_register_globals() 函数做了些什么工作呢？来看一下：

/* The layout of struct dictated by compiler */
struct kasan_global {
	/* 全局变量的起始地址 */
	const void *beg;		/* Address of the beginning of the global variable. */
	/* 全局变量占据空间大小 */
	size_t size;			/* Size of the global variable. */
	/* 全局变量的大小 + red zone 大小, 然后对齐到 32 字节 */
	size_t size_with_redzone;	/* Size of the variable + size of the red zone. 32 bytes aligned */
	const void *name;
	const void *module_name;	/* Name of the module where the global variable is declared. */
	unsigned long has_dynamic_init;	/* This needed for C++ */
#if KASAN_ABI_VERSION >= 4
	struct kasan_source_location *location;
#endif
#if KASAN_ABI_VERSION >= 5
	char *odr_indicator;
#endif
};

/* @globals 由编译器构建 */
void __asan_register_globals(struct kasan_global *globals, size_t size)
{
	int i;

	for (i = 0; i < size; i++)
		register_global(&globals[i]);
}
EXPORT_SYMBOL(__asan_register_globals);

#define KASAN_GLOBAL_REDZONE    0xFA  /* redzone for global variable */

/* 初始化 全局变量自身 和 其 redzone 的 kasan shadow */
static void register_global(struct kasan_global *global)
{
	size_t aligned_size = round_up(global->size, KASAN_SHADOW_SCALE_SIZE);

	/* 全局变量自身空间 的 shadow 区填充 */
	kasan_unpoison_shadow(global->beg, global->size);

	/* 全局变量 redzone 的 shadow 区填充为 0xFA */
	kasan_poison_shadow(global->beg + aligned_size,
		global->size_with_redzone - aligned_size,
		KASAN_GLOBAL_REDZONE);
}

/*
 * - size 对齐到 8 字节部分的 kasan shadow 清 0
 * - 如果 size 有未对齐到 8 字节的部分, 则剩余部分字节数
 *   写入对应的 kasan shadow
 */
void kasan_unpoison_shadow(const void *address, size_t size)
{
	kasan_poison_shadow(address, size, 0);

	if (size & KASAN_SHADOW_MASK) { /* 未对齐到 8 字节的剩余字节 */
		u8 *shadow = (u8 *)kasan_mem_to_shadow(address + size);
		*shadow = size & KASAN_SHADOW_MASK; /* shadow 写入未对齐的字节数 */
	}
}

/*
 * Poisons the shadow memory for 'size' bytes starting from 'addr'.
 * Memory addresses should be aligned to KASAN_SHADOW_SCALE_SIZE.
 */
/* 将 [address, address+size] 对应的 kasan shadow 设为 value */
static void kasan_poison_shadow(const void *address, size_t size, u8 value)
{
	void *shadow_start, *shadow_end;

	shadow_start = kasan_mem_to_shadow(address);
	shadow_end = kasan_mem_to_shadow(address + size);

	memset(shadow_start, value, shadow_end - shadow_start);
}

kasan_mem_to_shadow() 映射被访问虚拟地址 addr 到其对应的 shadow 内存虚拟地址：

#define KASAN_SHADOW_SCALE_SHIFT 3

/*
 * KASAN_SHADOW_START: beginning of the kernel virtual addresses.
 * KASAN_SHADOW_END: KASAN_SHADOW_START + 1/8 of kernel virtual addresses.
 */
#define KASAN_SHADOW_START      (VA_START)
#define KASAN_SHADOW_END        (KASAN_SHADOW_START + KASAN_SHADOW_SIZE)

/*
 * This value is used to map an address to the corresponding shadow
 * address by the following formula:
 *     shadow_addr = (address >> 3) + KASAN_SHADOW_OFFSET;
 *
 * (1 << 61) shadow addresses - [KASAN_SHADOW_OFFSET,KASAN_SHADOW_END]
 * cover all 64-bits of virtual addresses. So KASAN_SHADOW_OFFSET
 * should satisfy the following equation:
 *      KASAN_SHADOW_OFFSET = KASAN_SHADOW_END - (1ULL << 61)
 */
#define KASAN_SHADOW_OFFSET     (KASAN_SHADOW_END - (1ULL << (64 - 3)))

/* 返回 @addr 对应的 kasan shadow address */
static inline void *kasan_mem_to_shadow(const void *addr)
{
	return (void *)((unsigned long)addr >> KASAN_SHADOW_SCALE_SHIFT)
		+ KASAN_SHADOW_OFFSET;
}

小结一下，对全局变量的标记工作包括：

• 全局变量自身空间，对齐到 8 字节的部分，对应的 shadow 字节空间均标记为 0，未对齐到 8 字节的部分的对应 shadow 字节填充为未对齐到 8 字节的字节数；
• 编译器为全局变量尾部扩展的 redzone 对应的 shadow 字节填充为 0xFA (KASAN_GLOBAL_REDZONE)。

3.2.1.1 全局变量的访问检测

用链接 linux kernel 内存踩踏之KASAN（一） 的代码进行全局变量 OOB 问题测试：

// kasan_test.c

#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/init.h>
#include <linux/miscdevice.h>
#include <linux/workqueue.h>
#include <linux/jiffies.h>
#include <asm/page.h>
#include <linux/slab.h>
#include <linux/mm.h>
#include <linux/uaccess.h>
#include <linux/vmalloc.h>


#include "../../mm/kasan/kasan.h"

int global_kasan_value[2] = {996, 007};

struct kasan_test_type {
    int type;
};

static struct kasan_test_type *gptr = NULL;

enum kasan_test_case {
    slab_out_of_bounds = 0,
    page_out_of_bounds = 1,
    global_out_of_bounds = 2,
    stack_out_of_bounds = 3,
    use_after_free = 4,
    vmalloc_out_of_bounds = 5,
    alloca_out_of_bounds = 6,
    slab_use_after_free = 7,
    slab_repeate_free = 8,
};

static void kmalloc_oob_right(size_t size, int write_offset)
{
    char *ptr;

    ptr = kmalloc(size, GFP_KERNEL);
    pr_info("%s %llx\n", __func__, (unsigned long long)ptr);

    ptr[size - 1 + write_offset] = 'y';

    kfree(ptr);
}

static void kmalloc_use_after_free(size_t size)
{
    char *ptr;

    ptr = kmalloc(size, GFP_KERNEL);
    pr_info("%s %llx\n", __func__, (unsigned long long)ptr);

    ptr[0] = 'x';

    kfree(ptr);
    
    ptr[0] = 'y';
}

static void kmalloc_repeate_free(size_t size)
{
    char *ptr;

    ptr = kmalloc(size, GFP_KERNEL);
    pr_info("%s %llx\n", __func__, (unsigned long long)ptr);

    ptr[0] = 'x';

    kfree(ptr);
    kfree(ptr);
}

static void global_oob_left(void)
{
	pr_info("global arr oob access %d\n", global_kasan_value[2]);
}

static void pagealloc_oob_right(size_t order)
{
    char *ptr;
    struct page *pages;

    size_t size = (1UL << (PAGE_SHIFT + order));

    pages = alloc_pages(GFP_KERNEL, order);
    ptr = page_address(pages);
    pr_info("%s %llx\n", __func__, (unsigned long long)ptr);
    ptr[0] = ptr[size];
    free_pages((unsigned long)ptr, order);
}

static void pagealloc_uaf(size_t order)
{
	char *ptr;
	struct page *pages;

	pages = alloc_pages(GFP_KERNEL, order);
	ptr = page_address(pages);
	pr_info("%s %llx", __func__, (unsigned long long)ptr);
	free_pages((unsigned long)ptr, order);

	pr_info("%s %d\n", __func__, ptr[0]);
}

static void vmalloc_oob(size_t size)
{
    char *v_ptr;

    v_ptr = vmalloc(size);

    OPTIMIZER_HIDE_VAR(v_ptr);

    pr_info("%s %llx", __func__, (unsigned long long)v_ptr);
    /* Make sure in-bounds accesses are valid. */
    v_ptr[0] = 0;
    v_ptr[size - 1] = 0;

    /* trigger oob access */
    pr_info("%s %d\n", __func__, v_ptr[size]);

    vfree(v_ptr);
}

static void kasan_stack_oob(void)
{
	char stack_array[10];
	/* See comment in kasan_global_oob_right. */
	char *volatile array = stack_array;
	char *p = &array[ARRAY_SIZE(stack_array) + 4];
	pr_info("%s  %d\n", __func__, *p);
}

static void kasan_test_case(int type)
{
    bool multishot = kasan_save_enable_multi_shot();

    switch (type) {
        case slab_out_of_bounds:
            kmalloc_oob_right(128, 2); //alloc 128 byte and overwrite 2 offset
            break;

        case page_out_of_bounds:
            pagealloc_oob_right(0);
            break;

        case global_out_of_bounds:
            global_oob_left();
            break;

        case stack_out_of_bounds:
            kasan_stack_oob();
            break;

        case use_after_free:
            pagealloc_uaf(0);
            break;

        case vmalloc_out_of_bounds:
            vmalloc_oob(2048);
            break;

	case slab_use_after_free:
		kmalloc_use_after_free(18);
		break;

	case slab_repeate_free:
		kmalloc_repeate_free(88);
		break;

        default :
            pr_info("undef error type %d\n", type);
            break;
    }

    kasan_restore_multi_shot(multishot);
    pr_info("%s type %d\n", __func__, type);
}

static ssize_t kasan_testcase_write(struct file *filp, const char __user *buf,
                   size_t len, loff_t *off)
{
    char *kbuf;
    int ntcase;

    kbuf = kmalloc(len + 1, GFP_KERNEL);
    if (copy_from_user(kbuf, buf, len) != 0) {
        pr_info("copy the buff failed \n");
        goto done;
    }

    ntcase = simple_strtoul(kbuf, NULL, 0);

    kasan_test_case(ntcase);
done:
    return len;
}

static struct file_operations kasan_fops = {
    .owner  =   THIS_MODULE,
    .write  =   kasan_testcase_write,
    .llseek =   noop_llseek,
};

static struct miscdevice kasan_misc = {
    .minor  = MISC_DYNAMIC_MINOR,
    .name   = "kasan_test",
    .fops   = &kasan_fops,
};

static int __init kasan_start(void) 
{
    int ret;

    ret = misc_register(&kasan_misc);
    if (ret < 0) {
        printk(KERN_EMERG " kasan test register failed %d\n", ret);
        return ret;
    }
    gptr = kzalloc(sizeof(struct kasan_test_type), GFP_KERNEL);

    printk(KERN_INFO "kasan test register\n");
    return 0;
}

static void __exit kasan_end(void) 
{ 
    misc_deregister(&kasan_misc);
} 

MODULE_LICENSE("GPL");
MODULE_AUTHOR("geek");
MODULE_DESCRIPTION("A simple kasan test driver!");
MODULE_VERSION("0.1");
 
module_init(kasan_start);
module_exit(kasan_end);

编译运行，测试触发下面代码段的运行：

int global_kasan_value[2] = {996, 007};

static void global_oob_left(void)
{
	pr_info("global arr oob access %d\n", global_kasan_value[2]);
}

报错日志：

# insmod kasan_test.ko
root@qemu-ubuntu:~# echo 2 > /dev/kasan_test
[   83.940491] ==================================================================
[   83.942045] BUG: KASAN: global-out-of-bounds in kasan_testcase_write+0x37c/0x3c0 [kasan_test]
[   83.942486] Read of size 4 at addr ffff200000ea2008 by task bash/1388
[   83.942673] 
[   83.944585] 
[   83.944782] The buggy address belongs to the variable:
[   83.944944]  global_kasan_value+0x8/0xfffffffffffff000 [kasan_test]
[   83.945163] 
[   83.945284] Memory state around the buggy address:
[   83.945753]  ffff200000ea1f00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
[   83.945954]  ffff200000ea1f80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
[   83.946154] >ffff200000ea2000: 00 fa fa fa fa fa fa fa 00 00 00 00 00 00 00 00
[   83.946341]                       ^
[   83.946427]  ffff200000ea2080: 00 00 fa fa fa fa fa fa 00 00 00 00 00 00 00 00
[   83.946556]  ffff200000ea2100: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
[   83.946701] ==================================================================

看到了吧？访问到全局变量的 OOB 问题被爆出来了，怎么做到的？反汇编 kasan_test.ko，可以看到编译器在写 global_kasan_value 的代码前插入了 __asan_store1_noabort() 调用：

正是 __asan_store1_noabort() 对访问地址的 shadow 字节进行检测：

void __asan_store1(unsigned long addr)
{
	check_memory_region_inline(addr, 1, true, _RET_IP_);
}
__alias(__asan_store1) void __asan_store1_noabort(unsigned long); // __asan_store1() 的别名

static __always_inline void check_memory_region_inline(unsigned long addr,
						size_t size, bool write,
						unsigned long ret_ip)
{
	if (unlikely(size == 0))
		return;

	if (unlikely((void *)addr <
		kasan_shadow_to_mem((void *)KASAN_SHADOW_START))) { /* 非法地址 */
		kasan_report(addr, size, write, ret_ip); /* 报告错误 */
		return;
	}

	if (likely(!memory_is_poisoned(addr, size)))
		return; /* 可以正常访问地址 addr */

	/* 报告错误 */
	kasan_report(addr, size, write, ret_ip);
}

static __always_inline bool memory_is_poisoned(unsigned long addr, size_t size)
{
	if (__builtin_constant_p(size)) {
		switch (size) {
		case 1:
			return memory_is_poisoned_1(addr);
		case 2:
		case 4:
		case 8:
			return memory_is_poisoned_2_4_8(addr, size);
		case 16:
			return memory_is_poisoned_16(addr);
		default:
			BUILD_BUG();
		}
	}

	return memory_is_poisoned_n(addr, size);
}

static __always_inline bool memory_is_poisoned_1(unsigned long addr)
{
	s8 shadow_value = *(s8 *)kasan_mem_to_shadow((void *)addr); /* 读取 @addr 对应的 kasan shadow 字节内容 */

	/*
	 * shadow_value 可能是:
	 * - 某块内存末尾未对齐到 8 字节, shadow_value 存储未对齐到 8 的字节数
	 * - poisoned 字节: KASAN_FREE_PAGE, ...
	 */
	if (unlikely(shadow_value)) {
		/* 试图访问 @addr 未对齐到 8 部分, 在 8 字节内的偏移位置 */
		s8 last_accessible_byte = addr & KASAN_SHADOW_MASK;
		/*
		 * last_accessible_byte >= shadow_value 成立的两种可能:
		 * - @addr 想访问 8 字节内的 第 last_accessible_byte 个字节 
		 *   超出了 可访问字节 最大位置, 表示出现了 OOB BUG.
		 * - shadow_value 存储的是 poisoned 字节, 如 KASAN_GLOBAL_REDZONE
		 *   等, 这些系统定义的 poisoned 字节全都是小于 0 的, 因为
		 *   它们的最高位都为 1, 对于 s8 类型则为小于 0 的数.
		 */
		return unlikely(last_accessible_byte >= shadow_value);
	}

	return false; /* shadow_value == 0 表示从 @addr 连续的 8 字节均可访问 */
}

这里只贴出了 1 字节访问的检测，对于 2，4，8，16, 以及更多字节的写访问检测，读取可自行阅读源码中的 memory_is_poisoned_2_4_8(), memory_is_poisoned_16(), memory_is_poisoned_n() 函数，自行分析。

这里的例子是写访问检测，读访问检测的类似，编译器在读访问之前插入 __asan_loadN_noabort() 调用，以 1 字节的读访问为例，来看一下：

void __asan_load1(unsigned long addr)
{
	check_memory_region_inline(addr, 1, false, _RET_IP_);
}
__alias(__asan_load1) void __asan_load1_noabort(unsigned long); // __asan_load1() 的别名

check_memory_region_inline() 已经分析过了，就不再赘述。另外，kasan_report() 的内容本文不做展开，读者可自行阅读相关源码。

3.2.2 buddy 页面的标记和检测

3.2.2.1 buddy 页面的标记

首先，BOOT 期间在内存管理系统初始化过程中，释放内存页面给 buddy 时，调用 kasan_free_pages() 将对应页面的 shadow 内存字节填充为 0xFF，表示所有这些释放页面空间不可访问（未从 buddy 分配自然不可用）：

#define KASAN_FREE_PAGE         0xFF  /* page was freed */

void kasan_free_pages(struct page *page, unsigned int order)
{
	if (likely(!PageHighMem(page)))
		/* 已释放 page 的 kasan shadow 标记为 0xFF */
		kasan_poison_shadow(page_address(page),
				PAGE_SIZE << order,
				KASAN_FREE_PAGE);
}

然后在分配页面是调用 kasan_alloc_pages() 将页面的 shadow 内存字节清 0，表示对应页面空间可用：

prep_new_page()
	post_alloc_hook()
		kasan_alloc_pages()

void kasan_alloc_pages(struct page *page, unsigned int order)
{
	if (likely(!PageHighMem(page)))
		/*
		 * - size 对齐到 8 字节部分的 kasan shadow 清 0
		 * - 如果 size 有未对齐到 8 字节的部分, 则剩余部分字节数
		 *   写入对应的 kasan shadow
		 */
		kasan_unpoison_shadow(page_address(page), PAGE_SIZE << order);
}

释放页面时同样调用 kasan_free_pages()，标记释放的页面空间不再可用。

3.2.2.1 buddy 页面的访问检测

这和 2.2.1.1 全局变量的访问检测 并无不同，都是编译器在页面空间读写访问前，插入 __asan_{load,store}N_abort 调用，在此不再赘述。

这里给出 buddy 页面访问 OOB 报错的例子信息，对应的出错代码段如下：

static void pagealloc_oob_right(size_t order)
{
    char *ptr;
    struct page *pages;

    size_t size = (1UL << (PAGE_SHIFT + order));

    pages = alloc_pages(GFP_KERNEL, order);
    ptr = page_address(pages);
    pr_info("%s %llx\n", __func__, (unsigned long long)ptr);
    ptr[0] = ptr[size];
    free_pages((unsigned long)ptr, order);
}

报错日志：

# insmod kasan_test.ko
# echo 1 > /dev/kasan_test
[   51.847460] ==================================================================
[   51.848464] BUG: KASAN: use-after-free in kasan_testcase_write+0x200/0x3c0 [kasan_test]
[   51.848591] Read of size 1 at addr ffff800031eb3000 by task bash/1386
[   51.848681] 
[   51.849727] 
[   51.849839] Allocated by task 1334:
[   51.850066] 
[   51.850108] Freed by task 1335:
[   51.850230] 
[   51.850276] The buggy address belongs to the object at ffff800031eb3000
[   51.850276]  which belongs to the cache cred_jar(79:systemd-random-seed.service) of size 168
[   51.850476] The buggy address is located 0 bytes inside of
[   51.850476]  168-byte region [ffff800031eb3000, ffff800031eb30a8)
[   51.850661] The buggy address belongs to the page:
[   51.850833] page:ffff7e0000c7acc0 count:1 mapcount:0 mapping:          (null) index:0xffff800031eb3300
[   51.851036] flags: 0xfffc00000000100(slab)
[   51.851298] raw: 0fffc00000000100 0000000000000000 ffff800031eb3300 000000018010000e
[   51.851423] raw: dead000000000100 dead000000000200 ffff80003138f000 ffff8000347acd80
[   51.851540] page dumped because: kasan: bad access detected
[   51.851621] page->mem_cgroup:ffff8000347acd80
[   51.851752] 
[   51.851810] Memory state around the buggy address:
[   51.851999]  ffff800031eb2f00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
[   51.852114]  ffff800031eb2f80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
[   51.852208] >ffff800031eb3000: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
[   51.852329]                    ^
[   51.852388]  ffff800031eb3080: fb fb fb fb fb fc fc fc fc fc fc fc fc fc fc fc
[   51.852480]  ffff800031eb3100: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
[   51.852582] ==================================================================

你可能会奇怪，这里不应该是页面访问的 OOB 错误吗？怎么爆出了 UAF？原因是越界的地址是紧跟页面之后第一个字节的地址 ptr[0] = ptr[size];，其对应的 shadow 不对应页面本身的 shadow 区域，从而产生了误报。可见，KASAN 存在一定的缺陷。

3.2.3 slab 空间的标记和检测

3.2.3.1 slab 空间的标记

3.2.3.1.1 分配时的标记

前面有提到过，KASAN 会将 buddy 分配器分配的页面会标记为可用状态，所以，首先，slab 从 buddy 获取页面后，KASAN 会先将获取的页面标记为不可用状态，因为 slab 是按 object 进行分配的：

kmalloc()
	...
	slab_alloc_node()
		...
		allocate_slab()
			...
			page = alloc_slab_page(s, alloc_gfp, node, oo);
			...
			kasan_poison_slab(page);
			...

/* slab 页面空间 的 kasan shadow 填充为 KASAN_KMALLOC_REDZONE (0xFC) */
void kasan_poison_slab(struct page *page)
{
	kasan_poison_shadow(page_address(page),
			PAGE_SIZE << compound_order(page),
			KASAN_KMALLOC_REDZONE);
}

然后，在分配 object 的时候，KASAN 将 object 对应空间标记为可用，同时标记 object 的 redzone 空间，这可以放置越界访问(OOB)：

kmalloc()
	...
	slab_alloc_node()
		...
		slab_post_alloc_hook()

static inline void slab_post_alloc_hook(struct kmem_cache *s, gfp_t flags,
					size_t size, void **p)
{
	size_t i;

	flags &= gfp_allowed_mask;
	for (i = 0; i < size; i++) {
		void *object = p[i];

		...
		kasan_slab_alloc(s, object, flags);
	}

	...
}

void kasan_slab_alloc(struct kmem_cache *cache, void *object, gfp_t flags)
{
	kasan_kmalloc(cache, object, cache->object_size, flags);
}

#define KASAN_KMALLOC_REDZONE   0xFC  /* redzone inside slub object */

void kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size,
		   gfp_t flags)
{
	unsigned long redzone_start;
	unsigned long redzone_end;

	...

	if (unlikely(object == NULL))
		return;

	redzone_start = round_up((unsigned long)(object + size),
				KASAN_SHADOW_SCALE_SIZE);
	redzone_end = round_up((unsigned long)object + cache->object_size,
				KASAN_SHADOW_SCALE_SIZE);

	kasan_unpoison_shadow(object, size); /* kasan 标记 size 大小的 object 可用 */
	kasan_poison_shadow((void *)redzone_start, redzone_end - redzone_start,
		KASAN_KMALLOC_REDZONE); /* kasan 标记 redzone 空间: 读写都会导致 kasan BUG report */

	if (cache->flags & SLAB_KASAN)
		set_track(&get_alloc_info(cache, object)->alloc_track, flags);
}
EXPORT_SYMBOL(kasan_kmalloc);

slab 分配时大空间时，还有一种路径走 kmalloc_large()，KASAN 标记过程会有些不同，本文对此不作展开，读者可自行研究。

3.2.3.1.2 释放时的标记

释放时 KASAN 的标记动作，可以防止 UAF 和重复释放(double free)的问题：

kfree()
	slab_free()
		slab_free_freelist_hook()
			slab_free_hook()
				kasan_slab_free()

bool kasan_slab_free(struct kmem_cache *cache, void *object)
{
	s8 shadow_byte;

	/* RCU slabs could be legally used after free within the RCU period */
	if (unlikely(cache->flags & SLAB_TYPESAFE_BY_RCU))
		return false;

	shadow_byte = READ_ONCE(*(s8 *)kasan_mem_to_shadow(object));
	if (shadow_byte < 0 || shadow_byte >= KASAN_SHADOW_SCALE_SIZE) {
		kasan_report_double_free(cache, object,
				__builtin_return_address(1));
		return true;
	}

	kasan_poison_slab_free(cache, object);

	if (unlikely(!(cache->flags & SLAB_KASAN)))
		return false;

	set_track(&get_alloc_info(cache, object)->free_track, GFP_NOWAIT);
	quarantine_put(get_free_info(cache, object), cache);
	return true;
}

#define KASAN_KMALLOC_FREE      0xFB  /* object was freed (kmem_cache_free/kfree) */

static void kasan_poison_slab_free(struct kmem_cache *cache, void *object)
{
	unsigned long size = cache->object_size;
	unsigned long rounded_up_size = round_up(size, KASAN_SHADOW_SCALE_SIZE);

	/* RCU slabs could be legally used after free within the RCU period */
	if (unlikely(cache->flags & SLAB_TYPESAFE_BY_RCU))
		return;

	kasan_poison_shadow(object, rounded_up_size, KASAN_KMALLOC_FREE);
}

3.2.3.2 slab 空间的访问检测

这和 2.2.1.1 全局变量的访问检测 并无多大不同，都是编译器在页面空间读写访问前，插入 __asan_{load,store}N_abort 调用，在此不再赘述。

分别看下 slab OOB，UAF，重复释放(double free)这 3 种情形的示例。

首先看 OOB，触发代码：

static void kmalloc_oob_right(size_t size, int write_offset)
{
    char *ptr;

    ptr = kmalloc(size, GFP_KERNEL);
    pr_info("%s %llx\n", __func__, (unsigned long long)ptr);

    ptr[size - 1 + write_offset] = 'y';

    kfree(ptr);
}

报错日志：

# echo 0 > /dev/kasan_test
[  182.350903] ==================================================================
[  182.351236] BUG: KASAN: slab-out-of-bounds in kasan_testcase_write+0x2ac/0x4c8 [kasan_test]
[  182.351446] Write of size 1 at addr ffff8000325aa881 by task bash/1386
[  182.351594] 
[  182.352740] 
[  182.353024] Allocated by task 1386:
[  182.353580] 
[  182.353834] Freed by task 0:
[  182.354568] 
[  182.354763] The buggy address belongs to the object at ffff8000325aa800
[  182.354763]  which belongs to the cache kmalloc-128 of size 128
[  182.355252] The buggy address is located 1 bytes to the right of
[  182.355252]  128-byte region [ffff8000325aa800, ffff8000325aa880)
[  182.355509] The buggy address belongs to the page:
[  182.355652] page:ffff7e0000c96a80 count:1 mapcount:0 mapping:          (null) index:0x0
[  182.355848] flags: 0xfffc00000000100(slab)
[  182.355983] raw: 0fffc00000000100 0000000000000000 0000000000000000 0000000180100010
[  182.356158] raw: ffff7e0000cdcd00 0000000d00000002 ffff800035003c00 0000000000000000
[  182.356310] page dumped because: kasan: bad access detected
[  182.356425] 
[  182.356475] Memory state around the buggy address:
[  182.356589]  ffff8000325aa780: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
[  182.356737]  ffff8000325aa800: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
[  182.356879] >ffff8000325aa880: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
[  182.357011]                    ^
[  182.357091]  ffff8000325aa900: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
[  182.357224]  ffff8000325aa980: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
[  182.357347] ==================================================================

接着看 UAF，触发代码：

static void kmalloc_use_after_free(size_t size)
{
    char *ptr;

    ptr = kmalloc(size, GFP_KERNEL);
    pr_info("%s %llx\n", __func__, (unsigned long long)ptr);

    ptr[0] = 'x';

    kfree(ptr);

    ptr[0] = 'y';
}

报错日志：

# echo 7 > /dev/kasan_test 
[   81.257559] ==================================================================
[   81.259832] BUG: KASAN: use-after-free in kasan_testcase_write+0x468/0x4c8 [kasan_test]
[   81.261441] Write of size 1 at addr ffff800034487c00 by task bash/1386
[   81.262126] 
[   81.263600] 
[   81.263954] Allocated by task 1386:
[   81.264686] 
[   81.265004] Freed by task 1386:
[   81.265367] 
[   81.265481] The buggy address belongs to the object at ffff800034487c00
[   81.265481]  which belongs to the cache kmalloc-128 of size 128
[   81.266581] The buggy address is located 0 bytes inside of
[   81.266581]  128-byte region [ffff800034487c00, ffff800034487c80)
[   81.267555] The buggy address belongs to the page:
[   81.268264] page:ffff7e0000d121c0 count:1 mapcount:0 mapping:          (null) index:0x0
[   81.268873] flags: 0xfffc00000000100(slab)
[   81.269416] raw: 0fffc00000000100 0000000000000000 0000000000000000 0000000180100010
[   81.270147] raw: ffff7e0000c63440 0000000b0000000b ffff800035003c00 0000000000000000
[   81.270856] page dumped because: kasan: bad access detected
[   81.271324] 
[   81.271563] Memory state around the buggy address:
[   81.272280]  ffff800034487b00: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
[   81.272640]  ffff800034487b80: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
[   81.273689] >ffff800034487c00: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
[   81.274098]                    ^
[   81.274166]  ffff800034487c80: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
[   81.274258]  ffff800034487d00: 03 fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
[   81.274362] ==================================================================

最后看重复释放(double free)，触发代码：

static void kmalloc_double_free(size_t size)
{
    char *ptr;

    ptr = kmalloc(size, GFP_KERNEL);
    pr_info("%s %llx\n", __func__, (unsigned long long)ptr);

    ptr[0] = 'x';

    kfree(ptr);
    kfree(ptr);
}

报错日志：

# echo 8 > /dev/kasan_test 
[  358.378681] ==================================================================
[  358.378947] BUG: KASAN: double-free or invalid-free in           (null)
[  358.379107] 
[  358.380123] 
[  358.380376] Allocated by task 1386:
[  358.380905] 
[  358.381138] Freed by task 1386:
[  358.381849] 
[  358.382091] The buggy address belongs to the object at ffff800033f34900
[  358.382091]  which belongs to the cache kmalloc-128 of size 128
[  358.382722] The buggy address is located 0 bytes inside of
[  358.382722]  128-byte region [ffff800033f34900, ffff800033f34980)
[  358.382856] The buggy address belongs to the page:
[  358.382962] page:ffff7e0000cfcd00 count:1 mapcount:0 mapping:          (null) index:0x0
[  358.383083] flags: 0xfffc00000000100(slab)
[  358.383159] raw: 0fffc00000000100 0000000000000000 0000000000000000 0000000180100010
[  358.383263] raw: dead000000000100 dead000000000200 ffff800035003c00 0000000000000000
[  358.383355] page dumped because: kasan: bad access detected
[  358.383444] 
[  358.383477] Memory state around the buggy address:
[  358.383551]  ffff800033f34800: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
[  358.383643]  ffff800033f34880: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
[  358.383734] >ffff800033f34900: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
[  358.383818]                    ^
[  358.383872]  ffff800033f34980: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
[  358.383961]  ffff800033f34a00: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
[  358.384049] ==================================================================

3.2.4 栈空间的标记和检测

编译器在栈变量前后插入 redzone，然后编译器再插入对这两个 redzone 的 KASAN 标记代码。如下图所示：

栈空间的 OOB 包括以下几种情形：

/*
 * Stack redzone shadow values
 * (Those are compiler's ABI, don't change them)
 */
#define KASAN_STACK_LEFT        0xF1
#define KASAN_STACK_MID         0xF2
#define KASAN_STACK_RIGHT       0xF3
#define KASAN_STACK_PARTIAL     0xF4
#define KASAN_USE_AFTER_SCOPE   0xF8

栈 OOB 触发代码：

static void kasan_stack_oob(void)
{
	char stack_array[10];
	/* See comment in kasan_global_oob_right. */
	char *volatile array = stack_array;
	char *p = &array[ARRAY_SIZE(stack_array) + 4];
	pr_info("%s  %d\n", __func__, *p);
}

报错日志：

# insmod kasan_test.ko
# echo 3 > /dev/kasan_test
[  556.754000] ==================================================================
[  556.755603] BUG: KASAN: stack-out-of-bounds in kasan_testcase_write+0x36c/0x4c8 [kasan_test]
[  556.756005] Read of size 1 at addr ffff800031bffc9e by task bash/1387
[  556.756163] 
[  556.758828] 
[  556.759118] The buggy address belongs to the page:
[  556.759879] page:ffff7e0000c6ffc0 count:0 mapcount:0 mapping:          (null) index:0x0
[  556.761106] flags: 0xfffc00000000000()
[  556.762048] raw: 0fffc00000000000 0000000000000000 0000000000000000 00000000ffffffff
[  556.762386] raw: 0000000000000000 ffff7e0000c6ffe0 0000000000000000 0000000000000000
[  556.762556] page dumped because: kasan: bad access detected
[  556.762682] 
[  556.762749] Memory state around the buggy address:
[  556.762994]  ffff800031bffb80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
[  556.763284]  ffff800031bffc00: 00 00 00 00 00 00 00 00 00 00 f1 f1 f1 f1 00 f2
[  556.763414] >ffff800031bffc80: f2 f2 00 02 f3 f3 00 00 00 00 00 00 00 00 00 00
[  556.763569]                             ^
[  556.763652]  ffff800031bffd00: 00 00 f1 f1 f1 f1 00 00 f2 f2 00 00 00 00 00 f2
[  556.763771]  ffff800031bffd80: f2 f2 f2 f2 00 00 00 00 00 f3 f3 f3 f3 f3 00 00
[  556.763900] ==================================================================

4. KASAN 的优缺点和发展

KASAN 通过在内存访问指令前插入钩子函数，去检测内存访问的合法性，其优点是可以做到实时检测，不像 slub debug 的检测需要手动触发，或者释放时发现问题，这存在检测上存在延迟；和 slub debug 一样，KASAN 也会带来性能上的开销，本文讨论的版本不适应于生产环境；另外，KASAN 也会带来内核代码体积的增大。

为了克服上面提到的这些缺点，在高版本的内核增加了 Software TAG(仅 ARM64 支持，需要利用指针高位，利用了 ARM64 TBI 特性) 和 Hardware TAG(需要硬件支持，如 ARM64 的 MTE)，Hardware TAG 的方式带来的性能开销很小，可以用于生产环境。

另外，高版本内核也增加了对 VMALLOC 空间的 KASAN 支持。

5. 参考资料

[1] Kernel Address Sanitizer (KASAN)
[2] linux kernel 内存踩踏之KASAN（一）
[3] linux之kasan原理及解析
[4] KASAN(9)-全局变量和栈变量的实现