Hi Sergey,

Thank you for the great tool & research!

I am trying to do fuzzing of kafl_vuln_test using kAFL but I have a problem. I managed to install everything (qemu, KVM and etc.) according with the guide provided in README.md step by step including installation of vulnerable driver.

When I start kAFL it looks like it works but bitmap is always 0.00b and the file bitmap in the work directory is empty. I see qemu-system process in memory and I see multiple python processes working. My intuition that something is wrong on client side... I copied the address range of my driver from the output of kafl_info.py.

Do you have any ideas how to fix or debug that ?

Thank you in advance!

Hello.

Currently, I'm studying kAFL's KVM-PT and QEMU-PT to apply binary kernel fuzzing. So, I modified your kAFL to fit our fuzzer; only QEMU-PT. And I found some errors that occurs when the hypervisor sets the multiple IP ranges in QEMU-PT's decoder part.

As I understand in Intel PT decoder, after the pt_disable() is called, the hypervisor starts decoding about the stored Intel PT packets decode_buffer(). However, I got the different results when I set the single IP range (e.g., IP range A) and multiple IP range (e.g., IP range A, and B). For example, if I saw the control flow(ex. a->b->c) in single IP range A, I saw the different control flow(ex. a→d→e) when I set the multiple IP ranges, including a single IP range A.

I kept searching why it happens, and found that it has some errors in decoding the TNT packets. Each decoder which sets different IP ranges got influences from the another IP range's TNT infos. Therefore, it save the wrong control flows after decoding Intel PT packets.

I think the decoder part needs to be fixed.

Thank you.

I have written a custom fuzzing agent that creates another process then calls WaitForSingleObject on that process. I disabled the -nographic option of qemu to see what was actually going on in the fuzzed VM and saw that the program wasn't fully executed and stopped in the middle.

Is there some kind of timeout or mecanism that could be happening and cutting the execution?

Hi, I encountered some problems when I try to install kAFL.

• Problem 1:

when I try to run ./load.sh, It failed.

lzs243@E356-U180315:~/Documents/Kernel_Fuzzing/Tools/kAFL/kAFL-Fuzzer/vuln_drivers/simple/linux_x86-64$ sudo ./load.sh 
[sudo] password for lzs243: 
make -C /lib/modules/4.13.0-37-generic/build M=/home/lzs243/Documents/Kernel_Fuzzing/Tools/kAFL/kAFL-Fuzzer/vuln_drivers/simple/linux_x86-64 modules
make[1]: Entering directory '/usr/src/linux-headers-4.13.0-37-generic'
  CC [M]  /home/lzs243/Documents/Kernel_Fuzzing/Tools/kAFL/kAFL-Fuzzer/vuln_drivers/simple/linux_x86-64/kafl_vuln_test.o
In file included from /home/lzs243/Documents/Kernel_Fuzzing/Tools/kAFL/kAFL-Fuzzer/vuln_drivers/simple/linux_x86-64/kafl_vuln_test.c:6:0:
./arch/x86/include/asm/uaccess.h: In function ‘set_fs’:
./arch/x86/include/asm/uaccess.h:31:9: error: dereferencing pointer to incomplete type ‘struct task_struct’
  current->thread.addr_limit = fs;
         ^
/home/lzs243/Documents/Kernel_Fuzzing/Tools/kAFL/kAFL-Fuzzer/vuln_drivers/simple/linux_x86-64/kafl_vuln_test.c: In function ‘write_info’:
/home/lzs243/Documents/Kernel_Fuzzing/Tools/kAFL/kAFL-Fuzzer/vuln_drivers/simple/linux_x86-64/kafl_vuln_test.c:52:6: error: implicit declaration of function ‘copy_from_user’ [-Werror=implicit-function-declaration]
  if (copy_from_user(input, buff, len)) {
      ^
cc1: some warnings being treated as errors
scripts/Makefile.build:315: recipe for target '/home/lzs243/Documents/Kernel_Fuzzing/Tools/kAFL/kAFL-Fuzzer/vuln_drivers/simple/linux_x86-64/kafl_vuln_test.o' failed
make[2]: *** [/home/lzs243/Documents/Kernel_Fuzzing/Tools/kAFL/kAFL-Fuzzer/vuln_drivers/simple/linux_x86-64/kafl_vuln_test.o] Error 1
Makefile:1550: recipe for target '_module_/home/lzs243/Documents/Kernel_Fuzzing/Tools/kAFL/kAFL-Fuzzer/vuln_drivers/simple/linux_x86-64' failed
make[1]: *** [_module_/home/lzs243/Documents/Kernel_Fuzzing/Tools/kAFL/kAFL-Fuzzer/vuln_drivers/simple/linux_x86-64] Error 2
make[1]: Leaving directory '/usr/src/linux-headers-4.13.0-37-generic'
Makefile:4: recipe for target 'all' failed
make: *** [all] Error 2
insmod: ERROR: could not load module kafl_vuln_test.ko: No such file or directory
done

• Problem 2:

When I try to run the loader in /agents/linux_x86_64/loader, It says Illegal instruction (core dumped)

lzs243@E356-U180315:~/Documents/Kernel_Fuzzing/Tools/kAFL/kAFL-Fuzzer/agents/linux_x86_64/loader$ sudo ./loader 
Kernel Panic Handler Address:	ffffffffb0a870f3
Illegal instruction (core dumped)
lzs243@E356-U180315:~/Documents/Kernel_Fuzzing/Tools/kAFL/kAFL-Fuzzer/agents/linux_x86_64/loader$

• Problem 3:

when I run the command:

python kafl_info.py snapshot/ram.qcow2 snapshot/ agents/linux_x86_64/info/info 512 -v

It shows [Error] VMX_PT is not loaded!

kernel AFL: A feedback-driven general purpose ring-0 interface fuzzer for x86-64 operating systems.

Sergej Schumilo		<[email protected]>
Cornelius Aschermann	<[email protected]>
Robert Gawlik		<[email protected]>

Version: 0.1

(C) 2017
<< kafl_info.py: Kernel Address Dumper >>

**[Error] VMX_PT is not loaded!**

Hello there, I meet some problems during installation. In "Setup VM": When I run qemu-2.9.0/x86_64-softmmu/qemu-system-x86_64 -cpu host -enable-kvm -m 512 -hda linux.qcow2 -cdrom ubuntu.iso -usbdevice tablet I get an error: qemu-system-x86_64: error: failed to set MSR 0x38d to 0x0 qemu-system-x86_64: /home/c2hpxq/sectools/kAFL/qemu-2.9.0/target/i386/kvm.c:1833: kvm_put_msrs: Assertion ret == cpu->kvm_msr_buf->nmsrs' failed. I've tried use qemu-system-x86_64 installed by apt instead, the error not occurring anymore but when in "Compile and configure kAFL components", I run: python kafl_info.py /path/to/snapshot/ram.qcow2 /path/to/snapshot/ agents/linux_x86_64/info/info 512 -v I'm told VMX_PT not loaded.

Now I'm trying to comment out assertions in qemu-2.9.0/target/i386/kvm.c (line 1833 & 2204), and the VM installation seems working(I'm still on it when writing this answer) Do I get anything wrong? What's the right way to do it?

And by the way, when VM installation completed & rebooting, it get stuck at interface "Ubuntu 16.04", not continuing anymore... Kind of weird...

Please help me about it.

Update1: Still meet the problem [Error] VMX_PT is not loaded!

Update2: Can I do installation in a Ubuntu 16.04 virtual machine? i.e., qemu vm running in a virtual machine, does that matter? And where can I find information about whether a CPU supporting PT or not, it seems CPU specification doesn't list it in the product specification page, and I don't find anything useful in intel PT page. I don't know whether i7-7820HQ & i7-4712MQ are supported or not. I assume the former is supported and doubt the latter.

Hope for your help!

Is it lack of some components？Can't find any information about qemu slave vm,especially this line "-hda " + self.config.argument_values['overlay_dir'] + "/overlay_" + self.qemu_id + ".qcow2 " \， Could you give me some clue , thanks a lot .

I installed kAFL and setup the kafl_vuln_test test case according to the README.md from the latest (1ece095a7a835887477393b2bba88247b1536d4f) version in git. I also patched the fuzzer to set "socket.setdefaulttimeout(None)" and patched mapserver to import lz4.block. When I run the fuzzer it quickly finds 7 paths in the first 20 seconds. The corpus shows some progress towards inputs that will crash the test driver. But then the fuzzer makes no further progress even if I leave it running for hours. It looks like it is still running but it finds no more paths and adds no more entries to the corpus.

I've built on ubuntu 16.04.3 and installed 16.04.3 server in the guest. I was able to use the info program to get the ip0 range for kafl_vuln_test.

For testing purposes it would be much easier to have the compiled windows vulnerable driver or an easy way to have that. I have been struggling with mingw to get it compile, would you have any way to do it easily?

I met some troubles when following your instructions.

1. As your saying:

Execute loader binary which is in path/to/kAFL/kAFL-Fuzzer/agents/linux_x86_64/loader/ as root VM should freeze.

I execute loader binary as root but VM didn't freeze. It echoed a sentence that:

Kernel Panic Handler Address: ffffffff8118d734

Is this a right way?

2. How to Switch to the QEMU management console? I press CTRL-a + c but nothing happenned.

Looking forward to your answer.

After using this project， I can get the TNT and the TIP information， but I also need the time information，so can I ask for how to modify the code to get that message？

Has anyone got this working with the latest macOS?

I have found to run macOS on QEMU I need to add several CPU feature flags including the invtsc flag, however this flag makes the VM non-migratable so the savevm command no longer works.

qemu.py sends an extra 'R' message during initial handshake and soft reload. This releases the virtual machine that is waiting on hypercall_next_payload before qemu.py writes to the payload buffer. As a result, when the bitmap is returned, it is always measuring the previous test case, not the current test case (unless the vm is running slow enough, then qemu.py can occasionally win the race and get the buffer filled before the vm processes it). The fix is easy -- remove the extra send of the 'R' message in qemu.py's soft_reload and set_init_state methods.

I installed on ubuntu 16.04.3 server and followed the startup instructions in the readme to load the vulnerable test driver in the guest and start fuzzing it. After about three minutes the UI displays a python error

close failed in file object destructor: IOError: [Errno 9] Bad file descriptor

It does not provide much context and the fuzzer process seems to continue running.

Hi Sergey and mxmssh ， Thank you for the great tool & research!

I am trying to do fuzzing of kafl_vuln_test using kAFL but I have a problem. I managed to install everything (qemu, KVM and etc.) according with the guide provided in README.md step by step including installation of vulnerable driver. （Forgive me，mxmmsh）

But I got this error : FAIL 1 0xxxxxxx; After debugging, I found that the problem appeared here. write_virtual_memory function at memory_access.c

    phys_addr = cpu_get_phys_page_attrs_debug(cpu, (address & x86_64_PAGE_MASK), &attrs);

    if (phys_addr == -1){
        printf("FAIL 1 (%lx)!\n", address);
        return false;
    }

Do you have any ideas how to fix that ?

Thank you in advance!