Fuzzing tool (TFuzz): a fuzzing tool based on program transformation

Related tags

Deep Learning T-Fuzz
Overview

T-Fuzz

T-Fuzz consists of 2 components:

  • Fuzzing tool (TFuzz): a fuzzing tool based on program transformation
  • Crash Analyzer (CrashAnalyzer): a tool that verifies whether crashes found transformed programs are true bugs in the original program or not (coming soon).

OS support

The current version is tested only on Ubuntu-16.04, while trying to run the code, please use our tested OS.

Prerequisite

T-Fuzz system is built on several opensource tools.

Installing radare2

$ git clone https://github.com/radare/radare2.git
$ cd radare2
$ ./sys/install.sh

Installing python libraries

installing some dependent libraries

Note: to use apt-get build-dep, you need to uncomment the deb-src lines in your apt source file (/etc/apt/sources.list) and run apt-get update.

$ sudo apt-get install build-essential gcc-multilib libtool automake autoconf bison debootstrap debian-archive-keyring
$ sudo apt-get build-dep qemu-system
$ sudo apt-get install libacl1-dev

installing pip and setting up virtualenv & wrapper

$ sudo apt-get install python-pip python-virtualenv
$ pip install virtualenvwrapper

Add the following lines to your shell rc file (~/.bashrc or ~/.zshrc).

export WORKON_HOME=$HOME/.virtual_envs
source /usr/local/bin/virtualenvwrapper.sh

Creating a python virtual environment

$ mkvirtualenv tfuzz-env

Installing dependent libraries

This command will install all the dependent python libraries for you.

$ workon tfuzz-env
$ pip install -r req.txt

Fuzzing target programs with T-Fuzz

$ ./TFuzz  --program  
   
     --work_dir 
    
      --target_opts 
     

     
    
   

Where

  • : the path to the target program to fuzz
  • : the directory to save the results
  • : the options to pass to the target program, like AFL, use @@ as placeholder for files to mutate.

Examples

  1. Fuzzing base64 with T-Fuzz
$ ./TFuzz  --program  target_programs/base64  --work_dir workdir_base64 --target_opts "-d @@"
  1. Fuzzing uniq with T-Fuzz
$ ./TFuzz  --program  target_programs/uniq  --work_dir workdir_uniq --target_opts "@@"
  1. Fuzzing md5sum with T-Fuzz
$ ./TFuzz  --program  target_programs/md5sum  --work_dir workdir_md5sum --target_opts "-c @@"
  1. Fuzzing who with T-Fuzz
$ ./TFuzz  --program  target_programs/who  --work_dir workdir_who --target_opts "@@"

Using CrashAnalyzer to verify crashes

T-Fuzz CrashAnalyzer has been put in a docker image, however, it is still not working in all binaries we tested, we are still investigating it the cause.

Here is how:

Run the following command to run our docker image

$ [sudo] docker pull tfuzz/tfuzz-test
$ [sudo] docker run  --security-opt seccomp:unconfined -it tfuzz/tfuzz-test  /usr/bin/zsh 

In the container:

There are 3 directories:

  • release: contains code the built lava binaries
  • results: contains some results we found in lava-m dataset
  • radare2: it is a program used by T-Fuzz.

Currently, T-Fuzz may not work, because the tracer crashes accidentally. And the CrashAnalyzer can not work on all results. But some cases can be recovered.

For example:

To verify bugs in base64, first goto release and checkout ca_base64:

$ cd release
$ git checkout ca_base64

Then we use a transformed program to recover the crash in the original program:

  1. Choose a transformed program and run it on the input found by a fuzzer:
$ cd ~
$./results/ca_base64/554/base64_tfuzz_28/base64_tfuzz_28 -d ./results/ca_base64/554/crashing_inputs_from/results_saved_0_from 
[1]    131 segmentation fault (core dumped)  ./results/ca_base64/554/base64_tfuzz_28/base64_tfuzz_28 -d
  1. Recover an input from this transformed program and crashing input
). Re-hooking. WARNING | 2018-12-04 04:28:23,228 | angr.project | Address is already hooked, during hook(0x90dd000, ). Re-hooking. WARNING | 2018-12-04 04:28:23,229 | angr.simos.linux | Tracer has been heavily tested only for CGC. If you find it buggy for Linux binaries, we are sorry! Adding = 65) && (file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_0_0_8 <= 90)), ((file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_0_0_8 >= 97) && (file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_0_0_8 <= 122)), ((file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_0_0_8 >= 48) && (file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_0_0_8 <= 57)), (file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_0_0_8 == 43), (file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_0_0_8 == 47))> Adding = 65) && (file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_1_1_8 <= 90)), ((file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_1_1_8 >= 97) && (file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_1_1_8 <= 122)), ((file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_1_1_8 >= 48) && (file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_1_1_8 <= 57)), (file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_1_1_8 == 43), (file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_1_1_8 == 47))> Adding = 65) && (file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_2_2_8 <= 90)), ((file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_2_2_8 >= 97) && (file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_2_2_8 <= 122)), ((file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_2_2_8 >= 48) && (file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_2_2_8 <= 57)), (file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_2_2_8 == 43), (file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_2_2_8 == 47))> Adding = 65) && (file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_3_3_8 <= 90)), ((file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_3_3_8 >= 97) && (file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_3_3_8 <= 122)), ((file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_3_3_8 >= 48) && (file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_3_3_8 <= 57)), (file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_3_3_8 == 43), (file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_3_3_8 == 47))> results saved to /root/base64_result/recover_0 ">
$ ./release/CrashAnalyzer  --tprogram ./results/ca_base64/554/base64_tfuzz_28/base64_tfuzz_28 --target_opts "-d @@" --crash_input ./results/ca_base64/554/crashing_inputs_from/results_saved_0_from --result_dir base64_result --save_to recover
WARNING | 2018-12-04 04:28:22,350 | angr.analyses.disassembly_utils | Your verison of capstone does not support MIPS instruction groups.
Trying /root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from
WARNING | 2018-12-04 04:28:23,228 | angr.project | Address is already hooked, during hook(0x9021cd0, 
        
         ). Re-hooking.
WARNING | 2018-12-04 04:28:23,228 | angr.project | Address is already hooked, during hook(0x90dd000, 
         
          ). Re-hooking.
WARNING | 2018-12-04 04:28:23,229 | angr.simos.linux | Tracer has been heavily tested only for CGC. If you find it buggy for Linux binaries, we are sorry!
Adding 
          
           = 65) && (file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_0_0_8 <= 90)), ((file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_0_0_8 >= 97) && (file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_0_0_8 <= 122)), ((file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_0_0_8 >= 48) && (file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_0_0_8 <= 57)), (file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_0_0_8 == 43), (file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_0_0_8 == 47))>
Adding 
           
            = 65) && (file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_1_1_8 <= 90)), ((file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_1_1_8 >= 97) && (file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_1_1_8 <= 122)), ((file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_1_1_8 >= 48) && (file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_1_1_8 <= 57)), (file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_1_1_8 == 43), (file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_1_1_8 == 47))>
Adding 
            
             = 65) && (file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_2_2_8 <= 90)), ((file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_2_2_8 >= 97) && (file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_2_2_8 <= 122)), ((file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_2_2_8 >= 48) && (file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_2_2_8 <= 57)), (file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_2_2_8 == 43), (file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_2_2_8 == 47))> Adding 
             
              = 65) && (file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_3_3_8 <= 90)), ((file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_3_3_8 >= 97) && (file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_3_3_8 <= 122)), ((file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_3_3_8 >= 48) && (file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_3_3_8 <= 57)), (file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_3_3_8 == 43), (file_/root/results/ca_base64/554/crashing_inputs_from/results_saved_0_from_9_3_3_8 == 47))> results saved to /root/base64_result/recover_0 
             
            
           
          
         
        

Then /root/base64_result/recover_0 is generated, we can use it to trigger a crash in the original program.

  1. verify the input by running the generated input on the original program
$ ./results/base64 -d base64_result/recover_0 
Successfully triggered bug 554, crashing now!
Successfully triggered bug 554, crashing now!
Successfully triggered bug 554, crashing now!
[1]    177 segmentation fault (core dumped)  ./results/base64 -d base64_result/recover_0
Comments
  • Got error when I tried to fuzz the target_programs just like examples

    Got error when I tried to fuzz the target_programs just like examples

    After finished install all the dependent python library, I tried to fuzz the binary in target_programs just like the examples, and I got this error:

    WARNING | 2018-06-07 09:19:28,663 | angr.engines.successors | Exit state has over 257 possible solutions. Likely unconstrained; skipping. <BV32 global_c000013_64_32{UNINITIALIZED}> Deprecation warning: Use main_object instead of main_bin WARNING | 2018-06-07 09:19:31,096 | fuzzer.fuzzer | not forced Traceback (most recent call last): File "./TFuzz", line 64, in main() File "./TFuzz", line 55, in main tfuzzsys.run() File "/home/starpt/Documents/T-Fuzz/tfuzz/tfuzz_sys.py", line 160, in run if not self.__fuzz_one_program(): File "/home/starpt/Documents/T-Fuzz/tfuzz/tfuzz_sys.py", line 139, in __fuzz_one_program self.__current_fuzzer.start() File "/home/starpt/Documents/T-Fuzz/tfuzz/tfuzz_fuzzer.py", line 88, in start self._fuzzer.start() File "/home/starpt/.virtualenvs/t-fuzz/local/lib/python2.7/site-packages/fuzzer/fuzzer.py", line 232, in start self._start_afl() File "/home/starpt/.virtualenvs/t-fuzz/local/lib/python2.7/site-packages/fuzzer/fuzzer.py", line 543, in _start_afl master = self._start_afl_instance() # the master fuzzer File "/home/starpt/.virtualenvs/t-fuzz/local/lib/python2.7/site-packages/fuzzer/fuzzer.py", line 535, in _start_afl_instance return subprocess.Popen(args, stdout=fp, close_fds=True) File "/usr/lib/python2.7/subprocess.py", line 711, in init errread, errwrite) File "/usr/lib/python2.7/subprocess.py", line 1343, in _execute_child raise child_exception OSError: [Errno 2] No such file or directory

    I guessed it happened because I didn't export the AFL_PATH environment, so I tried to: export AFL_PATH=~/.virtualenvs/t-fuzz/bin, and I got same error again......

    How should I do to fixed this problem? Thank you

    opened by 0n3t04ll 7
  • Example issue

    Example issue

    Hi, all. I got error when executing this command in example:

    ./TFuzz  --program  target_programs/base64  --work_dir workdir_base64 --target_opts "-d @@"
    

    The following is the error information:

    WARNING | 2019-01-07 05:00:56,313 | angr.analyses.disassembly_utils | Your verison of capstone does not support MIPS instruction groups.
    /root/T-Fuzz/target_programs/base64
    WARNING | 2019-01-07 05:00:57,224 | angr.analyses.disassembly_utils | Your verison of capstone does not support MIPS instruction groups.
    WARNING | 2019-01-07 05:01:15,075 | angr.engines.successors | Exit state has over 257 possible solutions. Likely unconstrained; skipping. <BV32 global_c000000_14_32{UNINITIALIZED}>
    WARNING | 2019-01-07 05:01:15,134 | angr.engines.successors | Exit state has over 257 possible solutions. Likely unconstrained; skipping. <BV32 global_c000004_29_32{UNINITIALIZED}>
    WARNING | 2019-01-07 05:01:15,174 | angr.engines.successors | Exit state has over 257 possible solutions. Likely unconstrained; skipping. <BV32 global_c000008_35_32{UNINITIALIZED}>
    WARNING | 2019-01-07 05:01:18,414 | angr.engines.successors | Exit state has over 257 possible solutions. Likely unconstrained; skipping. <BV32 global_c00000e_55_32{UNINITIALIZED}>
    WARNING | 2019-01-07 05:01:18,576 | angr.engines.successors | Exit state has over 257 possible solutions. Likely unconstrained; skipping. <BV32 global_c000013_64_32{UNINITIALIZED}>
    WARNING | 2019-01-07 05:01:34,402 | angr.engines.successors | Exit state has over 257 possible solutions. Likely unconstrained; skipping. <BV32 global_c000001_24_32{UNINITIALIZED}>
    WARNING | 2019-01-07 05:01:34,466 | angr.engines.successors | Exit state has over 257 possible solutions. Likely unconstrained; skipping. <BV32 global_c000005_39_32{UNINITIALIZED}>
    WARNING | 2019-01-07 05:01:34,499 | angr.engines.successors | Exit state has over 257 possible solutions. Likely unconstrained; skipping. <BV32 global_c000009_45_32{UNINITIALIZED}>
    WARNING | 2019-01-07 05:01:36,095 | angr.engines.successors | Exit state has over 257 possible solutions. Likely unconstrained; skipping. <BV32 global_c00000e_54_32{UNINITIALIZED}>
    WARNING | 2019-01-07 05:01:36,154 | angr.engines.successors | Exit state has over 257 possible solutions. Likely unconstrained; skipping. <BV32 global_c000013_64_32{UNINITIALIZED}>
    Deprecation warning: Use main_object instead of main_bin
    WARNING | 2019-01-07 05:01:38,438 | fuzzer.fuzzer | not forced
    Traceback (most recent call last):
      File "./TFuzz", line 64, in <module>
        main()
      File "./TFuzz", line 55, in main
        tfuzzsys.run()
      File "/root/T-Fuzz/tfuzz/tfuzz_sys.py", line 160, in run
        if not self.__fuzz_one_program():
      File "/root/T-Fuzz/tfuzz/tfuzz_sys.py", line 139, in __fuzz_one_program
        self.__current_fuzzer.start()
      File "/root/T-Fuzz/tfuzz/tfuzz_fuzzer.py", line 88, in start
        self._fuzzer.start()
      File "/root/.virtual_envs/tfuzz-env/local/lib/python2.7/site-packages/fuzzer/fuzzer.py", line 232, in start
        self._start_afl()
      File "/root/.virtual_envs/tfuzz-env/local/lib/python2.7/site-packages/fuzzer/fuzzer.py", line 542, in _start_afl
        master = self._start_afl_instance() # the master fuzzer
      File "/root/.virtual_envs/tfuzz-env/local/lib/python2.7/site-packages/fuzzer/fuzzer.py", line 534, in _start_afl_instance
        return subprocess.Popen(args, stdout=fp, close_fds=True)
      File "/usr/lib/python2.7/subprocess.py", line 711, in __init__
        errread, errwrite)
      File "/usr/lib/python2.7/subprocess.py", line 1343, in _execute_child
        raise child_exception
    OSError: [Errno 2] No such file or directory
    
    opened by fouzhe 4
  • AFL Error: Suboptimal CPU scaling governor

    AFL Error: Suboptimal CPU scaling governor

    I tried T-fuzz, but I got the following error. I think this may be related to the change of angr. Because angr is changed to support python3 and its 'tracer' is merged into angr. So maybe you can push the whole codes including angr and shellphish fuzzer. my system is ubuntu 16.04. ./TFuzz --program target_programs/base64 --work_dir workdir_base64 --target_opts "-d @@" `WARNING | 2018-10-28 23:45:10,451 | angr.analyses.disassembly_utils | Your verison of capstone does not support MIPS instruction groups. ERROR | 2018-10-28 23:45:10,721 | fuzzer.fuzzer | AFL Error: Pipe at the beginning of core_pattern execute 'echo core | sudo tee /proc/sys/kernel/core_pattern' AFL Error: Suboptimal CPU scaling governor execute 'cd /sys/devices/system/cpu; echo performance | sudo tee cpu*/cpufreq/scaling_governor' AFL Warning: We probably want the fork() children to run first execute 'echo 1 | sudo tee /proc/sys/kernel/sched_child_runs_first'

    Traceback (most recent call last): File "./TFuzz", line 64, in main() File "./TFuzz", line 15, in main __angr_Fuzzer._perform_env_checks() File "/home/rosen/anaconda3/envs/T-Fuzz/lib/python2.7/site-packages/fuzzer/fuzzer.py", line 599, in _perform_env_checks raise InstallError(err) fuzzer.fuzzer.InstallError: AFL Error: Pipe at the beginning of core_pattern execute 'echo core | sudo tee /proc/sys/kernel/core_pattern' AFL Error: Suboptimal CPU scaling governor execute 'cd /sys/devices/system/cpu; echo performance | sudo tee cpu*/cpufreq/scaling_governor' AFL Warning: We probably want the fork() children to run first execute 'echo 1 | sudo tee /proc/sys/kernel/sched_child_runs_first'`

    Thanks for your time.

    opened by RosenZhu 4
  • Fuzzer says no program left

    Fuzzer says no program left

    Deprecation warning: Use main_object instead of main_bin WARNING | 2019-07-18 18:55:31,206 | tfuzz.tfuzz_sys | </home/ubuntu/T-Fuzz/workdir_who/who_tfuzz/who_tfuzz> failed to start, skip, but there still might be some crashes in crashing seeds WARNING | 2019-07-18 18:55:31,207 | tfuzz.tfuzz_sys | No program left I saw these messages and there was no progess in fuzzing for hours at all.

    Maybe manually installing python package shellphish-afl caused this error? I installed shellphish-afl 1.2.1 with source code, and installed other packages by req.txt

    Deprecation warning: Use main_object instead of main_bin

    at first, after above line, fuzzer had error because there was no ~/.virtual_envs/tfuzz_env/bin/afl-unix/afl-fuzz so i manually copied afl-fuzz to ~/.virtual_envs/tfuzz_env/bin/afl-unix/ does it caused problem?

    opened by geldkang 3
  • T-Fuzz stuck there.

    T-Fuzz stuck there.

    I ran the T-fuzz on docker, I don't know if that matter. The output is:

    root@5d42ab6b99cc:/apps/T-Fuzz# ./TFuzz --program target_programs/base64 --work_dir workdir_base64 --target_opts "-d @@" WARNING | 2018-11-29 12:03:16,387 | angr.analyses.disassembly_utils | Your verison of capstone does not support MIPS instruction groups. /apps/T-Fuzz/target_programs/base64 WARNING | 2018-11-29 12:03:17,409 | angr.analyses.disassembly_utils | Your verison of capstone does not support MIPS instruction groups. WARNING | 2018-11-29 12:03:38,370 | angr.engines.successors | Exit state has over 257 possible solutions. Likely unconstrained; skipping. <BV32 global_c000000_14_32{UNINITIALIZED}> WARNING | 2018-11-29 12:03:38,439 | angr.engines.successors | Exit state has over 257 possible solutions. Likely unconstrained; skipping. <BV32 global_c000004_29_32{UNINITIALIZED}> WARNING | 2018-11-29 12:03:38,485 | angr.engines.successors | Exit state has over 257 possible solutions. Likely unconstrained; skipping. <BV32 global_c000008_35_32{UNINITIALIZED}> WARNING | 2018-11-29 12:03:41,969 | angr.engines.successors | Exit state has over 257 possible solutions. Likely unconstrained; skipping. <BV32 global_c00000e_55_32{UNINITIALIZED}> WARNING | 2018-11-29 12:03:42,151 | angr.engines.successors | Exit state has over 257 possible solutions. Likely unconstrained; skipping. <BV32 global_c000013_64_32{UNINITIALIZED}> WARNING | 2018-11-29 12:04:00,529 | angr.engines.successors | Exit state has over 257 possible solutions. Likely unconstrained; skipping. <BV32 global_c000000_14_32{UNINITIALIZED}> WARNING | 2018-11-29 12:04:00,617 | angr.engines.successors | Exit state has over 257 possible solutions. Likely unconstrained; skipping. <BV32 global_c000004_29_32{UNINITIALIZED}> WARNING | 2018-11-29 12:04:00,662 | angr.engines.successors | Exit state has over 257 possible solutions. Likely unconstrained; skipping. <BV32 global_c000008_35_32{UNINITIALIZED}> WARNING | 2018-11-29 12:04:02,904 | angr.engines.successors | Exit state has over 257 possible solutions. Likely unconstrained; skipping. <BV32 global_c00000e_55_32{UNINITIALIZED}> WARNING | 2018-11-29 12:04:03,086 | angr.engines.successors | Exit state has over 257 possible solutions. Likely unconstrained; skipping. <BV32 global_c000013_64_32{UNINITIALIZED}> Deprecation warning: Use main_object instead of main_bin WARNING | 2018-11-29 12:04:06,019 | fuzzer.fuzzer | not forced

    Then, T-fuzz don't go further, it stucks here. The system on docker is ubuntu 16.04.

    opened by RosenZhu 3
  • T-Fuzz was aborted when fuzz the target_programs like examples

    T-Fuzz was aborted when fuzz the target_programs like examples

    T-Fuzz was aborted in a few minutes when I tried to fuzz the target_programs just like examples, and I have no idea about the reason. Please kindly check it.

    Here is part of the output:

    $ ./TFuzz --program target_programs/uniq --work_dir workdir_uniq --target_opts "@@"

    WARNING | 2018-07-30 04:45:40,595 | angr.analyses.disassembly_utils | Your verison of capstone does not support MIPS instruction groups. /home/public/.virtual_envs/tfuzz-env/T-Fuzz/target_programs/uniq WARNING | 2018-07-30 04:45:41,120 | angr.analyses.disassembly_utils | Your verison of capstone does not support MIPS instruction groups. WARNING | 2018-07-30 04:45:50,107 | angr.engines.successors | Exit state has over 257 possible solutions. Likely unconstrained; skipping. <BV32 global_c000000_14_32{UNINITIALIZED}> WARNING | 2018-07-30 04:45:50,214 | angr.engines.successors | Exit state has over 257 possible solutions. Likely unconstrained; skipping. <BV32 global_c000004_20_32{UNINITIALIZED}> WARNING | 2018-07-30 04:45:50,242 | angr.engines.successors | Exit state has over 257 possible solutions. Likely unconstrained ; skipping. <BV32 global_c000008_26_32{UNINITIALIZED}> WARNING | 2018-07-30 04:45:50,295 | angr.engines.successors | Exit state has over 257 possible solutions. Likely unconstrained ; skipping. <BV32 global_c00000d_35_32{UNINITIALIZED}> WARNING | 2018-07-30 04:45:50,362 | angr.engines.successors | Exit state has over 257 possible solutions. Likely unconstrained ; skipping. <BV32 global_c000011_50_32{UNINITIALIZED}> WARNING | 2018-07-30 04:45:52,092 | angr.engines.successors | Exit state has over 257 possible solutions. Likely unconstrained ; skipping. <BV32 global_c000016_60_32{UNINITIALIZED}> WARNING | 2018-07-30 04:45:52,199 | angr.engines.successors | Exit state has over 257 possible solutions. Likely unconstrained ; skipping. <BV32 global_c00001b_69_32{UNINITIALIZED}> WARNING | 2018-07-30 04:46:00,625 | angr.engines.successors | Exit state has over 257 possible solutions. Likely unconstrained ; skipping. <BV32 global_c000001_17_32{UNINITIALIZED}> WARNING | 2018-07-30 04:46:00,679 | angr.engines.successors | Exit state has over 257 possible solutions. Likely unconstrained ; skipping. <BV32 global_c000005_23_32{UNINITIALIZED}> WARNING | 2018-07-30 04:46:00,723 | angr.engines.successors | Exit state has over 257 possible solutions. Likely unconstrained ; skipping. <BV32 global_c000009_29_32{UNINITIALIZED}> WARNING | 2018-07-30 04:46:00,844 | angr.engines.successors | Exit state has over 257 possible solutions. Likely unconstrained ; skipping. <BV32 global_c00000d_44_32{UNINITIALIZED}> WARNING | 2018-07-30 04:46:00,871 | angr.engines.successors | Exit state has over 257 possible solutions. Likely unconstrained ; skipping. <BV32 global_c000011_50_32{UNINITIALIZED}> WARNING | 2018-07-30 04:46:02,015 | angr.engines.successors | Exit state has over 257 possible solutions. Likely unconstrained ; skipping. <BV32 global_c000016_60_32{UNINITIALIZED}> WARNING | 2018-07-30 04:46:02,122 | angr.engines.successors | Exit state has over 257 possible solutions. Likely unconstrained ; skipping. <BV32 global_c00001b_69_32{UNINITIALIZED}> WARNING | 2018-07-30 04:46:03,518 | fuzzer.fuzzer | not forced WARNING | 2018-07-30 04:47:05,627 | tfuzz.qemu_runner | One trace is found to be malformated, it is possible that the log file size exceeds the 1G limit, meaning that there might be infinite loops in the target program WARNING | 2018-07-30 04:47:05,684 | tfuzz.qemu_runner | One trace is found to be malformated, it is possible that the log file size exceeds the 1G limit, meaning that there might be infinite loops in the target program WARNING | 2018-07-30 04:47:05,738 | tfuzz.qemu_runner | One trace is found to be malformated, it is possible that the log file size exceeds the 1G limit, meaning that there might be infinite loops in the target program WARNING | 2018-07-30 04:47:05,796 | tfuzz.qemu_runner | One trace is found to be malformated, it is possible that the log file size exceeds the 1G limit, meaning that there might be infinite loops in the target program WARNING | 2018-07-30 04:47:05,853 | tfuzz.qemu_runner | One trace is found to be malformated, it is possible that the log file size exceeds the 1G limit, meaning that there might be infinite loops in the target program ... WARNING | 2018-07-30 04:47:08,787 | tfuzz.qemu_runner | One trace is found to be malformated, it is possible that the log file size exceeds the 1G limit, meaning that there might be infinite loops in the target program WARNING | 2018-07-30 04:47:08,842 | tfuzz.qemu_runner | One trace is found to be malformated, it is possible that the log file size exceeds the 1G limit, meaning that there might be infinite loops in the target program WARNING | 2018-07-30 04:47:08,899 | tfuzz.qemu_runner | One trace is found to be malformated, it is possible that the log file size exceeds the 1G limit, meaning that there might be infinite loops in the target program WARNING | 2018-07-30 04:47:08,905 | tfuzz.tfuzz_sys | No program left

    Then the fuzzer was aborted.

    opened by nopitydays 3
  • verison of capstone does not support MIPS instruction

    verison of capstone does not support MIPS instruction

    Hi,

    I have installed T-Fuzz on Windows Subsystem for Linux (Ubuntu). and I get the following error:

    WARNING | 2018-06-10 10:28:51,171 | angr.analyses.disassembly_utils | Your verison of capstone does not support MIPS instruction groups.
    Traceback (most recent call last):
      File "./TFuzz", line 64, in <module>
        main()
      File "./TFuzz", line 15, in main
        __angr_Fuzzer._perform_env_checks()
      File "/root/.virtual_envs/tfuzz-env/local/lib/python2.7/site-packages/fuzzer/fuzzer.py", line 572, in _perform_env_checks
        with open("/proc/sys/kernel/core_pattern") as f:
    IOError: [Errno 2] No such file or directory: '/proc/sys/kernel/core_pattern'
    

    Any pointers around this?

    opened by ksloven 3
  • Dockerfile request

    Dockerfile request

    Hi, The shellphish fuzzer repository is archived and angr tracer is deprecated. Seems also that you are using angr 7. Can you provide a Dockerfile for Ubuntu 16.04 that downloads all the correct versions of this old software? Or at least make a list of compatible versions. Thank you.

    Regards, Andrea

    opened by andreafioraldi 2
  • afl master only run first cycle?

    afl master only run first cycle?

    master.log shows:

        Test case count : 2 favored, 0 variable, 100 total
           Bitmap range : 1011 to 1013 bits (average: 1011.50 bits)
            Exec timing : 138k to 310k us (average: 237k us)
    
    [*] No -t option specified, so I'll use exec timeout of 480 ms.
    [+] All set and ready to roll!
    [*] Entering queue cycle 1.
    [*] Fuzzing test case #0 (100 total, 0 uniq crashes found)...
    

    In normal afl runs, there should be more logs?

    also, the runtime CPU is so low...

    CONTAINER ID        NAME                CPU %               MEM USAGE / LIMIT   MEM %               NET I/O             BLOCK I/O           PIDS
    2744d8ed2340        tfuzz3_30           3.25%               349.4MiB / 2GiB     17.06%              3.95kB / 0B         0B / 18.6MB         7
    

    For running 24 hours, the crash folder in sync/fuzzer-master/crashes is also empty for mp3gain 1.5.2-r2. And there is only one output file in sync/fuzzer-master/crashes for tiffsplit (libtiff-3.9.7)

    opened by zjuchenyuan 2
  • No matching distribution found for pkg-resources==0.0.0

    No matching distribution found for pkg-resources==0.0.0

    I've the error after:

    pip install -r req.txt
    

    Log:

    ...
    Collecting pkg-resources==0.0.0 (from -r req.txt (line 53))
      Could not find a version that satisfies the requirement pkg-resources==0.0.0 (from -r req.txt (line 53)) (from versions: )
    No matching distribution found for pkg-resources==0.0.0 (from -r req.txt (line 53))
    

    OS: Ubuntu

    Some people wrote that it's problem in Ubuntu and fix - should delete line pkg-resources==0.0.0

    opened by dukebarman 2
  • An error in create_dict in utils.py

    An error in create_dict in utils.py

    Hi, all! I built T-Fuzz in Ubuntu 16.04.5 LTS, when I ran the following example, T-Fuzz got stuck. $./TFuzz --program target_programs/base64 --work_dir workdir_base64 --target_opts "-d @@"

    Here is the output:

    WARNING | 2019-01-07 13:21:00,518 | angr.analyses.disassembly_utils | Your verison of capstone does not support MIPS instruction groups.
    /root/tfuzz/T-Fuzz/target_programs/base64
    WARNING | 2019-01-07 13:21:01,424 | angr.analyses.disassembly_utils | Your verison of capstone does not support MIPS instruction groups.
    WARNING | 2019-01-07 13:21:19,691 | angr.engines.successors | Exit state has over 257 possible solutions. Likely unconstrained; skipping. <BV32 global_c000000_23_32{UNINITIALIZED}>
    WARNING | 2019-01-07 13:21:19,730 | angr.engines.successors | Exit state has over 257 possible solutions. Likely unconstrained; skipping. <BV32 global_c000004_29_32{UNINITIALIZED}>
    WARNING | 2019-01-07 13:21:19,772 | angr.engines.successors | Exit state has over 257 possible solutions. Likely unconstrained; skipping. <BV32 global_c000008_35_32{UNINITIALIZED}>
    WARNING | 2019-01-07 13:21:22,801 | angr.engines.successors | Exit state has over 257 possible solutions. Likely unconstrained; skipping. <BV32 global_c00000e_54_32{UNINITIALIZED}>
    WARNING | 2019-01-07 13:21:22,866 | angr.engines.successors | Exit state has over 257 possible solutions. Likely unconstrained; skipping. <BV32 global_c000013_64_32{UNINITIALIZED}>
    WARNING | 2019-01-07 13:21:38,761 | angr.engines.successors | Exit state has over 257 possible solutions. Likely unconstrained; skipping. <BV32 global_c000001_24_32{UNINITIALIZED}>
    WARNING | 2019-01-07 13:21:38,801 | angr.engines.successors | Exit state has over 257 possible solutions. Likely unconstrained; skipping. <BV32 global_c000005_30_32{UNINITIALIZED}>
    WARNING | 2019-01-07 13:21:38,876 | angr.engines.successors | Exit state has over 257 possible solutions. Likely unconstrained; skipping. <BV32 global_c000009_45_32{UNINITIALIZED}>
    WARNING | 2019-01-07 13:21:40,609 | angr.engines.successors | Exit state has over 257 possible solutions. Likely unconstrained; skipping. <BV32 global_c00000e_55_32{UNINITIALIZED}>
    WARNING | 2019-01-07 13:21:40,752 | angr.engines.successors | Exit state has over 257 possible solutions. Likely unconstrained; skipping. <BV32 global_c000013_64_32{UNINITIALIZED}>
    Deprecation warning: Use main_object instead of main_bin
    WARNING | 2019-01-07 13:21:43,102 | fuzzer.fuzzer | not forced
    WARNING | 2019-01-07 13:21:45,116 | tfuzz.tfuzz_sys | </root/tfuzz/T-Fuzz/workdir_base64/base64_tfuzz/base64_tfuzz> failed to start,                 skip, but there still might be some                 crashes in crashing seeds
    WARNING | 2019-01-07 13:21:45,118 | tfuzz.tfuzz_sys | No program left
    

    It seems that the fuzzer failed to start: $cat workdir_base64/fuzzing_base64_tfuzz/base64_tfuzz/stat

    status:failed to start
    

    cat workdir_base64/fuzzing_base64_tfuzz/base64_tfuzz/fuzzer-master.log

    afl-fuzz 2.52b by <[email protected]>
    [+] Looks like we're not running on a tty, so I'll be a bit less verbose.
    [+] You have 80 CPU cores and 16 runnable tasks (utilization: 20%).
    [+] Try parallel jobs - see /usr/local/share/doc/afl/parallel_fuzzing.txt.
    [*] Checking CPU core loadout...
    [+] Found a free CPU core, binding to #0.
    [*] Checking core_pattern...
    [*] Setting up output directories...
    [*] Scanning '/root/tfuzz/T-Fuzz/workdir_base64/fuzzing_base64_tfuzz/base64_tfuzz/input'...
    [+] No auto-generated dictionary tokens to reuse.
    [*] Creating hard links for all input files...
    [*] Loading extra dictionary from '/root/tfuzz/T-Fuzz/workdir_base64/base64.dict' (level 0)...
    

    The issue occurs in the dictionary:

    $cat workdir_base64/base64.dict | head -10

    Deprecation warning: Use eval(expr, cast_to=str) instead of any_str
    string_0="\x00"
    string_1="\x5f\x5fAFL\x5fDEFER\x5fFORKSRV\x00"
    string_2="help"
    string_3="Written by \x25s\x2c \x25s\x2c \x25s\x2c\x0a\x25s\x2c \x25s\x2c \x25s\x2c \x25s\x2c\x0a\x25s\x2c \x25s\x2c and others.\x0a\x00"
    string_4="Written by \x25s\x2c \x25s\x2c \x25s\x2c\x0a\x25s\x2c \x25s\x2c \x25s\x2c \x25s\x2c\x0aand \x25s.\x0a\x00"
    string_5="\x28C\x29\x00"
    string_6="\x2550s"
    string_7="\x5f\x5fAFL\x5fSHM\x5fID\x00"
    string_8="Simon"
    

    This first line of the dict is a warning, which leads to AFL failure.

    I think this issue is caused by the following line, which attempts write all of stdout in the file dict:

    https://github.com/HexHive/T-Fuzz/blob/7d150e493237db72c421d423f9a315401cb94e44/tfuzz/utils.py#L13

    This issue can be solved by checking like this: https://github.com/fouzhe/T-Fuzz/commit/0aa9684c704c3f4f4ce6e7678af9f18bc2a1c6b6

    opened by fouzhe 0
  • T-Fuzz used wrong code to filter crashing seed?

    T-Fuzz used wrong code to filter crashing seed?

    From our experiment to fuzz wav2swf using T-Fuzz, we found an interesting result: The coverage result from the afl queue folder is smaller than provided seed.

    So, we investigate this finding, here may be the cause:

    https://github.com/HexHive/T-Fuzz/blob/7d150e493237db72c421d423f9a315401cb94e44/tfuzz/executor.py#L39-L48

    https://github.com/HexHive/T-Fuzz/blob/7d150e493237db72c421d423f9a315401cb94e44/tfuzz/tfuzz_fuzzer.py#L40-L50

    you can see that whenever a CalledProcessError is raised, T-Fuzz will ignore this seed, copy to crashing_seed. And if the whole seed folder all leads to crash, then T-Fuzz will generate a random seed for fuzzing.

    However, CalledProcessError will also be raised if the exit code is not 0:

    >>> subprocess32.check_output(["/d/p/justafl/4.wav2swf", "-o","/dev/null","crash_seed_12"], timeout=1)
    Unsupported bitspersample value: 24
    Error: no mp3 soundstream support compiled in.
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/usr/local/lib/python2.7/dist-packages/subprocess32.py", line 638, in check_output
        raise CalledProcessError(retcode, process.args, output=output)
    subprocess32.CalledProcessError: Command '['/d/p/justafl/4.wav2swf', '-o', '/dev/null', 'crash_seed_12']' returned non-zero exit status 1
    

    I think exit status 1 does not mean a crash, only those be killed by a signal should be.

    opened by zjuchenyuan 0
  • Problems during installation

    Problems during installation

    when i try to install T-Fuzz following instructions, there are some problems with shellphish-afl, python package. req.txt install shellphish-afl version 1.1, but building wheel for shellphish-afl fails. So i installed shellphish-afl 1.21 manually, but then errors occur when running TFuzz. When i tried to install shellphish-afl 1.1 manually, it failed. but i didn't go deeper to it. is shellphish-afl 1.21 also usable in TFuzz? or should i manually install shellphish-afl 1.1?

    opened by geldkang 2
  • T-Fuzz generated too many transformed binaires which take too much disk space

    T-Fuzz generated too many transformed binaires which take too much disk space

    image

    When fuzzing programes like flvmeta, we found that T-Fuzz has generated too many transformed binaries within 24 hours (du -sh shows 9.1G), which take too much disk space and impact I/O performance.

    According to your code, whenever the afl gots stuck, NCCDector will be run to generate a bunch of transformed binaries, and then fuzzing them one by one. Can this be improved by set a maxinum number of generated programs? Or immediately start new fuzzing process after one transformed binary generated?

    opened by zjuchenyuan 0
  • TFuzz take too long time for create_dict when fuzzing big binaries

    TFuzz take too long time for create_dict when fuzzing big binaries

    When fuzzing big binaries, like ffmpeg (about 130MB), it takes a very long time and huge memory to create dict.

    I think create dict is not necessary and should be skipped or time-limited for big binaries.

    opened by zjuchenyuan 0
  • tfuzz did not consider cannot-renamed programs

    tfuzz did not consider cannot-renamed programs

    There is a type of programs whose behaviour depend to which name they are. busybox is a good example, only one file is real ELF file and others are just symlink to it.

    In this scenario, it's important for fuzzer to keep the original ELF filename, otherwise desired program behaviour will not be triggered and fuzzed.

    I want to fuzz infotocap, which is a symlink to tic

    But tfuzz actually runs: (which means it has been renamed to infotocap_tfuzz)

     /usr/bin/afl-unix/afl-fuzz -i /d/output/reruninfotocap_tfuzz6_1/fuzzing_infotocap_tfuzz/infotocap_tfuzz/input -o /d/output/reruninfotocap_tfuzz6_1/fuzzing_infotocap_tfuzz/infotocap_tfuzz/sync -m 8G -M fuzzer-master -x /d/output/reruninfotocap_tfuzz6_1/infotocap.dict -- /d/output/reruninfotocap_tfuzz6_1/infotocap_tfuzz/infotocap_tfuzz -o /dev/null @@
    

    Could you consider this type of program? Thanks~

    opened by zjuchenyuan 4
Owner
HexHive
Enforcing memory safety guarantees and type safety guarantees at the compiler and runtime level
HexHive
Attention-based Transformation from Latent Features to Point Clouds (AAAI 2022)

Attention-based Transformation from Latent Features to Point Clouds This repository contains a PyTorch implementation of the paper: Attention-based Tr

null 12 Nov 11, 2022
Efficient Householder transformation in PyTorch

Efficient Householder Transformation in PyTorch This repository implements the Householder transformation algorithm for calculating orthogonal matrice

Anton Obukhov 49 Nov 20, 2022
Data manipulation and transformation for audio signal processing, powered by PyTorch

torchaudio: an audio library for PyTorch The aim of torchaudio is to apply PyTorch to the audio domain. By supporting PyTorch, torchaudio follows the

null 1.9k Dec 28, 2022
git《Tangent Space Backpropogation for 3D Transformation Groups》(CVPR 2021) GitHub:1]

LieTorch: Tangent Space Backpropagation Introduction The LieTorch library generalizes PyTorch to 3D transformation groups. Just as torch.Tensor is a m

Princeton Vision & Learning Lab 482 Jan 6, 2023
Meta Representation Transformation for Low-resource Cross-lingual Learning

MetaXL: Meta Representation Transformation for Low-resource Cross-lingual Learning This repo hosts the code for MetaXL, published at NAACL 2021. [Meta

Microsoft 36 Aug 17, 2022
[CVPR'21] Locally Aware Piecewise Transformation Fields for 3D Human Mesh Registration

Locally Aware Piecewise Transformation Fields for 3D Human Mesh Registration This repository contains the implementation of our paper Locally Aware Pi

sfwang 70 Dec 19, 2022
Improving Contrastive Learning by Visualizing Feature Transformation, ICCV 2021 Oral

Improving Contrastive Learning by Visualizing Feature Transformation This project hosts the codes, models and visualization tools for the paper: Impro

Bingchen Zhao 83 Dec 15, 2022
GraphGT: Machine Learning Datasets for Graph Generation and Transformation

GraphGT: Machine Learning Datasets for Graph Generation and Transformation Dataset Website | Paper Installation Using pip To install the core environm

y6q9 50 Aug 18, 2022
[CVPR'21] Projecting Your View Attentively: Monocular Road Scene Layout Estimation via Cross-view Transformation

Projecting Your View Attentively: Monocular Road Scene Layout Estimation via Cross-view Transformation Weixiang Yang, Qi Li, Wenxi Liu, Yuanlong Yu, Y

null 118 Dec 26, 2022
Affine / perspective transformation in Pose Estimation with Tensorflow 2

Pose Transformation Affine / Perspective transformation in Pose Estimation with Tensorflow 2 Introduction 이 repo는 pose estimation을 연구하고 개발하는 데 도움이 되기

Kim Junho 1 Dec 22, 2021
Finite-temperature variational Monte Carlo calculation of uniform electron gas using neural canonical transformation.

CoulombGas This code implements the neural canonical transformation approach to the thermodynamic properties of uniform electron gas. Building on JAX,

FermiFlow 9 Mar 3, 2022
Apply a perspective transformation to a raster image inside Inkscape (no need to use an external software such as GIMP or Krita).

Raster Perspective Apply a perspective transformation to bitmap image using the selected path as envelope, without the need to use an external softwar

s.ouchene 19 Dec 22, 2022
Differential fuzzing for the masses!

NEZHA NEZHA is an efficient and domain-independent differential fuzzer developed at Columbia University. NEZHA exploits the behavioral asymmetries bet

null 147 Dec 5, 2022
InsTrim: Lightweight Instrumentation for Coverage-guided Fuzzing

InsTrim The paper: InsTrim: Lightweight Instrumentation for Coverage-guided Fuzzing Build Prerequisite llvm-8.0-dev clang-8.0 cmake >= 3.2 Make git cl

null 75 Dec 23, 2022
ProFuzzBench - A Benchmark for Stateful Protocol Fuzzing

ProFuzzBench - A Benchmark for Stateful Protocol Fuzzing ProFuzzBench is a benchmark for stateful fuzzing of network protocols. It includes a suite of

null 155 Jan 8, 2023
Emulation and Feedback Fuzzing of Firmware with Memory Sanitization

BaseSAFE This repository contains the BaseSAFE Rust APIs, introduced by "BaseSAFE: Baseband SAnitized Fuzzing through Emulation". The example/ directo

Security in Telecommunications 138 Dec 16, 2022
A fuzzing framework for SMT solvers

yinyang A fuzzing framework for SMT solvers. Given a set of seed SMT formulas, yinyang generates mutant formulas to stress-test SMT solvers. yinyang c

Project Yin-Yang for SMT Solver Testing 145 Jan 4, 2023
AntiFuzz: Impeding Fuzzing Audits of Binary Executables

AntiFuzz: Impeding Fuzzing Audits of Binary Executables Get the paper here: https://www.usenix.org/system/files/sec19-guler.pdf Usage: The python scri

Chair for Sys­tems Se­cu­ri­ty 88 Dec 21, 2022
Fuzzification helps developers protect the released, binary-only software from attackers who are capable of applying state-of-the-art fuzzing techniques

About Fuzzification Fuzzification helps developers protect the released, binary-only software from attackers who are capable of applying state-of-the-

gts3.org (SSLab@Gatech) 55 Oct 25, 2022