BinTuner is a cost-efficient auto-tuning framework, which can deliver a near-optimal binary code that reveals much more differences than -Ox settings.

Hi, I'm trying to run the bzip2 example mentioned in README.md. Under the Dev/BinTuner/examples/gccflags directory, I tried to run the command python2 bzip2_gcc.py 10. Then Python complains the file Dev/BinTuner/examples/gccflags/../../opentuner/utils/adddeps.py cannot be found. I manually looked up the source directory and also could not find the file adddeps.py.

The full error information is attached:

Traceback (most recent call last):
  File "bzip2_gcc.py", line 3, in <module>
    import adddeps
  File "/...../Dev/BinTuner/examples/gccflags/adddeps.py", line 6, in <module>
    execfile(target, dict(__file__=target))
IOError: [Errno 2] No such file or directory: '/....../Dev/BinTuner/examples/gccflags/../../open
tuner/utils/adddeps.py'

Thanks, XZ-X

Program Start ['GGA'] beformutation ************************ Z3 ************************

Thread 1 "python" received signal SIGSEGV, Segmentation fault. api::object::inc_ref (this=this@entry=0xffffffff918ba0a8) at /z3/src/api/api_context.cpp:38 38 void object::inc_ref() { m_ref_count++; } (gdb) bt #0 api::object::inc_ref (this=this@entry=0xffffffff918ba0a8) at /z3/src/api/api_context.cpp:38 #1 0x00007f451a0f0c88 in Z3_solver_inc_ref (c=0x55a49197c7a8, s=0xffffffff918ba0a8) at /z3/src/api/api_solver.cpp:379 #2 0x00007f45bd37cdae in ffi_call_unix64 () from /usr/lib/x86_64-linux-gnu/libffi.so.6 #3 0x00007f45bd37c71f in ffi_call () from /usr/lib/x86_64-linux-gnu/libffi.so.6 #4 0x00007f45bd58fbf4 in _ctypes_callproc () from /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so #5 0x00007f45bd58f5f5 in ?? () from /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so #6 0x000055a4902da6c0 in PyEval_EvalFrameEx () #7 0x000055a4902da4c6 in PyEval_EvalFrameEx () #8 0x000055a4902d2d5a in PyEval_EvalCodeEx () #9 0x000055a4902eebc9 in ?? () #10 0x000055a490306efe in ?? () #11 0x000055a4902be8be in PyObject_Call () #12 0x000055a4902f23cc in PyInstance_New () #13 0x000055a4902da6c0 in PyEval_EvalFrameEx () #14 0x000055a4902d2d5a in PyEval_EvalCodeEx () #15 0x000055a4902dae3c in PyEval_EvalFrameEx () #16 0x000055a4902da4c6 in PyEval_EvalFrameEx () #17 0x000055a4902da4c6 in PyEval_EvalFrameEx () #18 0x000055a4902d2d5a in PyEval_EvalCodeEx () #19 0x000055a4902dae3c in PyEval_EvalFrameEx () #20 0x000055a4902d2d5a in PyEval_EvalCodeEx () #21 0x000055a4902dae3c in PyEval_EvalFrameEx () #22 0x000055a4902da4c6 in PyEval_EvalFrameEx () #23 0x000055a4902da4c6 in PyEval_EvalFrameEx () #24 0x000055a4902d2d5a in PyEval_EvalCodeEx () #25 0x000055a4902dae3c in PyEval_EvalFrameEx () #26 0x000055a4902d2d5a in PyEval_EvalCodeEx () #27 0x000055a4902d2899 in PyEval_EvalCode () #28 0x000055a4903030ef in ?? () #29 0x000055a4902fe2f2 in PyRun_FileExFlags () #30 0x000055a4902fdded in PyRun_SimpleFileExFlags () #31 0x000055a4902acc99 in Py_Main () #32 0x00007f45bf87fbf7 in __libc_start_main (main=0x55a4902ac710

Currently BinTuner focuses on quality. To make it a practical tool, it'd need to provide subsecond answers for small programs (say 1000 SLOC of C).

Please share your ideas here how could that be achieved without much compromising the quality of the outcomes.

1. Based on some fast heuristics (e.g. based on experience with "BinTuning" of previous apps) or source code available or some annotations in debug build or whatever, choose only a very small percentage (maybe less than 0.5%) of all instructions at interesting places and run BinTuner only on those.
2. Restrict or extend the operations the genetic algorithm can do leading to faster convergence.
3. Pre-prepare the binary in some way so that the initial phases the genetic algorithm currently does can be "skipped". In other words, shuffle with the instructions before the genetic algorithm starts in a way resembling the first few hundreds/thousands iterations/populations of the genetic algorithm but in one step. Yet in other words, prepare quickly more optimal default initial state of the input instead of relying on the genetic algorithm all the way down from the very beginning.
4. Techniques used by superoptimizers.

Hello, thanks for your execellent research and friendly released code. However, it seems you do not mention how to use this this tool to get the nearly optimal compiler flags to create binaries. May I ask whether you will detial the steps to use BinTuner? Thanks in advance.

Could I asked a question. GCC is used as the optimization option in the original code. The paper also conducted experiments on llvm optimization options. What specific optimization options are used?

Hi, I am compiling the SPEC using main_clang.py. Basically, I can generate the tuned binary, which is exciting :) After that, I am wondering how to set the termination condition. From the README and the main_clang.py, there are two paths monitoring the change of MAX value. After the MAX value remains unchanged 10 times, the increase is less than 0.05 for 10 times, or the sum of trigger time of these two conditions has reached 10, it would terminate. So what's the argument (for the command "python main.py 10" is 10) used for? I have changed it to 100 but the termination is still triggered by the above two paths. Should I change the termination threshold to higher, like 100, or 10 is enough? Thanks for your time and have a nice day!

Hello, I was interested to run the LLVM version of BinTuner but cannot find any related files for compiling via clang. is there an equivalent python file that compiles using clang similar to examples/gccflags ? Are there instructions on how to specify constraints for LLVM options and get the correct combination of options for clang?

Hello, I am very interested in your work. At present, I have encountered a problem. When I compiler benchmarks with GCC, I turn on the specified single or several optimization options based on -O0. Unfortunately, I found that the optimization options did not work. I noticed that in your paper it is mentioned that "In some cases, two options negatively influence each other, and turning on them together leads to a compilation error. Some other compilation options only work when a certain option has been activated. For example, -fpartial-inlining has any effect only when -finline-functions is turned on. To avoid compilation errors caused by such constraints, we manually translate them into logical first-order formulas offline after understanding the compiler manual.".

I can't find the constraint file in the code https://github.com/BinTuner/Dev , could you please share it with me? Also, do you have any good suggestions to solve this problem? Thank you!

Hello, thanks for your fancy research and your code. There is one question bother me: I mention that you only deal with GCC Flags i n the source code of BinTuner, but in the paper, you use both GCC and Clang.

Both LLVM and GCC explicitly specify a set of constraints between optimization ?ags, including adverse interactions and dependency relationships.

I check their website and find that it's clear in GCC, but nobody mentioned this kind of interaction in LLVM.

So does that mean there's no such interaction? Or it's just so hard for us to figure it out?

I would appreciate it if you could help me with this.