Hello and thank you for the outstanding course!

I built a benchmark with the use of make_benchmark_library_vs2019.cmd script first. But then, when I tried to build a warmup library, it failed. Am I missing anything? The full log is below:

PS C:\Work\perf-ninja\labs\misc\warmup> cmake -E make_directory build
PS C:\Work\perf-ninja\labs\misc\warmup> cd build
PS C:\Work\perf-ninja\labs\misc\warmup\build> cmake -DCMAKE_BUILD_TYPE=Release ..
-- Building for: Visual Studio 16 2019
-- Selecting Windows SDK version 10.0.19041.0 to target Windows 10.0.22000.
-- The C compiler identification is MSVC 19.29.30141.0
-- The CXX compiler identification is MSVC 19.29.30141.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/BuildTools/VC/Tools/MSVC/14.29.30133/bin/Hostx64/x64/cl.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/BuildTools/VC/Tools/MSVC/14.29.30133/bin/Hostx64/x64/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Performing Test SUPPORT_MSVC_AVX512
-- Performing Test SUPPORT_MSVC_AVX512 - Failed
-- Performing Test SUPPORT_MSVC_AVX2
-- Performing Test SUPPORT_MSVC_AVX2 - Success
-- Performing Test SUPPORT_MSVC_AVX
-- Performing Test SUPPORT_MSVC_AVX - Success
-- Configuring done
-- Generating done
-- Build files have been written to: C:/Work/perf-ninja/labs/misc/warmup/build
PS C:\Work\perf-ninja\labs\misc\warmup\build> cmake --build . --config Release --parallel 8
Microsoft (R) Build Engine version 16.11.2+f32259642 for .NET Framework
Copyright (C) Microsoft Corporation. All rights reserved.

  Checking Build System
  Building Custom Rule C:/Work/perf-ninja/labs/misc/warmup/CMakeLists.txt
  Building Custom Rule C:/Work/perf-ninja/labs/misc/warmup/CMakeLists.txt
  bench.cpp
  validate.cpp
C:\Work\perf-ninja\tools\benchmark\include\benchmark/benchmark.h(190,10): fatal error C1083: Cannot open include file: 'benchmark/export.h': No such file or d
irectory [C:\Work\perf-ninja\labs\misc\warmup\build\lab.vcxproj]
  solution.cpp
  Generating Code...
  solution.cpp
  Generating Code...
  validate.vcxproj -> C:\Work\perf-ninja\labs\misc\warmup\build\Release\validate.exe

Request I recieved: "It would be nice if there was a script or something to run the baseline program and the improved program to tell me how well I'm faring, instead of having to take note what was the latest score, what was the baseline score etc. It would reduce the ping-ponging with CI."

Hello Denis,

Following solution in the summary video failed to pass the CI for the data packing lab.

struct S {
  float d;
  long long l:16;
  int i:8;
  short s:7;
  bool b:1;

  bool operator<(const S &s) const { return this->i < s.i; }
};

In particular, short s:7; failed to pass the validation.

Scanning dependencies of target validateLab
Validation Failed. Value s is -28. Expected is 100 for intialization values 0 and 100
Validation Failed. Value s is -29. Expected is 99 for intialization values 1 and 99
Validation Failed. Value s is -28. Expected is 100 for intialization values 100 and 100
make[3]: *** [CMakeFiles/validateLab.dir/build.make:57: CMakeFiles/validateLab] Error 2
make[2]: *** [CMakeFiles/Makefile2:134: CMakeFiles/validateLab.dir/all] Error 2
make[1]: *** [CMakeFiles/Makefile2:141: CMakeFiles/validateLab.dir/rule] Error 2
make: *** [Makefile:144: validateLab] Error 2
Will benchmark the lab: memory_bound:data_packing
Build and Validate the solution
Prepare build directory - OK
CMake - OK
Build - OK
Validation - Failed

If we just set short s;, then the new version would be slower:

Benchmark                   Time             CPU      Time Old      Time New       CPU Old       CPU New
--------------------------------------------------------------------------------------------------------
bench1                   +0.0307         -0.0028           419           432           420           419
New version is slower. Submission for the lab memory_bound:data_packing failed.

Hello, I'm looking forward following the course, but I ran into an issue at the start:

When I do cmake --build . --config Release --parallel 8 for the warmup, I get the error: undefined reference to 'main'

With --verbose=1:

~/workspace/perf-ninja/labs/misc/warmup/build$ cmake --build . --config Release --parallel 8 -v
/snap/cmake/1005/bin/cmake -S/home/jhovan/workspace/perf-ninja/labs/misc/warmup -B/home/jhovan/workspace/perf-ninja/labs/misc/warmup/build --check-build-system CMakeFiles/Makefile.cmake 0
/snap/cmake/1005/bin/cmake -E cmake_progress_start /home/jhovan/workspace/perf-ninja/labs/misc/warmup/build/CMakeFiles /home/jhovan/workspace/perf-ninja/labs/misc/warmup/build//CMakeFiles/progress.marks
/usr/bin/make  -f CMakeFiles/Makefile2 all
make[1]: Entering directory '/home/jhovan/workspace/perf-ninja/labs/misc/warmup/build'
/usr/bin/make  -f CMakeFiles/lab.dir/build.make CMakeFiles/lab.dir/depend
/usr/bin/make  -f CMakeFiles/validate.dir/build.make CMakeFiles/validate.dir/depend
make[2]: Entering directory '/home/jhovan/workspace/perf-ninja/labs/misc/warmup/build'
cd /home/jhovan/workspace/perf-ninja/labs/misc/warmup/build && /snap/cmake/1005/bin/cmake -E cmake_depends "Unix Makefiles" /home/jhovan/workspace/perf-ninja/labs/misc/warmup /home/jhovan/workspace/perf-ninja/labs/misc/warmup /home/jhovan/workspace/perf-ninja/labs/misc/warmup/build /home/jhovan/workspace/perf-ninja/labs/misc/warmup/build /home/jhovan/workspace/perf-ninja/labs/misc/warmup/build/CMakeFiles/lab.dir/DependInfo.cmake --color=
make[2]: Entering directory '/home/jhovan/workspace/perf-ninja/labs/misc/warmup/build'
cd /home/jhovan/workspace/perf-ninja/labs/misc/warmup/build && /snap/cmake/1005/bin/cmake -E cmake_depends "Unix Makefiles" /home/jhovan/workspace/perf-ninja/labs/misc/warmup /home/jhovan/workspace/perf-ninja/labs/misc/warmup /home/jhovan/workspace/perf-ninja/labs/misc/warmup/build /home/jhovan/workspace/perf-ninja/labs/misc/warmup/build /home/jhovan/workspace/perf-ninja/labs/misc/warmup/build/CMakeFiles/validate.dir/DependInfo.cmake --color=
make[2]: Leaving directory '/home/jhovan/workspace/perf-ninja/labs/misc/warmup/build'
make[2]: Leaving directory '/home/jhovan/workspace/perf-ninja/labs/misc/warmup/build'
/usr/bin/make  -f CMakeFiles/lab.dir/build.make CMakeFiles/lab.dir/build
/usr/bin/make  -f CMakeFiles/validate.dir/build.make CMakeFiles/validate.dir/build
make[2]: Entering directory '/home/jhovan/workspace/perf-ninja/labs/misc/warmup/build'
make[2]: Nothing to be done for 'CMakeFiles/validate.dir/build'.
make[2]: Leaving directory '/home/jhovan/workspace/perf-ninja/labs/misc/warmup/build'
make[2]: Entering directory '/home/jhovan/workspace/perf-ninja/labs/misc/warmup/build'
[ 50%] Linking CXX executable lab
[ 66%] Built target validate
/snap/cmake/1005/bin/cmake -E cmake_link_script CMakeFiles/lab.dir/link.txt --verbose=1
/usr/bin/clang++-12 -O3 -ffast-math -march=native   -rdynamic CMakeFiles/lab.dir/bench.cpp.o CMakeFiles/lab.dir/solution.cpp.o -o lab  /usr/local/lib/libbenchmark.a -lpthread -lm 
/usr/bin/ld: /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/crt1.o: in function `_start':
(.text+0x24): undefined reference to `main'
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [CMakeFiles/lab.dir/build.make:114: lab] Error 1
make[2]: Leaving directory '/home/jhovan/workspace/perf-ninja/labs/misc/warmup/build'
make[1]: *** [CMakeFiles/Makefile2:87: CMakeFiles/lab.dir/all] Error 2
make[1]: Leaving directory '/home/jhovan/workspace/perf-ninja/labs/misc/warmup/build'
make: *** [Makefile:91: all] Error 2

Any help would be greatly appreciated!

Hi! I tried to push my private branch, but I get the following error :

ERROR: Permission to dendibakh/perf-ninja.git denied to goandrei.
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

Is the course still free to use?

Hi, thanks again for this wonderful repository.

Running the local experiments (as described in GetStarted.md) fails with the following output.

λ python3 tools/check_speedup.py -lab_path labs/misc/warmup -bench_lib_path tools/benchmark -num_runs 1 -v
Running: cmake -E make_directory baselineBuild0
Running: cmake -G Ninja -DCMAKE_CXX_COMPILER=clang++ -DCMAKE_C_COMPILER=clang -DCMAKE_BUILD_TYPE=Release  "labs/misc/warmup"
CMake Error: The source directory "/Users/tp5/code/perf-ninja/baselineBuild0/labs/misc/warmup" does not exist.
Specify --help for usage, or press the help button on the CMake GUI.
baseline: iteration 0 - Failed
Running: cmake -E make_directory solutionBuild0
Running: cmake -G Ninja -DCMAKE_CXX_COMPILER=clang++ -DCMAKE_C_COMPILER=clang -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_FLAGS=-DSOLUTION "labs/misc/warmup"
CMake Error: The source directory "/Users/tp5/code/perf-ninja/solutionBuild0/labs/misc/warmup" does not exist.
Specify --help for usage, or press the help button on the CMake GUI.
solution: iteration 0 - Failed
Running: python "tools/benchmark/tools/compare.py" benchmarks "/Users/tp5/code/perf-ninja/baselineBuild0/result.json" "/Users/tp5/code/perf-ninja/solutionBuild0/result.json"
usage: compare.py benchmarks [-h] test_baseline test_contender ...
compare.py benchmarks: error: argument test_baseline: can't open '/Users/tp5/code/perf-ninja/baselineBuild0/result.json': [Errno 2] No such file or directory: '/Users/tp5/code/perf-ninja/baselineBuild0/result.json'

The link to the source directory seems incorrect. I could fix the error locally on my machine by modifying the following

https://github.com/dendibakh/perf-ninja/blob/d2d6b12f734a897f95fb3e6a568cd2b6f5088263/tools/check_speedup.py#L62

to

    callWrapper("cmake -DCMAKE_CXX_COMPILER=clang++ -DCMAKE_C_COMPILER=clang -DCMAKE_BUILD_TYPE=Release " + cmakeFlags + " \"../" + labRootPath + "\"")

(I added a ../) before the labRootPath. Can you take a look if this bug is reproducible or if it's something that's incorrectly configured on my machine? Thanks.

Sometimes we have to deal with different speedups on different platforms, e.g. Linux vs. Windows. This may cause CI failures on one platform but success on another, which is annoying.

Example: measurements for two CI jobs for the same commit differ by 2x: Linux: https://github.com/dendibakh/perf-ninja/actions/runs/1382087200 Windows: https://github.com/dendibakh/perf-ninja/actions/runs/1382087204

Possible solutions:

• Define two sets of thresholds for Windows and Linux
• For big differences (like 2x) investigate the reason. Maybe something is not configured properly on one of the platforms.

This fixes issue #2. For GCC/Clang compilers it is safe to leave -march=native since they will automatically determine the target arch. But for MSVC I'm not sure since the default for x86 in MSVC is SSE2. For now, let's not touch it.

Trying to complete cmake -DCMAKE_BUILD_TYPE=Release .. command, result:

CMake Error at /opt/homebrew/Cellar/cmake/3.22.3/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
  Could NOT find Threads (missing: Threads_FOUND)
Call Stack (most recent call first):
  /opt/homebrew/Cellar/cmake/3.22.3/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:594 (_FPHSA_FAILURE_MESSAGE)
  /opt/homebrew/Cellar/cmake/3.22.3/share/cmake/Modules/FindThreads.cmake:238 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
  /opt/homebrew/Cellar/cmake/3.22.3/share/cmake/Modules/CMakeFindDependencyMacro.cmake:47 (find_package)
  /usr/local/lib/cmake/benchmark/benchmarkConfig.cmake:12 (find_dependency)
  /Users/user/Projects/perf-ninja/tools/labs.cmake:54 (find_package)
  CMakeLists.txt:6 (include)

Possible Solution: Add

+IF(APPLE)
    set(CMAKE_THREAD_LIBS_INIT "-lpthread")
    set(CMAKE_HAVE_THREADS_LIBRARY 1)
    set(CMAKE_USE_WIN32_THREADS_INIT 0)
    set(CMAKE_USE_PTHREADS_INIT 1)
    set(THREADS_PREFER_PTHREAD_FLAG ON)
ENDIF()

in warmup/CMakeLists.txt just after cmake_minimum_required .. After that I had a problem:

$ cmake --build . --config Release --parallel 8
[ 66%] Building CXX object CMakeFiles/lab.dir/bench.cpp.o
[ 66%] Building CXX object CMakeFiles/validate.dir/validate.cpp.o
[ 66%] Building CXX object CMakeFiles/validate.dir/solution.cpp.o
[ 66%] Building CXX object CMakeFiles/lab.dir/solution.cpp.o
clang: error: the clang compiler does not support '-march=native'
clang: error: the clang compiler does not support '-march=native'
clang: error: the clang compiler does not support '-march=native'
clang: error: the clang compiler does not support '-march=native'

Possible solution: Change in tools/labs.cmake

-  set(CMAKE_C_FLAGS "-O3 -ffast-math -march=native ${CMAKE_C_FLAGS}")
+  set(CMAKE_C_FLAGS "-O3 -ffast-math  ${CMAKE_C_FLAGS}")

Note: do not commit this changes.

It would be nice for visitors to this repo to know what they're expected to already be familiar with, as well as necessary "zeroth" steps to take to meet those requirements.

Hi,

I followed the guides attached in README.md and completed the setup. But to test if the setup is working correctly I am trying to run the given command in the warmup lab folder but it fails at the third command cmake -DCMAKE_BUILD_TYPE=Release ... Following is the attached error and log files.

Machine: Mac M1 Ventura

-- The C compiler identification is Clang 15.0.3
-- The CXX compiler identification is Clang 15.0.3
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - failed
-- Check for working C compiler: /opt/homebrew/opt/llvm/bin/clang
-- Check for working C compiler: /opt/homebrew/opt/llvm/bin/clang - broken
CMake Error at /opt/homebrew/Cellar/cmake/3.22.2/share/cmake/Modules/CMakeTestCCompiler.cmake:69 (message):
  The C compiler

    "/opt/homebrew/opt/llvm/bin/clang"

  is not able to compile a simple test program.

  It fails with the following output:

    Change Dir: /Users/tuba/Playgrounds/Perfomance_Sandbox/perf-ninja/labs/misc/warmup/build/CMakeFiles/CMakeTmp
    
    Run Build Command(s):/opt/homebrew/bin/gmake -f Makefile cmTC_7ee35/fast && /opt/homebrew/bin/gmake  -f CMakeFiles/cmTC_7ee35.dir/build.make CMakeFiles/cmTC_7ee35.dir/build
    gmake[1]: Entering directory '/Users/tuba/Playgrounds/Perfomance_Sandbox/perf-ninja/labs/misc/warmup/build/CMakeFiles/CMakeTmp'
    Building C object CMakeFiles/cmTC_7ee35.dir/testCCompiler.c.o
    /opt/homebrew/opt/llvm/bin/clang   -arch arm64 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.3.sdk -MD -MT CMakeFiles/cmTC_7ee35.dir/testCCompiler.c.o -MF CMakeFiles/cmTC_7ee35.dir/testCCompiler.c.o.d -o CMakeFiles/cmTC_7ee35.dir/testCCompiler.c.o -c /Users/tuba/Playgrounds/Perfomance_Sandbox/perf-ninja/labs/misc/warmup/build/CMakeFiles/CMakeTmp/testCCompiler.c
    Linking C executable cmTC_7ee35
    /opt/homebrew/Cellar/cmake/3.22.2/bin/cmake -E cmake_link_script CMakeFiles/cmTC_7ee35.dir/link.txt --verbose=1
    /opt/homebrew/opt/llvm/bin/clang  -arch arm64 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.3.sdk -Wl,-search_paths_first -Wl,-headerpad_max_install_names CMakeFiles/cmTC_7ee35.dir/testCCompiler.c.o -o cmTC_7ee35 
    ld: library not found for -lSystem
    clang-15: error: linker command failed with exit code 1 (use -v to see invocation)
    gmake[1]: *** [CMakeFiles/cmTC_7ee35.dir/build.make:100: cmTC_7ee35] Error 1
    gmake[1]: Leaving directory '/Users/tuba/Playgrounds/Perfomance_Sandbox/perf-ninja/labs/misc/warmup/build/CMakeFiles/CMakeTmp'
    gmake: *** [Makefile:127: cmTC_7ee35/fast] Error 2
    
    

  

  CMake will not be able to correctly generate this project.
Call Stack (most recent call first):
  CMakeLists.txt:3 (project)


-- Configuring incomplete, errors occurred!
See also "/Users/tuba/Playgrounds/Perfomance_Sandbox/perf-ninja/labs/misc/warmup/build/CMakeFiles/CMakeOutput.log".
See also "/Users/tuba/Playgrounds/Perfomance_Sandbox/perf-ninja/labs/misc/warmup/build/CMakeFiles/CMakeError.log".

Compiling the C compiler identification source file "CMakeCCompilerId.c" failed.
Compiler: /opt/homebrew/opt/llvm/bin/clang 
Build flags: 
Id flags:  

The output was:
1
ld: library not found for -lSystem
clang-15: error: linker command failed with exit code 1 (use -v to see invocation)


Compiling the CXX compiler identification source file "CMakeCXXCompilerId.cpp" failed.
Compiler: /opt/homebrew/opt/llvm/bin/clang++ 
Build flags: 
Id flags:  

The output was:
1
ld: library not found for -lc++
clang-15: error: linker command failed with exit code 1 (use -v to see invocation)


Detecting C compiler ABI info failed to compile with the following output:
Change Dir: /Users/tuba/Playgrounds/Perfomance_Sandbox/perf-ninja/labs/misc/warmup/build/CMakeFiles/CMakeTmp

Run Build Command(s):/opt/homebrew/bin/gmake -f Makefile cmTC_86d51/fast && /opt/homebrew/bin/gmake  -f CMakeFiles/cmTC_86d51.dir/build.make CMakeFiles/cmTC_86d51.dir/build
gmake[1]: Entering directory '/Users/tuba/Playgrounds/Perfomance_Sandbox/perf-ninja/labs/misc/warmup/build/CMakeFiles/CMakeTmp'
Building C object CMakeFiles/cmTC_86d51.dir/CMakeCCompilerABI.c.o
/opt/homebrew/opt/llvm/bin/clang   -arch arm64 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.3.sdk   -v -Wl,-v -MD -MT CMakeFiles/cmTC_86d51.dir/CMakeCCompilerABI.c.o -MF CMakeFiles/cmTC_86d51.dir/CMakeCCompilerABI.c.o.d -o CMakeFiles/cmTC_86d51.dir/CMakeCCompilerABI.c.o -c /opt/homebrew/Cellar/cmake/3.22.2/share/cmake/Modules/CMakeCCompilerABI.c
Homebrew clang version 15.0.3
Target: arm64-apple-darwin22.1.0
Thread model: posix
InstalledDir: /opt/homebrew/opt/llvm/bin
clang-15: warning: -Wl,-v: 'linker' input unused [-Wunused-command-line-argument]
 (in-process)
 "/opt/homebrew/Cellar/llvm/15.0.3/bin/clang-15" -cc1 -triple arm64-apple-macosx12.3.0 -Wundef-prefix=TARGET_OS_ -Werror=undef-prefix -Wdeprecated-objc-isa-usage -Werror=deprecated-objc-isa-usage -emit-obj -mrelax-all --mrelax-relocations -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name CMakeCCompilerABI.c -mrelocation-model pic -pic-level 2 -mframe-pointer=non-leaf -ffp-contract=on -fno-rounding-math -funwind-tables=2 -target-sdk-version=12.3 -fcompatibility-qualified-id-block-type-checking -fvisibility-inlines-hidden-static-local-var -target-cpu apple-m1 -target-feature +v8.5a -target-feature +crc -target-feature +lse -target-feature +rdm -target-feature +crypto -target-feature +dotprod -target-feature +fp-armv8 -target-feature +neon -target-feature +fp16fml -target-feature +ras -target-feature +rcpc -target-feature +zcm -target-feature +zcz -target-feature +fullfp16 -target-feature +sm4 -target-feature +sha3 -target-feature +sha2 -target-feature +aes -target-abi darwinpcs -fallow-half-arguments-and-returns -mllvm -treat-scalable-fixed-error-as-warning -debugger-tuning=lldb -target-linker-version 820.1 -v -fcoverage-compilation-dir=/Users/tuba/Playgrounds/Perfomance_Sandbox/perf-ninja/labs/misc/warmup/build/CMakeFiles/CMakeTmp -resource-dir /opt/homebrew/Cellar/llvm/15.0.3/lib/clang/15.0.3 -dependency-file CMakeFiles/cmTC_86d51.dir/CMakeCCompilerABI.c.o.d -MT CMakeFiles/cmTC_86d51.dir/CMakeCCompilerABI.c.o -sys-header-deps -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.3.sdk -internal-isystem /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.3.sdk/usr/local/include -internal-isystem /opt/homebrew/Cellar/llvm/15.0.3/lib/clang/15.0.3/include -internal-externc-isystem /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.3.sdk/usr/include -fdebug-compilation-dir=/Users/tuba/Playgrounds/Perfomance_Sandbox/perf-ninja/labs/misc/warmup/build/CMakeFiles/CMakeTmp -ferror-limit 19 -stack-protector 1 -fblocks -fencode-extended-block-signature -fregister-global-dtors-with-atexit -fgnuc-version=4.2.1 -fmax-type-align=16 -D__GCC_HAVE_DWARF2_CFI_ASM=1 -o CMakeFiles/cmTC_86d51.dir/CMakeCCompilerABI.c.o -x c /opt/homebrew/Cellar/cmake/3.22.2/share/cmake/Modules/CMakeCCompilerABI.c
clang -cc1 version 15.0.3 based upon LLVM 15.0.3 default target arm64-apple-darwin22.1.0
ignoring nonexistent directory "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.3.sdk/usr/local/include"
ignoring nonexistent directory "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.3.sdk/Library/Frameworks"
#include "..." search starts here:
#include <...> search starts here:
 /opt/homebrew/Cellar/llvm/15.0.3/lib/clang/15.0.3/include
 /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.3.sdk/usr/include
 /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.3.sdk/System/Library/Frameworks (framework directory)
End of search list.
Linking C executable cmTC_86d51
/opt/homebrew/Cellar/cmake/3.22.2/bin/cmake -E cmake_link_script CMakeFiles/cmTC_86d51.dir/link.txt --verbose=1
/opt/homebrew/opt/llvm/bin/clang  -arch arm64 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.3.sdk -Wl,-search_paths_first -Wl,-headerpad_max_install_names  -v -Wl,-v CMakeFiles/cmTC_86d51.dir/CMakeCCompilerABI.c.o -o cmTC_86d51 
Homebrew clang version 15.0.3
Target: arm64-apple-darwin22.1.0
Thread model: posix
InstalledDir: /opt/homebrew/opt/llvm/bin
 "/usr/bin/ld" -demangle -lto_library /opt/homebrew/Cellar/llvm/15.0.3/lib/libLTO.dylib -dynamic -arch arm64 -platform_version macos 12.3.0 12.3 -syslibroot /Library/Developer/CommandLineTools/SDKs/MacOSX13.sdk -o cmTC_86d51 -search_paths_first -headerpad_max_install_names -v CMakeFiles/cmTC_86d51.dir/CMakeCCompilerABI.c.o -lSystem /opt/homebrew/Cellar/llvm/15.0.3/lib/clang/15.0.3/lib/darwin/libclang_rt.osx.a
@(#)PROGRAM:ld  PROJECT:ld64-819.6
BUILD 21:47:32 Sep 13 2022
configured to support archs: armv6 armv7 armv7s arm64 arm64e arm64_32 i386 x86_64 x86_64h armv6m armv7k armv7m armv7em
Library search paths:
Framework search paths:
ld: library not found for -lSystem
clang-15: error: linker command failed with exit code 1 (use -v to see invocation)
gmake[1]: *** [CMakeFiles/cmTC_86d51.dir/build.make:100: cmTC_86d51] Error 1
gmake[1]: Leaving directory '/Users/tuba/Playgrounds/Perfomance_Sandbox/perf-ninja/labs/misc/warmup/build/CMakeFiles/CMakeTmp'
gmake: *** [Makefile:127: cmTC_86d51/fast] Error 2




Determining if the C compiler works failed with the following output:
Change Dir: /Users/tuba/Playgrounds/Perfomance_Sandbox/perf-ninja/labs/misc/warmup/build/CMakeFiles/CMakeTmp

Run Build Command(s):/opt/homebrew/bin/gmake -f Makefile cmTC_9dee8/fast && /opt/homebrew/bin/gmake  -f CMakeFiles/cmTC_9dee8.dir/build.make CMakeFiles/cmTC_9dee8.dir/build
gmake[1]: Entering directory '/Users/tuba/Playgrounds/Perfomance_Sandbox/perf-ninja/labs/misc/warmup/build/CMakeFiles/CMakeTmp'
Building C object CMakeFiles/cmTC_9dee8.dir/testCCompiler.c.o
/opt/homebrew/opt/llvm/bin/clang   -arch arm64 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.3.sdk -MD -MT CMakeFiles/cmTC_9dee8.dir/testCCompiler.c.o -MF CMakeFiles/cmTC_9dee8.dir/testCCompiler.c.o.d -o CMakeFiles/cmTC_9dee8.dir/testCCompiler.c.o -c /Users/tuba/Playgrounds/Perfomance_Sandbox/perf-ninja/labs/misc/warmup/build/CMakeFiles/CMakeTmp/testCCompiler.c
Linking C executable cmTC_9dee8
/opt/homebrew/Cellar/cmake/3.22.2/bin/cmake -E cmake_link_script CMakeFiles/cmTC_9dee8.dir/link.txt --verbose=1
/opt/homebrew/opt/llvm/bin/clang  -arch arm64 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.3.sdk -Wl,-search_paths_first -Wl,-headerpad_max_install_names CMakeFiles/cmTC_9dee8.dir/testCCompiler.c.o -o cmTC_9dee8 
ld: library not found for -lSystem
clang-15: error: linker command failed with exit code 1 (use -v to see invocation)
gmake[1]: *** [CMakeFiles/cmTC_9dee8.dir/build.make:100: cmTC_9dee8] Error 1
gmake[1]: Leaving directory '/Users/tuba/Playgrounds/Perfomance_Sandbox/perf-ninja/labs/misc/warmup/build/CMakeFiles/CMakeTmp'
gmake: *** [Makefile:127: cmTC_9dee8/fast] Error 2



Compiling the C compiler identification source file "CMakeCCompilerId.c" failed.
Compiler: /opt/homebrew/opt/llvm/bin/clang 
Build flags: 
Id flags:  

The output was:
1
ld: library not found for -lSystem
clang-15: error: linker command failed with exit code 1 (use -v to see invocation)


Compiling the CXX compiler identification source file "CMakeCXXCompilerId.cpp" failed.
Compiler: /opt/homebrew/opt/llvm/bin/clang++ 
Build flags: 
Id flags:  

The output was:
1
ld: library not found for -lc++
clang-15: error: linker command failed with exit code 1 (use -v to see invocation)


Detecting C compiler ABI info failed to compile with the following output:
Change Dir: /Users/tuba/Playgrounds/Perfomance_Sandbox/perf-ninja/labs/misc/warmup/build/CMakeFiles/CMakeTmp

Run Build Command(s):/opt/homebrew/bin/gmake -f Makefile cmTC_a057a/fast && /opt/homebrew/bin/gmake  -f CMakeFiles/cmTC_a057a.dir/build.make CMakeFiles/cmTC_a057a.dir/build
gmake[1]: Entering directory '/Users/tuba/Playgrounds/Perfomance_Sandbox/perf-ninja/labs/misc/warmup/build/CMakeFiles/CMakeTmp'
Building C object CMakeFiles/cmTC_a057a.dir/CMakeCCompilerABI.c.o
/opt/homebrew/opt/llvm/bin/clang   -arch arm64 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.3.sdk   -v -Wl,-v -MD -MT CMakeFiles/cmTC_a057a.dir/CMakeCCompilerABI.c.o -MF CMakeFiles/cmTC_a057a.dir/CMakeCCompilerABI.c.o.d -o CMakeFiles/cmTC_a057a.dir/CMakeCCompilerABI.c.o -c /opt/homebrew/Cellar/cmake/3.22.2/share/cmake/Modules/CMakeCCompilerABI.c
Homebrew clang version 15.0.3
Target: arm64-apple-darwin22.1.0
Thread model: posix
InstalledDir: /opt/homebrew/opt/llvm/bin
clang-15: warning: -Wl,-v: 'linker' input unused [-Wunused-command-line-argument]
 (in-process)
 "/opt/homebrew/Cellar/llvm/15.0.3/bin/clang-15" -cc1 -triple arm64-apple-macosx12.3.0 -Wundef-prefix=TARGET_OS_ -Werror=undef-prefix -Wdeprecated-objc-isa-usage -Werror=deprecated-objc-isa-usage -emit-obj -mrelax-all --mrelax-relocations -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name CMakeCCompilerABI.c -mrelocation-model pic -pic-level 2 -mframe-pointer=non-leaf -ffp-contract=on -fno-rounding-math -funwind-tables=2 -target-sdk-version=12.3 -fcompatibility-qualified-id-block-type-checking -fvisibility-inlines-hidden-static-local-var -target-cpu apple-m1 -target-feature +v8.5a -target-feature +crc -target-feature +lse -target-feature +rdm -target-feature +crypto -target-feature +dotprod -target-feature +fp-armv8 -target-feature +neon -target-feature +fp16fml -target-feature +ras -target-feature +rcpc -target-feature +zcm -target-feature +zcz -target-feature +fullfp16 -target-feature +sm4 -target-feature +sha3 -target-feature +sha2 -target-feature +aes -target-abi darwinpcs -fallow-half-arguments-and-returns -mllvm -treat-scalable-fixed-error-as-warning -debugger-tuning=lldb -target-linker-version 820.1 -v -fcoverage-compilation-dir=/Users/tuba/Playgrounds/Perfomance_Sandbox/perf-ninja/labs/misc/warmup/build/CMakeFiles/CMakeTmp -resource-dir /opt/homebrew/Cellar/llvm/15.0.3/lib/clang/15.0.3 -dependency-file CMakeFiles/cmTC_a057a.dir/CMakeCCompilerABI.c.o.d -MT CMakeFiles/cmTC_a057a.dir/CMakeCCompilerABI.c.o -sys-header-deps -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.3.sdk -internal-isystem /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.3.sdk/usr/local/include -internal-isystem /opt/homebrew/Cellar/llvm/15.0.3/lib/clang/15.0.3/include -internal-externc-isystem /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.3.sdk/usr/include -fdebug-compilation-dir=/Users/tuba/Playgrounds/Perfomance_Sandbox/perf-ninja/labs/misc/warmup/build/CMakeFiles/CMakeTmp -ferror-limit 19 -stack-protector 1 -fblocks -fencode-extended-block-signature -fregister-global-dtors-with-atexit -fgnuc-version=4.2.1 -fmax-type-align=16 -D__GCC_HAVE_DWARF2_CFI_ASM=1 -o CMakeFiles/cmTC_a057a.dir/CMakeCCompilerABI.c.o -x c /opt/homebrew/Cellar/cmake/3.22.2/share/cmake/Modules/CMakeCCompilerABI.c
clang -cc1 version 15.0.3 based upon LLVM 15.0.3 default target arm64-apple-darwin22.1.0
ignoring nonexistent directory "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.3.sdk/usr/local/include"
ignoring nonexistent directory "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.3.sdk/Library/Frameworks"
#include "..." search starts here:
#include <...> search starts here:
 /opt/homebrew/Cellar/llvm/15.0.3/lib/clang/15.0.3/include
 /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.3.sdk/usr/include
 /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.3.sdk/System/Library/Frameworks (framework directory)
End of search list.
Linking C executable cmTC_a057a
/opt/homebrew/Cellar/cmake/3.22.2/bin/cmake -E cmake_link_script CMakeFiles/cmTC_a057a.dir/link.txt --verbose=1
/opt/homebrew/opt/llvm/bin/clang  -arch arm64 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.3.sdk -Wl,-search_paths_first -Wl,-headerpad_max_install_names  -v -Wl,-v CMakeFiles/cmTC_a057a.dir/CMakeCCompilerABI.c.o -o cmTC_a057a 
Homebrew clang version 15.0.3
Target: arm64-apple-darwin22.1.0
Thread model: posix
InstalledDir: /opt/homebrew/opt/llvm/bin
 "/usr/bin/ld" -demangle -lto_library /opt/homebrew/Cellar/llvm/15.0.3/lib/libLTO.dylib -dynamic -arch arm64 -platform_version macos 12.3.0 12.3 -syslibroot /Library/Developer/CommandLineTools/SDKs/MacOSX13.sdk -o cmTC_a057a -search_paths_first -headerpad_max_install_names -v CMakeFiles/cmTC_a057a.dir/CMakeCCompilerABI.c.o -lSystem /opt/homebrew/Cellar/llvm/15.0.3/lib/clang/15.0.3/lib/darwin/libclang_rt.osx.a
@(#)PROGRAM:ld  PROJECT:ld64-819.6
BUILD 21:47:32 Sep 13 2022
configured to support archs: armv6 armv7 armv7s arm64 arm64e arm64_32 i386 x86_64 x86_64h armv6m armv7k armv7m armv7em
Library search paths:
Framework search paths:
ld: library not found for -lSystem
clang-15: error: linker command failed with exit code 1 (use -v to see invocation)
gmake[1]: *** [CMakeFiles/cmTC_a057a.dir/build.make:100: cmTC_a057a] Error 1
gmake[1]: Leaving directory '/Users/tuba/Playgrounds/Perfomance_Sandbox/perf-ninja/labs/misc/warmup/build/CMakeFiles/CMakeTmp'
gmake: *** [Makefile:127: cmTC_a057a/fast] Error 2




Determining if the C compiler works failed with the following output:
Change Dir: /Users/tuba/Playgrounds/Perfomance_Sandbox/perf-ninja/labs/misc/warmup/build/CMakeFiles/CMakeTmp

Run Build Command(s):/opt/homebrew/bin/gmake -f Makefile cmTC_7ee35/fast && /opt/homebrew/bin/gmake  -f CMakeFiles/cmTC_7ee35.dir/build.make CMakeFiles/cmTC_7ee35.dir/build
gmake[1]: Entering directory '/Users/tuba/Playgrounds/Perfomance_Sandbox/perf-ninja/labs/misc/warmup/build/CMakeFiles/CMakeTmp'
Building C object CMakeFiles/cmTC_7ee35.dir/testCCompiler.c.o
/opt/homebrew/opt/llvm/bin/clang   -arch arm64 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.3.sdk -MD -MT CMakeFiles/cmTC_7ee35.dir/testCCompiler.c.o -MF CMakeFiles/cmTC_7ee35.dir/testCCompiler.c.o.d -o CMakeFiles/cmTC_7ee35.dir/testCCompiler.c.o -c /Users/tuba/Playgrounds/Perfomance_Sandbox/perf-ninja/labs/misc/warmup/build/CMakeFiles/CMakeTmp/testCCompiler.c
Linking C executable cmTC_7ee35
/opt/homebrew/Cellar/cmake/3.22.2/bin/cmake -E cmake_link_script CMakeFiles/cmTC_7ee35.dir/link.txt --verbose=1
/opt/homebrew/opt/llvm/bin/clang  -arch arm64 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.3.sdk -Wl,-search_paths_first -Wl,-headerpad_max_install_names CMakeFiles/cmTC_7ee35.dir/testCCompiler.c.o -o cmTC_7ee35 
ld: library not found for -lSystem
clang-15: error: linker command failed with exit code 1 (use -v to see invocation)
gmake[1]: *** [CMakeFiles/cmTC_7ee35.dir/build.make:100: cmTC_7ee35] Error 1
gmake[1]: Leaving directory '/Users/tuba/Playgrounds/Perfomance_Sandbox/perf-ninja/labs/misc/warmup/build/CMakeFiles/CMakeTmp'
gmake: *** [Makefile:127: cmTC_7ee35/fast] Error 2

Could you please guide me here on how to resolve? I have tried Google search results for hours but didn't help.

The lab shouldn't necessarily be very complicated. The goal is to show students how to offload computations to a GPU. SYCL + Intel's DPCPP compiler should be good candidates.

Currently the following labs don't have solutions for Mac M1 platform:

  ["memory_bound"]["huge_pages_1"]          - need to check huge pages on Mac
  ["misc"]["io_opt1"]                       - mmap on Mac
  ["core_bound"]["compiler_intrinsics_1"]   - NEON version
  ["core_bound"]["compiler_intrinsics_2"]   - NEON version

This prevents automated benchmarking of their speedups in CI.

warning: profile data may be out of date: of 97 functions, 1 has mismatched data that will be ignored [-Wprofile-instr-out-of-date]

So far it didn't generate a performance problem, but it's definitely something to watch out for in the future.

Still up for discussion...

There was an awesome job done by @grahamking of porting most of the labs into Rust: perf-ninja-rs. We can think about merging two projects to have rust labs automatically benchmarked.