Tattoo - System for automating the Gentoo arch testing process

Related tags

Testing tattoo
Overview

Naming origin

Well, naming things is very hard. Thankfully we have an excellent team at Gentoo capable of naming things - The Infra team. Alec Warner (antarus) was kind enough to decide on the name "tattoo", which came from combining "tatt" (package testing tool used here) and "Gentoo".

Setup

Developer's own machine

  1. Install the following dependencies:
    • app-portage/nattka
    • net-misc/openssh
  2. Select the directory from which we will work. Always run and set files inside this directory. It can be the source files directory.
  3. Create a ssh_config file using the ssh_config.in template. Fill your username, select the wanted path for socket on every host, and define the various hosts and theirs hostname.
  4. Add an environment variable (for example into .bashrc) named ARCHTESTER_BUGZILLA_APIKEY which equals to an API key from bugzilla.

Remote machine - machine manager

  1. Install the following dependencies:
    • dev-lang/python[sqlite(+)] (tested on >=3.9)
    • app-portage/nattka
  2. Select the directory from which we will work. Always run and set files inside this directory. It can be the source files directory. It must be the same directory as set in the ssh_config directory (by default ~/tattoo)

Remote machine - testing container

  1. Install the following dependencies:
    • app-portage/nattka
    • app-portage/tatt
  2. Make sure the working directory of the machine manager is mount bound into the container. The mount destination inside the container would be the working directory for the testing container.
  3. Create a corresponding ~/.tatt file inside container, for example (IMPORTANT: replace ARCH)
    arch=arm64
    emergeopts="--autounmask --autounmask-continue --autounmask-write"
    repodir="/var/db/repos/gentoo/"
    ignoreprefix="elibc_","video_cards_","linguas_","python_targets_","python_single_target_","kdeenablefinal","test","debug","qemu_user_","qemu_softmmu_","libressl","static-libs","systemd","sdjournal","eloginid","doc","ruby_targets_"
    buildlogdir=/root/logs
    rdeps=0
    usecombis=1
    

Running and using

Load all remote machines

  1. Run in selected directory (for example ~/tattoo) the command ./manager.py.
    • Inside this directory a file named tattoo.socket will be created. Through this socket all communication will occur.
    • A SQLite DB named tattoo.db will hold all successes and failures of test runs.
  2. In every container on that machine, run the command ./tester.py -n [NAME] -a [ARCH] -j [JOBS] where NAME is just a nice textual name to know which container did what, ARCH is the arch to test, with amd64 for stable bugs, and ~arm for keyword bugs. JOBS is the maximal concurrent testing jobs.
    • This command must be ran in the mount bound dir from manager, where the tattoo.socket is created (so it can communicate).
  3. Check that the manager logs all containers connecting to it.

Control from developer's own machine

  1. Connect to remote servers listed in ssh_config using ./controller.py -c. Various sockets are created inside /tmp/tattoo/ directory
  2. Send specific bugs using ./controller.py -b {NUM} {NUM} ... or initiate full scan for open bugs per arch using ./controller.py -s
  3. You can use "follower" mode to get live logs from all machines and testers using ./controller.py follower
  4. When bugs are ready, use ./controller.py fetch -n to view all done bugs, but in dry-run mode (no update for bugzilla, and no update last-seen bugs). Btw, the output corresponds to sam's at-commit script.
  5. When ready to apply, run ./controller.py fetch -ar -d [REPO] where REPO is the ::gentoo repo to apply on it the commits. This command also un-CC and closes bugs for what passed. After success, it saves in small file the last seen bugs, so you don't try to reapply them.
  6. From REPO push the commits (if you are unlucky, git pull --rebase before)
  7. Send, fetch, apply how much you want
  8. Disconnect from all using ./controller.py -d
You might also like...
Sixpack is a language-agnostic a/b-testing framework

Sixpack Sixpack is a framework to enable A/B testing across multiple programming languages. It does this by exposing a simple API for client libraries

Automatically mock your HTTP interactions to simplify and speed up testing

VCR.py 📼 This is a Python version of Ruby's VCR library. Source code https://github.com/kevin1024/vcrpy Documentation https://vcrpy.readthedocs.io/ R

fsociety Hacking Tools Pack – A Penetration Testing Framework
fsociety Hacking Tools Pack – A Penetration Testing Framework

Fsociety Hacking Tools Pack A Penetration Testing Framework, you will have every script that a hacker needs. Works with Python 2. For a Python 3 versi

Scalable user load testing tool written in Python
Scalable user load testing tool written in Python

Locust Locust is an easy to use, scriptable and scalable performance testing tool. You define the behaviour of your users in regular Python code, inst

Automatically mock your HTTP interactions to simplify and speed up testing

VCR.py 📼 This is a Python version of Ruby's VCR library. Source code https://github.com/kevin1024/vcrpy Documentation https://vcrpy.readthedocs.io/ R

Language-agnostic HTTP API Testing Tool
Language-agnostic HTTP API Testing Tool

Dredd — HTTP API Testing Framework Dredd is a language-agnostic command-line tool for validating API description document against backend implementati

Web testing library for Robot Framework

SeleniumLibrary Contents Introduction Keyword Documentation Installation Browser drivers Usage Extending SeleniumLibrary Community Versions History In

✅ Python web automation and testing. 🚀 Fast, easy, reliable. 💠
✅ Python web automation and testing. 🚀 Fast, easy, reliable. 💠

Build fast, reliable, end-to-end tests. SeleniumBase is a Python framework for web automation, end-to-end testing, and more. Tests are run with "pytes

A command-line tool and Python library and Pytest plugin for automated testing of RESTful APIs, with a simple, concise and flexible YAML-based syntax

1.0 Release See here for details about breaking changes with the upcoming 1.0 release: https://github.com/taverntesting/tavern/issues/495 Easier API t

Owner
Arthur Zamarin
Arthur Zamarin
Main purpose of this project is to provide the service to automate the API testing process

PPTester project Main purpose of this project is to provide the service to automate the API testing process. In order to deploy this service use you s

null 4 Dec 16, 2021
PENBUD is penetration testing buddy which helps you in penetration testing by making various important tools interactive.

penbud - Penetration Tester Buddy PENBUD is penetration testing buddy which helps you in penetration testing by making various important tools interac

Himanshu Shukla 15 Feb 1, 2022
pytest plugin for distributed testing and loop-on-failures testing modes.

xdist: pytest distributed testing plugin The pytest-xdist plugin extends pytest with some unique test execution modes: test run parallelization: if yo

pytest-dev 1.1k Dec 30, 2022
PacketPy is an open-source solution for stress testing network devices using different testing methods

PacketPy About PacketPy is an open-source solution for stress testing network devices using different testing methods. Currently, there are only two c

null 4 Sep 22, 2022
A testing system for catching visual regressions in Web applications.

Huxley Watches you browse, takes screenshots, tells you when they change Huxley is a test-like system for catching visual regressions in Web applicati

Facebook Archive 4.1k Nov 30, 2022
FFPuppet is a Python module that automates browser process related tasks to aid in fuzzing

FFPuppet FFPuppet is a Python module that automates browser process related tasks to aid in fuzzing. Happy bug hunting! Are you fuzzing the browser? G

Mozilla Fuzzing Security 24 Oct 25, 2022
Hypothesis is a powerful, flexible, and easy to use library for property-based testing.

Hypothesis Hypothesis is a family of testing libraries which let you write tests parametrized by a source of examples. A Hypothesis implementation the

Hypothesis 6.4k Jan 5, 2023
Generic automation framework for acceptance testing and RPA

Robot Framework Introduction Installation Example Usage Documentation Support and contact Contributing License Introduction Robot Framework is a gener

Robot Framework 7.7k Jan 7, 2023
Scalable user load testing tool written in Python

Locust Locust is an easy to use, scriptable and scalable performance testing tool. You define the behaviour of your users in regular Python code, inst

Locust.io 20.4k Jan 4, 2023
A modern API testing tool for web applications built with Open API and GraphQL specifications.

Schemathesis Schemathesis is a modern API testing tool for web applications built with Open API and GraphQL specifications. It reads the application s

Schemathesis.io 1.6k Jan 6, 2023