Pylint also with Python3.10. Other workflows (mypy, pytest...) are still run against Python3.9. Suggested at https://github.com/google/scaaml/pull/87#discussion_r917855371

@jmichelp Should we also bump Python version for other workflows?

Python notebooks contain typos.

There are some typos which are a part of the api or need more attention (bibtex name burzteindc):

burzteindc chipwispher comparaison filepattern

scaaml.aes still contains non-word variables.

Part of issue #38

Because we're using google codestyle in yapf, pylint configuration file has been updated to the matching one in Google public style guide: https://github.com/google/styleguide/blob/gh-pages/pyguide.md

Rationale behind splitting workflows is the granularity of branch protection which is at the yaml file level. It's also not so bad for maintenance considering it's a python project.

Currently the leak mapping code don't work when unicorn is not installed but don't crash and as a result silently fail

[Emulating target]
Starting emulation
Traceback (most recent call last):
  File "_ctypes/callbacks.c", line 234, in 'calling callback function'
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/unicorn-1.0.2rc4-py3.6.egg/unicorn/unicorn.py", line 479, in _hookcode_cb
  File "/Users/elieb/git/scaaml/scaaml/scald/tracer/devices/rainbow/generics/cortexm.py", line 109, in block_handler
    self.base_block_handler(address)
  File "/Users/elieb/git/scaaml/scaaml/scald/tracer/devices/rainbow/rainbow.py", line 414, in base_block_handler
    r = self.stubbed_functions[f](self)
  File "/Users/elieb/git/scaaml/scaaml/scald/tracer/chipwhisperer_leakage_automaton.py", line 194, in trigger_high
    self.trigger_hook = e.emu.hook_add(uc.UC_HOOK_MEM_WRITE,
NameError: name 'uc' is not defined
Traceback (most recent call last):
  File "_ctypes/callbacks.c", line 234, in 'calling callback function'
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/unicorn-1.0.2rc4-py3.6.egg/unicorn/unicorn.py", line 479, in _hookcode_cb
  File "/Users/elieb/git/scaaml/scaaml/scald/tracer/devices/rainbow/generics/cortexm.py", line 109, in block_handler
    self.base_block_handler(address)
  File "/Users/elieb/git/scaaml/scaaml/scald/tracer/devices/rainbow/rainbow.py", line 414, in base_block_handler
    r = self.stubbed_functions[f](self)
  File "/Users/elieb/git/scaaml/scaaml/scald/tracer/chipwhisperer_leakage_automaton.py", line 205, in trigger_low
    e.emu.hook_del(self.trigger_hook)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/unicorn-1.0.2rc4-py3.6.egg/unicorn/unicorn.py", line 592, in hook_del
TypeError: an integer is required (got type NoneType)
Encryption: FAIL
KEY=2b7e151628aed2a6abf7158809cf4f3c
 PT=6bc1bee22e409f96e93d7e117393172a
EXP=3ad77bb40d7a3660a89ecaf32466ef97

Currently the group and part counters are reset to zero when restarting a failed capture. This produces shards which belong to wrong groups / parts. An existing dataset can be fixed using scaaml.io.Dataset.reshape_into_new_dataset.

The SScope is expected to be initialized using capture_info (to have all information present in capture_info). But capture_info might contain more information which is not needed to initialize SScope. This commit allows SScope to ignore the rest of the capture_info.

Currently when a capture is restarted, the progress bar is showing how many traces need to be captured and starts at zero percent. This is confusing, since it looks like progress has been lost. It should instead start where it ended (show all shards that have been captured so far as the immediate progress).

When resuming capture (see PR #24 and PR #25) the info.json file gets rewritten instead of updated. This results in the shard file names not being present in the info.json and possibly broken metadata (such as min and max values).

This is useful for combining predictions. One way to combine predictions is to compute arithmetic mean, the other way is to do geometric mean. For numerical stability sum log_10(prediction + epsilon), but this gives negative values which do not play well with optimistic=False).

By default mypy does not check types in functions without type signature. Add a flag to allow type checking inside such functions. This also removes extraneous comments on PRs.

When loading a dataset using as_tfdataset and more bytes of the same attack point are loaded they would have the same value. This is because a dictionary would share state instead of being copied. Thus all attack points would have the same byte index causing the same value to be loaded multiple times.

Fixes issue #107

There is a state being shared during loading of dataset. Thus if two bytes of an attack point should be loaded two copies of the same get returned (e.g., key byte 1 and byte 3 should be loaded, then the same value is repeated as key_1 and key_3).

When examples_per_shard == 1 and a restart happens there is no way to tell if the previous shard was already captured and saved or not. Also the scaaml.io.Dataset or DatasetFiller are not equipped to handle saving single example multiple times. To make this reliable there needs to be a way to say "shard saved".

SScope base class should have a method such as def get_scope_settings(self) -> dict[str, str] so that the capture script can do capture_info.update(scope.get_scope_info()) And potentially also something like: def setup_scope(self, capture_info: dict[str, str]) -> None that takes capture_info data and setup of scope using the keys it can understand. This way get_scope_settings() would return all the settings. Sometimes, especially with picoscopes, the scope adjust the settings you give to something it can actually do.

This could significantly speed up (save days, maybe a week) both capture and converting from other formats to our format.

Potential pitfalls:

• running out of memory for very long traces
• need proper integration with resuming capture

Checking for key duplicities (if the same key is in train and test shards at the same time) fails with "key is not an attack point" when key is not an attack point.