Open-questions - Open questions for Bellingcat technical contributors

I have been reading about landslide detection, some of the tools and techniques seems like they would be a useful addition. This is for selecting useful and unobstructed images, the parts about calculating slope and elevation may not be applicable but could be removed, or modified for more useful annotation.

Dear Bellingcat,

I've got an open source github project that I froze yesterday because of Bellingcat's announcement of their time-and-space map. However, that map is highly curated. My question is constructing such a map from a strait upload of photos valuable? Obviously, this would be input to curation and corroboration, not a replacement for it. I'm willing to work it, but despite asking everywhere I can I've gotten no feedback (either positive or negative) from journalists or OSINT analysts. Obviously, they are busy. I'm a serious programmer and founder of Public Invention, but have no experience or connections in the OSINT community. My email is [email protected].

A lot of witness provide videos of large explosions.

There is a well-known physic analysis to estimate the power of the explosion, that was unveiled by the British physicist G. I. Taylor from declassified images of the Trinity test in New Mexico in 1945. It is based on Buckingham π theorem.

Look at this doc for explanation.

To perform this computation, we need timed frames and the fov of the camera to estimate the scale of the fire ball.

In order to constrain the problem space somewhat, and provide a scaffolding for contributors to get started with, it would be helpful to have a sample dataset collection of image:lat/lon/angle pairs for experimenting with. This could initially be small, perhaps a dozen images.

To generate a larger dataset in the future, perhaps online images with known geocodes could be used?

Summary

Long read below so:

1. The workflow engine described exists, is open source and ready now
2. Hosting the control plane can be done from a single node kubernetes cluster (k3d, kind or minikube is fine) or use a commercial control plane hosting provider
3. All that's required here is to start modelling workflows

Background

Ability to provide access to toolings and workflows to as many people as possible. These tools and capabilities must be "consumable" as easily as possible to ensure widespread use and adoption.

Problem Statements

1. Installing and using tools often requires advanced technical knowledge and thus is not suitable for widespread adoption
2. Tooling and "providers" (the how) can change often whereas the processes (or aims) rarely change.

Example of 1: Ability to use the instagram-location-search capability first requires that you know what Python is. Second, that you can install it, third that you are savvy enough to open dev tools and find your session ID. All of this adds an unneccessarily high barrier to entry.

Example of 2: Perform a phone number lookup (the aim or "what") will remain fairly static over time (the tooling or "how") is subject to potentially frequent change. If, tomorrow, I want to lookup a number not from instagram but from another provider, I need to write an entirely new program.

Solution

1. Provision a workflow engine in front of these capabilities with an easy to understand, standardised and open source eventing mechanism.
2. Provide the concept of sequences which can consist of one or more tasks. Optionally provide the ability to link sequences together in larger workflows.
3. Provide a standardised "control plane" that a user (or app) can interact with to trigger sequences and workflows.
4. Provide flexibility in where and how the "tooling" runs (alongside the control plane or remotely, in containers or "standard" processes).
5. Ablility to write tooling in any language - the consumer shouldn't know or care about implementation
6. Provide flexibility to adjust tooling without altering overall capability (eg. swap Instagram for Tool X without impacting a user)

Simple Sequence (Single Task)

Using the above example of a phone number lookup, we could model it like this (stages modelling the available environments - in this case, the development environment):

spec:
  stages:
    - name: dev
      sequences:
        - name: sequence-one
          tasks:
          - name: checknumber

If someone wants to check a phone number:

1. They HTTP POST a dev.sequence-one.triggered event
2. The workflow engine knows that it must now trigger a task called checknumber so the workflow engine triggers a checknumber.triggered event
3. The "Phone number checker" service listens for checknumber.triggered events and knows that it must start working
4. The "Phone number checker" service informs the engine that it has started by sending a checknumber.started event
5. Optionally during execution, the "phone number checker" can send updates with the checknumber.status.changed event
6. The "Phone number checker" has interacted with the Instagram APIs and done it's work. It sends a checknumber.finished event along with any result back as an event.
7. The workflow engine knows that the checknumber task is done and that it is the only task in the sequence, therefore the sequence is done.
8. The final result is returned to the user (or web app)

Complex Sequence (Multiple Tasks)

Take a more complex sequence which consists of multiple tasks. This one may model a process to judge a photo's authenticity:

spec:
  stages:
    - name: dev
      sequences:
        - name: sequence-two
          tasks:
          - name: extract-metadata
          - name: geolookup
          - name: author-search
          - name: evaluate

In this sequence, the engine won't start the geolookup task before it receives a .finished event from the extract-metadata task. The final task assumes that we have some metrics (perhaps from the previous steps) which we can use to judge the authenticity of an image. The evaluate task takes, as input, a set of metrics, their corresponding thresholds and ultimately produces a decision pass, warning or fail and a score which indicates the authenticity.

This evaluate logic is already available out-of-the-box with the engine so no additional implementation is required here. Just tell the engine:

1. What are the metrics?
2. Where are the metrics to be retrieved from?
3. What are the thresholds?

The engine will do the evaluation and produce the report.

As I say, the engine and all eventing logic is ready. Reach out if you want to trial some sequence and task definitions.