An attempt to map the areas with active conflict in Ukraine using open source twitter data.

As the project grows, we need a way to make sure it is easy to test incoming PRs instead of me manually pulling and testing everything. The best way would be to setup a CI/CD pipeline using Github Actions. At the present moment we don't have any tests except the one introduced in #21. Ideally I would like to setup tests and then a CI/CD pipeline. However, the server running the website right now, is manually using cron jobs. I would instead like to wait for #16 to finish and then create a pipeline with the docker image.

Meanwhile, feel free to add tests and help me write a CI pipeline.

Ok, so I wish to contribute to this project. I made some minor changes, here they are:

1. Cap tweet preview for the folium marker preview to 100 letters.
2. Add one more hashtag for tweepy to search (#OSINT).
3. Adjust default coordinates and zoom.
4. Change tileset to something more fitting (in my opinion).
5. Change User Agent which geopy uses for Nominatim.

Please inform me of anything wrong I did (this is my first pr).

instead of using a third-party GitHub action.

#52 I've used just the token secret, the gitlab ref name is hardcoded to master.

I haven't tested it, as I'd need the secret value and gitlab access.

I think it would be a pretty good idea to implement a ui for this instead of just generating a PNG.

If needed i could spare some time in Vue. By all means i have a free weekend right now.

Fix #5

Excluding accounts that spreads misinformation

This pullrequest added:

• [x] Changes in README (adding point 4)

• [x] Adds new env variable to handle blacklisted userIDs
• [x] Add test folder and some test coverage of the implemented feature

Don't exit the whole application if fetching geolocation data fails. It will look like this:

Adding markers... (This may take a while)
 10%|█████████████████▌                                       | 50/512 [01:12<14:03,  1.83s/it] 
Getting geolocation data for the South China Sea failed
 11%|███████████████████                                      | 55/512 [01:26<19:59,  2.63s/it] 

I was trying to get a domain on freenom, but it doesn't let me register, for the time being I'll deploy it on my custom domain but if anyone has a spare domain lying around or if they can register a free "liveactionmap" domain on freenom that would be great.

Since we're no longer dependent on the cronjob, we need to remove the tweets in the python code itself. However deleting the file after each run as done in an earlier PR is not useful because there's too few data points, we can try to increase the number of results we get from Twitter but that's a separate issue.

Also let's make sure the schedule time for the scraping and removing tweets don't overlap (they are not multiples) otherwise the tweets file might be deleted while the bot is running outputting an empty map.

Moving to a dynamic backend should be it's own issue

Originally posted by @Krishna-Sivakumar in https://github.com/kinshukdua/LiveActionMap/issues/24#issuecomment-1053660218

Self-hosting a Nominatim service will help us cut down on time wasted doing API calls. Map building takes the most time. And right now, most of the time while building the map is spent on doing 2 API calls to a public Nominatim service per tweet, which adds a lot of latency.

Since we already have a docker build, it doesn't hurt to add one or two more containers to our deployment.

Links to refer: https://github.com/mediagis/nominatim-docker https://nominatim.org/release-docs/latest/admin/Installation/

This will help us shift to a dynamic backend in the long run.

Create a GET API serving /api/zonesproviding a response containing all currently available markers.

Expected response

[
    {
        "positions": [
            [
                50.36429316995319,
                30.228621662109
            ],
            [
                50.51303377189189,
                30.28741424805162
            ],
            [
                50.60122461757218,
                30.787151228563896
            ],
            [
                50.44548234821721,
                30.77081995469095
            ]
        ],
        "color": "green",
        "title": "Safe Zone",
        "content": "Some safe zone over here"
    }
]

This is so people can access the site using tor, might make some people feel safer.

This would need a separate frontend that doesn't contain any tracking and no links to anything but out service

Create a GET API serving /api/markers providing a response containing all currently available markers.

Expected response

{
    "markers": [
        {
            "position": [
                49.038230248475905,
                31.450182690663947
            ],
            "title": "Twit Title with image",
            "content": "",
            "user": "Twitter User",
            "uri": "https://twitter/postUrl",
            "image": "imageURL",
            "timestamp": "unixtimestamp of tweet"
        },
        {
            "position": [
                48.038230248475905,
                31.450182690663947
            ],
            "title": "Twit Title without image",
            "content": "Twit Text",
            "user": "Twitter User",
            "uri": "https://twitter/postUrl",
            "timestamp": "unixtimestamp of tweet"
        }
    ],
    "timestamp": "unixTimeStamp"
}

It would be neat to see how many hits the site gets.

I'd recommend either google or unami(this is more gdpr complaint and can be self hosted on Heroku for free)

How about I make a quick mockup with vuejs that would just read an api. That way once the project is evolved enough it would be enough to just hook one and the other together.

Originally posted by @Krishna-Sivakumar in https://github.com/kinshukdua/LiveActionMap/issues/24#issuecomment-1053660218