Introduction
Apiestas is a project composed of a backend powered by the awesome framework FastAPI and a crawler powered by Scrapy.
This project has followed code examples from RealWorld apps, specifically the following projects:
- FastAPI RealWorld App
- FastAPI MongoDB RealWorld App (A fork of the previous)
- Full Stack FastAPI PostgreSQL
The crawler inserts and updates data from the MongoDB database by using the Apiestas REST API and the data is exposed through this API. The REST API communicates with the database by using Motor - the async Python driver for MongoDB. Finally, this application uses Typer to create the Apiestas CLI, which is the main entrypoint of the application.
Quickstart
First, set environment variables and create database. For example using docker:
export MONGO_DB=rwdb MONGO_PORT=5432 MONGO_USER=MONGO MONGO_PASSWORD=MONGO
docker run --name mongodb --rm -e MONGO_USER="$MONGO_USER" -e MONGO_PASSWORD="$MONGO_PASSWORD" -e MONGO_DB="$MONGO_DB" MONGO
export MONGO_HOST=$(docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' pgdb)
mongo --host=$MONGO_HOST --port=$MONGO_PORT --username=$MONGO_USER $MONGO_DB
Then run the following commands to bootstrap your environment with pipenv:
git clone https://github.com/franloza/apiestas cd apiestas pipenv install pipenv shell
Then create .env file (or rename and modify .env.example) in api or crawling folders and set environment variables for every application:
cd api touch .env echo DB_CONNECTION=mongo://$MONGO_USER:$MONGO_PASSWORD@$MONGO_HOST:$MONGO_PORT/$MONGO_DB >> .env
To run the web application in debug use:
python main.py api --reload
Development with Docker
You must have docker and docker-compose tools installed to work with material in this section. Then just run:
cd docker docker-compose up -d
The API will be available on localhost:9000 in your browser.
If you want to enable the surebets calculation feature, you need to use the extended Docker Compose file for Kafka environment. This file is docker-compose.kafka.yml. However, instead of executing this file directly along with docker-compose.yml file, execute run-with-kafka.sh as it is necessary to set up Kafka Connect, MongoDB Replica Set and wait for the systems to be ready. containers initialization
If you run Apiestas with Kafka and Kafka Connect, you will enable Kafka UI, where you can to examine the topics and other info.: http://localhost:9021 or http://localhost:8001/
- The matches topic should have the crawled bets and matches.
- The mongo.apiestas.matches topic should contain the change events.
You can also examine the collections in the MongoDB by executing:
docker-compose exec mongo /usr/bin/mongo
To see the logs of the different services, you can execute the following command:
docker-compose -f docker-compose.yml -f docker-compose.kafka.yml logs -f api surebets crawler
Run tests with Docker
cd docker docker-compose -f docker-compose-test.yml run tests
Web routes
All routes are available on /docs or /redoc paths with Swagger or ReDoc.
Docs
Redoc
Data sources
Currently the application implements two working crawlers:
oddsportalcom- Used as ground truth for matches and oddselcomparador.com- for odds dataCodere- for odds data
Architecture
TODO
- Add support for more bet types calculation
- Support time series visualization
1 Nov 24, 2022
83 Dec 19, 2022
2 Apr 19, 2022
60 Dec 17, 2022
18 Oct 31, 2022
2 Mar 14, 2022
371 Jan 1, 2023
2.2k Jan 8, 2023
1.6k Dec 31, 2022
4 Mar 18, 2022
2 Jun 21, 2022
31 Dec 9, 2022
106 Jan 6, 2023
472 Jan 2, 2023
2 Nov 7, 2021
1 Dec 21, 2021
8 Feb 8, 2022
28 Oct 25, 2022
39 Nov 24, 2022