Hi. I apologize if this seems sudden and out of the blue, or like a lot of work. I was hoping that I might offer my services in a bit of an uplift.

In preparation for containerization a little bit of cleanup is required.

• Configuration module is extracted which allows previous configuration but also adds the possibility to configure via environment variables.
• Statistics module is extracted to separate the non-functional JSON variant from the MongoDB which is current.
• Link Cache module is extracted providing the previous JSON behavior, as well as MongoDB, configurable by configuration.
• Storage Module is extracted to handle the current media folder for downloading. In addition the storage location is now configurable.

Next steps:

• Use a Python packaging tool for Virtual Environments. I suggest Poetry
• Creating a Dockerfile and emulating existing deployment via a docker-compose file.
• Implementation of alternative cache, storage, statistics modules.
• Creating a deployment script which allows serverless deployment using the docker container.
• • I've focused on the area I know best, which is the Google Cloud Platform
• • Cloud Run handles the service.
• • Firestore as database.
• • Cloud Storage as media store. (with possibility for automatic expiry)

(jk. I've got all of the above ready.)

Future:

• This should scale up quite significantly without too much overhead, it might even work just fine under googles free tier.
• There is potential for hangups during media downloads, one proposal is to use Task Queue to run a download-to-storage module while letting the request either complete or await completion.

The naming conventions for variables, functions, methods and objects are laid out in this document.

The names of the variables and functions have been changed to match that of the python spec.

See https://revolt.chat/ The source code that uses this user agent is here: https://github.com/revoltchat/january/blob/b0fab877838a71f90c18fc5301b8461d0e3c58e5/src/util/request.rs#L10

You can test this without registering to Revolt by making a GET request to its API (no authentication required), i.e.:

https://jan.revolt.chat/embed?url=https://fxtwitter.com/dm4catbot/status/1413797797187440640

When I tried to open a link, I had to manually change the URL from fxtwitter.com back to twitter.com in order to be redirected to the orignal tweet. So here's a small suggestion to help others who are experiencing the same problem.

• Fix video quality sorting (i didnt quite understand what a dict was)
• Fix screen_name naming consistency
  • screen_name seems to be blank in prod (fxtwitter.com) but works fine in dev env
• Fix img accessed before declaration
• Hopefully tweet text content shows now in telegram
• Add flask_cors dependency
• Remove unnecesary react-helmet garbage

By comparing against a regex we can detect if the link is in the proper format, and allow for excluding https://twitter.com/" in the url. this allows for swapping outtwitter.comfortwtfix.me` in place, and having the link instantly work.

Also removed some unused imports that were causing issue as they were not in the requires.txt

This implements the change I suggested at https://github.com/robinuniverse/TwitFix/issues/44#issuecomment-1065567675

Additionally cleans up the matching a bit, tweaks the readme, fixes a missing entry in requirements.txt, and simplifies local testing a little.

I haven't fully tested this end-to-end, as I don't have twitter API access set up, but everything seems to behave as expected in the logs.