changedetection.io - The best and simplest self-hosted website change detection monitoring service

Last update: Jan 1, 2023

Related tags

Logging docker open-source monitoring self-hosted website-monitor watchtower url-monitor change-alert change-detection changedetection website-change-monitor website-monitoring change-monitoring visualping page-change-alert

Overview

changedetection.io

Self-hosted change monitoring of web pages.

Know when web pages change! Stay ontop of new information!

Example use cases

Know when ...

Government department updates (changes are often only on their websites)
Local government news (changes are often only on their websites)
New software releases, security advisories when you're not on their mailing list.
Festivals with changes
Realestate listing changes

Get monitoring now! super simple, one command!

docker run -d --restart always -p "127.0.0.1:5000:5000" -v datastore-volume:/datastore --name changedetection.io dgtlmoon/changedetection.io

Now visit http://127.0.0.1:5000 , You should now be able to access the UI.

Updating to latest version

Highly recommended :)

docker pull dgtlmoon/changedetection.io
docker kill $(docker ps -a|grep changedetection.io|awk '{print $1}')
docker rm $(docker ps -a|grep changedetection.io|awk '{print $1}')
docker run -d --restart always -p "127.0.0.1:5000:5000" -v datastore-volume:/datastore --name changedetection.io dgtlmoon/changedetection.io

Screenshots

Examining differences in content.

Future plans

Greater configuration of check interval times, page request headers.
~~General options for timeout, default headers~~
On change detection, callout to another API (handy for notices/issue trackers)
~~Explore the differences that were detected~~
Add more options to explore versions of differences
Use a graphic/rendered page difference instead of text (see the experimental selenium-screenshot-diff branch)

Please ⭐ star ⭐ this project and help it grow! https://github.com/dgtlmoon/changedetection.io/

Comments

Heroku Support?

I'm trying to get changedetection.io to work with heroku free plan. Adding a "Deploy to heroku" button would make this service much more accessible

opened by harshfolio 33
Add filter to remove elements by CSS rule from HTML before change detection is run

This PR adds a filter that will exclude header, footer and nav tags from HTML.

This is useful in situations where it is expected that interesting changes only happen outside of header, footer or navigation (in most cases that means in the main body of the document).

I get some failing tests, but tbh I have no clue why - I'm adding something very similar to PR #444, which passes, and the failing tests don't seem to be related directly to my changes.

The tests I added for this work (which pass) I adapted from test_css_selector.py and am not 100% sure about test_body_filter_full - @dgtlmoon perhaps you can have a closer look to make sure my intention with the test is executed correctly?

opened by timlod 32
Reverse proxy - Password protection randomly disables when clicking logout

Describe the bug Not exactly sure what causes this. It's very random and does not always happen. Cannot find anything about this in the logs.

Version v0.39.13.1
bug

opened by kzshantonu 27
Content from a different URL used in diff
Describe the bug

Notifications are being triggered and when I check the diff, the content for the previous version is the content from one of the other URLs I have configured.

For example, the watch I have configured for https://www.churnetvalleyrailway.co.uk/events/rail-ale-trail.html#timetable shows the new lines from that website, however, the old lines in the diff are from https://www.retrotink.com/product-page/2x-mini (another watch I have configured).

I have only seen this where the previous is wrong in the diff, not the current but I may have missed it.

Version

v0.39.19.1

I am aware this is an older version and have upgraded to v0.39.21.1 to see if it resolves the issue. I've not seen this in previous version but have just changed my hosting environment.

To Reproduce

I'm not sure what triggers this. Only that it is using other configured URLs so at least 2 URLs must be configured.

https://changedetection.io/share/CRdC1_e-W6Ea https://changedetection.io/share/O-fx5cPQLmMa

Expected behavior

Configured URLs should only diff and send notifications on changes to their own URLs

Screenshots

Host Environment (please complete the following information):

OS: [e.g. iOS] Raspberry Pi OS, Kubernetes, local path volume

Browser [e.g. chrome, safari] Firefox 106.0.4

~~I'm unsure if this would cause the issue but I am aware of a problem with the SMB volumes where data access is much slower than an application would usually expect. I am looking to resolve this but I wouldn't expect that to cause this behaviour in an app.~~ After checking, this container is using a local path volume rather than SMB.

Additional context

I'm open to ideas on how to get more information on this.
triage
opened by deosrc 25
[feature] ability to import distill.io's/visualping.io exported JSON data
Distill.io offers the ability to export your data https://distill.io/docs/web-monitor/how-export-and-import-monitors/

wanted! please attach your distill.io/visualping export data if possible so we can learn more!

It would be amazing to be able to import their JSON data, atleast

[x] URLs

[x] Filters (xpath/css)

[x] Top frame only (we dont dig deeper into embedded iframes atm)

Mockup - I think putting the others under a tab each is good because there's more than likely going to be some checkbox options that apply to each

enhancement
opened by dgtlmoon 23
Docker - All data and config have been reset

Hi,

I have installed change detection in a Synology Docker and it has been working fine for a while. Then all URLs and config have been reset two days ago. Does anyone have the same issue?
bug

opened by sqsuica 22
Chrome/Webdriver support for Javascript websites
This actually provides a pluggable interface for adding different fetch/handling mechanisms, first of which is the requests HTTP plain html fetcher, and Chrome Webdriver support! 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉

DOES NOT WORK ON ARM / RaspberryPi type machines!

Todo

[ ] Make tests use the full docker stack with the chromedriver + relevant settings

[ ] Lower the number of workers (make it configurable per backend?) 10 is too many (is it?), but this is totally fine for requests/plaintext

[ ] Some kind of dom-wait-until-fully loaded or other

[ ] Handle failures better (better feedback so we know if its from chromedriver, or from the website)

[ ] Save last screenshot?

TO USE

Grab the docker-compose.yml from https://github.com/dgtlmoon/changedetection.io/blob/javascript-browser/docker-compose.yml

docker-compose up -d or how ever you like it

Enable the 'chrome' fetcher in [settings] tab at http://localhost:5000
opened by dgtlmoon 22
Feature request: Email notification

Looking to use this instead of visualping but I need email notifications.

Whats the best way to hack that together with this application? (if at all possible) #
enhancement

opened by monkeydust 22
(windows) Connection to URL timed out, SOCKSHTTPSConnectionPool error?

SOCKSHTTPSConnectionPool(host='_URL_', port=443): Max retries exceeded with url: _encodedURLstring?_ (Caused by ConnectTimeoutError(<urllib3.contrib.socks.SOCKSHTTPSConnection object at 0x7f21e011fd60>, 'Connection to _URL_ timed out. (connect timeout=15)'))

I'm getting this error for all URLs I try to enter, as well as the default ones.

Installed the docker image today so should have the latest version, running on Windows 10 in Docker Desktop on localhost:5000.

opened by benninkcorien 21
Smart ignore

Better would be a way to click on the diff text (green/red insert/delete) and have some mechanism that lets us tell the machine that we want to ignore that block

can be based on context of the above 2 and below 2 lines

run before generating the checksum

maybe using the text highlight JS to get the lines/text selected, then we know where to set the context

and/or click on an individual line to add that ignore text to a config
enhancement

opened by dgtlmoon 20

telegram:// diff notifications, don't send anything other than , , , and

When using a telegram notification, which includes {diff} in the body/other sometimes there are errors 2022-04-21 13:51:42,987 - WARNING - Failed to send Telegram notification to -628107617: Bad Request: can't parse entities: Unsupported start tag "br/" at byte offset 120, error=400. <b>, <i>, <a>,<code> and <pre> https://github.com/Eleirbag89/TelegramBotPHP/issues/152#issuecomment-395220066 bug Notifications systems opened by dgtlmoon 19

[feature] Increase maximum number of Browser Steps Version and OS All Is your feature request related to a problem? Please describe. I have a particular process which is longer than 10 steps that I would like to automate. Describe the solution you'd like Increased maximum number of steps. 15 or 20 would fit my use case. Describe the use-case and give concrete real-world examples Automatically navigate through more than 10 actions with the easy to use Browser Steps feature. enhancement opened by evanofslack 0 [feature] Template for Fetch Version and OS newest docker image Is your feature request related to a problem? Please describe. I fetch several products from the same page so would be good to have some option like template, so i do not need to go thru all the options over and over but only change the site adders and all the options would be the same Describe the solution you'd like to have ability to use Template option for a same domain but different subpage as all the other settings in Filters and Triggers will be the same Describe the use-case and give concrete real-world examples site.com/product1.html site.com/product2.html for now i have to setup all over from the beginning but for product 2 i could use template to fill all the predefined options without using page preview etc... Additional context Add any other context or screenshots about the feature request here. enhancement opened by MG-Sky 0 [draft] add follow redirects options for html_requests fetcher For https://github.com/dgtlmoon/changedetection.io/issues/1257. Starting another draft because the other has been closed and cant re-open it :monocle_face: . opened by Eldorico 0 JSON-Filter doesn't work correctly on this particular page Describe the bug JSON-Filter doesn't work correctly. Version v0.40.0.4 To Reproduce As an example I have this price comparison. The original result is: { "@type": "AggregateOffer", "lowPrice": 182.0, "highPrice": 342.49, "priceCurrency": "EUR", "offerCount": 64, "availability": "https://schema.org/InStock", "itemCondition": "https://schema.org/NewCondition" } Now I just want to filter only the price (at Filters & Triggers ➜ CSS/JSONPath/JQ/XPath Filters). Result with filter jq:.lowPrice: null{ "@type": "AggregateOffer", "lowPrice": 182.0, "highPrice": 342.49, "priceCurrency": "EUR", "offerCount": 64, "availability": "https://schema.org/InStock", "itemCondition": "https://schema.org/NewCondition" } And when I use JSONPath with json:$.lowPrice, the result remains unchanged from the original output. The tests on https://jqplay.org/ and https://jsonpath.com/ work as they should. Is this a mistake on my part (please excuse me) or is it a bug? triage opened by geimist 1 Cannot read properties of null (reading 'scrollTop') at eval Describe the bug I often get an error for checks: TypeError: Cannot read properties of null (reading 'scrollTop') at eval (eval at evaluate (:191:30), :12:67) at UtilityScript.evaluate (:198:19) at UtilityScript. (:1:44) If i manually click recheck it works (sometime it takes two tries) without changing the element. What causes this? Version v0.40.0.3 To Reproduce You cant. It happens randomly. But there are multiple elements that have that error and it are not always the same items it seems. ! ALWAYS INCLUDE AN EXAMPLE URL WHERE IT IS POSSIBLE TO RE-CREATE THE ISSUE - USE THE 'SHARE WATCH' FEATURE AND PASTE IN THE SHARE-LINK! As said it happens randomly and works if I click recheck - URL is useless but here: https://www.parallels.com/de/products/desktop/buy/?full Expected behavior A clear and concise description of what you expected to happen. Screenshots Github throws an error uploading a jpg / png. Desktop (please complete the following information): Docker with playwright triage opened by blippercop 4 'list' object has no attribute 'lower' Describe the bug changedetection only shows this error when I watch that specify URL. Version 0.40.0.3 To Reproduce Steps to reproduce the behavior: Go to self-hosted changedetection Add new watch with URL https://www.maklerpoint.at/wohnungssuche?f%5Ball%5D%5Bzip_code%5D=1150&f%5Boccupancy%5D=all&from=119162 Let changedetection check this watch Error is listed on the main page. Expected behavior URL is being watched. Screenshots If applicable, add screenshots to help explain your problem. Desktop (please complete the following information): OS: Windows 11 22H2 Browser Chrome Version 108 Smartphone (please complete the following information): Device: [e.g. iPhone6] OS: [e.g. iOS8.1] Browser [e.g. stock browser, safari] Version [e.g. 22] Additional context If I'm not mistaken, this shouldn't go into the discussion section, because it looks like an python error with. bug opened by dom6770 1

Releases(0.40.0.4) 0.40.0.4(Dec 25, 2022) Upgrade container to Python 3.10 Many fixes and improvements What's Changed Run test as built docker container by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/1245 Initial PDF fetcher support with basic text extraction by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/1244 UI - Tidy-up list icons by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/1250 Container updates by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/1249 Ability to configure extra proxies via the UI by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/1235 Filter failure notification tokens by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/1256 UI - Suggest adding proxy for watch when 403 access denied is reached by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/1260 Full Changelog: https://github.com/dgtlmoon/changedetection.io/compare/0.40.0.3...0.40.0.4 Source code(tar.gz) Source code(zip) 0.40.0.3(Dec 19, 2022) What's Changed Playwright - fetch improvements - Disable service workers and just fetch after page starts to load+delay by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/1229 Skip process if the previous checksum and the just fetched one was the same by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/925 Fetcher + VisualSelector - xpath filter with attribute filter was breaking the element finder by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/1232 Always sort the key order of JSON content for less false alerts Re #1219 by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/1234 Playwright - Fetch reliability improvements by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/1238 Check the most overdue watch first by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/1242 Visual Selector - Select smallest/most precise element first, better filtering of zero size elements by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/1246 Full Changelog: https://github.com/dgtlmoon/changedetection.io/compare/0.40.0.2...0.40.0.3 Source code(tar.gz) Source code(zip) 0.40.0.2(Dec 8, 2022) What's Changed Re #1207 - Dont scan for ldjson data when 'no' was clicked on the suggestion by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/1208 Full Changelog: https://github.com/dgtlmoon/changedetection.io/compare/0.40.0...0.40.0.2 Source code(tar.gz) Source code(zip) 0.40.0(Dec 8, 2022) Huge release Main items Browser steps - Interact with the browser, login to websites, accept cookie boxes, fill forms, click X,Y and more (requires playwright), runs before changedetection so you can log into a website and then get a change alert Auto suggestion to follow product LD JSON data use existing product data to get better price change alerts without needing visual-filter/xpath/css selectors (where the website supports LD-JSON) Dark Mode - Thanks to @tanc for the killer styling and functionality! Notifications , Improved simple gets:// post:// callbacks and Jinja2 compatible templates, all variables, tokens now available in notification settings (you could even put if statements in your notification body if you wanted, as an idea) Extract historical data, Use a regex to generate CSV files (great for scraping websites over time, for example the number of available houses for rent/buy over time) Loads of fixes Help spread the word about changedetection.io! What's Changed Update README.md by @t5k6 in https://github.com/dgtlmoon/changedetection.io/pull/1158 Configurable browser steps (enter text, scroll, wait for text, etc) by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/478 Extra validation for URLs with jinja2 template by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/1166 Faster BrowserStep screenshot updates and enable gzip compression for all content replies in the UI by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/1171 BrowserSteps - Make the UI require an extra step so it doesnt slow down the experience when clicking through the tabs by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/1175 Favicon multiplatform update by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/1176 add changedetection container dependencies by @peppetemp in https://github.com/dgtlmoon/changedetection.io/pull/1178 Simple extract data by regex from all historical watch text by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/1191 Extra validation for regex extract field by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/1192 test fix - use local address by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/1193 Notifications - tokens/jinja2 templating by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/1184 Notification - Support for standard API calls post:// posts:// get:// gets:// delete:// deletes:// put:// puts:// by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/1194 Dark mode - Template tidy up by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/1197 Update runtime.txt to work on all heroku stacks by @amrohendawi in https://github.com/dgtlmoon/changedetection.io/pull/1198 Test cleanups by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/1196 New Contributors @t5k6 made their first contribution in https://github.com/dgtlmoon/changedetection.io/pull/1158 @peppetemp made their first contribution in https://github.com/dgtlmoon/changedetection.io/pull/1178 @amrohendawi made their first contribution in https://github.com/dgtlmoon/changedetection.io/pull/1198 Full Changelog: https://github.com/dgtlmoon/changedetection.io/compare/0.39.22.1...0.40.0 Source code(tar.gz) Source code(zip) 0.39.22.1(Nov 22, 2022) What's Changed Re #1148 - Notification screenshot/JPEG was not being regenerated correctly by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/1149 Full Changelog: https://github.com/dgtlmoon/changedetection.io/compare/0.39.22...0.39.22.1 Source code(tar.gz) Source code(zip) 0.39.22(Nov 20, 2022) New big things Watch filters can now accept a list/multiple CSS/xPath filters per page Send a screenshot when a website page change is detected Updated AppRise library ( Mastodon support, email fixes and other minor fixes) What's Changed Re #623 - Filters should accept a list/multiple filters by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/1064 Adding check_count and sorting vars by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/1099 Use deepcopy to stop data corruption (possible?) by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/1108 Stability fix related to counter by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/1113 Re-test under HIDE_REFERER, use strtobool so you can use 'False' by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/1121 Bump notification library by @hmmbob in https://github.com/dgtlmoon/changedetection.io/pull/1128 Make link to notification debug log easier to find by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/1130 Make VisualSelector show first available multiple selector, refactor to make more maintainable by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/1132 Diff js maint by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/1137 Option to attach screenshot to notification by @m42e in https://github.com/dgtlmoon/changedetection.io/pull/1127 No need to re-check, fix timing. by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/1141 Screenshot option should only be available to webdriver/playwright watches, screenshot sent as JPEG to save bandwidth, Simplify the logic around screenshot, by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/1140 New Contributors @m42e made their first contribution in https://github.com/dgtlmoon/changedetection.io/pull/1127 Full Changelog: https://github.com/dgtlmoon/changedetection.io/compare/0.39.21.1...0.39.22 Source code(tar.gz) Source code(zip) 0.39.21.1(Nov 2, 2022) What's Changed Playwright conditional fix musl alpine by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/1087 Full Changelog: https://github.com/dgtlmoon/changedetection.io/compare/0.39.21...0.39.21.1 Source code(tar.gz) Source code(zip) 0.39.21(Nov 2, 2022) What's Changed Hide the Referer header from monitored websites (as env option) by @michaelmcmillan in https://github.com/dgtlmoon/changedetection.io/pull/996 Be sure that only valid history index lines are read by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/1042 update path for validation in the CONTRIBUTING.md by @Entepotenz in https://github.com/dgtlmoon/changedetection.io/pull/1046 use same version of playwright while running tests as in production builds by @Entepotenz in https://github.com/dgtlmoon/changedetection.io/pull/1047 Fix syntax by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/1050 Basic system info/system state API by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/1051 Re #1052 - Dynamic URLs by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/1057 Backup zip - always use correct relative path of the snapshot text instead of the datadir path by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/1054 Re #1052 - Watch 'open' link should use any dynamic/template info by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/1063 Make fetch error readable by @SuperSandro2000 in https://github.com/dgtlmoon/changedetection.io/pull/1038 Use pip conditional requirements to not install playwright for ARM (unsupported) by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/1067 Jq conditional pip requirements.txt include in linux by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/1066 Remove unused code by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/1070 Disable version check when pytest is running by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/1084 New Contributors @michaelmcmillan made their first contribution in https://github.com/dgtlmoon/changedetection.io/pull/996 @Entepotenz made their first contribution in https://github.com/dgtlmoon/changedetection.io/pull/1046 @SuperSandro2000 made their first contribution in https://github.com/dgtlmoon/changedetection.io/pull/1038 Full Changelog: https://github.com/dgtlmoon/changedetection.io/compare/0.39.20.4...0.39.21 Source code(tar.gz) Source code(zip) 0.39.20.4(Oct 17, 2022) What's Changed Better cross-platform UTF-8 handling by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/1034 Full Changelog: https://github.com/dgtlmoon/changedetection.io/compare/0.39.20.3...0.39.20.4 Source code(tar.gz) Source code(zip) 0.39.20.3(Oct 17, 2022) What's Changed Windows diff file handling improvements by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/1031 Include test suite in pip Full Changelog: https://github.com/dgtlmoon/changedetection.io/compare/0.39.20.2...0.39.20.3 Source code(tar.gz) Source code(zip) 0.39.20.2(Oct 12, 2022) What's Changed Test that the 'execute JS before' works by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/1006 JQ - Make library optional so it doesnt break Windows by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/1009 Full Changelog: https://github.com/dgtlmoon/changedetection.io/compare/0.39.20.1...0.39.20.2 Source code(tar.gz) Source code(zip) 0.39.20.1(Oct 9, 2022) What's Changed Docker containers did not build correctly because jq (JSON query) needed make on arm v6 and arm v7 (rPi for example) Added test to test build docker containers on changes to critical files Don't use default Requests user-agent and accept headers in playwright+selenium requests, breaks sites such as united.com - by @dgtlmoon in #1004 Full Changelog: https://github.com/dgtlmoon/changedetection.io/compare/0.39.20...0.39.20.1 Source code(tar.gz) Source code(zip) 0.39.20(Oct 9, 2022) What's Changed Fixes and improvements for playwright fetching errors Use proxies.json instead of proxies.txt by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/945 Tidy up proxies.json logic, adding tests by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/955 Bump notification library by @hmmbob in https://github.com/dgtlmoon/changedetection.io/pull/1002 Brightdata Proxy info by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/1003 Adds support for jq JSON query by @YusefOuda in https://github.com/dgtlmoon/changedetection.io/pull/1001 New Contributors @YusefOuda made their first contribution in https://github.com/dgtlmoon/changedetection.io/pull/1001 Full Changelog: https://github.com/dgtlmoon/changedetection.io/compare/0.39.19.1...0.39.20 Source code(tar.gz) Source code(zip) 0.39.19.1(Sep 11, 2022) Mainly - The system may have not correctly used your system/global notification settings. Now available is a 'Use default notifications' button when using the group selector checkbox in the main watch overview list. What's Changed Fix typo by @ammgws in https://github.com/dgtlmoon/changedetection.io/pull/924 New Contributors @ammgws made their first contribution in https://github.com/dgtlmoon/changedetection.io/pull/924 Full Changelog: https://github.com/dgtlmoon/changedetection.io/compare/0.39.19...0.39.19.1 Source code(tar.gz) Source code(zip) 0.39.19(Sep 8, 2022) What's Changed Remove old function - not needed by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/875 Cleaner separation of watch/global notification settings by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/894 Test the visual selector data loads as JSON by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/895 Prevent crash when UUID doesnt exist here by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/915 Notification settings refine by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/910 Use play and pause icon by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/919 Full Changelog: https://github.com/dgtlmoon/changedetection.io/compare/0.39.18...0.39.19 Source code(tar.gz) Source code(zip) 0.39.18(Aug 19, 2022) TLDR Massive improvements to making errors from websites more easier to understand UI and usability improvements Bug fixes around saving data and processing the queue Home-Assistant integration via Apprise URLs in notifications Various fixes What's Changed Bump apprise to 1.0.0 by @hmmbob in https://github.com/dgtlmoon/changedetection.io/pull/818 Don't process a watch if it was paused AFTER being queued by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/825 Crash protection - handle the case where watch was deleted by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/833 Upgrade playwright python driver by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/834 More concise handling of non-200 HTTP errors by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/824 use smarter way to managed 'last changed' time by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/835 Column sorting by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/838 Make the table header easier to understand when sorting by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/840 Crash fix: Data store sub-directories werent always being created when needed by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/842 'Save chrome screenshot' checkbox never used, removing, we always save the screenshot. by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/844 Be sure visual-selector data is set when filter is not found by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/851 Playwright didnt report errors correctly by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/852 Bug fix for automatically queuing watches to check by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/860 UI - Basic checkbox/group operations by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/861 More checks by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/863 New Contributors @hmmbob made their first contribution in https://github.com/dgtlmoon/changedetection.io/pull/818 Full Changelog: https://github.com/dgtlmoon/changedetection.io/compare/0.39.17.2...0.39.18 Source code(tar.gz) Source code(zip) 0.39.17.2(Aug 5, 2022) What's Changed Feature - mute notifications by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/791 Preview text fix by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/796 Feature - priority queue - edited and added watches should get checked before automatically queued watches by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/799 Remove [save & preview] button, the preview is not updated live so it can lead to confusion by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/801 Handle SIGINT somewhat by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/737 Re #598 - Password could be unset accidently by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/808 Full Changelog: https://github.com/dgtlmoon/changedetection.io/compare/0.39.17.1...0.39.17.2 Source code(tar.gz) Source code(zip) 0.39.17.1(Jul 29, 2022) What's Changed Filter failure detection notification interfering with change-detection results by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/786 Full Changelog: https://github.com/dgtlmoon/changedetection.io/compare/0.39.17...0.39.17.1 Source code(tar.gz) Source code(zip) 0.39.17(Jul 28, 2022) What's Changed Remove duplicate environment variables by @marvin8 in https://github.com/dgtlmoon/changedetection.io/pull/738 Ability to specify JS before running change-detection by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/744 Return all regex results instead of first match by @freddieleeman in https://github.com/dgtlmoon/changedetection.io/pull/730 Added discord webhook base url to truncation rules by @bwees in https://github.com/dgtlmoon/changedetection.io/pull/753 Regex clarification by @rocket357 in https://github.com/dgtlmoon/changedetection.io/pull/766 Notifications - email - Correctly send plaintext notification as plaintext by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/767 Small cleanups by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/768 Bug fix for alerting when xPath based filters are not present by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/772 Bug fix - filter "Only trigger when new lines appear" should check all history, not only the first item by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/777 Enhancement - support xPath text() function filter, for example "//title/text()" in RSS feeds by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/778 UI Feature - Add watch in "paused" state, saving then unpauses by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/779 New Contributors @marvin8 made their first contribution in https://github.com/dgtlmoon/changedetection.io/pull/738 @freddieleeman made their first contribution in https://github.com/dgtlmoon/changedetection.io/pull/730 @bwees made their first contribution in https://github.com/dgtlmoon/changedetection.io/pull/753 @rocket357 made their first contribution in https://github.com/dgtlmoon/changedetection.io/pull/766 Full Changelog: https://github.com/dgtlmoon/changedetection.io/compare/0.39.16...0.39.17 Source code(tar.gz) Source code(zip) 0.39.16(Jul 5, 2022) What's Changed Re #580 - Jitter delay by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/681 Notifications - log what was sent after cleaning up by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/691 Some changes werent getting triggered because the previous checksum only recorded when an event occurred by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/697 Improve docker-compose.yml browserless/playwright docker container, add env var for STEALTH and BLOCK_ADS by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/701 Only set last_changed on the second snapshot, update existing watches by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/715 Minor diff page improvements - list should be sorted 'newest first' and no need to include the current version to compare against by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/716 lang: prefer 'clear (snap) history' to 'scrub' by @jtagcat in https://github.com/dgtlmoon/changedetection.io/pull/721 Re #664 - Use HTTP Request Headers override (Cookie, etc) in playwright by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/723 [feature] Filter option - Trigger only when NEW content (lines) are detected ( compared to earlier text snapshots ) by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/685 Use our own apprise asset format by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/733 New Contributors @jtagcat made their first contribution in https://github.com/dgtlmoon/changedetection.io/pull/721 Full Changelog: https://github.com/dgtlmoon/changedetection.io/compare/0.39.15...0.39.16 Source code(tar.gz) Source code(zip) 0.39.15(Jun 12, 2022) What's Changed Remove tag length validation by @timlod in https://github.com/dgtlmoon/changedetection.io/pull/645 Fixed: Calculate offsetX and offsetY for Visual Selector in Firefox by @lflare in https://github.com/dgtlmoon/changedetection.io/pull/646 Move history data to a textfile by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/638 "Mark all viewed" button - #651 - Adding test and fixing accidently broken implementation by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/652 Fixing RSS feed HTML content formatting by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/662 "Extract text" filter ability by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/624 Improve error message by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/666 Scrub single watch history by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/672 Improve logging by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/671 Also log normal notification activity, make logs easier to find by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/682 Add small preview section check by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/683 New Contributors @lflare made their first contribution in https://github.com/dgtlmoon/changedetection.io/pull/646 Full Changelog: https://github.com/dgtlmoon/changedetection.io/compare/0.39.14...0.39.15 Source code(tar.gz) Source code(zip) 0.39.14.1(May 23, 2022) Adding Visual Selector tool Important PIP install fix Full Changelog: https://github.com/dgtlmoon/changedetection.io/compare/0.39.14...0.39.14.1 Source code(tar.gz) Source code(zip) 0.39.14(May 23, 2022) What's Changed Fix PLAYWRIGHT_DRIVER_URL default value by @weeix in https://github.com/dgtlmoon/changedetection.io/pull/599 Re #214 - configurable extra wait seconds for webdriver requests by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/606 Option to control if pages with no renderable content are a change (like JS webapps that dont render any text sometimes etc) by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/608 Input field tidyup by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/611 Proxy list settings on watch should have a "[ ] default" option by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/610 API Interface by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/617 Full Changelog: https://github.com/dgtlmoon/changedetection.io/compare/0.39.13.1...0.39.14 Source code(tar.gz) Source code(zip) 0.39.13.1(May 10, 2022) What's Changed Update MANIFEST to include model/ subdir for Pip installs by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/594 Full Changelog: https://github.com/dgtlmoon/changedetection.io/compare/0.39.13...0.39.13.1 Source code(tar.gz) Source code(zip) 0.39.13(May 10, 2022) What's Changed wtforms 3 update by @tbhi in https://github.com/dgtlmoon/changedetection.io/pull/523 HTTP Fetcher code improvements by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/539 source: fetcher (compare source code) by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/540 Refactor form handling by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/548 Import speed improvements, and adding an import URL batch size of 5,000 to stop accidental CPU overload by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/549 Shows which items are already in the queue, disables adding to the queue if already in the recheck queue by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/552 Seconds/minutes/hours/days between checks by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/512 Share watch by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/563 Re #569 corrupted diffs by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/571 Handle deletions better by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/570 Bugfix - dont update deleted by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/573 Simplify scrub/remove all history operation by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/575 Add new fetch method: Playwright Chromium (Selenium/WebDriver alterna… by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/489 Toggle 'requests' extra options when not used by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/584 Make proxy configuration more consistent by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/585 Fix PLAYWRIGHT_DRIVER_URL example by @ginkel in https://github.com/dgtlmoon/changedetection.io/pull/588 Ability to specify a list of proxies to choose from, always using the first one by default by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/591 Distill.io import by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/592 New Contributors @ginkel made their first contribution in https://github.com/dgtlmoon/changedetection.io/pull/588 Full Changelog: https://github.com/dgtlmoon/changedetection.io/compare/0.39.12...0.39.13 Source code(tar.gz) Source code(zip) 0.39.12(Apr 9, 2022) What's Changed Minor UI cleanups (mobile and font sizing) by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/503 {diff} and {diff_full} fixes by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/509 RSS feed should contain CDATA of rendered differences by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/510 Remove 'unviewed' status in watch table when clicked by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/514 Screenshot last check by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/516 AJAX button for sending test notifications by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/519 [Add email] button to notification settings with a prefix set from NOTIFICATION_MAIL_BUTTON_PREFIX env var when defined. by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/528 Discord:// notifications should be cut to 2000 chars or Discord will not process them. by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/531 Render hyperlinks option by @AndriiG13 in https://github.com/dgtlmoon/changedetection.io/pull/534 General form cleanups by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/535 Full Changelog: https://github.com/dgtlmoon/changedetection.io/compare/0.39.11...0.39.12 Source code(tar.gz) Source code(zip) 0.39.11(Mar 23, 2022) What's Changed Missed typo ignore -> remove in global settings form by @timlod in https://github.com/dgtlmoon/changedetection.io/pull/464 JSON use the original char encoding by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/465 Auto-byte encoding for utf-8 when its not mentioned in the content-type header reply by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/472 Security update - Protect against file:/// type access by webdriver/chrome. by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/483 Security update - Use CSRF token protection for forms, make remove password use HTTP Post by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/484 Allow changedetector to ignore status codes as a per-site setting (#479) by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/485 Windows pip install fixes by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/492 Full Changelog: https://github.com/dgtlmoon/changedetection.io/compare/0.39.10...0.39.11 Source code(tar.gz) Source code(zip) 0.39.10(Mar 12, 2022) What's Changed Improved CPU usage around site watch management by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/438 Include link to changedetection.io hosted option by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/439 Tweak support tabs and text by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/440 Refactor tests by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/443 Upgrade inscriptis to 2.2~ by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/434 Set auth cookie control path relative to X-Forwarded-Prefix when running via reverse proxy subdirectory by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/446 Handle the case where the visitor is already logged-in and tries to login again by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/447 Refactor HTTP Request type check by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/453 Fix XPath text by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/457 Add filter to remove elements by CSS rule from HTML before change detection is run by @timlod in https://github.com/dgtlmoon/changedetection.io/pull/445 Full Changelog: https://github.com/dgtlmoon/changedetection.io/compare/0.39.9...0.39.10 Source code(tar.gz) Source code(zip) 0.39.9(Feb 24, 2022) What's Changed Dont allow redirect on login, it's safer and more reliable this way by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/414 Bump node-sass from 6.0.1 to 7.0.0 in /changedetectionio/static/styles by @dependabot in https://github.com/dgtlmoon/changedetection.io/pull/415 Make errors from the HTTP fetchers easier to find by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/421 Add re: namespace for regEx support in xpath by @michael-kerbel in https://github.com/dgtlmoon/changedetection.io/pull/428 UI/highlight ignores by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/426 "Recheck" button should work when entry is in paused state by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/435 Minor reliability upgrade for large datasets - retry deepcopy by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/436 New Contributors @michael-kerbel made their first contribution in https://github.com/dgtlmoon/changedetection.io/pull/428 Full Changelog: https://github.com/dgtlmoon/changedetection.io/compare/0.39.8...0.39.9 Source code(tar.gz) Source code(zip) 0.39.8(Feb 8, 2022) What's Changed Fix bug where diff and diff_full were switched in notification templates. by @tjhowse in https://github.com/dgtlmoon/changedetection.io/pull/380 Add option for tags on import by @timlod in https://github.com/dgtlmoon/changedetection.io/pull/377 text/plain mime type fix - Don't lose line-feeds by parsing it as HTML ( text/plain documents were appearing as all on one line ) by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/391 #323 Adding note about discord:// 2000 char limit by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/392 Ability to use a generated salted password in deployments as env var SALTED_PASS by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/397 fixed the reference to wiki for rpi section by @rursache in https://github.com/dgtlmoon/changedetection.io/pull/402 Notification error log handler by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/403 Notification error log handler by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/404 Introduce -h option by @Al2Klimov in https://github.com/dgtlmoon/changedetection.io/pull/406 Fix typo by @Al2Klimov in https://github.com/dgtlmoon/changedetection.io/pull/405 Number of fetching workers can be overriden by Env "FETCH_WORKERS" by @dgtlmoon in https://github.com/dgtlmoon/changedetection.io/pull/413 New Contributors @tjhowse made their first contribution in https://github.com/dgtlmoon/changedetection.io/pull/380 @timlod made their first contribution in https://github.com/dgtlmoon/changedetection.io/pull/377 @rursache made their first contribution in https://github.com/dgtlmoon/changedetection.io/pull/402 @Al2Klimov made their first contribution in https://github.com/dgtlmoon/changedetection.io/pull/406 Full Changelog: https://github.com/dgtlmoon/changedetection.io/compare/0.39.7...0.39.8 Source code(tar.gz) Source code(zip)