Your self hosted Youtube media server

Overview

banner-tube-archivist-light.png

The Tube Archivist
Your self hosted Youtube media server

Core functionality

  • Subscribe to your favourite Youtube channels
  • Download Videos using yt-dlp
  • Index and make videos searchable
  • Play videos
  • Keep track of viewed and unviewed videos

Screenshots

home screenshot
Home Page

channels screenshot
All Channels

single channel screenshot
Single Channel

video page screenshot
Video Page

video page screenshot
Downloads Page

Problem Tube Archivist tries to solve

Once your Youtube video collection grows, it becomes hard to search and find a specific video. That's where Tube Archivist comes in: By indexing your video collection with metadata from Youtube, you can organize, search and enjoy your archived Youtube videos without hassle offline through a convenient web interface.

Installation

Take a look at the example docker-compose.yml file provided. Tube Archivist depends on three main components split up into seperate docker containers:

Tube Archivist

The main Python application that displays and serves your video collection, built with Django.

  • Serves the interface on port 8000
  • Needs a mandatory volume for the video archive at /youtube
  • And another recommended volume to save the cache for thumbnails and artwork at /cache.
  • The environment variables ES_URL and REDIS_HOST are needed to tell Tube Archivist where Elasticsearch and Redis respectively are located.
  • The environment variables HOST_UID and HOST_GID allowes Tube Archivist to chown the video files to the main host system user instead of the container user.

Elasticsearch

Stores video meta data and makes everything searchable. Also keeps track of the download queue.

  • Needs to be accessable over the default port 9200
  • Needs a volume at /usr/share/elasticsearch/data to store data

Follow the documentation for additional installation details.

Redis JSON

Functions as a cache and temporary link between the application and the filesystem. Used to store and display messages and configuration variables.

  • Needs to be accessable over the default port 6379
  • Takes an optional volume at /data to make your configuration changes permanent.

Getting Started

  1. Go through the settings page and look at the available options. Particularly set Download Format to your desired video quality before downloading.
  2. Subscribe to some of your favourite Youtube channels on the channels page.
  3. On the downloads page, click on Rescan subscriptions to add videos from the subscribed channels to your Download queue or click on Add to download queue to manually add Video IDs, links, channels or playlists.
  4. Click on Download queue and let Tube Archivist to it's thing.
  5. Enjoy your archived collection!

Potential pitfalls

Elastic Search in Docker requires the kernel setting of the host machine vm.max_map_count to be set to least 262144.

To temporary set the value run:

sudo sysctl -w vm.max_map_count=262144

To apply the change permanently depends on your host operating system:

  • For example on Ubuntu Server add vm.max_map_count = 262144 to the file /etc/sysctl.conf.
  • On Arch based systems create a file /etc/sysctl.d/max_map_count.conf with the content vm.max_map_count = 262144.
  • On any other platform look up in the documentation on how to pass kernel parameters.

Roadmap

This should be considered as a minimal viable product, there is an exstensive list of future functions and improvements planned:

  • Scan your filesystem to manually add videos
  • Access controll
  • User roles
  • Delete videos and channel
  • Create playlists
  • Show similar videos on video page
  • Import existing downloaded archive
  • Multi language support
  • Backup and restore

Known limitations

  • Video files created by Tube Archivist need to be mp4 video files for best browser compatibility.
  • Every limitation of yt-dlp will also be present in Tube Archivist. If yt-dlp can't download or extract a video for any reason, Tube Archivist won't be able to either.
  • For now this is meant to be run in a trusted network environment.
Issues
  • 500 error on some channels

    500 error on some channels

    https://www.youtube.com/c/BlakeWeber https://www.youtube.com/user/DrunkenWoodworker

    cant add channel. there are a few others. All use the /c/channelname

    try to add it and 500 error. if i add a video from the channel it adds it fine

    opened by Code-Slave 14
  • Videos wont play

    Videos wont play

    Ive left all setting at default. a couple things are happening

    It seems dls are stalling and i have to hit download que multiple time to get it going again. nothing helpfule in logs

    when playing a video in firefox (linux) i get no video with supported format and mimetime found. on chrome i get no controls and or anything, just a screen with the beginning videothumnail i assume

    opened by Code-Slave 14
  • ElasticsearchException[failed to bind service] with new TA install

    ElasticsearchException[failed to bind service] with new TA install

    Hey, just saw your post on Reddit and thought I'd give it a whirl. Ran into an issue getting ElasticSearch started, not sure why. I thought (based on the "Access Denied" in the logs) it was a permission error, so I already chown -R admin:admin (which is who UID and GID for 1001 is on my system) all the folders that TA uses to run, brought the project down and back up again, no dice.

    System Details:

    OS: Debian 10
    Docker version: 20.10.8, build 3967b7d
    

    docker-compose.yml file:

    version: '3.3'
    
    services:
      tubearchivist:
        container_name: tubearchivist
        restart: always
        image: bbilly1/tubearchivist:latest
        ports:
          - 8008:8000
        volumes:
          - /mnt/storage/media/youtube/media:/youtube
          - /mnt/storage/media/youtube/cache:/cache
        environment:
          - ES_URL=http://archivist-es:9200
          - REDIS_HOST=archivist-redis
          - HOST_UID=1001
          - HOST_GID=1001
        depends_on:
          - archivist-es
          - archivist-redis
      archivist-redis:
        image: redislabs/rejson:latest
        container_name: archivist-redis
        restart: always
        ports:
          - 6379:6379
        volumes:
          - /mnt/storage/media/youtube/redis:/data
        depends_on:
          - archivist-es
      archivist-es:
        image: docker.elastic.co/elasticsearch/elasticsearch:7.14.1
        container_name: archivist-es
        restart: always
        environment:
          - "discovery.type=single-node"
          - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
        ulimits:
          memlock:
            soft: -1
            hard: -1
        volumes:
          - /mnt/storage/media/youtube/es:/usr/share/elasticsearch/data
        ports:
          - 9200:9200
    

    docker logs -f archivist-es output:

    {"type": "server", "timestamp": "2021-09-15T21:42:17,960Z", "level": "ERROR", "component": "o.e.b.ElasticsearchUncaughtExceptionHandler", "cluster.name": "docker-cluster", "node.name": "2928085f95ba", "message": "uncaught exception in thread [main]", 
    "stacktrace": ["org.elasticsearch.bootstrap.StartupException: ElasticsearchException[failed to bind service]; nested: AccessDeniedException[/usr/share/elasticsearch/data/nodes];",
    "at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:163) ~[elasticsearch-7.14.1.jar:7.14.1]",
    "at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150) ~[elasticsearch-7.14.1.jar:7.14.1]",
    "at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:75) ~[elasticsearch-7.14.1.jar:7.14.1]",
    "at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:116) ~[elasticsearch-cli-7.14.1.jar:7.14.1]",
    "at org.elasticsearch.cli.Command.main(Command.java:79) ~[elasticsearch-cli-7.14.1.jar:7.14.1]",
    "at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:115) ~[elasticsearch-7.14.1.jar:7.14.1]",
    "at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:81) ~[elasticsearch-7.14.1.jar:7.14.1]",
    "Caused by: org.elasticsearch.ElasticsearchException: failed to bind service",
    "at org.elasticsearch.node.Node.<init>(Node.java:798) ~[elasticsearch-7.14.1.jar:7.14.1]",
    "at org.elasticsearch.node.Node.<init>(Node.java:281) ~[elasticsearch-7.14.1.jar:7.14.1]",
    "at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:219) ~[elasticsearch-7.14.1.jar:7.14.1]",
    "at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:219) ~[elasticsearch-7.14.1.jar:7.14.1]",
    "at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:399) ~[elasticsearch-7.14.1.jar:7.14.1]",
    "at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-7.14.1.jar:7.14.1]",
    "... 6 more",
    "Caused by: java.nio.file.AccessDeniedException: /usr/share/elasticsearch/data/nodes",
    "at sun.nio.fs.UnixException.translateToIOException(UnixException.java:90) ~[?:?]",
    "at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106) ~[?:?]",
    "at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111) ~[?:?]",
    "at sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:396) ~[?:?]",
    "at java.nio.file.Files.createDirectory(Files.java:694) ~[?:?]",
    "at java.nio.file.Files.createAndCheckIsDirectory(Files.java:801) ~[?:?]",
    "at java.nio.file.Files.createDirectories(Files.java:787) ~[?:?]",
    "at org.elasticsearch.env.NodeEnvironment.lambda$new$0(NodeEnvironment.java:265) ~[elasticsearch-7.14.1.jar:7.14.1]",
    "at org.elasticsearch.env.NodeEnvironment$NodeLock.<init>(NodeEnvironment.java:202) ~[elasticsearch-7.14.1.jar:7.14.1]",
    "at org.elasticsearch.env.NodeEnvironment.<init>(NodeEnvironment.java:262) ~[elasticsearch-7.14.1.jar:7.14.1]",
    "at org.elasticsearch.node.Node.<init>(Node.java:376) ~[elasticsearch-7.14.1.jar:7.14.1]",
    "at org.elasticsearch.node.Node.<init>(Node.java:281) ~[elasticsearch-7.14.1.jar:7.14.1]",
    "at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:219) ~[elasticsearch-7.14.1.jar:7.14.1]",
    "at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:219) ~[elasticsearch-7.14.1.jar:7.14.1]",
    "at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:399) ~[elasticsearch-7.14.1.jar:7.14.1]",
    "at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-7.14.1.jar:7.14.1]",
    "... 6 more"] }
    uncaught exception in thread [main]
    ElasticsearchException[failed to bind service]; nested: AccessDeniedException[/usr/share/elasticsearch/data/nodes];
    Likely root cause: java.nio.file.AccessDeniedException: /usr/share/elasticsearch/data/nodes
    	at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:90)
    	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)
    	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
    	at java.base/sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:396)
    	at java.base/java.nio.file.Files.createDirectory(Files.java:694)
    	at java.base/java.nio.file.Files.createAndCheckIsDirectory(Files.java:801)
    	at java.base/java.nio.file.Files.createDirectories(Files.java:787)
    	at org.elasticsearch.env.NodeEnvironment.lambda$new$0(NodeEnvironment.java:265)
    	at org.elasticsearch.env.NodeEnvironment$NodeLock.<init>(NodeEnvironment.java:202)
    	at org.elasticsearch.env.NodeEnvironment.<init>(NodeEnvironment.java:262)
    	at org.elasticsearch.node.Node.<init>(Node.java:376)
    	at org.elasticsearch.node.Node.<init>(Node.java:281)
    	at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:219)
    	at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:219)
    	at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:399)
    	at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159)
    	at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150)
    	at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:75)
    	at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:116)
    	at org.elasticsearch.cli.Command.main(Command.java:79)
    	at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:115)
    	at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:81)
    For complete error details, refer to the log at /usr/share/elasticsearch/logs/docker-cluster.log
    

    Also, I've checked /mnt/storage/media/youtube/es and there's no log file there, sadly. Let me know if you need anymore information.

    opened by Alcatraz077 11
  • Help setting this up

    Help setting this up

    Hey! I'm running trying to set this up on a QNAP NAS. It's running QTS's Docker Station (I'm not sure if any of this will be important information). Anyways, I pulled the Docker-compose file and just changed the locations for the files it'll pull. When I navigate to the URL for the server, the browser just displays this json:

    { "name" : "305f54c99168", "cluster_name" : "docker-cluster", "cluster_uuid" : "B270t1WITSWJ1m1ooeqA1Q", "version" : { "number" : "7.14.1", "build_flavor" : "default", "build_type" : "docker", "build_hash" : "66b55ebfa59c92c15db3f69a335d500018b3331e", "build_date" : "2021-08-26T09:01:05.390870785Z", "build_snapshot" : false, "lucene_version" : "8.9.0", "minimum_wire_compatibility_version" : "6.8.0", "minimum_index_compatibility_version" : "6.0.0-beta1" }, "tagline" : "You Know, for Search" }

    I also got a TON of messages in the console terminal, I'm not sure what any of them mean. I think I might actually have more of this, the terminal might've cleared away some of it when I was checking on something else. It's attached here --> https://pastebin.com/kggPyasL Some of these are about my currently high disk usage - funnily enough, a lot of those are YouTube videos, and I'd love to move them onto Tubearchivist as soon as I can get it working.

    opened by Brancliff 9
  • Scanning of subscriptions

    Scanning of subscriptions

    Currently, the current channel page size (default=50) is the number of videos scanned from the newest backwards, however if I just started to archive a new channel, it would probably mean that 50 is insufficient. Instead of manually changing the value to something else, is it possible to instead let it scan for the latest 50 not downloaded videos?

    opened by ClementNgZiQian 8
  • Manual media import not available

    Manual media import not available

    Not really sure what I'm doing wrong here, but the manual import still says "Coming soon". I've tried removing and re-creating the containers, re-downloading the code form GitHub but nothing seems to help. What am I overlooking?

    opened by kajatak 8
  • Question about populating the archive manually

    Question about populating the archive manually

    Looks like a great project. I was trying to use Plex Media Server to manage and share my music video collection, but Plex is not very good with music videos.

    However, before I install & use it, I do have a few questions about TA (short for the tube archivist):

    1. I have 15-20 YT playlists, each having 10's to 100's of videos. Is it possible to directly populate TA from a playlist?
    2. I have an existing library on my hard disk - a collection of 1000's of MP4 videos - already downloaded from YT. The docs say that for each video I need to create a JSON file matching the video file. What are the contents of the JSON, or is empty?

    Thanks and keep up the good work!

    opened by akrk1986 5
  • Any way to get nfo files for Emby/Plex Metadata?

    Any way to get nfo files for Emby/Plex Metadata?

    My main goal is to get the embedded thumbnails to display in Emby. I've come to realize this can't be done without an nfo file? I'm not sure if this can be done or not with tubearchivist. Can it create nfo files for metadata? FYI I do have the embedded metadata option enabled in settings.

    opened by geekedtv 5
  • Server Not Responding with deploy.sh test [with workaround]

    Server Not Responding with deploy.sh test [with workaround]

    Issue:

    When building the local environment using the deploy.sh script, the tubearchivist image does not properly instantiate the web server.

    Details:

    Using the base build details from the current testing or primary branches and then attempting to start the tubearchivist docker image, the server does not start appropriately. This is because the web service is unable to build the style.css file properly due to missing references (.tff.woff files for fonts). The server supplies a 500 response because there isn't a good home page (or any page, for that matter) to reference.

    Expected Result:

    Server returns a 200 status response and supplies the appropriate web page response.

    Actual Result:

    Server returns a 500 status response and supplies no web page with the response.

    Workaround Found:

    Docker has a copy command, via docker cp, which can copy files locally. Copying the requested files (pulled from an existing, previous version and working container) into the expected location and then restarting the container allows for it to restart the web service properly and supply the Expected Result.

    Note:

    Attempting to create the directory in the build location before creating the docker image did not produce the Expected Result. Only copying the files into the directory via docker cp worked to actually bring the web server into a responsive state.

    Logs and command outputs:

    Building with deploy.sh test

    building file list ...
    114 files to consider
    sending incremental file list
    docker-compose.yml
              1,098 100%  388.67kB/s    0:00:00 (xfr#1, to-chk=0/1)
    Sending build context to Docker daemon  471.6kB
    Step 1/19 : FROM python:3.9.7-slim-bullseye
     ---> e455ca30507a
    Step 2/19 : ENV PYTHONUNBUFFERED 1
     ---> Using cache
     ---> e3bb53f200c1
    Step 3/19 : RUN apt-get clean && apt-get -y update && apt-get -y install --no-install-recommends     build-essential     nginx     curl && rm -rf /var/lib/apt/lists/*
     ---> Using cache
     ---> d8a0c4648191
    Step 4/19 : RUN curl -s https://api.github.com/repos/yt-dlp/FFmpeg-Builds/releases/latest     | grep browser_download_url     | grep linux64-nonfree.tar.xz     | cut -d '"' -f 4     | xargs curl -L --output ffmpeg.tar.xz &&     tar -xf ffmpeg.tar.xz --strip-components=2 --no-anchored -C /usr/bin/ "ffmpeg" &&     tar -xf ffmpeg.tar.xz --strip-components=2 --no-anchored -C /usr/bin/ "ffprobe" &&     rm ffmpeg.tar.xz
     ---> Using cache
     ---> d50733852bcb
    Step 5/19 : COPY nginx.conf /etc/nginx/conf.d/
     ---> Using cache
     ---> ef6168e041bf
    Step 6/19 : RUN mkdir /cache
     ---> Using cache
     ---> 24ff5e3af03c
    Step 7/19 : RUN mkdir /youtube
     ---> Using cache
     ---> ce16b112c7a5
    Step 8/19 : RUN mkdir /app
     ---> Using cache
     ---> c2c10588e4b3
    Step 9/19 : COPY ./tubearchivist/requirements.txt /requirements.txt
     ---> Using cache
     ---> f9384103603b
    Step 10/19 : RUN pip install --no-cache-dir -r requirements.txt --src /usr/local/src
     ---> Using cache
     ---> 4dba2dd95a6b
    Step 11/19 : COPY ./tubearchivist /app
     ---> Using cache
     ---> 7cae16ac0c75
    Step 12/19 : COPY ./run.sh /app
     ---> Using cache
     ---> 5e11c3b6453e
    Step 13/19 : COPY ./uwsgi.ini /app
     ---> Using cache
     ---> c5ef45c842eb
    Step 14/19 : VOLUME /cache
     ---> Using cache
     ---> c2c2b0302cac
    Step 15/19 : VOLUME /youtube
     ---> Using cache
     ---> c3d30c993f20
    Step 16/19 : WORKDIR /app
     ---> Using cache
     ---> cd9f6c1f7960
    Step 17/19 : EXPOSE 8000
     ---> Using cache
     ---> 5b974a0e9dfc
    Step 18/19 : RUN chmod +x ./run.sh
     ---> Using cache
     ---> 43caaee852bd
    Step 19/19 : CMD ["./run.sh"]
     ---> Using cache
     ---> d2c750d9fec2
    Successfully built d2c750d9fec2
    Successfully tagged bbilly1/tubearchivist:latest
    archivist-es is up-to-date
    archivist-redis is up-to-date
    tubearchivist is up-to-date
    

    Logs showing archivist-es is running and working

    [email protected]:~/tubearchivist-build$ docker logs -f archivist-es
    {"type": "server", "timestamp": "2021-10-05T21:18:13,432Z", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "version[7.15.0], pid[8], build[default/docker/79d65f6e357953a5b3cbcc5e2c7c21073d89aa29/2021-09-16T03:05:29.143308416Z], OS[Linux/5.4.0-88-generic/amd64], JVM[Eclipse Foundation/OpenJDK 64-Bit Server VM/16.0.2/16.0.2+7]" }
    {"type": "server", "timestamp": "2021-10-05T21:18:13,451Z", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "JVM home [/usr/share/elasticsearch/jdk], using bundled JDK [true]" }
    {"type": "server", "timestamp": "2021-10-05T21:18:13,453Z", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "JVM arguments [-Xshare:auto, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -XX:+ShowCodeDetailsInExceptionMessages, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.locale.providers=SPI,COMPAT, --add-opens=java.base/java.io=ALL-UNNAMED, -XX:+UseG1GC, -Djava.io.tmpdir=/tmp/elasticsearch-3159410211284329604, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -Des.cgroups.hierarchy.override=/, -Xms512m, -Xmx512m, -XX:MaxDirectMemorySize=268435456, -XX:G1HeapRegionSize=4m, -XX:InitiatingHeapOccupancyPercent=30, -XX:G1ReservePercent=15, -Des.path.home=/usr/share/elasticsearch, -Des.path.conf=/usr/share/elasticsearch/config, -Des.distribution.flavor=default, -Des.distribution.type=docker, -Des.bundled_jdk=true]" }
    {"type": "server", "timestamp": "2021-10-05T21:18:24,628Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "loaded module [aggs-matrix-stats]" }
    {"type": "server", "timestamp": "2021-10-05T21:18:24,629Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "loaded module [analysis-common]" }
    {"type": "server", "timestamp": "2021-10-05T21:18:24,631Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "loaded module [constant-keyword]" }
    {"type": "server", "timestamp": "2021-10-05T21:18:24,633Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "loaded module [frozen-indices]" }
    {"type": "server", "timestamp": "2021-10-05T21:18:24,635Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "loaded module [ingest-common]" }
    {"type": "server", "timestamp": "2021-10-05T21:18:24,644Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "loaded module [ingest-geoip]" }
    {"type": "server", "timestamp": "2021-10-05T21:18:24,645Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "loaded module [ingest-user-agent]" }
    {"type": "server", "timestamp": "2021-10-05T21:18:24,646Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "loaded module [kibana]" }
    {"type": "server", "timestamp": "2021-10-05T21:18:24,647Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "loaded module [lang-expression]" }
    {"type": "server", "timestamp": "2021-10-05T21:18:24,647Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "loaded module [lang-mustache]" }
    {"type": "server", "timestamp": "2021-10-05T21:18:24,648Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "loaded module [lang-painless]" }
    {"type": "server", "timestamp": "2021-10-05T21:18:24,649Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "loaded module [mapper-extras]" }
    {"type": "server", "timestamp": "2021-10-05T21:18:24,652Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "loaded module [mapper-version]" }
    {"type": "server", "timestamp": "2021-10-05T21:18:24,654Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "loaded module [parent-join]" }
    {"type": "server", "timestamp": "2021-10-05T21:18:24,655Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "loaded module [percolator]" }
    {"type": "server", "timestamp": "2021-10-05T21:18:24,656Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "loaded module [rank-eval]" }
    {"type": "server", "timestamp": "2021-10-05T21:18:24,658Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "loaded module [reindex]" }
    {"type": "server", "timestamp": "2021-10-05T21:18:24,658Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "loaded module [repositories-metering-api]" }
    {"type": "server", "timestamp": "2021-10-05T21:18:24,659Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "loaded module [repository-encrypted]" }
    {"type": "server", "timestamp": "2021-10-05T21:18:24,661Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "loaded module [repository-url]" }
    {"type": "server", "timestamp": "2021-10-05T21:18:24,663Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "loaded module [runtime-fields-common]" }
    {"type": "server", "timestamp": "2021-10-05T21:18:24,666Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "loaded module [search-business-rules]" }
    {"type": "server", "timestamp": "2021-10-05T21:18:24,667Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "loaded module [searchable-snapshots]" }
    {"type": "server", "timestamp": "2021-10-05T21:18:24,668Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "loaded module [snapshot-repo-test-kit]" }
    {"type": "server", "timestamp": "2021-10-05T21:18:24,669Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "loaded module [spatial]" }
    {"type": "server", "timestamp": "2021-10-05T21:18:24,670Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "loaded module [transform]" }
    {"type": "server", "timestamp": "2021-10-05T21:18:24,672Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "loaded module [transport-netty4]" }
    {"type": "server", "timestamp": "2021-10-05T21:18:24,673Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "loaded module [unsigned-long]" }
    {"type": "server", "timestamp": "2021-10-05T21:18:24,674Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "loaded module [vector-tile]" }
    {"type": "server", "timestamp": "2021-10-05T21:18:24,675Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "loaded module [vectors]" }
    {"type": "server", "timestamp": "2021-10-05T21:18:24,677Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "loaded module [wildcard]" }
    {"type": "server", "timestamp": "2021-10-05T21:18:24,678Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "loaded module [x-pack-aggregate-metric]" }
    {"type": "server", "timestamp": "2021-10-05T21:18:24,679Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "loaded module [x-pack-analytics]" }
    {"type": "server", "timestamp": "2021-10-05T21:18:24,681Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "loaded module [x-pack-async]" }
    {"type": "server", "timestamp": "2021-10-05T21:18:24,682Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "loaded module [x-pack-async-search]" }
    {"type": "server", "timestamp": "2021-10-05T21:18:24,684Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "loaded module [x-pack-autoscaling]" }
    {"type": "server", "timestamp": "2021-10-05T21:18:24,685Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "loaded module [x-pack-ccr]" }
    {"type": "server", "timestamp": "2021-10-05T21:18:24,687Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "loaded module [x-pack-core]" }
    {"type": "server", "timestamp": "2021-10-05T21:18:24,688Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "loaded module [x-pack-data-streams]" }
    {"type": "server", "timestamp": "2021-10-05T21:18:24,690Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "loaded module [x-pack-deprecation]" }
    {"type": "server", "timestamp": "2021-10-05T21:18:24,691Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "loaded module [x-pack-enrich]" }
    {"type": "server", "timestamp": "2021-10-05T21:18:24,692Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "loaded module [x-pack-eql]" }
    {"type": "server", "timestamp": "2021-10-05T21:18:24,694Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "loaded module [x-pack-fleet]" }
    {"type": "server", "timestamp": "2021-10-05T21:18:24,696Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "loaded module [x-pack-graph]" }
    {"type": "server", "timestamp": "2021-10-05T21:18:24,698Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "loaded module [x-pack-identity-provider]" }
    {"type": "server", "timestamp": "2021-10-05T21:18:24,699Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "loaded module [x-pack-ilm]" }
    {"type": "server", "timestamp": "2021-10-05T21:18:24,700Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "loaded module [x-pack-logstash]" }
    {"type": "server", "timestamp": "2021-10-05T21:18:24,702Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "loaded module [x-pack-ml]" }
    {"type": "server", "timestamp": "2021-10-05T21:18:24,704Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "loaded module [x-pack-monitoring]" }
    {"type": "server", "timestamp": "2021-10-05T21:18:24,706Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "loaded module [x-pack-ql]" }
    {"type": "server", "timestamp": "2021-10-05T21:18:24,707Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "loaded module [x-pack-rollup]" }
    {"type": "server", "timestamp": "2021-10-05T21:18:24,708Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "loaded module [x-pack-security]" }
    {"type": "server", "timestamp": "2021-10-05T21:18:24,709Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "loaded module [x-pack-shutdown]" }
    {"type": "server", "timestamp": "2021-10-05T21:18:24,711Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "loaded module [x-pack-sql]" }
    {"type": "server", "timestamp": "2021-10-05T21:18:24,715Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "loaded module [x-pack-stack]" }
    {"type": "server", "timestamp": "2021-10-05T21:18:24,717Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "loaded module [x-pack-text-structure]" }
    {"type": "server", "timestamp": "2021-10-05T21:18:24,721Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "loaded module [x-pack-voting-only-node]" }
    {"type": "server", "timestamp": "2021-10-05T21:18:24,722Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "loaded module [x-pack-watcher]" }
    {"type": "server", "timestamp": "2021-10-05T21:18:24,730Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "no plugins loaded" }
    {"type": "server", "timestamp": "2021-10-05T21:18:24,923Z", "level": "INFO", "component": "o.e.e.NodeEnvironment", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "using [1] data paths, mounts [[/usr/share/elasticsearch/data (/dev/mapper/ubuntu--vg-ubuntu--lv)]], net usable_space [13.4gb], net total_space [23.9gb], types [ext4]" }
    {"type": "server", "timestamp": "2021-10-05T21:18:24,924Z", "level": "INFO", "component": "o.e.e.NodeEnvironment", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "heap size [512mb], compressed ordinary object pointers [true]" }
    {"type": "server", "timestamp": "2021-10-05T21:18:25,022Z", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "node name [1f8e61b3fb75], node ID [0cVnt5KFTCu9dyZTyx9xxA], cluster name [docker-cluster], roles [transform, data_frozen, master, remote_cluster_client, data, ml, data_content, data_hot, data_warm, data_cold, ingest]" }
    {"type": "server", "timestamp": "2021-10-05T21:18:50,172Z", "level": "INFO", "component": "o.e.x.m.p.l.CppLogMessageHandler", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "[controller/209] [[email protected]] controller (64 bit): Version 7.15.0 (Build d0ab43b6c551f8) Copyright (c) 2021 Elasticsearch BV" }
    {"type": "server", "timestamp": "2021-10-05T21:18:51,655Z", "level": "INFO", "component": "o.e.x.s.a.s.FileRolesStore", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "parsed [0] roles from file [/usr/share/elasticsearch/config/roles.yml]" }
    {"type": "server", "timestamp": "2021-10-05T21:18:54,181Z", "level": "INFO", "component": "o.e.i.g.LocalDatabases", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "initialized default databases [[GeoLite2-Country.mmdb, GeoLite2-City.mmdb, GeoLite2-ASN.mmdb]], config databases [[]] and watching [/usr/share/elasticsearch/config/ingest-geoip] for changes" }
    {"type": "server", "timestamp": "2021-10-05T21:18:54,187Z", "level": "INFO", "component": "o.e.i.g.DatabaseRegistry", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "initialized database registry, using geoip-databases directory [/tmp/elasticsearch-3159410211284329604/geoip-databases/0cVnt5KFTCu9dyZTyx9xxA]" }
    {"type": "server", "timestamp": "2021-10-05T21:18:58,222Z", "level": "INFO", "component": "o.e.t.NettyAllocator", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "creating NettyAllocator with the following configs: [name=unpooled, suggested_max_allocation_size=1mb, factors={es.unsafe.use_unpooled_allocator=null, g1gc_enabled=true, g1gc_region_size=4mb, heap_size=512mb}]" }
    {"type": "server", "timestamp": "2021-10-05T21:18:58,702Z", "level": "INFO", "component": "o.e.d.DiscoveryModule", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "using discovery type [single-node] and seed hosts providers [settings]" }
    {"type": "server", "timestamp": "2021-10-05T21:19:02,273Z", "level": "INFO", "component": "o.e.g.DanglingIndicesState", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "gateway.auto_import_dangling_indices is disabled, dangling indices will not be automatically detected or imported and must be managed manually" }
    {"type": "server", "timestamp": "2021-10-05T21:19:05,567Z", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "initialized" }
    {"type": "server", "timestamp": "2021-10-05T21:19:05,571Z", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "starting ..." }
    {"type": "server", "timestamp": "2021-10-05T21:19:06,631Z", "level": "INFO", "component": "o.e.x.s.c.f.PersistentCache", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "persistent cache index loaded" }
    {"type": "server", "timestamp": "2021-10-05T21:19:07,161Z", "level": "INFO", "component": "o.e.t.TransportService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "publish_address {172.24.0.2:9300}, bound_addresses {0.0.0.0:9300}" }
    {"type": "server", "timestamp": "2021-10-05T21:19:10,834Z", "level": "WARN", "component": "o.e.b.BootstrapChecks", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]" }
    {"type": "server", "timestamp": "2021-10-05T21:19:10,875Z", "level": "INFO", "component": "o.e.c.c.Coordinator", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "setting initial configuration to VotingConfiguration{0cVnt5KFTCu9dyZTyx9xxA}" }
    {"type": "server", "timestamp": "2021-10-05T21:19:13,366Z", "level": "INFO", "component": "o.e.c.s.MasterService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "elected-as-master ([1] nodes joined)[{1f8e61b3fb75}{0cVnt5KFTCu9dyZTyx9xxA}{GCjWF8ZcT-CVDKYXpxDpfQ}{172.24.0.2}{172.24.0.2:9300}{cdfhilmrstw} elect leader, _BECOME_MASTER_TASK_, _FINISH_ELECTION_], term: 1, version: 1, delta: master node changed {previous [], current [{1f8e61b3fb75}{0cVnt5KFTCu9dyZTyx9xxA}{GCjWF8ZcT-CVDKYXpxDpfQ}{172.24.0.2}{172.24.0.2:9300}{cdfhilmrstw}]}" }
    {"type": "server", "timestamp": "2021-10-05T21:19:13,952Z", "level": "INFO", "component": "o.e.c.c.CoordinationState", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "cluster UUID set to [o4COgb3RTwOsisd4j6DiZg]" }
    {"type": "server", "timestamp": "2021-10-05T21:19:14,429Z", "level": "INFO", "component": "o.e.c.s.ClusterApplierService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "master node changed {previous [], current [{1f8e61b3fb75}{0cVnt5KFTCu9dyZTyx9xxA}{GCjWF8ZcT-CVDKYXpxDpfQ}{172.24.0.2}{172.24.0.2:9300}{cdfhilmrstw}]}, term: 1, version: 1, reason: Publication{term=1, version=1}" }
    {"type": "server", "timestamp": "2021-10-05T21:19:14,722Z", "level": "INFO", "component": "o.e.h.AbstractHttpServerTransport", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "publish_address {172.24.0.2:9200}, bound_addresses {0.0.0.0:9200}", "cluster.uuid": "o4COgb3RTwOsisd4j6DiZg", "node.id": "0cVnt5KFTCu9dyZTyx9xxA"  }
    {"type": "server", "timestamp": "2021-10-05T21:19:14,724Z", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "started", "cluster.uuid": "o4COgb3RTwOsisd4j6DiZg", "node.id": "0cVnt5KFTCu9dyZTyx9xxA"  }
    {"type": "server", "timestamp": "2021-10-05T21:19:15,772Z", "level": "INFO", "component": "o.e.g.GatewayService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "recovered [0] indices into cluster_state", "cluster.uuid": "o4COgb3RTwOsisd4j6DiZg", "node.id": "0cVnt5KFTCu9dyZTyx9xxA"  }
    {"type": "server", "timestamp": "2021-10-05T21:19:16,816Z", "level": "INFO", "component": "o.e.c.m.MetadataIndexTemplateService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "adding index template [.ml-stats] for index patterns [.ml-stats-*]", "cluster.uuid": "o4COgb3RTwOsisd4j6DiZg", "node.id": "0cVnt5KFTCu9dyZTyx9xxA"  }
    {"type": "server", "timestamp": "2021-10-05T21:19:17,487Z", "level": "INFO", "component": "o.e.c.m.MetadataIndexTemplateService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "adding component template [data-streams-mappings]", "cluster.uuid": "o4COgb3RTwOsisd4j6DiZg", "node.id": "0cVnt5KFTCu9dyZTyx9xxA"  }
    {"type": "server", "timestamp": "2021-10-05T21:19:17,969Z", "level": "INFO", "component": "o.e.c.m.MetadataIndexTemplateService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "adding index template [.ml-state] for index patterns [.ml-state*]", "cluster.uuid": "o4COgb3RTwOsisd4j6DiZg", "node.id": "0cVnt5KFTCu9dyZTyx9xxA"  }
    {"type": "server", "timestamp": "2021-10-05T21:19:18,663Z", "level": "INFO", "component": "o.e.c.m.MetadataIndexTemplateService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "adding component template [logs-mappings]", "cluster.uuid": "o4COgb3RTwOsisd4j6DiZg", "node.id": "0cVnt5KFTCu9dyZTyx9xxA"  }
    {"type": "server", "timestamp": "2021-10-05T21:19:19,553Z", "level": "INFO", "component": "o.e.c.m.MetadataIndexTemplateService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "adding index template [.ml-anomalies-] for index patterns [.ml-anomalies-*]", "cluster.uuid": "o4COgb3RTwOsisd4j6DiZg", "node.id": "0cVnt5KFTCu9dyZTyx9xxA"  }
    {"type": "server", "timestamp": "2021-10-05T21:19:20,167Z", "level": "INFO", "component": "o.e.c.m.MetadataIndexTemplateService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "adding index template [.ml-notifications-000002] for index patterns [.ml-notifications-000002]", "cluster.uuid": "o4COgb3RTwOsisd4j6DiZg", "node.id": "0cVnt5KFTCu9dyZTyx9xxA"  }
    {"type": "server", "timestamp": "2021-10-05T21:19:20,695Z", "level": "INFO", "component": "o.e.c.m.MetadataIndexTemplateService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "adding component template [logs-settings]", "cluster.uuid": "o4COgb3RTwOsisd4j6DiZg", "node.id": "0cVnt5KFTCu9dyZTyx9xxA"  }
    {"type": "server", "timestamp": "2021-10-05T21:19:21,359Z", "level": "INFO", "component": "o.e.c.m.MetadataIndexTemplateService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "adding component template [synthetics-settings]", "cluster.uuid": "o4COgb3RTwOsisd4j6DiZg", "node.id": "0cVnt5KFTCu9dyZTyx9xxA"  }
    {"type": "server", "timestamp": "2021-10-05T21:19:21,896Z", "level": "INFO", "component": "o.e.c.m.MetadataIndexTemplateService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "adding component template [synthetics-mappings]", "cluster.uuid": "o4COgb3RTwOsisd4j6DiZg", "node.id": "0cVnt5KFTCu9dyZTyx9xxA"  }
    {"type": "server", "timestamp": "2021-10-05T21:19:22,353Z", "level": "INFO", "component": "o.e.c.m.MetadataIndexTemplateService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "adding component template [metrics-mappings]", "cluster.uuid": "o4COgb3RTwOsisd4j6DiZg", "node.id": "0cVnt5KFTCu9dyZTyx9xxA"  }
    {"type": "server", "timestamp": "2021-10-05T21:19:22,776Z", "level": "INFO", "component": "o.e.c.m.MetadataIndexTemplateService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "adding component template [metrics-settings]", "cluster.uuid": "o4COgb3RTwOsisd4j6DiZg", "node.id": "0cVnt5KFTCu9dyZTyx9xxA"  }
    {"type": "server", "timestamp": "2021-10-05T21:19:23,647Z", "level": "INFO", "component": "o.e.c.m.MetadataIndexTemplateService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "adding index template [.watch-history-13] for index patterns [.watcher-history-13*]", "cluster.uuid": "o4COgb3RTwOsisd4j6DiZg", "node.id": "0cVnt5KFTCu9dyZTyx9xxA"  }
    {"type": "server", "timestamp": "2021-10-05T21:19:24,093Z", "level": "INFO", "component": "o.e.c.m.MetadataIndexTemplateService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "adding index template [ilm-history] for index patterns [ilm-history-5*]", "cluster.uuid": "o4COgb3RTwOsisd4j6DiZg", "node.id": "0cVnt5KFTCu9dyZTyx9xxA"  }
    {"type": "server", "timestamp": "2021-10-05T21:19:24,503Z", "level": "INFO", "component": "o.e.c.m.MetadataIndexTemplateService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "adding index template [.slm-history] for index patterns [.slm-history-5*]", "cluster.uuid": "o4COgb3RTwOsisd4j6DiZg", "node.id": "0cVnt5KFTCu9dyZTyx9xxA"  }
    {"type": "server", "timestamp": "2021-10-05T21:19:24,941Z", "level": "INFO", "component": "o.e.c.m.MetadataIndexTemplateService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "adding component template [.deprecation-indexing-mappings]", "cluster.uuid": "o4COgb3RTwOsisd4j6DiZg", "node.id": "0cVnt5KFTCu9dyZTyx9xxA"  }
    {"type": "server", "timestamp": "2021-10-05T21:19:25,521Z", "level": "INFO", "component": "o.e.c.m.MetadataIndexTemplateService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "adding component template [.deprecation-indexing-settings]", "cluster.uuid": "o4COgb3RTwOsisd4j6DiZg", "node.id": "0cVnt5KFTCu9dyZTyx9xxA"  }
    {"type": "server", "timestamp": "2021-10-05T21:19:25,887Z", "level": "INFO", "component": "o.e.c.m.MetadataIndexTemplateService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "adding template [.monitoring-alerts-7] for index patterns [.monitoring-alerts-7]", "cluster.uuid": "o4COgb3RTwOsisd4j6DiZg", "node.id": "0cVnt5KFTCu9dyZTyx9xxA"  }
    {"type": "server", "timestamp": "2021-10-05T21:19:26,506Z", "level": "INFO", "component": "o.e.c.m.MetadataIndexTemplateService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "adding template [.monitoring-es] for index patterns [.monitoring-es-7-*]", "cluster.uuid": "o4COgb3RTwOsisd4j6DiZg", "node.id": "0cVnt5KFTCu9dyZTyx9xxA"  }
    {"type": "server", "timestamp": "2021-10-05T21:19:26,994Z", "level": "INFO", "component": "o.e.c.m.MetadataIndexTemplateService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "adding template [.monitoring-kibana] for index patterns [.monitoring-kibana-7-*]", "cluster.uuid": "o4COgb3RTwOsisd4j6DiZg", "node.id": "0cVnt5KFTCu9dyZTyx9xxA"  }
    {"type": "server", "timestamp": "2021-10-05T21:19:27,605Z", "level": "INFO", "component": "o.e.c.m.MetadataIndexTemplateService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "adding template [.monitoring-logstash] for index patterns [.monitoring-logstash-7-*]", "cluster.uuid": "o4COgb3RTwOsisd4j6DiZg", "node.id": "0cVnt5KFTCu9dyZTyx9xxA"  }
    {"type": "server", "timestamp": "2021-10-05T21:19:28,045Z", "level": "INFO", "component": "o.e.c.m.MetadataIndexTemplateService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "adding template [.monitoring-beats] for index patterns [.monitoring-beats-7-*]", "cluster.uuid": "o4COgb3RTwOsisd4j6DiZg", "node.id": "0cVnt5KFTCu9dyZTyx9xxA"  }
    {"type": "server", "timestamp": "2021-10-05T21:19:28,597Z", "level": "INFO", "component": "o.e.c.m.MetadataIndexTemplateService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "adding index template [logs] for index patterns [logs-*-*]", "cluster.uuid": "o4COgb3RTwOsisd4j6DiZg", "node.id": "0cVnt5KFTCu9dyZTyx9xxA"  }
    {"type": "server", "timestamp": "2021-10-05T21:19:28,983Z", "level": "INFO", "component": "o.e.c.m.MetadataIndexTemplateService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "adding index template [synthetics] for index patterns [synthetics-*-*]", "cluster.uuid": "o4COgb3RTwOsisd4j6DiZg", "node.id": "0cVnt5KFTCu9dyZTyx9xxA"  }
    {"type": "server", "timestamp": "2021-10-05T21:19:29,426Z", "level": "INFO", "component": "o.e.c.m.MetadataCreateIndexService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "[ta_channel] creating index, cause [api], templates [], shards [1]/[0]", "cluster.uuid": "o4COgb3RTwOsisd4j6DiZg", "node.id": "0cVnt5KFTCu9dyZTyx9xxA"  }
    {"type": "server", "timestamp": "2021-10-05T21:19:30,620Z", "level": "INFO", "component": "o.e.c.m.MetadataIndexTemplateService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "adding index template [metrics] for index patterns [metrics-*-*]", "cluster.uuid": "o4COgb3RTwOsisd4j6DiZg", "node.id": "0cVnt5KFTCu9dyZTyx9xxA"  }
    {"type": "server", "timestamp": "2021-10-05T21:19:31,374Z", "level": "INFO", "component": "o.e.c.m.MetadataIndexTemplateService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "adding index template [.deprecation-indexing-template] for index patterns [.logs-deprecation.*]", "cluster.uuid": "o4COgb3RTwOsisd4j6DiZg", "node.id": "0cVnt5KFTCu9dyZTyx9xxA"  }
    {"type": "server", "timestamp": "2021-10-05T21:19:32,104Z", "level": "INFO", "component": "o.e.c.r.a.AllocationService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[ta_channel][0]]]).", "cluster.uuid": "o4COgb3RTwOsisd4j6DiZg", "node.id": "0cVnt5KFTCu9dyZTyx9xxA"  }
    {"type": "server", "timestamp": "2021-10-05T21:19:32,680Z", "level": "INFO", "component": "o.e.x.i.a.TransportPutLifecycleAction", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "adding index lifecycle policy [ml-size-based-ilm-policy]", "cluster.uuid": "o4COgb3RTwOsisd4j6DiZg", "node.id": "0cVnt5KFTCu9dyZTyx9xxA"  }
    {"type": "server", "timestamp": "2021-10-05T21:19:33,307Z", "level": "INFO", "component": "o.e.c.m.MetadataCreateIndexService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "[ta_video] creating index, cause [api], templates [], shards [1]/[0]", "cluster.uuid": "o4COgb3RTwOsisd4j6DiZg", "node.id": "0cVnt5KFTCu9dyZTyx9xxA"  }
    {"type": "server", "timestamp": "2021-10-05T21:19:34,077Z", "level": "INFO", "component": "o.e.x.i.a.TransportPutLifecycleAction", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "adding index lifecycle policy [logs]", "cluster.uuid": "o4COgb3RTwOsisd4j6DiZg", "node.id": "0cVnt5KFTCu9dyZTyx9xxA"  }
    {"type": "server", "timestamp": "2021-10-05T21:19:34,782Z", "level": "INFO", "component": "o.e.x.i.a.TransportPutLifecycleAction", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "adding index lifecycle policy [metrics]", "cluster.uuid": "o4COgb3RTwOsisd4j6DiZg", "node.id": "0cVnt5KFTCu9dyZTyx9xxA"  }
    {"type": "server", "timestamp": "2021-10-05T21:19:35,291Z", "level": "INFO", "component": "o.e.c.r.a.AllocationService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[ta_video][0]]]).", "cluster.uuid": "o4COgb3RTwOsisd4j6DiZg", "node.id": "0cVnt5KFTCu9dyZTyx9xxA"  }
    {"type": "server", "timestamp": "2021-10-05T21:19:35,736Z", "level": "INFO", "component": "o.e.x.i.a.TransportPutLifecycleAction", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "adding index lifecycle policy [synthetics]", "cluster.uuid": "o4COgb3RTwOsisd4j6DiZg", "node.id": "0cVnt5KFTCu9dyZTyx9xxA"  }
    {"type": "server", "timestamp": "2021-10-05T21:19:37,868Z", "level": "INFO", "component": "o.e.c.m.MetadataCreateIndexService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "[ta_download] creating index, cause [api], templates [], shards [1]/[0]", "cluster.uuid": "o4COgb3RTwOsisd4j6DiZg", "node.id": "0cVnt5KFTCu9dyZTyx9xxA"  }
    {"type": "server", "timestamp": "2021-10-05T21:19:38,757Z", "level": "INFO", "component": "o.e.x.i.a.TransportPutLifecycleAction", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "adding index lifecycle policy [watch-history-ilm-policy]", "cluster.uuid": "o4COgb3RTwOsisd4j6DiZg", "node.id": "0cVnt5KFTCu9dyZTyx9xxA"  }
    {"type": "server", "timestamp": "2021-10-05T21:19:40,132Z", "level": "INFO", "component": "o.e.x.i.a.TransportPutLifecycleAction", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "adding index lifecycle policy [ilm-history-ilm-policy]", "cluster.uuid": "o4COgb3RTwOsisd4j6DiZg", "node.id": "0cVnt5KFTCu9dyZTyx9xxA"  }
    {"type": "server", "timestamp": "2021-10-05T21:19:40,810Z", "level": "INFO", "component": "o.e.c.r.a.AllocationService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[ta_download][0]]]).", "cluster.uuid": "o4COgb3RTwOsisd4j6DiZg", "node.id": "0cVnt5KFTCu9dyZTyx9xxA"  }
    {"type": "server", "timestamp": "2021-10-05T21:19:41,583Z", "level": "INFO", "component": "o.e.x.i.a.TransportPutLifecycleAction", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "adding index lifecycle policy [slm-history-ilm-policy]", "cluster.uuid": "o4COgb3RTwOsisd4j6DiZg", "node.id": "0cVnt5KFTCu9dyZTyx9xxA"  }
    {"type": "server", "timestamp": "2021-10-05T21:19:42,413Z", "level": "INFO", "component": "o.e.x.i.a.TransportPutLifecycleAction", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "adding index lifecycle policy [.deprecation-indexing-ilm-policy]", "cluster.uuid": "o4COgb3RTwOsisd4j6DiZg", "node.id": "0cVnt5KFTCu9dyZTyx9xxA"  }
    {"type": "server", "timestamp": "2021-10-05T21:19:43,825Z", "level": "INFO", "component": "o.e.x.i.a.TransportPutLifecycleAction", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "adding index lifecycle policy [.fleet-actions-results-ilm-policy]", "cluster.uuid": "o4COgb3RTwOsisd4j6DiZg", "node.id": "0cVnt5KFTCu9dyZTyx9xxA"  }
    {"type": "server", "timestamp": "2021-10-05T21:19:45,781Z", "level": "ERROR", "component": "o.e.x.m.e.l.LocalExporter", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "failed to set monitoring pipeline [xpack_monitoring_7]", "cluster.uuid": "o4COgb3RTwOsisd4j6DiZg", "node.id": "0cVnt5KFTCu9dyZTyx9xxA" ,
    "stacktrace": ["org.elasticsearch.cluster.metadata.ProcessClusterEventTimeoutException: failed to process cluster event (put-pipeline-xpack_monitoring_7) within 30s",
    "at org.elasticsearch.cluster.service.MasterService$Batcher.lambda$onTimeout$0(MasterService.java:147) [elasticsearch-7.15.0.jar:7.15.0]",
    "at java.util.ArrayList.forEach(ArrayList.java:1511) [?:?]",
    "at org.elasticsearch.cluster.service.MasterService$Batcher.lambda$onTimeout$1(MasterService.java:146) [elasticsearch-7.15.0.jar:7.15.0]",
    "at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:678) [elasticsearch-7.15.0.jar:7.15.0]",
    "at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) [?:?]",
    "at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) [?:?]",
    "at java.lang.Thread.run(Thread.java:831) [?:?]"] }
    {"type": "server", "timestamp": "2021-10-05T21:19:45,913Z", "level": "INFO", "component": "o.e.i.g.GeoIpDownloader", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "updating geoip databases", "cluster.uuid": "o4COgb3RTwOsisd4j6DiZg", "node.id": "0cVnt5KFTCu9dyZTyx9xxA"  }
    {"type": "server", "timestamp": "2021-10-05T21:19:45,915Z", "level": "INFO", "component": "o.e.i.g.GeoIpDownloader", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "fetching geoip databases overview from [https://geoip.elastic.co/v1/database?elastic_geoip_service_tos=agree]", "cluster.uuid": "o4COgb3RTwOsisd4j6DiZg", "node.id": "0cVnt5KFTCu9dyZTyx9xxA"  }
    {"type": "server", "timestamp": "2021-10-05T21:19:46,697Z", "level": "INFO", "component": "o.e.i.g.GeoIpDownloader", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "updating geoip database [GeoLite2-ASN.mmdb]", "cluster.uuid": "o4COgb3RTwOsisd4j6DiZg", "node.id": "0cVnt5KFTCu9dyZTyx9xxA"  }
    {"type": "server", "timestamp": "2021-10-05T21:19:46,877Z", "level": "ERROR", "component": "o.e.x.m.e.l.LocalExporter", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "failed to set monitoring pipeline [xpack_monitoring_6]", "cluster.uuid": "o4COgb3RTwOsisd4j6DiZg", "node.id": "0cVnt5KFTCu9dyZTyx9xxA" ,
    "stacktrace": ["org.elasticsearch.cluster.metadata.ProcessClusterEventTimeoutException: failed to process cluster event (put-pipeline-xpack_monitoring_6) within 30s",
    "at org.elasticsearch.cluster.service.MasterService$Batcher.lambda$onTimeout$0(MasterService.java:147) [elasticsearch-7.15.0.jar:7.15.0]",
    "at java.util.ArrayList.forEach(ArrayList.java:1511) [?:?]",
    "at org.elasticsearch.cluster.service.MasterService$Batcher.lambda$onTimeout$1(MasterService.java:146) [elasticsearch-7.15.0.jar:7.15.0]",
    "at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:678) [elasticsearch-7.15.0.jar:7.15.0]",
    "at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) [?:?]",
    "at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) [?:?]",
    "at java.lang.Thread.run(Thread.java:831) [?:?]"] }
    {"type": "server", "timestamp": "2021-10-05T21:19:47,581Z", "level": "INFO", "component": "o.e.l.LicenseService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "license [ef60de8b-2797-417c-91bb-a76658a86b7b] mode [basic] - valid", "cluster.uuid": "o4COgb3RTwOsisd4j6DiZg", "node.id": "0cVnt5KFTCu9dyZTyx9xxA"  }
    {"type": "server", "timestamp": "2021-10-05T21:19:47,597Z", "level": "INFO", "component": "o.e.x.s.s.SecurityStatusChangeListener", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "Active license is now [BASIC]; Security is disabled", "cluster.uuid": "o4COgb3RTwOsisd4j6DiZg", "node.id": "0cVnt5KFTCu9dyZTyx9xxA"  }
    {"type": "server", "timestamp": "2021-10-05T21:19:47,599Z", "level": "WARN", "component": "o.e.x.s.s.SecurityStatusChangeListener", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "Elasticsearch built-in security features are not enabled. Without authentication, your cluster could be accessible to anyone. See https://www.elastic.co/guide/en/elasticsearch/reference/7.15/security-minimal-setup.html to enable security.", "cluster.uuid": "o4COgb3RTwOsisd4j6DiZg", "node.id": "0cVnt5KFTCu9dyZTyx9xxA"  }
    {"type": "deprecation.elasticsearch", "timestamp": "2021-10-05T21:19:47,603Z", "level": "DEPRECATION", "component": "o.e.d.x.s.s.SecurityStatusChangeListener", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "The default behavior of disabling security on basic licenses is deprecated. In a later version of Elasticsearch, the value of [xpack.security.enabled] will default to \"true\" , regardless of the license level. See https://www.elastic.co/guide/en/elasticsearch/reference/7.15/security-minimal-setup.html to enable security, or explicitly disable security by setting [xpack.security.enabled] to false in elasticsearch.yml", "key": "security_implicitly_disabled", "cluster.uuid": "o4COgb3RTwOsisd4j6DiZg", "node.id": "0cVnt5KFTCu9dyZTyx9xxA"  }
    {"type": "server", "timestamp": "2021-10-05T21:19:47,976Z", "level": "INFO", "component": "o.e.c.m.MetadataCreateIndexService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "[.geoip_databases] creating index, cause [auto(bulk api)], templates [], shards [1]/[0]", "cluster.uuid": "o4COgb3RTwOsisd4j6DiZg", "node.id": "0cVnt5KFTCu9dyZTyx9xxA"  }
    {"type": "server", "timestamp": "2021-10-05T21:19:51,234Z", "level": "INFO", "component": "o.e.c.r.a.AllocationService", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[.geoip_databases][0]]]).", "cluster.uuid": "o4COgb3RTwOsisd4j6DiZg", "node.id": "0cVnt5KFTCu9dyZTyx9xxA"  }
    {"type": "server", "timestamp": "2021-10-05T21:19:56,510Z", "level": "INFO", "component": "o.e.i.g.DatabaseRegistry", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "downloading geoip database [GeoLite2-ASN.mmdb] to [/tmp/elasticsearch-3159410211284329604/geoip-databases/0cVnt5KFTCu9dyZTyx9xxA/GeoLite2-ASN.mmdb.tmp.gz]", "cluster.uuid": "o4COgb3RTwOsisd4j6DiZg", "node.id": "0cVnt5KFTCu9dyZTyx9xxA"  }
    {"type": "server", "timestamp": "2021-10-05T21:19:56,731Z", "level": "INFO", "component": "o.e.i.g.GeoIpDownloader", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "updated geoip database [GeoLite2-ASN.mmdb]", "cluster.uuid": "o4COgb3RTwOsisd4j6DiZg", "node.id": "0cVnt5KFTCu9dyZTyx9xxA"  }
    {"type": "server", "timestamp": "2021-10-05T21:19:56,987Z", "level": "INFO", "component": "o.e.i.g.GeoIpDownloader", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "updating geoip database [GeoLite2-City.mmdb]", "cluster.uuid": "o4COgb3RTwOsisd4j6DiZg", "node.id": "0cVnt5KFTCu9dyZTyx9xxA"  }
    {"type": "server", "timestamp": "2021-10-05T21:19:59,350Z", "level": "INFO", "component": "o.e.i.g.DatabaseRegistry", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "successfully reloaded changed geoip database file [/tmp/elasticsearch-3159410211284329604/geoip-databases/0cVnt5KFTCu9dyZTyx9xxA/GeoLite2-ASN.mmdb]", "cluster.uuid": "o4COgb3RTwOsisd4j6DiZg", "node.id": "0cVnt5KFTCu9dyZTyx9xxA"  }
    {"type": "server", "timestamp": "2021-10-05T21:20:23,599Z", "level": "INFO", "component": "o.e.i.g.DatabaseRegistry", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "downloading geoip database [GeoLite2-City.mmdb] to [/tmp/elasticsearch-3159410211284329604/geoip-databases/0cVnt5KFTCu9dyZTyx9xxA/GeoLite2-City.mmdb.tmp.gz]", "cluster.uuid": "o4COgb3RTwOsisd4j6DiZg", "node.id": "0cVnt5KFTCu9dyZTyx9xxA"  }
    {"type": "server", "timestamp": "2021-10-05T21:20:23,753Z", "level": "INFO", "component": "o.e.i.g.GeoIpDownloader", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "updated geoip database [GeoLite2-City.mmdb]", "cluster.uuid": "o4COgb3RTwOsisd4j6DiZg", "node.id": "0cVnt5KFTCu9dyZTyx9xxA"  }
    {"type": "server", "timestamp": "2021-10-05T21:20:23,763Z", "level": "INFO", "component": "o.e.i.g.GeoIpDownloader", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "updating geoip database [GeoLite2-Country.mmdb]", "cluster.uuid": "o4COgb3RTwOsisd4j6DiZg", "node.id": "0cVnt5KFTCu9dyZTyx9xxA"  }
    {"type": "server", "timestamp": "2021-10-05T21:20:27,965Z", "level": "INFO", "component": "o.e.i.g.DatabaseRegistry", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "successfully reloaded changed geoip database file [/tmp/elasticsearch-3159410211284329604/geoip-databases/0cVnt5KFTCu9dyZTyx9xxA/GeoLite2-City.mmdb]", "cluster.uuid": "o4COgb3RTwOsisd4j6DiZg", "node.id": "0cVnt5KFTCu9dyZTyx9xxA"  }
    {"type": "server", "timestamp": "2021-10-05T21:20:44,016Z", "level": "INFO", "component": "o.e.i.g.DatabaseRegistry", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "downloading geoip database [GeoLite2-Country.mmdb] to [/tmp/elasticsearch-3159410211284329604/geoip-databases/0cVnt5KFTCu9dyZTyx9xxA/GeoLite2-Country.mmdb.tmp.gz]", "cluster.uuid": "o4COgb3RTwOsisd4j6DiZg", "node.id": "0cVnt5KFTCu9dyZTyx9xxA"  }
    {"type": "server", "timestamp": "2021-10-05T21:20:44,064Z", "level": "INFO", "component": "o.e.i.g.GeoIpDownloader", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "updated geoip database [GeoLite2-Country.mmdb]", "cluster.uuid": "o4COgb3RTwOsisd4j6DiZg", "node.id": "0cVnt5KFTCu9dyZTyx9xxA"  }
    {"type": "server", "timestamp": "2021-10-05T21:20:44,359Z", "level": "INFO", "component": "o.e.i.g.DatabaseRegistry", "cluster.name": "docker-cluster", "node.name": "1f8e61b3fb75", "message": "successfully reloaded changed geoip database file [/tmp/elasticsearch-3159410211284329604/geoip-databases/0cVnt5KFTCu9dyZTyx9xxA/GeoLite2-Country.mmdb]", "cluster.uuid": "o4COgb3RTwOsisd4j6DiZg", "node.id": "0cVnt5KFTCu9dyZTyx9xxA"  }
    

    Logs showing archivist-redis is running and working

    [email protected]:~/tubearchivist-build$ docker logs -f archivist-redis
    1:C 05 Oct 2021 21:17:59.423 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
    1:C 05 Oct 2021 21:17:59.424 # Redis version=6.2.5, bits=64, commit=00000000, modified=0, pid=1, just started
    1:C 05 Oct 2021 21:17:59.424 # Configuration loaded
    1:M 05 Oct 2021 21:17:59.429 * monotonic clock: POSIX clock_gettime
    1:M 05 Oct 2021 21:17:59.440 * Running mode=standalone, port=6379.
    1:M 05 Oct 2021 21:17:59.441 # Server initialized
    1:M 05 Oct 2021 21:17:59.441 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
    1:M 05 Oct 2021 21:17:59.441 # <ReJSON> JSON data type for Redis v1.0.8 [encver 0]
    1:M 05 Oct 2021 21:17:59.442 * Module 'ReJSON' loaded from /usr/lib/redis/modules/rejson.so
    1:M 05 Oct 2021 21:17:59.443 * Ready to accept connections
    

    Logs showing that tubearchivist does not start appropriately. Error is copied below.

    [email protected]:~/tubearchivist-build$ docker logs -f tubearchivist
    waiting for elastic search to start
    waiting for elastic search to start
    waiting for elastic search to start
    waiting for elastic search to start
    waiting for elastic search to start
    waiting for elastic search to start
    waiting for elastic search to start
    waiting for elastic search to start
    waiting for elastic search to start
    waiting for elastic search to start
    waiting for elastic search to start
    waiting for elastic search to start
    failed to connect to elastic search, exiting...
    waiting for elastic search to start
    waiting for elastic search to start
    {
      "name" : "1f8e61b3fb75",
      "cluster_name" : "docker-cluster",
      "cluster_uuid" : "o4COgb3RTwOsisd4j6DiZg",
      "version" : {
        "number" : "7.15.0",
        "build_flavor" : "default",
        "build_type" : "docker",
        "build_hash" : "79d65f6e357953a5b3cbcc5e2c7c21073d89aa29",
        "build_date" : "2021-09-16T03:05:29.143308416Z",
        "build_snapshot" : false,
        "lucene_version" : "8.9.0",
        "minimum_wire_compatibility_version" : "6.8.0",
        "minimum_index_compatibility_version" : "6.0.0-beta1"
      },
      "tagline" : "You Know, for Search"
    }
    create new blank index with name ta_channel...
    create new blank index with name ta_video...
    create new blank index with name ta_download...
    sync redis
    Operations to perform:
      Apply all migrations: admin, auth, contenttypes, sessions
    Running migrations:
      Applying contenttypes.0001_initial... OK
      Applying auth.0001_initial... OK
      Applying admin.0001_initial... OK
      Applying admin.0002_logentry_remove_auto_add... OK
      Applying admin.0003_logentry_add_action_flag_choices... OK
      Applying contenttypes.0002_remove_content_type_name... OK
      Applying auth.0002_alter_permission_name_max_length... OK
      Applying auth.0003_alter_user_email_max_length... OK
      Applying auth.0004_alter_user_username_opts... OK
      Applying auth.0005_alter_user_last_login_null... OK
      Applying auth.0006_require_contenttypes_0002... OK
      Applying auth.0007_alter_validators_add_error_messages... OK
      Applying auth.0008_alter_user_username_max_length... OK
      Applying auth.0009_alter_user_last_name_max_length... OK
      Applying auth.0010_alter_group_name_max_length... OK
      Applying auth.0011_update_proxy_permissions... OK
      Applying auth.0012_alter_user_first_name_max_length... OK
      Applying sessions.0001_initial... OK
    ta_channel index is created and up to date...
    ta_video index is created and up to date...
    ta_download index is created and up to date...
    sync redis
    Post-processing 'css/style.css' failed!
    
    Traceback (most recent call last):
      File "/app/manage.py", line 23, in <module>
        main()
      File "/app/manage.py", line 19, in main
        execute_from_command_line(sys.argv)
      File "/usr/local/lib/python3.9/site-packages/django/core/management/__init__.py", line 419, in execute_from_command_line
        utility.execute()
      File "/usr/local/lib/python3.9/site-packages/django/core/management/__init__.py", line 413, in execute
        self.fetch_command(subcommand).run_from_argv(self.argv)
      File "/usr/local/lib/python3.9/site-packages/django/core/management/base.py", line 354, in run_from_argv
        self.execute(*args, **cmd_options)
      File "/usr/local/lib/python3.9/site-packages/django/core/management/base.py", line 398, in execute
        output = self.handle(*args, **options)
      File "/usr/local/lib/python3.9/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 187, in handle
        collected = self.collect()
      File "/usr/local/lib/python3.9/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 134, in collect
        raise processed
    whitenoise.storage.MissingFileError: The file 'font/Sen-Bold.ttf.woff' could not be found with <whitenoise.storage.CompressedManifestStaticFilesStorage object at 0x7f720b5eafd0>.
    
    The CSS file 'css/style.css' references a file which could not be found:
      font/Sen-Bold.ttf.woff
    
    Please check the URL references in this CSS file, particularly any
    relative paths which might be pointing to the wrong location.
    
    [uWSGI] getting INI configuration from uwsgi.ini
    *** Starting uWSGI 2.0.19.1 (64bit) on [Tue Oct  5 21:19:55 2021] ***
    compiled with version: 10.2.1 20210110 on 05 October 2021 21:13:25
    os: Linux-5.4.0-88-generic #99-Ubuntu SMP Thu Sep 23 17:29:00 UTC 2021
    nodename: 283f1e8b559e
    machine: x86_64
    clock source: unix
    detected number of CPU cores: 2
    current working directory: /app
    writing pidfile to /tmp/project-master.pid
    detected binary path: /usr/local/bin/uwsgi
    !!! no internal routing support, rebuild with pcre support !!!
    uWSGI running as root, you can use --uid/--gid/--chroot options
    *** WARNING: you are running uWSGI as root !!! (use the --uid flag) ***
    your memory page size is 4096 bytes
    detected max file descriptor number: 1048576
    lock engine: pthread robust mutexes
    thunder lock: disabled (you can enable it with --thunder-lock)
    uwsgi socket 0 bound to TCP address :8080 fd 3
    uWSGI running as root, you can use --uid/--gid/--chroot options
    *** WARNING: you are running uWSGI as root !!! (use the --uid flag) ***
    Python version: 3.9.7 (default, Sep 28 2021, 18:41:28)  [GCC 10.2.1 20210110]
    *** Python threads support is disabled. You can enable it with --enable-threads ***
    Python main interpreter initialized at 0x55d547c95ff0
    uWSGI running as root, you can use --uid/--gid/--chroot options
    *** WARNING: you are running uWSGI as root !!! (use the --uid flag) ***
    your server socket listen backlog is limited to 100 connections
    your mercy for graceful operations on workers is 60 seconds
    mapped 145808 bytes (142 KB) for 1 cores
    *** Operational MODE: single process ***
    ta_channel index is created and up to date...
    ta_video index is created and up to date...
    ta_download index is created and up to date...
    sync redis
    /usr/local/lib/python3.9/site-packages/celery/platforms.py:834: SecurityWarning: You're running the worker with superuser privileges: this is
    absolutely not recommended!
    
    Please specify a different user using the --uid option.
    
    User information: uid=0 euid=0 gid=0 egid=0
    
      warnings.warn(SecurityWarning(ROOT_DISCOURAGED.format(
    WSGI app 0 (mountpoint='') ready in 10 seconds on interpreter 0x55d547c95ff0 pid: 21 (default app)
    uWSGI running as root, you can use --uid/--gid/--chroot options
    *** WARNING: you are running uWSGI as root !!! (use the --uid flag) ***
    *** uWSGI is running in multiple interpreter mode ***
    spawned uWSGI master process (pid: 21)
    spawned uWSGI worker 1 (pid: 27, cores: 1)
    
     -------------- [email protected] v5.1.2 (sun-harmonics)
    --- ***** -----
    -- ******* ---- Linux-5.4.0-88-generic-x86_64-with-glibc2.31 2021-10-05 16:20:05
    - *** --- * ---
    - ** ---------- [config]
    - ** ---------- .> app:         tasks:0x7f0543aebee0
    - ** ---------- .> transport:   redis://archivist-redis:6379//
    - ** ---------- .> results:     disabled://
    - *** --- * --- .> concurrency: 2 (prefork)
    -- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)
    --- ***** -----
     -------------- [queues]
                    .> celery           exchange=celery(direct) key=celery
    
    
    [tasks]
      . home.tasks.check_reindex
      . home.tasks.download_pending
      . home.tasks.download_single
      . home.tasks.extrac_dl
      . home.tasks.run_backup
      . home.tasks.run_manual_import
      . home.tasks.run_restore_backup
      . home.tasks.update_subscribed
    
    [2021-10-05 16:20:06,189: INFO/MainProcess] Connected to redis://archivist-redis:6379//
    [2021-10-05 16:20:06,217: INFO/MainProcess] mingle: searching for neighbors
    [2021-10-05 16:20:07,296: INFO/MainProcess] mingle: all alone
    [2021-10-05 16:20:07,445: INFO/MainProcess] [email protected] ready.
    [pid: 27|app: 0|req: 1/1] 192.168.0.150 () {42 vars in 753 bytes} [Tue Oct  5 21:22:18 2021] GET / => generated 145 bytes in 547 msecs (HTTP/1.1 500) 6 headers in 184 bytes (1 switches on core 0)
    

    Specific error message:

    Post-processing 'css/style.css' failed!
    
    
    Traceback (most recent call last):
      File "/app/manage.py", line 23, in <module>
        main()
      File "/app/manage.py", line 19, in main
        execute_from_command_line(sys.argv)
      File "/usr/local/lib/python3.9/site-packages/django/core/management/__init__.py", line 419, in execute_from_command_line
        utility.execute()
      File "/usr/local/lib/python3.9/site-packages/django/core/management/__init__.py", line 413, in execute
        self.fetch_command(subcommand).run_from_argv(self.argv)
      File "/usr/local/lib/python3.9/site-packages/django/core/management/base.py", line 354, in run_from_argv
        self.execute(*args, **cmd_options)
      File "/usr/local/lib/python3.9/site-packages/django/core/management/base.py", line 398, in execute
        output = self.handle(*args, **options)
      File "/usr/local/lib/python3.9/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 187, in handle
        collected = self.collect()
      File "/usr/local/lib/python3.9/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 134, in collect
        raise processed
    whitenoise.storage.MissingFileError: The file 'font/Sen-Bold.ttf.woff' could not be found with <whitenoise.storage.CompressedManifestStaticFilesStorage object at 0x7f720b5eafd0>.
    
    The CSS file 'css/style.css' references a file which could not be found:
      font/Sen-Bold.ttf.woff
    
    Please check the URL references in this CSS file, particularly any
    relative paths which might be pointing to the wrong location.
    

    docker exec within the container showing that the font directory is missing from the relative location.

    [email protected]:~/tubearchivist-build$ docker exec -it tubearchivist '/bin/bash'
    [email protected]:/app# ls
    config  db.sqlite3  home  manage.py  requirements.txt  run.sh  static  staticfiles  testing.sh  uwsgi.ini
    [email protected]:/app# which ls
    /bin/ls
    [email protected]:/app# cd static
    [email protected]:/app/static# ls
    css  favicon.ico  img  progress.js  script.js
    [email protected]:/app/static# ls css/
    dark.css  light.css  style.css
    [email protected]:/app/static# cd ../staticfiles/
    [email protected]:/app/staticfiles# ls
    admin  css  favicon.ico  img  progress.js  script.js
    [email protected]:/app/staticfiles# ls css/
    dark.934e9cd71cc4.css  dark.css  light.css  style.css
    

    docker exec within the container showing the .css reference that failed to find the font files.

    [email protected]:/app/static# grep -C5 '.ttf.woff' css/style.css
    @font-face {
    font-family: 'Sen-Bold';
        src:  url('../font/Sen-Bold.ttf.woff');
        font-family: 'Sen-Bold';
    }
    
    @font-face {
    font-family: 'Sen-Regular';
        src:  url('../font/Sen-Regular.ttf.woff');
        font-family: 'Sen-Regular';
    }
    
    * {
        margin: 0;
    

    Commands showing the Workaround

    [email protected]:~$ ls
    docker  Dockerfile  font  tubearchivist  tubearchivist-build
    [email protected]:~$ docker cp font tubearchivist:/app/static/font
    [email protected]:~$ docker cp font tubearchivist:/app/staticfiles/font
    [email protected]:~$ docker exec -it tubearchivist '/bin/bash'
    [email protected]:/app# cd static
    [email protected]:/app/static# ls
    css  favicon.ico  font  img  progress.js  script.js
    [email protected]:/app/static# ls font/
    Sen-Bold.ttf.woff  Sen-Regular.ttf.woff
    [email protected]:/app/static# cd ..
    [email protected]:/app# cd staticfiles/
    [email protected]:/app/staticfiles# ls
    admin  css  favicon.ico  font  img  progress.js  script.js
    [email protected]:/app/staticfiles# ls font/
    Sen-Bold.ttf.woff  Sen-Regular.ttf.woff
    [email protected]:/app/staticfiles#
    

    Showing that tubearchivist provides Expected Response after Workaround

    Note: Log snippet is redacted to exclude previously provided log snippet, since docker container is restarted, not redeployed.
    [email protected]:~$ docker restart tubearchivist
    tubearchivist
    [email protected]:~$ docker logs -f tubearchivist
    [...]
    {
      "name" : "1f8e61b3fb75",
      "cluster_name" : "docker-cluster",
      "cluster_uuid" : "o4COgb3RTwOsisd4j6DiZg",
      "version" : {
        "number" : "7.15.0",
        "build_flavor" : "default",
        "build_type" : "docker",
        "build_hash" : "79d65f6e357953a5b3cbcc5e2c7c21073d89aa29",
        "build_date" : "2021-09-16T03:05:29.143308416Z",
        "build_snapshot" : false,
        "lucene_version" : "8.9.0",
        "minimum_wire_compatibility_version" : "6.8.0",
        "minimum_index_compatibility_version" : "6.0.0-beta1"
      },
      "tagline" : "You Know, for Search"
    }
    ta_channel index is created and up to date...
    ta_video index is created and up to date...
    ta_download index is created and up to date...
    sync redis
    Operations to perform:
      Apply all migrations: admin, auth, contenttypes, sessions
    Running migrations:
      No migrations to apply.
    ta_channel index is created and up to date...
    ta_video index is created and up to date...
    ta_download index is created and up to date...
    sync redis
    Deleting 'script.js'
    Deleting 'favicon.ico'
    Deleting 'progress.js'
    Deleting 'admin/fonts/Roboto-Bold-webfont.50d75e48e0a3.woff'
    Deleting 'admin/fonts/README.txt'
    Deleting 'admin/fonts/LICENSE.txt'
    Deleting 'admin/fonts/Roboto-Bold-webfont.woff'
    Deleting 'admin/fonts/Roboto-Light-webfont.c73eb1ceba33.woff'
    Deleting 'admin/fonts/Roboto-Light-webfont.woff'
    Deleting 'admin/fonts/Roboto-Regular-webfont.woff'
    Deleting 'admin/fonts/Roboto-Regular-webfont.35b07eb2f871.woff'
    Deleting 'admin/fonts/LICENSE.d273d63619c9.txt'
    Deleting 'admin/fonts/README.ab99e6b541ea.txt'
    Deleting 'admin/css/changelists.c70d77c47e69.css'
    Deleting 'admin/css/fonts.168bab448fee.css'
    Deleting 'admin/css/responsive.css'
    Deleting 'admin/css/nav_sidebar.css'
    Deleting 'admin/css/login.c35adf41bb6e.css'
    Deleting 'admin/css/forms.647cb5f1dee9.css'
    Deleting 'admin/css/responsive.b128bdf0edef.css'
    Deleting 'admin/css/autocomplete.4a81fc4242d0.css'
    Deleting 'admin/css/responsive_rtl.css'
    Deleting 'admin/css/forms.css'
    Deleting 'admin/css/changelists.css'
    Deleting 'admin/css/base.1f418065fc2c.css'
    Deleting 'admin/css/rtl.4bc23eb90919.css'
    Deleting 'admin/css/widgets.css'
    Deleting 'admin/css/login.css'
    Deleting 'admin/css/widgets.694d845b2cb1.css'
    Deleting 'admin/css/nav_sidebar.0fd434145f4d.css'
    Deleting 'admin/css/autocomplete.css'
    Deleting 'admin/css/responsive_rtl.e13ae754cceb.css'
    Deleting 'admin/css/rtl.css'
    Deleting 'admin/css/fonts.css'
    Deleting 'admin/css/dashboard.be83f13e4369.css'
    Deleting 'admin/css/dashboard.css'
    Deleting 'admin/css/base.css'
    Deleting 'admin/css/vendor/select2/select2.css'
    Deleting 'admin/css/vendor/select2/LICENSE-SELECT2.f94142512c91.md'
    Deleting 'admin/css/vendor/select2/select2.min.css'
    Deleting 'admin/css/vendor/select2/LICENSE-SELECT2.md'
    Deleting 'admin/css/vendor/select2/select2.a2194c262648.css'
    Deleting 'admin/css/vendor/select2/select2.min.9f54e6414f87.css'
    Deleting 'admin/img/icon-viewlink.41eb31f7826e.svg'
    Deleting 'admin/img/inline-delete.svg'
    Deleting 'admin/img/icon-viewlink.svg'
    Deleting 'admin/img/LICENSE'
    Deleting 'admin/img/README.txt'
    Deleting 'admin/img/icon-unknown.svg'
    Deleting 'admin/img/icon-deletelink.svg'
    Deleting 'admin/img/sorting-icons.3a097b59f104.svg'
    Deleting 'admin/img/icon-unknown.a18cb4398978.svg'
    Deleting 'admin/img/icon-no.439e821418cd.svg'
    Deleting 'admin/img/selector-icons.svg'
    Deleting 'admin/img/icon-alert.svg'
    Deleting 'admin/img/icon-yes.svg'
    Deleting 'admin/img/icon-calendar.svg'
    Deleting 'admin/img/icon-clock.svg'
    Deleting 'admin/img/calendar-icons.39b290681a8b.svg'
    Deleting 'admin/img/selector-icons.b4555096cea2.svg'
    Deleting 'admin/img/icon-unknown-alt.svg'
    Deleting 'admin/img/icon-unknown-alt.81536e128bb6.svg'
    Deleting 'admin/img/icon-addlink.d519b3bab011.svg'
    Deleting 'admin/img/icon-addlink.svg'
    Deleting 'admin/img/tooltag-arrowright.svg'
    Deleting 'admin/img/search.7cf54ff789c6.svg'
    Deleting 'admin/img/README.a70711a38d87.txt'
    Deleting 'admin/img/icon-clock.e1d4dfac3f2b.svg'
    Deleting 'admin/img/tooltag-add.svg'
    Deleting 'admin/img/calendar-icons.svg'
    Deleting 'admin/img/icon-changelink.svg'
    Deleting 'admin/img/LICENSE.2c54f4e1ca1c'
    Deleting 'admin/img/icon-alert.034cc7d8a67f.svg'
    Deleting 'admin/img/icon-yes.d2f9f035226a.svg'
    Deleting 'admin/img/icon-deletelink.564ef9dc3854.svg'
    Deleting 'admin/img/icon-calendar.ac7aea671bea.svg'
    Deleting 'admin/img/tooltag-add.e59d620a9742.svg'
    Deleting 'admin/img/search.svg'
    Deleting 'admin/img/icon-no.svg'
    Deleting 'admin/img/sorting-icons.svg'
    Deleting 'admin/img/inline-delete.fec1b761f254.svg'
    Deleting 'admin/img/icon-changelink.18d2fd706348.svg'
    Deleting 'admin/img/tooltag-arrowright.bbfb788a849e.svg'
    Deleting 'admin/img/gis/move_vertex_on.svg'
    Deleting 'admin/img/gis/move_vertex_on.0047eba25b67.svg'
    Deleting 'admin/img/gis/move_vertex_off.7a23bf31ef8a.svg'
    Deleting 'admin/img/gis/move_vertex_off.svg'
    Deleting 'admin/js/prepopulate.js'
    Deleting 'admin/js/popup_response.js'
    Deleting 'admin/js/cancel.js'
    Deleting 'admin/js/SelectFilter2.js'
    Deleting 'admin/js/autocomplete.js'
    Deleting 'admin/js/collapse.f84e7410290f.js'
    Deleting 'admin/js/nav_sidebar.7605597ddf52.js'
    Deleting 'admin/js/actions.a6d23e8853fd.js'
    Deleting 'admin/js/calendar.js'
    Deleting 'admin/js/popup_response.c6cc78ea5551.js'
    Deleting 'admin/js/urlify.js'
    Deleting 'admin/js/prepopulate_init.js'
    Deleting 'admin/js/cancel.ecc4c5ca7b32.js'
    Deleting 'admin/js/change_form.js'
    Deleting 'admin/js/jquery.init.b7781a0897fc.js'
    Deleting 'admin/js/jquery.init.js'
    Deleting 'admin/js/prepopulate_init.e056047b7a7e.js'
    Deleting 'admin/js/core.ccd84108ec57.js'
    Deleting 'admin/js/inlines.7596b7fd289e.js'
    Deleting 'admin/js/actions.js'
    Deleting 'admin/js/nav_sidebar.js'
    Deleting 'admin/js/urlify.25cc3eac8123.js'
    Deleting 'admin/js/prepopulate.bd2361dfd64d.js'
    Deleting 'admin/js/autocomplete.b6b77d0e5906.js'
    Deleting 'admin/js/SelectBox.8161741c7647.js'
    Deleting 'admin/js/collapse.js'
    Deleting 'admin/js/inlines.js'
    Deleting 'admin/js/SelectFilter2.d250dcb52a9a.js'
    Deleting 'admin/js/change_form.9d8ca4f96b75.js'
    Deleting 'admin/js/SelectBox.js'
    Deleting 'admin/js/core.js'
    Deleting 'admin/js/calendar.f8a5d055eb33.js'
    Deleting 'admin/js/admin/RelatedObjectLookups.b4d76b6aaf0b.js'
    Deleting 'admin/js/admin/RelatedObjectLookups.js'
    Deleting 'admin/js/admin/DateTimeShortcuts.js'
    Deleting 'admin/js/admin/DateTimeShortcuts.5548f99471bf.js'
    Deleting 'admin/js/vendor/select2/select2.full.min.js'
    Deleting 'admin/js/vendor/select2/select2.full.min.fcd7500d8e13.js'
    Deleting 'admin/js/vendor/select2/LICENSE.md'
    Deleting 'admin/js/vendor/select2/LICENSE.f94142512c91.md'
    Deleting 'admin/js/vendor/select2/select2.full.js'
    Deleting 'admin/js/vendor/select2/select2.full.c2afdeda3058.js'
    Deleting 'admin/js/vendor/select2/i18n/af.4f6fcd73488c.js'
    Deleting 'admin/js/vendor/select2/i18n/zh-CN.js'
    Deleting 'admin/js/vendor/select2/i18n/bn.6d42b4dd5665.js'
    Deleting 'admin/js/vendor/select2/i18n/hr.js'
    Deleting 'admin/js/vendor/select2/i18n/ja.170ae885d74f.js'
    Deleting 'admin/js/vendor/select2/i18n/hy.js'
    Deleting 'admin/js/vendor/select2/i18n/pl.6031b4f16452.js'
    Deleting 'admin/js/vendor/select2/i18n/az.js'
    Deleting 'admin/js/vendor/select2/i18n/vi.097a5b75b3e1.js'
    Deleting 'admin/js/vendor/select2/i18n/ro.js'
    Deleting 'admin/js/vendor/select2/i18n/nl.js'
    Deleting 'admin/js/vendor/select2/i18n/nb.da2fce143f27.js'
    Deleting 'admin/js/vendor/select2/i18n/ps.js'
    Deleting 'admin/js/vendor/select2/i18n/cs.js'
    Deleting 'admin/js/vendor/select2/i18n/lt.js'
    Deleting 'admin/js/vendor/select2/i18n/km.js'
    Deleting 'admin/js/vendor/select2/i18n/dsb.56372c92d2f1.js'
    Deleting 'admin/js/vendor/select2/i18n/eu.adfe5c97b72c.js'
    Deleting 'admin/js/vendor/select2/i18n/ar.65aa8e36bf5d.js'
    Deleting 'admin/js/vendor/select2/i18n/fr.js'
    Deleting 'admin/js/vendor/select2/i18n/ro.f75cb460ec3b.js'
    Deleting 'admin/js/vendor/select2/i18n/fa.js'
    Deleting 'admin/js/vendor/select2/i18n/ps.38dfa47af9e0.js'
    Deleting 'admin/js/vendor/select2/i18n/sv.7a9c2f71e777.js'
    Deleting 'admin/js/vendor/select2/i18n/gl.js'
    Deleting 'admin/js/vendor/select2/i18n/sl.131a78bc0752.js'
    Deleting 'admin/js/vendor/select2/i18n/sk.js'
    Deleting 'admin/js/vendor/select2/i18n/ja.js'
    Deleting 'admin/js/vendor/select2/i18n/sq.js'
    Deleting 'admin/js/vendor/select2/i18n/fa.3b5bd1961cfd.js'
    Deleting 'admin/js/vendor/select2/i18n/ko.e7be6c20e673.js'
    Deleting 'admin/js/vendor/select2/i18n/nb.js'
    Deleting 'admin/js/vendor/select2/i18n/cs.4f43e8e7d33a.js'
    Deleting 'admin/js/vendor/select2/i18n/ca.js'
    Deleting 'admin/js/vendor/select2/i18n/zh-CN.2cff662ec5f9.js'
    Deleting 'admin/js/vendor/select2/i18n/tr.b5a0643d1545.js'
    Deleting 'admin/js/vendor/select2/i18n/hsb.js'
    Deleting 'admin/js/vendor/select2/i18n/tk.js'
    Deleting 'admin/js/vendor/select2/i18n/pt-BR.js'
    Deleting 'admin/js/vendor/select2/i18n/es.js'
    Deleting 'admin/js/vendor/select2/i18n/th.f38c20b0221b.js'
    Deleting 'admin/js/vendor/select2/i18n/ka.js'
    Deleting 'admin/js/vendor/select2/i18n/hsb.fa3b55265efe.js'
    Deleting 'admin/js/vendor/select2/i18n/is.js'
    Deleting 'admin/js/vendor/select2/i18n/pt-BR.e1b294433e7f.js'
    Deleting 'admin/js/vendor/select2/i18n/ms.js'
    Deleting 'admin/js/vendor/select2/i18n/hu.6ec6039cb8a3.js'
    Deleting 'admin/js/vendor/select2/i18n/fi.js'
    Deleting 'admin/js/vendor/select2/i18n/et.2b96fd98289d.js'
    Deleting 'admin/js/vendor/select2/i18n/he.js'
    Deleting 'admin/js/vendor/select2/i18n/lt.23c7ce903300.js'
    Deleting 'admin/js/vendor/select2/i18n/vi.js'
    Deleting 'admin/js/vendor/select2/i18n/gl.d99b1fedaa86.js'
    Deleting 'admin/js/vendor/select2/i18n/de.js'
    Deleting 'admin/js/vendor/select2/i18n/ms.4ba82c9a51ce.js'
    Deleting 'admin/js/vendor/select2/i18n/km.c23089cb06ca.js'
    Deleting 'admin/js/vendor/select2/i18n/zh-TW.04554a227c2b.js'
    Deleting 'admin/js/vendor/select2/i18n/pl.js'
    Deleting 'admin/js/vendor/select2/i18n/hi.js'
    Deleting 'admin/js/vendor/select2/i18n/lv.08e62128eac1.js'
    Deleting 'admin/js/vendor/select2/i18n/sv.js'
    Deleting 'admin/js/vendor/select2/i18n/it.be4fe8d365b5.js'
    Deleting 'admin/js/vendor/select2/i18n/tr.js'
    Deleting 'admin/js/vendor/select2/i18n/es.66dbc2652fb1.js'
    Deleting 'admin/js/vendor/select2/i18n/uk.js'
    Deleting 'admin/js/vendor/select2/i18n/sr.js'
    Deleting 'admin/js/vendor/select2/i18n/ru.934aa95f5b5f.js'
    Deleting 'admin/js/vendor/select2/i18n/el.27097f071856.js'
    Deleting 'admin/js/vendor/select2/i18n/bg.39b8be30d4f0.js'
    Deleting 'admin/js/vendor/select2/i18n/ar.js'
    Deleting 'admin/js/vendor/select2/i18n/ru.js'
    Deleting 'admin/js/vendor/select2/i18n/pt.js'
    Deleting 'admin/js/vendor/select2/i18n/bg.js'
    Deleting 'admin/js/vendor/select2/i18n/sr-Cyrl.js'
    Deleting 'admin/js/vendor/select2/i18n/zh-TW.js'
    Deleting 'admin/js/vendor/select2/i18n/en.cf932ba09a98.js'
    Deleting 'admin/js/vendor/select2/i18n/el.js'
    Deleting 'admin/js/vendor/select2/i18n/id.js'
    Deleting 'admin/js/vendor/select2/i18n/sq.5636b60d29c9.js'
    Deleting 'admin/js/vendor/select2/i18n/pt.33b4a3b44d43.js'
    Deleting 'admin/js/vendor/select2/i18n/fr.05e0542fcfe6.js'
    Deleting 'admin/js/vendor/select2/i18n/ne.3d79fd3f08db.js'
    Deleting 'admin/js/vendor/select2/i18n/bn.js'
    Deleting 'admin/js/vendor/select2/i18n/af.js'
    Deleting 'admin/js/vendor/select2/i18n/hi.70640d41628f.js'
    Deleting 'admin/js/vendor/select2/i18n/he.e420ff6cd3ed.js'
    Deleting 'admin/js/vendor/select2/i18n/tk.7c572a68c78f.js'
    Deleting 'admin/js/vendor/select2/i18n/it.js'
    Deleting 'admin/js/vendor/select2/i18n/bs.js'
    Deleting 'admin/js/vendor/select2/i18n/ko.js'
    Deleting 'admin/js/vendor/select2/i18n/uk.8cede7f4803c.js'
    Deleting 'admin/js/vendor/select2/i18n/nl.997868a37ed8.js'
    Deleting 'admin/js/vendor/select2/i18n/eu.js'
    Deleting 'admin/js/vendor/select2/i18n/da.766346afe4dd.js'
    Deleting 'admin/js/vendor/select2/i18n/dsb.js'
    Deleting 'admin/js/vendor/select2/i18n/mk.dabbb9087130.js'
    Deleting 'admin/js/vendor/select2/i18n/ca.a166b745933a.js'
    Deleting 'admin/js/vendor/select2/i18n/sr.5ed85a48f483.js'
    Deleting 'admin/js/vendor/select2/i18n/en.js'
    Deleting 'admin/js/vendor/select2/i18n/lv.js'
    Deleting 'admin/js/vendor/select2/i18n/et.js'
    Deleting 'admin/js/vendor/select2/i18n/da.js'
    Deleting 'admin/js/vendor/select2/i18n/id.04debded514d.js'
    Deleting 'admin/js/vendor/select2/i18n/ka.2083264a54f0.js'
    Deleting 'admin/js/vendor/select2/i18n/hu.js'
    Deleting 'admin/js/vendor/select2/i18n/sk.33d02cef8d11.js'
    Deleting 'admin/js/vendor/select2/i18n/bs.91624382358e.js'
    Deleting 'admin/js/vendor/select2/i18n/sr-Cyrl.f254bb8c4c7c.js'
    Deleting 'admin/js/vendor/select2/i18n/fi.614ec42aa9ba.js'
    Deleting 'admin/js/vendor/select2/i18n/is.3ddd9a6a97e9.js'
    Deleting 'admin/js/vendor/select2/i18n/th.js'
    Deleting 'admin/js/vendor/select2/i18n/sl.js'
    Deleting 'admin/js/vendor/select2/i18n/ne.js'
    Deleting 'admin/js/vendor/select2/i18n/az.270c257daf81.js'
    Deleting 'admin/js/vendor/select2/i18n/hr.a2b092cc1147.js'
    Deleting 'admin/js/vendor/select2/i18n/hy.c7babaeef5a6.js'
    Deleting 'admin/js/vendor/select2/i18n/de.8a1c222b0204.js'
    Deleting 'admin/js/vendor/select2/i18n/mk.js'
    Deleting 'admin/js/vendor/xregexp/LICENSE.bf79e414957a.txt'
    Deleting 'admin/js/vendor/xregexp/LICENSE.txt'
    Deleting 'admin/js/vendor/xregexp/xregexp.min.js'
    Deleting 'admin/js/vendor/xregexp/xregexp.efda034b9537.js'
    Deleting 'admin/js/vendor/xregexp/xregexp.js'
    Deleting 'admin/js/vendor/xregexp/xregexp.min.b0439563a5d3.js'
    Deleting 'admin/js/vendor/jquery/jquery.min.js'
    Deleting 'admin/js/vendor/jquery/LICENSE.txt'
    Deleting 'admin/js/vendor/jquery/jquery.min.dc5e7f18c8d3.js'
    Deleting 'admin/js/vendor/jquery/jquery.23c7c5d2d131.js'
    Deleting 'admin/js/vendor/jquery/LICENSE.75308107741f.txt'
    Deleting 'admin/js/vendor/jquery/jquery.js'
    Deleting 'css/dark.css'
    Deleting 'css/dark.934e9cd71cc4.css'
    Deleting 'css/style.css'
    Deleting 'css/light.css'
    Deleting 'font/Sen-Bold.ttf.woff'
    Deleting 'font/Sen-Regular.ttf.woff'
    Deleting 'img/icon-close.svg'
    Deleting 'img/banner-tube-archivist-dark.png'
    Deleting 'img/banner-tube-archivist-light.png'
    Deleting 'img/icon-seen.svg'
    Deleting 'img/icon-play.svg'
    Deleting 'img/icon-stop.svg'
    Deleting 'img/icon-help.svg'
    Deleting 'img/icon-gear.svg'
    Deleting 'img/icon-gridview.svg'
    Deleting 'img/icon-rescan.svg'
    Deleting 'img/icon-unseen.svg'
    Deleting 'img/icon-download.svg'
    Deleting 'img/icon-add.svg'
    Deleting 'img/icon-search.svg'
    Deleting 'img/icon-listview.svg'
    Deleting 'img/icon-thumb.svg'
    
    152 static files copied to '/app/staticfiles', 476 post-processed.
    [uWSGI] getting INI configuration from uwsgi.ini
    *** Starting uWSGI 2.0.19.1 (64bit) on [Tue Oct  5 21:43:41 2021] ***
    compiled with version: 10.2.1 20210110 on 05 October 2021 21:13:25
    os: Linux-5.4.0-88-generic #99-Ubuntu SMP Thu Sep 23 17:29:00 UTC 2021
    nodename: 283f1e8b559e
    machine: x86_64
    clock source: unix
    detected number of CPU cores: 2
    current working directory: /app
    writing pidfile to /tmp/project-master.pid
    detected binary path: /usr/local/bin/uwsgi
    !!! no internal routing support, rebuild with pcre support !!!
    uWSGI running as root, you can use --uid/--gid/--chroot options
    *** WARNING: you are running uWSGI as root !!! (use the --uid flag) ***
    your memory page size is 4096 bytes
    detected max file descriptor number: 1048576
    lock engine: pthread robust mutexes
    thunder lock: disabled (you can enable it with --thunder-lock)
    uwsgi socket 0 bound to TCP address :8080 fd 3
    uWSGI running as root, you can use --uid/--gid/--chroot options
    *** WARNING: you are running uWSGI as root !!! (use the --uid flag) ***
    Python version: 3.9.7 (default, Sep 28 2021, 18:41:28)  [GCC 10.2.1 20210110]
    *** Python threads support is disabled. You can enable it with --enable-threads ***
    Python main interpreter initialized at 0x56295e4ccff0
    uWSGI running as root, you can use --uid/--gid/--chroot options
    *** WARNING: you are running uWSGI as root !!! (use the --uid flag) ***
    your server socket listen backlog is limited to 100 connections
    your mercy for graceful operations on workers is 60 seconds
    mapped 145808 bytes (142 KB) for 1 cores
    *** Operational MODE: single process ***
    ta_channel index is created and up to date...
    ta_video index is created and up to date...
    ta_download index is created and up to date...
    sync redis
    WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x56295e4ccff0 pid: 15 (default app)
    uWSGI running as root, you can use --uid/--gid/--chroot options
    *** WARNING: you are running uWSGI as root !!! (use the --uid flag) ***
    *** uWSGI is running in multiple interpreter mode ***
    spawned uWSGI master process (pid: 15)
    spawned uWSGI worker 1 (pid: 21, cores: 1)
    /usr/local/lib/python3.9/site-packages/celery/platforms.py:834: SecurityWarning: You're running the worker with superuser privileges: this is
    absolutely not recommended!
    
    Please specify a different user using the --uid option.
    
    User information: uid=0 euid=0 gid=0 egid=0
    
      warnings.warn(SecurityWarning(ROOT_DISCOURAGED.format(
    
     -------------- [email protected] v5.1.2 (sun-harmonics)
    --- ***** -----
    -- ******* ---- Linux-5.4.0-88-generic-x86_64-with-glibc2.31 2021-10-05 16:43:47
    - *** --- * ---
    - ** ---------- [config]
    - ** ---------- .> app:         tasks:0x7f302cdad910
    - ** ---------- .> transport:   redis://archivist-redis:6379//
    - ** ---------- .> results:     disabled://
    - *** --- * --- .> concurrency: 2 (prefork)
    -- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)
    --- ***** -----
     -------------- [queues]
                    .> celery           exchange=celery(direct) key=celery
    
    
    [tasks]
      . home.tasks.check_reindex
      . home.tasks.download_pending
      . home.tasks.download_single
      . home.tasks.extrac_dl
      . home.tasks.run_backup
      . home.tasks.run_manual_import
      . home.tasks.run_restore_backup
      . home.tasks.update_subscribed
    
    [2021-10-05 16:43:48,398: INFO/MainProcess] Connected to redis://archivist-redis:6379//
    [2021-10-05 16:43:48,479: INFO/MainProcess] mingle: searching for neighbors
    [2021-10-05 16:43:49,559: INFO/MainProcess] mingle: all alone
    [2021-10-05 16:43:49,634: INFO/MainProcess] [email protected] ready.
    [pid: 21|app: 0|req: 1/1] 192.168.0.150 () {42 vars in 753 bytes} [Tue Oct  5 21:44:43 2021] GET / => generated 4192 bytes in 274 msecs (HTTP/1.1 200) 7 headers in 349 bytes (1 switches on core 0)
    [pid: 21|app: 0|req: 2/2] 192.168.0.150 () {42 vars in 764 bytes} [Tue Oct  5 21:44:43 2021] GET /static/css/dark.934e9cd71cc4.css => generated 237 bytes in 4 msecs via sendfile() (HTTP/1.1 200) 10 headers in 344 bytes (0 switches on core 0)
    [pid: 21|app: 0|req: 3/3] 192.168.0.150 () {42 vars in 766 bytes} [Tue Oct  5 21:44:43 2021] GET /static/css/style.7ba40e75dac4.css => generated 2663 bytes in 2 msecs via sendfile() (HTTP/1.1 200) 10 headers in 346 bytes (0 switches on core 0)
    [pid: 21|app: 0|req: 4/4] 192.168.0.150 () {42 vars in 743 bytes} [Tue Oct  5 21:44:43 2021] GET /static/script.1de5bb2ef080.js => generated 2413 bytes in 2 msecs via sendfile() (HTTP/1.1 200) 10 headers in 353 bytes (0 switches on core 0)
    [pid: 21|app: 0|req: 5/5] 192.168.0.150 () {42 vars in 854 bytes} [Tue Oct  5 21:44:43 2021] GET /static/img/banner-tube-archivist-dark.edd9d4b3a1e5.png => generated 52255 bytes in 4 msecs via sendfile() (HTTP/1.1 200) 8 headers in 284 bytes (0 switches on core 0)
    [pid: 21|app: 0|req: 6/6] 192.168.0.150 () {42 vars in 820 bytes} [Tue Oct  5 21:44:43 2021] GET /static/img/icon-help.1d655f2f80a1.svg => generated 1756 bytes in 2 msecs via sendfile() (HTTP/1.1 200) 10 headers in 333 bytes (0 switches on core 0)
    [pid: 21|app: 0|req: 7/7] 192.168.0.150 () {42 vars in 820 bytes} [Tue Oct  5 21:44:43 2021] GET /static/img/icon-gear.3b8c10795a9c.svg => generated 1512 bytes in 2 msecs via sendfile() (HTTP/1.1 200) 10 headers in 333 bytes (0 switches on core 0)
    [pid: 21|app: 0|req: 8/8] 192.168.0.150 () {42 vars in 824 bytes} [Tue Oct  5 21:44:43 2021] GET /static/img/icon-search.fd22f6656b26.svg => generated 1320 bytes in 3 msecs via sendfile() (HTTP/1.1 200) 10 headers in 333 bytes (0 switches on core 0)
    [pid: 21|app: 0|req: 9/9] 192.168.0.150 () {42 vars in 828 bytes} [Tue Oct  5 21:44:43 2021] GET /static/img/icon-gridview.0c117ea41097.svg => generated 1212 bytes in 3 msecs via sendfile() (HTTP/1.1 200) 10 headers in 334 bytes (0 switches on core 0)
    [pid: 21|app: 0|req: 10/10] 192.168.0.150 () {42 vars in 828 bytes} [Tue Oct  5 21:44:43 2021] GET /static/img/icon-listview.48ef792820a6.svg => generated 1290 bytes in 2 msecs via sendfile() (HTTP/1.1 200) 10 headers in 333 bytes (0 switches on core 0)
    [pid: 21|app: 0|req: 11/11] 192.168.0.150 () {44 vars in 847 bytes} [Tue Oct  5 21:44:43 2021] GET /static/font/Sen-Regular.ttf.218bb0c177f2.woff => generated 28956 bytes in 2 msecs via sendfile() (HTTP/1.1 200) 8 headers in 296 bytes (0 switches on core 0)
    [pid: 21|app: 0|req: 12/12] 192.168.0.150 () {44 vars in 841 bytes} [Tue Oct  5 21:44:43 2021] GET /static/font/Sen-Bold.ttf.b032f49273fe.woff => generated 24936 bytes in 2 msecs via sendfile() (HTTP/1.1 200) 8 headers in 296 bytes (0 switches on core 0)
    [pid: 21|app: 0|req: 13/13] 192.168.0.150 () {46 vars in 861 bytes} [Tue Oct  5 21:44:43 2021] GET /static/favicon.b21cb294ff64.ico => generated 7661 bytes in 1 msecs via sendfile() (HTTP/1.1 200) 10 headers in 333 bytes (0 switches on core 0)
    [pid: 21|app: 0|req: 14/14] 192.168.0.150 () {44 vars in 868 bytes} [Tue Oct  5 21:45:01 2021] GET /channel/ => generated 4547 bytes in 110 msecs (HTTP/1.1 200) 7 headers in 349 bytes (1 switches on core 0)
    [pid: 21|app: 0|req: 15/15] 192.168.0.150 () {42 vars in 826 bytes} [Tue Oct  5 21:45:01 2021] GET /static/img/icon-add.19182e5c4ca3.svg => generated 1299 bytes in 4 msecs via sendfile() (HTTP/1.1 200) 10 headers in 333 bytes (0 switches on core 0)
    [pid: 21|app: 0|req: 16/16] 192.168.0.150 () {44 vars in 860 bytes} [Tue Oct  5 21:45:02 2021] GET / => generated 4192 bytes in 57 msecs (HTTP/1.1 200) 7 headers in 349 bytes (1 switches on core 0)
    [pid: 21|app: 0|req: 17/17] 192.168.0.150 () {44 vars in 868 bytes} [Tue Oct  5 21:45:06 2021] GET /channel/ => generated 4547 bytes in 62 msecs (HTTP/1.1 200) 7 headers in 349 bytes (1 switches on core 0)
    [pid: 21|app: 0|req: 18/18] 192.168.0.150 () {44 vars in 880 bytes} [Tue Oct  5 21:45:07 2021] GET /downloads/ => generated 4218 bytes in 77 msecs (HTTP/1.1 200) 7 headers in 349 bytes (1 switches on core 0)
    [pid: 21|app: 0|req: 19/19] 192.168.0.150 () {42 vars in 834 bytes} [Tue Oct  5 21:45:07 2021] GET /static/img/icon-rescan.4d903b41a4f8.svg => generated 1556 bytes in 2 msecs via sendfile() (HTTP/1.1 200) 10 headers in 333 bytes (0 switches on core 0)
    [pid: 21|app: 0|req: 20/20] 192.168.0.150 () {42 vars in 838 bytes} [Tue Oct  5 21:45:07 2021] GET /static/img/icon-download.ab2a56dc6336.svg => generated 1226 bytes in 3 msecs via sendfile() (HTTP/1.1 200) 10 headers in 333 bytes (0 switches on core 0)
    [pid: 21|app: 0|req: 21/21] 192.168.0.150 () {42 vars in 757 bytes} [Tue Oct  5 21:45:07 2021] GET /static/progress.8f315ada545f.js => generated 945 bytes in 2 msecs via sendfile() (HTTP/1.1 200) 10 headers in 351 bytes (0 switches on core 0)
    [pid: 21|app: 0|req: 22/22] 192.168.0.150 () {42 vars in 731 bytes} [Tue Oct  5 21:45:08 2021] GET /downloads/progress => generated 17 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 157 bytes (1 switches on core 0)
    [pid: 21|app: 0|req: 23/23] 192.168.0.150 () {44 vars in 878 bytes} [Tue Oct  5 21:45:14 2021] GET /channel/ => generated 4547 bytes in 50 msecs (HTTP/1.1 200) 7 headers in 349 bytes (1 switches on core 0)
    [pid: 21|app: 0|req: 24/24] 192.168.0.150 () {44 vars in 860 bytes} [Tue Oct  5 21:45:17 2021] GET / => generated 4192 bytes in 72 msecs (HTTP/1.1 200) 7 headers in 349 bytes (1 switches on core 0)
    [pid: 21|app: 0|req: 25/25] 192.168.0.150 () {44 vars in 868 bytes} [Tue Oct  5 21:45:18 2021] GET /channel/ => generated 4547 bytes in 45 msecs (HTTP/1.1 200) 7 headers in 349 bytes (1 switches on core 0)
    [pid: 21|app: 0|req: 26/26] 192.168.0.150 () {44 vars in 880 bytes} [Tue Oct  5 21:45:19 2021] GET /downloads/ => generated 4218 bytes in 66 msecs (HTTP/1.1 200) 7 headers in 349 bytes (1 switches on core 0)
    [pid: 21|app: 0|req: 27/27] 192.168.0.150 () {42 vars in 731 bytes} [Tue Oct  5 21:45:20 2021] GET /downloads/progress => generated 17 bytes in 12 msecs (HTTP/1.1 200) 5 headers in 157 bytes (1 switches on core 0)
    [pid: 21|app: 0|req: 28/28] 192.168.0.150 () {44 vars in 878 bytes} [Tue Oct  5 21:45:22 2021] GET /channel/ => generated 4547 bytes in 43 msecs (HTTP/1.1 200) 7 headers in 349 bytes (1 switches on core 0)
    
    opened by lamusmaser 5
  • Installation instructions for existing Redis and Elasticsearch instances

    Installation instructions for existing Redis and Elasticsearch instances

    Hello,

    Can directions be provided for how to integrate with already-existing Redis and ES instances? Thank you.

    opened by suckerfish 4
  • S3 storage backend

    S3 storage backend

    S3 storage backend support would be great.

    opened by 12nick12 1
  • Some more url formats would be nice

    Some more url formats would be nice

    Channel links with the Channel name don't work:

    tubearchivist      | {'csrfmiddlewaretoken': ['AY0tO0Dav2zXgzjttPYb4lEsQ5qgYn5NE4Fzk66r983FaufAOvSgNKSTb3mBAUFQ'], 'subscribe': ['https://www.youtube.com/c/veritasium']}
    tubearchivist      | parsing subscribe ids failed!
    tubearchivist      | ['https://www.youtube.com/c/veritasium']
    

    As a workaround I currently copy the link from the channel name when watching a video. This link has the neded channel id.

    Playlist links in this format: https://www.youtube.com/watch?v=aFPJf-wKTd0&list=UUHnyfMqiRRG1u-2MsSQLbXA&index=2 are parsed like one video but not as list.

    opened by MSDev201 11
  • some requirements

    some requirements

    issues

    • unwatch (uncheck) video not possible
    • "show / hide subscribed channels" is shown twice, on channel page and config

    tubearchivist

    • deleting videos/channels
    • blacklist videos/channels
    • playlists, similar to channel subscriptions

    load only information, and imho unique: extended sorting (length,date(s),channel,titel,views,likes) < favorite i'll work on :)

    project / community / wiki

    • some docker commands could be useful, or a "how to" keep all up to date
    • for more space/storage, how to mount host volumes instead of docker volumes (and how to migrate)
    • enable discussions (if you agree of course) https://docs.github.com/en/discussions/quickstart

    dev / contributors

    how to setup a dev environment for tubearchivist :-)

    opened by chfmunich 1
  • [Security] Hardcoded Django Secret Key

    [Security] Hardcoded Django Secret Key

    There is a hardcoded SECRET_KEY value here, in the settings.py of the project.

    From the Django docs regarding SECRET_KEY:

    Warning

    Keep this value secret.

    Running Django with a known SECRET_KEY defeats many of Django’s security protections, and can lead to privilege escalation and remote code execution vulnerabilities.

    I recommend changing this line to something like:

    SECRET_KEY = str(os.getenv("DJANGO_SECRET_KEY"))
    

    Or better yet:

    if os.getenv("DJANGO_SECRET_KEY_FILE"):
      open('os.getenv("DJANGO_SECRET_KEY_FILE", "r") as secret_file:
        SECRET_KEY = secret_file.read().strip()
    else:
      SECRET_KEY = os.getenv("DJANGO_SECRET_KEY")
    

    Which gives the option to pass the value more securely via Docker or Kubernetes Secrets.

    (There are also Django settings packages which can reduce the boilerplate code here, provide this kind of file/env var support and do type casting for any setting in Django. I'm out of the loop on Django development these days or I'd recommend one, but I have seen that they exist.)

    opened by tpansino 3
  • Multi-arch images?

    Multi-arch images?

    Thanks for working on this, it is a promising project!

    I was trying to spin-up tubearchivist on a RPi 4 running 64-bit Raspbian OS and while the containers get built fine, I run into the standard_init_linux.go:228: exec user process caused: exec format error for the redis JSON and the tubearchivist containers (the Elasticsearch seems to install fine).

    Is there a plan for building multi-arch images especially for arm64?

    opened by abhilesh 17
  • Awesome app. some feedback

    Awesome app. some feedback

    Want to say I love this and Its really going the way I want. I archive LOTS of channels so if you need any stress testing please let me know.

    A couple ideas from others ive used.

    Allow custom naming - I use plex or other plugins that need a specific name Creat .nfo files for plex/kodi etc. Save metadata and a thumbnail in the channel folder

    Alternative channel layout (grid). My archive has 350-400 channels in it so a single column is rough

    But seriously. keep on keeping on. Im pretty excited with this

    opened by Code-Slave 9
Releases(v0.0.5)
  • v0.0.5(Oct 3, 2021)

    Added

    • Added grid and list view switch for all archive pages
    • The Downloads view now has a ignored view list toggle to show all previously ignored videos and provides options to unignore them
    • The Github wiki is now where all the user documentation is located.
    • Tube Archivist can now integrate with a custom RedisJSON port.
    • Added a section in the contributing page about how to set up your testing environment.

    Changed

    • Tube Archivist now utilizes the patched nightly builds of ffmpeg for best compatibility with yt-dlp, #37 #26
    • The About page contains now just useful links as the documentation is now consolidated into the github wiki.
    • Converted some true/false dropdowns to a toggle switch.
    • The “Download Queue” button on the download page is now called “Start Download” for better clarity.

    Fixed

    • Cleaned up startup functions into a dedicated Django ready method to avoid double execution.
    • There is still a lot of refactoring and cleaning up going on.
    Source code(tar.gz)
    Source code(zip)
  • v0.0.4(Sep 26, 2021)

    Added

    • Added a readme section about updating Tube Archivist and expected future changes.
    • Added some donating links, #29
    • Added a docs folder to start working on a Tube Archivist wiki. Asking for help @TechnicallyOffbeat

    Changed

    • Changed how the download queue works: Is now dynamic to allow for gracefully stopping and ungracefully killing the process.
    • Default download limit value is now disabled on new installations, as the dynamic queue offers better ways to stop the download process.
    • “Download now” function allows to set the video as a priority download infront of an already running queue.
    • Additionally the download order is more logical now: New videos get added to the back of the queue, videos start downloading from the top of the queue.

    Fixed

    • Better error handling in add to download form
    • Sanitizing directory scan output from hidden and temporary files, #30
    Source code(tar.gz)
    Source code(zip)
  • v0.0.3(Sep 22, 2021)

    Added

    • Added support to restore index from backup zip file
    • Post-processors support for yt-dlp and optional embedding of metadata into media file, #21 shout out to @nifoc
    • Now showing current version number in the footer for easy reference
    • There is now a CONTRIBUTING.md file
    • Linting and code formatting rules, shout out to @cclauss

    Changed

    • Lots and lots of improvements, refactoring, cleaning in the code base to make things presentable

    Fixed

    • Fixed lots of grammar and spelling issues, shout out to @TechnicallyOffbeat
    Source code(tar.gz)
    Source code(zip)
  • v0.0.2(Sep 17, 2021)

    Added

    • backup metadata db to disk
    • readme section about elastic search permission error

    Changed

    • Download view now has pagination to avoid loading too many thumbnails at once

    Fixed

    • now publishing same image to docker for latest and newest semantic version
    • fixed blocking issue with download now
    • fixed staticfile collection throwing an error on container restart because files are already there
    Source code(tar.gz)
    Source code(zip)
  • v0.0.1(Sep 15, 2021)

    Added

    • Importing and indexing existing video collection into archive.

    Changed

    • Subscribe to channel now takes a list of channels

    Fixed

    • Fixed scraping issue with EU cookie consent screen #2
    • Fixed issue where subscribing to invalid channel ids froze interface
    Source code(tar.gz)
    Source code(zip)
Owner
Simon
Free and open source software enthusiast.
Simon
Your self hosted Youtube media server

The Tube Archivist Your self hosted Youtube media server Core functionality Subscribe to your favourite Youtube channels Download Videos using yt-dlp

Simon 365 Oct 14, 2021
High-performance cross-platform Video Processing Python framework powerpacked with unique trailblazing features :fire:

Releases | Gears | Documentation | Installation | License VidGear is a High-Performance Video Processing Python Library that provides an easy-to-use,

Abhishek Thakur 1.9k Oct 22, 2021
Youtube as covert-channel - Control systems remotely and execute commands by uploading videos to Youtube

covert-tube A program to control systems remotely by uploading videos to Youtube using Python to create the videos and the listener, emulating some ma

Ricardo Ruiz 65 Oct 20, 2021
Python Script for Streaming YouTube Videos in VLC Media Player.

Short Description Use this Simple Script to stream YouTube Video to VLC

Sijey 6 May 27, 2021
A tool to fuck a video/audio quality using FFmpeg

Media quality fucker A tool to fuck a video/audio quality using FFmpeg How to use Download the source Download Python Extract FFmpeg Put what you want

Maizena 4 Sep 24, 2021
A youtube video link or id to video thumbnail python package.

Youtube-Video-Thumbnail A youtube video link or id to video thumbnail python package. Made with Python3

Fayas Noushad 7 Oct 18, 2021
Boltstream Live Video Streaming Website + Backend

Boltstream Self-hosted Live Video Streaming Website + Backend Reference

Ben Wilber 1.7k Oct 18, 2021
Docker container to expose a local RTMP, RTSP, and HLS stream for all your Wyze cameras including v3

Docker container to expose a local RTMP, RTSP, and HLS stream for all your Wyze cameras including v3. No Third-party or special firmware required.

null 458 Oct 19, 2021
MPV remote controller is a program for remote controlling mpv player with device in your local network through web browser.

MPV remote controller is a program for remote controlling mpv player with device in your local network through web browser.

null 3 Jul 30, 2021
KonomiTV: Kind and Optimized Next brOadcast watching systeM Infrastructure for TV

備考・注意事項 現在 α 版で、まだ実験的なプロダクトです。通常利用には耐えないでしょうし、サポートもできません。 安定しているとは到底言いがたい品質ですが、それでも構わない方のみ導入してください。 使い方などの説明も用意できていないため、自力でトラブルに対処できるエンジニアの方以外には現状おすすめ

tsukumi 27 Oct 21, 2021
Text2Video's purpose is to help people create videos quickly and easily by simply typing out the video’s script and a description of images to include in the video.

Text2Video Text2Video's purpose is to help people create videos quickly and easily by simply typing out the video’s script and a description of images

Josh Chen 8 Sep 14, 2021
Home Assistant custom component for viewing IP cameras RTSP stream in real time using WebRTC technology

WebRTC Camera Home Assistant custom component for viewing IP cameras RTSP stream in real time using WebRTC technology. Based on: Pion - pure Go implem

Alex X 350 Oct 22, 2021
This application makes a webrtc video call with jitsi meet signaling

gstreamer-jitsi-meet This application makes a webrtc video call with jitsi meet signaling. Other end can be any jitsi meet app or web app. It doesn't

Linh 4 Aug 19, 2021
A Simple Telegram Bot By @Tellybots to add Subtitle Files in Video

Video-subtitle-merger A Simple Telegram Bot By @Tellybots to add Subtitle Files in Video Features Force Sub Button Added Soon Support Media Type Such

null 4 Oct 23, 2021
This is an example of building a video Question-Answer system using Jina.

example-video-search This is an example of building a video Question-Answer system using Jina. The index data is subtitle files of YouTube videos. Aft

Jina AI 2 Oct 18, 2021
iYTDL - Asynchronous Standalone Inline YouTube-DL Module

iYTDL Asynchronous Standalone Inline YouTube-DL Module ⬇️ Installing Install pip3 install iytdl Upgrade pip3 install -U iytdl ⭐️ Features Fully Asynch

iYTDL 38 Oct 18, 2021
A Python media index

pyvideo https://pyvideo.org is simply an index of Python-related media records. The raw data being used here comes out of the pyvideo/data repo. Befor

pyvideo 227 Oct 5, 2021
plumi video sharing

December 2017 update We are moving tickets from the Plumi tracker (trac.plumi.org) here, for historical reasons. Plumi video sharing system Plumi is a

Plumi 108 Sep 25, 2021
Stream music with ffmpeg and python

youtube-stream Stream music with ffmpeg and python original Usage set the KEY in stream.sh run server.py run stream.sh (You can use Git bash or WSL in

Giyoung Ryu 12 Oct 7, 2021