bump TA_VERSION

Changed:
- Added additional sort by fields
- [API] Chaned primary stats endpoints
- [API] Added separate video stats endpoints
- Added fallback for some manual import values
- Fix comment extration for members video
- Fix reindex outdated query

.dockerignore

@@ -17,8 +17,5 @@ venv/
 # Unneeded graphics
 assets/*
 
-# Unneeded docs
-docs/*
-
 # for local testing only
 testing.sh

.github/ISSUE_TEMPLATE/INSTALLATION-HELP.yml

@@ -38,6 +38,6 @@ body:
     attributes:
       label: Relevant log output
       description: Please copy and paste any relevant Docker logs. This will be automatically formatted into code, so no need for backticks.
-      render: shell
+      render: Shell
     validations:
       required: true

Dockerfile

@@ -7,7 +7,7 @@ FROM python:3.11.3-slim-bullseye AS builder
 ARG TARGETPLATFORM
 
 RUN apt-get update && apt-get install -y --no-install-recommends \
-    build-essential gcc libldap2-dev libsasl2-dev libssl-dev
+    build-essential gcc libldap2-dev libsasl2-dev libssl-dev git
 
 # install requirements
 COPY ./tubearchivist/requirements.txt /requirements.txt

README.md

@@ -58,7 +58,12 @@ Take a look at the example [docker-compose.yml](https://github.com/tubearchivist
 | TZ | Set your timezone for the scheduler | Required |
 | TA_PORT | Overwrite Nginx port | Optional |
 | TA_UWSGI_PORT | Overwrite container internal uwsgi port | Optional |
+| TA_ENABLE_AUTH_PROXY | Enables support for forwarding auth in reverse proxies | [Read more](https://docs.tubearchivist.com/configuration/forward-auth/) |
+| TA_AUTH_PROXY_USERNAME_HEADER | Header containing username to log in | Optional |
+| TA_AUTH_PROXY_LOGOUT_URL | Logout URL for forwarded auth | Optional |
 | ES_URL | URL That ElasticSearch runs on | Optional |
+| ES_DISABLE_VERIFY_SSL | Disable ElasticSearch SSL certificate verification | Optional |
+| ES_SNAPSHOT_DIR | Custom path where elastic search stores snapshots for master/data nodes | Optional |
 | HOST_GID | Allow TA to own the video files instead of container user | Optional |
 | HOST_UID | Allow TA to own the video files instead of container user | Optional |
 | ELASTIC_USER | Change the default ElasticSearch user | Optional |
@@ -77,7 +82,7 @@ Take a look at the example [docker-compose.yml](https://github.com/tubearchivist
 ## Update
 Always use the *latest* (the default) or a named semantic version tag for the docker images. The *unstable* tags are only for your testing environment, there might not be an update path for these testing builds. 
 
-You will see the current version number of **Tube Archivist** in the footer of the interface. There is a daily version check task querying tubearchivist.com, notifying you of any new releases in the footer. To take advantage of the latest fixes and improvements, make sure you are running the *latest and greatest*. After updating, check the footer to verify you are running the expected version.  
+You will see the current version number of **Tube Archivist** in the footer of the interface. There is a daily version check task querying tubearchivist.com, notifying you of any new releases in the footer. To update, you need to update the docker images, the method for which will depend on your platform. For example, if you're using `docker-compose`, run `docker-compose pull` and then restart with `docker-compose up -d`. After updating, check the footer to verify you are running the expected version.  
 
   - This project is tested for updates between one or two releases maximum. Further updates back may or may not be supported and you might have to reset your index and configurations to update. Ideally apply new updates at least once per month.  
   - There can be breaking changes between updates, particularly as the application grows, new environment variables or settings might be required for you to set in the your docker-compose file. *Always* check the **release notes**: Any breaking changes will be marked there.  
@@ -130,6 +135,11 @@ The Elasticsearch index will turn to ***read only*** if the disk usage of the co
 
 Similar to that, TubeArchivist will become all sorts of messed up when running out of disk space. There are some error messages in the logs when that happens, but it's best to make sure to have enough disk space before starting to download.
 
+## `error setting rlimit`
+If you are seeing errors like `failed to create shim: OCI runtime create failed` and `error during container init: error setting rlimits`, this means docker can't set these limits, usually because they are set at another place or are incompatible because of other reasons. Solution is to remove the `ulimits` key from the ES container in your docker compose and start again.
+
+This can happen if you have nested virtualizations, e.g. LXC running Docker in Proxmox.
+
 ## Known limitations
 - Video files created by Tube Archivist need to be playable in your browser of choice. Not every codec is compatible with every browser and might require some testing with format selection. 
 - Every limitation of **yt-dlp** will also be present in Tube Archivist. If **yt-dlp** can't download or extract a video for any reason, Tube Archivist won't be able to either.
@@ -139,18 +149,20 @@ Similar to that, TubeArchivist will become all sorts of messed up when running o
 We have come far, nonetheless we are not short of ideas on how to improve and extend this project. Issues waiting for you to be tackled in no particular order:
 
 - [ ] User roles
+- [ ] Audio download
 - [ ] Podcast mode to serve channel as mp3
-- [ ] Implement [PyFilesystem](https://github.com/PyFilesystem/pyfilesystem2) for flexible video storage
 - [ ] User created playlists, random and repeat controls ([#108](https://github.com/tubearchivist/tubearchivist/issues/108), [#220](https://github.com/tubearchivist/tubearchivist/issues/220))
 - [ ] Auto play or play next link ([#226](https://github.com/tubearchivist/tubearchivist/issues/226))
 - [ ] Multi language support
 - [ ] Show total video downloaded vs total videos available in channel
-- [ ] Add statistics of index
-- [ ] Download speed schedule ([#198](https://github.com/tubearchivist/tubearchivist/issues/198))
 - [ ] Download or Ignore videos by keyword ([#163](https://github.com/tubearchivist/tubearchivist/issues/163))
 - [ ] Custom searchable notes to videos, channels, playlists ([#144](https://github.com/tubearchivist/tubearchivist/issues/144))
+- [ ] Search comments
+- [ ] Search download queue
+- [ ] Configure shorts, streams and video sizes per channel
 
 Implemented:
+- [X] Add statistics of index [2023-09-03]
 - [X] Implement [Apprise](https://github.com/caronc/apprise) for notifications [2023-08-05]
 - [X] Download video comments [2022-11-30]
 - [X] Show similar videos on video page [2022-11-30]

docker-compose.yml

@@ -1,4 +1,4 @@
-version: '3.3'
+version: '3.5'
 
 services:
   tubearchivist:
@@ -20,6 +20,12 @@ services:
       - TA_PASSWORD=verysecret              # your initial TA credentials
       - ELASTIC_PASSWORD=verysecret         # set password for Elasticsearch
       - TZ=America/New_York                 # set your time zone
+    healthcheck:
+      test: ["CMD", "curl", "-f", "http://localhost:8000/health"]
+      interval: 2m
+      timeout: 10s
+      retries: 3
+      start_period: 30s
     depends_on:
       - archivist-es
       - archivist-redis
@@ -34,7 +40,7 @@ services:
     depends_on:
       - archivist-es
   archivist-es:
-    image: bbilly1/tubearchivist-es         # only for amd64, or use official es 8.9.0
+    image: bbilly1/tubearchivist-es         # only for amd64, or use official es 8.11.0
     container_name: archivist-es
     restart: unless-stopped
     environment:

docs/Channels.md

@@ -1,3 +0,0 @@
-All user documentation has moved to a more flexible, easier to extend and modify documentation platform accessible [here](https://docs.tubearchivist.com) and built from [here](https://github.com/tubearchivist/docs). Don't make any more changes here, keeping this around for some time to keep old links alive.
-
-Equivalent Channels page is: [here](https://docs.tubearchivist.com/channels/).

docs/Downloads.md

@@ -1,3 +0,0 @@
-All user documentation has moved to a more flexible, easier to extend and modify documentation platform accessible [here](https://docs.tubearchivist.com) and built from [here](https://github.com/tubearchivist/docs). Don't make any more changes here, keeping this around for some time to keep old links alive.
-
-Equivalent Downloads page is: [here](https://docs.tubearchivist.com/downloads/).

docs/FAQ.md

@@ -1,3 +0,0 @@
-All user documentation has moved to a more flexible, easier to extend and modify documentation platform accessible [here](https://docs.tubearchivist.com) and built from [here](https://github.com/tubearchivist/docs). Don't make any more changes here, keeping this around for some time to keep old links alive.
-
-Equivalent FAQ page is: [here](https://docs.tubearchivist.com/faq/).

docs/Home.md

@@ -1 +0,0 @@
-All user documentation has moved to a more flexible, easier to extend and modify documentation platform accessible [here](https://docs.tubearchivist.com) and built from [here](https://github.com/tubearchivist/docs). Don't make any more changes here, keeping this around for some time to keep old links alive.

docs/Installation.md

@@ -1,3 +0,0 @@
-All user documentation has moved to a more flexible, easier to extend and modify documentation platform accessible [here](https://docs.tubearchivist.com) and built from [here](https://github.com/tubearchivist/docs). Don't make any more changes here, keeping this around for some time to keep old links alive.
-
-Equivalent pages are located under *installation* on the left.

docs/Playlists.md

@@ -1,3 +0,0 @@
-All user documentation has moved to a more flexible, easier to extend and modify documentation platform accessible [here](https://docs.tubearchivist.com) and built from [here](https://github.com/tubearchivist/docs). Don't make any more changes here, keeping this around for some time to keep old links alive.
-
-Equivalent Playlist page is: [here](https://docs.tubearchivist.com/playlists/).

docs/Search.md

@@ -1,3 +0,0 @@
-All user documentation has moved to a more flexible, easier to extend and modify documentation platform accessible [here](https://docs.tubearchivist.com) and built from [here](https://github.com/tubearchivist/docs). Don't make any more changes here, keeping this around for some time to keep old links alive.
-
-Equivalent Search page is: [here](https://docs.tubearchivist.com/search/).

docs/Settings.md

@@ -1,3 +0,0 @@
-All user documentation has moved to a more flexible, easier to extend and modify documentation platform accessible [here](https://docs.tubearchivist.com) and built from [here](https://github.com/tubearchivist/docs). Don't make any more changes here, keeping this around for some time to keep old links alive.
-
-Equivalent Settings page is: [here](https://docs.tubearchivist.com/settings/).

docs/Users.md

@@ -1,3 +0,0 @@
-All user documentation has moved to a more flexible, easier to extend and modify documentation platform accessible [here](https://docs.tubearchivist.com) and built from [here](https://github.com/tubearchivist/docs). Don't make any more changes here, keeping this around for some time to keep old links alive.
-
-Equivalent Users page is: [here](https://docs.tubearchivist.com/users/).

docs/Video.md

@@ -1,3 +0,0 @@
-All user documentation has moved to a more flexible, easier to extend and modify documentation platform accessible [here](https://docs.tubearchivist.com) and built from [here](https://github.com/tubearchivist/docs). Don't make any more changes here, keeping this around for some time to keep old links alive.
-
-Equivalent Video page is: [here](https://docs.tubearchivist.com/video/).

tubearchivist/api/src/aggs.py

@@ -1,7 +1,8 @@
 """aggregations"""
 
 from home.src.es.connect import ElasticWrap
-from home.src.index.video_streams import DurationConverter
+from home.src.ta.helper import get_duration_str
+from home.src.ta.settings import EnvironmentSettings
 
 
 class AggBase:
@@ -23,62 +24,159 @@ class AggBase:
         raise NotImplementedError
 
 
-class Primary(AggBase):
+class Video(AggBase):
-    """primary aggregation for total documents indexed"""
+    """get video stats"""
 
-    name = "primary"
+    name = "video_stats"
-    path = "ta_video,ta_channel,ta_playlist,ta_subtitle,ta_download/_search"
+    path = "ta_video/_search"
     data = {
         "size": 0,
         "aggs": {
             "video_type": {
-                "filter": {"exists": {"field": "active"}},
+                "terms": {"field": "vid_type"},
-                "aggs": {"filtered": {"terms": {"field": "vid_type"}}},
+                "aggs": {
+                    "media_size": {"sum": {"field": "media_size"}},
+                    "duration": {"sum": {"field": "player.duration"}},
                 },
-            "channel_total": {"value_count": {"field": "channel_active"}},
+            },
-            "channel_sub": {"terms": {"field": "channel_subscribed"}},
+            "video_active": {
-            "playlist_total": {"value_count": {"field": "playlist_active"}},
+                "terms": {"field": "active"},
-            "playlist_sub": {"terms": {"field": "playlist_subscribed"}},
+                "aggs": {
-            "download": {"terms": {"field": "status"}},
+                    "media_size": {"sum": {"field": "media_size"}},
+                    "duration": {"sum": {"field": "player.duration"}},
+                },
+            },
+            "video_media_size": {"sum": {"field": "media_size"}},
+            "video_count": {"value_count": {"field": "youtube_id"}},
+            "duration": {"sum": {"field": "player.duration"}},
         },
     }
 
     def process(self):
-        """make the call"""
+        """process aggregation"""
         aggregations = self.get()
 
-        videos = {"total": aggregations["video_type"].get("doc_count")}
+        duration = int(aggregations["duration"]["value"])
-        videos.update(
+        response = {
+            "doc_count": aggregations["video_count"]["value"],
+            "media_size": int(aggregations["video_media_size"]["value"]),
+            "duration": duration,
+            "duration_str": get_duration_str(duration),
+        }
+        for bucket in aggregations["video_type"]["buckets"]:
+            duration = int(bucket["duration"].get("value"))
+            response.update(
                 {
-                i.get("key"): i.get("doc_count")
+                    f"type_{bucket['key']}": {
-                for i in aggregations["video_type"]["filtered"]["buckets"]
+                        "doc_count": bucket.get("doc_count"),
+                        "media_size": int(bucket["media_size"].get("value")),
+                        "duration": duration,
+                        "duration_str": get_duration_str(duration),
+                    }
                 }
             )
-        channels = {"total": aggregations["channel_total"].get("value")}
+
-        channels.update(
+        for bucket in aggregations["video_active"]["buckets"]:
+            duration = int(bucket["duration"].get("value"))
+            response.update(
                 {
-                "sub_" + i.get("key_as_string"): i.get("doc_count")
+                    f"active_{bucket['key_as_string']}": {
-                for i in aggregations["channel_sub"]["buckets"]
+                        "doc_count": bucket.get("doc_count"),
+                        "media_size": int(bucket["media_size"].get("value")),
+                        "duration": duration,
+                        "duration_str": get_duration_str(duration),
+                    }
                 }
             )
-        playlists = {"total": aggregations["playlist_total"].get("value")}
+
-        playlists.update(
+        return response
-            {
+
-                "sub_" + i.get("key_as_string"): i.get("doc_count")
+
-                for i in aggregations["playlist_sub"]["buckets"]
+class Channel(AggBase):
-            }
+    """get channel stats"""
-        )
+
-        downloads = {
+    name = "channel_stats"
-            i.get("key"): i.get("doc_count")
+    path = "ta_channel/_search"
-            for i in aggregations["download"]["buckets"]
+    data = {
+        "size": 0,
+        "aggs": {
+            "channel_count": {"value_count": {"field": "channel_id"}},
+            "channel_active": {"terms": {"field": "channel_active"}},
+            "channel_subscribed": {"terms": {"field": "channel_subscribed"}},
+        },
     }
 
+    def process(self):
+        """process aggregation"""
+        aggregations = self.get()
+
         response = {
-            "videos": videos,
+            "doc_count": aggregations["channel_count"].get("value"),
-            "channels": channels,
-            "playlists": playlists,
-            "downloads": downloads,
         }
+        for bucket in aggregations["channel_active"]["buckets"]:
+            key = f"active_{bucket['key_as_string']}"
+            response.update({key: bucket.get("doc_count")})
+        for bucket in aggregations["channel_subscribed"]["buckets"]:
+            key = f"subscribed_{bucket['key_as_string']}"
+            response.update({key: bucket.get("doc_count")})
+
+        return response
+
+
+class Playlist(AggBase):
+    """get playlist stats"""
+
+    name = "playlist_stats"
+    path = "ta_playlist/_search"
+    data = {
+        "size": 0,
+        "aggs": {
+            "playlist_count": {"value_count": {"field": "playlist_id"}},
+            "playlist_active": {"terms": {"field": "playlist_active"}},
+            "playlist_subscribed": {"terms": {"field": "playlist_subscribed"}},
+        },
+    }
+
+    def process(self):
+        """process aggregation"""
+        aggregations = self.get()
+        response = {"doc_count": aggregations["playlist_count"].get("value")}
+        for bucket in aggregations["playlist_active"]["buckets"]:
+            key = f"active_{bucket['key_as_string']}"
+            response.update({key: bucket.get("doc_count")})
+        for bucket in aggregations["playlist_subscribed"]["buckets"]:
+            key = f"subscribed_{bucket['key_as_string']}"
+            response.update({key: bucket.get("doc_count")})
+
+        return response
+
+
+class Download(AggBase):
+    """get downloads queue stats"""
+
+    name = "download_queue_stats"
+    path = "ta_download/_search"
+    data = {
+        "size": 0,
+        "aggs": {
+            "status": {"terms": {"field": "status"}},
+            "video_type": {
+                "filter": {"term": {"status": "pending"}},
+                "aggs": {"type_pending": {"terms": {"field": "vid_type"}}},
+            },
+        },
+    }
+
+    def process(self):
+        """process aggregation"""
+        aggregations = self.get()
+        response = {}
+        for bucket in aggregations["status"]["buckets"]:
+            response.update({bucket["key"]: bucket.get("doc_count")})
+
+        for bucket in aggregations["video_type"]["type_pending"]["buckets"]:
+            key = f"pending_{bucket['key']}"
+            response.update({key: bucket.get("doc_count")})
 
         return response
 
@@ -117,9 +215,9 @@ class WatchProgress(AggBase):
         all_duration = int(aggregations["total_duration"].get("value"))
         response.update(
             {
-                "all": {
+                "total": {
                     "duration": all_duration,
-                    "duration_str": DurationConverter().get_str(all_duration),
+                    "duration_str": get_duration_str(all_duration),
                     "items": aggregations["total_vids"].get("value"),
                 }
             }
@@ -135,7 +233,7 @@ class WatchProgress(AggBase):
         """parse bucket"""
 
         duration = int(bucket["watch_docs"]["duration"]["value"])
-        duration_str = DurationConverter().get_str(duration)
+        duration_str = get_duration_str(duration)
         items = bucket["watch_docs"]["true_count"]["value"]
         if bucket["key_as_string"] == "false":
             key = "unwatched"
@@ -168,13 +266,22 @@ class DownloadHist(AggBase):
                     "calendar_interval": "day",
                     "format": "yyyy-MM-dd",
                     "order": {"_key": "desc"},
+                    "time_zone": EnvironmentSettings.TZ,
                 },
                 "aggs": {
-                    "total_videos": {"value_count": {"field": "youtube_id"}}
+                    "total_videos": {"value_count": {"field": "youtube_id"}},
+                    "media_size": {"sum": {"field": "media_size"}},
                 },
             }
         },
-        "query": {"range": {"date_downloaded": {"gte": "now-7d/d"}}},
+        "query": {
+            "range": {
+                "date_downloaded": {
+                    "gte": "now-7d/d",
+                    "time_zone": EnvironmentSettings.TZ,
+                }
+            }
+        },
     }
 
     def process(self):
@@ -186,6 +293,7 @@ class DownloadHist(AggBase):
             {
                 "date": i.get("key_as_string"),
                 "count": i.get("doc_count"),
+                "media_size": i["media_size"].get("value"),
             }
             for i in buckets
         ]
@@ -196,6 +304,9 @@ class DownloadHist(AggBase):
 class BiggestChannel(AggBase):
     """get channel aggregations"""
 
+    def __init__(self, order):
+        self.data["aggs"][self.name]["multi_terms"]["order"] = {order: "desc"}
+
     name = "channel_stats"
     path = "ta_video/_search"
     data = {
@@ -231,9 +342,7 @@ class BiggestChannel(AggBase):
                 "name": i["key"][0].title(),
                 "doc_count": i["doc_count"]["value"],
                 "duration": i["duration"]["value"],
-                "duration_str": DurationConverter().get_str(
+                "duration_str": get_duration_str(int(i["duration"]["value"])),
-                    i["duration"]["value"]
-                ),
                 "media_size": i["media_size"]["value"],
             }
             for i in buckets

tubearchivist/api/src/search_processor.py

@@ -7,15 +7,14 @@ Functionality:
 import urllib.parse
 
 from home.src.download.thumbnails import ThumbManager
-from home.src.ta.config import AppConfig
+from home.src.ta.helper import date_praser, get_duration_str
-from home.src.ta.helper import date_praser
+from home.src.ta.settings import EnvironmentSettings
 
 
 class SearchProcess:
     """process search results"""
 
-    CONFIG = AppConfig().config
+    CACHE_DIR = EnvironmentSettings.CACHE_DIR
-    CACHE_DIR = CONFIG["application"]["cache_dir"]
 
     def __init__(self, response):
         self.response = response
@@ -50,6 +49,16 @@ class SearchProcess:
             processed = self._process_download(result["_source"])
         if index == "ta_comment":
             processed = self._process_comment(result["_source"])
+        if index == "ta_subtitle":
+            processed = self._process_subtitle(result)
+
+        if isinstance(processed, dict):
+            processed.update(
+                {
+                    "_index": index,
+                    "_score": round(result.get("_score") or 0, 2),
+                }
+            )
 
         return processed
 
@@ -139,3 +148,29 @@ class SearchProcess:
                 processed_comments[-1]["comment_replies"].append(comment)
 
         return processed_comments
+
+    def _process_subtitle(self, result):
+        """take complete result dict to extract highlight"""
+        subtitle_dict = result["_source"]
+        highlight = result.get("highlight")
+        if highlight:
+            # replace lines with the highlighted markdown
+            subtitle_line = highlight.get("subtitle_line")[0]
+            subtitle_dict.update({"subtitle_line": subtitle_line})
+
+        thumb_path = ThumbManager(subtitle_dict["youtube_id"]).vid_thumb_path()
+        subtitle_dict.update({"vid_thumb_url": f"/cache/{thumb_path}"})
+
+        return subtitle_dict
+
+
+def process_aggs(response):
+    """convert aggs duration to str"""
+
+    if response.get("aggregations"):
+        aggs = response["aggregations"]
+        if "total_duration" in aggs:
+            duration_sec = int(aggs["total_duration"]["value"])
+            aggs["total_duration"].update(
+                {"value_str": get_duration_str(duration_sec)}
+            )

tubearchivist/api/urls.py

@@ -96,6 +96,16 @@ urlpatterns = [
         views.SnapshotApiView.as_view(),
         name="api-snapshot",
     ),
+    path(
+        "backup/",
+        views.BackupApiListView.as_view(),
+        name="api-backup-list",
+    ),
+    path(
+        "backup/<str:filename>/",
+        views.BackupApiView.as_view(),
+        name="api-backup",
+    ),
     path(
         "task-name/",
         views.TaskListView.as_view(),
@@ -111,6 +121,11 @@ urlpatterns = [
         views.TaskIDView.as_view(),
         name="api-task-id",
     ),
+    path(
+        "config/user/",
+        views.UserConfigView.as_view(),
+        name="api-config-user",
+    ),
     path(
         "cookie/",
         views.CookieView.as_view(),
@@ -137,9 +152,24 @@ urlpatterns = [
         name="api-notification",
     ),
     path(
-        "stats/primary/",
+        "stats/video/",
-        views.StatPrimaryView.as_view(),
+        views.StatVideoView.as_view(),
-        name="api-stats-primary",
+        name="api-stats-video",
+    ),
+    path(
+        "stats/channel/",
+        views.StatChannelView.as_view(),
+        name="api-stats-channel",
+    ),
+    path(
+        "stats/playlist/",
+        views.StatPlaylistView.as_view(),
+        name="api-stats-playlist",
+    ),
+    path(
+        "stats/download/",
+        views.StatDownloadView.as_view(),
+        name="api-stats-download",
     ),
     path(
         "stats/watch/",

tubearchivist/api/views.py

@@ -1,6 +1,14 @@
 """all API views"""
 
-from api.src.aggs import BiggestChannel, DownloadHist, Primary, WatchProgress
+from api.src.aggs import (
+    BiggestChannel,
+    Channel,
+    Download,
+    DownloadHist,
+    Playlist,
+    Video,
+    WatchProgress,
+)
 from api.src.search_processor import SearchProcess
 from home.src.download.queue import PendingInteract
 from home.src.download.subscriptions import (
@@ -8,6 +16,7 @@ from home.src.download.subscriptions import (
     PlaylistSubscription,
 )
 from home.src.download.yt_dlp_base import CookieHandler
+from home.src.es.backup import ElasticBackup
 from home.src.es.connect import ElasticWrap
 from home.src.es.snapshot import ElasticSnapshot
 from home.src.frontend.searching import SearchForm
@@ -18,38 +27,69 @@ from home.src.index.playlist import YoutubePlaylist
 from home.src.index.reindex import ReindexProgress
 from home.src.index.video import SponsorBlock, YoutubeVideo
 from home.src.ta.config import AppConfig, ReleaseVersion
+from home.src.ta.settings import EnvironmentSettings
 from home.src.ta.ta_redis import RedisArchivist
 from home.src.ta.task_manager import TaskCommand, TaskManager
 from home.src.ta.urlparser import Parser
+from home.src.ta.users import UserConfig
 from home.tasks import (
     BaseTask,
     check_reindex,
     download_pending,
     extrac_dl,
+    run_restore_backup,
     subscribe_to,
 )
+from rest_framework import permissions
 from rest_framework.authentication import (
     SessionAuthentication,
     TokenAuthentication,
 )
 from rest_framework.authtoken.models import Token
 from rest_framework.authtoken.views import ObtainAuthToken
-from rest_framework.permissions import IsAuthenticated
 from rest_framework.response import Response
 from rest_framework.views import APIView
 
 
+def check_admin(user):
+    """check for admin permission for restricted views"""
+    return user.is_staff or user.groups.filter(name="admin").exists()
+
+
+class AdminOnly(permissions.BasePermission):
+    """allow only admin"""
+
+    def has_permission(self, request, view):
+        return check_admin(request.user)
+
+
+class AdminWriteOnly(permissions.BasePermission):
+    """allow only admin writes"""
+
+    def has_permission(self, request, view):
+        if request.method in permissions.SAFE_METHODS:
+            return permissions.IsAuthenticated().has_permission(request, view)
+
+        return check_admin(request.user)
+
+
 class ApiBaseView(APIView):
     """base view to inherit from"""
 
     authentication_classes = [SessionAuthentication, TokenAuthentication]
-    permission_classes = [IsAuthenticated]
+    permission_classes = [permissions.IsAuthenticated]
     search_base = ""
     data = ""
 
     def __init__(self):
         super().__init__()
-        self.response = {"data": False, "config": AppConfig().config}
+        self.response = {
+            "data": False,
+            "config": {
+                "enable_cast": EnvironmentSettings.ENABLE_CAST,
+                "downloads": AppConfig().config["downloads"],
+            },
+        }
         self.data = {"query": {"match_all": {}}}
         self.status_code = False
         self.context = False
@@ -102,6 +142,7 @@ class VideoApiView(ApiBaseView):
     """
 
     search_base = "ta_video/_doc/"
+    permission_classes = [AdminWriteOnly]
 
     def get(self, request, video_id):
         # pylint: disable=unused-argument
@@ -165,7 +206,6 @@ class VideoProgressView(ApiBaseView):
         message = {"position": position, "youtube_id": video_id}
         RedisArchivist().set_message(key, message)
         self.response = request.data
-
         return Response(self.response)
 
     def delete(self, request, video_id):
@@ -276,6 +316,7 @@ class ChannelApiView(ApiBaseView):
     """
 
     search_base = "ta_channel/_doc/"
+    permission_classes = [AdminWriteOnly]
 
     def get(self, request, channel_id):
         # pylint: disable=unused-argument
@@ -306,6 +347,7 @@ class ChannelApiListView(ApiBaseView):
 
     search_base = "ta_channel/_search/"
     valid_filter = ["subscribed"]
+    permission_classes = [AdminWriteOnly]
 
     def get(self, request):
         """get request"""
@@ -419,6 +461,7 @@ class PlaylistApiListView(ApiBaseView):
     """
 
     search_base = "ta_playlist/_search/"
+    permission_classes = [AdminWriteOnly]
 
     def get(self, request):
         """handle get request"""
@@ -467,6 +510,7 @@ class PlaylistApiView(ApiBaseView):
     """
 
     search_base = "ta_playlist/_doc/"
+    permission_classes = [AdminWriteOnly]
 
     def get(self, request, playlist_id):
         # pylint: disable=unused-argument
@@ -513,6 +557,7 @@ class DownloadApiView(ApiBaseView):
 
     search_base = "ta_download/_doc/"
     valid_status = ["pending", "ignore", "priority"]
+    permission_classes = [AdminOnly]
 
     def get(self, request, video_id):
         # pylint: disable=unused-argument
@@ -559,6 +604,7 @@ class DownloadApiListView(ApiBaseView):
 
     search_base = "ta_download/_search/"
     valid_filter = ["pending", "ignore"]
+    permission_classes = [AdminOnly]
 
     def get(self, request):
         """get request"""
@@ -663,10 +709,12 @@ class LoginApiView(ObtainAuthToken):
 
 class SnapshotApiListView(ApiBaseView):
     """resolves to /api/snapshot/
-    GET: returns snashot config plus list of existing snapshots
+    GET: returns snapshot config plus list of existing snapshots
     POST: take snapshot now
     """
 
+    permission_classes = [AdminOnly]
+
     @staticmethod
     def get(request):
         """handle get request"""
@@ -691,6 +739,8 @@ class SnapshotApiView(ApiBaseView):
     DELETE: delete a snapshot
     """
 
+    permission_classes = [AdminOnly]
+
     @staticmethod
     def get(request, snapshot_id):
         """handle get request"""
@@ -725,11 +775,87 @@ class SnapshotApiView(ApiBaseView):
         return Response(response)
 
 
+class BackupApiListView(ApiBaseView):
+    """resolves to /api/backup/
+    GET: returns list of available zip backups
+    POST: take zip backup now
+    """
+
+    permission_classes = [AdminOnly]
+    task_name = "run_backup"
+
+    @staticmethod
+    def get(request):
+        """handle get request"""
+        # pylint: disable=unused-argument
+        backup_files = ElasticBackup().get_all_backup_files()
+        return Response(backup_files)
+
+    def post(self, request):
+        """handle post request"""
+        # pylint: disable=unused-argument
+        response = TaskCommand().start(self.task_name)
+        message = {
+            "message": "backup task started",
+            "task_id": response["task_id"],
+        }
+
+        return Response(message)
+
+
+class BackupApiView(ApiBaseView):
+    """resolves to /api/backup/<filename>/
+    GET: return a single backup
+    POST: restore backup
+    DELETE: delete backup
+    """
+
+    permission_classes = [AdminOnly]
+    task_name = "restore_backup"
+
+    @staticmethod
+    def get(request, filename):
+        """get single backup"""
+        # pylint: disable=unused-argument
+        backup_file = ElasticBackup().build_backup_file_data(filename)
+        if not backup_file:
+            message = {"message": "file not found"}
+            return Response(message, status=404)
+
+        return Response(backup_file)
+
+    def post(self, request, filename):
+        """restore backup file"""
+        # pylint: disable=unused-argument
+        task = run_restore_backup.delay(filename)
+        message = {
+            "message": "backup restore task started",
+            "filename": filename,
+            "task_id": task.id,
+        }
+        return Response(message)
+
+    @staticmethod
+    def delete(request, filename):
+        """delete backup file"""
+        # pylint: disable=unused-argument
+
+        backup_file = ElasticBackup().delete_file(filename)
+        if not backup_file:
+            message = {"message": "file not found"}
+            return Response(message, status=404)
+
+        message = {"message": f"file {filename} deleted"}
+        return Response(message)
+
+
 class TaskListView(ApiBaseView):
     """resolves to /api/task-name/
     GET: return a list of all stored task results
     """
 
+    permission_classes = [AdminOnly]
+
     def get(self, request):
         """handle get request"""
         # pylint: disable=unused-argument
@@ -744,6 +870,8 @@ class TaskNameListView(ApiBaseView):
     POST: start new background process
     """
 
+    permission_classes = [AdminOnly]
+
     def get(self, request, task_name):
         """handle get request"""
         # pylint: disable=unused-argument
@@ -782,6 +910,7 @@ class TaskIDView(ApiBaseView):
     """
 
     valid_commands = ["stop", "kill"]
+    permission_classes = [AdminOnly]
 
     def get(self, request, task_id):
         """handle get request"""
@@ -833,6 +962,8 @@ class RefreshView(ApiBaseView):
     POST: start a manual refresh task
     """
 
+    permission_classes = [AdminOnly]
+
     def get(self, request):
         """handle get request"""
         request_type = request.GET.get("type")
@@ -859,6 +990,42 @@ class RefreshView(ApiBaseView):
         return Response(data)
 
 
+class UserConfigView(ApiBaseView):
+    """resolves to /api/config/user/
+    GET: return current user config
+    POST: update user config
+    """
+
+    def get(self, request):
+        """get config"""
+        user_id = request.user.id
+        response = UserConfig(user_id).get_config()
+        response.update({"user_id": user_id})
+
+        return Response(response)
+
+    def post(self, request):
+        """update config"""
+        user_id = request.user.id
+        data = request.data
+
+        user_conf = UserConfig(user_id)
+        for key, value in data.items():
+            try:
+                user_conf.set_value(key, value)
+            except ValueError as err:
+                message = {
+                    "status": "Bad Request",
+                    "message": f"failed updating {key} to '{value}', {err}",
+                }
+                return Response(message, status=400)
+
+        response = user_conf.get_config()
+        response.update({"user_id": user_id})
+
+        return Response(response)
+
+
 class CookieView(ApiBaseView):
     """resolves to /api/cookie/
     GET: check if cookie is enabled
@@ -866,6 +1033,8 @@ class CookieView(ApiBaseView):
     PUT: import cookie
     """
 
+    permission_classes = [AdminOnly]
+
     @staticmethod
     def get(request):
         """handle get request"""
@@ -953,6 +1122,8 @@ class TokenView(ApiBaseView):
     DELETE: revoke the token
     """
 
+    permission_classes = [AdminOnly]
+
     @staticmethod
     def delete(request):
         print("revoke API token")
@@ -978,16 +1149,52 @@ class NotificationView(ApiBaseView):
         return Response(RedisArchivist().list_items(query))
 
 
-class StatPrimaryView(ApiBaseView):
+class StatVideoView(ApiBaseView):
-    """resolves to /api/stats/primary/
+    """resolves to /api/stats/video/
-    GET: return document count
+    GET: return video stats
     """
 
     def get(self, request):
         """get stats"""
         # pylint: disable=unused-argument
 
-        return Response(Primary().process())
+        return Response(Video().process())
+
+
+class StatChannelView(ApiBaseView):
+    """resolves to /api/stats/channel/
+    GET: return channel stats
+    """
+
+    def get(self, request):
+        """get stats"""
+        # pylint: disable=unused-argument
+
+        return Response(Channel().process())
+
+
+class StatPlaylistView(ApiBaseView):
+    """resolves to /api/stats/playlist/
+    GET: return playlist stats
+    """
+
+    def get(self, request):
+        """get stats"""
+        # pylint: disable=unused-argument
+
+        return Response(Playlist().process())
+
+
+class StatDownloadView(ApiBaseView):
+    """resolves to /api/stats/download/
+    GET: return download stats
+    """
+
+    def get(self, request):
+        """get stats"""
+        # pylint: disable=unused-argument
+
+        return Response(Download().process())
 
 
 class StatWatchProgress(ApiBaseView):
@@ -1025,9 +1232,9 @@ class StatBiggestChannel(ApiBaseView):
     def get(self, request):
         """handle get request"""
 
-        order = request.GET.get("order", False)
+        order = request.GET.get("order", "doc_count")
         if order and order not in self.order_choices:
             message = {"message": f"invalid order parameter {order}"}
             return Response(message, status=400)
 
-        return Response(BiggestChannel().process())
+        return Response(BiggestChannel(order).process())

tubearchivist/config/management/commands/ta_connection.py

@@ -8,6 +8,7 @@ from time import sleep
 import requests
 from django.core.management.base import BaseCommand, CommandError
 from home.src.es.connect import ElasticWrap
+from home.src.ta.settings import EnvironmentSettings
 from home.src.ta.ta_redis import RedisArchivist
 
 TOPIC = """
@@ -132,7 +133,19 @@ class Command(BaseCommand):
         """check that path.repo var is set"""
         self.stdout.write("[5] check ES path.repo env var")
         response, _ = ElasticWrap("_nodes/_all/settings").get()
+        snaphost_roles = [
+            "data",
+            "data_cold",
+            "data_content",
+            "data_frozen",
+            "data_hot",
+            "data_warm",
+            "master",
+        ]
         for node in response["nodes"].values():
+            if not (set(node["roles"]) & set(snaphost_roles)):
+                continue
+
             if node["settings"]["path"].get("repo"):
                 self.stdout.write(
                     self.style.SUCCESS("    ✓ path.repo env var is set")
@@ -142,8 +155,9 @@ class Command(BaseCommand):
             message = (
                 "    🗙 path.repo env var not found. "
                 + "set the following env var to the ES container:\n"
-                + "    path.repo=/usr/share/elasticsearch/data/snapshot"
+                + "    path.repo="
+                + EnvironmentSettings.ES_SNAPSHOT_DIR
             )
-            self.stdout.write(self.style.ERROR(f"{message}"))
+            self.stdout.write(self.style.ERROR(message))
             sleep(60)
             raise CommandError(message)

tubearchivist/config/management/commands/ta_envcheck.py

@@ -11,6 +11,7 @@ import re
 
 from django.core.management.base import BaseCommand, CommandError
 from home.models import Account
+from home.src.ta.settings import EnvironmentSettings
 
 LOGO = """
 
@@ -96,18 +97,14 @@ class Command(BaseCommand):
 
     def _elastic_user_overwrite(self):
         """check for ELASTIC_USER overwrite"""
-        self.stdout.write("[2] set default ES user")
+        self.stdout.write("[2] check ES user overwrite")
-        if not os.environ.get("ELASTIC_USER"):
+        env = EnvironmentSettings.ES_USER
-            os.environ.setdefault("ELASTIC_USER", "elastic")
-
-        env = os.environ.get("ELASTIC_USER")
-
         self.stdout.write(self.style.SUCCESS(f"    ✓ ES user is set to {env}"))
 
     def _ta_port_overwrite(self):
         """set TA_PORT overwrite for nginx"""
         self.stdout.write("[3] check TA_PORT overwrite")
-        overwrite = os.environ.get("TA_PORT")
+        overwrite = EnvironmentSettings.TA_PORT
         if not overwrite:
             self.stdout.write(self.style.SUCCESS("    TA_PORT is not set"))
             return
@@ -125,7 +122,7 @@ class Command(BaseCommand):
     def _ta_uwsgi_overwrite(self):
         """set TA_UWSGI_PORT overwrite"""
         self.stdout.write("[4] check TA_UWSGI_PORT overwrite")
-        overwrite = os.environ.get("TA_UWSGI_PORT")
+        overwrite = EnvironmentSettings.TA_UWSGI_PORT
         if not overwrite:
             message = "    TA_UWSGI_PORT is not set"
             self.stdout.write(self.style.SUCCESS(message))
@@ -151,7 +148,7 @@ class Command(BaseCommand):
     def _enable_cast_overwrite(self):
         """cast workaround, remove auth for static files in nginx"""
         self.stdout.write("[5] check ENABLE_CAST overwrite")
-        overwrite = os.environ.get("ENABLE_CAST")
+        overwrite = EnvironmentSettings.ENABLE_CAST
         if not overwrite:
             self.stdout.write(self.style.SUCCESS("    ENABLE_CAST is not set"))
             return
@@ -174,8 +171,8 @@ class Command(BaseCommand):
             self.stdout.write(self.style.SUCCESS(message))
             return
 
-        name = os.environ.get("TA_USERNAME")
+        name = EnvironmentSettings.TA_USERNAME
-        password = os.environ.get("TA_PASSWORD")
+        password = EnvironmentSettings.TA_PASSWORD
         Account.objects.create_superuser(name, password)
         message = f"    ✓ new superuser with name {name} created"
         self.stdout.write(self.style.SUCCESS(message))

tubearchivist/config/management/commands/ta_migpath.py

@@ -6,8 +6,8 @@ import shutil
 
 from django.core.management.base import BaseCommand
 from home.src.es.connect import ElasticWrap, IndexPaginate
-from home.src.ta.config import AppConfig
 from home.src.ta.helper import ignore_filelist
+from home.src.ta.settings import EnvironmentSettings
 
 TOPIC = """
 
@@ -58,8 +58,7 @@ class FolderMigration:
     """migrate video archive folder"""
 
     def __init__(self):
-        self.config = AppConfig().config
+        self.videos = EnvironmentSettings.MEDIA_DIR
-        self.videos = self.config["application"]["videos"]
         self.bulk_list = []
 
     def get_to_migrate(self):
@@ -84,8 +83,8 @@ class FolderMigration:
 
     def create_folders(self, to_migrate):
         """create required channel folders"""
-        host_uid = self.config["application"]["HOST_UID"]
+        host_uid = EnvironmentSettings.HOST_UID
-        host_gid = self.config["application"]["HOST_GID"]
+        host_gid = EnvironmentSettings.HOST_GID
         all_channel_ids = {i["channel"]["channel_id"] for i in to_migrate}
 
         for channel_id in all_channel_ids:

tubearchivist/config/management/commands/ta_startup.py

@@ -8,14 +8,14 @@ import os
 from time import sleep
 
 from django.core.management.base import BaseCommand, CommandError
-from home.src.es.connect import ElasticWrap, IndexPaginate
 from home.src.es.index_setup import ElasitIndexWrap
 from home.src.es.snapshot import ElasticSnapshot
-from home.src.index.video_streams import MediaStreamExtractor
 from home.src.ta.config import AppConfig, ReleaseVersion
 from home.src.ta.helper import clear_dl_cache
+from home.src.ta.settings import EnvironmentSettings
 from home.src.ta.ta_redis import RedisArchivist
 from home.src.ta.task_manager import TaskManager
+from home.src.ta.users import UserConfig
 
 TOPIC = """
 
@@ -42,8 +42,7 @@ class Command(BaseCommand):
         self._version_check()
         self._mig_index_setup()
         self._mig_snapshot_check()
-        self._mig_set_streams()
+        self._mig_move_users_to_es()
-        self._mig_set_autostart()
 
     def _sync_redis_state(self):
         """make sure redis gets new config.json values"""
@@ -67,7 +66,7 @@ class Command(BaseCommand):
             "playlists",
             "videos",
         ]
-        cache_dir = AppConfig().config["application"]["cache_dir"]
+        cache_dir = EnvironmentSettings.CACHE_DIR
         for folder in folders:
             folder_path = os.path.join(cache_dir, folder)
             os.makedirs(folder_path, exist_ok=True)
@@ -117,8 +116,7 @@ class Command(BaseCommand):
     def _clear_dl_cache(self):
         """clear leftover files from dl cache"""
         self.stdout.write("[5] clear leftover files from dl cache")
-        config = AppConfig().config
+        leftover_files = clear_dl_cache(EnvironmentSettings.CACHE_DIR)
-        leftover_files = clear_dl_cache(config)
         if leftover_files:
             self.stdout.write(
                 self.style.SUCCESS(f"    ✓ cleared {leftover_files} files")
@@ -147,75 +145,92 @@ class Command(BaseCommand):
         self.stdout.write("[MIGRATION] setup snapshots")
         ElasticSnapshot().setup()
 
-    def _mig_set_streams(self):
+    def _mig_move_users_to_es(self):  # noqa: C901
-        """migration: update from 0.3.5 to 0.3.6, set streams and media_size"""
+        """migration: update from 0.4.1 to 0.4.2 move user config to ES"""
-        self.stdout.write("[MIGRATION] index streams and media size")
+        self.stdout.write("[MIGRATION] move user configuration to ES")
-        videos = AppConfig().config["application"]["videos"]
+        redis = RedisArchivist()
-        data = {
+
-            "query": {
+        # 1: Find all users in Redis
-                "bool": {"must_not": [{"exists": {"field": "streams"}}]}
+        users = {i.split(":")[0] for i in redis.list_keys("[0-9]*:")}
-            },
+        if not users:
-            "_source": ["media_url", "youtube_id"],
+            self.stdout.write("    no users needed migrating to ES")
-        }
-        all_missing = IndexPaginate("ta_video", data).get_results()
-        if not all_missing:
-            self.stdout.write("    no videos need updating")
             return
 
-        total = len(all_missing)
+        # 2: Write all Redis user settings to ES
-        for idx, missing in enumerate(all_missing):
+        # 3: Remove user settings from Redis
-            media_url = missing["media_url"]
+        try:
-            youtube_id = missing["youtube_id"]
+            for user in users:
-            media_path = os.path.join(videos, media_url)
+                new_conf = UserConfig(user)
-            if not os.path.exists(media_path):
-                self.stdout.write(f"    file not found: {media_path}")
-                self.stdout.write("    run file system rescan to fix")
-                continue
 
-            media = MediaStreamExtractor(media_path)
+                stylesheet_key = f"{user}:color"
-            vid_data = {
+                stylesheet = redis.get_message(stylesheet_key).get("status")
-                "doc": {
+                if stylesheet:
-                    "streams": media.extract_metadata(),
+                    new_conf.set_value("stylesheet", stylesheet)
-                    "media_size": media.get_file_size(),
+                    redis.del_message(stylesheet_key)
-                }
-            }
-            path = f"ta_video/_update/{youtube_id}"
-            response, status_code = ElasticWrap(path).post(data=vid_data)
-            if not status_code == 200:
-                self.stdout.errors(
-                    f"    update failed: {path}, {response}, {status_code}"
-                )
 
-            if idx % 100 == 0:
+                sort_by_key = f"{user}:sort_by"
-                self.stdout.write(f"    progress {idx}/{total}")
+                sort_by = redis.get_message(sort_by_key).get("status")
+                if sort_by:
+                    new_conf.set_value("sort_by", sort_by)
+                    redis.del_message(sort_by_key)
+
+                page_size_key = f"{user}:page_size"
+                page_size = redis.get_message(page_size_key).get("status")
+                if page_size:
+                    new_conf.set_value("page_size", page_size)
+                    redis.del_message(page_size_key)
+
+                sort_order_key = f"{user}:sort_order"
+                sort_order = redis.get_message(sort_order_key).get("status")
+                if sort_order:
+                    new_conf.set_value("sort_order", sort_order)
+                    redis.del_message(sort_order_key)
+
+                grid_items_key = f"{user}:grid_items"
+                grid_items = redis.get_message(grid_items_key).get("status")
+                if grid_items:
+                    new_conf.set_value("grid_items", grid_items)
+                    redis.del_message(grid_items_key)
+
+                hide_watch_key = f"{user}:hide_watched"
+                hide_watch = redis.get_message(hide_watch_key).get("status")
+                if hide_watch:
+                    new_conf.set_value("hide_watched", hide_watch)
+                    redis.del_message(hide_watch_key)
+
+                ignore_only_key = f"{user}:show_ignored_only"
+                ignore_only = redis.get_message(ignore_only_key).get("status")
+                if ignore_only:
+                    new_conf.set_value("show_ignored_only", ignore_only)
+                    redis.del_message(ignore_only_key)
+
+                subed_only_key = f"{user}:show_subed_only"
+                subed_only = redis.get_message(subed_only_key).get("status")
+                if subed_only:
+                    new_conf.set_value("show_subed_only", subed_only)
+                    redis.del_message(subed_only_key)
+
+                for view in ["channel", "playlist", "home", "downloads"]:
+                    view_key = f"{user}:view:{view}"
+                    view_style = redis.get_message(view_key).get("status")
+                    if view_style:
+                        new_conf.set_value(f"view_style_{view}", view_style)
+                        redis.del_message(view_key)
 
-    def _mig_set_autostart(self):
-        """migration: update from 0.3.5 to 0.3.6 set auto_start to false"""
-        self.stdout.write("[MIGRATION] set default download auto_start")
-        data = {
-            "query": {
-                "bool": {"must_not": [{"exists": {"field": "auto_start"}}]}
-            },
-            "script": {"source": "ctx._source['auto_start'] = false"},
-        }
-        path = "ta_download/_update_by_query"
-        response, status_code = ElasticWrap(path).post(data=data)
-        if status_code == 200:
-            updated = response.get("updated", 0)
-            if updated:
                 self.stdout.write(
                     self.style.SUCCESS(
-                        f"    ✓ {updated} videos updated in ta_download"
+                        f"    ✓ Settings for user '{user}' migrated to ES"
                     )
                 )
+        except Exception as err:
+            message = "    🗙 user migration to ES failed"
+            self.stdout.write(self.style.ERROR(message))
+            self.stdout.write(self.style.ERROR(err))
+            sleep(60)
+            raise CommandError(message) from err
         else:
             self.stdout.write(
-                    "    no videos needed updating in ta_download"
+                self.style.SUCCESS(
+                    "    ✓ Settings for all users migrated to ES"
+                )
             )
-            return
-
-        message = "    🗙 ta_download auto_start update failed"
-        self.stdout.write(self.style.ERROR(message))
-        self.stdout.write(response)
-        sleep(60)
-        raise CommandError(message)

tubearchivist/config/settings.py

@@ -17,8 +17,8 @@ from pathlib import Path
 import ldap
 from corsheaders.defaults import default_headers
 from django_auth_ldap.config import LDAPSearch
-from home.src.ta.config import AppConfig
 from home.src.ta.helper import ta_host_parser
+from home.src.ta.settings import EnvironmentSettings
 
 # Build paths inside the project like this: BASE_DIR / 'subdir'.
 BASE_DIR = Path(__file__).resolve().parent.parent
@@ -27,7 +27,7 @@ BASE_DIR = Path(__file__).resolve().parent.parent
 # Quick-start development settings - unsuitable for production
 # See https://docs.djangoproject.com/en/3.2/howto/deployment/checklist/
 
-PW_HASH = hashlib.sha256(environ["TA_PASSWORD"].encode())
+PW_HASH = hashlib.sha256(EnvironmentSettings.TA_PASSWORD.encode())
 SECRET_KEY = PW_HASH.hexdigest()
 
 # SECURITY WARNING: don't run with debug turned on in production!
@@ -64,6 +64,7 @@ MIDDLEWARE = [
     "django.contrib.auth.middleware.AuthenticationMiddleware",
     "django.contrib.messages.middleware.MessageMiddleware",
     "django.middleware.clickjacking.XFrameOptionsMiddleware",
+    "home.src.ta.health.HealthCheckMiddleware",
 ]
 
 ROOT_URLCONF = "config.urls"
@@ -174,13 +175,12 @@ if bool(environ.get("TA_LDAP")):
             ldap.OPT_X_TLS_REQUIRE_CERT: ldap.OPT_X_TLS_NEVER,
         }
 
-    global AUTHENTICATION_BACKENDS
     AUTHENTICATION_BACKENDS = ("django_auth_ldap.backend.LDAPBackend",)
 
 # Database
 # https://docs.djangoproject.com/en/3.2/ref/settings/#databases
 
-CACHE_DIR = AppConfig().config["application"]["cache_dir"]
+CACHE_DIR = EnvironmentSettings.CACHE_DIR
 DB_PATH = path.join(CACHE_DIR, "db.sqlite3")
 DATABASES = {
     "default": {
@@ -210,12 +210,25 @@ AUTH_PASSWORD_VALIDATORS = [
 
 AUTH_USER_MODEL = "home.Account"
 
+# Forward-auth authentication
+if bool(environ.get("TA_ENABLE_AUTH_PROXY")):
+    TA_AUTH_PROXY_USERNAME_HEADER = (
+        environ.get("TA_AUTH_PROXY_USERNAME_HEADER") or "HTTP_REMOTE_USER"
+    )
+    TA_AUTH_PROXY_LOGOUT_URL = environ.get("TA_AUTH_PROXY_LOGOUT_URL")
+
+    MIDDLEWARE.append("home.src.ta.auth.HttpRemoteUserMiddleware")
+
+    AUTHENTICATION_BACKENDS = (
+        "django.contrib.auth.backends.RemoteUserBackend",
+    )
+
 
 # Internationalization
 # https://docs.djangoproject.com/en/3.2/topics/i18n/
 
 LANGUAGE_CODE = "en-us"
-TIME_ZONE = environ.get("TZ") or "UTC"
+TIME_ZONE = EnvironmentSettings.TZ
 USE_I18N = True
 USE_L10N = True
 USE_TZ = True
@@ -256,4 +269,4 @@ CORS_ALLOW_HEADERS = list(default_headers) + [
 
 # TA application settings
 TA_UPSTREAM = "https://github.com/tubearchivist/tubearchivist"
-TA_VERSION = "v0.4.1"
+TA_VERSION = "v0.4.4"

tubearchivist/home/config.json

@@ -1,18 +1,5 @@
 {
-    "archive": {
-        "sort_by": "published",
-        "sort_order": "desc",
-        "page_size": 12
-    },
-    "default_view": {
-        "home": "grid",
-        "channel": "list",
-        "downloads": "list",
-        "playlist": "grid",
-        "grid_items": 3
-    },
     "subscriptions": {
-        "auto_download": false,
         "channel_size": 50,
         "live_channel_size": 50,
         "shorts_channel_size": 50,
@@ -38,11 +25,6 @@
         "integrate_sponsorblock": false
     },
     "application": {
-        "app_root": "/app",
-        "cache_dir": "/cache",
-        "videos": "/youtube",
-        "colors": "dark",
-        "enable_cast": false,
         "enable_snapshot": true
     },
     "scheduler": {

tubearchivist/home/src/download/queue.py

@@ -16,9 +16,8 @@ from home.src.download.yt_dlp_base import YtWrap
 from home.src.es.connect import ElasticWrap, IndexPaginate
 from home.src.index.playlist import YoutubePlaylist
 from home.src.index.video_constants import VideoTypeEnum
-from home.src.index.video_streams import DurationConverter
 from home.src.ta.config import AppConfig
-from home.src.ta.helper import is_shorts
+from home.src.ta.helper import get_duration_str, is_shorts
 
 
 class PendingIndex:
@@ -229,6 +228,11 @@ class PendingList(PendingIndex):
         """add all videos of playlist to list"""
         playlist = YoutubePlaylist(url)
         playlist.build_json()
+        if not playlist.json_data:
+            message = f"{playlist.youtube_id}: failed to extract metadata"
+            print(message)
+            raise ValueError(message)
+
         video_results = playlist.json_data.get("playlist_entries")
         youtube_ids = [i["youtube_id"] for i in video_results]
         for video_id in youtube_ids:
@@ -335,9 +339,6 @@ class PendingList(PendingIndex):
     def _parse_youtube_details(self, vid, vid_type=VideoTypeEnum.VIDEOS):
         """parse response"""
         vid_id = vid.get("id")
-        duration_str = DurationConverter.get_str(vid["duration"])
-        if duration_str == "NA":
-            print(f"skip extracting duration for: {vid_id}")
         published = datetime.strptime(vid["upload_date"], "%Y%m%d").strftime(
             "%Y-%m-%d"
         )
@@ -349,7 +350,7 @@ class PendingList(PendingIndex):
             "vid_thumb_url": vid["thumbnail"],
             "title": vid["title"],
             "channel_id": vid["channel_id"],
-            "duration": duration_str,
+            "duration": get_duration_str(vid["duration"]),
             "published": published,
             "timestamp": int(datetime.now().timestamp()),
             # Pulling enum value out so it is serializable

tubearchivist/home/src/download/subscriptions.py

@@ -187,6 +187,11 @@ class PlaylistSubscription:
             playlist_h = YoutubePlaylist(playlist_id)
             playlist_h.all_youtube_ids = all_youtube_ids
             playlist_h.build_json()
+            if not playlist_h.json_data:
+                message = f"{playlist_h.youtube_id}: failed to extract data"
+                print(message)
+                raise ValueError(message)
+
             playlist_h.json_data["playlist_subscribed"] = subscribed
             playlist_h.upload_to_es()
             playlist_h.add_vids_to_playlist()

tubearchivist/home/src/download/thumbnails.py

@@ -11,7 +11,7 @@ from time import sleep
 
 import requests
 from home.src.es.connect import ElasticWrap, IndexPaginate
-from home.src.ta.config import AppConfig
+from home.src.ta.settings import EnvironmentSettings
 from mutagen.mp4 import MP4, MP4Cover
 from PIL import Image, ImageFile, ImageFilter, UnidentifiedImageError
 
@@ -21,8 +21,7 @@ ImageFile.LOAD_TRUNCATED_IMAGES = True
 class ThumbManagerBase:
     """base class for thumbnail management"""
 
-    CONFIG = AppConfig().config
+    CACHE_DIR = EnvironmentSettings.CACHE_DIR
-    CACHE_DIR = CONFIG["application"]["cache_dir"]
     VIDEO_DIR = os.path.join(CACHE_DIR, "videos")
     CHANNEL_DIR = os.path.join(CACHE_DIR, "channels")
     PLAYLIST_DIR = os.path.join(CACHE_DIR, "playlists")
@@ -70,7 +69,7 @@ class ThumbManagerBase:
             img_raw = Image.open(self.fallback)
             return img_raw
 
-        app_root = self.CONFIG["application"]["app_root"]
+        app_root = EnvironmentSettings.APP_DIR
         default_map = {
             "video": os.path.join(
                 app_root, "static/img/default-video-thumb.jpg"
@@ -246,9 +245,10 @@ class ThumbManager(ThumbManagerBase):
 class ValidatorCallback:
     """handle callback validate thumbnails page by page"""
 
-    def __init__(self, source, index_name):
+    def __init__(self, source, index_name, counter=0):
         self.source = source
         self.index_name = index_name
+        self.counter = counter
 
     def run(self):
         """run the task for page"""
@@ -379,14 +379,14 @@ class ThumbFilesystem:
 class EmbedCallback:
     """callback class to embed thumbnails"""
 
-    CONFIG = AppConfig().config
+    CACHE_DIR = EnvironmentSettings.CACHE_DIR
-    CACHE_DIR = CONFIG["application"]["cache_dir"]
+    MEDIA_DIR = EnvironmentSettings.MEDIA_DIR
-    MEDIA_DIR = CONFIG["application"]["videos"]
     FORMAT = MP4Cover.FORMAT_JPEG
 
-    def __init__(self, source, index_name):
+    def __init__(self, source, index_name, counter=0):
         self.source = source
         self.index_name = index_name
+        self.counter = counter
 
     def run(self):
         """run embed"""

tubearchivist/home/src/download/yt_dlp_base.py

@@ -10,6 +10,7 @@ from http import cookiejar
 from io import StringIO
 
 import yt_dlp
+from home.src.ta.settings import EnvironmentSettings
 from home.src.ta.ta_redis import RedisArchivist
 
 
@@ -86,6 +87,7 @@ class CookieHandler:
     def __init__(self, config):
         self.cookie_io = False
         self.config = config
+        self.cache_dir = EnvironmentSettings.CACHE_DIR
 
     def get(self):
         """get cookie io stream"""
@@ -95,8 +97,9 @@ class CookieHandler:
 
     def import_cookie(self):
         """import cookie from file"""
-        cache_path = self.config["application"]["cache_dir"]
+        import_path = os.path.join(
-        import_path = os.path.join(cache_path, "import", "cookies.google.txt")
+            self.cache_dir, "import", "cookies.google.txt"
+        )
 
         try:
             with open(import_path, encoding="utf-8") as cookie_file:
@@ -111,7 +114,7 @@ class CookieHandler:
         print("cookie: import successful")
 
     def set_cookie(self, cookie):
-        """set cookie str and activate in cofig"""
+        """set cookie str and activate in config"""
         RedisArchivist().set_message("cookie", cookie, save=True)
         path = ".downloads.cookie_import"
         RedisArchivist().set_message("config", True, path=path, save=True)

tubearchivist/home/src/download/yt_dlp_handler.py

@@ -21,6 +21,7 @@ from home.src.index.video import YoutubeVideo, index_new_video
 from home.src.index.video_constants import VideoTypeEnum
 from home.src.ta.config import AppConfig
 from home.src.ta.helper import ignore_filelist
+from home.src.ta.settings import EnvironmentSettings
 
 
 class DownloadPostProcess:
@@ -153,6 +154,8 @@ class VideoDownloader:
         self.youtube_id_list = youtube_id_list
         self.task = task
         self.config = AppConfig().config
+        self.cache_dir = EnvironmentSettings.CACHE_DIR
+        self.media_dir = EnvironmentSettings.MEDIA_DIR
         self._build_obs()
         self.channels = set()
         self.videos = set()
@@ -262,10 +265,7 @@ class VideoDownloader:
         """initial obs"""
         self.obs = {
             "merge_output_format": "mp4",
-            "outtmpl": (
+            "outtmpl": (self.cache_dir + "/download/%(id)s.mp4"),
-                self.config["application"]["cache_dir"]
-                + "/download/%(id)s.mp4"
-            ),
             "progress_hooks": [self._progress_hook],
             "noprogress": True,
             "continuedl": True,
@@ -340,7 +340,7 @@ class VideoDownloader:
         if format_overwrite:
             obs["format"] = format_overwrite
 
-        dl_cache = self.config["application"]["cache_dir"] + "/download/"
+        dl_cache = self.cache_dir + "/download/"
 
         # check if already in cache to continue from there
         all_cached = ignore_filelist(os.listdir(dl_cache))
@@ -370,20 +370,20 @@ class VideoDownloader:
 
     def move_to_archive(self, vid_dict):
         """move downloaded video from cache to archive"""
-        videos = self.config["application"]["videos"]
+        host_uid = EnvironmentSettings.HOST_UID
-        host_uid = self.config["application"]["HOST_UID"]
+        host_gid = EnvironmentSettings.HOST_GID
-        host_gid = self.config["application"]["HOST_GID"]
         # make folder
-        folder = os.path.join(videos, vid_dict["channel"]["channel_id"])
+        folder = os.path.join(
+            self.media_dir, vid_dict["channel"]["channel_id"]
+        )
         if not os.path.exists(folder):
             os.makedirs(folder)
             if host_uid and host_gid:
                 os.chown(folder, host_uid, host_gid)
         # move media file
         media_file = vid_dict["youtube_id"] + ".mp4"
-        cache_dir = self.config["application"]["cache_dir"]
+        old_path = os.path.join(self.cache_dir, "download", media_file)
-        old_path = os.path.join(cache_dir, "download", media_file)
+        new_path = os.path.join(self.media_dir, vid_dict["media_url"])
-        new_path = os.path.join(videos, vid_dict["media_url"])
         # move media file and fix permission
         shutil.move(old_path, new_path, copy_function=shutil.copyfile)
         if host_uid and host_gid:
@@ -417,7 +417,7 @@ class VideoDownloader:
                 "lang": "painless",
             },
         }
-        response, _ = ElasticWrap(path, config=self.config).post(data=data)
+        response, _ = ElasticWrap(path).post(data=data)
         updated = response.get("updated")
         if updated:
             print(f"[download] reset auto start on {updated} videos.")

tubearchivist/home/src/es/backup.py

@@ -13,14 +13,18 @@ from datetime import datetime
 from home.src.es.connect import ElasticWrap, IndexPaginate
 from home.src.ta.config import AppConfig
 from home.src.ta.helper import get_mapping, ignore_filelist
+from home.src.ta.settings import EnvironmentSettings
 
 
 class ElasticBackup:
     """dump index to nd-json files for later bulk import"""
 
+    INDEX_SPLIT = ["comment"]
+    CACHE_DIR = EnvironmentSettings.CACHE_DIR
+    BACKUP_DIR = os.path.join(CACHE_DIR, "backup")
+
     def __init__(self, reason=False, task=False):
         self.config = AppConfig().config
-        self.cache_dir = self.config["application"]["cache_dir"]
         self.timestamp = datetime.now().strftime("%Y%m%d")
         self.index_config = get_mapping()
         self.reason = reason
@@ -51,14 +55,18 @@ class ElasticBackup:
 
     def backup_index(self, index_name):
         """export all documents of a single index"""
-        paginate = IndexPaginate(
+        paginate_kwargs = {
-            f"ta_{index_name}",
+            "data": {"query": {"match_all": {}}},
-            data={"query": {"match_all": {}}},
+            "keep_source": True,
-            keep_source=True,
+            "callback": BackupCallback,
-            callback=BackupCallback,
+            "task": self.task,
-            task=self.task,
+            "total": self._get_total(index_name),
-            total=self._get_total(index_name),
+        }
-        )
+
+        if index_name in self.INDEX_SPLIT:
+            paginate_kwargs.update({"size": 200})
+
+        paginate = IndexPaginate(f"ta_{index_name}", **paginate_kwargs)
         _ = paginate.get_results()
 
     @staticmethod
@@ -72,14 +80,13 @@ class ElasticBackup:
     def zip_it(self):
         """pack it up into single zip file"""
         file_name = f"ta_backup-{self.timestamp}-{self.reason}.zip"
-        folder = os.path.join(self.cache_dir, "backup")
 
         to_backup = []
-        for file in os.listdir(folder):
+        for file in os.listdir(self.BACKUP_DIR):
             if file.endswith(".json"):
-                to_backup.append(os.path.join(folder, file))
+                to_backup.append(os.path.join(self.BACKUP_DIR, file))
 
-        backup_file = os.path.join(folder, file_name)
+        backup_file = os.path.join(self.BACKUP_DIR, file_name)
 
         comp = zipfile.ZIP_DEFLATED
         with zipfile.ZipFile(backup_file, "w", compression=comp) as zip_f:
@@ -92,7 +99,7 @@ class ElasticBackup:
 
     def post_bulk_restore(self, file_name):
         """send bulk to es"""
-        file_path = os.path.join(self.cache_dir, file_name)
+        file_path = os.path.join(self.CACHE_DIR, file_name)
         with open(file_path, "r", encoding="utf-8") as f:
             data = f.read()
 
@@ -103,9 +110,7 @@ class ElasticBackup:
 
     def get_all_backup_files(self):
         """build all available backup files for view"""
-        backup_dir = os.path.join(self.cache_dir, "backup")
+        all_backup_files = ignore_filelist(os.listdir(self.BACKUP_DIR))
-        backup_files = os.listdir(backup_dir)
-        all_backup_files = ignore_filelist(backup_files)
         all_available_backups = [
             i
             for i in all_backup_files
@@ -114,8 +119,19 @@ class ElasticBackup:
         all_available_backups.sort(reverse=True)
 
         backup_dicts = []
-        for backup_file in all_available_backups:
+        for filename in all_available_backups:
-            file_split = backup_file.split("-")
+            data = self.build_backup_file_data(filename)
+            backup_dicts.append(data)
+
+        return backup_dicts
+
+    def build_backup_file_data(self, filename):
+        """build metadata of single backup file"""
+        file_path = os.path.join(self.BACKUP_DIR, filename)
+        if not os.path.exists(file_path):
+            return False
+
+        file_split = filename.split("-")
         if len(file_split) == 2:
             timestamp = file_split[1].strip(".zip")
             reason = False
@@ -123,14 +139,15 @@ class ElasticBackup:
             timestamp = file_split[1]
             reason = file_split[2].strip(".zip")
 
-            to_add = {
+        data = {
-                "filename": backup_file,
+            "filename": filename,
+            "file_path": file_path,
+            "file_size": os.path.getsize(file_path),
             "timestamp": timestamp,
             "reason": reason,
         }
-            backup_dicts.append(to_add)
 
-        return backup_dicts
+        return data
 
     def restore(self, filename):
         """
@@ -142,22 +159,19 @@ class ElasticBackup:
 
     def _unpack_zip_backup(self, filename):
         """extract backup zip and return filelist"""
-        backup_dir = os.path.join(self.cache_dir, "backup")
+        file_path = os.path.join(self.BACKUP_DIR, filename)
-        file_path = os.path.join(backup_dir, filename)
 
         with zipfile.ZipFile(file_path, "r") as z:
             zip_content = z.namelist()
-            z.extractall(backup_dir)
+            z.extractall(self.BACKUP_DIR)
 
         return zip_content
 
     def _restore_json_files(self, zip_content):
         """go through the unpacked files and restore"""
-        backup_dir = os.path.join(self.cache_dir, "backup")
-
         for idx, json_f in enumerate(zip_content):
             self._notify_restore(idx, json_f, len(zip_content))
-            file_name = os.path.join(backup_dir, json_f)
+            file_name = os.path.join(self.BACKUP_DIR, json_f)
 
             if not json_f.startswith("es_") or not json_f.endswith(".json"):
                 os.remove(file_name)
@@ -194,22 +208,32 @@ class ElasticBackup:
             print("no backup files to rotate")
             return
 
-        backup_dir = os.path.join(self.cache_dir, "backup")
-
         all_to_delete = auto[rotate:]
         for to_delete in all_to_delete:
-            file_path = os.path.join(backup_dir, to_delete["filename"])
+            self.delete_file(to_delete["filename"])
+
+    def delete_file(self, filename):
+        """delete backup file"""
+        file_path = os.path.join(self.BACKUP_DIR, filename)
+        if not os.path.exists(file_path):
+            print(f"backup file not found: {filename}")
+            return False
+
         print(f"remove old backup file: {file_path}")
         os.remove(file_path)
 
+        return file_path
+
 
 class BackupCallback:
     """handle backup ndjson writer as callback for IndexPaginate"""
 
-    def __init__(self, source, index_name):
+    def __init__(self, source, index_name, counter=0):
         self.source = source
         self.index_name = index_name
+        self.counter = counter
         self.timestamp = datetime.now().strftime("%Y%m%d")
+        self.cache_dir = EnvironmentSettings.CACHE_DIR
 
     def run(self):
         """run the junk task"""
@@ -236,8 +260,8 @@ class BackupCallback:
 
     def _write_es_json(self, file_content):
         """write nd-json file for es _bulk API to disk"""
-        cache_dir = AppConfig().config["application"]["cache_dir"]
+        index = self.index_name.lstrip("ta_")
-        file_name = f"es_{self.index_name.lstrip('ta_')}-{self.timestamp}.json"
+        file_name = f"es_{index}-{self.timestamp}-{self.counter}.json"
-        file_path = os.path.join(cache_dir, "backup", file_name)
+        file_path = os.path.join(self.cache_dir, "backup", file_name)
         with open(file_path, "a+", encoding="utf-8") as f:
             f.write(file_content)

tubearchivist/home/src/es/connect.py

@@ -6,9 +6,11 @@ functionality:
 # pylint: disable=missing-timeout
 
 import json
+from typing import Any
 
 import requests
-from home.src.ta.config import AppConfig
+import urllib3
+from home.src.ta.settings import EnvironmentSettings
 
 
 class ElasticWrap:
@@ -16,61 +18,94 @@ class ElasticWrap:
     returns response json and status code tuple
     """
 
-    def __init__(self, path, config=False):
+    def __init__(self, path: str):
-        self.url = False
+        self.url: str = f"{EnvironmentSettings.ES_URL}/{path}"
-        self.auth = False
+        self.auth: tuple[str, str] = (
-        self.path = path
+            EnvironmentSettings.ES_USER,
-        self.config = config
+            EnvironmentSettings.ES_PASS,
-        self._get_config()
-
-    def _get_config(self):
-        """add config if not passed"""
-        if not self.config:
-            self.config = AppConfig().config
-
-        es_url = self.config["application"]["es_url"]
-        self.auth = self.config["application"]["es_auth"]
-        self.url = f"{es_url}/{self.path}"
-
-    def get(self, data=False, timeout=10, print_error=True):
-        """get data from es"""
-        if data:
-            response = requests.get(
-                self.url, json=data, auth=self.auth, timeout=timeout
         )
-        else:
+
-            response = requests.get(self.url, auth=self.auth, timeout=timeout)
+        if EnvironmentSettings.ES_DISABLE_VERIFY_SSL:
+            urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
+
+    def get(
+        self,
+        data: bool | dict = False,
+        timeout: int = 10,
+        print_error: bool = True,
+    ) -> tuple[dict, int]:
+        """get data from es"""
+
+        kwargs: dict[str, Any] = {
+            "auth": self.auth,
+            "timeout": timeout,
+        }
+
+        if EnvironmentSettings.ES_DISABLE_VERIFY_SSL:
+            kwargs["verify"] = False
+
+        if data:
+            kwargs["json"] = data
+
+        response = requests.get(self.url, **kwargs)
+
         if print_error and not response.ok:
             print(response.text)
 
         return response.json(), response.status_code
 
-    def post(self, data=False, ndjson=False):
+    def post(
+        self, data: bool | dict = False, ndjson: bool = False
+    ) -> tuple[dict, int]:
         """post data to es"""
-        if ndjson:
-            headers = {"Content-type": "application/x-ndjson"}
-            payload = data
-        else:
-            headers = {"Content-type": "application/json"}
-            payload = json.dumps(data)
 
-        if data:
+        kwargs: dict[str, Any] = {"auth": self.auth}
-            response = requests.post(
+
-                self.url, data=payload, headers=headers, auth=self.auth
+        if ndjson and data:
+            kwargs.update(
+                {
+                    "headers": {"Content-type": "application/x-ndjson"},
+                    "data": data,
+                }
             )
-        else:
+        elif data:
-            response = requests.post(self.url, headers=headers, auth=self.auth)
+            kwargs.update(
+                {
+                    "headers": {"Content-type": "application/json"},
+                    "data": json.dumps(data),
+                }
+            )
+
+        if EnvironmentSettings.ES_DISABLE_VERIFY_SSL:
+            kwargs["verify"] = False
+
+        response = requests.post(self.url, **kwargs)
 
         if not response.ok:
             print(response.text)
 
         return response.json(), response.status_code
 
-    def put(self, data, refresh=False):
+    def put(
+        self,
+        data: bool | dict = False,
+        refresh: bool = False,
+    ) -> tuple[dict, Any]:
         """put data to es"""
+
         if refresh:
             self.url = f"{self.url}/?refresh=true"
-        response = requests.put(f"{self.url}", json=data, auth=self.auth)
+
+        kwargs: dict[str, Any] = {
+            "json": data,
+            "auth": self.auth,
+        }
+
+        if EnvironmentSettings.ES_DISABLE_VERIFY_SSL:
+            kwargs["verify"] = False
+
+        response = requests.put(self.url, **kwargs)
+
         if not response.ok:
             print(response.text)
             print(data)
@@ -78,14 +113,25 @@ class ElasticWrap:
 
         return response.json(), response.status_code
 
-    def delete(self, data=False, refresh=False):
+    def delete(
+        self,
+        data: bool | dict = False,
+        refresh: bool = False,
+    ) -> tuple[dict, Any]:
         """delete document from es"""
+
         if refresh:
             self.url = f"{self.url}/?refresh=true"
+
+        kwargs: dict[str, Any] = {"auth": self.auth}
+
         if data:
-            response = requests.delete(self.url, json=data, auth=self.auth)
+            kwargs["json"] = data
-        else:
+
-            response = requests.delete(self.url, auth=self.auth)
+        if EnvironmentSettings.ES_DISABLE_VERIFY_SSL:
+            kwargs["verify"] = False
+
+        response = requests.delete(self.url, **kwargs)
 
         if not response.ok:
             print(response.text)
@@ -156,7 +202,9 @@ class IndexPaginate:
                     all_results.append(hit["_source"])
 
             if self.kwargs.get("callback"):
-                self.kwargs.get("callback")(all_hits, self.index_name).run()
+                self.kwargs.get("callback")(
+                    all_hits, self.index_name, counter=counter
+                ).run()
 
             if self.kwargs.get("task"):
                 print(f"{self.index_name}: processing page {counter}")

tubearchivist/home/src/es/index_mapping.json

@@ -1,5 +1,17 @@
 {
     "index_config": [{
+            "index_name": "config",
+            "expected_map": {
+                "config": {
+                    "type": "object",
+                    "enabled": false
+                }
+            },
+            "expected_set": {
+                "number_of_replicas": "0"
+            }
+        },
+        {
             "index_name": "channel",
             "expected_map": {
                 "channel_id": {
@@ -62,7 +74,7 @@
                             "type": "boolean"
                         },
                         "integrate_sponsorblock": {
-                            "type" : "boolean"
+                            "type": "boolean"
                         }
                     }
                 }
@@ -156,7 +168,7 @@
                                     "type": "boolean"
                                 },
                                 "integrate_sponsorblock": {
-                                    "type" : "boolean"
+                                    "type": "boolean"
                                 }
                             }
                         }
@@ -224,19 +236,37 @@
                 "comment_count": {
                     "type": "long"
                 },
-                "stats" : {
+                "stats": {
-                    "properties" : {
+                    "properties": {
-                        "average_rating" : {
+                        "average_rating": {
-                            "type" : "float"
+                            "type": "float"
                         },
-                        "dislike_count" : {
+                        "dislike_count": {
-                            "type" : "long"
+                            "type": "long"
                         },
-                        "like_count" : {
+                        "like_count": {
-                            "type" : "long"
+                            "type": "long"
                         },
-                        "view_count" : {
+                        "view_count": {
-                            "type" : "long"
+                            "type": "long"
+                        }
+                    }
+                },
+                "player": {
+                    "properties": {
+                        "duration": {
+                            "type": "long"
+                        },
+                        "duration_str": {
+                            "type": "keyword",
+                            "index": false
+                        },
+                        "watched": {
+                            "type": "boolean"
+                        },
+                        "watched_date": {
+                            "type": "date",
+                            "format": "epoch_second"
                         }
                     }
                 },
@@ -302,28 +332,28 @@
                         "is_enabled": {
                             "type": "boolean"
                         },
-                        "segments" : {
+                        "segments": {
-                            "properties" : {
+                            "properties": {
-                                "UUID" : {
+                                "UUID": {
                                     "type": "keyword"
                                 },
-                                "actionType" : {
+                                "actionType": {
                                     "type": "keyword"
                                 },
-                                "category" : {
+                                "category": {
                                     "type": "keyword"
                                 },
-                                "locked" : {
+                                "locked": {
-                                    "type" : "short"
+                                    "type": "short"
                                 },
-                                "segment" : {
+                                "segment": {
-                                    "type" : "float"
+                                    "type": "float"
                                 },
-                                "videoDuration" : {
+                                "videoDuration": {
-                                    "type" : "float"
+                                    "type": "float"
                                 },
-                                "votes" : {
+                                "votes": {
-                                    "type" : "long"
+                                    "type": "long"
                                 }
                             }
                         }
@@ -447,6 +477,41 @@
                 "playlist_last_refresh": {
                     "type": "date",
                     "format": "epoch_second"
+                },
+                "playlist_entries": {
+                    "properties": {
+                        "downloaded": {
+                            "type": "boolean"
+                        },
+                        "idx": {
+                            "type": "long"
+                        },
+                        "title": {
+                            "type": "text",
+                            "analyzer": "english",
+                            "fields": {
+                                "keyword": {
+                                    "type": "keyword",
+                                    "ignore_above": 256,
+                                    "normalizer": "to_lower"
+                                }
+                            }
+                        },
+                        "uploader": {
+                            "type": "text",
+                            "analyzer": "english",
+                            "fields": {
+                                "keyword": {
+                                    "type": "keyword",
+                                    "ignore_above": 256,
+                                    "normalizer": "to_lower"
+                                }
+                            }
+                        },
+                        "youtube_id": {
+                            "type": "keyword"
+                        }
+                    }
                 }
             },
             "expected_set": {
@@ -504,7 +569,7 @@
                     "format": "epoch_second"
                 },
                 "subtitle_index": {
-                    "type" : "long"
+                    "type": "long"
                 },
                 "subtitle_lang": {
                     "type": "keyword"
@@ -513,7 +578,7 @@
                     "type": "keyword"
                 },
                 "subtitle_line": {
-                    "type" : "text",
+                    "type": "text",
                     "analyzer": "english"
                 }
             },
@@ -548,14 +613,14 @@
                             "type": "keyword"
                         },
                         "comment_text": {
-                            "type" : "text"
+                            "type": "text"
                         },
                         "comment_timestamp": {
                             "type": "date",
                             "format": "epoch_second"
                         },
                         "comment_time_text": {
-                            "type" : "text"
+                            "type": "text"
                         },
                         "comment_likecount": {
                             "type": "long"

tubearchivist/home/src/es/snapshot.py

@@ -4,12 +4,12 @@ functionality:
 """
 
 from datetime import datetime
-from os import environ
 from time import sleep
 from zoneinfo import ZoneInfo
 
 from home.src.es.connect import ElasticWrap
 from home.src.ta.helper import get_mapping
+from home.src.ta.settings import EnvironmentSettings
 
 
 class ElasticSnapshot:
@@ -19,7 +19,7 @@ class ElasticSnapshot:
     REPO_SETTINGS = {
         "compress": "true",
         "chunk_size": "1g",
-        "location": "/usr/share/elasticsearch/data/snapshot",
+        "location": EnvironmentSettings.ES_SNAPSHOT_DIR,
     }
     POLICY = "ta_daily"
 
@@ -254,7 +254,7 @@ class ElasticSnapshot:
         expected_format = "%Y-%m-%dT%H:%M:%S.%fZ"
         date = datetime.strptime(date_utc, expected_format)
         local_datetime = date.replace(tzinfo=ZoneInfo("localtime"))
-        converted = local_datetime.astimezone(ZoneInfo(environ.get("TZ")))
+        converted = local_datetime.astimezone(ZoneInfo(EnvironmentSettings.TZ))
         converted_str = converted.strftime("%Y-%m-%d %H:%M")
 
         return converted_str

tubearchivist/home/src/frontend/api_calls.py

@@ -1,104 +0,0 @@
-"""
-Functionality:
-- collection of functions and tasks from frontend
-- called via user input
-"""
-
-from home.src.ta.ta_redis import RedisArchivist
-from home.tasks import run_restore_backup
-
-
-class PostData:
-    """
-    map frontend http post values to backend funcs
-    handover long running tasks to celery
-    """
-
-    def __init__(self, post_dict, current_user):
-        self.post_dict = post_dict
-        self.to_exec, self.exec_val = list(post_dict.items())[0]
-        self.current_user = current_user
-
-    def run_task(self):
-        """execute and return task result"""
-        to_exec = self.exec_map()
-        task_result = to_exec()
-        return task_result
-
-    def exec_map(self):
-        """map dict key and return function to execute"""
-        exec_map = {
-            "change_view": self._change_view,
-            "change_grid": self._change_grid,
-            "sort_order": self._sort_order,
-            "hide_watched": self._hide_watched,
-            "show_subed_only": self._show_subed_only,
-            "show_ignored_only": self._show_ignored_only,
-            "db-restore": self._db_restore,
-        }
-
-        return exec_map[self.to_exec]
-
-    def _change_view(self):
-        """process view changes in home, channel, and downloads"""
-        origin, new_view = self.exec_val.split(":")
-        key = f"{self.current_user}:view:{origin}"
-        print(f"change view: {key} to {new_view}")
-        RedisArchivist().set_message(key, {"status": new_view})
-        return {"success": True}
-
-    def _change_grid(self):
-        """process change items in grid"""
-        grid_items = int(self.exec_val)
-        grid_items = max(grid_items, 3)
-        grid_items = min(grid_items, 7)
-
-        key = f"{self.current_user}:grid_items"
-        print(f"change grid items: {grid_items}")
-        RedisArchivist().set_message(key, {"status": grid_items})
-        return {"success": True}
-
-    def _sort_order(self):
-        """change the sort between published to downloaded"""
-        sort_order = {"status": self.exec_val}
-        if self.exec_val in ["asc", "desc"]:
-            RedisArchivist().set_message(
-                f"{self.current_user}:sort_order", sort_order
-            )
-        else:
-            RedisArchivist().set_message(
-                f"{self.current_user}:sort_by", sort_order
-            )
-        return {"success": True}
-
-    def _hide_watched(self):
-        """toggle if to show watched vids or not"""
-        key = f"{self.current_user}:hide_watched"
-        message = {"status": bool(int(self.exec_val))}
-        print(f"toggle {key}: {message}")
-        RedisArchivist().set_message(key, message)
-        return {"success": True}
-
-    def _show_subed_only(self):
-        """show or hide subscribed channels only on channels page"""
-        key = f"{self.current_user}:show_subed_only"
-        message = {"status": bool(int(self.exec_val))}
-        print(f"toggle {key}: {message}")
-        RedisArchivist().set_message(key, message)
-        return {"success": True}
-
-    def _show_ignored_only(self):
-        """switch view on /downloads/ to show ignored only"""
-        show_value = self.exec_val
-        key = f"{self.current_user}:show_ignored_only"
-        value = {"status": show_value}
-        print(f"Filter download view ignored only: {show_value}")
-        RedisArchivist().set_message(key, value)
-        return {"success": True}
-
-    def _db_restore(self):
-        """restore es zip from settings page"""
-        print("restoring index from backup zip")
-        filename = self.exec_val
-        run_restore_backup.delay(filename)
-        return {"success": True}

tubearchivist/home/src/frontend/forms.py

@@ -2,9 +2,12 @@
 - hold all form classes used in the views
 """
 
+import os
+
 from django import forms
 from django.contrib.auth.forms import AuthenticationForm
 from django.forms.widgets import PasswordInput, TextInput
+from home.src.ta.helper import get_stylesheets
 
 
 class CustomAuthForm(AuthenticationForm):
@@ -29,14 +32,16 @@ class CustomAuthForm(AuthenticationForm):
 class UserSettingsForm(forms.Form):
     """user configurations values"""
 
-    CHOICES = [
+    STYLESHEET_CHOICES = [("", "-- change stylesheet --")]
-        ("", "-- change color scheme --"),
+    STYLESHEET_CHOICES.extend(
-        ("dark", "Dark"),
+        [
-        ("light", "Light"),
+            (stylesheet, os.path.splitext(stylesheet)[0].title())
+            for stylesheet in get_stylesheets()
         ]
+    )
 
-    colors = forms.ChoiceField(
+    stylesheet = forms.ChoiceField(
-        widget=forms.Select, choices=CHOICES, required=False
+        widget=forms.Select, choices=STYLESHEET_CHOICES, required=False
     )
     page_size = forms.IntegerField(required=False)
 

tubearchivist/home/src/frontend/searching.py

@@ -6,116 +6,19 @@ Functionality:
 - calculate pagination values
 """
 
-import urllib.parse
-from datetime import datetime
 
-from home.src.download.thumbnails import ThumbManager
+from api.src.search_processor import SearchProcess
 from home.src.es.connect import ElasticWrap
-from home.src.index.video_streams import DurationConverter
-from home.src.ta.config import AppConfig
-
-
-class SearchHandler:
-    """search elastic search"""
-
-    def __init__(self, path, config, data=False):
-        self.max_hits = None
-        self.aggs = None
-        self.path = path
-        self.config = config
-        self.data = data
-
-    def get_data(self):
-        """get the data"""
-        response, _ = ElasticWrap(self.path, config=self.config).get(self.data)
-
-        if "hits" in response.keys():
-            self.max_hits = response["hits"]["total"]["value"]
-            return_value = response["hits"]["hits"]
-        else:
-            # simulate list for single result to reuse rest of class
-            return_value = [response]
-
-        if not return_value:
-            return False
-
-        for idx, hit in enumerate(return_value):
-            return_value[idx] = self.hit_cleanup(hit)
-
-        if response.get("aggregations"):
-            self.aggs = response["aggregations"]
-            if "total_duration" in self.aggs:
-                duration_sec = self.aggs["total_duration"]["value"]
-                self.aggs["total_duration"].update(
-                    {"value_str": DurationConverter().get_str(duration_sec)}
-                )
-
-        return return_value
-
-    @staticmethod
-    def hit_cleanup(hit):
-        """clean up and parse data from a single hit"""
-        hit["source"] = hit.pop("_source")
-        hit_keys = hit["source"].keys()
-        if "media_url" in hit_keys:
-            parsed_url = urllib.parse.quote(hit["source"]["media_url"])
-            hit["source"]["media_url"] = parsed_url
-
-        if "published" in hit_keys:
-            published = hit["source"]["published"]
-            date_pub = datetime.strptime(published, "%Y-%m-%d")
-            date_str = datetime.strftime(date_pub, "%d %b, %Y")
-            hit["source"]["published"] = date_str
-
-        if "vid_last_refresh" in hit_keys:
-            vid_last_refresh = hit["source"]["vid_last_refresh"]
-            date_refresh = datetime.fromtimestamp(vid_last_refresh)
-            date_str = datetime.strftime(date_refresh, "%d %b, %Y")
-            hit["source"]["vid_last_refresh"] = date_str
-
-        if "playlist_last_refresh" in hit_keys:
-            playlist_last_refresh = hit["source"]["playlist_last_refresh"]
-            date_refresh = datetime.fromtimestamp(playlist_last_refresh)
-            date_str = datetime.strftime(date_refresh, "%d %b, %Y")
-            hit["source"]["playlist_last_refresh"] = date_str
-
-        if "vid_thumb_url" in hit_keys:
-            youtube_id = hit["source"]["youtube_id"]
-            thumb_path = ThumbManager(youtube_id).vid_thumb_path()
-            hit["source"]["vid_thumb_url"] = f"/cache/{thumb_path}"
-
-        if "channel_last_refresh" in hit_keys:
-            refreshed = hit["source"]["channel_last_refresh"]
-            date_refresh = datetime.fromtimestamp(refreshed)
-            date_str = datetime.strftime(date_refresh, "%d %b, %Y")
-            hit["source"]["channel_last_refresh"] = date_str
-
-        if "channel" in hit_keys:
-            channel_keys = hit["source"]["channel"].keys()
-            if "channel_last_refresh" in channel_keys:
-                refreshed = hit["source"]["channel"]["channel_last_refresh"]
-                date_refresh = datetime.fromtimestamp(refreshed)
-                date_str = datetime.strftime(date_refresh, "%d %b, %Y")
-                hit["source"]["channel"]["channel_last_refresh"] = date_str
-
-        if "subtitle_fragment_id" in hit_keys:
-            youtube_id = hit["source"]["youtube_id"]
-            thumb_path = ThumbManager(youtube_id).vid_thumb_path()
-            hit["source"]["vid_thumb_url"] = f"/cache/{thumb_path}"
-
-        return hit
 
 
 class SearchForm:
     """build query from search form data"""
 
-    CONFIG = AppConfig().config
-
     def multi_search(self, search_query):
         """searching through index"""
         path, query, query_type = SearchParser(search_query).run()
-        look_up = SearchHandler(path, config=self.CONFIG, data=query)
+        response, _ = ElasticWrap(path).get(data=query)
-        search_results = look_up.get_data()
+        search_results = SearchProcess(response).process()
         all_results = self.build_results(search_results)
 
         return {"results": all_results, "queryType": query_type}
@@ -465,7 +368,6 @@ class QueryBuilder:
 
         query = {
             "size": 30,
-            "_source": {"excludes": "subtitle_line"},
             "query": {"bool": {"must": must_list}},
             "highlight": {
                 "fields": {

tubearchivist/home/src/index/channel.py

@@ -14,6 +14,7 @@ from home.src.download.yt_dlp_base import YtWrap
 from home.src.es.connect import ElasticWrap, IndexPaginate
 from home.src.index.generic import YouTubeItem
 from home.src.index.playlist import YoutubePlaylist
+from home.src.ta.settings import EnvironmentSettings
 
 
 class YoutubeChannel(YouTubeItem):
@@ -22,20 +23,13 @@ class YoutubeChannel(YouTubeItem):
     es_path = False
     index_name = "ta_channel"
     yt_base = "https://www.youtube.com/channel/"
-    yt_obs = {
+    yt_obs = {"playlist_items": "0,0"}
-        "extract_flat": True,
-        "allow_playlist_files": True,
-    }
 
     def __init__(self, youtube_id, task=False):
         super().__init__(youtube_id)
         self.all_playlists = False
         self.task = task
 
-    def build_yt_url(self):
-        """overwrite base to use channel about page"""
-        return f"{self.yt_base}{self.youtube_id}/about"
-
     def build_json(self, upload=False, fallback=False):
         """get from es or from youtube"""
         self.get_from_es()
@@ -68,7 +62,7 @@ class YoutubeChannel(YouTubeItem):
             "channel_banner_url": self._get_banner_art(),
             "channel_thumb_url": self._get_thumb_art(),
             "channel_tvart_url": self._get_tv_art(),
-            "channel_views": self.youtube_meta.get("view_count", 0),
+            "channel_views": self.youtube_meta.get("view_count") or 0,
         }
 
     def _parse_tags(self, tags):
@@ -134,7 +128,7 @@ class YoutubeChannel(YouTubeItem):
     def _info_json_fallback(self):
         """read channel info.json for additional metadata"""
         info_json = os.path.join(
-            self.config["application"]["cache_dir"],
+            EnvironmentSettings.CACHE_DIR,
             "import",
             f"{self.youtube_id}.info.json",
         )
@@ -178,7 +172,7 @@ class YoutubeChannel(YouTubeItem):
     def get_folder_path(self):
         """get folder where media files get stored"""
         folder_path = os.path.join(
-            self.app_conf["videos"],
+            EnvironmentSettings.MEDIA_DIR,
             self.json_data["channel_id"],
         )
         return folder_path

tubearchivist/home/src/index/comments.py

@@ -77,7 +77,7 @@ class Comments:
     def get_yt_comments(self):
         """get comments from youtube"""
         yt_obs = self.build_yt_obs()
-        info_json = YtWrap(yt_obs).extract(self.youtube_id)
+        info_json = YtWrap(yt_obs, config=self.config).extract(self.youtube_id)
         if not info_json:
             return False, False
 

tubearchivist/home/src/index/filesystem.py

@@ -8,14 +8,14 @@ import os
 from home.src.es.connect import ElasticWrap, IndexPaginate
 from home.src.index.comments import CommentList
 from home.src.index.video import YoutubeVideo, index_new_video
-from home.src.ta.config import AppConfig
 from home.src.ta.helper import ignore_filelist
+from home.src.ta.settings import EnvironmentSettings
 
 
 class Scanner:
     """scan index and filesystem"""
 
-    VIDEOS: str = AppConfig().config["application"]["videos"]
+    VIDEOS: str = EnvironmentSettings.MEDIA_DIR
 
     def __init__(self, task=False) -> None:
         self.task = task
@@ -83,7 +83,7 @@ class Scanner:
             if self.task:
                 self.task.send_progress(
                     message_lines=[
-                        f"Index missing video {youtube_id}, {idx}/{total}"
+                        f"Index missing video {youtube_id}, {idx + 1}/{total}"
                     ],
                     progress=(idx + 1) / total,
                 )

tubearchivist/home/src/index/generic.py

@@ -8,7 +8,7 @@ import math
 from home.src.download.yt_dlp_base import YtWrap
 from home.src.es.connect import ElasticWrap
 from home.src.ta.config import AppConfig
-from home.src.ta.ta_redis import RedisArchivist
+from home.src.ta.users import UserConfig
 
 
 class YouTubeItem:
@@ -26,7 +26,6 @@ class YouTubeItem:
         self.youtube_id = youtube_id
         self.es_path = f"{self.index_name}/_doc/{youtube_id}"
         self.config = AppConfig().config
-        self.app_conf = self.config["application"]
         self.youtube_meta = False
         self.json_data = False
 
@@ -100,13 +99,7 @@ class Pagination:
 
     def get_page_size(self):
         """get default or user modified page_size"""
-        key = f"{self.request.user.id}:page_size"
+        return UserConfig(self.request.user.id).get_value("page_size")
-        page_size = RedisArchivist().get_message(key)["status"]
-        if not page_size:
-            config = AppConfig().config
-            page_size = config["archive"]["page_size"]
-
-        return page_size
 
     def first_guess(self):
         """build first guess before api call"""

tubearchivist/home/src/index/manual.py

@@ -16,6 +16,7 @@ from home.src.index.comments import CommentList
 from home.src.index.video import YoutubeVideo
 from home.src.ta.config import AppConfig
 from home.src.ta.helper import ignore_filelist
+from home.src.ta.settings import EnvironmentSettings
 from PIL import Image
 from yt_dlp.utils import ISO639Utils
 
@@ -28,7 +29,7 @@ class ImportFolderScanner:
     """
 
     CONFIG = AppConfig().config
-    CACHE_DIR = CONFIG["application"]["cache_dir"]
+    CACHE_DIR = EnvironmentSettings.CACHE_DIR
     IMPORT_DIR = os.path.join(CACHE_DIR, "import")
 
     """All extensions should be in lowercase until better handling is in place.
@@ -433,9 +434,9 @@ class ManualImport:
 
     def _move_to_archive(self, json_data):
         """move identified media file to archive"""
-        videos = self.config["application"]["videos"]
+        videos = EnvironmentSettings.MEDIA_DIR
-        host_uid = self.config["application"]["HOST_UID"]
+        host_uid = EnvironmentSettings.HOST_UID
-        host_gid = self.config["application"]["HOST_GID"]
+        host_gid = EnvironmentSettings.HOST_GID
 
         channel, file = os.path.split(json_data["media_url"])
         channel_folder = os.path.join(videos, channel)
@@ -472,7 +473,7 @@ class ManualImport:
                 os.remove(subtitle_file)
 
         channel_info = os.path.join(
-            self.config["application"]["cache_dir"],
+            EnvironmentSettings.CACHE_DIR,
             "import",
             f"{json_data['channel']['channel_id']}.info.json",
         )

tubearchivist/home/src/index/reindex.py

@@ -19,6 +19,7 @@ from home.src.index.comments import Comments
 from home.src.index.playlist import YoutubePlaylist
 from home.src.index.video import YoutubeVideo
 from home.src.ta.config import AppConfig
+from home.src.ta.settings import EnvironmentSettings
 from home.src.ta.ta_redis import RedisQueue
 
 
@@ -104,11 +105,13 @@ class ReindexPopulate(ReindexBase):
         """get total hits from index"""
         index_name = reindex_config["index_name"]
         active_key = reindex_config["active_key"]
-        path = f"{index_name}/_search?filter_path=hits.total"
+        data = {
-        data = {"query": {"match": {active_key: True}}}
+            "query": {"term": {active_key: {"value": True}}},
-        response, _ = ElasticWrap(path).post(data=data)
+            "_source": False,
-        total_hits = response["hits"]["total"]["value"]
+        }
-        return total_hits
+        total = IndexPaginate(index_name, data, keep_source=True).get_results()
+
+        return len(total)
 
     def _get_daily_should(self, total_hits):
         """calc how many should reindex daily"""
@@ -122,7 +125,7 @@ class ReindexPopulate(ReindexBase):
         """get outdated from index_name"""
         index_name = reindex_config["index_name"]
         refresh_key = reindex_config["refresh_key"]
-        now_lte = self.now - self.interval * 24 * 60 * 60
+        now_lte = str(self.now - self.interval * 24 * 60 * 60)
         must_list = [
             {"match": {reindex_config["active_key"]: True}},
             {"range": {refresh_key: {"lte": now_lte}}},
@@ -293,7 +296,7 @@ class Reindex(ReindexBase):
 
         # get new
         media_url = os.path.join(
-            self.config["application"]["videos"], es_meta["media_url"]
+            EnvironmentSettings.MEDIA_DIR, es_meta["media_url"]
         )
         video.build_json(media_path=media_url)
         if not video.youtube_meta:
@@ -311,10 +314,6 @@ class Reindex(ReindexBase):
             video.json_data["playlist"] = es_meta.get("playlist")
 
         video.upload_to_es()
-        if es_meta.get("media_url") != video.json_data["media_url"]:
-            self._rename_media_file(
-                es_meta.get("media_url"), video.json_data["media_url"]
-            )
 
         thumb_handler = ThumbManager(youtube_id)
         thumb_handler.delete_video_thumb()
@@ -325,14 +324,6 @@ class Reindex(ReindexBase):
 
         return
 
-    def _rename_media_file(self, media_url_is, media_url_should):
-        """handle title change"""
-        print(f"[reindex] fix media_url {media_url_is} to {media_url_should}")
-        videos = self.config["application"]["videos"]
-        old_path = os.path.join(videos, media_url_is)
-        new_path = os.path.join(videos, media_url_should)
-        os.rename(old_path, new_path)
-
     def _reindex_single_channel(self, channel_id):
         """refresh channel data and sync to videos"""
         # read current state

tubearchivist/home/src/index/subtitle.py

@@ -12,6 +12,7 @@ from datetime import datetime
 import requests
 from home.src.es.connect import ElasticWrap
 from home.src.ta.helper import requests_headers
+from home.src.ta.settings import EnvironmentSettings
 
 
 class YoutubeSubtitle:
@@ -113,7 +114,7 @@ class YoutubeSubtitle:
 
     def download_subtitles(self, relevant_subtitles):
         """download subtitle files to archive"""
-        videos_base = self.video.config["application"]["videos"]
+        videos_base = EnvironmentSettings.MEDIA_DIR
         indexed = []
         for subtitle in relevant_subtitles:
             dest_path = os.path.join(videos_base, subtitle["media_url"])
@@ -149,8 +150,8 @@ class YoutubeSubtitle:
         with open(dest_path, "w", encoding="utf-8") as subfile:
             subfile.write(subtitle_str)
 
-        host_uid = self.video.config["application"]["HOST_UID"]
+        host_uid = EnvironmentSettings.HOST_UID
-        host_gid = self.video.config["application"]["HOST_GID"]
+        host_gid = EnvironmentSettings.HOST_GID
         if host_uid and host_gid:
             os.chown(dest_path, host_uid, host_gid)
 
@@ -162,7 +163,7 @@ class YoutubeSubtitle:
     def delete(self, subtitles=False):
         """delete subtitles from index and filesystem"""
         youtube_id = self.video.youtube_id
-        videos_base = self.video.config["application"]["videos"]
+        videos_base = EnvironmentSettings.MEDIA_DIR
         # delete files
         if subtitles:
             files = [i["media_url"] for i in subtitles]

tubearchivist/home/src/index/video.py

@@ -16,12 +16,10 @@ from home.src.index import playlist as ta_playlist
 from home.src.index.generic import YouTubeItem
 from home.src.index.subtitle import YoutubeSubtitle
 from home.src.index.video_constants import VideoTypeEnum
-from home.src.index.video_streams import (
+from home.src.index.video_streams import MediaStreamExtractor
-    DurationConverter,
+from home.src.ta.helper import get_duration_sec, get_duration_str, randomizor
-    MediaStreamExtractor,
+from home.src.ta.settings import EnvironmentSettings
-)
+from home.src.ta.users import UserConfig
-from home.src.ta.helper import randomizor
-from home.src.ta.ta_redis import RedisArchivist
 from ryd_client import ryd_client
 
 
@@ -35,17 +33,16 @@ class SponsorBlock:
         self.user_agent = f"{settings.TA_UPSTREAM} {settings.TA_VERSION}"
         self.last_refresh = int(datetime.now().timestamp())
 
-    def get_sb_id(self):
+    def get_sb_id(self) -> str:
-        """get sponsorblock userid or generate if needed"""
+        """get sponsorblock for the userid or generate if needed"""
         if not self.user_id:
-            print("missing request user id")
+            raise ValueError("missing request user id")
-            raise ValueError
 
-        key = f"{self.user_id}:id_sponsorblock"
+        user = UserConfig(self.user_id)
-        sb_id = RedisArchivist().get_message(key)
+        sb_id = user.get_value("sponsorblock_id")
-        if not sb_id["status"]:
+        if not sb_id:
-            sb_id = {"status": randomizor(32)}
+            sb_id = randomizor(32)
-            RedisArchivist().set_message(key, sb_id)
+            user.set_value("sponsorblock_id", sb_id)
 
         return sb_id
 
@@ -91,7 +88,7 @@ class SponsorBlock:
 
     def post_timestamps(self, youtube_id, start_time, end_time):
         """post timestamps to api"""
-        user_id = self.get_sb_id().get("status")
+        user_id = self.get_sb_id()
         data = {
             "videoID": youtube_id,
             "startTime": start_time,
@@ -108,7 +105,7 @@ class SponsorBlock:
 
     def vote_on_segment(self, uuid, vote):
         """send vote on existing segment"""
-        user_id = self.get_sb_id().get("status")
+        user_id = self.get_sb_id()
         data = {
             "UUID": uuid,
             "userID": user_id,
@@ -191,11 +188,11 @@ class YoutubeVideo(YouTubeItem, YoutubeSubtitle):
         # build json_data basics
         self.json_data = {
             "title": self.youtube_meta["title"],
-            "description": self.youtube_meta["description"],
+            "description": self.youtube_meta.get("description", ""),
-            "category": self.youtube_meta["categories"],
+            "category": self.youtube_meta.get("categories", []),
             "vid_thumb_url": self.youtube_meta["thumbnail"],
             "vid_thumb_base64": base64_blur,
-            "tags": self.youtube_meta["tags"],
+            "tags": self.youtube_meta.get("tags", []),
             "published": published,
             "vid_last_refresh": last_refresh,
             "date_downloaded": last_refresh,
@@ -213,31 +210,24 @@ class YoutubeVideo(YouTubeItem, YoutubeSubtitle):
 
     def _add_stats(self):
         """add stats dicst to json_data"""
-        # likes
+        stats = {
-        like_count = self.youtube_meta.get("like_count", 0)
+            "view_count": self.youtube_meta.get("view_count", 0),
-        dislike_count = self.youtube_meta.get("dislike_count", 0)
+            "like_count": self.youtube_meta.get("like_count", 0),
-        average_rating = self.youtube_meta.get("average_rating", 0)
+            "dislike_count": self.youtube_meta.get("dislike_count", 0),
-        self.json_data.update(
+            "average_rating": self.youtube_meta.get("average_rating", 0),
-            {
-                "stats": {
-                    "view_count": self.youtube_meta["view_count"],
-                    "like_count": like_count,
-                    "dislike_count": dislike_count,
-                    "average_rating": average_rating,
         }
-            }
+        self.json_data.update({"stats": stats})
-        )
 
     def build_dl_cache_path(self):
         """find video path in dl cache"""
-        cache_dir = self.app_conf["cache_dir"]
+        cache_dir = EnvironmentSettings.CACHE_DIR
         video_id = self.json_data["youtube_id"]
         cache_path = f"{cache_dir}/download/{video_id}.mp4"
         if os.path.exists(cache_path):
             return cache_path
 
         channel_path = os.path.join(
-            self.app_conf["videos"],
+            EnvironmentSettings.MEDIA_DIR,
             self.json_data["channel"]["channel_id"],
             f"{video_id}.mp4",
         )
@@ -249,16 +239,14 @@ class YoutubeVideo(YouTubeItem, YoutubeSubtitle):
     def add_player(self, media_path=False):
         """add player information for new videos"""
         vid_path = media_path or self.build_dl_cache_path()
+        duration = get_duration_sec(vid_path)
 
-        duration_handler = DurationConverter()
-        duration = duration_handler.get_sec(vid_path)
-        duration_str = duration_handler.get_str(duration)
         self.json_data.update(
             {
                 "player": {
                     "watched": False,
                     "duration": duration,
-                    "duration_str": duration_str,
+                    "duration_str": get_duration_str(duration),
                 }
             }
         )
@@ -288,7 +276,7 @@ class YoutubeVideo(YouTubeItem, YoutubeSubtitle):
         if not self.json_data:
             raise FileNotFoundError
 
-        video_base = self.app_conf["videos"]
+        video_base = EnvironmentSettings.MEDIA_DIR
         media_url = self.json_data.get("media_url")
         file_path = os.path.join(video_base, media_url)
         try:

tubearchivist/home/src/index/video_streams.py

@@ -5,60 +5,6 @@ import subprocess
 from os import stat
 
 
-class DurationConverter:
-    """
-    using ffmpeg to get and parse duration from filepath
-    """
-
-    @staticmethod
-    def get_sec(file_path):
-        """read duration from file"""
-        duration = subprocess.run(
-            [
-                "ffprobe",
-                "-v",
-                "error",
-                "-show_entries",
-                "format=duration",
-                "-of",
-                "default=noprint_wrappers=1:nokey=1",
-                file_path,
-            ],
-            capture_output=True,
-            check=True,
-        )
-        duration_raw = duration.stdout.decode().strip()
-        if duration_raw == "N/A":
-            return 0
-
-        duration_sec = int(float(duration_raw))
-        return duration_sec
-
-    @staticmethod
-    def get_str(seconds):
-        """takes duration in sec and returns clean string"""
-        if not seconds:
-            # failed to extract
-            return "NA"
-
-        days = int(seconds // (24 * 3600))
-        hours = int((seconds % (24 * 3600)) // 3600)
-        minutes = int((seconds % 3600) // 60)
-        seconds = int(seconds % 60)
-
-        duration_str = str()
-        if days:
-            duration_str = f"{days}d "
-        if hours:
-            duration_str = duration_str + str(hours).zfill(2) + ":"
-        if minutes:
-            duration_str = duration_str + str(minutes).zfill(2) + ":"
-        else:
-            duration_str = duration_str + "00:"
-        duration_str = duration_str + str(seconds).zfill(2)
-        return duration_str
-
-
 class MediaStreamExtractor:
     """extract stream metadata"""
 

tubearchivist/home/src/ta/auth.py

@@ -0,0 +1,10 @@
+from django.conf import settings
+from django.contrib.auth.middleware import PersistentRemoteUserMiddleware
+
+
+class HttpRemoteUserMiddleware(PersistentRemoteUserMiddleware):
+    """This class allows authentication via HTTP_REMOTE_USER which is set for
+    example by certain SSO applications.
+    """
+
+    header = settings.TA_AUTH_PROXY_USERNAME_HEADER

tubearchivist/home/src/ta/config.py

@@ -5,7 +5,6 @@ Functionality:
 """
 
 import json
-import os
 import re
 from random import randint
 from time import sleep
@@ -17,12 +16,10 @@ from home.src.ta.ta_redis import RedisArchivist
 
 
 class AppConfig:
-    """handle user settings and application variables"""
+    """handle application variables"""
 
-    def __init__(self, user_id=False):
+    def __init__(self):
-        self.user_id = user_id
         self.config = self.get_config()
-        self.colors = self.get_colors()
 
     def get_config(self):
         """get config from default file or redis if changed"""
@@ -30,13 +27,6 @@ class AppConfig:
         if not config:
             config = self.get_config_file()
 
-        if self.user_id:
-            key = f"{self.user_id}:page_size"
-            page_size = RedisArchivist().get_message(key)["status"]
-            if page_size:
-                config["archive"]["page_size"] = page_size
-
-        config["application"].update(self.get_config_env())
         return config
 
     def get_config_file(self):
@@ -44,27 +34,8 @@ class AppConfig:
         with open("home/config.json", "r", encoding="utf-8") as f:
             config_file = json.load(f)
 
-        config_file["application"].update(self.get_config_env())
-
         return config_file
 
-    @staticmethod
-    def get_config_env():
-        """read environment application variables"""
-        es_pass = os.environ.get("ELASTIC_PASSWORD")
-        es_user = os.environ.get("ELASTIC_USER", default="elastic")
-
-        application = {
-            "REDIS_HOST": os.environ.get("REDIS_HOST"),
-            "es_url": os.environ.get("ES_URL"),
-            "es_auth": (es_user, es_pass),
-            "HOST_UID": int(os.environ.get("HOST_UID", False)),
-            "HOST_GID": int(os.environ.get("HOST_GID", False)),
-            "enable_cast": bool(os.environ.get("ENABLE_CAST")),
-        }
-
-        return application
-
     @staticmethod
     def get_config_redis():
         """read config json set from redis to overwrite defaults"""
@@ -103,30 +74,6 @@ class AppConfig:
         RedisArchivist().set_message("config", self.config, save=True)
         return updated
 
-    @staticmethod
-    def set_user_config(form_post, user_id):
-        """set values in redis for user settings"""
-        for key, value in form_post.items():
-            if not value:
-                continue
-
-            message = {"status": value}
-            redis_key = f"{user_id}:{key}"
-            RedisArchivist().set_message(redis_key, message, save=True)
-
-    def get_colors(self):
-        """overwrite config if user has set custom values"""
-        colors = False
-        if self.user_id:
-            col_dict = RedisArchivist().get_message(f"{self.user_id}:colors")
-            colors = col_dict["status"]
-
-        if not colors:
-            colors = self.config["application"]["colors"]
-
-        self.config["application"]["colors"] = colors
-        return colors
-
     @staticmethod
     def _build_rand_daily():
         """build random daily schedule per installation"""

tubearchivist/home/src/ta/health.py

@@ -0,0 +1,11 @@
+from django.http import HttpResponse
+
+
+class HealthCheckMiddleware:
+    def __init__(self, get_response):
+        self.get_response = get_response
+
+    def __call__(self, request):
+        if request.path == "/health":
+            return HttpResponse("ok")
+        return self.get_response(request)

tubearchivist/home/src/ta/helper.py

@@ -7,10 +7,12 @@ import json
 import os
 import random
 import string
+import subprocess
 from datetime import datetime
 from urllib.parse import urlparse
 
 import requests
+from home.src.ta.settings import EnvironmentSettings
 
 
 def ignore_filelist(filelist: list[str]) -> list[str]:
@@ -111,13 +113,13 @@ def time_parser(timestamp: str) -> float:
     return int(hours) * 60 * 60 + int(minutes) * 60 + float(seconds)
 
 
-def clear_dl_cache(config: dict) -> int:
+def clear_dl_cache(cache_dir: str) -> int:
     """clear leftover files from dl cache"""
     print("clear download cache")
-    cache_dir = os.path.join(config["application"]["cache_dir"], "download")
+    download_cache_dir = os.path.join(cache_dir, "download")
-    leftover_files = ignore_filelist(os.listdir(cache_dir))
+    leftover_files = ignore_filelist(os.listdir(download_cache_dir))
     for cached in leftover_files:
-        to_delete = os.path.join(cache_dir, cached)
+        to_delete = os.path.join(download_cache_dir, cached)
         os.remove(to_delete)
 
     return len(leftover_files)
@@ -141,6 +143,47 @@ def is_shorts(youtube_id: str) -> bool:
     return response.status_code == 200
 
 
+def get_duration_sec(file_path: str) -> int:
+    """get duration of media file from file path"""
+
+    duration = subprocess.run(
+        [
+            "ffprobe",
+            "-v",
+            "error",
+            "-show_entries",
+            "format=duration",
+            "-of",
+            "default=noprint_wrappers=1:nokey=1",
+            file_path,
+        ],
+        capture_output=True,
+        check=True,
+    )
+    duration_raw = duration.stdout.decode().strip()
+    if duration_raw == "N/A":
+        return 0
+
+    duration_sec = int(float(duration_raw))
+    return duration_sec
+
+
+def get_duration_str(seconds: int) -> str:
+    """Return a human-readable duration string from seconds."""
+    if not seconds:
+        return "NA"
+
+    units = [("y", 31536000), ("d", 86400), ("h", 3600), ("m", 60), ("s", 1)]
+    duration_parts = []
+
+    for unit_label, unit_seconds in units:
+        if seconds >= unit_seconds:
+            unit_count, seconds = divmod(seconds, unit_seconds)
+            duration_parts.append(f"{unit_count:02}{unit_label}")
+
+    return " ".join(duration_parts)
+
+
 def ta_host_parser(ta_host: str) -> tuple[list[str], list[str]]:
     """parse ta_host env var for ALLOWED_HOSTS and CSRF_TRUSTED_ORIGINS"""
     allowed_hosts: list[str] = [
@@ -161,3 +204,21 @@ def ta_host_parser(ta_host: str) -> tuple[list[str], list[str]]:
         csrf_trusted_origins.append(f"{parsed.scheme}://{parsed.hostname}")
 
     return allowed_hosts, csrf_trusted_origins
+
+
+def get_stylesheets():
+    """Get all valid stylesheets from /static/css"""
+    app_root = EnvironmentSettings.APP_DIR
+    stylesheets = os.listdir(os.path.join(app_root, "static/css"))
+    stylesheets.remove("style.css")
+    stylesheets.sort()
+    stylesheets = list(filter(lambda x: x.endswith(".css"), stylesheets))
+    return stylesheets
+
+
+def check_stylesheet(stylesheet: str):
+    """Check if a stylesheet exists. Return dark.css as a fallback"""
+    if stylesheet in get_stylesheets():
+        return stylesheet
+
+    return "dark.css"

tubearchivist/home/src/ta/settings.py

@@ -0,0 +1,95 @@
+"""
+Functionality:
+- read and write application config backed by ES
+- encapsulate persistence of application properties
+"""
+
+from os import environ
+
+
+class EnvironmentSettings:
+    """
+    Handle settings for the application that are driven from the environment.
+    These will not change when the user is using the application.
+    These settings are only provided only on startup.
+    """
+
+    HOST_UID: int = int(environ.get("HOST_UID", False))
+    HOST_GID: int = int(environ.get("HOST_GID", False))
+    ENABLE_CAST: bool = bool(environ.get("ENABLE_CAST"))
+    TZ: str = str(environ.get("TZ", "UTC"))
+    TA_PORT: int = int(environ.get("TA_PORT", False))
+    TA_UWSGI_PORT: int = int(environ.get("TA_UWSGI_PORT", False))
+    TA_USERNAME: str = str(environ.get("TA_USERNAME"))
+    TA_PASSWORD: str = str(environ.get("TA_PASSWORD"))
+
+    # Application Paths
+    MEDIA_DIR: str = str(environ.get("TA_MEDIA_DIR", "/youtube"))
+    APP_DIR: str = str(environ.get("TA_APP_DIR", "/app"))
+    CACHE_DIR: str = str(environ.get("TA_CACHE_DIR", "/cache"))
+
+    # Redis
+    REDIS_HOST: str = str(environ.get("REDIS_HOST"))
+    REDIS_PORT: int = int(environ.get("REDIS_PORT", 6379))
+    REDIS_NAME_SPACE: str = str(environ.get("REDIS_NAME_SPACE", "ta:"))
+
+    # ElasticSearch
+    ES_URL: str = str(environ.get("ES_URL"))
+    ES_PASS: str = str(environ.get("ELASTIC_PASSWORD"))
+    ES_USER: str = str(environ.get("ELASTIC_USER", "elastic"))
+    ES_SNAPSHOT_DIR: str = str(
+        environ.get(
+            "ES_SNAPSHOT_DIR", "/usr/share/elasticsearch/data/snapshot"
+        )
+    )
+    ES_DISABLE_VERIFY_SSL: bool = bool(environ.get("ES_DISABLE_VERIFY_SSL"))
+
+    def print_generic(self):
+        """print generic env vars"""
+        print(
+            f"""
+            HOST_UID: {self.HOST_UID}
+            HOST_GID: {self.HOST_GID}
+            TZ: {self.TZ}
+            ENABLE_CAST: {self.ENABLE_CAST}
+            TA_PORT: {self.TA_PORT}
+            TA_UWSGI_PORT: {self.TA_UWSGI_PORT}
+            TA_USERNAME: {self.TA_USERNAME}
+            TA_PASSWORD: *****"""
+        )
+
+    def print_paths(self):
+        """debug paths set"""
+        print(
+            f"""
+            MEDIA_DIR: {self.MEDIA_DIR}
+            APP_DIR: {self.APP_DIR}
+            CACHE_DIR: {self.CACHE_DIR}"""
+        )
+
+    def print_redis_conf(self):
+        """debug redis conf paths"""
+        print(
+            f"""
+            REDIS_HOST: {self.REDIS_HOST}
+            REDIS_PORT: {self.REDIS_PORT}
+            REDIS_NAME_SPACE: {self.REDIS_NAME_SPACE}"""
+        )
+
+    def print_es_paths(self):
+        """debug es conf"""
+        print(
+            f"""
+            ES_URL: {self.ES_URL}
+            ES_PASS: *****
+            ES_USER: {self.ES_USER}
+            ES_SNAPSHOT_DIR: {self.ES_SNAPSHOT_DIR}
+            ES_DISABLE_VERIFY_SSL: {self.ES_DISABLE_VERIFY_SSL}"""
+        )
+
+    def print_all(self):
+        """print all"""
+        self.print_generic()
+        self.print_paths()
+        self.print_redis_conf()
+        self.print_es_paths()

tubearchivist/home/src/ta/ta_redis.py

@@ -6,20 +6,21 @@ functionality:
 """
 
 import json
-import os
 
 import redis
+from home.src.ta.settings import EnvironmentSettings
 
 
 class RedisBase:
     """connection base for redis"""
 
-    REDIS_HOST: str = str(os.environ.get("REDIS_HOST"))
+    NAME_SPACE: str = EnvironmentSettings.REDIS_NAME_SPACE
-    REDIS_PORT: int = int(os.environ.get("REDIS_PORT") or 6379)
-    NAME_SPACE: str = "ta:"
 
     def __init__(self):
-        self.conn = redis.Redis(host=self.REDIS_HOST, port=self.REDIS_PORT)
+        self.conn = redis.Redis(
+            host=EnvironmentSettings.REDIS_HOST,
+            port=EnvironmentSettings.REDIS_PORT,
+        )
 
 
 class RedisArchivist(RedisBase):

tubearchivist/home/src/ta/users.py

@@ -0,0 +1,142 @@
+"""
+Functionality:
+- read and write user config backed by ES
+- encapsulate persistence of user properties
+"""
+
+from typing import TypedDict
+
+from home.src.es.connect import ElasticWrap
+from home.src.ta.helper import get_stylesheets
+
+
+class UserConfigType(TypedDict, total=False):
+    """describes the user configuration"""
+
+    stylesheet: str
+    page_size: int
+    sort_by: str
+    sort_order: str
+    view_style_home: str
+    view_style_channel: str
+    view_style_downloads: str
+    view_style_playlist: str
+    grid_items: int
+    hide_watched: bool
+    show_ignored_only: bool
+    show_subed_only: bool
+    sponsorblock_id: str
+
+
+class UserConfig:
+    """Handle settings for an individual user"""
+
+    _DEFAULT_USER_SETTINGS = UserConfigType(
+        stylesheet="dark.css",
+        page_size=12,
+        sort_by="published",
+        sort_order="desc",
+        view_style_home="grid",
+        view_style_channel="list",
+        view_style_downloads="list",
+        view_style_playlist="grid",
+        grid_items=3,
+        hide_watched=False,
+        show_ignored_only=False,
+        show_subed_only=False,
+        sponsorblock_id=None,
+    )
+
+    VALID_STYLESHEETS = get_stylesheets()
+    VALID_VIEW_STYLE = ["grid", "list"]
+    VALID_SORT_ORDER = ["asc", "desc"]
+    VALID_SORT_BY = [
+        "published",
+        "downloaded",
+        "views",
+        "likes",
+        "duration",
+        "filesize",
+    ]
+    VALID_GRID_ITEMS = range(3, 8)
+
+    def __init__(self, user_id: str):
+        self._user_id: str = user_id
+        self._config: UserConfigType = self.get_config()
+
+    def get_value(self, key: str):
+        """Get the given key from the users configuration
+
+        Throws a KeyError if the requested Key is not a permitted value"""
+        if key not in self._DEFAULT_USER_SETTINGS:
+            raise KeyError(f"Unable to read config for unknown key '{key}'")
+
+        return self._config.get(key) or self._DEFAULT_USER_SETTINGS.get(key)
+
+    def set_value(self, key: str, value: str | bool | int):
+        """Set or replace a configuration value for the user"""
+        self._validate(key, value)
+        old = self.get_value(key)
+        self._config[key] = value
+
+        # Upsert this property (creating a record if not exists)
+        es_payload = {"doc": {"config": {key: value}}, "doc_as_upsert": True}
+        es_document_path = f"ta_config/_update/user_{self._user_id}"
+        response, status = ElasticWrap(es_document_path).post(es_payload)
+        if status < 200 or status > 299:
+            raise ValueError(f"Failed storing user value {status}: {response}")
+
+        print(f"User {self._user_id} value '{key}' change: {old} -> {value}")
+
+    def _validate(self, key, value):
+        """validate key and value"""
+        if not self._user_id:
+            raise ValueError("Unable to persist config for null user_id")
+
+        if key not in self._DEFAULT_USER_SETTINGS:
+            raise KeyError(
+                f"Unable to persist config for an unknown key '{key}'"
+            )
+
+        valid_values = {
+            "stylesheet": self.VALID_STYLESHEETS,
+            "sort_by": self.VALID_SORT_BY,
+            "sort_order": self.VALID_SORT_ORDER,
+            "view_style_home": self.VALID_VIEW_STYLE,
+            "view_style_channel": self.VALID_VIEW_STYLE,
+            "view_style_download": self.VALID_VIEW_STYLE,
+            "view_style_playlist": self.VALID_VIEW_STYLE,
+            "grid_items": self.VALID_GRID_ITEMS,
+            "page_size": int,
+            "hide_watched": bool,
+            "show_ignored_only": bool,
+            "show_subed_only": bool,
+        }
+        validation_value = valid_values.get(key)
+
+        if isinstance(validation_value, (list, range)):
+            if value not in validation_value:
+                raise ValueError(f"Invalid value for {key}: {value}")
+        elif validation_value == int:
+            if not isinstance(value, int):
+                raise ValueError(f"Invalid value for {key}: {value}")
+        elif validation_value == bool:
+            if not isinstance(value, bool):
+                raise ValueError(f"Invalid value for {key}: {value}")
+
+    def get_config(self) -> UserConfigType:
+        """get config from ES or load from the application defaults"""
+        if not self._user_id:
+            # this is for a non logged-in user so use all the defaults
+            return {}
+
+        # Does this user have configuration stored in ES
+        es_document_path = f"ta_config/_doc/user_{self._user_id}"
+        response, status = ElasticWrap(es_document_path).get(print_error=False)
+        if status == 200 and "_source" in response.keys():
+            source = response.get("_source")
+            if "config" in source.keys():
+                return source.get("config")
+
+        # There is no config in ES
+        return {}

tubearchivist/home/tasks.py

@@ -24,13 +24,14 @@ from home.src.index.manual import ImportFolderScanner
 from home.src.index.reindex import Reindex, ReindexManual, ReindexPopulate
 from home.src.ta.config import AppConfig, ReleaseVersion, ScheduleBuilder
 from home.src.ta.notify import Notifications
+from home.src.ta.settings import EnvironmentSettings
 from home.src.ta.ta_redis import RedisArchivist
 from home.src.ta.task_manager import TaskManager
 from home.src.ta.urlparser import Parser
 
 CONFIG = AppConfig().config
-REDIS_HOST = os.environ.get("REDIS_HOST")
+REDIS_HOST = EnvironmentSettings.REDIS_HOST
-REDIS_PORT = os.environ.get("REDIS_PORT") or 6379
+REDIS_PORT = EnvironmentSettings.REDIS_PORT
 
 os.environ.setdefault("DJANGO_SETTINGS_MODULE", "config.settings")
 app = Celery(
@@ -39,9 +40,11 @@ app = Celery(
     backend=f"redis://{REDIS_HOST}:{REDIS_PORT}",
     result_extended=True,
 )
-app.config_from_object("django.conf:settings", namespace="ta:")
+app.config_from_object(
+    "django.conf:settings", namespace=EnvironmentSettings.REDIS_NAME_SPACE
+)
 app.autodiscover_tasks()
-app.conf.timezone = os.environ.get("TZ") or "UTC"
+app.conf.timezone = EnvironmentSettings.TZ
 
 
 class BaseTask(Task):
@@ -291,7 +294,7 @@ def run_restore_backup(self, filename):
     if manager.is_pending(self):
         print(f"[task][{self.name}] restore is already running")
         self.send_progress("Restore is already running.")
-        return
+        return None
 
     manager.init(self)
     self.send_progress(["Reset your Index"])
@@ -299,6 +302,8 @@ def run_restore_backup(self, filename):
     ElasticBackup(task=self).restore(filename)
     print("index restore finished")
 
+    return f"backup restore completed: {filename}"
+
 
 @shared_task(bind=True, name="rescan_filesystem", base=BaseTask)
 def rescan_filesystem(self):

tubearchivist/home/templates/home/base.html

@@ -1,4 +1,5 @@
 {% load static %}
+{% load auth_extras %}
 <!DOCTYPE html>
 <html lang="en">
 <head>
@@ -22,11 +23,7 @@
     {% else %}
         <title>TubeArchivist</title>
     {% endif %}
-    {% if colors == "dark" %}
+    <link rel="stylesheet" href="{% static 'css/' %}{{ stylesheet }}">
-        <link rel="stylesheet" href="{% static 'css/dark.css' %}">
-    {% else %}
-        <link rel="stylesheet" href="{% static 'css/light.css' %}">
-    {% endif %}
     <script type="text/javascript" src="{% static 'script.js' %}"></script>
     {% if cast %}
         <script type="text/javascript" src="https://www.gstatic.com/cv/js/sender/v1/cast_sender.js?loadCastFramework=1"></script>
@@ -38,12 +35,7 @@
         <div class="boxed-content">
             <div class="top-banner">
                 <a href="{% url 'home' %}">
-                    {% if colors == 'dark' %}
+                    <img alt="tube-archivist-banner">
-                        <img src="{% static 'img/banner-tube-archivist-dark.png' %}" alt="tube-archivist-banner">
-                    {% endif %}
-                    {% if colors == 'light' %}
-                        <img src="{% static 'img/banner-tube-archivist-light.png' %}" alt="tube-archivist-banner">
-                    {% endif %}
                 </a>
             </div>
             <div class="top-nav">
@@ -57,9 +49,11 @@
                     <a href="{% url 'playlist' %}">
                         <div class="nav-item">playlists</div>
                     </a>
+                    {% if request.user|has_group:"admin" or request.user.is_staff %}
                     <a href="{% url 'downloads' %}">
                         <div class="nav-item">downloads</div>
                     </a>
+                    {% endif %}
                 </div>
                 <div class="nav-icons">
                     <a href="{% url 'search' %}">

tubearchivist/home/templates/home/base_settings.html

@@ -1,14 +1,17 @@
 {# Base file for all of the settings pages to ensure a common menu #}
 {% extends "home/base.html" %}
 {% load static %}
+{% load auth_extras %}
 {% block content %}
 <div class="boxed-content">
     <div class="info-box-item child-page-nav">
         <a href="{% url 'settings' %}"><h3>Dashboard</h3></a>
         <a href="{% url 'settings_user' %}"><h3>User</h3></a>
+        {% if request.user|has_group:"admin" or request.user.is_staff %}
             <a href="{% url 'settings_application' %}"><h3>Application</h3></a>
             <a href="{% url 'settings_scheduling' %}"><h3>Scheduling</h3></a>
             <a href="{% url 'settings_actions' %}"><h3>Actions</h3></a>
+        {% endif %}
     </div>
     <div id="notifications" data=""></div>
     {% block settings_content %}{% endblock %}

tubearchivist/home/templates/home/channel.html

@@ -2,11 +2,13 @@
 {% load static %}
 {% load humanize %}
 {% block content %}
+{% load auth_extras %}
 <div class="boxed-content">
     <div class="title-split">
         <div class="title-bar">
             <h1>Channels</h1>
         </div>
+        {% if request.user|has_group:"admin" or request.user.is_staff %}
         <div class="title-split-form">
             <img id="animate-icon" onclick="showForm()" src="{% static 'img/icon-add.svg' %}" alt="add-icon" title="Subscribe to Channels">
             <div class="show-form">
@@ -17,6 +19,7 @@
                 </form>
             </div>
         </div>
+        {% endif %}
     </div>
     <div id="notifications" data="subscription"></div>
     <div class="view-controls">
@@ -42,33 +45,33 @@
             {% for channel in results %}
                 <div class="channel-item {{ view_style }}">
                     <div class="channel-banner {{ view_style }}">
-                        <a href="{% url 'channel_id' channel.source.channel_id %}">
+                        <a href="{% url 'channel_id' channel.channel_id %}">
-                            <img src="/cache/channels/{{ channel.source.channel_id }}_banner.jpg" alt="{{ channel.source.channel_id }}-banner">
+                            <img src="/cache/channels/{{ channel.channel_id }}_banner.jpg" alt="{{ channel.channel_id }}-banner">
                         </a>
                     </div>
                     <div class="info-box info-box-2 {{ view_style }}">
                         <div class="info-box-item">
                             <div class="round-img">
-                                <a href="{% url 'channel_id' channel.source.channel_id %}">
+                                <a href="{% url 'channel_id' channel.channel_id %}">
-                                    <img src="/cache/channels/{{ channel.source.channel_id }}_thumb.jpg" alt="channel-thumb">
+                                    <img src="/cache/channels/{{ channel.channel_id }}_thumb.jpg" alt="channel-thumb">
                                 </a>
                             </div>
                             <div>
-                                <h3><a href="{% url 'channel_id' channel.source.channel_id %}">{{ channel.source.channel_name }}</a></h3>
+                                <h3><a href="{% url 'channel_id' channel.channel_id %}">{{ channel.channel_name }}</a></h3>
-                                {% if channel.source.channel_subs >= 1000000 %}
+                                {% if channel.channel_subs >= 1000000 %}
-                                    <p>Subscribers: {{ channel.source.channel_subs|intword }}</p>
+                                    <p>Subscribers: {{ channel.channel_subs|intword }}</p>
                                 {% else %}
-                                    <p>Subscribers: {{ channel.source.channel_subs|intcomma }}</p>
+                                    <p>Subscribers: {{ channel.channel_subs|intcomma }}</p>
                                 {% endif %}
                             </div>
                         </div>
                         <div class="info-box-item">
                             <div>
-                                <p>Last refreshed: {{ channel.source.channel_last_refresh }}</p>
+                                <p>Last refreshed: {{ channel.channel_last_refresh }}</p>
-                                {% if channel.source.channel_subscribed %}
+                                {% if channel.channel_subscribed %}
-                                    <button class="unsubscribe" type="button" data-type="channel" data-subscribe="" data-id="{{ channel.source.channel_id }}" onclick="subscribeStatus(this)" title="Unsubscribe from {{ channel.source.channel_name }}">Unsubscribe</button>
+                                    <button class="unsubscribe" type="button" data-type="channel" data-subscribe="" data-id="{{ channel.channel_id }}" onclick="subscribeStatus(this)" title="Unsubscribe from {{ channel.channel_name }}">Unsubscribe</button>
                                 {% else %}
-                                    <button type="button" data-type="channel" data-subscribe="true" data-id="{{ channel.source.channel_id }}" onclick="subscribeStatus(this)" title="Subscribe to {{ channel.source.channel_name }}">Subscribe</button>
+                                    <button type="button" data-type="channel" data-subscribe="true" data-id="{{ channel.channel_id }}" onclick="subscribeStatus(this)" title="Subscribe to {{ channel.channel_name }}">Subscribe</button>
                                 {% endif %}
                             </div>
                         </div>

tubearchivist/home/templates/home/channel_id.html

@@ -2,6 +2,8 @@
 {% block content %}
 {% load static %}
 {% load humanize %}
+{% load auth_extras %}
+
 <div class="boxed-content">
     <div class="channel-banner">
         <a href="/channel/{{ channel_info.channel_id }}/"><img src="/cache/channels/{{ channel_info.channel_id }}_banner.jpg" alt="channel_banner"></a>
@@ -19,8 +21,10 @@
         {% endif %}
         <a href="{% url 'channel_id_about' channel_info.channel_id %}"><h3>About</h3></a>
         {% if has_pending %}
+            {% if request.user|has_group:"admin" or request.user.is_staff %}
                 <a href="{% url 'downloads' %}?channel={{ channel_info.channel_id }}"><h3>Downloads</h3></a>
             {% endif %}
+        {% endif %}
     </div>
     <div id="notifications" data="channel reindex"></div>
     <div class="info-box info-box-2">
@@ -38,7 +42,9 @@
                     <p>Subscribers: {{ channel_info.channel_subs|intcomma }}</p>
                 {% endif %}
                 {% if channel_info.channel_subscribed %}
+                    {% if request.user|has_group:"admin" or request.user.is_staff %}
                     <button class="unsubscribe" type="button" data-type="channel" data-subscribe="" data-id="{{ channel_info.channel_id }}" onclick="subscribeStatus(this)" title="Unsubscribe from {{ channel_info.channel_name }}">Unsubscribe</button>
+                    {% endif %}
                 {% else %}
                     <button type="button" data-type="channel" data-subscribe="true" data-id="{{ channel_info.channel_id }}" onclick="subscribeStatus(this)" title="Subscribe to {{ channel_info.channel_name }}">Subscribe</button>
                 {% endif %}
@@ -47,7 +53,10 @@
         <div class="info-box-item">
             {% if aggs %}
                 <p>{{ aggs.total_items.value }} videos <span class="space-carrot">|</span> {{ aggs.total_duration.value_str }} playback <span class="space-carrot">|</span> Total size {{ aggs.total_size.value|filesizeformat }}</p>
+                <div class="button-box">
                     <button title="Mark all videos from {{ channel_info.channel_name }} as watched" type="button" id="watched-button" data-id="{{ channel_info.channel_id }}" onclick="isWatchedButton(this)">Mark as watched</button>
+                    <button title="Mark all videos from {{ channel_info.channel_name }} as unwatched" type="button" id="unwatched-button" data-id="{{ channel_info.channel_id }}" onclick="isUnwatchedButton(this)">Mark as unwatched</button>
+                </div>
             {% endif %}
         </div>
     </div>
@@ -68,13 +77,15 @@
         <div class="sort">
             <div id="hidden-form">
                 <span>Sort by:</span>
-                <select name="sort" id="sort" onchange="sortChange(this.value)">
+                <select name="sort_by" id="sort" onchange="sortChange(this)">
                     <option value="published" {% if sort_by == "published" %}selected{% endif %}>date published</option>
                     <option value="downloaded" {% if sort_by == "downloaded" %}selected{% endif %}>date downloaded</option>
                     <option value="views" {% if sort_by == "views" %}selected{% endif %}>views</option>
                     <option value="likes" {% if sort_by == "likes" %}selected{% endif %}>likes</option>
+                    <option value="duration" {% if sort_by == "duration" %}selected{% endif %}>duration</option>
+                    <option value="filesize" {% if sort_by == "filesize" %}selected{% endif %}>file size</option>
                 </select>
-                <select name="sord-order" id="sort-order" onchange="sortChange(this.value)">
+                <select name="sort_order" id="sort-order" onchange="sortChange(this)">
                     <option value="asc" {% if sort_order == "asc" %}selected{% endif %}>asc</option>
                     <option value="desc" {% if sort_order == "desc" %}selected{% endif %}>desc</option>
                 </select>
@@ -103,14 +114,14 @@
         {% if results %}
             {% for video in results %}
                 <div class="video-item {{ view_style }}">
-                    <a href="#player" data-id="{{ video.source.youtube_id }}" onclick="createPlayer(this)">
+                    <a href="#player" data-id="{{ video.youtube_id }}" onclick="createPlayer(this)">
                         <div class="video-thumb-wrap {{ view_style }}">
                             <div class="video-thumb">
-                                <img src="{{ video.source.vid_thumb_url }}" alt="video-thumb">
+                                <img src="{{ video.vid_thumb_url }}" alt="video-thumb">
-                                {% if video.source.player.progress %}
+                                {% if video.player.progress %}
-                                    <div class="video-progress-bar" id="progress-{{ video.source.youtube_id }}" style="width: {{video.source.player.progress}}%;"></div>
+                                    <div class="video-progress-bar" id="progress-{{ video.youtube_id }}" style="width: {{video.player.progress}}%;"></div>
                                 {% else %}
-                                    <div class="video-progress-bar" id="progress-{{ video.source.youtube_id }}" style="width: 0%;"></div>
+                                    <div class="video-progress-bar" id="progress-{{ video.youtube_id }}" style="width: 0%;"></div>
                                 {% endif %}
                             </div>
                             <div class="video-play">
@@ -119,16 +130,16 @@
                         </div>
                     </a>
                     <div class="video-desc {{ view_style }}">
-                        <div class="video-desc-player" id="video-info-{{ video.source.youtube_id }}">
+                        <div class="video-desc-player" id="video-info-{{ video.youtube_id }}">
-                            {% if video.source.player.watched %}
+                            {% if video.player.watched %}
-                                <img src="{% static 'img/icon-seen.svg' %}" alt="seen-icon" data-id="{{ video.source.youtube_id }}" data-status="watched" onclick="updateVideoWatchStatus(this)" class="watch-button" title="Mark as unwatched">
+                                <img src="{% static 'img/icon-seen.svg' %}" alt="seen-icon" data-id="{{ video.youtube_id }}" data-status="watched" onclick="updateVideoWatchStatus(this)" class="watch-button" title="Mark as unwatched">
                             {% else %}
-                                <img src="{% static 'img/icon-unseen.svg' %}" alt="unseen-icon" data-id="{{ video.source.youtube_id }}" data-status="unwatched" onclick="updateVideoWatchStatus(this)" class="watch-button" title="Mark as watched">
+                                <img src="{% static 'img/icon-unseen.svg' %}" alt="unseen-icon" data-id="{{ video.youtube_id }}" data-status="unwatched" onclick="updateVideoWatchStatus(this)" class="watch-button" title="Mark as watched">
                             {% endif %}
-                            <span>{{ video.source.published }} | {{ video.source.player.duration_str }}</span>
+                            <span>{{ video.published }} | {{ video.player.duration_str }}</span>
                         </div>
                         <div>
-                            <a class="video-more" href="{% url 'video' video.source.youtube_id %}"><h2>{{ video.source.title }}</h2></a>
+                            <a class="video-more" href="{% url 'video' video.youtube_id %}"><h2>{{ video.title }}</h2></a>
                         </div>
                     </div>
                 </div>

tubearchivist/home/templates/home/channel_id_about.html

@@ -2,6 +2,7 @@
 {% block content %}
 {% load static %}
 {% load humanize %}
+{% load auth_extras %}
 <div class="boxed-content">
     <div class="channel-banner">
         <a href="{% url 'channel_id' channel_info.channel_id %}"><img src="{{ channel_info.channel_banner_url }}" alt="channel_banner"></a>
@@ -19,8 +20,10 @@
         {% endif %}
         <a href="{% url 'channel_id_about' channel_info.channel_id %}"><h3>About</h3></a>
         {% if has_pending %}
+            {% if request.user|has_group:"admin" or request.user.is_staff %}
                 <a href="{% url 'downloads' %}?channel={{ channel_info.channel_id }}"><h3>Downloads</h3></a>
             {% endif %}
+        {% endif %}
     </div>
     <div id="notifications" data="channel reindex"></div>
     <div class="info-box info-box-3">
@@ -56,6 +59,7 @@
                 {% elif channel_info.channel_views > 0 %}
                     <p>Channel views: {{ channel_info.channel_views|intcomma }}</p>
                 {% endif %}
+                {% if request.user|has_group:"admin" or request.user.is_staff %}
                     <div class="button-box">
                         <button onclick="deleteConfirm()" id="delete-item">Delete Channel</button>
                         <div class="delete-confirm" id="delete-button">
@@ -70,6 +74,7 @@
                             <button data-id="{{ channel_info.channel_id }}" data-type="channel" data-extract-videos="true" onclick="reindex(this)" title="Reindex Videos of {{ channel_info.channel_name }}">Reindex Videos</button>
                         </div>
                     {% endif %}
+                {% endif %}
             </div>
         </div>
     </div>
@@ -90,6 +95,7 @@
             </div>
         </div>
     {% endif %}
+    {% if request.user|has_group:"admin" or request.user.is_staff %}
         <div id="overwrite-form" class="info-box">
             <div class="info-box-item">
                 <h2>Customize {{ channel_info.channel_name }}</h2>
@@ -137,6 +143,7 @@
                 </form>
             </div>
         </div>
+    {% endif %}
 </div>
 <script type="text/javascript" src="{% static 'progress.js' %}"></script>
 {% endblock content %}

tubearchivist/home/templates/home/channel_id_playlist.html

@@ -2,6 +2,7 @@
 {% block content %}
 {% load static %}
 {% load humanize %}
+{% load auth_extras %}
 <div class="boxed-content">
     <div class="channel-banner">
         <a href="{% url 'channel_id' channel_info.channel_id %}"><img src="{{ channel_info.channel_banner_url }}" alt="channel_banner"></a>
@@ -19,8 +20,10 @@
         {% endif %}
         <a href="{% url 'channel_id_about' channel_info.channel_id %}"><h3>About</h3></a>
         {% if has_pending %}
+            {% if request.user|has_group:"admin" or request.user.is_staff %}
                 <a href="{% url 'downloads' %}?channel={{ channel_info.channel_id }}"><h3>Downloads</h3></a>
             {% endif %}
+        {% endif %}
     </div>
     <div id="notifications" data="channel reindex"></div>
     <div class="view-controls">
@@ -45,18 +48,18 @@
             {% for playlist in results %}
             <div class="playlist-item {{ view_style }}">
                 <div class="playlist-thumbnail">
-                    <a href="{% url 'playlist_id' playlist.source.playlist_id %}">
+                    <a href="{% url 'playlist_id' playlist.playlist_id %}">
-                        <img src="/cache/playlists/{{ playlist.source.playlist_id }}.jpg" alt="{{ playlist.source.playlist_id }}-thumbnail">
+                        <img src="/cache/playlists/{{ playlist.playlist_id }}.jpg" alt="{{ playlist.playlist_id }}-thumbnail">
                     </a>
                 </div>
                 <div class="playlist-desc {{ view_style }}">
-                    <a href="{% url 'channel_id' playlist.source.playlist_channel_id %}"><h3>{{ playlist.source.playlist_channel }}</h3></a>
+                    <a href="{% url 'channel_id' playlist.playlist_channel_id %}"><h3>{{ playlist.playlist_channel }}</h3></a>
-                    <a href="{% url 'playlist_id' playlist.source.playlist_id %}"><h2>{{ playlist.source.playlist_name }}</h2></a>
+                    <a href="{% url 'playlist_id' playlist.playlist_id %}"><h2>{{ playlist.playlist_name }}</h2></a>
-                    <p>Last refreshed: {{ playlist.source.playlist_last_refresh }}</p>
+                    <p>Last refreshed: {{ playlist.playlist_last_refresh }}</p>
-                    {% if playlist.source.playlist_subscribed %}
+                    {% if playlist.playlist_subscribed and request.user|has_group:"admin" or request.user.is_staff %}
-                        <button class="unsubscribe" type="button" data-type="playlist" data-subscribe="" data-id="{{ playlist.source.playlist_id }}" onclick="subscribeStatus(this)" title="Unsubscribe from {{ playlist.source.playlist_name }}">Unsubscribe</button>
+                        <button class="unsubscribe" type="button" data-type="playlist" data-subscribe="" data-id="{{ playlist.playlist_id }}" onclick="subscribeStatus(this)" title="Unsubscribe from {{ playlist.playlist_name }}">Unsubscribe</button>
                     {% else %}
-                        <button type="button" data-type="playlist" data-subscribe="true" data-id="{{ playlist.source.playlist_id }}" onclick="subscribeStatus(this)" title="Subscribe to {{ playlist.source.playlist_name }}">Subscribe</button>
+                        <button type="button" data-type="playlist" data-subscribe="true" data-id="{{ playlist.playlist_id }}" onclick="subscribeStatus(this)" title="Subscribe to {{ playlist.playlist_name }}">Subscribe</button>
                     {% endif %}
                 </div>
             </div>

tubearchivist/home/templates/home/downloads.html

@@ -70,18 +70,18 @@
     <div class="video-list {{ view_style }} {% if view_style == "grid" %}grid-{{ grid_items }}{% endif %}">
         {% if results %}
             {% for video in results %}
-                <div class="video-item {{ view_style }}" id="dl-{{ video.source.youtube_id }}">
+                <div class="video-item {{ view_style }}" id="dl-{{ video.youtube_id }}">
                     <div class="video-thumb-wrap {{ view_style }}">
                         <div class="video-thumb">
-                            <img src="{{ video.source.vid_thumb_url }}" alt="video_thumb">
+                            <img src="{{ video.vid_thumb_url }}" alt="video_thumb">
                             <div class="video-tags">
                                 {% if show_ignored_only %}
                                     <span>ignored</span>
                                 {% else %}
                                     <span>queued</span>
                                 {% endif %}
-                                <span>{{ video.source.vid_type }}</span>
+                                <span>{{ video.vid_type }}</span>
-                                {% if video.source.auto_start %}
+                                {% if video.auto_start %}
                                     <span>auto</span>
                                 {% endif %}
                             </div>
@@ -89,27 +89,27 @@
                     </div>
                     <div class="video-desc {{ view_style }}">
                         <div>
-                            {% if video.source.channel_indexed %}
+                            {% if video.channel_indexed %}
-                            <a href="{% url 'channel_id' video.source.channel_id %}">{{ video.source.channel_name }}</a>
+                            <a href="{% url 'channel_id' video.channel_id %}">{{ video.channel_name }}</a>
                             {% else %}
-                                <span>{{ video.source.channel_name }}</span>
+                                <span>{{ video.channel_name }}</span>
                             {% endif %}
-                            <a href="https://www.youtube.com/watch?v={{ video.source.youtube_id }}" target="_blank"><h3>{{ video.source.title }}</h3></a>
+                            <a href="https://www.youtube.com/watch?v={{ video.youtube_id }}" target="_blank"><h3>{{ video.title }}</h3></a>
                         </div>
-                        <p>Published: {{ video.source.published }} | Duration: {{ video.source.duration }} | {{ video.source.youtube_id }}</p>
+                        <p>Published: {{ video.published }} | Duration: {{ video.duration }} | {{ video.youtube_id }}</p>
-                        {% if video.source.message %}
+                        {% if video.message %}
-                            <p class="danger-zone">{{ video.source.message }}</p>
+                            <p class="danger-zone">{{ video.message }}</p>
                         {% endif %}
                         <div>
                             {% if show_ignored_only %}
-                                <button data-id="{{ video.source.youtube_id }}" onclick="forgetIgnore(this)">Forget</button>
+                                <button data-id="{{ video.youtube_id }}" onclick="forgetIgnore(this)">Forget</button>
-                                <button data-id="{{ video.source.youtube_id }}" onclick="addSingle(this)">Add to queue</button>
+                                <button data-id="{{ video.youtube_id }}" onclick="addSingle(this)">Add to queue</button>
                             {% else %}
-                                <button data-id="{{ video.source.youtube_id }}" onclick="toIgnore(this)">Ignore</button>
+                                <button data-id="{{ video.youtube_id }}" onclick="toIgnore(this)">Ignore</button>
-                                <button id="{{ video.source.youtube_id }}" data-id="{{ video.source.youtube_id }}" onclick="downloadNow(this)">Download now</button>
+                                <button id="{{ video.youtube_id }}" data-id="{{ video.youtube_id }}" onclick="downloadNow(this)">Download now</button>
                             {% endif %}
-                            {% if video.source.message %}
+                            {% if video.message %}
-                                <button class="danger-button" data-id="{{ video.source.youtube_id }}" onclick="forgetIgnore(this)">Delete</button>
+                                <button class="danger-button" data-id="{{ video.youtube_id }}" onclick="forgetIgnore(this)">Delete</button>
                             {% endif %}
                         </div>
                     </div>

tubearchivist/home/templates/home/home.html

@@ -9,14 +9,14 @@
         <div class="video-list {{ view_style }} {% if view_style == "grid" %}grid-{{ grid_items }}{% endif %}">
             {% for video in continue_vids %}
                 <div class="video-item {{ view_style }}">
-                    <a href="#player" data-id="{{ video.source.youtube_id }}" onclick="createPlayer(this)">
+                    <a href="#player" data-id="{{ video.youtube_id }}" onclick="createPlayer(this)">
                         <div class="video-thumb-wrap {{ view_style }}">
                             <div class="video-thumb">
-                                <img src="{{ video.source.vid_thumb_url }}" alt="video-thumb">
+                                <img src="{{ video.vid_thumb_url }}" alt="video-thumb">
-                                {% if video.source.player.progress %}
+                                {% if video.player.progress %}
-                                    <div class="video-progress-bar" id="progress-{{ video.source.youtube_id }}" style="width: {{video.source.player.progress}}%;"></div>
+                                    <div class="video-progress-bar" id="progress-{{ video.youtube_id }}" style="width: {{video.player.progress}}%;"></div>
                                 {% else %}
-                                    <div class="video-progress-bar" id="progress-{{ video.source.youtube_id }}" style="width: 0%;"></div>
+                                    <div class="video-progress-bar" id="progress-{{ video.youtube_id }}" style="width: 0%;"></div>
                                 {% endif %}
                             </div>
                             <div class="video-play">
@@ -25,17 +25,17 @@
                         </div>
                     </a>
                     <div class="video-desc {{ view_style }}">
-                        <div class="video-desc-player" id="video-info-{{ video.source.youtube_id }}">
+                        <div class="video-desc-player" id="video-info-{{ video.youtube_id }}">
-                            {% if video.source.player.watched %}
+                            {% if video.player.watched %}
-                                <img src="{% static 'img/icon-seen.svg' %}" alt="seen-icon" data-id="{{ video.source.youtube_id }}" data-status="watched" onclick="updateVideoWatchStatus(this)" class="watch-button" title="Mark as unwatched">
+                                <img src="{% static 'img/icon-seen.svg' %}" alt="seen-icon" data-id="{{ video.youtube_id }}" data-status="watched" onclick="updateVideoWatchStatus(this)" class="watch-button" title="Mark as unwatched">
                             {% else %}
-                                <img src="{% static 'img/icon-unseen.svg' %}" alt="unseen-icon" data-id="{{ video.source.youtube_id }}" data-status="unwatched" onclick="updateVideoWatchStatus(this)" class="watch-button" title="Mark as watched">
+                                <img src="{% static 'img/icon-unseen.svg' %}" alt="unseen-icon" data-id="{{ video.youtube_id }}" data-status="unwatched" onclick="updateVideoWatchStatus(this)" class="watch-button" title="Mark as watched">
                             {% endif %}
-                            <span>{{ video.source.published }} | {{ video.source.player.duration_str }}</span>
+                            <span>{{ video.published }} | {{ video.player.duration_str }}</span>
                         </div>
                         <div>
-                            <a href="{% url 'channel_id' video.source.channel.channel_id %}"><h3>{{ video.source.channel.channel_name }}</h3></a>
+                            <a href="{% url 'channel_id' video.channel.channel_id %}"><h3>{{ video.channel.channel_name }}</h3></a>
-                            <a class="video-more" href="{% url 'video' video.source.youtube_id %}"><h2>{{ video.source.title }}</h2></a>
+                            <a class="video-more" href="{% url 'video' video.youtube_id %}"><h2>{{ video.title }}</h2></a>
                         </div>
                     </div>
                 </div>
@@ -60,13 +60,15 @@
         <div class="sort">
             <div id="hidden-form">
                 <span>Sort by:</span>
-                <select name="sort" id="sort" onchange="sortChange(this.value)">
+                <select name="sort_by" id="sort" onchange="sortChange(this)">
                     <option value="published" {% if sort_by == "published" %}selected{% endif %}>date published</option>
                     <option value="downloaded" {% if sort_by == "downloaded" %}selected{% endif %}>date downloaded</option>
                     <option value="views" {% if sort_by == "views" %}selected{% endif %}>views</option>
                     <option value="likes" {% if sort_by == "likes" %}selected{% endif %}>likes</option>
+                    <option value="duration" {% if sort_by == "duration" %}selected{% endif %}>duration</option>
+                    <option value="filesize" {% if sort_by == "filesize" %}selected{% endif %}>file size</option>
                 </select>
-                <select name="sord-order" id="sort-order" onchange="sortChange(this.value)">
+                <select name="sort_order" id="sort-order" onchange="sortChange(this)">
                     <option value="asc" {% if sort_order == "asc" %}selected{% endif %}>asc</option>
                     <option value="desc" {% if sort_order == "desc" %}selected{% endif %}>desc</option>
                 </select>
@@ -95,14 +97,14 @@
         {% if results %}
             {% for video in results %}
                 <div class="video-item {{ view_style }}">
-                    <a href="#player" data-id="{{ video.source.youtube_id }}" onclick="createPlayer(this)">
+                    <a href="#player" data-id="{{ video.youtube_id }}" onclick="createPlayer(this)">
                         <div class="video-thumb-wrap {{ view_style }}">
                             <div class="video-thumb">
-                                <img src="{{ video.source.vid_thumb_url }}" alt="video-thumb">
+                                <img src="{{ video.vid_thumb_url }}" alt="video-thumb">
-                                {% if video.source.player.progress %}
+                                {% if video.player.progress %}
-                                    <div class="video-progress-bar" id="progress-{{ video.source.youtube_id }}" style="width: {{video.source.player.progress}}%;"></div>
+                                    <div class="video-progress-bar" id="progress-{{ video.youtube_id }}" style="width: {{video.player.progress}}%;"></div>
                                 {% else %}
-                                    <div class="video-progress-bar" id="progress-{{ video.source.youtube_id }}" style="width: 0%;"></div>
+                                    <div class="video-progress-bar" id="progress-{{ video.youtube_id }}" style="width: 0%;"></div>
                                 {% endif %}
                             </div>
                             <div class="video-play">
@@ -111,17 +113,17 @@
                         </div>
                     </a>
                     <div class="video-desc {{ view_style }}">
-                        <div class="video-desc-player" id="video-info-{{ video.source.youtube_id }}">
+                        <div class="video-desc-player" id="video-info-{{ video.youtube_id }}">
-                            {% if video.source.player.watched %}
+                            {% if video.player.watched %}
-                                <img src="{% static 'img/icon-seen.svg' %}" alt="seen-icon" data-id="{{ video.source.youtube_id }}" data-status="watched" onclick="updateVideoWatchStatus(this)" class="watch-button" title="Mark as unwatched">
+                                <img src="{% static 'img/icon-seen.svg' %}" alt="seen-icon" data-id="{{ video.youtube_id }}" data-status="watched" onclick="updateVideoWatchStatus(this)" class="watch-button" title="Mark as unwatched">
                             {% else %}
-                                <img src="{% static 'img/icon-unseen.svg' %}" alt="unseen-icon" data-id="{{ video.source.youtube_id }}" data-status="unwatched" onclick="updateVideoWatchStatus(this)" class="watch-button" title="Mark as watched">
+                                <img src="{% static 'img/icon-unseen.svg' %}" alt="unseen-icon" data-id="{{ video.youtube_id }}" data-status="unwatched" onclick="updateVideoWatchStatus(this)" class="watch-button" title="Mark as watched">
                             {% endif %}
-                            <span>{{ video.source.published }} | {{ video.source.player.duration_str }}</span>
+                            <span>{{ video.published }} | {{ video.player.duration_str }}</span>
                         </div>
                         <div>
-                            <a href="{% url 'channel_id' video.source.channel.channel_id %}"><h3>{{ video.source.channel.channel_name }}</h3></a>
+                            <a href="{% url 'channel_id' video.channel.channel_id %}"><h3>{{ video.channel.channel_name }}</h3></a>
-                            <a class="video-more" href="{% url 'video' video.source.youtube_id %}"><h2>{{ video.source.title }}</h2></a>
+                            <a class="video-more" href="{% url 'video' video.youtube_id %}"><h2>{{ video.title }}</h2></a>
                         </div>
                     </div>
                 </div>

tubearchivist/home/templates/home/login.html

@@ -18,20 +18,11 @@
     <meta name="msapplication-TileColor" content="#01202e">
     <meta name="msapplication-config" content="{% static 'favicon/browserconfig.xml' %}">
     <meta name="theme-color" content="#01202e">
-    {% if colors == "dark" %}
+    <link rel="stylesheet" href="{% static 'css/' %}{{ stylesheet }}">
-        <link rel="stylesheet" href="{% static 'css/dark.css' %}">
-    {% else %}
-        <link rel="stylesheet" href="{% static 'css/light.css' %}">
-    {% endif %}
 </head>
 <body>
     <div class="boxed-content login-page">
-        {% if colors == 'dark' %}
+        <img alt="tube-archivist-logo">
-            <img src="{% static 'img/logo-tube-archivist-dark.png' %}" alt="tube-archivist-logo">
-        {% endif %}
-        {% if colors == 'light' %}
-            <img src="{% static 'img/logo-tube-archivist-light.png' %}" alt="tube-archivist-banner">
-        {% endif %}
         <h1>Tube Archivist</h1>
         <h2>Your Self Hosted YouTube Media Server</h2>
         {% if form_error %}

tubearchivist/home/templates/home/playlist.html

@@ -1,11 +1,15 @@
 {% extends "home/base.html" %}
 {% load static %}
 {% block content %}
+{% load auth_extras %}
+
 <div class="boxed-content">
     <div class="title-split">
         <div class="title-bar">
             <h1>Playlists</h1>
         </div>
+        {% if request.user|has_group:"admin" or request.user.is_staff %}
+
         <div class="title-split-form">
             <img id="animate-icon" onclick="showForm()" src="{% static 'img/icon-add.svg' %}" alt="add-icon" title="Subscribe to Playlists">
             <div class="show-form">
@@ -16,6 +20,8 @@
                 </form>
             </div>
         </div>
+        {% endif %}
+
     </div>
     <div id="notifications" data="subscription"></div>
     <div class="view-controls">
@@ -40,18 +46,18 @@
             {% for playlist in results %}
             <div class="playlist-item {{ view_style }}">
                 <div class="playlist-thumbnail">
-                    <a href="{% url 'playlist_id' playlist.source.playlist_id %}">
+                    <a href="{% url 'playlist_id' playlist.playlist_id %}">
-                        <img src="/cache/playlists/{{ playlist.source.playlist_id }}.jpg" alt="{{ playlist.source.playlist_id }}-thumbnail">
+                        <img src="/cache/playlists/{{ playlist.playlist_id }}.jpg" alt="{{ playlist.playlist_id }}-thumbnail">
                     </a>
                 </div>
                 <div class="playlist-desc {{ view_style }}">
-                    <a href="{% url 'channel_id' playlist.source.playlist_channel_id %}"><h3>{{ playlist.source.playlist_channel }}</h3></a>
+                    <a href="{% url 'channel_id' playlist.playlist_channel_id %}"><h3>{{ playlist.playlist_channel }}</h3></a>
-                    <a href="{% url 'playlist_id' playlist.source.playlist_id %}"><h2>{{ playlist.source.playlist_name }}</h2></a>
+                    <a href="{% url 'playlist_id' playlist.playlist_id %}"><h2>{{ playlist.playlist_name }}</h2></a>
-                    <p>Last refreshed: {{ playlist.source.playlist_last_refresh }}</p>
+                    <p>Last refreshed: {{ playlist.playlist_last_refresh }}</p>
-                    {% if playlist.source.playlist_subscribed %}
+                    {% if playlist.playlist_subscribed %}
-                        <button class="unsubscribe" type="button" data-type="playlist" data-subscribe="" data-id="{{ playlist.source.playlist_id }}" onclick="subscribeStatus(this)" title="Unsubscribe from {{ playlist.source.playlist_name }}">Unsubscribe</button>
+                        <button class="unsubscribe" type="button" data-type="playlist" data-subscribe="" data-id="{{ playlist.playlist_id }}" onclick="subscribeStatus(this)" title="Unsubscribe from {{ playlist.playlist_name }}">Unsubscribe</button>
                     {% else %}
-                        <button type="button" data-type="playlist" data-subscribe="true" data-id="{{ playlist.source.playlist_id }}" onclick="subscribeStatus(this)" title="Subscribe to {{ playlist.source.playlist_name }}">Subscribe</button>
+                        <button type="button" data-type="playlist" data-subscribe="true" data-id="{{ playlist.playlist_id }}" onclick="subscribeStatus(this)" title="Subscribe to {{ playlist.playlist_name }}">Subscribe</button>
                     {% endif %}
                 </div>
             </div>

tubearchivist/home/templates/home/playlist_id.html

@@ -2,6 +2,8 @@
 {% load static %}
 {% load humanize %}
 {% block content %}
+{% load auth_extras %}
+
 <div class="boxed-content">
     <div class="title-bar">
         <h1>{{ playlist_info.playlist_name }}</h1>
@@ -27,7 +29,9 @@
                 <p>Last refreshed: {{ playlist_info.playlist_last_refresh }}</p>
                 <p>Playlist:
                     {% if playlist_info.playlist_subscribed %}
+                        {% if request.user|has_group:"admin" or request.user.is_staff %}
                         <button class="unsubscribe" type="button" data-type="playlist" data-subscribe="" data-id="{{ playlist_info.playlist_id }}" onclick="subscribeStatus(this)" title="Unsubscribe from {{ playlist_info.playlist_name }}">Unsubscribe</button>
+                        {% endif %}
                     {% else %}
                         <button type="button" data-type="playlist" data-subscribe="true" data-id="{{ playlist_info.playlist_id }}" onclick="subscribeStatus(this)" title="Subscribe to {{ playlist_info.playlist_name }}">Subscribe</button>
                     {% endif %}
@@ -50,7 +54,10 @@
             <div>
                 {% if max_hits %}
                     <p>Total Videos archived: {{ max_hits }}/{{ playlist_info.playlist_entries|length }}</p>
-                    <p>Watched: <button title="Mark all videos from {{ playlist_info.playlist_name }} as watched" type="button" id="watched-button" data-id="{{ playlist_info.playlist_id }}" onclick="isWatchedButton(this)">Mark as watched</button></p>
+                    <div id="watched-button" class="button-box">
+                        <button title="Mark all videos from {{ playlist_info.playlist_name }} as watched" type="button" id="watched-button" data-id="{{ playlist_info.playlist_id }}" onclick="isWatchedButton(this)">Mark as watched</button>
+                        <button title="Mark all videos from {{ playlist_info.playlist_name }} as unwatched" type="button" id="unwatched-button" data-id="{{ playlist_info.playlist_id }}" onclick="isUnwatchedButton(this)">Mark as unwatched</button>
+                    </div>
                 {% endif %}
                 {% if reindex %}
                     <p>Reindex scheduled</p>
@@ -107,14 +114,14 @@
         {% if results %}
             {% for video in results %}
                 <div class="video-item {{ view_style }}">
-                    <a href="#player" data-id="{{ video.source.youtube_id }}" onclick="createPlayer(this)">
+                    <a href="#player" data-id="{{ video.youtube_id }}" onclick="createPlayer(this)">
                         <div class="video-thumb-wrap {{ view_style }}">
                             <div class="video-thumb">
-                                <img src="{{ video.source.vid_thumb_url }}" alt="video-thumb">
+                                <img src="{{ video.vid_thumb_url }}" alt="video-thumb">
-                                {% if video.source.player.progress %}
+                                {% if video.player.progress %}
-                                    <div class="video-progress-bar" id="progress-{{ video.source.youtube_id }}" style="width: {{video.source.player.progress}}%;"></div>
+                                    <div class="video-progress-bar" id="progress-{{ video.youtube_id }}" style="width: {{video.player.progress}}%;"></div>
                                 {% else %}
-                                    <div class="video-progress-bar" id="progress-{{ video.source.youtube_id }}" style="width: 0%;"></div>
+                                    <div class="video-progress-bar" id="progress-{{ video.youtube_id }}" style="width: 0%;"></div>
                                 {% endif %}
                             </div>
                             <div class="video-play">
@@ -123,16 +130,16 @@
                         </div>
                     </a>
                     <div class="video-desc {{ view_style }}">
-                        <div class="video-desc-player" id="video-info-{{ video.source.youtube_id }}">
+                        <div class="video-desc-player" id="video-info-{{ video.youtube_id }}">
-                            {% if video.source.player.watched %}
+                            {% if video.player.watched %}
-                                <img src="{% static 'img/icon-seen.svg' %}" alt="seen-icon" data-id="{{ video.source.youtube_id }}" data-status="watched" onclick="updateVideoWatchStatus(this)" class="watch-button" title="Mark as unwatched">
+                                <img src="{% static 'img/icon-seen.svg' %}" alt="seen-icon" data-id="{{ video.youtube_id }}" data-status="watched" onclick="updateVideoWatchStatus(this)" class="watch-button" title="Mark as unwatched">
                             {% else %}
-                                <img src="{% static 'img/icon-unseen.svg' %}" alt="unseen-icon" data-id="{{ video.source.youtube_id }}" data-status="unwatched" onclick="updateVideoWatchStatus(this)" class="watch-button" title="Mark as watched">
+                                <img src="{% static 'img/icon-unseen.svg' %}" alt="unseen-icon" data-id="{{ video.youtube_id }}" data-status="unwatched" onclick="updateVideoWatchStatus(this)" class="watch-button" title="Mark as watched">
                             {% endif %}
-                            <span>{{ video.source.published }} | {{ video.source.player.duration_str }}</span>
+                            <span>{{ video.published }} | {{ video.player.duration_str }}</span>
                         </div>
                         <div>
-                            <a class="video-more" href="{% url 'video' video.source.youtube_id %}"><h2>{{ video.source.title }}</h2></a>
+                            <a class="video-more" href="{% url 'video' video.youtube_id %}"><h2>{{ video.title }}</h2></a>
                         </div>
                     </div>
                 </div>

tubearchivist/home/templates/home/settings.html

@@ -5,14 +5,26 @@
     <h1>Your Archive</h1>
 </div>
 <div class="settings-item">
-    <h2>Main overview</h2>
+    <h2>Overview</h2>
-    <div id="primaryBox" class="info-box info-box-4">
+    <div id="activeBox" class="info-box info-box-3">
+        <p id="loading">Loading...</p>
+    </div>
+</div>
+<div class="settings-item">
+    <h2>Video Type</h2>
+    <div id="videoTypeBox" class="info-box info-box-3">
+        <p id="loading">Loading...</p>
+    </div>
+</div>
+<div class="settings-item">
+    <h2>Application</h2>
+    <div id="secondaryBox" class="info-box info-box-3">
         <p id="loading">Loading...</p>
     </div>
 </div>
 <div class="settings-item">
     <h2>Watch Progress</h2>
-    <div id="watchBox" class="info-box info-box-3">
+    <div id="watchBox" class="info-box info-box-2">
         <p id="loading">Loading...</p>
     </div>
 </div>
@@ -22,21 +34,47 @@
         <p id="loading">Loading...</p>
     </div>
 </div>
+
 <div class="settings-item">
     <h2>Biggest Channels</h2>
-    <div class="info-box description-box">
+    <div class="info-box info-box-3">
-        <table>
+        <div class="info-box-item">
+            <table class="agg-channel-table">
                 <thead>
                     <tr>
-                    <th class="agg-channel-name">Name</th>
+                        <th>Name</th>
-                    <th>Videos</th>
+                        <th class="agg-channel-right-align">Videos</th>
-                    <th>Duration</th>
-                    <th>Media Size</th>
                     </tr>
                 </thead>
-            <tbody id="biggestChannelTable"></tbody>
+                <tbody id="biggestChannelTableVideos"></tbody>
             </table>
         </div>
+
+        <div class="info-box-item">
+            <table class="agg-channel-table">
+                <thead>
+                    <tr>
+                        <th>Name</th>
+                        <th class="agg-channel-right-align">Duration</th>
+                    </tr>
+                </thead>
+                <tbody id="biggestChannelTableDuration"></tbody>
+            </table>
+        </div>
+        
+        <div class="info-box-item">
+            <table class="agg-channel-table">
+                <thead>
+                    <tr>
+                        <th>Name</th>
+                        <th class="agg-channel-right-align">Media Size</th>
+                    </tr>
+                </thead>
+                <tbody id="biggestChannelTableMediaSize"></tbody>
+            </table>
+        </div>
+    </div>
 </div>
+
 <script type="text/javascript" src="{% static 'stats.js' %}"></script>
 {% endblock settings_content %}

tubearchivist/home/templates/home/settings_actions.html

@@ -12,7 +12,7 @@
 </div>
 <div class="settings-group">
     <h2>Manual media files import.</h2>
-    <p>Add files to the <span class="settings-current">cache/import</span> folder. Make sure to follow the instructions in the Github <a href="https://docs.tubearchivist.com/settings/" target="_blank">Wiki</a>.</p>
+    <p>Add files to the <span class="settings-current">cache/import</span> folder. Make sure to follow the instructions in the Github <a href="https://docs.tubearchivist.com/settings/actions/#manual-media-files-import" target="_blank">Wiki</a>.</p>
     <div id="manual-import">
         <button onclick="manualImport()">Start import</button>
     </div>
@@ -58,7 +58,7 @@
 <div class="settings-group">
     <h2>Rescan filesystem</h2>
     <p><span class="danger-zone">Danger Zone</span>: This will delete the metadata of deleted videos from the filesystem.</p>
-    <p>Rescan your media folder looking for missing videos and clean up index. More infos on the Github <a href="https://docs.tubearchivist.com/settings/" target="_blank">Wiki</a>.</p>
+    <p>Rescan your media folder looking for missing videos and clean up index. More infos on the Github <a href="https://docs.tubearchivist.com/settings/actions/#rescan-filesystem" target="_blank">Wiki</a>.</p>
     <div id="fs-rescan">
         <button onclick="fsRescan()">Rescan filesystem</button>
     </div>

tubearchivist/home/templates/home/settings_application.html

@@ -141,7 +141,7 @@
         <div class="settings-item">
             <p>Import YouTube cookie: <span class="settings-current">{{ config.downloads.cookie_import }}</span><br></p>
             <p>For automatic cookie import use <b>Tube Archivist Companion</b> <a href="https://github.com/tubearchivist/browser-extension" target="_blank">browser extension</a>.</p> 
-            <i>For manual cookie import, place your cookie file named <span class="settings-current">cookies.google.txt</span> in <span class="settings-current">cache/import</span> before enabling. Instructions in the <a href="https://docs.tubearchivist.com/settings/" target="_blank">Wiki.</a></i><br>
+            <i>For manual cookie import, place your cookie file named <span class="settings-current">cookies.google.txt</span> in <span class="settings-current">cache/import</span> before enabling. Instructions in the <a href="https://docs.tubearchivist.com/settings/application/#cookie" target="_blank">Wiki.</a></i><br>
             {{ app_form.downloads_cookie_import }}<br>
             {% if config.downloads.cookie_import %}
                 <div id="cookieMessage">
@@ -174,7 +174,7 @@
         <h2 id="snapshots">Snapshots</h2>
         <div class="settings-item">
             <p>Current system snapshot: <span class="settings-current">{{ config.application.enable_snapshot }}</span></p>
-            <i>Automatically create daily deduplicated snapshots of the index, stored in Elasticsearch. Read first before activating: <a target="_blank" href="https://docs.tubearchivist.com/settings/#snapshots">Wiki</a>.</i><br>
+            <i>Automatically create daily deduplicated snapshots of the index, stored in Elasticsearch. Read first before activating: <a target="_blank" href="https://docs.tubearchivist.com/settings/application/#snapshots">Wiki</a>.</i><br>
             {{ app_form.application_enable_snapshot }}
         </div>
         <div>

tubearchivist/home/templates/home/settings_scheduling.html

@@ -51,7 +51,7 @@
         </div>
     </div>
     <div class="settings-group">
-        <h2>Start download</h2>
+        <h2>Start Download</h2>
         <div class="settings-item">
             <p>Current Download schedule: <span class="settings-current">
                 {% if config.scheduler.download_pending %}
@@ -112,7 +112,7 @@
         </div>
     </div>
     <div class="settings-group">
-        <h2>Thumbnail check</h2>
+        <h2>Thumbnail Check</h2>
         <div class="settings-item">
             <p>Current thumbnail check schedule: <span class="settings-current">
                 {% if config.scheduler.thumbnail_check %}

tubearchivist/home/templates/home/settings_user.html

@@ -7,17 +7,17 @@
 <form action="{% url 'settings_user' %}" method="POST" name="user-update">
     {% csrf_token %}
     <div class="settings-group">
-        <h2>Color scheme</h2>
+        <h2>Stylesheet</h2>
         <div class="settings-item">
-            <p>Current color scheme: <span class="settings-current">{{ config.application.colors }}</span></p>
+            <p>Current stylesheet: <span class="settings-current">{{ stylesheet }}</span></p>
-            <i>Select your preferred color scheme between dark and light mode.</i><br>
+            <i>Select your preferred stylesheet.</i><br>
-            {{ user_form.colors }}
+            {{ user_form.stylesheet }}
         </div>
     </div>
     <div class="settings-group">
         <h2>Archive View</h2>
         <div class="settings-item">
-            <p>Current page size: <span class="settings-current">{{ config.archive.page_size }}</span></p>
+            <p>Current page size: <span class="settings-current">{{ page_size }}</span></p>
             <i>Result of videos showing in archive page</i><br>
             {{ user_form.page_size }}
         </div>

tubearchivist/home/templates/home/video.html

@@ -2,6 +2,7 @@
 {% block content %}
 {% load static %}
 {% load humanize %}
+{% load auth_extras %}
 <div id="player" class="player-wrapper">
     <div class="video-main">
         <div class="video-modal"><span class="video-modal-text"></span></div>
@@ -81,15 +82,19 @@
                 {% if reindex %}
                     <p>Reindex scheduled</p>
                 {% else %}
+                  {% if request.user|has_group:"admin" or request.user.is_staff %}
                     <div id="reindex-button" class="button-box">
                         <button data-id="{{ video.youtube_id }}" data-type="video" onclick="reindex(this)" title="Reindex {{ video.title }}">Reindex</button>
                     </div>
                   {% endif %}
-                <a download="" href="/media/{{ video.media_url }}"><button id="download-item">Download File</button></a>
+                {% endif %}
+                <a download="" href="{{ video.media_url }}"><button id="download-item">Download File</button></a>
+                {% if request.user|has_group:"admin" or request.user.is_staff %}
                 <button onclick="deleteConfirm()" id="delete-item">Delete Video</button>
                 <div class="delete-confirm" id="delete-button">
                     <span>Are you sure? </span><button class="danger-button" onclick="deleteVideo(this)" data-id="{{ video.youtube_id }}" data-redirect = "{{ video.channel.channel_id }}">Delete</button> <button onclick="cancelDelete()">Cancel</button>
                 </div>
+                {% endif %}
             </div>
         </div>
         <div class="info-box-item">

tubearchivist/home/templatetags/auth_extras.py

@@ -0,0 +1,8 @@
+from django import template
+
+register = template.Library()
+
+
+@register.filter(name="has_group")
+def has_group(user, group_name):
+    return user.groups.filter(name=group_name).exists()

tubearchivist/home/urls.py

@@ -3,18 +3,30 @@
 from django.conf import settings
 from django.contrib.auth.decorators import login_required
 from django.contrib.auth.views import LogoutView
+from django.shortcuts import redirect
 from django.urls import path
 from home import views
 
-urlpatterns = [
+if hasattr(settings, "TA_AUTH_PROXY_LOGOUT_URL"):
-    path("", login_required(views.HomeView.as_view()), name="home"),
+    logout_path = path(
-    path("login/", views.LoginView.as_view(), name="login"),
+        "logout/",
-    path(
+        lambda request: redirect(
+            settings.TA_AUTH_PROXY_LOGOUT_URL, permanent=False
+        ),
+        name="logout",
+    )
+else:
+    logout_path = path(
         "logout/",
         LogoutView.as_view(),
         {"next_page": settings.LOGOUT_REDIRECT_URL},
         name="logout",
-    ),
+    )
+
+urlpatterns = [
+    path("", login_required(views.HomeView.as_view()), name="home"),
+    path("login/", views.LoginView.as_view(), name="login"),
+    logout_path,
     path("about/", views.AboutView.as_view(), name="about"),
     path(
         "downloads/",
@@ -46,7 +58,6 @@ urlpatterns = [
         login_required(views.SettingsActionsView.as_view()),
         name="settings_actions",
     ),
-    path("process/", login_required(views.process), name="process"),
     path(
         "channel/",
         login_required(views.ChannelView.as_view()),

tubearchivist/home/views.py

@@ -4,23 +4,24 @@ Functionality:
 - holds base classes to inherit from
 """
 import enum
-import json
 import urllib.parse
 from time import sleep
 
-from api.src.search_processor import SearchProcess
+from api.src.search_processor import SearchProcess, process_aggs
+from api.views import check_admin
 from django.conf import settings
 from django.contrib.auth import login
+from django.contrib.auth.decorators import user_passes_test
 from django.contrib.auth.forms import AuthenticationForm
-from django.http import JsonResponse
+from django.http import Http404
 from django.shortcuts import redirect, render
+from django.utils.decorators import method_decorator
 from django.views import View
 from home.src.download.queue import PendingInteract
 from home.src.download.yt_dlp_base import CookieHandler
 from home.src.es.backup import ElasticBackup
 from home.src.es.connect import ElasticWrap
 from home.src.es.snapshot import ElasticSnapshot
-from home.src.frontend.api_calls import PostData
 from home.src.frontend.forms import (
     AddToQueueForm,
     ApplicationSettingsForm,
@@ -32,15 +33,16 @@ from home.src.frontend.forms import (
     SubscribeToPlaylistForm,
     UserSettingsForm,
 )
-from home.src.frontend.searching import SearchHandler
 from home.src.index.channel import channel_overwrites
 from home.src.index.generic import Pagination
 from home.src.index.playlist import YoutubePlaylist
 from home.src.index.reindex import ReindexProgress
 from home.src.index.video_constants import VideoTypeEnum
 from home.src.ta.config import AppConfig, ReleaseVersion, ScheduleBuilder
-from home.src.ta.helper import time_parser
+from home.src.ta.helper import check_stylesheet, time_parser
+from home.src.ta.settings import EnvironmentSettings
 from home.src.ta.ta_redis import RedisArchivist
+from home.src.ta.users import UserConfig
 from home.tasks import index_channel_playlists, subscribe_to
 from rest_framework.authtoken.models import Token
 
@@ -52,93 +54,38 @@ class ArchivistViewConfig(View):
         super().__init__()
         self.view_origin = view_origin
         self.user_id = False
-        self.user_conf = False
+        self.user_conf: UserConfig = False
-        self.default_conf = False
         self.context = False
 
-    def _get_sort_by(self):
-        """return sort_by config var"""
-        messag_key = f"{self.user_id}:sort_by"
-        sort_by = self.user_conf.get_message(messag_key)["status"]
-        if not sort_by:
-            sort_by = self.default_conf["archive"]["sort_by"]
-
-        return sort_by
-
-    def _get_sort_order(self):
-        """return sort_order config var"""
-        sort_order_key = f"{self.user_id}:sort_order"
-        sort_order = self.user_conf.get_message(sort_order_key)["status"]
-        if not sort_order:
-            sort_order = self.default_conf["archive"]["sort_order"]
-
-        return sort_order
-
-    def _get_view_style(self):
-        """return view_style config var"""
-        view_key = f"{self.user_id}:view:{self.view_origin}"
-        view_style = self.user_conf.get_message(view_key)["status"]
-        if not view_style:
-            view_style = self.default_conf["default_view"][self.view_origin]
-
-        return view_style
-
-    def _get_grid_items(self):
-        """return items per row to show in grid view"""
-        grid_key = f"{self.user_id}:grid_items"
-        grid_items = self.user_conf.get_message(grid_key)["status"]
-        if not grid_items:
-            grid_items = self.default_conf["default_view"]["grid_items"]
-
-        return grid_items
-
     def get_all_view_styles(self):
-        """get dict of all view stiles for search form"""
+        """get dict of all view styles for search form"""
-        all_keys = ["channel", "playlist", "home"]
         all_styles = {}
-        for view_origin in all_keys:
+        for view_origin in ["channel", "playlist", "home", "downloads"]:
-            view_key = f"{self.user_id}:view:{view_origin}"
+            all_styles[view_origin] = self.user_conf.get_value(
-            view_style = self.user_conf.get_message(view_key)["status"]
+                f"view_style_{view_origin}"
-            if not view_style:
+            )
-                view_style = self.default_conf["default_view"][view_origin]
-            all_styles[view_origin] = view_style
 
         return all_styles
 
-    def _get_hide_watched(self):
-        hide_watched_key = f"{self.user_id}:hide_watched"
-        hide_watched = self.user_conf.get_message(hide_watched_key)["status"]
-
-        return hide_watched
-
-    def _get_show_ignore_only(self):
-        ignored_key = f"{self.user_id}:show_ignored_only"
-        show_ignored_only = self.user_conf.get_message(ignored_key)["status"]
-
-        return show_ignored_only
-
-    def _get_show_subed_only(self):
-        sub_only_key = f"{self.user_id}:show_subed_only"
-        show_subed_only = self.user_conf.get_message(sub_only_key)["status"]
-
-        return show_subed_only
-
     def config_builder(self, user_id):
         """build default context for every view"""
         self.user_id = user_id
-        self.user_conf = RedisArchivist()
+        self.user_conf = UserConfig(self.user_id)
-        self.default_conf = AppConfig(self.user_id).config
 
         self.context = {
-            "colors": self.default_conf["application"]["colors"],
+            "stylesheet": check_stylesheet(
-            "cast": self.default_conf["application"]["enable_cast"],
+                self.user_conf.get_value("stylesheet")
-            "sort_by": self._get_sort_by(),
+            ),
-            "sort_order": self._get_sort_order(),
+            "cast": EnvironmentSettings.ENABLE_CAST,
-            "view_style": self._get_view_style(),
+            "sort_by": self.user_conf.get_value("sort_by"),
-            "grid_items": self._get_grid_items(),
+            "sort_order": self.user_conf.get_value("sort_order"),
-            "hide_watched": self._get_hide_watched(),
+            "view_style": self.user_conf.get_value(
-            "show_ignored_only": self._get_show_ignore_only(),
+                f"view_style_{self.view_origin}"
-            "show_subed_only": self._get_show_subed_only(),
+            ),
+            "grid_items": self.user_conf.get_value("grid_items"),
+            "hide_watched": self.user_conf.get_value("hide_watched"),
+            "show_ignored_only": self.user_conf.get_value("show_ignored_only"),
+            "show_subed_only": self.user_conf.get_value("show_subed_only"),
             "version": settings.TA_VERSION,
             "ta_update": ReleaseVersion().get_update(),
         }
@@ -164,6 +111,8 @@ class ArchivistResultsView(ArchivistViewConfig):
             "likes": "stats.like_count",
             "downloaded": "date_downloaded",
             "published": "published",
+            "duration": "player.duration",
+            "filesize": "media_size",
         }
         sort_by = sort_by_map[self.context["sort_by"]]
 
@@ -199,8 +148,7 @@ class ArchivistResultsView(ArchivistViewConfig):
         self.context["continue_vids"] = self.get_in_progress(results)
 
         in_progress = {i["youtube_id"]: i["position"] for i in results}
-        for hit in self.context["results"]:
+        for video in self.context["results"]:
-            video = hit["source"]
             if video["youtube_id"] in in_progress:
                 played_sec = in_progress.get(video["youtube_id"])
                 total = video["player"]["duration"]
@@ -212,32 +160,35 @@ class ArchivistResultsView(ArchivistViewConfig):
         """get all videos in progress"""
         ids = [{"match": {"youtube_id": i.get("youtube_id")}} for i in results]
         data = {
-            "size": self.default_conf["archive"]["page_size"],
+            "size": UserConfig(self.user_id).get_value("page_size"),
             "query": {"bool": {"should": ids}},
             "sort": [{"published": {"order": "desc"}}],
         }
-        search = SearchHandler(
+        response, _ = ElasticWrap("ta_video/_search").get(data)
-            "ta_video/_search", self.default_conf, data=data
+        videos = SearchProcess(response).process()
-        )
+
-        videos = search.get_data()
         if not videos:
             return False
 
         for video in videos:
-            youtube_id = video["source"]["youtube_id"]
+            youtube_id = video["youtube_id"]
             matched = [i for i in results if i["youtube_id"] == youtube_id]
             played_sec = matched[0]["position"]
-            total = video["source"]["player"]["duration"]
+            total = video["player"]["duration"]
             if not total:
                 total = matched[0].get("position") * 2
-            video["source"]["player"]["progress"] = 100 * (played_sec / total)
+            video["player"]["progress"] = 100 * (played_sec / total)
 
         return videos
 
     def single_lookup(self, es_path):
         """retrieve a single item from url"""
-        search = SearchHandler(es_path, config=self.default_conf)
+        response, status_code = ElasticWrap(es_path).get()
-        result = search.get_data()[0]["source"]
+        if not status_code == 200:
+            raise Http404
+
+        result = SearchProcess(response).process()
+
         return result
 
     def initiate_vars(self, request):
@@ -251,14 +202,19 @@ class ArchivistResultsView(ArchivistViewConfig):
 
     def find_results(self):
         """add results and pagination to context"""
-        search = SearchHandler(
+        response, _ = ElasticWrap(self.es_search).get(self.data)
-            self.es_search, config=self.default_conf, data=self.data
+        process_aggs(response)
+        results = SearchProcess(response).process()
+        max_hits = response["hits"]["total"]["value"]
+        self.pagination_handler.validate(max_hits)
+        self.context.update(
+            {
+                "results": results,
+                "max_hits": max_hits,
+                "pagination": self.pagination_handler.pagination,
+                "aggs": response.get("aggregations"),
+            }
         )
-        self.context["results"] = search.get_data()
-        self.pagination_handler.validate(search.max_hits)
-        self.context["max_hits"] = search.max_hits
-        self.context["pagination"] = self.pagination_handler.pagination
-        self.context["aggs"] = search.aggs
 
 
 class MinView(View):
@@ -268,7 +224,9 @@ class MinView(View):
     def get_min_context(request):
         """build minimal vars for context"""
         return {
-            "colors": AppConfig(request.user.id).colors,
+            "stylesheet": check_stylesheet(
+                UserConfig(request.user.id).get_value("stylesheet")
+            ),
             "version": settings.TA_VERSION,
             "ta_update": ReleaseVersion().get_update(),
         }
@@ -365,6 +323,7 @@ class AboutView(MinView):
         return render(request, "home/about.html", context)
 
 
+@method_decorator(user_passes_test(check_admin), name="dispatch")
 class DownloadView(ArchivistResultsView):
     """resolves to /download/
     handle the download queue
@@ -552,7 +511,7 @@ class ChannelIdView(ChannelIdBaseView):
         self.channel_pages(channel_id)
 
         if self.context["results"]:
-            channel_info = self.context["results"][0]["source"]["channel"]
+            channel_info = self.context["results"][0]["channel"]
             channel_name = channel_info["channel_name"]
         else:
             # fall back channel lookup if no videos found
@@ -645,6 +604,7 @@ class ChannelIdAboutView(ChannelIdBaseView):
 
         return render(request, "home/channel_id_about.html", self.context)
 
+    @method_decorator(user_passes_test(check_admin), name="dispatch")
     @staticmethod
     def post(request, channel_id):
         """handle post request"""
@@ -729,6 +689,7 @@ class ChannelView(ArchivistResultsView):
                 "term": {"channel_subscribed": {"value": True}}
             }
 
+    @method_decorator(user_passes_test(check_admin), name="dispatch")
     @staticmethod
     def post(request):
         """handle http post requests"""
@@ -872,6 +833,7 @@ class PlaylistView(ArchivistResultsView):
                 }
             }
 
+    @method_decorator(user_passes_test(check_admin), name="dispatch")
     @staticmethod
     def post(request):
         """handle post from search form"""
@@ -892,9 +854,10 @@ class VideoView(MinView):
 
     def get(self, request, video_id):
         """get single video"""
-        config_handler = AppConfig(request.user.id)
+        config_handler = AppConfig()
-        look_up = SearchHandler(f"ta_video/_doc/{video_id}", config=False)
+        response, _ = ElasticWrap(f"ta_video/_doc/{video_id}").get()
-        video_data = look_up.get_data()[0]["source"]
+        video_data = SearchProcess(response).process()
+
         try:
             rating = video_data["stats"]["average_rating"]
             video_data["stats"]["average_rating"] = self.star_creator(rating)
@@ -917,7 +880,7 @@ class VideoView(MinView):
                 "video": video_data,
                 "playlist_nav": playlist_nav,
                 "title": video_data.get("title"),
-                "cast": config_handler.config["application"]["enable_cast"],
+                "cast": EnvironmentSettings.ENABLE_CAST,
                 "config": config_handler.config,
                 "position": time_parser(request.GET.get("t")),
                 "reindex": reindex.get("state"),
@@ -1005,7 +968,9 @@ class SettingsUserView(MinView):
         context.update(
             {
                 "title": "User Settings",
-                "config": AppConfig(request.user.id).config,
+                "page_size": UserConfig(request.user.id).get_value(
+                    "page_size"
+                ),
                 "user_form": UserSettingsForm(),
             }
         )
@@ -1015,15 +980,23 @@ class SettingsUserView(MinView):
     def post(self, request):
         """handle form post to update settings"""
         user_form = UserSettingsForm(request.POST)
+        config_handler = UserConfig(request.user.id)
         if user_form.is_valid():
             user_form_post = user_form.cleaned_data
-            if any(user_form_post.values()):
+            if user_form_post.get("stylesheet"):
-                AppConfig().set_user_config(user_form_post, request.user.id)
+                config_handler.set_value(
+                    "stylesheet", user_form_post.get("stylesheet")
+                )
+            if user_form_post.get("page_size"):
+                config_handler.set_value(
+                    "page_size", user_form_post.get("page_size")
+                )
 
         sleep(1)
         return redirect("settings_user", permanent=True)
 
 
+@method_decorator(user_passes_test(check_admin), name="dispatch")
 class SettingsApplicationView(MinView):
     """resolves to /settings/application/
     handle the settings sub-page for application configuration,
@@ -1037,7 +1010,7 @@ class SettingsApplicationView(MinView):
         context.update(
             {
                 "title": "Application Settings",
-                "config": AppConfig(request.user.id).config,
+                "config": AppConfig().config,
                 "api_token": self.get_token(request),
                 "app_form": ApplicationSettingsForm(),
                 "snapshots": ElasticSnapshot().get_snapshot_stats(),
@@ -1113,6 +1086,7 @@ class SettingsApplicationView(MinView):
         RedisArchivist().set_message(key, message=message, expire=True)
 
 
+@method_decorator(user_passes_test(check_admin), name="dispatch")
 class SettingsSchedulingView(MinView):
     """resolves to /settings/scheduling/
     handle the settings sub-page for scheduling settings,
@@ -1126,7 +1100,7 @@ class SettingsSchedulingView(MinView):
         context.update(
             {
                 "title": "Scheduling Settings",
-                "config": AppConfig(request.user.id).config,
+                "config": AppConfig().config,
                 "scheduler_form": SchedulerSettingsForm(),
             }
         )
@@ -1146,6 +1120,7 @@ class SettingsSchedulingView(MinView):
         return redirect("settings_scheduling", permanent=True)
 
 
+@method_decorator(user_passes_test(check_admin), name="dispatch")
 class SettingsActionsView(MinView):
     """resolves to /settings/actions/
     handle the settings actions sub-page
@@ -1162,16 +1137,3 @@ class SettingsActionsView(MinView):
         )
 
         return render(request, "home/settings_actions.html", context)
-
-
-def process(request):
-    """handle all the buttons calls via POST ajax"""
-    if request.method == "POST":
-        current_user = request.user.id
-        post_dict = json.loads(request.body.decode())
-        post_handler = PostData(post_dict, current_user)
-        if post_handler.to_exec:
-            task_result = post_handler.run_task()
-            return JsonResponse(task_result)
-
-    return JsonResponse({"success": False})

tubearchivist/requirements.txt

@@ -1,13 +1,13 @@
-apprise==1.4.5
+apprise==1.6.0
-celery==5.3.1
+celery==5.3.5
-Django==4.2.4
+Django==4.2.7
-django-auth-ldap==4.5.0
+django-auth-ldap==4.6.0
-django-cors-headers==4.2.0
+django-cors-headers==4.3.1
 djangorestframework==3.14.0
-Pillow==10.0.0
+Pillow==10.1.0
-redis==5.0.0
+redis==5.0.1
 requests==2.31.0
 ryd-client==0.0.6
-uWSGI==2.0.22
+uWSGI==2.0.23
-whitenoise==6.5.0
+whitenoise==6.6.0
-yt_dlp==2023.7.6
+yt-dlp==2023.11.16

tubearchivist/static/css/dark.css

@@ -9,4 +9,6 @@
     --accent-font-light: #97d4c8;
     --img-filter: invert(50%) sepia(9%) saturate(2940%) hue-rotate(122deg) brightness(94%) contrast(90%);
     --img-filter-error: invert(16%) sepia(60%) saturate(3717%) hue-rotate(349deg) brightness(86%) contrast(120%);
+    --banner: url("../img/banner-tube-archivist-dark.png");
+    --logo: url("../img/logo-tube-archivist-dark.png");
 }

tubearchivist/static/css/light.css

@@ -9,4 +9,6 @@
     --accent-font-light: #35b399;
     --img-filter: invert(50%) sepia(9%) saturate(2940%) hue-rotate(122deg) brightness(94%) contrast(90%);
     --img-filter-error: invert(16%) sepia(60%) saturate(3717%) hue-rotate(349deg) brightness(86%) contrast(120%);
+    --banner: url("../img/banner-tube-archivist-light.png");
+    --logo: url("../img/logo-tube-archivist-light.png");
 }

tubearchivist/static/css/matrix.css

@@ -0,0 +1,67 @@
+:root {
+    --main-bg: #000000;
+    --highlight-bg: #080808;
+    --highlight-error: #880000;
+    --highlight-error-light: #aa0000;
+    --highlight-bg-transparent: #0c0c0caf;
+    --main-font: #00aa00;
+    --accent-font-dark: #007700;
+    --accent-font-light: #00aa00;
+    --img-filter: brightness(0) saturate(100%) invert(45%) sepia(100%) saturate(3710%) hue-rotate(96deg) brightness(100%) contrast(102%);
+    --img-filter-error: invert(16%) sepia(60%) saturate(3717%) hue-rotate(349deg) brightness(86%) contrast(120%);
+    --banner: url("../img/banner-tube-archivist-dark.png");
+    --logo: url("../img/logo-tube-archivist-dark.png");
+    --outline: 1px solid green;
+    --filter: hue-rotate(310deg);
+}
+
+.settings-group {
+    outline: var(--outline);
+}
+
+.info-box-item {
+    outline: var(--outline);
+}
+
+.footer {
+    outline: var(--outline);
+}
+
+.top-banner img {
+    filter: var(--filter);
+}
+
+.icon-text {
+    outline: var(--outline);
+}
+
+.video-item {
+    outline: var(--outline);
+}
+
+.channel-banner {
+    outline: var(--outline);
+}
+
+.description-box {
+    outline: var(--outline);
+}
+
+.video-player {
+    outline: var(--outline);
+}
+
+#notification {
+    outline: var(--outline);
+}
+
+textarea {
+    background-color: var(--highlight-bg);
+    outline: var(--outline);
+    color: var(--main-font);
+}
+
+input {
+    background-color: var(--highlight-bg);
+    color: var(--main-font);
+}

tubearchivist/static/css/midnight.css

@@ -0,0 +1,14 @@
+:root {
+    --main-bg: #000000;
+    --highlight-bg: #0c0c0c;
+    --highlight-error: #220000;
+    --highlight-error-light: #330000;
+    --highlight-bg-transparent: #0c0c0caf;
+    --main-font: #888888;
+    --accent-font-dark: #555555;
+    --accent-font-light: #999999;
+    --img-filter: invert(50%) sepia(9%) saturate(2940%) hue-rotate(122deg) brightness(94%) contrast(90%);
+    --img-filter-error: invert(16%) sepia(60%) saturate(3717%) hue-rotate(349deg) brightness(86%) contrast(120%);
+    --banner: url("../img/banner-tube-archivist-dark.png");
+    --logo: url("../img/logo-tube-archivist-dark.png");
+}

tubearchivist/static/css/style.css

@@ -165,12 +165,13 @@ button:hover {
 .top-banner img {
     width: 100%;
     max-width: 700px;
+    content: var(--banner);
 }
 
 .footer {
     margin: 0;
     padding: 20px 0;
-    background-color: var(--accent-font-dark);
+    background-color: var(--highlight-bg);
     grid-row-start: 2;
     grid-row-end: 3;
 }
@@ -179,6 +180,10 @@ button:hover {
     text-decoration: underline;
 }
 
+.footer .boxed-content {
+    text-align: center;
+}
+
 /* toggle on-off */
 .toggle {
     display: flex;
@@ -725,6 +730,7 @@ video:-webkit-full-screen {
     max-width: 200px;
     max-height: 200px;
     margin-bottom: 40px;
+    content: var(--logo);
 }
 
 .login-page form {
@@ -1099,6 +1105,15 @@ video:-webkit-full-screen {
     min-width: 300px;
 }
 
+.settings-item .agg-channel-table {
+    width: 100%;
+}
+
+.settings-item .agg-channel-right-align {
+    white-space: nowrap;
+    text-align: right;
+}
+
 .danger-zone {
     background-color: var(--highlight-error);
     color: #fff;
@@ -1316,11 +1331,6 @@ video:-webkit-full-screen {
     .playlist-nav-item img {
         width: 100%;
     }
-    .agg-channel-name {
-        min-width: 50px;
-        width: 100px;
-        max-width: 200px;
-    }
     .td, th, span, label {
         text-align: unset;
     }

tubearchivist/static/img/icon-add.svg

@@ -1,71 +1,13 @@
-<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<?xml version="1.0" encoding="utf-8"?>
-<!-- Created with Inkscape (http://www.inkscape.org/) -->
+<!-- Generator: Adobe Illustrator 27.5.0, SVG Export Plug-In . SVG Version: 6.00 Build 0)  -->
-
+<svg version="1.1"
-<svg
+	 id="svg1303" xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape" xmlns:cc="http://creativecommons.org/ns#" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:svg="http://www.w3.org/2000/svg" xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
-   xmlns:dc="http://purl.org/dc/elements/1.1/"
+	 xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px" viewBox="0 0 500 500"
-   xmlns:cc="http://creativecommons.org/ns#"
+	 style="enable-background:new 0 0 500 500;" xml:space="preserve">
-   xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
+<sodipodi:namedview  bordercolor="#666666" borderopacity="1.0" id="base" inkscape:current-layer="layer1" inkscape:cx="-97.380081" inkscape:cy="261.09215" inkscape:document-units="mm" inkscape:guide-bbox="true" inkscape:pageopacity="0.0" inkscape:pageshadow="2" inkscape:window-height="1017" inkscape:window-maximized="1" inkscape:window-width="1920" inkscape:window-x="-8" inkscape:window-y="-8" inkscape:zoom="0.85859018" pagecolor="#ffffff" showgrid="false" showguides="true" units="px">
-   xmlns:svg="http://www.w3.org/2000/svg"
+	<sodipodi:guide  id="guide1072" inkscape:locked="false" orientation="1,0" position="-221.87586,143.2945"></sodipodi:guide>
-   xmlns="http://www.w3.org/2000/svg"
+</sodipodi:namedview>
-   xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+<path d="M431.9,194.4H305.6V68.1c0-30.6-25-55.6-55.6-55.6h0c-30.6,0-55.6,25-55.6,55.6v126.3H68.1c-30.6,0-55.6,25-55.6,55.6v0
-   xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+	c0,30.6,25,55.6,55.6,55.6h126.3v126.3c0,30.6,25,55.6,55.6,55.6h0c30.6,0,55.6-25,55.6-55.6V305.6h126.3c30.6,0,55.6-25,55.6-55.6
-   width="500"
+	v0C487.5,219.4,462.5,194.4,431.9,194.4z"/>
-   height="500"
-   viewBox="0 0 132.29197 132.29167"
-   version="1.1"
-   id="svg1303"
-   inkscape:version="0.92.4 (5da689c313, 2019-01-14)"
-   sodipodi:docname="Icon_add.svg">
-  <defs
-     id="defs1297" />
-  <sodipodi:namedview
-     id="base"
-     pagecolor="#ffffff"
-     bordercolor="#666666"
-     borderopacity="1.0"
-     inkscape:pageopacity="0.0"
-     inkscape:pageshadow="2"
-     inkscape:zoom="0.85859018"
-     inkscape:cx="-97.380081"
-     inkscape:cy="261.09215"
-     inkscape:document-units="mm"
-     inkscape:current-layer="layer1"
-     showgrid="false"
-     units="px"
-     inkscape:window-width="1920"
-     inkscape:window-height="1017"
-     inkscape:window-x="-8"
-     inkscape:window-y="-8"
-     inkscape:window-maximized="1"
-     showguides="true"
-     inkscape:guide-bbox="true">
-    <sodipodi:guide
-       position="-221.87586,143.2945"
-       orientation="1,0"
-       id="guide1072"
-       inkscape:locked="false" />
-  </sodipodi:namedview>
-  <metadata
-     id="metadata1300">
-    <rdf:RDF>
-      <cc:Work
-         rdf:about="">
-        <dc:format>image/svg+xml</dc:format>
-        <dc:type
-           rdf:resource="http://purl.org/dc/dcmitype/StillImage" />
-        <dc:title></dc:title>
-      </cc:Work>
-    </rdf:RDF>
-  </metadata>
-  <g
-     inkscape:label="Ebene 1"
-     inkscape:groupmode="layer"
-     id="layer1"
-     transform="translate(0,-164.70764)">
-    <path
-       style="opacity:1;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0;stroke-linecap:round;stroke-linejoin:bevel;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;paint-order:markers fill stroke"
-       d="m 58.600542,170.62113 c -1.62283,1.61686 -2.626579,3.8573 -2.631447,6.33943 l -0.08037,43.5977 -43.597706,-0.0803 c -4.9648228,-0.009 -8.9691711,3.98046 -8.9784054,8.94536 l -0.00482,2.62846 c -0.00925,4.9649 3.9804459,8.96933 8.9452674,8.97832 l 43.597694,0.0805 -0.08027,43.59778 c -0.0093,4.96488 3.980368,8.96922 8.945263,8.9783 l 2.628471,0.005 c 4.964897,0.009 8.969245,-3.98054 8.978406,-8.94536 l 0.08035,-43.59771 43.597715,0.0803 c 4.96484,0.009 8.96917,-3.98046 8.9784,-8.94534 l 0.005,-2.62847 c 0.009,-4.96489 -3.98037,-8.96923 -8.94525,-8.97831 l -43.597784,-0.0805 0.08034,-43.59771 c 0.0093,-4.96481 -3.980379,-8.96923 -8.945267,-8.97831 l -2.628469,-0.005 c -2.482483,-0.005 -4.724106,0.98906 -6.346936,2.60592 z"
-       id="rect1073"
-       inkscape:connector-curvature="0" />
-  </g>
 </svg>

tubearchivist/static/img/icon-close.svg

@@ -1,63 +1,10 @@
-<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<?xml version="1.0" encoding="utf-8"?>
-<!-- Created with Inkscape (http://www.inkscape.org/) -->
+<!-- Generator: Adobe Illustrator 27.5.0, SVG Export Plug-In . SVG Version: 6.00 Build 0)  -->
-
+<svg version="1.1"
-<svg
+	 id="svg1303" xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape" xmlns:cc="http://creativecommons.org/ns#" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:svg="http://www.w3.org/2000/svg" xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
-   xmlns:dc="http://purl.org/dc/elements/1.1/"
+	 xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px" viewBox="0 0 500 500"
-   xmlns:cc="http://creativecommons.org/ns#"
+	 style="enable-background:new 0 0 500 500;" xml:space="preserve">
-   xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
+<path d="M473.5,392.9L329.1,250.5l144.3-145.6c21.6-21.6,21.6-57,0-78.6l0,0c-21.6-21.6-57-21.6-78.6,0L250.5,171.9L106.3,26.2
-   xmlns:svg="http://www.w3.org/2000/svg"
+	c-21.6-21.6-57-21.6-78.6,0l0,0C6,47.9,6,83.2,27.6,104.9l144.2,145.6L26.4,393.9c-21.6,21.6-21.6,57,0,78.6l0,0
-   xmlns="http://www.w3.org/2000/svg"
+	c21.6,21.6,57,21.6,78.6,0l145.4-143.4l144.3,142.4c21.6,21.6,57,21.6,78.6,0l0,0C495.1,449.9,495.1,414.6,473.5,392.9z"/>
-   xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
-   xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
-   width="500"
-   height="500"
-   viewBox="0 0 132.29197 132.29167"
-   version="1.1"
-   id="svg1303"
-   inkscape:version="0.92.4 (5da689c313, 2019-01-14)"
-   sodipodi:docname="Icons_close.svg">
-  <defs
-     id="defs1297" />
-  <sodipodi:namedview
-     id="base"
-     pagecolor="#ffffff"
-     bordercolor="#666666"
-     borderopacity="1.0"
-     inkscape:pageopacity="0.0"
-     inkscape:pageshadow="2"
-     inkscape:zoom="1.4291625"
-     inkscape:cx="122.66624"
-     inkscape:cy="202.38142"
-     inkscape:document-units="mm"
-     inkscape:current-layer="layer1"
-     showgrid="false"
-     units="px"
-     inkscape:window-width="1920"
-     inkscape:window-height="1017"
-     inkscape:window-x="-8"
-     inkscape:window-y="-8"
-     inkscape:window-maximized="1" />
-  <metadata
-     id="metadata1300">
-    <rdf:RDF>
-      <cc:Work
-         rdf:about="">
-        <dc:format>image/svg+xml</dc:format>
-        <dc:type
-           rdf:resource="http://purl.org/dc/dcmitype/StillImage" />
-        <dc:title />
-      </cc:Work>
-    </rdf:RDF>
-  </metadata>
-  <g
-     inkscape:label="Ebene 1"
-     inkscape:groupmode="layer"
-     id="layer1"
-     transform="translate(0,-164.70764)">
-    <path
-       style="opacity:1;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0;stroke-linecap:round;stroke-linejoin:bevel;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;paint-order:markers fill stroke"
-       d="m 115.5244,167.09017 c -3.04348,0 -6.08905,1.16742 -8.42111,3.49893 L 66.146003,211.54641 25.18874,170.5891 c -4.664143,-4.66413 -12.173961,-4.66413 -16.8382429,0 l -2.4692132,2.46935 c -4.664282,4.66413 -4.664282,12.17408 0,16.83807 l 40.9572491,40.9573 -40.9572491,40.95745 c -4.664282,4.66412 -4.664282,12.17393 0,16.83806 l 2.4692132,2.46935 c 4.6642819,4.66413 12.1740999,4.66413 16.8382429,0 l 40.957263,-40.9573 40.957287,40.9573 c 4.66413,4.66413 12.17393,4.66413 16.8382,0 l 2.46921,-2.46935 c 4.66427,-4.66413 4.66427,-12.17394 0,-16.83806 L 85.453463,230.85382 126.4107,189.89652 c 4.66427,-4.66399 4.66427,-12.17394 0,-16.83807 l -2.46921,-2.46935 c -2.33221,-2.33206 -5.37361,-3.49893 -8.41709,-3.49893 z"
-       id="rect1073"
-       inkscape:connector-curvature="0" />
-  </g>
 </svg>

tubearchivist/static/img/icon-download.svg

@@ -1,63 +1,14 @@
-<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<?xml version="1.0" encoding="utf-8"?>
-<!-- Created with Inkscape (http://www.inkscape.org/) -->
+<!-- Generator: Adobe Illustrator 27.5.0, SVG Export Plug-In . SVG Version: 6.00 Build 0)  -->
-
+<svg version="1.1"
-<svg
+	 id="svg1303" xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape" xmlns:cc="http://creativecommons.org/ns#" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:svg="http://www.w3.org/2000/svg" xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
-   xmlns:dc="http://purl.org/dc/elements/1.1/"
+	 xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px" viewBox="0 0 500 500"
-   xmlns:cc="http://creativecommons.org/ns#"
+	 style="enable-background:new 0 0 500 500;" xml:space="preserve">
-   xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
+<g>
-   xmlns:svg="http://www.w3.org/2000/svg"
+	<path d="M231.3,382c4.8,5.3,11.6,8.3,18.7,8.3c7.1,0,13.9-3,18.7-8.3l112.1-124.2c6.7-7.4,8.4-18,4.3-27.1
-   xmlns="http://www.w3.org/2000/svg"
+		c-4-9.1-13.1-14.9-23-14.9h-52.9V64.9c0-13.9-11.3-25.2-25.2-25.2h-68c-13.9,0-25.2,11.3-25.2,25.2v150.9h-52.9
-   xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+		c-10,0-19,5.9-23,14.9c-4,9.1-2.3,19.7,4.3,27.1L231.3,382z"/>
-   xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+	<path d="M436.1,408.8H63.9c-13.6,0-24.7,11.1-24.7,24.7c0,13.6,11.1,24.7,24.7,24.7h372.2c13.6,0,24.7-11.1,24.7-24.7
-   width="500"
+		C460.8,419.9,449.7,408.8,436.1,408.8z"/>
-   height="500"
+</g>
-   viewBox="0 0 132.29197 132.29167"
-   version="1.1"
-   id="svg1303"
-   inkscape:version="0.92.4 (5da689c313, 2019-01-14)"
-   sodipodi:docname="Download.svg">
-  <defs
-     id="defs1297" />
-  <sodipodi:namedview
-     id="base"
-     pagecolor="#ffffff"
-     bordercolor="#666666"
-     borderopacity="1.0"
-     inkscape:pageopacity="0.0"
-     inkscape:pageshadow="2"
-     inkscape:zoom="0.71458126"
-     inkscape:cx="-376.84122"
-     inkscape:cy="260.65826"
-     inkscape:document-units="mm"
-     inkscape:current-layer="layer1"
-     showgrid="false"
-     units="px"
-     inkscape:window-width="1920"
-     inkscape:window-height="1017"
-     inkscape:window-x="-8"
-     inkscape:window-y="-8"
-     inkscape:window-maximized="1" />
-  <metadata
-     id="metadata1300">
-    <rdf:RDF>
-      <cc:Work
-         rdf:about="">
-        <dc:format>image/svg+xml</dc:format>
-        <dc:type
-           rdf:resource="http://purl.org/dc/dcmitype/StillImage" />
-        <dc:title></dc:title>
-      </cc:Work>
-    </rdf:RDF>
-  </metadata>
-  <g
-     inkscape:label="Ebene 1"
-     inkscape:groupmode="layer"
-     id="layer1"
-     transform="translate(0,-164.70764)">
-    <path
-       style="opacity:1;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0;stroke-linecap:round;stroke-linejoin:bevel;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;paint-order:markers fill stroke"
-       d="M 249.89062 39.6875 L 249.88477 39.693359 C 229.11581 39.693359 212.39453 59.135896 212.39453 83.289062 L 212.39453 289.66016 L 159.20312 236.46875 L 127.63867 268.05469 L 218.74805 359.14258 L 218.73242 359.1582 L 249.93555 390.33789 L 281.52148 358.77344 L 372.36133 267.91211 L 341.1582 236.73047 L 287.37891 290.50977 L 287.37891 83.283203 C 287.37891 59.130339 270.65971 39.6875 249.89062 39.6875 z M 50.566406 407.88867 C 44.284822 407.88867 39.228516 412.94498 39.228516 419.22656 L 39.228516 444.62695 C 39.228516 450.90854 44.284822 455.96484 50.566406 455.96484 L 449.43359 455.96484 C 455.71518 455.96484 460.77148 450.90854 460.77148 444.62695 L 460.77148 419.22656 C 460.77148 412.94498 455.71518 407.88867 449.43359 407.88867 L 50.566406 407.88867 z "
-       transform="matrix(0.26458394,0,0,0.26458394,0,164.70749)"
-       id="rect1208" />
-  </g>
 </svg>

tubearchivist/static/img/icon-exit.svg

@@ -1,75 +1,19 @@
-<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<?xml version="1.0" encoding="utf-8"?>
-<!-- Created with Inkscape (http://www.inkscape.org/) -->
+<!-- Generator: Adobe Illustrator 27.5.0, SVG Export Plug-In . SVG Version: 6.00 Build 0)  -->
-
+<svg version="1.1"
-<svg
+	 id="svg1566" xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape" xmlns:cc="http://creativecommons.org/ns#" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:svg="http://www.w3.org/2000/svg" xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
-   xmlns:dc="http://purl.org/dc/elements/1.1/"
+	 xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px" viewBox="0 0 595.3 595.3"
-   xmlns:cc="http://creativecommons.org/ns#"
+	 style="enable-background:new 0 0 595.3 595.3;" xml:space="preserve">
-   xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
+<g>
-   xmlns:svg="http://www.w3.org/2000/svg"
+	<path d="M99,295.1c0,105.5,75.4,194.1,177.1,209.1c68.9,10.1,128.8-9.3,179.1-57.3c14.4-13.7,30.2-19,49-11.3
-   xmlns="http://www.w3.org/2000/svg"
+		c27.8,11.3,35.3,47.2,14.3,68.5c-43.9,44.6-96.7,72.7-158.4,83.6c-65,11.4-127.4,2.9-185.7-27.8c-85.2-44.9-138.5-115-155.9-210.1
-   xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+		C2.5,261.9,23,182,78.9,112.4C129,50.1,194.6,13.3,274.5,4.6c94.2-10.3,175.3,18.6,243,84.9c13.3,13,16.6,31.3,9.4,47.7
-   xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+		c-7.1,16-23.4,26.2-40.9,25.4c-11.6-0.6-21.3-5.5-29.6-13.7c-28.7-28.3-62.6-47.1-102-56.1c-117-26.9-234.8,53.1-252.9,171.9
-   width="210mm"
+		C99.9,275.4,98.7,286.2,99,295.1z"/>
-   height="210mm"
+	<path d="M434.1,339.9c-2.6,0-5.2,0-7.8,0c-69,0-138.1,0-207.1,0c-19.1,0-34.3-10.6-40.8-28.2c-6-16.2-1.2-34.6,12.4-46.4
-   viewBox="0 0 210 210"
+		c8-7,17.5-10.2,28.3-10.2c69,0.1,138.1,0,207.1,0c2.6,0,5.2,0,8.7,0c-2.2-4.7-5.9-7-8.7-10.1c-11.1-12.6-15.3-26.9-9.9-43.1
-   version="1.1"
+		c5.3-15.8,16.6-25.5,32.9-28.6c14.2-2.7,27.1,1.3,37.3,11.4c28,27.6,55.9,55.3,83.4,83.5c16.1,16.5,16.1,42.2,0.1,58.7
-   id="svg1566"
+		c-27.6,28.3-55.6,56.3-83.9,83.8c-12.7,12.3-28.4,15.4-44.9,8.8c-16.7-6.7-25.9-19.8-27.1-37.7c-0.8-12,3.7-22.7,12.1-31.5
-   inkscape:version="0.92.4 (5da689c313, 2019-01-14)"
+		c2.8-3,6.1-5.5,9.2-8.3C435,341.3,434.5,340.6,434.1,339.9z"/>
-   sodipodi:docname="Icons_exit 05.svg">
+</g>
-  <defs
-     id="defs1560" />
-  <sodipodi:namedview
-     id="base"
-     pagecolor="#ffffff"
-     bordercolor="#666666"
-     borderopacity="1.0"
-     inkscape:pageopacity="0.0"
-     inkscape:pageshadow="2"
-     inkscape:zoom="0.35355339"
-     inkscape:cx="963.7258"
-     inkscape:cy="291.01609"
-     inkscape:document-units="mm"
-     inkscape:current-layer="layer1"
-     showgrid="false"
-     inkscape:window-width="1920"
-     inkscape:window-height="1009"
-     inkscape:window-x="-8"
-     inkscape:window-y="-8"
-     inkscape:window-maximized="1" />
-  <metadata
-     id="metadata1563">
-    <rdf:RDF>
-      <cc:Work
-         rdf:about="">
-        <dc:format>image/svg+xml</dc:format>
-        <dc:type
-           rdf:resource="http://purl.org/dc/dcmitype/StillImage" />
-        <dc:title></dc:title>
-      </cc:Work>
-    </rdf:RDF>
-  </metadata>
-  <g
-     inkscape:label="Ebene 1"
-     inkscape:groupmode="layer"
-     id="layer1"
-     transform="translate(0,-87)">
-    <path
-       style="opacity:1;fill:#000000;fill-opacity:1;stroke:none;stroke-width:2.35654187;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;paint-order:markers fill stroke"
-       d="M 106.49932,87.901069 C 49.504302,87.900974 3.3006913,134.10459 3.3007713,191.0996 c 0,0.30098 0.003,0.60131 0.005,0.90167 v 0 c -0.003,0.29952 -0.006,0.59901 -0.006,0.89912 -8e-5,56.99502 46.2035307,103.19865 103.1985287,103.19854 23.01714,-0.0773 45.34783,-7.84709 63.44155,-22.07425 0,0 9.01874,-8.71006 2.40579,-16.41737 -6.61297,-7.70731 -19.11222,0.3185 -19.11222,0.3185 -13.60985,9.81394 -29.95596,15.11012 -46.73512,15.14236 -44.275428,0 -80.167758,-35.89234 -80.167758,-80.16778 0,-0.30097 0.003,-0.60148 0.006,-0.90166 h -5.2e-4 c -0.003,-0.29934 -0.006,-0.59901 -0.006,-0.89913 0,-44.27545 35.89234,-80.16777 80.167778,-80.16777 16.77916,0.0322 33.12527,5.32843 46.73512,15.14236 0,0 12.49925,8.02581 19.11222,0.3185 6.61295,-7.70732 -2.4058,-16.41739 -2.4058,-16.41739 C 151.84561,95.74815 129.51494,87.97828 106.4978,87.901069 Z m 54.30959,56.450221 -12.13663,11.69622 20.15864,20.93332 -93.932488,-1.4899 c -9.22763,-0.17349 -16.77655,6.07423 -16.92587,14.00904 l 0.002,0.002 c -0.0149,1.82673 -0.0235,3.40102 0,4.99598 l -0.002,0.002 c 0.14932,7.93483 7.69824,14.18254 16.92587,14.00905 l 93.932488,-1.48991 -20.15864,20.93333 12.13663,11.69622 34.0585,-35.35536 11.82982,-12.29208 h 0.003 l -9.9e-4,-0.002 9.9e-4,-9.9e-4 h -0.003 l -11.82982,-12.29208 z"
-       id="path1405"
-       inkscape:connector-curvature="0"
-       sodipodi:nodetypes="cccccccsccsccsccscccccccccccccccccccccc" />
-    <path
-       style="opacity:1;fill:#000000;fill-opacity:1;stroke:none;stroke-width:2.39729571;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;paint-order:markers fill stroke"
-       d="m 506.57967,92.503023 c -57.98068,-1e-4 -104.98336,47.002567 -104.98326,104.983257 1.9e-4,57.98049 47.00276,104.98284 104.98326,104.98273 23.42489,-0.0758 46.15146,-7.98387 57.83458,-18.08923 11.68313,-10.10537 12.15613,-18.62993 7.38675,-23.04107 v -0.002 c -4.7711,-4.41269 -12.38099,-1.9587 -17.69245,2.25103 -13.83538,9.99805 -30.45915,15.40285 -47.52888,15.4528 -45.04116,0 -81.55421,-36.51305 -81.5542,-81.55419 0,-45.04114 36.51307,-81.5542 81.5542,-81.5542 17.06933,0.0328 33.21884,5.19482 43.16812,12.86758 9.94929,7.67275 17.33418,9.17607 22.1053,4.76338 v -0.002 c 4.77116,-4.41278 5.55882,-12.9887 -0.73482,-18.60197 -18.40654,-14.47308 -41.1234,-22.377337 -64.5386,-22.455877 z m 55.24881,57.426467 -12.34652,11.8985 20.50728,21.29534 -95.55697,-1.51567 c -9.38721,-0.17649 -17.06669,6.17929 -17.21858,14.25133 l 0.003,0.002 c -0.15192,8.07203 7.28245,14.71295 16.66978,14.88953 l 95.22519,1.50947 -21.06332,20.28455 12.04579,12.49846 36.06808,-34.74464 0.005,0.005 12.34654,-11.89954 -12.03701,-12.50724 z m 35.17874,98.71801 0.69918,0.67386 c 0.13539,-0.22412 0.26991,-0.44874 0.4036,-0.67386 z"
-       id="path1405-6"
-       inkscape:connector-curvature="0"
-       sodipodi:nodetypes="ccccccccsczccccccccccccccccccccccc" />
-    <path
-       style="opacity:1;fill:#000000;fill-opacity:1;stroke:none;stroke-width:2.39729571;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;paint-order:markers fill stroke"
-       d="m 740.89945,94.730897 c -57.98068,-9.6e-5 -104.98334,47.002563 -104.98325,104.983253 1.9e-4,57.98049 47.00276,104.98284 104.98325,104.98274 23.42488,-0.0758 46.15145,-7.98387 64.5635,-22.46581 l -17.03461,-16.41553 c -13.83537,9.99805 -30.45916,15.40285 -47.52889,15.4528 -45.04113,0 -81.55419,-36.51306 -81.55419,-81.5542 0,-45.04114 36.51306,-81.55419 81.55419,-81.55419 17.06934,0.0328 33.69814,5.42058 47.54336,15.40423 l 16.99534,-16.3773 c -18.40663,-14.4732 -41.12349,-22.377447 -64.5387,-22.455993 z m 55.24882,57.426473 -12.34653,11.8985 20.50728,21.29534 -95.55696,-1.51567 c -9.38721,-0.17649 -17.06668,6.17928 -17.21858,14.25132 l 0.002,0.002 c -0.1519,8.07203 7.28245,14.71295 16.66978,14.88953 l 95.22519,1.50947 -21.06332,20.28455 12.04578,12.49846 36.06808,-34.74465 0.005,0.005 12.34653,-11.89953 -12.03699,-12.50725 z m 35.17873,98.718 0.69919,0.67386 c 0.13538,-0.22412 0.26991,-0.44874 0.40359,-0.67386 z"
-       id="path1405-9"
-       inkscape:connector-curvature="0"
-       sodipodi:nodetypes="ccccccsccccccccccccccccccccccc" />
-  </g>
 </svg>

tubearchivist/static/img/icon-eye.svg

@@ -1,17 +1,15 @@
 <?xml version="1.0" encoding="utf-8"?>
-<!-- Generator: Adobe Illustrator 26.5.0, SVG Export Plug-In . SVG Version: 6.00 Build 0)  -->
+<!-- Generator: Adobe Illustrator 27.5.0, SVG Export Plug-In . SVG Version: 6.00 Build 0)  -->
 <svg version="1.1"
-	 id="svg1303" xmlns:cc="http://creativecommons.org/ns#" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape" xmlns:svg="http://www.w3.org/2000/svg" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd" inkscape:version="0.92.4 (5da689c313, 2019-01-14)" sodipodi:docname="Icons_seen.svg"
+	 id="svg1303" xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape" xmlns:cc="http://creativecommons.org/ns#" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:svg="http://www.w3.org/2000/svg" xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
 	 xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px" viewBox="0 0 500 500"
 	 style="enable-background:new 0 0 500 500;" xml:space="preserve">
-<sodipodi:namedview  bordercolor="#666666" borderopacity="1.0" id="base" inkscape:current-layer="layer1" inkscape:cx="84.208758" inkscape:cy="136.94831" inkscape:document-units="mm" inkscape:pageopacity="0.0" inkscape:pageshadow="2" inkscape:window-height="1017" inkscape:window-maximized="1" inkscape:window-width="1920" inkscape:window-x="-8" inkscape:window-y="-8" inkscape:zoom="1.0105705" pagecolor="#ffffff" showgrid="false" units="px">
+<g>
-	</sodipodi:namedview>
+	<path d="M249.8,406.5C142.2,401.9,59.2,355.3,3.6,261.9c-4.7-8-4.8-16.2-0.1-24.2c46.6-79.1,115-127.3,205.9-141.1
-<g id="layer1" transform="translate(0,-164.70764)" inkscape:groupmode="layer" inkscape:label="Ebene 1">
+		c113.5-17.1,224.7,36.5,283.5,134.6c1,1.7,1.9,3.5,3,5.2c5.6,8.8,5.4,17.5,0.1,26.5c-29,50.1-69.2,88.2-121,113.9
-	<path id="path1091" inkscape:connector-curvature="0" d="M250,259.4c-105.2,0.1-201.5,60-250,155.4C48.5,510.1,144.9,569.9,250,570
+		c-33.4,16.6-68.7,26.2-106,28.5C262.7,405.7,256.4,406.2,249.8,406.5z M139.6,249.7c0,61.4,49.1,110.5,110.6,110.5
-		c105.2-0.1,201.5-60,250-155.4C451.5,319.3,355.1,259.5,250,259.4z M230.7,287.8c-18.7,2.8-36.6,9.8-52.4,20.3
+		c61.2,0,110.3-49.1,110.4-110.2c0.1-61.5-48.9-110.7-110.4-110.7C188.7,139.2,139.6,188.3,139.6,249.7z"/>
-		c2.8-0.5,5.7-0.7,8.6-0.7c27.8,0,50.4,22.6,50.4,50.4c0,0,0,0,0,0c0,27.8-22.6,50.4-50.4,50.4c0,0,0,0,0,0
+	<path d="M174.9,249.8c0-42.1,33.2-75.3,75.4-75.2c41.7,0,75,33.3,75,75.2c0,42.1-33.3,75.3-75.5,75.3
-		c-27.8,0-50.4-22.6-50.4-50.4c0,0,0,0,0,0c0-1.2,0.1-2.4,0.2-3.6c-10,18.6-15.2,39.4-15.2,60.5c0,62.6,45,116,106.7,126.7
+		C208.1,324.9,174.9,291.6,174.9,249.8z"/>
-		c-78.3-7.6-147.6-55.2-183.9-126.6C81,342.6,151.3,294.7,230.7,287.8L230.7,287.8z M271.8,288c78.3,7.6,147.6,55.2,183.9,126.6
-		c-36.7,72.1-106.9,120-186.2,127c62.6-9.7,108.9-63.5,108.9-126.9C378.5,352.1,333.5,298.7,271.8,288L271.8,288z"/>
 </g>
 </svg>

tubearchivist/static/img/icon-play.svg

@@ -1,75 +1,13 @@
-<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<?xml version="1.0" encoding="utf-8"?>
-<!-- Created with Inkscape (http://www.inkscape.org/) -->
+<!-- Generator: Adobe Illustrator 27.5.0, SVG Export Plug-In . SVG Version: 6.00 Build 0)  -->
-
+<svg version="1.1"
-<svg
+	 id="svg1303" xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape" xmlns:cc="http://creativecommons.org/ns#" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:svg="http://www.w3.org/2000/svg" xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
-   xmlns:dc="http://purl.org/dc/elements/1.1/"
+	 xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px" viewBox="0 0 500 500"
-   xmlns:cc="http://creativecommons.org/ns#"
+	 style="enable-background:new 0 0 500 500;" xml:space="preserve">
-   xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
+<g>
-   xmlns:svg="http://www.w3.org/2000/svg"
+	<path d="M175.5,421.6c-4,0-8,0-12.1,0c-13.9-3.4-21-13-24.1-26.3c-1-4.5-1.5-9-1.5-13.6c0-87.7,0-175.4,0-263.1
-   xmlns="http://www.w3.org/2000/svg"
+		c0-5.5,0.6-10.9,2.2-16.2c3.7-12.5,11-21.5,24.5-23.6c9-1.4,17.6,0.7,25.5,5c10.9,5.9,21.6,12.1,32.4,18.2
-   xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+		c66.6,38.2,133.2,76.4,199.8,114.6c7.1,4.1,13.7,8.7,18.8,15.4c8.7,11.5,8.7,25,0,36.5c-5,6.6-11.7,11.3-18.8,15.4
-   xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+		c-74,42.5-148.1,84.9-222.1,127.4C192.2,415.7,184.4,419.8,175.5,421.6z"/>
-   width="500"
+</g>
-   height="500"
-   viewBox="0 0 132.29197 132.29167"
-   version="1.1"
-   id="svg1303"
-   inkscape:version="0.92.4 (5da689c313, 2019-01-14)"
-   sodipodi:docname="Icons_play.svg">
-  <defs
-     id="defs1297" />
-  <sodipodi:namedview
-     id="base"
-     pagecolor="#ffffff"
-     bordercolor="#666666"
-     borderopacity="1.0"
-     inkscape:pageopacity="0.0"
-     inkscape:pageshadow="2"
-     inkscape:zoom="3.4343607"
-     inkscape:cx="51.709313"
-     inkscape:cy="230.85077"
-     inkscape:document-units="mm"
-     inkscape:current-layer="layer1"
-     showgrid="false"
-     units="px"
-     inkscape:window-width="1920"
-     inkscape:window-height="1017"
-     inkscape:window-x="-8"
-     inkscape:window-y="-8"
-     inkscape:window-maximized="1" />
-  <metadata
-     id="metadata1300">
-    <rdf:RDF>
-      <cc:Work
-         rdf:about="">
-        <dc:format>image/svg+xml</dc:format>
-        <dc:type
-           rdf:resource="http://purl.org/dc/dcmitype/StillImage" />
-        <dc:title />
-      </cc:Work>
-    </rdf:RDF>
-  </metadata>
-  <g
-     inkscape:label="Ebene 1"
-     inkscape:groupmode="layer"
-     id="layer1"
-     transform="translate(0,-164.70764)">
-    <path
-       sodipodi:type="star"
-       style="opacity:1;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0;stroke-linecap:round;stroke-linejoin:bevel;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;paint-order:markers fill stroke"
-       id="path1206"
-       sodipodi:sides="3"
-       sodipodi:cx="-142.60593"
-       sodipodi:cy="272.75879"
-       sodipodi:r1="30.817924"
-       sodipodi:r2="15.408962"
-       sodipodi:arg1="0.52359878"
-       sodipodi:arg2="1.5707963"
-       inkscape:flatsided="false"
-       inkscape:rounded="-3.46945e-18"
-       inkscape:randomized="0"
-       d="m -115.91682,288.16775 -26.68911,0 -26.6891,0 13.34455,-23.11344 13.34455,-23.11345 13.34456,23.11345 z"
-       transform="matrix(0,1.7014617,-1.7014617,0,530.04498,473.492)"
-       inkscape:transform-center-x="-18.792756" />
-  </g>
 </svg>

tubearchivist/static/img/icon-rescan.svg

@@ -1,63 +1,23 @@
-<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<?xml version="1.0" encoding="utf-8"?>
-<!-- Created with Inkscape (http://www.inkscape.org/) -->
+<!-- Generator: Adobe Illustrator 27.5.0, SVG Export Plug-In . SVG Version: 6.00 Build 0)  -->
-
+<svg version="1.1"
-<svg
+	 id="svg1303" xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape" xmlns:cc="http://creativecommons.org/ns#" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:svg="http://www.w3.org/2000/svg" xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
-   xmlns:dc="http://purl.org/dc/elements/1.1/"
+	 xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px" viewBox="0 0 500 500"
-   xmlns:cc="http://creativecommons.org/ns#"
+	 style="enable-background:new 0 0 500 500;" xml:space="preserve">
-   xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
+<sodipodi:namedview  bordercolor="#666666" borderopacity="1.0" id="base" inkscape:current-layer="layer1" inkscape:cx="125.63373" inkscape:cy="156.7016" inkscape:document-units="mm" inkscape:pageopacity="0.0" inkscape:pageshadow="2" inkscape:window-height="1017" inkscape:window-maximized="1" inkscape:window-width="1920" inkscape:window-x="-8" inkscape:window-y="-8" inkscape:zoom="0.85859018" pagecolor="#ffffff" showgrid="false" units="px">
-   xmlns:svg="http://www.w3.org/2000/svg"
+	</sodipodi:namedview>
-   xmlns="http://www.w3.org/2000/svg"
+<g>
-   xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+	<path d="M269,28.7c10.8,1.6,21.6,2.8,32.3,5.3c38.3,9.2,71.9,27.2,100.7,54.1c3.4,3.2,4.7,2.4,6.8-1.1c3.3-5.5,7.1-10.9,10.7-16.2
-   xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+		c4.8-7,11.3-9.7,18.5-7.6c6.8,1.9,11.5,8.2,11.5,16.5c-0.1,26.3,0.3,52.7-0.5,79c-0.4,12.5-0.1,25-0.2,37.6
-   width="500"
+		c-0.1,14-11.1,21.3-23.9,15.6c-35.2-15.7-70.3-31.6-105.5-47.4c-7.7-3.5-11.8-10.1-10.8-16.9c1.1-7.8,6.9-13.2,15.6-14.4
-   height="500"
+		c4-0.6,8-1.2,11.9-1.8c0.9-0.1,2.3,0.2,2.6-1.1c0.3-1.2-1-1.6-1.8-2.2c-12.2-8.9-25.5-15.7-39.8-20.3
-   viewBox="0 0 132.29197 132.29167"
+		c-72.7-23.2-148.9,9.9-181.6,78.9c-4.1,8.7-8.3,17-17.1,22.1c-13.7,8-30.1,6.6-42.2-3.8c-11.3-9.6-15.6-26.6-9.6-40.5
-   version="1.1"
+		c15.7-36.9,39.4-67.7,71.6-91.8c29.8-22.4,63.2-36.3,100-41.9c4.8-0.7,9.7-1.3,14.5-2C244.8,28.7,256.9,28.7,269,28.7z"/>
-   id="svg1303"
+	<path d="M52.1,361.3c0.6-20,0.8-39,0.8-58c0-14,11.1-21.2,23.9-15.5c25.2,11.2,50.4,22.6,75.6,33.9c10.1,4.5,20.2,9,30.3,13.7
-   inkscape:version="0.92.4 (5da689c313, 2019-01-14)"
+		c6.4,2.9,10.5,7.8,10.5,15c0,6.4-3.2,11.2-9,14.2c-3.8,2-8.1,2-12.1,2.8c-2.1,0.4-4.3,0.6-6.4,1c-0.9,0.1-2.3-0.2-2.6,1.1
-   sodipodi:docname="Rescan Subscriptions.svg">
+		c-0.3,1.2,1,1.6,1.8,2.2c12.2,8.9,25.5,15.7,39.8,20.3c72.7,23.2,148.9-9.9,181.6-78.9c4.2-8.8,8.5-17.3,17.4-22.4
-  <defs
+		c13.6-7.7,29.8-6.3,41.8,4c11.3,9.6,15.6,26.6,9.7,40.5c-15.7,36.9-39.4,67.7-71.6,91.8c-29.7,22.3-62.9,36.5-99.6,41.8
-     id="defs1297" />
+		c-57.8,8.3-110.9-4-159.1-36.9c-9.3-6.3-17.9-13.5-26.1-21.2c-2.1-2-3.3-2.1-4.9,0.5c-3.4,5.5-7.1,10.8-10.7,16.2
-  <sodipodi:namedview
+		c-5.7,8.5-12.2,11.4-19.9,8.9c-7.3-2.3-11.1-8.6-11.1-18.5C52.1,398.6,52.1,379.4,52.1,361.3z"/>
-     id="base"
+</g>
-     pagecolor="#ffffff"
-     bordercolor="#666666"
-     borderopacity="1.0"
-     inkscape:pageopacity="0.0"
-     inkscape:pageshadow="2"
-     inkscape:zoom="0.85859018"
-     inkscape:cx="125.63373"
-     inkscape:cy="156.7016"
-     inkscape:document-units="mm"
-     inkscape:current-layer="layer1"
-     showgrid="false"
-     units="px"
-     inkscape:window-width="1920"
-     inkscape:window-height="1017"
-     inkscape:window-x="-8"
-     inkscape:window-y="-8"
-     inkscape:window-maximized="1" />
-  <metadata
-     id="metadata1300">
-    <rdf:RDF>
-      <cc:Work
-         rdf:about="">
-        <dc:format>image/svg+xml</dc:format>
-        <dc:type
-           rdf:resource="http://purl.org/dc/dcmitype/StillImage" />
-        <dc:title />
-      </cc:Work>
-    </rdf:RDF>
-  </metadata>
-  <g
-     inkscape:label="Ebene 1"
-     inkscape:groupmode="layer"
-     id="layer1"
-     transform="translate(0,-164.70764)">
-    <path
-       style="opacity:1;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0;stroke-linecap:round;stroke-linejoin:bevel;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;paint-order:markers fill stroke"
-       d="M 240.35352 18.810547 L 240.35352 29.246094 A 221.29301 221.29301 0 0 0 64.912109 129.45898 L 95.701172 150.14453 A 184.02261 184.02261 0 0 1 240.35352 66.369141 L 240.35352 87.705078 L 270.18359 70.480469 L 300.01562 53.257812 L 270.18359 36.035156 L 240.35352 18.810547 z M 312.28516 37.798828 L 301.08398 73.353516 A 184.02261 184.02261 0 0 1 422.6875 187.33594 L 406.34375 192.03125 L 431.13281 215.94922 L 455.92188 239.86719 L 464.24219 206.44141 L 472.56055 173.01367 L 458.55078 177.03711 A 221.29301 221.29301 0 0 0 312.28516 37.798828 z M 73.375 144.88281 L 48.585938 168.80078 L 23.796875 192.71875 L 35.617188 196.11328 A 221.29301 221.29301 0 0 0 28.707031 250 A 221.29301 221.29301 0 0 0 77.992188 388.9375 L 107.99805 366.83789 A 184.02261 184.02261 0 0 1 65.976562 250 A 184.02261 184.02261 0 0 1 71.474609 206.41211 L 90.013672 211.73633 L 81.693359 178.30859 L 73.375 144.88281 z M 433.95312 248.4375 A 184.02261 184.02261 0 0 1 434.02344 250 A 184.02261 184.02261 0 0 1 367.40625 391.49219 L 358.92773 380.14844 L 345.34375 411.80469 L 331.75977 443.45898 L 365.9668 439.39648 L 400.17188 435.33203 L 389.71484 421.33984 A 221.29301 221.29301 0 0 0 471.29297 250 A 221.29301 221.29301 0 0 0 471.23047 248.77539 L 433.95312 248.4375 z M 94.9375 385.80273 L 108.97852 417.25781 L 123.02148 448.71289 L 131.60156 436.87891 A 221.29301 221.29301 0 0 0 250 471.29297 A 221.29301 221.29301 0 0 0 328.64648 456.57227 L 315.99023 421.74805 A 184.02261 184.02261 0 0 1 250 434.02344 A 184.02261 184.02261 0 0 1 153.63867 406.48047 L 163.45898 392.93555 L 129.19727 389.36914 L 94.9375 385.80273 z "
-       transform="matrix(0.26458394,0,0,0.26458394,0,164.70749)"
-       id="path826" />
-  </g>
 </svg>

tubearchivist/static/img/icon-search.svg

@@ -1,64 +1,15 @@
-<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<?xml version="1.0" encoding="utf-8"?>
-<!-- Created with Inkscape (http://www.inkscape.org/) -->
+<!-- Generator: Adobe Illustrator 27.5.0, SVG Export Plug-In . SVG Version: 6.00 Build 0)  -->
-
+<svg version="1.1"
-<svg
+	 id="svg1303" xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape" xmlns:cc="http://creativecommons.org/ns#" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:svg="http://www.w3.org/2000/svg" xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
-   xmlns:dc="http://purl.org/dc/elements/1.1/"
+	 xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px" viewBox="0 0 500 500"
-   xmlns:cc="http://creativecommons.org/ns#"
+	 style="enable-background:new 0 0 500 500;" xml:space="preserve">
-   xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
+<g>
-   xmlns:svg="http://www.w3.org/2000/svg"
+	<path d="M492.4,462.2c-1,2.5-1.8,5.1-3,7.6C478,495,448.2,501.9,427.2,484c-1.8-1.5-3.5-3.2-5.2-4.9
-   xmlns="http://www.w3.org/2000/svg"
+		c-34.1-34.8-68.3-69.5-102.3-104.4c-3.3-3.4-5.6-3.6-9.5-1c-30.5,20.3-64.1,31.1-100.6,32c-60.1,1.5-110.4-20.8-151.1-65.6
-   xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+		c-28.2-31-45.1-67.8-49.5-109.8C1.6,161.2,23.6,102.8,74.3,56.1c28.7-26.4,62.7-42.4,101-47.9c52.7-7.6,101,4.5,143.9,36.5
-   xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+		c43.7,32.6,70,76.6,78.4,131c7.1,45.8-1.1,89.2-23.7,129.6c-3.7,6.6-3.8,6.6,1.5,11.9c33.3,34,66.6,68,100.1,101.9
-   width="500"
+		c7.6,7.7,14.2,15.9,16.9,26.7C494.1,451.5,493.6,456.9,492.4,462.2z M335.2,205.9c0.5-72.8-58.3-134.2-131.9-134
-   height="500"
+		C132.9,72,72,129.3,72,206c0,74.8,58.1,134.2,131.7,134.2C278.8,340.2,335.8,278.1,335.2,205.9z"/>
-   viewBox="0 0 132.29197 132.29167"
+</g>
-   version="1.1"
-   id="svg1303"
-   inkscape:version="0.92.4 (5da689c313, 2019-01-14)"
-   sodipodi:docname="Icons_Lupe.svg">
-  <defs
-     id="defs1297" />
-  <sodipodi:namedview
-     id="base"
-     pagecolor="#ffffff"
-     bordercolor="#666666"
-     borderopacity="1.0"
-     inkscape:pageopacity="0.0"
-     inkscape:pageshadow="2"
-     inkscape:zoom="1.0105705"
-     inkscape:cx="-23.955221"
-     inkscape:cy="207.58555"
-     inkscape:document-units="mm"
-     inkscape:current-layer="layer1"
-     showgrid="false"
-     units="px"
-     inkscape:window-width="1920"
-     inkscape:window-height="1017"
-     inkscape:window-x="-8"
-     inkscape:window-y="-8"
-     inkscape:window-maximized="1" />
-  <metadata
-     id="metadata1300">
-    <rdf:RDF>
-      <cc:Work
-         rdf:about="">
-        <dc:format>image/svg+xml</dc:format>
-        <dc:type
-           rdf:resource="http://purl.org/dc/dcmitype/StillImage" />
-        <dc:title />
-      </cc:Work>
-    </rdf:RDF>
-  </metadata>
-  <g
-     inkscape:label="Ebene 1"
-     inkscape:groupmode="layer"
-     id="layer1"
-     transform="translate(0,-164.70764)">
-    <path
-       style="opacity:1;fill:#002130;fill-opacity:1;stroke:none;stroke-width:0;stroke-linecap:round;stroke-linejoin:bevel;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;paint-order:markers fill stroke"
-       d="m 15.593586,179.85791 c 18.094683,-18.09394 47.431291,-18.09355 65.52552,7.7e-4 16.496472,16.53809 18.102694,42.7696 3.747207,61.19678 17.821687,15.21024 24.490877,18.93088 42.171347,34.20969 4.41254,4.71717 4.29213,12.08299 -0.27223,16.65347 -4.57089,4.56126 -11.93421,4.68001 -16.65007,0.26874 l -8.4e-4,9.1e-4 -0.0173,-0.0152 C 94.335453,273.50898 90.211408,265.24542 76.792795,249.15281 58.360539,263.5048 32.12693,261.89037 15.593512,245.38662 c -18.0947127,-18.09455 -18.0947127,-47.43175 0,-65.52636 z m 7.908493,7.90846 c -13.7275962,13.72683 -13.7279739,35.98299 -8.23e-4,49.71019 13.72719,13.72714 35.983444,13.72678 49.710173,-8.2e-4 13.726834,-13.72692 13.726834,-35.98245 0,-49.70937 -13.726863,-13.72676 -35.982478,-13.72676 -49.70935,0 z"
-       id="rect826"
-       inkscape:connector-curvature="0"
-       sodipodi:nodetypes="ccccccccccccccccc" />
-  </g>
 </svg>

tubearchivist/static/img/icon-sort.svg

@@ -1,67 +1,21 @@
-<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<?xml version="1.0" encoding="utf-8"?>
-<!-- Created with Inkscape (http://www.inkscape.org/) -->
+<!-- Generator: Adobe Illustrator 27.5.0, SVG Export Plug-In . SVG Version: 6.00 Build 0)  -->
-
+<svg version="1.1"
-<svg
+	 id="svg1303" xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape" xmlns:cc="http://creativecommons.org/ns#" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:svg="http://www.w3.org/2000/svg" xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
-   xmlns:dc="http://purl.org/dc/elements/1.1/"
+	 xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px" viewBox="0 0 500 500"
-   xmlns:cc="http://creativecommons.org/ns#"
+	 style="enable-background:new 0 0 500 500;" xml:space="preserve">
-   xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
+<g>
-   xmlns:svg="http://www.w3.org/2000/svg"
+	<path d="M147.7,416.2c-8.7-3.4-11.9-8.1-11.9-17.5c0-53.5,0-107,0-160.5c0-9.3-4.2-15-12.6-16.7c-1.7-0.4-3.4-0.4-5.2-0.4
-   xmlns="http://www.w3.org/2000/svg"
+		c-8.3,0-16.6,0-25,0c-7.3,0-12.9-3-16-9.5c-3.1-6.6-2.4-13,2.3-18.8c21.1-26.4,42.1-52.9,63.2-79.4c5.7-7.2,11.4-14.4,17.1-21.6
-   xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+		c3.2-4.1,7-7.4,12.5-7.9c6.6-0.6,11.4,2.3,15.3,7.3c5.7,7.4,11.5,14.9,17.3,22.3c20.5,26.4,40.9,52.8,61.5,79
-   xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+		c7.2,9.2,4.3,19.8-2,24.9c-3.2,2.6-6.8,3.6-10.8,3.6c-8.6,0-17.3,0-25.9,0c-10.6,0.1-16.2,5.6-16.2,16.2
-   width="500"
+		c0,53.1-0.1,106.1,0.1,159.2c0,9.6-3.4,16.3-12.6,19.7C181.9,416.2,164.8,416.2,147.7,416.2z"/>
-   height="500"
+</g>
-   viewBox="0 0 132.29197 132.29167"
+<g>
-   version="1.1"
+	<path d="M369.5,83.8c8.7,3.4,11.9,8.1,11.9,17.5c0,53.5,0,106.9,0,160.4c0,9.3,4.2,15,12.6,16.7c1.7,0.3,3.4,0.4,5.2,0.4
-   id="svg1303"
+		c8.3,0,16.6,0,25,0c7.3,0,12.9,3,16,9.5c3.1,6.6,2.4,13-2.3,18.8c-21.1,26.4-42.1,52.9-63.2,79.3c-5.7,7.2-11.4,14.4-17.1,21.6
-   inkscape:version="0.92.4 (5da689c313, 2019-01-14)"
+		c-3.2,4.1-6.9,7.4-12.5,7.9c-6.6,0.6-11.4-2.3-15.3-7.3c-5.7-7.4-11.5-14.8-17.3-22.3c-20.5-26.4-40.8-52.8-61.5-79
-   sodipodi:docname="Icons_sort.svg">
+		c-7.2-9.2-4.3-19.8,2-24.9c3.2-2.6,6.8-3.6,10.8-3.6c8.6,0,17.3,0,25.9,0c10.6-0.1,16.2-5.6,16.2-16.2c0-53,0.1-106.1-0.1-159.1
-  <defs
+		c0-9.6,3.4-16.3,12.6-19.7C335.4,83.8,352.5,83.8,369.5,83.8z"/>
-     id="defs1297" />
+</g>
-  <sodipodi:namedview
-     id="base"
-     pagecolor="#ffffff"
-     bordercolor="#666666"
-     borderopacity="1.0"
-     inkscape:pageopacity="0.0"
-     inkscape:pageshadow="2"
-     inkscape:zoom="0.6482696"
-     inkscape:cx="214.8721"
-     inkscape:cy="136.02434"
-     inkscape:document-units="mm"
-     inkscape:current-layer="g855"
-     showgrid="false"
-     units="px"
-     inkscape:window-width="957"
-     inkscape:window-height="893"
-     inkscape:window-x="941"
-     inkscape:window-y="13"
-     inkscape:window-maximized="0" />
-  <metadata
-     id="metadata1300">
-    <rdf:RDF>
-      <cc:Work
-         rdf:about="">
-        <dc:format>image/svg+xml</dc:format>
-        <dc:type
-           rdf:resource="http://purl.org/dc/dcmitype/StillImage" />
-        <dc:title></dc:title>
-      </cc:Work>
-    </rdf:RDF>
-  </metadata>
-  <g
-     inkscape:label="Ebene 1"
-     inkscape:groupmode="layer"
-     id="layer1"
-     transform="translate(0,-164.70764)">
-    <g
-       id="g855"
-       transform="matrix(1.9016362,0,0,1.9016362,-197.93838,-58.9418)">
-      <path
-         style="opacity:1;fill:#000000;fill-opacity:1;stroke:none;stroke-width:2.55118108;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;paint-order:markers fill stroke"
-         d="M 341.125 82.275391 L 315.50977 106.92773 L 241.84375 177.89258 L 266.21289 203.17969 L 309.82617 161.17969 L 306.72266 379.375 C 306.36114 398.60059 319.37807 414.32761 335.91016 414.63867 L 335.91406 414.63477 C 352.44586 414.94597 366.04647 399.71987 366.4082 380.49414 L 369.5 162.97852 L 411.04297 206.11719 L 436.64062 181.44727 L 365.48242 107.57812 L 365.49609 107.5625 L 341.125 82.275391 z M 175.93359 82.277344 L 175.92969 82.28125 C 159.39782 81.970041 145.79728 97.196144 145.43555 116.42188 L 142.34375 333.9375 L 100.80078 290.79883 L 75.203125 315.46875 L 146.36133 389.33789 L 146.3457 389.35156 L 170.7168 414.63867 L 196.33203 389.98633 L 270 319.02344 L 245.63086 293.73633 L 202.01758 335.73633 L 205.12109 117.54102 C 205.48261 98.315428 192.46568 82.588409 175.93359 82.277344 z "
-         transform="matrix(0.13913489,0,0,0.13913489,104.08846,117.60887)"
-         id="path814" />
-    </g>
-  </g>
 </svg>

tubearchivist/static/img/icon-star-empty.svg

@@ -1,65 +1,25 @@
-<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<?xml version="1.0" encoding="utf-8"?>
-<!-- Created with Inkscape (http://www.inkscape.org/) -->
+<!-- Generator: Adobe Illustrator 27.5.0, SVG Export Plug-In . SVG Version: 6.00 Build 0)  -->
-
+<svg version="1.1"
-<svg
+	 id="svg1303" xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape" xmlns:cc="http://creativecommons.org/ns#" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:svg="http://www.w3.org/2000/svg" xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
-   width="500"
+	 xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px" viewBox="0 0 500 500"
-   height="500"
+	 style="enable-background:new 0 0 500 500;" xml:space="preserve">
-   viewBox="0 0 132.29197 132.29167"
+<g>
-   version="1.1"
+	<path d="M492.6,214.8c-2.7,7.3-7.9,12.7-13.8,17.7c-32.6,28.4-65.1,57-97.7,85.5c-2.4,2.1-2.4,4.1-1.8,6.7
-   id="svg1303"
+		c9.9,43.5,19.8,87.1,29.7,130.6c3.8,16.8-4.6,30.8-20.6,34.1c-7.2,1.5-13.7-0.8-19.8-4.4c-37.7-22.5-75.4-45-113-67.7
-   inkscape:version="1.1.1 (3bf5ae0d25, 2021-09-20)"
+		c-4.2-2.6-7.2-2.5-11.4,0.1c-37.7,22.8-75.6,45.3-113.4,67.9c-17.6,10.5-37.4,1.6-40.9-18.2c-0.7-3.8,0-7.5,0.8-11.2
-   sodipodi:docname="icon-star-empty.svg"
+		c9.8-43.4,19.7-86.8,29.6-130.2c0.8-3.7,0.4-5.9-2.6-8.5c-33.2-28.9-66.3-58.1-99.5-87c-4.9-4.3-8.5-9.3-10.9-15.2
-   xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+		c-1-4.4-1.4-8.8,0-13.3c4.9-13.7,14.9-19.5,29.3-20.4c22.5-1.4,44.9-4,67.4-6.1c19-1.7,38-3.6,57-5c6.3-0.5,10.3-2.2,13-8.8
-   xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+		c16.4-39.3,33.3-78.3,50-117.4c2.6-6.2,5.7-12,11.5-15.8c13.6-8.9,31.4-4.1,38.3,11.1c8.8,19.4,17,39.1,25.3,58.7
-   xmlns="http://www.w3.org/2000/svg"
+		c9.4,22.1,18.9,44.1,28.2,66.2c1.4,3.3,3.3,4.9,6.9,5.2c15.9,1.3,31.7,2.8,47.6,4.3c23.1,2.1,46.2,4.2,69.2,6.4
-   xmlns:svg="http://www.w3.org/2000/svg"
+		c4.2,0.4,8.5,1.1,12.7,1.1c14.3,0.2,23.9,6.8,28.8,20.3C494,206,493.8,210.4,492.6,214.8z M50.8,211.9c-1.9,0.3-2.5,2.7-1.1,3.9
-   xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
+		c0.1,0.1,0.2,0.2,0.3,0.3c16.9,14.8,33.7,29.6,50.6,44.4c14.2,12.5,28.5,25,42.7,37.5c8.8,7.7,11.7,17.1,9,28.6
-   xmlns:cc="http://creativecommons.org/ns#"
+		c-8.4,36.3-16.6,72.6-24.8,108.9c-0.9,4.1-2.1,8.1-2.8,12.2c-0.3,2.1,2.1,3.6,3.8,2.4c0.4-0.3,0.8-0.5,1.1-0.8
-   xmlns:dc="http://purl.org/dc/elements/1.1/">
+		c34.9-20.8,69.7-41.6,104.5-62.4c3.7-2.2,7.4-4,11.8-4.7c8.6-1.4,15.6,2.2,22.7,6.4c34.1,20.5,68.3,40.9,102.5,61.3
-  <defs
+		c1.2,0.7,2.6,2.6,4.1,1.3c1.1-1,0-2.7-0.3-4c-9-39.7-17.7-79.5-27-119.1c-3-12.8,0.1-22.8,9.9-31.3c24.3-21.1,48.5-42.4,72.7-63.6
-     id="defs1297" />
+		c6.7-5.8,13.3-11.7,20.2-17.8c1.4-1.2,0.7-3.6-1.2-3.8c0,0,0,0,0,0c-17.9-1.6-35.8-3.3-53.7-4.8c-22.6-2-45.2-4.2-67.9-5.9
-  <sodipodi:namedview
+		c-14.2-1.1-22.9-8-28.4-21.2c-15.4-37.2-31.5-74.1-47.3-111.1c-0.5-1.2-0.6-3.2-2.3-3.2s-1.8,2-2.3,3.2c-7.7,18-15.4,36-23.1,54
-     id="base"
+		c-8.6,20.2-17.2,40.4-25.9,60.6c-4,9.2-10.6,15.6-20.7,16.7c-20.2,2.3-40.5,4-60.8,5.8c-19,1.7-38,3.2-57,5.1
-     pagecolor="#ffffff"
+		C57.2,211,54.2,211.3,50.8,211.9z"/>
-     bordercolor="#666666"
+</g>
-     borderopacity="1.0"
-     inkscape:pageopacity="0.0"
-     inkscape:pageshadow="2"
-     inkscape:zoom="2.4005076"
-     inkscape:cx="40.40812"
-     inkscape:cy="211.20533"
-     inkscape:document-units="mm"
-     inkscape:current-layer="layer1"
-     showgrid="false"
-     units="px"
-     inkscape:window-width="3840"
-     inkscape:window-height="2112"
-     inkscape:window-x="0"
-     inkscape:window-y="48"
-     inkscape:window-maximized="1"
-     inkscape:pagecheckerboard="0" />
-  <metadata
-     id="metadata1300">
-    <rdf:RDF>
-      <cc:Work
-         rdf:about="">
-        <dc:format>image/svg+xml</dc:format>
-        <dc:type
-           rdf:resource="http://purl.org/dc/dcmitype/StillImage" />
-        <dc:title />
-      </cc:Work>
-    </rdf:RDF>
-  </metadata>
-  <g
-     inkscape:label="Ebene 1"
-     inkscape:groupmode="layer"
-     id="layer1"
-     transform="translate(0,-164.70764)">
-    <path
-       id="path1255"
-       style="opacity:1;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0;stroke-linecap:round;stroke-linejoin:bevel;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;paint-order:markers fill stroke"
-       inkscape:transform-center-y="-6.4703827"
-       d="M 250 21.158203 L 189.78906 194.37891 L 6.4375 198.11523 L 152.57617 308.9082 L 99.470703 484.43945 L 250 379.69141 L 400.5293 484.43945 L 347.42383 308.9082 L 493.5625 198.11523 L 310.21094 194.37891 L 250 21.158203 z M 250 96.408203 L 291.69141 216.34766 L 418.64453 218.93555 L 317.45703 295.65039 L 354.22852 417.18945 L 250 344.66211 L 145.77148 417.18945 L 182.54297 295.65039 L 81.355469 218.93555 L 208.30859 216.34766 L 250 96.408203 z "
-       transform="matrix(0.26458394,0,0,0.26458394,0,164.70749)" />
-  </g>
 </svg>

tubearchivist/static/img/icon-star-full.svg

@@ -1,74 +1,19 @@
-<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<?xml version="1.0" encoding="utf-8"?>
-<!-- Created with Inkscape (http://www.inkscape.org/) -->
+<!-- Generator: Adobe Illustrator 27.5.0, SVG Export Plug-In . SVG Version: 6.00 Build 0)  -->
-
+<svg version="1.1"
-<svg
+	 id="svg1303" xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape" xmlns:cc="http://creativecommons.org/ns#" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:svg="http://www.w3.org/2000/svg" xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
-   xmlns:dc="http://purl.org/dc/elements/1.1/"
+	 xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px" viewBox="0 0 500 500"
-   xmlns:cc="http://creativecommons.org/ns#"
+	 style="enable-background:new 0 0 500 500;" xml:space="preserve">
-   xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
+<sodipodi:namedview  bordercolor="#666666" borderopacity="1.0" id="base" inkscape:current-layer="layer1" inkscape:cx="61.891881" inkscape:cy="148.25167" inkscape:document-units="mm" inkscape:pageopacity="0.0" inkscape:pageshadow="2" inkscape:window-height="1017" inkscape:window-maximized="1" inkscape:window-width="1920" inkscape:window-x="-8" inkscape:window-y="-8" inkscape:zoom="1.0105705" pagecolor="#ffffff" showgrid="false" units="px">
-   xmlns:svg="http://www.w3.org/2000/svg"
+	</sodipodi:namedview>
-   xmlns="http://www.w3.org/2000/svg"
+<g>
-   xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+	<path d="M493.2,207.8c-2.7,7.3-7.9,12.7-13.8,17.7c-32.6,28.4-65.1,57-97.7,85.5c-2.4,2.1-2.4,4.1-1.8,6.7
-   xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+		c9.9,43.5,19.8,87.1,29.7,130.6c3.8,16.8-4.6,30.8-20.6,34.1c-7.2,1.5-13.7-0.8-19.8-4.4c-37.7-22.5-75.4-45-113-67.7
-   width="500"
+		c-4.2-2.6-7.2-2.5-11.4,0.1c-37.7,22.8-75.6,45.3-113.4,67.9c-17.6,10.5-37.4,1.6-40.9-18.2c-0.7-3.8,0-7.5,0.8-11.2
-   height="500"
+		c9.8-43.4,19.7-86.8,29.6-130.2c0.8-3.7,0.4-5.9-2.6-8.5C85.1,281.1,52,252,18.8,223c-4.9-4.3-8.5-9.3-10.9-15.2
-   viewBox="0 0 132.29197 132.29167"
+		c-1-4.4-1.4-8.8,0-13.3c4.9-13.7,14.9-19.5,29.3-20.4c22.5-1.4,44.9-4,67.4-6.1c19-1.7,38-3.6,57-5c6.3-0.5,10.3-2.2,13-8.8
-   version="1.1"
+		c16.3-39.2,33.2-78.2,49.8-117.2c2.6-6.2,5.7-12,11.5-15.8c13.6-8.9,31.4-4.1,38.3,11.1c8.8,19.4,17,39.1,25.3,58.7
-   id="svg1303"
+		c9.4,22.1,18.9,44.1,28.2,66.2c1.4,3.3,3.3,4.9,6.9,5.2c15.9,1.3,31.7,2.8,47.6,4.3c23.1,2.1,46.2,4.2,69.2,6.4
-   inkscape:version="0.92.4 (5da689c313, 2019-01-14)"
+		c4.2,0.4,8.5,1.1,12.7,1.1c14.3,0.2,23.9,6.8,28.8,20.3C494.5,199,494.3,203.4,493.2,207.8z"/>
-   sodipodi:docname="Icons_star.svg">
+</g>
-  <defs
-     id="defs1297" />
-  <sodipodi:namedview
-     id="base"
-     pagecolor="#ffffff"
-     bordercolor="#666666"
-     borderopacity="1.0"
-     inkscape:pageopacity="0.0"
-     inkscape:pageshadow="2"
-     inkscape:zoom="1.0105705"
-     inkscape:cx="61.891881"
-     inkscape:cy="148.25167"
-     inkscape:document-units="mm"
-     inkscape:current-layer="layer1"
-     showgrid="false"
-     units="px"
-     inkscape:window-width="1920"
-     inkscape:window-height="1017"
-     inkscape:window-x="-8"
-     inkscape:window-y="-8"
-     inkscape:window-maximized="1" />
-  <metadata
-     id="metadata1300">
-    <rdf:RDF>
-      <cc:Work
-         rdf:about="">
-        <dc:format>image/svg+xml</dc:format>
-        <dc:type
-           rdf:resource="http://purl.org/dc/dcmitype/StillImage" />
-        <dc:title />
-      </cc:Work>
-    </rdf:RDF>
-  </metadata>
-  <g
-     inkscape:label="Ebene 1"
-     inkscape:groupmode="layer"
-     id="layer1"
-     transform="translate(0,-164.70764)">
-    <path
-       sodipodi:type="star"
-       style="opacity:1;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0;stroke-linecap:round;stroke-linejoin:bevel;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;paint-order:markers fill stroke"
-       id="path1255"
-       sodipodi:sides="5"
-       sodipodi:cx="66.145981"
-       sodipodi:cy="237.32387"
-       sodipodi:r1="67.758904"
-       sodipodi:r2="27.103561"
-       sodipodi:arg1="-1.5707963"
-       sodipodi:arg2="-0.9424778"
-       inkscape:flatsided="false"
-       inkscape:rounded="-3.46945e-18"
-       inkscape:randomized="0"
-       d="m 66.145983,169.56496 15.931071,45.83167 48.511476,0.98859 L 91.923,245.69933 105.97366,292.14197 66.145981,264.42743 26.318295,292.14197 40.368962,245.69933 1.7034347,216.38521 50.214907,215.39663 Z"
-       inkscape:transform-center-y="-6.4703827" />
-  </g>
 </svg>

tubearchivist/static/img/icon-star-half.svg

@@ -1,66 +1,22 @@
-<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<?xml version="1.0" encoding="utf-8"?>
-<!-- Created with Inkscape (http://www.inkscape.org/) -->
+<!-- Generator: Adobe Illustrator 27.5.0, SVG Export Plug-In . SVG Version: 6.00 Build 0)  -->
-
+<svg version="1.1"
-<svg
+	 id="svg1303" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd" xmlns:svg="http://www.w3.org/2000/svg" xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:cc="http://creativecommons.org/ns#"
-   width="500"
+	 xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px" viewBox="0 0 500 500"
-   height="500"
+	 style="enable-background:new 0 0 500 500;" xml:space="preserve">
-   viewBox="0 0 132.29197 132.29167"
+<g>
-   version="1.1"
+	<path d="M7.5,207.8c2.8,7.3,8,12.6,13.8,17.7C54,253.9,86.4,282.6,119,311c2.4,2.1,2.4,4.1,1.8,6.7c-9.9,43.5-19.8,87.1-29.7,130.6
-   id="svg1303"
+		c-3.9,16.9,4.6,30.9,20.6,34.1c7.2,1.5,13.7-0.7,19.8-4.4c37.7-22.6,75.4-45,112.9-68c1.9-1.3,3.7-1.9,5.4-1.9v0.3
-   inkscape:version="1.1.1 (3bf5ae0d25, 2021-09-20)"
+		c1.9,0,3.7,0.6,5.8,1.9c37.7,22.8,75.5,45.4,113.4,67.9c17.6,10.5,37.4,1.6,40.9-18.2c0.7-3.8,0-7.5-0.8-11.2
-   sodipodi:docname="icon-star-half.svg"
+		c-9.8-43.4-19.6-86.8-29.6-130.2c-0.9-3.6-0.4-5.9,2.6-8.5c33.3-28.9,66.3-58,99.5-87c4.9-4.3,8.5-9.3,10.9-15.2
-   xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+		c1-4.5,1.4-8.9,0-13.3c-5-13.7-14.9-19.5-29.3-20.4c-22.5-1.4-44.9-4.1-67.4-6.1c-19-1.8-38-3.6-57-5c-6.3-0.5-10.2-2.2-13-8.8
-   xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+		c-16.2-39.2-33.2-78.1-49.8-117.2c-2.6-6.2-5.7-12-11.5-15.8c-4.6-3-9.5-4.5-14.5-4.5v-0.1c-9.8,0-19.2,5.5-23.8,15.6
-   xmlns="http://www.w3.org/2000/svg"
+		c-8.8,19.4-16.9,39.1-25.3,58.7c-9.5,22-18.9,44.1-28.2,66.2c-1.4,3.3-3.3,4.9-6.9,5.2c-15.9,1.3-31.7,2.8-47.6,4.3
-   xmlns:svg="http://www.w3.org/2000/svg"
+		c-23,2.2-46.1,4.3-69.2,6.4c-4.2,0.4-8.5,1-12.7,1.1C22,174.4,12.4,181,7.5,194.5C6.1,199,6.3,203.4,7.5,207.8z M250,374.9V58.2
-   xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
+		c1.6,0.1,1.7,2,2.2,3.2c7.7,18,15.4,36,23.1,54c8.7,20.2,17.3,40.4,25.9,60.6c3.9,9.2,10.6,15.6,20.7,16.7
-   xmlns:cc="http://creativecommons.org/ns#"
+		c20.3,2.3,40.5,4,60.8,5.8c19,1.7,38,3.1,57,5.1l6,0.6c3.3,0.3,4.6,4.3,2.1,6.5c-15.6,13.8-33.6,29.7-48.4,42.7
-   xmlns:dc="http://purl.org/dc/elements/1.1/">
+		c-14.2,12.5-28.5,25-42.7,37.5c-8.7,7.7-11.6,17.2-9,28.6c8.4,36.2,16.6,72.6,24.8,108.9c0.9,4,2.1,8,2.8,12.1
-  <defs
+		c0.3,2.2-2.1,3.7-3.9,2.5c-0.4-0.3-0.7-0.5-1.1-0.7c-34.8-20.8-69.7-41.6-104.5-62.4c-3.7-2.2-7.5-4-11.8-4.7
-     id="defs1297" />
+		C252.8,375,251.6,374.9,250,374.9L250,374.9z"/>
-  <sodipodi:namedview
+</g>
-     id="base"
-     pagecolor="#ffffff"
-     bordercolor="#666666"
-     borderopacity="1.0"
-     inkscape:pageopacity="0.0"
-     inkscape:pageshadow="2"
-     inkscape:zoom="1.0105705"
-     inkscape:cx="44.034533"
-     inkscape:cy="168.22181"
-     inkscape:document-units="mm"
-     inkscape:current-layer="layer1"
-     showgrid="false"
-     units="px"
-     inkscape:window-width="3840"
-     inkscape:window-height="2078"
-     inkscape:window-x="0"
-     inkscape:window-y="82"
-     inkscape:window-maximized="1"
-     inkscape:pagecheckerboard="0" />
-  <metadata
-     id="metadata1300">
-    <rdf:RDF>
-      <cc:Work
-         rdf:about="">
-        <dc:format>image/svg+xml</dc:format>
-        <dc:type
-           rdf:resource="http://purl.org/dc/dcmitype/StillImage" />
-        <dc:title />
-      </cc:Work>
-    </rdf:RDF>
-  </metadata>
-  <g
-     inkscape:label="Ebene 1"
-     inkscape:groupmode="layer"
-     id="layer1"
-     transform="translate(0,-164.70764)">
-    <path
-       id="path1255"
-       style="opacity:1;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0;stroke-linecap:round;stroke-linejoin:bevel;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;paint-order:markers fill stroke"
-       inkscape:transform-center-x="1.1220981e-06"
-       inkscape:transform-center-y="-6.4703827"
-       d="M 250 21.158203 L 189.78906 194.37891 L 6.4375 198.11523 L 152.57617 308.9082 L 99.470703 484.43945 L 250 379.69141 L 400.5293 484.43945 L 347.42383 308.9082 L 493.5625 198.11523 L 310.21094 194.37891 L 250 21.158203 z M 250 96.408203 L 291.69141 216.34766 L 418.64453 218.93555 L 317.45703 295.65039 L 354.22852 417.18945 L 250 344.66211 L 250 96.408203 z "
-       transform="matrix(0.26458394,0,0,0.26458394,0,164.70749)" />
-  </g>
 </svg>

tubearchivist/static/img/icon-substract.svg

@@ -1,75 +1,12 @@
-<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<?xml version="1.0" encoding="utf-8"?>
-<!-- Created with Inkscape (http://www.inkscape.org/) -->
+<!-- Generator: Adobe Illustrator 27.5.0, SVG Export Plug-In . SVG Version: 6.00 Build 0)  -->
-
+<svg version="1.1"
-<svg
+	 id="svg1303" xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape" xmlns:cc="http://creativecommons.org/ns#" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:svg="http://www.w3.org/2000/svg" xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
-   width="474.99609"
+	 xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px" viewBox="0 0 475 111.2"
-   height="78.428696"
+	 style="enable-background:new 0 0 475 111.2;" xml:space="preserve">
-   viewBox="0 0 125.67634 20.750926"
+<sodipodi:namedview  bordercolor="#666666" borderopacity="1.0" id="base" inkscape:current-layer="layer1" inkscape:cx="-97.380081" inkscape:cy="261.09215" inkscape:deskcolor="#d1d1d1" inkscape:document-units="mm" inkscape:guide-bbox="true" inkscape:pagecheckerboard="0" inkscape:pageopacity="0.0" inkscape:pageshadow="2" inkscape:showpageshadow="2" inkscape:window-height="1017" inkscape:window-maximized="1" inkscape:window-width="1920" inkscape:window-x="-8" inkscape:window-y="-8" inkscape:zoom="0.85859018" pagecolor="#ffffff" showgrid="false" showguides="true" units="px">
-   version="1.1"
+	<sodipodi:guide  id="guide1072" inkscape:locked="false" orientation="1,0" position="-225.18364,87.524084"></sodipodi:guide>
-   id="svg1303"
+</sodipodi:namedview>
-   inkscape:version="0.92.4 (5da689c313, 2019-01-14)"
+<path d="M181.9,0H55.6C25,0,0,25,0,55.6l0,0c0,30.6,25,55.6,55.6,55.6h126.3h111.2h126.3c30.6,0,55.6-25,55.6-55.6l0,0
-   sodipodi:docname="icon-add.svg"
+	C475,25,450,0,419.4,0H293.1H181.9z"/>
-   xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
-   xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
-   xmlns="http://www.w3.org/2000/svg"
-   xmlns:svg="http://www.w3.org/2000/svg"
-   xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
-   xmlns:cc="http://creativecommons.org/ns#"
-   xmlns:dc="http://purl.org/dc/elements/1.1/">
-  <defs
-     id="defs1297" />
-  <sodipodi:namedview
-     id="base"
-     pagecolor="#ffffff"
-     bordercolor="#666666"
-     borderopacity="1.0"
-     inkscape:pageopacity="0.0"
-     inkscape:pageshadow="2"
-     inkscape:zoom="0.85859018"
-     inkscape:cx="-97.380081"
-     inkscape:cy="261.09215"
-     inkscape:document-units="mm"
-     inkscape:current-layer="layer1"
-     showgrid="false"
-     units="px"
-     inkscape:window-width="1920"
-     inkscape:window-height="1017"
-     inkscape:window-x="-8"
-     inkscape:window-y="-8"
-     inkscape:window-maximized="1"
-     showguides="true"
-     inkscape:guide-bbox="true"
-     inkscape:showpageshadow="2"
-     inkscape:pagecheckerboard="0"
-     inkscape:deskcolor="#d1d1d1">
-    <sodipodi:guide
-       position="-225.18364,87.524084"
-       orientation="1,0"
-       id="guide1072"
-       inkscape:locked="false" />
-  </sodipodi:namedview>
-  <metadata
-     id="metadata1300">
-    <rdf:RDF>
-      <cc:Work
-         rdf:about="">
-        <dc:format>image/svg+xml</dc:format>
-        <dc:type
-           rdf:resource="http://purl.org/dc/dcmitype/StillImage" />
-        <dc:title />
-      </cc:Work>
-    </rdf:RDF>
-  </metadata>
-  <g
-     inkscape:label="Ebene 1"
-     inkscape:groupmode="layer"
-     id="layer1"
-     transform="translate(-3.3077777,-220.4781)">
-    <path
-       style="opacity:1;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0;stroke-linecap:round;stroke-linejoin:bevel;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;paint-order:markers fill stroke"
-       d="m 12.291019,220.47796 c -4.9648228,-0.009 -8.9691711,3.98046 -8.9784054,8.94536 l -0.00482,2.62846 c -0.00925,4.9649 3.9804459,8.96933 8.9452674,8.97832 107.703889,0.29274 22.266017,0.0414 107.747629,0.19881 4.96484,0.009 8.96917,-3.98046 8.9784,-8.94534 l 0.005,-2.62847 c 0.009,-4.96489 -3.98037,-8.96923 -8.94525,-8.97831 -107.850822,-0.0255 -20.413204,-0.038 -107.747821,-0.19883 z"
-       id="rect1073"
-       inkscape:connector-curvature="0"
-       sodipodi:nodetypes="ccccccccc" />
-  </g>
 </svg>

tubearchivist/static/img/icon-thumb.svg

@@ -1,18 +1,21 @@
 <?xml version="1.0" encoding="utf-8"?>
-<!-- Generator: Adobe Illustrator 26.5.0, SVG Export Plug-In . SVG Version: 6.00 Build 0)  -->
+<!-- Generator: Adobe Illustrator 27.5.0, SVG Export Plug-In . SVG Version: 6.00 Build 0)  -->
 <svg version="1.1"
-	 id="svg1303" xmlns:cc="http://creativecommons.org/ns#" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape" xmlns:svg="http://www.w3.org/2000/svg" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd" inkscape:version="0.92.4 (5da689c313, 2019-01-14)" sodipodi:docname="icon_thumb.svg"
+	 id="svg1303" xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape" xmlns:cc="http://creativecommons.org/ns#" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:svg="http://www.w3.org/2000/svg" xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
 	 xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px" viewBox="0 0 500 500"
 	 style="enable-background:new 0 0 500 500;" xml:space="preserve">
 <sodipodi:namedview  bordercolor="#666666" borderopacity="1.0" id="base" inkscape:current-layer="layer1" inkscape:cx="35.718548" inkscape:cy="203.39339" inkscape:document-units="mm" inkscape:guide-bbox="true" inkscape:pageopacity="0.0" inkscape:pageshadow="2" inkscape:window-height="1009" inkscape:window-maximized="1" inkscape:window-width="1920" inkscape:window-x="-8" inkscape:window-y="-8" inkscape:zoom="1.5046797" pagecolor="#ffffff" showgrid="false" showguides="true" units="px">
 	<sodipodi:guide  id="guide816" inkscape:locked="false" orientation="0,1" position="80.99058,65.965029"></sodipodi:guide>
 	<sodipodi:guide  id="guide818" inkscape:locked="false" orientation="1,0" position="65.965178,49.107299"></sodipodi:guide>
 </sodipodi:namedview>
-<g id="layer1" transform="translate(0,-164.70764)" inkscape:groupmode="layer" inkscape:label="Ebene 1">
+<g>
-	<path id="rect1278" inkscape:connector-curvature="0" d="M285.3,164.7c-3.1-0.1-6.4,0.1-9.8,0.7c-73.7,12.2,24,119.6-61.8,180.9
+	<path d="M142.3,330.2c0-38.6,0.1-77.1,0-115.7c-0.1-16.2,4.1-31,12.8-44.7c29.5-46.9,58.7-94,88.1-141C256.7,7.3,281-3.8,304.5,0.5
-		c-71.7,59.8-97.8,41.1-98.5,60.7v192.6c10.8,28.5,82.3,57.6,158.4,63.8c76.1,6.2,182.5-9.9,183.6-43c0.6-30-16.5-30.7-16.5-30.7
+		c25.2,4.7,43.6,22.6,48.9,47.9c2.4,11.8,0.8,23.2-3.4,34.3c-12.5,33.1-24.8,66.3-37.2,99.4c-2.7,7.1-2.6,7.1,5.2,7.1
-		s37.5-2.3,38.7-34.8c0.6-31.9-36.3-36.4-36.3-36.4s47.3-1.3,47.3-34.2c0-33.6-36.4-36.3-47.6-36.8c7-0.4,34.5-10,34.4-43.2
+		c40.9,0,81.9-0.2,122.8,0.1c21.5,0.1,37.7,9.5,46.8,29.6c4.4,9.8,4.4,20,2.7,30.5c-8.1,52.1-15.9,104.3-24.2,156.4
-		c-0.2-34.3-52.8-31.5-147.3-37.7C340.8,302.9,348.4,166.8,285.3,164.7L285.3,164.7z M25.6,382.8c-8.9,0-16,6-16,13.4v217.5
+		c-2.6,16.4-3.3,33.3-9.6,48.9c-10.7,26.5-36.5,44.1-65,44.2c-66.5,0.1-133,0.1-199.5,0c-29,0-49.6-20.7-49.7-49.9
-		c0,7.4,7.1,13.4,16,13.4h59.8c8.9,0,16-6,16-13.4V396.2c0-7.4-7.1-13.4-16-13.4H25.6z"/>
+		C142.3,409.6,142.3,369.9,142.3,330.2z"/>
+	<path d="M0,338.5c0-37.3-0.1-74.5,0-111.8c0.1-23.5,14.9-43.1,37.5-47.5c15.4-3,31.5-3.1,46.9,0.4c21.7,5,35.5,24,35.6,49.1
+		c0.3,73.2,0.3,146.5,0,219.7c-0.1,26.4-15.1,45.5-38.9,49.7c-13.8,2.5-28.2,2.4-42.1,0.1c-23.3-3.9-39-23.9-39.1-47.9
+		C-0.1,413,0,375.8,0,338.5z"/>
 </g>
 </svg>

tubearchivist/static/script.js

@@ -2,9 +2,11 @@
 
 /* globals checkMessages */
 
-function sortChange(sortValue) {
+function sortChange(button) {
-  let payload = JSON.stringify({ sort_order: sortValue });
+  let apiEndpoint = '/api/config/user/';
-  sendPost(payload);
+  let data = {};
+  data[button.name] = button.value;
+  apiRequest(apiEndpoint, 'POST', data);
   setTimeout(function () {
     location.reload();
   }, 500);
@@ -64,7 +66,15 @@ function isWatchedButton(button) {
   let youtube_id = button.getAttribute('data-id');
   let apiEndpoint = '/api/watched/';
   let data = { id: youtube_id, is_watched: true };
-  button.remove();
+  apiRequest(apiEndpoint, 'POST', data);
+  setTimeout(function () {
+    location.reload();
+  }, 1000);
+}
+function isUnwatchedButton(button) {
+  let youtube_id = button.getAttribute('data-id');
+  let apiEndpoint = '/api/watched/';
+  let data = { id: youtube_id, is_watched: false };
   apiRequest(apiEndpoint, 'POST', data);
   setTimeout(function () {
     location.reload();
@@ -97,17 +107,21 @@ function subscribeStatus(subscribeButton) {
 function changeView(image) {
   let sourcePage = image.getAttribute('data-origin');
   let newView = image.getAttribute('data-value');
-  let payload = JSON.stringify({ change_view: sourcePage + ':' + newView });
+  let apiEndpoint = '/api/config/user/';
-  sendPost(payload);
+  let data = {};
+  data[`view_style_${sourcePage}`] = newView;
+  console.log(data);
+  apiRequest(apiEndpoint, 'POST', data);
   setTimeout(function () {
     location.reload();
   }, 500);
 }
 
 function changeGridItems(image) {
-  let operator = image.getAttribute('data-value');
+  let newGridItems = Number(image.getAttribute('data-value'));
-  let payload = JSON.stringify({ change_grid: operator });
+  let apiEndpoint = '/api/config/user/';
-  sendPost(payload);
+  let data = { grid_items: newGridItems };
+  apiRequest(apiEndpoint, 'POST', data);
   setTimeout(function () {
     location.reload();
   }, 500);
@@ -115,12 +129,10 @@ function changeGridItems(image) {
 
 function toggleCheckbox(checkbox) {
   // pass checkbox id as key and checkbox.checked as value
-  let toggleId = checkbox.id;
+  let apiEndpoint = '/api/config/user/';
-  let toggleVal = checkbox.checked;
+  let data = {};
-  let payloadDict = {};
+  data[checkbox.id] = checkbox.checked;
-  payloadDict[toggleId] = toggleVal;
+  apiRequest(apiEndpoint, 'POST', data);
-  let payload = JSON.stringify(payloadDict);
-  sendPost(payload);
   setTimeout(function () {
     let currPage = window.location.pathname;
     window.location.replace(currPage);
@@ -275,7 +287,7 @@ function reEmbed() {
 }
 
 function dbBackup() {
-  let apiEndpoint = '/api/task-name/run_backup/';
+  let apiEndpoint = '/api/backup/';
   apiRequest(apiEndpoint, 'POST');
   // clear button
   let message = document.createElement('p');
@@ -291,8 +303,8 @@ function dbBackup() {
 
 function dbRestore(button) {
   let fileName = button.getAttribute('data-id');
-  let payload = JSON.stringify({ 'db-restore': fileName });
+  let apiEndpoint = `/api/backup/${fileName}/`;
-  sendPost(payload);
+  apiRequest(apiEndpoint, 'POST');
   // clear backup row
   let message = document.createElement('p');
   message.innerText = 'restoring from backup';
@@ -462,7 +474,7 @@ function createPlayer(button) {
 
   // If cast integration is enabled create cast button
   let castButton = '';
-  if (videoData.config.application.enable_cast) {
+  if (videoData.config.enable_cast) {
     castButton = `<google-cast-launcher id="castbutton"></google-cast-launcher>`;
   }
 
@@ -534,7 +546,7 @@ function createVideoTag(videoData, videoProgress) {
   }
 
   let videoTag = `
-    <video poster="${videoThumbUrl}" ontimeupdate="onVideoProgress()" onpause="onVideoPause()" onended="onVideoEnded()" controls autoplay width="100%" playsinline id="video-item">
+    <video poster="${videoThumbUrl}" onvolumechange="onVolumeChange(this)" onloadstart="this.volume=getPlayerVolume()" ontimeupdate="onVideoProgress()" onpause="onVideoPause()" onended="onVideoEnded()" controls autoplay width="100%" playsinline id="video-item">
         <source src="${videoUrl}#t=${videoProgress}" type="video/mp4" id="video-source" videoid="${videoId}">
         ${subtitles}
     </video>
@@ -542,6 +554,14 @@ function createVideoTag(videoData, videoProgress) {
   return videoTag;
 }
 
+function onVolumeChange(videoTag) {
+  localStorage.setItem('playerVolume', videoTag.volume);
+}
+
+function getPlayerVolume() {
+  return localStorage.getItem('playerVolume') ?? 1;
+}
+
 // Gets video tag
 function getVideoPlayer() {
   let videoElement = document.getElementById('video-item');
@@ -938,7 +958,7 @@ function populateMultiSearchResults(allResults, queryType) {
   videoBox.parentElement.style.display = 'block';
   if (allVideos.length > 0) {
     for (let index = 0; index < allVideos.length; index++) {
-      const video = allVideos[index].source;
+      const video = allVideos[index];
       const videoDiv = createVideo(video, defaultVideo);
       videoBox.appendChild(videoDiv);
     }
@@ -957,7 +977,7 @@ function populateMultiSearchResults(allResults, queryType) {
   channelBox.parentElement.style.display = 'block';
   if (allChannels.length > 0) {
     for (let index = 0; index < allChannels.length; index++) {
-      const channel = allChannels[index].source;
+      const channel = allChannels[index];
       const channelDiv = createChannel(channel, defaultChannel);
       channelBox.appendChild(channelDiv);
     }
@@ -976,7 +996,7 @@ function populateMultiSearchResults(allResults, queryType) {
   playlistBox.parentElement.style.display = 'block';
   if (allPlaylists.length > 0) {
     for (let index = 0; index < allPlaylists.length; index++) {
-      const playlist = allPlaylists[index].source;
+      const playlist = allPlaylists[index];
       const playlistDiv = createPlaylist(playlist, defaultPlaylist);
       playlistBox.appendChild(playlistDiv);
     }
@@ -995,7 +1015,7 @@ function populateMultiSearchResults(allResults, queryType) {
   if (allFullText.length > 0) {
     for (let i = 0; i < allFullText.length; i++) {
       const fullText = allFullText[i];
-      if ('highlight' in fullText) {
+      if ('subtitle_line' in fullText) {
         const fullTextDiv = createFulltext(fullText);
         fullTextBox.appendChild(fullTextDiv);
       }
@@ -1132,19 +1152,14 @@ function createPlaylist(playlist, viewStyle) {
 }
 
 function createFulltext(fullText) {
-  const videoId = fullText.source.youtube_id;
+  const videoId = fullText.youtube_id;
-  const videoTitle = fullText.source.title;
+  const subtitle_start = fullText.subtitle_start.split('.')[0];
-  const thumbUrl = fullText.source.vid_thumb_url;
+  const subtitle_end = fullText.subtitle_end.split('.')[0];
-  const channelId = fullText.source.subtitle_channel_id;
-  const channelName = fullText.source.subtitle_channel;
-  const subtitleLine = fullText.highlight.subtitle_line[0];
-  const subtitle_start = fullText.source.subtitle_start.split('.')[0];
-  const subtitle_end = fullText.source.subtitle_end.split('.')[0];
   const markup = `
     <a href="#player" data-id="${videoId}" data-position="${subtitle_start}" onclick="createPlayer(this)">
         <div class="video-thumb-wrap list">
             <div class="video-thumb">
-                <img src="${thumbUrl}" alt="video-thumb">
+                <img src="${fullText.vid_thumb_url}" alt="video-thumb">
             </div>
             <div class="video-play">
                 <img src="/static/img/icon-play.svg" alt="play-icon">
@@ -1152,12 +1167,13 @@ function createFulltext(fullText) {
         </div>
     </a>
     <div class="video-desc list">
-        <p>${subtitle_start} - ${subtitle_end}</p>
-        <p>${subtitleLine}</p>
         <div>
-            <a href="/channel/${channelId}/"><h3>${channelName}</h3></a>
+          <a href="/channel/${fullText.subtitle_channel_id}/"><h3>${fullText.subtitle_channel}</h3></a>
-            <a class="video-more" href="/video/${videoId}/?t=${subtitle_start}"><h2>${videoTitle}</h2></a>
+          <a class="video-more" href="/video/${videoId}/?t=${subtitle_start}"><h2>${fullText.title}</h2></a>
         </div>
+        <p>${subtitle_start} - ${subtitle_end}</p>
+        <p>${fullText.subtitle_line}</p>
+        <span class="settings-current">Score: ${fullText._score}</span>
     </div>
     `;
   const fullTextDiv = document.createElement('div');
@@ -1300,14 +1316,6 @@ function populateEmpty() {
 
 // generic
 
-function sendPost(payload) {
-  let http = new XMLHttpRequest();
-  http.open('POST', '/process/', true);
-  http.setRequestHeader('X-CSRFToken', getCookie('csrftoken'));
-  http.setRequestHeader('Content-type', 'application/json');
-  http.send(payload);
-}
-
 function getCookie(c_name) {
   if (document.cookie.length > 0) {
     let c_start = document.cookie.indexOf(c_name + '=');

tubearchivist/static/stats.js

@@ -5,72 +5,183 @@
 /* globals apiRequest */
 
 function primaryStats() {
-  let apiEndpoint = '/api/stats/primary/';
+  let apiVideoEndpoint = '/api/stats/video/';
-  let responseData = apiRequest(apiEndpoint, 'GET');
+  let responseData = apiRequest(apiVideoEndpoint, 'GET');
-  let primaryBox = document.getElementById('primaryBox');
+
-  clearLoading(primaryBox);
+  let activeBox = document.getElementById('activeBox');
-  let videoTile = buildVideoTile(responseData);
+  clearLoading(activeBox);
-  primaryBox.appendChild(videoTile);
+
-  let channelTile = buildChannelTile(responseData);
+  let totalTile = buildTotalVideoTile(responseData);
-  primaryBox.appendChild(channelTile);
+  activeBox.appendChild(totalTile);
-  let playlistTile = buildPlaylistTile(responseData);
+  let activeTile = buildActiveVideoTile(responseData);
-  primaryBox.appendChild(playlistTile);
+  activeBox.appendChild(activeTile);
-  let downloadTile = buildDownloadTile(responseData);
+  let inActiveTile = buildInActiveVideoTile(responseData);
-  primaryBox.appendChild(downloadTile);
+  activeBox.appendChild(inActiveTile);
+
+  let videoTypeBox = document.getElementById('videoTypeBox');
+  clearLoading(videoTypeBox);
+
+  let videosTypeTile = buildVideosTypeTile(responseData);
+  videoTypeBox.appendChild(videosTypeTile);
+  let shortsTypeTile = buildShortsTypeTile(responseData);
+  videoTypeBox.appendChild(shortsTypeTile);
+  let streamsTypeTile = buildStreamsTypeTile(responseData);
+  videoTypeBox.appendChild(streamsTypeTile);
 }
 
-function clearLoading(dashBox) {
+function secondaryStats() {
-  dashBox.querySelector('#loading').remove();
+  let apiChannelEndpoint = '/api/stats/channel/';
+  let channelResponseData = apiRequest(apiChannelEndpoint, 'GET');
+  let secondaryBox = document.getElementById('secondaryBox');
+  clearLoading(secondaryBox);
+  let channelTile = buildChannelTile(channelResponseData);
+  secondaryBox.appendChild(channelTile);
+
+  let apiPlaylistEndpoint = '/api/stats/playlist/';
+  let playlistResponseData = apiRequest(apiPlaylistEndpoint, 'GET');
+  let playlistTile = buildPlaylistTile(playlistResponseData);
+  secondaryBox.appendChild(playlistTile);
+
+  let apiDownloadEndpoint = '/api/stats/download/';
+  let downloadResponseData = apiRequest(apiDownloadEndpoint, 'GET');
+  let downloadTile = buildDownloadTile(downloadResponseData);
+  secondaryBox.appendChild(downloadTile);
 }
 
-function buildTile(titleText) {
+function buildTotalVideoTile(responseData) {
-  let tile = document.createElement('div');
+  const totalCount = responseData.doc_count || 0;
-  tile.classList.add('info-box-item');
+  const totalSize = humanFileSize(responseData.media_size || 0);
-  let title = document.createElement('h3');
+  const content = {
-  title.innerText = titleText;
+    Videos: `${totalCount}`,
-  tile.appendChild(title);
+    'Media Size': `${totalSize}`,
+    Duration: responseData.duration_str,
+  };
+  const tile = buildTile('All: ');
+  const table = buildTileContenTable(content, 2);
+  tile.appendChild(table);
   return tile;
 }
 
-function buildVideoTile(responseData) {
+function buildActiveVideoTile(responseData) {
-  let tile = buildTile(`Total Videos: ${responseData.videos.total || 0}`);
+  const activeCount = responseData?.active_true?.doc_count || 0;
-  let message = document.createElement('p');
+  const activeSize = humanFileSize(responseData?.active_true?.media_size || 0);
-  message.innerHTML = `
+  const duration = responseData?.active_true?.duration_str || 'NA';
-        videos: ${responseData.videos.videos || 0}<br>
+  const content = {
-        shorts: ${responseData.videos.shorts || 0}<br>
+    Videos: `${activeCount}`,
-        streams: ${responseData.videos.streams || 0}<br>
+    'Media Size': `${activeSize}`,
-    `;
+    Duration: duration,
-  tile.appendChild(message);
+  };
+  const tile = buildTile('Active: ');
+  const table = buildTileContenTable(content, 2);
+  tile.appendChild(table);
+  return tile;
+}
 
+function buildInActiveVideoTile(responseData) {
+  const inActiveCount = responseData?.active_false?.doc_count || 0;
+  const inActiveSize = humanFileSize(responseData?.active_false?.media_size || 0);
+  const duration = responseData?.active_false?.duration_str || 'NA';
+  const content = {
+    Videos: `${inActiveCount}`,
+    'Media Size': `${inActiveSize}`,
+    Duration: duration,
+  };
+  const tile = buildTile('Inactive: ');
+  const table = buildTileContenTable(content, 2);
+  tile.appendChild(table);
+  return tile;
+}
+
+function buildVideosTypeTile(responseData) {
+  const videosCount = responseData?.type_videos?.doc_count || 0;
+  const videosSize = humanFileSize(responseData?.type_videos?.media_size || 0);
+  const duration = responseData?.type_videos?.duration_str || 'NA';
+  const content = {
+    Videos: `${videosCount}`,
+    'Media Size': `${videosSize}`,
+    Duration: duration,
+  };
+  const tile = buildTile('Regular Videos: ');
+  const table = buildTileContenTable(content, 2);
+  tile.appendChild(table);
+  return tile;
+}
+
+function buildShortsTypeTile(responseData) {
+  const shortsCount = responseData?.type_shorts?.doc_count || 0;
+  const shortsSize = humanFileSize(responseData?.type_shorts?.media_size || 0);
+  const duration = responseData?.type_shorts?.duration_str || 'NA';
+  const content = {
+    Videos: `${shortsCount}`,
+    'Media Size': `${shortsSize}`,
+    Duration: duration,
+  };
+  const tile = buildTile('Shorts: ');
+  const table = buildTileContenTable(content, 2);
+  tile.appendChild(table);
+  return tile;
+}
+
+function buildStreamsTypeTile(responseData) {
+  const streamsCount = responseData?.type_streams?.doc_count || 0;
+  const streamsSize = humanFileSize(responseData?.type_streams?.media_size || 0);
+  const duration = responseData?.type_streams?.duration_str || 'NA';
+  const content = {
+    Videos: `${streamsCount}`,
+    'Media Size': `${streamsSize}`,
+    Duration: duration,
+  };
+  const tile = buildTile('Streams: ');
+  const table = buildTileContenTable(content, 2);
+  tile.appendChild(table);
   return tile;
 }
 
 function buildChannelTile(responseData) {
-  let tile = buildTile(`Total Channels: ${responseData.channels.total || 0}`);
+  let tile = buildTile('Channels: ');
-  let message = document.createElement('p');
+  const total = responseData.doc_count || 0;
-  message.innerHTML = `subscribed: ${responseData.channels.sub_true || 0}`;
+  const subscribed = responseData.subscribed_true || 0;
-  tile.appendChild(message);
+  const active = responseData.active_true || 0;
+  const content = {
+    Subscribed: subscribed,
+    Active: active,
+    Total: total,
+  };
+  const table = buildTileContenTable(content, 3);
+  tile.appendChild(table);
 
   return tile;
 }
 
 function buildPlaylistTile(responseData) {
-  let tile = buildTile(`Total Playlists: ${responseData.playlists.total || 0}`);
+  let tile = buildTile('Playlists: ');
-  let message = document.createElement('p');
+  const total = responseData.doc_count || 0;
-  message.innerHTML = `subscribed: ${responseData.playlists.sub_true || 0}`;
+  const subscribed = responseData.subscribed_true || 0;
-  tile.appendChild(message);
+  const active = responseData.active_true || 0;
+  const content = {
+    Subscribed: subscribed,
+    Active: active,
+    Total: total,
+  };
+  const table = buildTileContenTable(content, 2);
+  tile.appendChild(table);
 
   return tile;
 }
 
 function buildDownloadTile(responseData) {
-  let tile = buildTile('Downloads');
+  const pendingTotal = responseData.pending || 0;
-  let message = document.createElement('p');
+  let tile = buildTile(`Downloads Pending: ${pendingTotal}`);
-  message.innerHTML = `
+  const pendingVideos = responseData.pending_videos || 0;
-        pending: ${responseData.downloads.pending || 0}<br>
+  const pendingShorts = responseData.pending_shorts || 0;
-        ignored: ${responseData.downloads.ignore || 0}<br>
+  const pendingStreams = responseData.pending_streams || 0;
-    `;
+  const content = {
-  tile.appendChild(message);
+    Videos: pendingVideos,
+    Shorts: pendingShorts,
+    Streams: pendingStreams,
+  };
+  const table = buildTileContenTable(content, 3);
+  tile.appendChild(table);
 
   return tile;
 }
@@ -81,24 +192,38 @@ function watchStats() {
   let watchBox = document.getElementById('watchBox');
   clearLoading(watchBox);
 
-  for (const property in responseData) {
+  let watchedTile = buildWatchTile('watched', responseData.watched);
-    let tile = buildWatchTile(property, responseData[property]);
+  watchBox.appendChild(watchedTile);
-    watchBox.appendChild(tile);
+
-  }
+  let unwatchedTile = buildWatchTile('unwatched', responseData.unwatched);
+  watchBox.appendChild(unwatchedTile);
 }
 
 function buildWatchTile(title, watchDetail) {
-  let tile = buildTile(`Total ${title}`);
+  const items = watchDetail?.items ?? 0;
-  let message = document.createElement('p');
+  const duration = watchDetail?.duration ?? 0;
-  message.innerHTML = `
+  const duration_str = watchDetail?.duration_str ?? '0s';
-    ${watchDetail.items} Videos<br>
+  const hasProgess = !!watchDetail?.progress;
-    ${watchDetail.duration} Seconds<br>
+  const progress = (Number(watchDetail?.progress) * 100).toFixed(2) ?? '0';
-    ${watchDetail.duration_str} Playback
+
-  `;
+  let titleCapizalized = capitalizeFirstLetter(title);
-  if (watchDetail.progress) {
+
-    message.innerHTML += `<br>${Number(watchDetail.progress * 100).toFixed(2)}%`;
+  if (hasProgess) {
+    titleCapizalized = `${progress}% ` + titleCapizalized;
   }
-  tile.appendChild(message);
+
+  let tile = buildTile(titleCapizalized);
+
+  const content = {
+    Videos: items,
+    Seconds: duration,
+    Duration: duration_str,
+  };
+
+  const table = buildTileContenTable(content, 3);
+
+  tile.appendChild(table);
+
   return tile;
 }
 
@@ -123,31 +248,83 @@ function downloadHist() {
 function buildDailyStat(dailyStat) {
   let tile = buildTile(dailyStat.date);
   let message = document.createElement('p');
-  message.innerText = `new videos: ${dailyStat.count}`;
+  const isExactlyOne = dailyStat.count === 1;
+
+  let text = 'Videos';
+  if (isExactlyOne) {
+    text = 'Video';
+  }
+
+  message.innerText = `+${dailyStat.count} ${text}
+  ${humanFileSize(dailyStat.media_size)}`;
+
   tile.appendChild(message);
   return tile;
 }
 
-function biggestChannel() {
+function buildChannelRow(id, name, value) {
-  let apiEndpoint = '/api/stats/biggestchannels/';
+  let tableRow = document.createElement('tr');
-  let responseData = apiRequest(apiEndpoint, 'GET');
+
-  let tBody = document.getElementById('biggestChannelTable');
+  tableRow.innerHTML = `
+    <td class="agg-channel-name"><a href="/channel/${id}/">${name}</a></td>
+    <td class="agg-channel-right-align">${value}</td>
+  `;
+
+  return tableRow;
+}
+
+function addBiggestChannelByDocCount() {
+  let tBody = document.getElementById('biggestChannelTableVideos');
+
+  let apiEndpoint = '/api/stats/biggestchannels/?order=doc_count';
+  const responseData = apiRequest(apiEndpoint, 'GET');
+
   for (let i = 0; i < responseData.length; i++) {
-    const channelData = responseData[i];
+    const { id, name, doc_count } = responseData[i];
-    let tableRow = buildChannelRow(channelData);
+
+    let tableRow = buildChannelRow(id, name, doc_count);
+
     tBody.appendChild(tableRow);
   }
 }
 
-function buildChannelRow(channelData) {
+function addBiggestChannelByDuration() {
-  let tableRow = document.createElement('tr');
+  const tBody = document.getElementById('biggestChannelTableDuration');
-  tableRow.innerHTML = `
+
-    <td class="agg-channel-name"><a href="/channel/${channelData.id}/">${channelData.name}</a></td>
+  let apiEndpoint = '/api/stats/biggestchannels/?order=duration';
-    <td>${channelData.doc_count}</td>
+  const responseData = apiRequest(apiEndpoint, 'GET');
-    <td>${channelData.duration_str}</td>
+
-    <td>${humanFileSize(channelData.media_size)}</td>
+  for (let i = 0; i < responseData.length; i++) {
-  `;
+    const { id, name, duration_str } = responseData[i];
-  return tableRow;
+
+    let tableRow = buildChannelRow(id, name, duration_str);
+
+    tBody.appendChild(tableRow);
+  }
+}
+
+function addBiggestChannelByMediaSize() {
+  let tBody = document.getElementById('biggestChannelTableMediaSize');
+
+  let apiEndpoint = '/api/stats/biggestchannels/?order=media_size';
+  const responseData = apiRequest(apiEndpoint, 'GET');
+
+  for (let i = 0; i < responseData.length; i++) {
+    const { id, name, media_size } = responseData[i];
+
+    let tableRow = buildChannelRow(id, name, humanFileSize(media_size));
+
+    tBody.appendChild(tableRow);
+  }
+}
+
+function clearLoading(dashBox) {
+  dashBox.querySelector('#loading').remove();
+}
+
+function capitalizeFirstLetter(string) {
+  // source: https://stackoverflow.com/a/1026087
+  return string.charAt(0).toUpperCase() + string.slice(1);
 }
 
 function humanFileSize(size) {
@@ -155,8 +332,76 @@ function humanFileSize(size) {
   return (size / Math.pow(1024, i)).toFixed(1) * 1 + ' ' + ['B', 'kB', 'MB', 'GB', 'TB'][i];
 }
 
+function buildTile(titleText) {
+  let tile = document.createElement('div');
+  tile.classList.add('info-box-item');
+
+  let title = document.createElement('h3');
+
+  title.innerText = titleText;
+  tile.appendChild(title);
+
+  return tile;
+}
+
+function buildTileContenTable(content, rowsWanted) {
+  let contentEntries = Object.entries(content);
+
+  const nbsp = '\u00A0'; // No-Break Space https://www.compart.com/en/unicode/U+00A0
+
+  // Do not add spacing rows when on mobile device
+  const isMobile = window.matchMedia('(max-width: 600px)');
+  if (!isMobile.matches) {
+    if (contentEntries.length < rowsWanted) {
+      const rowsToAdd = rowsWanted - contentEntries.length;
+
+      for (let i = 0; i < rowsToAdd; i++) {
+        contentEntries.push([nbsp, nbsp]);
+      }
+    }
+  }
+
+  const table = document.createElement('table');
+  table.classList.add('agg-channel-table');
+  const tableBody = document.createElement('tbody');
+
+  for (const [key, value] of contentEntries) {
+    const row = document.createElement('tr');
+
+    const leftCell = document.createElement('td');
+    leftCell.classList.add('agg-channel-name');
+
+    // Do not add ":" when its a spacing entry
+    const keyText = key === nbsp ? key : `${key}: `;
+    const leftText = document.createTextNode(keyText);
+    leftCell.appendChild(leftText);
+
+    const rightCell = document.createElement('td');
+    rightCell.classList.add('agg-channel-right-align');
+
+    const rightText = document.createTextNode(value);
+    rightCell.appendChild(rightText);
+
+    row.appendChild(leftCell);
+    row.appendChild(rightCell);
+
+    tableBody.appendChild(row);
+  }
+
+  table.appendChild(tableBody);
+
+  return table;
+}
+
+function biggestChannel() {
+  addBiggestChannelByDocCount();
+  addBiggestChannelByDuration();
+  addBiggestChannelByMediaSize();
+}
+
 async function buildStats() {
   primaryStats();
+  secondaryStats();
   watchStats();
   downloadHist();
   biggestChannel();