bump TA_VERSION

Changed:
- Bump yt-dlp to 2025.01.15
- Fix comment extraction

README.md

@@ -25,7 +25,7 @@
 ## Core functionality
 Once your YouTube video collection grows, it becomes hard to search and find a specific video. That's where Tube Archivist comes in: By indexing your video collection with metadata from YouTube, you can organize, search and enjoy your archived YouTube videos without hassle offline through a convenient web interface. This includes:  
 * Subscribe to your favorite YouTube channels
-* Download Videos using **yt-dlp**
+* Download Videos using **[yt-dlp](https://github.com/yt-dlp/yt-dlp)**
 * Index and make videos searchable
 * Play videos
 * Keep track of viewed and unviewed videos
@@ -201,6 +201,7 @@ This is your time to shine, [read this](https://github.com/tubearchivist/tubearc
 - [lamusmaser/create_info_json](https://gist.github.com/lamusmaser/837fb58f73ea0cad784a33497932e0dd): Script to generate `.info.json` files using `ffmpeg` collecting information from downloaded videos.
 - [lamusmaser/ta_fix_for_video_redirection](https://github.com/lamusmaser/ta_fix_for_video_redirection): Script to fix videos that were incorrectly indexed by YouTube's "Video is Unavailable" response. 
 - [RoninTech/ta-helper](https://github.com/RoninTech/ta-helper): Helper script to provide a symlink association to reference TubeArchivist videos with their original titles.
+- [tangyjoust/Tautulli-Notify-TubeArchivist-of-Plex-Watched-State](https://github.com/tangyjoust/Tautulli-Notify-TubeArchivist-of-Plex-Watched-State) Mark videos watched in Plex (through streaming not manually) through Tautulli back to TubeArchivist
 
 ## Donate
 The best donation to **Tube Archivist** is your time, take a look at the [contribution page](CONTRIBUTING.md) to get started.  
@@ -212,6 +213,8 @@ Second best way to support the development is to provide for caffeinated beverag
 
 ## Notable mentions
 This is a selection of places where this project has been featured on reddit, in the news, blogs or any other online media, newest on top.
+* **xda-developers.com**: 5 obscure self-hosted services worth checking out - Tube Archivist - To save your essential YouTube videos, [2024-10-13][[link](https://www.xda-developers.com/obscure-self-hosted-services/)]
+* **selfhosted.show**: why we're trying Tube Archivist, [2024-06-14][[link](https://selfhosted.show/125)]
 * **ycombinator**: Tube Archivist on Hackernews front page, [2023-07-16][[link](https://news.ycombinator.com/item?id=36744395)]
 * **linux-community.de**: Tube Archivist bringt Ordnung in die Youtube-Sammlung, [German][2023-05-01][[link](https://www.linux-community.de/ausgaben/linuxuser/2023/05/tube-archivist-bringt-ordnung-in-die-youtube-sammlung/)]
 * **noted.lol**: Dev Debrief, An Interview With the Developer of Tube Archivist, [2023-03-30] [[link](https://noted.lol/dev-debrief-tube-archivist/)]
@@ -223,13 +226,3 @@ This is a selection of places where this project has been featured on reddit, in
 * **reddit.com**: Celebrating TubeArchivist v0.1, [2022-01-09] [[link](https://www.reddit.com/r/selfhosted/comments/rzh084/celebrating_tubearchivist_v01/)]
 * **linuxunplugged.com**: Pick: tubearchivist — Your self-hosted YouTube media server, [2021-09-11] [[link](https://linuxunplugged.com/425)] and [2021-10-05] [[link](https://linuxunplugged.com/426)]
 * **reddit.com**: Introducing Tube Archivist, your self hosted Youtube media server, [2021-09-12] [[link](https://www.reddit.com/r/selfhosted/comments/pmj07b/introducing_tube_archivist_your_self_hosted/)]
-
-
-## Sponsor
-Big thank you to [Digitalocean](https://www.digitalocean.com/) for generously donating credit for the tubearchivist.com VPS and buildserver. 
-<p>
-  <a href="https://www.digitalocean.com/">
-    <img src="https://opensource.nyc3.cdn.digitaloceanspaces.com/attribution/assets/PoweredByDO/DO_Powered_by_Badge_blue.svg" width="201px">
-  </a>
-</p>
-

deploy.sh

@@ -157,7 +157,7 @@ function sync_docker {
     fi
 
     echo "latest tags:"
-    git tag | tail -n 5 | sort -r
+    git tag | sort -rV | head -n 5
 
     printf "\ncreate new version:\n"
     read -r VERSION
@@ -189,7 +189,7 @@ function sync_docker_old {
     fi
 
     echo "latest tags:"
-    git tag | tail -n 5 | sort -r
+    git tag | sort -rV | head -n 5
 
     printf "\ncreate new version:\n"
     read -r VERSION

docker-compose.yml

@@ -40,7 +40,7 @@ services:
     depends_on:
       - archivist-es
   archivist-es:
-    image: bbilly1/tubearchivist-es         # only for amd64, or use official es 8.14.3
+    image: bbilly1/tubearchivist-es         # only for amd64, or use official es 8.16.0
     container_name: archivist-es
     restart: unless-stopped
     environment:

tubearchivist/config/management/commands/ta_config_backup.py

@@ -0,0 +1,76 @@
+"""backup config for sqlite reset and restore"""
+
+import json
+from pathlib import Path
+
+from django.contrib.auth import get_user_model
+from django.core.management.base import BaseCommand
+from home.models import CustomPeriodicTask
+from home.src.ta.settings import EnvironmentSettings
+from rest_framework.authtoken.models import Token
+
+User = get_user_model()
+
+
+class Command(BaseCommand):
+    """export"""
+
+    help = "Exports all users and their auth tokens to a JSON file"
+    FILE = Path(EnvironmentSettings.CACHE_DIR) / "backup" / "migration.json"
+
+    def handle(self, *args, **kwargs):
+        """entry point"""
+
+        data = {
+            "user_data": self.get_users(),
+            "schedule_data": self.get_schedules(),
+        }
+
+        with open(self.FILE, "w", encoding="utf-8") as json_file:
+            json_file.write(json.dumps(data))
+
+    def get_users(self):
+        """get users"""
+
+        users = User.objects.all()
+
+        user_data = []
+
+        for user in users:
+            user_info = {
+                "username": user.name,
+                "is_staff": user.is_staff,
+                "is_superuser": user.is_superuser,
+                "password": user.password,
+                "tokens": [],
+            }
+
+            try:
+                token = Token.objects.get(user=user)
+                user_info["tokens"] = [token.key]
+            except Token.DoesNotExist:
+                user_info["tokens"] = []
+
+            user_data.append(user_info)
+
+        return user_data
+
+    def get_schedules(self):
+        """get schedules"""
+
+        all_schedules = CustomPeriodicTask.objects.all()
+        schedule_data = []
+
+        for schedule in all_schedules:
+            schedule_info = {
+                "name": schedule.name,
+                "crontab": {
+                    "minute": schedule.crontab.minute,
+                    "hour": schedule.crontab.hour,
+                    "day_of_week": schedule.crontab.day_of_week,
+                },
+            }
+
+            schedule_data.append(schedule_info)
+
+        return schedule_data

tubearchivist/config/settings.py

@@ -276,4 +276,4 @@ CORS_ALLOW_HEADERS = list(default_headers) + [
 
 # TA application settings
 TA_UPSTREAM = "https://github.com/tubearchivist/tubearchivist"
-TA_VERSION = "v0.4.10"
+TA_VERSION = "v0.4.13"

tubearchivist/home/src/index/comments.py

@@ -69,7 +69,6 @@ class Comments:
                 "youtube": {
                     "max_comments": max_comments_list,
                     "comment_sort": [comment_sort],
-                    "player_client": ["ios", "web"],  # workaround yt-dlp #9554
                 }
             },
         }

tubearchivist/home/src/index/playlist.py

@@ -93,13 +93,10 @@ class YoutubePlaylist(YouTubeItem):
         """get all videos in playlist, match downloaded with ids_found"""
         all_members = []
         for idx, entry in enumerate(self.youtube_meta["entries"]):
-            if not entry["channel"]:
-                continue
-
             to_append = {
                 "youtube_id": entry["id"],
                 "title": entry["title"],
-                "uploader": entry["channel"],
+                "uploader": entry.get("channel"),
                 "idx": idx,
                 "downloaded": entry["id"] in ids_found,
             }

tubearchivist/requirements-dev.txt

@@ -1,10 +1,11 @@
 -r requirements.txt
-black
-codespell
-flake8
-isort
-pylint
-pylint-django
-pytest
-pytest-django
-types-requests
+black==24.10.0
+codespell==2.3.0
+flake8==7.1.1
+isort==5.13.2
+pylint-django==2.6.1
+pylint==3.3.3
+pytest-django==4.9.0
+pytest==8.3.4
+requirementscheck==0.0.5
+types-requests==2.32.0.20241016

tubearchivist/requirements.txt

@@ -1,14 +1,14 @@
-apprise==1.8.1
+apprise==1.9.2
 celery==5.4.0
-Django==5.0.7
-django-auth-ldap==4.8.0
-django-celery-beat==2.6.0
-django-cors-headers==4.4.0
+django-auth-ldap==5.1.0
+django-celery-beat==2.7.0
+django-cors-headers==4.6.0
+Django==5.1.5
 djangorestframework==3.15.2
-Pillow==10.4.0
-redis==5.0.8
+Pillow==11.1.0
+redis==5.2.1
 requests==2.32.3
 ryd-client==0.0.6
-uWSGI==2.0.26
-whitenoise==6.7.0
-yt-dlp==2024.8.1
+uWSGI==2.0.28
+whitenoise==6.8.2
+yt-dlp[default]==2025.1.15