bump TA_VERSION

Changed:
- Bump yt-dlp to 2025.01.15
- Fix comment extraction

.github/ISSUE_TEMPLATE/FEATURE-REQUEST.yml

@@ -1,34 +1,12 @@
 name: Feature Request
-description: I have an idea for a great addition to this project
+description: This Project currently doesn't take any new feature requests.
 title: "[Feature Request]: "
 
 body:
-  - type: markdown
-    attributes:
-      value: |
-        Thanks for taking the time to help improve this project! This project is *very* selective with accepting new feature requests. Please read the [how to open an issue](https://github.com/tubearchivist/tubearchivist/blob/master/CONTRIBUTING.md#how-to-open-an-issue) guide carefully before continuing.
-
   - type: checkboxes
-    id: already
+    id: block
     attributes:
-      label: "I've read the documentation"
+      label: "This project doesn't accept any new feature requests for the forseeable future. There is no shortage of ideas and the next development steps are clear for years to come."
       options:
-        - label: I have read the [how to open an issue](https://github.com/tubearchivist/tubearchivist/blob/master/CONTRIBUTING.md#how-to-open-an-issue) guide, particularly the [feature request](https://github.com/tubearchivist/tubearchivist/blob/master/CONTRIBUTING.md#feature-request) section.
+        - label: I understand that this issue will be closed without comment.
           required: true
-
-  - type: textarea
-    id: description
-    attributes:
-      label: Your Feature Request
-      value: "## Is your feature request related to a problem? Please describe.\n\n## Describe the solution you'd like\n\n## Additional context"
-      placeholder: Tell us what you see!
-    validations:
-      required: true
-
-  - type: checkboxes
-    id: help
-    attributes:
-      label: Your help is needed!
-      description: This project is ambitious as it is, please contribute.
-      options:
-        - label: Yes I will work on this in the next few days or weeks.

CONTRIBUTING.md

@@ -62,22 +62,7 @@ Please keep in mind:
 - A bug that can't be reproduced, is difficult or sometimes even impossible to fix. Provide very clear steps *how to reproduce*.
 
 ### Feature Request
-This project needs your help to grow further. There is no shortage of ideas, see the open [issues on GH](https://github.com/tubearchivist/tubearchivist/issues?q=is%3Aopen+is%3Aissue+label%3Aenhancement) and the [roadmap](https://github.com/tubearchivist/tubearchivist#roadmap), what this project lacks is contributors interested in helping with overall improvements of the application. Focus is *not* on adding new features, but improving existing ones.
-
-Existing ideas are easily *multiple years* worth of development effort, at least at current speed. This project is *very* selective with accepting new feature requests at this point.  
-
-Good feature requests usually fall into one or more of these categories:
-- You want to work on your own small scoped idea within the next few days or weeks.
-- Your idea is beneficial for a wide range of users, not just for you.
-- Your idea extends the current project by building on and improving existing functionality.
-- Your idea is quick and easy to implement, for an experienced as well as for a first time contributor.
-
-Your request is likely going to be rejected if:
-- Your idea requires multiple days worth of development time and is unrealistic to be implemented any time soon.
-- There are already other ways to do what you are trying to do.
-- You are trying to do something that only applies to your platform, your specific workflow or your specific setup.
-- Your idea would fundamentally change how the project works or it wouldn't be able to be implemented with backwards compatibility.
-- Your idea is not a good fit for this project.
+This project doesn't take any new feature requests. This project doesn't lack ideas, see the currently open tasks and roadmap. New feature requests aren't helpful at this point in time. Thank you for your understanding.
 
 ### Installation Help
 GitHub is most likely not the best place to ask for installation help. That's inherently individual and one on one.

README.md

@@ -25,7 +25,7 @@
 ## Core functionality
 Once your YouTube video collection grows, it becomes hard to search and find a specific video. That's where Tube Archivist comes in: By indexing your video collection with metadata from YouTube, you can organize, search and enjoy your archived YouTube videos without hassle offline through a convenient web interface. This includes:  
 * Subscribe to your favorite YouTube channels
-* Download Videos using **yt-dlp**
+* Download Videos using **[yt-dlp](https://github.com/yt-dlp/yt-dlp)**
 * Index and make videos searchable
 * Play videos
 * Keep track of viewed and unviewed videos
@@ -148,7 +148,6 @@ This can happen if you have nested virtualizations, e.g. LXC running Docker in P
 ## Roadmap
 We have come far, nonetheless we are not short of ideas on how to improve and extend this project. Issues waiting for you to be tackled in no particular order:
 
-- [ ] User roles
 - [ ] Audio download
 - [ ] Podcast mode to serve channel as mp3
 - [ ] Random and repeat controls ([#108](https://github.com/tubearchivist/tubearchivist/issues/108), [#220](https://github.com/tubearchivist/tubearchivist/issues/220))
@@ -159,10 +158,12 @@ We have come far, nonetheless we are not short of ideas on how to improve and ex
 - [ ] Custom searchable notes to videos, channels, playlists ([#144](https://github.com/tubearchivist/tubearchivist/issues/144))
 - [ ] Search comments
 - [ ] Search download queue
-- [ ] Configure shorts, streams and video sizes per channel
+- [ ] Per user videos/channel/playlists
 
 Implemented:
+- [X] Configure shorts, streams and video sizes per channel [2024-07-15]
 - [X] User created playlists [2024-04-10]
+- [X] User roles, aka read only user [2023-11-10]
 - [X] Add statistics of index [2023-09-03]
 - [X] Implement [Apprise](https://github.com/caronc/apprise) for notifications [2023-08-05]
 - [X] Download video comments [2022-11-30]
@@ -200,6 +201,7 @@ This is your time to shine, [read this](https://github.com/tubearchivist/tubearc
 - [lamusmaser/create_info_json](https://gist.github.com/lamusmaser/837fb58f73ea0cad784a33497932e0dd): Script to generate `.info.json` files using `ffmpeg` collecting information from downloaded videos.
 - [lamusmaser/ta_fix_for_video_redirection](https://github.com/lamusmaser/ta_fix_for_video_redirection): Script to fix videos that were incorrectly indexed by YouTube's "Video is Unavailable" response. 
 - [RoninTech/ta-helper](https://github.com/RoninTech/ta-helper): Helper script to provide a symlink association to reference TubeArchivist videos with their original titles.
+- [tangyjoust/Tautulli-Notify-TubeArchivist-of-Plex-Watched-State](https://github.com/tangyjoust/Tautulli-Notify-TubeArchivist-of-Plex-Watched-State) Mark videos watched in Plex (through streaming not manually) through Tautulli back to TubeArchivist
 
 ## Donate
 The best donation to **Tube Archivist** is your time, take a look at the [contribution page](CONTRIBUTING.md) to get started.  
@@ -211,6 +213,8 @@ Second best way to support the development is to provide for caffeinated beverag
 
 ## Notable mentions
 This is a selection of places where this project has been featured on reddit, in the news, blogs or any other online media, newest on top.
+* **xda-developers.com**: 5 obscure self-hosted services worth checking out - Tube Archivist - To save your essential YouTube videos, [2024-10-13][[link](https://www.xda-developers.com/obscure-self-hosted-services/)]
+* **selfhosted.show**: why we're trying Tube Archivist, [2024-06-14][[link](https://selfhosted.show/125)]
 * **ycombinator**: Tube Archivist on Hackernews front page, [2023-07-16][[link](https://news.ycombinator.com/item?id=36744395)]
 * **linux-community.de**: Tube Archivist bringt Ordnung in die Youtube-Sammlung, [German][2023-05-01][[link](https://www.linux-community.de/ausgaben/linuxuser/2023/05/tube-archivist-bringt-ordnung-in-die-youtube-sammlung/)]
 * **noted.lol**: Dev Debrief, An Interview With the Developer of Tube Archivist, [2023-03-30] [[link](https://noted.lol/dev-debrief-tube-archivist/)]
@@ -222,13 +226,3 @@ This is a selection of places where this project has been featured on reddit, in
 * **reddit.com**: Celebrating TubeArchivist v0.1, [2022-01-09] [[link](https://www.reddit.com/r/selfhosted/comments/rzh084/celebrating_tubearchivist_v01/)]
 * **linuxunplugged.com**: Pick: tubearchivist — Your self-hosted YouTube media server, [2021-09-11] [[link](https://linuxunplugged.com/425)] and [2021-10-05] [[link](https://linuxunplugged.com/426)]
 * **reddit.com**: Introducing Tube Archivist, your self hosted Youtube media server, [2021-09-12] [[link](https://www.reddit.com/r/selfhosted/comments/pmj07b/introducing_tube_archivist_your_self_hosted/)]
-
-
-## Sponsor
-Big thank you to [Digitalocean](https://www.digitalocean.com/) for generously donating credit for the tubearchivist.com VPS and buildserver. 
-<p>
-  <a href="https://www.digitalocean.com/">
-    <img src="https://opensource.nyc3.cdn.digitaloceanspaces.com/attribution/assets/PoweredByDO/DO_Powered_by_Badge_blue.svg" width="201px">
-  </a>
-</p>
-

deploy.sh

@@ -157,7 +157,7 @@ function sync_docker {
     fi
 
     echo "latest tags:"
-    git tag | tail -n 5 | sort -r
+    git tag | sort -rV | head -n 5
 
     printf "\ncreate new version:\n"
     read -r VERSION
@@ -189,7 +189,7 @@ function sync_docker_old {
     fi
 
     echo "latest tags:"
-    git tag | tail -n 5 | sort -r
+    git tag | sort -rV | head -n 5
 
     printf "\ncreate new version:\n"
     read -r VERSION

docker-compose.yml

@@ -40,7 +40,7 @@ services:
     depends_on:
       - archivist-es
   archivist-es:
-    image: bbilly1/tubearchivist-es         # only for amd64, or use official es 8.13.2
+    image: bbilly1/tubearchivist-es         # only for amd64, or use official es 8.16.0
     container_name: archivist-es
     restart: unless-stopped
     environment:

tubearchivist/config/management/commands/ta_config_backup.py

@@ -0,0 +1,76 @@
+"""backup config for sqlite reset and restore"""
+
+import json
+from pathlib import Path
+
+from django.contrib.auth import get_user_model
+from django.core.management.base import BaseCommand
+from home.models import CustomPeriodicTask
+from home.src.ta.settings import EnvironmentSettings
+from rest_framework.authtoken.models import Token
+
+User = get_user_model()
+
+
+class Command(BaseCommand):
+    """export"""
+
+    help = "Exports all users and their auth tokens to a JSON file"
+    FILE = Path(EnvironmentSettings.CACHE_DIR) / "backup" / "migration.json"
+
+    def handle(self, *args, **kwargs):
+        """entry point"""
+
+        data = {
+            "user_data": self.get_users(),
+            "schedule_data": self.get_schedules(),
+        }
+
+        with open(self.FILE, "w", encoding="utf-8") as json_file:
+            json_file.write(json.dumps(data))
+
+    def get_users(self):
+        """get users"""
+
+        users = User.objects.all()
+
+        user_data = []
+
+        for user in users:
+            user_info = {
+                "username": user.name,
+                "is_staff": user.is_staff,
+                "is_superuser": user.is_superuser,
+                "password": user.password,
+                "tokens": [],
+            }
+
+            try:
+                token = Token.objects.get(user=user)
+                user_info["tokens"] = [token.key]
+            except Token.DoesNotExist:
+                user_info["tokens"] = []
+
+            user_data.append(user_info)
+
+        return user_data
+
+    def get_schedules(self):
+        """get schedules"""
+
+        all_schedules = CustomPeriodicTask.objects.all()
+        schedule_data = []
+
+        for schedule in all_schedules:
+            schedule_info = {
+                "name": schedule.name,
+                "crontab": {
+                    "minute": schedule.crontab.minute,
+                    "hour": schedule.crontab.hour,
+                    "day_of_week": schedule.crontab.day_of_week,
+                },
+            }
+
+            schedule_data.append(schedule_info)
+
+        return schedule_data

tubearchivist/config/settings.py

@@ -276,4 +276,4 @@ CORS_ALLOW_HEADERS = list(default_headers) + [
 
 # TA application settings
 TA_UPSTREAM = "https://github.com/tubearchivist/tubearchivist"
-TA_VERSION = "v0.4.9"
+TA_VERSION = "v0.4.13"

tubearchivist/home/src/index/comments.py

@@ -69,7 +69,6 @@ class Comments:
                 "youtube": {
                     "max_comments": max_comments_list,
                     "comment_sort": [comment_sort],
-                    "player_client": ["ios", "web"],  # workaround yt-dlp #9554
                 }
             },
         }

tubearchivist/home/src/index/playlist.py

@@ -93,13 +93,10 @@ class YoutubePlaylist(YouTubeItem):
         """get all videos in playlist, match downloaded with ids_found"""
         all_members = []
         for idx, entry in enumerate(self.youtube_meta["entries"]):
-            if not entry["channel"]:
-                continue
-
             to_append = {
                 "youtube_id": entry["id"],
                 "title": entry["title"],
-                "uploader": entry["channel"],
+                "uploader": entry.get("channel"),
                 "idx": idx,
                 "downloaded": entry["id"] in ids_found,
             }

tubearchivist/requirements-dev.txt

@@ -1,10 +1,11 @@
 -r requirements.txt
-black
-codespell
-flake8
-isort
-pylint
-pylint-django
-pytest
-pytest-django
-types-requests
+black==24.10.0
+codespell==2.3.0
+flake8==7.1.1
+isort==5.13.2
+pylint-django==2.6.1
+pylint==3.3.3
+pytest-django==4.9.0
+pytest==8.3.4
+requirementscheck==0.0.5
+types-requests==2.32.0.20241016

tubearchivist/requirements.txt

@@ -1,14 +1,14 @@
-apprise==1.8.0
+apprise==1.9.2
 celery==5.4.0
-Django==5.0.7
-django-auth-ldap==4.8.0
-django-celery-beat==2.6.0
-django-cors-headers==4.4.0
+django-auth-ldap==5.1.0
+django-celery-beat==2.7.0
+django-cors-headers==4.6.0
+Django==5.1.5
 djangorestframework==3.15.2
-Pillow==10.4.0
-redis==5.0.7
+Pillow==11.1.0
+redis==5.2.1
 requests==2.32.3
 ryd-client==0.0.6
-uWSGI==2.0.26
-whitenoise==6.7.0
-yt-dlp==2024.7.9
+uWSGI==2.0.28
+whitenoise==6.8.2
+yt-dlp[default]==2025.1.15