bumping versions for new release

.dockerignore

@@ -17,5 +17,8 @@ venv/
 # Unneeded graphics
 assets/*
 
+# Unneeded docs
+docs/*
+
 # for local testing only
 testing.sh

CONTRIBUTING.md

@@ -6,7 +6,32 @@ If you haven't already, the best place to start is the README. This will give yo
 ## Report a bug
 
 If you notice something is not working as expected, check to see if it has been previously reported in the [open issues](https://github.com/bbilly1/tubearchivist/issues).
-If it has not yet been disclosed, go ahead and create an issue.
+If it has not yet been disclosed, go ahead and create an issue.  
+If the issue doesn't move forward due to a lack of response, I assume it's solved and will close it after some time to keep the list fresh. 
+
+## Wiki
+
+WIP: The wiki is where all user functions are explained in detail. These pages are mirrored into the **docs** folder of the repo. This allows for pull requests and all other features like regular code. Make any changes there, and I'll sync them  with the wiki tab.
+
+## Development Environment
+
+I have learned the hard way, that working on a dockerized application outside of docker is very error prone and in general not a good idea. So if you want to test your changes, it's best to run them in a docker testing environment.  
+
+This is my setup I have landed on, YMMV:
+- Clone the repo, work on it with your favorite code editor in your local filesystem. *testing* branch is the where all the changes are happening, might be unstable and is WIP.
+- Then I have a VM on KVM hypervisor running standard Ubuntu Server LTS with docker installed. The VM keeps my projects separate and offers convenient snapshot functionality. The VM also offers ways to simulate lowend environments by limiting CPU cores and memory. But you could also just run docker on your host system.
+- Additionally to the required services as listed in the example docker-compose file, the **Dev Tools** of [Kibana](https://www.elastic.co/guide/en/kibana/current/docker.html) are invaluable for running and testing Elasticsearch queries.
+- The `Dockerfile` is structured in a way that the actual application code is in the last layer so rebuilding the image with only code changes utilizes the build cache for everything else and will take just 2-3 secs.
+- Take a look at the `deploy.sh` file. I have my local DNS resolve `tubearchivist.local` to the IP of the VM for convenience. To deploy the latest changes and rebuild the application to the testing VM run:
+```bash
+./deploy.sh test
+```
+- The command above will also copy the file `tubarchivist/testing.sh` into the working folder of the container. Running this script will install additional debugging tools I regularly use in testing. 
+- This `deploy.sh` file is not meant to be universally usable for every possible environment but could serve as an idea on how to automatically rebuild containers to test changes - customize to your liking. 
+
+## Implementing a new feature
+
+Do you see anything on the roadmap that you would like to take a closer look at but you are not sure, what's the best way to tackle that? Or anything not on there yet you'd like to implement but are not sure how? Open up an issue and we try to find a solution together.
 
 ## Making changes
 

Dockerfile

@@ -7,10 +7,20 @@ ENV PYTHONUNBUFFERED 1
 # install distro packages needed
 RUN apt-get clean && apt-get -y update && apt-get -y install --no-install-recommends \
     build-essential \
-    ffmpeg \
     nginx \
     curl && rm -rf /var/lib/apt/lists/*
 
+# get newest patched ffmpeg and ffprobe builds
+RUN curl -s https://api.github.com/repos/yt-dlp/FFmpeg-Builds/releases/latest \
+    | grep browser_download_url \
+    | grep linux64 \
+    | grep ffmpeg-n \
+    | cut -d '"' -f 4 \
+    | xargs curl -L --output ffmpeg.tar.xz && \
+    tar -xf ffmpeg.tar.xz --strip-components=2 --no-anchored -C /usr/bin/ "ffmpeg" && \
+    tar -xf ffmpeg.tar.xz --strip-components=2 --no-anchored -C /usr/bin/ "ffprobe" && \
+    rm ffmpeg.tar.xz
+
 # copy config files
 COPY nginx.conf /etc/nginx/conf.d/
 

README.md

@@ -2,6 +2,19 @@
 
 <center><h1>Your self hosted YouTube media server</h1></center>
 
+## Table of contents:
+* [Wiki](https://github.com/bbilly1/tubearchivist/wiki) for a detailed documentation
+* [Core functionality](#core-functionality)
+* [Screenshots](#screenshots)
+* [Problem Tube Archivist tries to solve](#problem-tube-archivist-tries-to-solve)
+* [Installing and updating](#installing-and-updating)
+* [Getting Started](#getting-started)
+* [Potential pitfalls](#potential-pitfalls)
+* [Roadmap](#roadmap)
+* [Known limitations](#known-limitations)
+* [Donate](#donate)
+
+------------------------
 
 ## Core functionality
 * Subscribe to your favorite YouTube channels
@@ -29,7 +42,7 @@
 ## Problem Tube Archivist tries to solve
 Once your YouTube video collection grows, it becomes hard to search and find a specific video. That's where Tube Archivist comes in: By indexing your video collection with metadata from YouTube, you can organize, search and enjoy your archived YouTube videos without hassle offline through a convenient web interface.
 
-## Installation
+## Installing and updating
 Take a look at the example `docker-compose.yml` file provided. Tube Archivist depends on three main components split up into separate docker containers:  
 
 ### Tube Archivist
@@ -52,40 +65,16 @@ Functions as a cache and temporary link between the application and the file sys
   - Needs to be accessible over the default port `6379`
   - Takes an optional volume at **/data** to make your configuration changes permanent.
 
-## Getting Started
-1. Go through the **settings** page and look at the available options. Particularly set *Download Format* to your desired video quality before downloading. **Tube Archivist** downloads the best available quality by default.
-2. Subscribe to some of your favorite YouTube channels on the **channels** page. 
-3. On the **downloads** page, click on *Rescan subscriptions* to add videos from the subscribed channels to your Download queue or click on *Add to download queue* to manually add Video IDs, links, channels or playlists.
-4. Click on *Download queue* and let Tube Archivist to it's thing. 
-5. Enjoy your archived collection!
-  
-## Import your existing library
-So far this depends on the video you are trying to import to be still available on YouTube to get the metadata. Add the files you like to import to the */cache/import* folder. Then start the process from the settings page *Manual media files import*. Make sure to follow one of the two methods below.
+### Redis on a custom port
+For some architectures it might be required to run Redis JSON on a nonstandard port. To for example change the Redis port to **6380**, set the following values:
+- Set the environment variable `REDIS_PORT=6380` to the *tubearchivist* service.
+- For the *archivist-redis* service, change the ports to `6380:6380`
+- Additionally set the following value to the *archivist-redis* service: `command: --port 6380 --loadmodule /usr/lib/redis/modules/rejson.so`
 
-### Method 1:
-Add a matching *.json* file with the media file. Both files need to have the same base name, for example:
-- For the media file: \<base-name>.mp4
-- For the JSON file: \<base-name>.info.json
-- Alternate JSON file: \<base-name>.json
-
-**Tube Archivist** then looks for the 'id' key within the JSON file to identify the video.
-
-### Method 2:
-Detect the YouTube ID from filename, this accepts the default yt-dlp naming convention for file names like:
-- \<base-name>[\<youtube-id>].mp4
-- The YouTube ID in square brackets at the end of the filename is the crucial part.
-
-### Some notes:
-- This will **consume** the files you put into the import folder: Files will get converted to mp4 if needed (this might take a long time...) and moved to the archive, *.json* files will get deleted upon completion to avoid having duplicates on the next run.
-- Maybe start with a subset of your files to import to make sure everything goes well...
-- Follow the logs to monitor progress and errors: `docker-compose logs -f tubearchivist`.
-
-## Backup and restore
-From the settings page you can backup your metadata into a zip file. The file will get stored at *cache/backup* and will contain the necessary files to restore the Elasticsearch index formatted **nd-json** files as well a complete export of the index in a set of conventional **json** files.  
-
-The restore functionality will expect the same zip file in *cache/backup* and will recreate the index from the snapshot.  
-
-BE AWARE: This will **replace** your current index with the one from the backup file.
+### Updating Tube Archivist
+You will see the current version number of **Tube Archivist** in the footer of the interface so you can compare it with the latest release to make sure you are running the *latest and greatest*.  
+* There can be breaking changes between updates, particularly as the application grows, new environment variables or settings might be required for you to set in the your docker-compose file. Any breaking changes will be marked in the **release notes**.  
+* All testing and development is done with the Elasticsearch version number as mentioned in the provided *docker-compose.yml* file. This will be updated when a new release of Elasticsearch is available. Running an older version of Elasticsearch is most likely not going to result in any issues, but it's still recommended to run the same version as mentioned.
 
 ## Potential pitfalls
 ### vm.max_map_count
@@ -109,6 +98,13 @@ chown 1000:0 /path/to/mount/point
 ```
 This will match the permissions with the **UID** and **GID** of elasticsearch within the container and should fix the issue.
 
+## Getting Started
+1. Go through the **settings** page and look at the available options. Particularly set *Download Format* to your desired video quality before downloading. **Tube Archivist** downloads the best available quality by default.
+2. Subscribe to some of your favorite YouTube channels on the **channels** page. 
+3. On the **downloads** page, click on *Rescan subscriptions* to add videos from the subscribed channels to your Download queue or click on *Add to download queue* to manually add Video IDs, links, channels or playlists.
+4. Click on *Start download* and let **Tube Archivist** to it's thing. 
+5. Enjoy your archived collection!
+  
 ## Roadmap
 This should be considered as a **minimal viable product**, there is an extensive list of future functions and improvements planned.
 
@@ -119,8 +115,9 @@ This should be considered as a **minimal viable product**, there is an extensive
 - [ ] Create playlists
 - [ ] Podcast mode to serve channel as mp3
 - [ ] Implement [PyFilesystem](https://github.com/PyFilesystem/pyfilesystem2) for flexible video storage
-- [ ] Dynamic download queue
 - [ ] Un-ignore videos
+- [ ] Add thumbnail embed option
+- [X] Dynamic download queue [2021-09-26]
 - [X] Backup and restore [2021-09-22]
 - [X] Scan your file system to index already downloaded videos [2021-09-14]
 
@@ -135,4 +132,12 @@ This should be considered as a **minimal viable product**, there is an extensive
 ## Known limitations
 - Video files created by Tube Archivist need to be **mp4** video files for best browser compatibility.
 - Every limitation of **yt-dlp** will also be present in Tube Archivist. If **yt-dlp** can't download or extract a video for any reason, Tube Archivist won't be able to either.
-- For now this is meant to be run in a trusted network environment.
+- For now this is meant to be run in a trusted network environment. There is *no* security.
+
+
+## Donate
+The best donation to **Tube Archivist** is your time, take a look at the [contribution page](CONTRIBUTING.md) to get started.  
+Second best way to support the development is to provide for caffeinated beverages:
+* [Paypal.me](https://paypal.me/bbilly1) for a one time coffee
+* [Paypal Subscription](https://www.paypal.com/webapps/billing/plans/subscribe?plan_id=P-03770005GR991451KMFGVPMQ) for a monthly coffee
+* [co-fi.com](https://ko-fi.com/bbilly1) for an alternative platform

deploy.sh

@@ -108,7 +108,8 @@ function sync_docker {
     printf "\nlatest images:\n"
     sudo docker image ls bbilly1/tubearchivist
 
-    read -s "Push?"
+    echo "continue?"
+    read -rn 1
 
     # push to docker
     echo "pushing latest:"

docker-compose.yml

@@ -29,7 +29,7 @@ services:
     depends_on:
       - archivist-es
   archivist-es:
-    image: docker.elastic.co/elasticsearch/elasticsearch:7.14.1
+    image: docker.elastic.co/elasticsearch/elasticsearch:7.15.0
     container_name: archivist-es
     restart: always
     environment:

docs/Channels.md

@@ -0,0 +1,23 @@
+# Channels Overview and Channel Detail Page
+
+The channels are organized on two different levels:
+
+## Channels Overview
+Accessible at `/channel/` of your Tube Archivist, the **Overview Page** shows a list of all channels you have indexed. 
+- You can filter that list to show or hide subscribed channels from the drop down menu. Clicking on the channel banner or the channel name will direct you to the *Channel Detail Page*.
+- If you are subscribed to a channel a *Unsubscribe* button will show.
+
+The **Subscribe to Channels** button <img src="assets/icon-add.png?raw=true" alt="add icon" width="20px" style="margin:0 5px;"> opens a text field to subscribe to a channel. You have a few options:
+- Enter the YouTube channel ID, a 25 character alphanumeric string. For example *UCBa659QWEk1AI4Tg--mrJ2A*
+- Enter the URL to the channel page on YouTube. For example *https://www.youtube.com/channel/UCBa659QWEk1AI4Tg--mrJ2A*
+- Enter the video URL for any video and let Tube Archivist extract the channel ID for you. For example *https://www.youtube.com/watch?v=2tdiKTSdE9Y*
+- Add one per line.
+- **Note**: Adding a link to a YouTube channel name is not yet supported, for example: *https://www.youtube.com/c/TomScottGo* will fail.
+
+The search icon <img src="assets/icon-search.png?raw=true" alt="search icon" width="20px" style="margin:0 5px;"> opens a text box to search for indexed channel names. Possible matches will show as you type. 
+
+## Channel Detail
+Each channel will get a dedicated channel detail page accessible at `/channel/<channel-id>/` of your Tube Archivist. This page shows all the videos you have downloaded from this channel plus additional metadata. 
+- If you are subscribed to the channel, an *Unsubscribe* button will show.
+- You can *Show* the channel description, that matches with the *About* tab on YouTube.
+- The **Mark as Watched** button will mark all videos of this channel as watched.

docs/Downloads.md

@@ -0,0 +1,35 @@
+# Downloads Page
+Accessible at `/downloads/` of your Tube Archivist, this page handles all the download functionality.
+
+
+## Rescan Subscriptions
+The **Rescan Subscriptions** icon <img src="assets/icon-rescan.png?raw=true" alt="rescan icon" width="20px" style="margin:0 5px;"> will start a background task to look for new videos from the channels you are subscribed to. You can define the channel page size on the [settings page](Settings#subscriptions). With the default channel page size, expect this process to take around 2-3 seconds for each channel you are subscribed to. A status message will show the progress.
+
+Then for every video found, **Tube Archivist** will skip the video if it has already been downloaded or if you added it to the *ignored* list before. All the other videos will get added to the download queue. Expect this to take around 1 second for each video as **Tube Archivist** needs to grab some additional metadata. New videos will get added at the bottom of the download queue.
+
+## Download Queue
+The **Start Download** icon <img src="assets/icon-download.png?raw=true" alt="download icon" width="20px" style="margin:0 5px;"> will start the download process starting from the top of the queue. Take a look at the relevant settings on the [Settings Page](Settings#downloads). Once the process started, a progress message will show with additional details and controls: 
+- The stop icon <img src="assets/icon-stop.png?raw=true" alt="stop icon" width="20px" style="margin:0 5px;"> will gracefully stop the download process, once the current video has been finished successfully.
+- The cancel icon <img src="assets/icon-close-red.png?raw=true" alt="close icon" width="20px" style="margin:0 5px;"> is equivalent to killing the process and will stop the download immediately. Any leftover files will get deleted, the canceled video will still be available in the download queue.
+
+## Add to Download Queue
+The **Add to Download Queue** icon <img src="assets/icon-add.png?raw=true" alt="add icon" width="20px" style="margin:0 5px;"> opens a text field to manually add videos to the download queue. You have a few options:
+- Add a link to a YouTube video. For example *https://www.youtube.com/watch?v=2tdiKTSdE9Y*.
+- Add a YouTube video ID. For example *2tdiKTSdE9Y*.
+- Add a link to a YouTube video by providing the shortened URL, for example *https://youtu.be/2tdiKTSdE9Y*.
+- Add a Channel ID or Channel URL to add every available video to the download queue. This will ignore the channel page size as described before and is meant for an initial download of the whole channel. You can still ignore selected videos before starting the download.
+- Add a playlist ID or URL to add every available video in the list to the download queue, for example *https://www.youtube.com/playlist?list=PL96C35uN7xGLLeET0dOWaKHkAlPsrkcha* or *PL96C35uN7xGLLeET0dOWaKHkAlPsrkcha*. Note that when you add a link to a video in a playlist, Tube Archivist assumes you want to download only the specific video and not the whole playlist, for example *https://www.youtube.com/watch?v=CINVwWHlzTY&list=PL96C35uN7xGLLeET0dOWaKHkAlPsrkcha* will only add one video *CINVwWHlzTY* to the queue.
+- Add one link per line.
+- **Note**: Adding a link to a YouTube channel name is not yet supported, for example: *https://www.youtube.com/c/TomScottGo* will fail.
+
+## The Download Queue
+Below the three buttons you find the download queue. New items will get added at the bottom of the queue, the next video to download once you click on **Start Download** will be the first in the list.
+
+Every video in the download queue has two buttons:
+- **Ignore**: This will remove that video from the download queue and this video will not get added again, even when you **Rescan Subscriptions**.
+- **Download now**: This will give priority to this video. If the download process is already running, the prioritized video will get downloaded as soon as the current video is finished. If there is no download process running, this will start downloading this single video and stop after that.
+
+You can flip the view by activating **Show Only Ignored Videos**. This will show all videos you have previously *ignored*.  
+Every video in the ignored list has two buttons:
+- **Forget**: This will delete the item form the ignored list.
+- **Add to Queue**: This will add the ignored video back to the download queue.

docs/Home.md

@@ -0,0 +1,29 @@
+# Tube Archivist Wiki
+
+**WIP**: This is work in progress!
+
+Welcome to the official Tube Archivist Wiki. This is an up-to-date documentation of user functionality.
+
+Table of contents:
+* [Main](Main): Tube Archivist landing page
+* [Channels](Channels): Browse your channels, handle subscriptions
+* [Downloads](Downloads): Scanning subscriptions, handle download queue
+* [Settings](Settings): All the configuration options
+
+## Getting Started
+1. [Subscribe](Channels#channels-overview) to some of your favourite YouTube channels.
+2. [Scan](Downloads#rescan-subscriptions) subscriptions to add the latest videos to the download queue.
+3. [Add](Downloads#add-to-download-queue) additional videos, channels or playlist - ignore the ones you don't want to download.
+4. [Download](Downloads#download-queue) and let **Tube Archivist** do it's thing.
+5. Sit back and enjoy your archived and indexed collection!
+
+## General Navigation
+* Clicking on the channel name or the channel icon brings you to the dedicated channel page to show videos from that channel.
+* Clicking on a video title brings you to the dedicated video page and shows additional details.
+* Clicking on a video thumbnail opens the video player and starts streaming the selected video.
+* Hover over the playing video to show additional control options.
+
+
+An empty checkbox icon <img src="assets/icon-unseen.png?raw=true" alt="unseen icon" width="20px" style="margin:0 5px;"> will show for videos you haven't marked as watched. Click on it and the icon will change to a filled checkbox <img src="assets/icon-seen.png?raw=true" alt="seen icon" width="20px" style="margin:0 5px;"> indicating it as watched.
+
+When available the <img src="assets/icon-gridview.png?raw=true" alt="gridview icon" width="20px" style="margin:0 5px;"> gridview icon will display the list in a grid, the <img src="assets/icon-listview.png?raw=true" alt="listview icon" width="20px" style="margin:0 5px;"> listview icon will arrange the items in a list.

docs/Main.md

@@ -0,0 +1,10 @@
+# Tube Archivist Home Page Functionality
+
+This is the landing page, when you first open **Tube Archivist**. You have a few options to sort and filter that view:
+- With the **Sort Order** you can select how the "Recent Videos" are sorted: 
+  - **Date Published**: Sorts the list by date when the video was published on YouTube, newest on top.
+  - **Date Downloaded**: Sorts the list based on when you have downloaded the video to your archive, newest on top.
+- With **Hide Watched** you can filter out videos you have already marked as watched to only show unwatched videos. 
+- You can use those two options together to for example filter the list to *Hide Watched* videos **and** sort by date downloaded.
+
+Additionally the search icon <img src="assets/icon-search.png?raw=true" alt="search icon" width="20px" style="margin:0 5px;"> opens a text field to search your collection.

docs/Settings.md

@@ -0,0 +1,60 @@
+# Settings Page
+Accessible at `/settings/` of your **Tube Archivist**, this page holds all the configurations and additional functionality related to the database.
+
+Click on **Update Settings** at the bottom of the form to apply your configurations.
+
+## Color scheme
+Switch between the easy on the eyes dark theme and the burning bright theme.
+
+## Archive View
+- **Page Size**: Defines how many results get displayed on a given page. Same value goes for all archive views.
+
+## Subscriptions
+Settings related to the channel management.
+- **Channel Page Size**: Defines how many pages will get analyzed by **Tube Archivist** each time you click on *Rescan Subscriptions*. The default page size used by yt-dlp is **50**, that's also the recommended value to set here. Any value higher will slow down the rescan process, for example if you set the value to 51, that means yt-dlp will have to go through 2 pages of results instead of 1 and by that doubling the time that process takes.
+
+## Downloads
+Settings related to the download process.
+- **Download Limit**: Stop the download process after downloading the set quantity of videos.
+- **Download Speed Limit**: Set your download speed limit in KB/s. This will pass the option `--limit-rate` to yt-dlp.
+- **Sleep Interval**: Time in seconds to sleep between requests to YouTube. It's a good idea to set this to **3** seconds. Might be necessary to avoid throttling.
+
+## Download Format
+Additional settings passed to yt-dlp.
+- **Format**: This controls which streams get downloaded and is equivalent to passing `--format` to yt-dlp. Use one of the recommended one or look at the documentation of [yt-dlp](https://github.com/yt-dlp/yt-dlp#format-selection). Please note: The option `--merge-output-format mp4` is automatically passed to yt-dlp to guarantee browser compatibility.
+- **Embed Metadata**: This saves the available tags directly into the media file by passing `--embed-metadata` to yt-dlp.
+
+
+# Actions
+Additional database functionality.
+
+## Manual Media Files Import
+So far this depends on the video you are trying to import to be still available on YouTube to get the metadata. Add the files you like to import to the */cache/import* folder. Then start the process from the settings page *Manual Media Files Import*. Make sure to follow one of the two methods below.
+
+### Method 1:
+Add a matching *.json* file with the media file. Both files need to have the same base name, for example:
+- For the media file: \<base-name>.mp4
+- For the JSON file: \<base-name>.info.json
+- Alternate JSON file: \<base-name>.json
+
+**Tube Archivist** then looks for the 'id' key within the JSON file to identify the video.
+
+### Method 2:
+Detect the YouTube ID from filename, this accepts the default yt-dlp naming convention for file names like:
+- \<base-name>[\<youtube-id>].mp4
+- The YouTube ID in square brackets at the end of the filename is the crucial part.
+
+### Some notes:
+- This will **consume** the files you put into the import folder: Files will get converted to mp4 if needed (this might take a long time...) and moved to the archive, *.json* files will get deleted upon completion to avoid having duplicates on the next run.
+- Maybe start with a subset of your files to import to make sure everything goes well...
+- Follow the logs to monitor progress and errors: `docker-compose logs -f tubearchivist`.
+
+## Backup Database
+This will backup your metadata into a zip file. The file will get stored at *cache/backup* and will contain the necessary files to restore the Elasticsearch index formatted **nd-json** files plus a complete export of the index in a set of conventional **json** files.  
+
+BE AWARE: This will **not** backup any media files, just the metadata from the Elasticsearch.
+
+## Restore From Backup
+The restore functionality will expect the same zip file in *cache/backup* as created from the **Backup database** function. This will recreate the index from the snapshot. If there are multiple backup files in the folder, the newest one will take priority. 
+
+BE AWARE: This will **replace** your current index with the one from the backup file. This won't restore any media files.

tubearchivist/home/__init__.py

@@ -1,42 +1,5 @@
-""" handle startup """
-
-import os
-
-from home.src.config import AppConfig
-from home.src.helper import set_message
-from home.src.index_management import index_check
+""" handle celery startup """
 
 from .tasks import app as celery_app
 
-
-def sync_redis_state():
-    """make sure redis gets the config.json values"""
-    print("sync redis")
-    config_handler = AppConfig()
-    config_handler.load_new_defaults()
-    config = config_handler.config
-    sort_order = config["archive"]["sort"]
-    set_message("sort_order", sort_order, expire=False)
-    hide_watched = bool(int(config["archive"]["hide_watched"]))
-    set_message("hide_watched", hide_watched, expire=False)
-    show_subed_only = bool(int(config["archive"]["show_subed_only"]))
-    set_message("show_subed_only", show_subed_only, expire=False)
-
-
-def make_folders():
-    """make needed cache folders here so docker doesn't mess it up"""
-    folders = ["download", "channels", "videos", "import", "backup"]
-    config = AppConfig().config
-    cache_dir = config["application"]["cache_dir"]
-    for folder in folders:
-        folder_path = os.path.join(cache_dir, folder)
-        try:
-            os.makedirs(folder_path)
-        except FileExistsError:
-            continue
-
-
 __all__ = ("celery_app",)
-make_folders()
-sync_redis_state()
-index_check()

tubearchivist/home/apps.py

@@ -1,6 +1,42 @@
+"""handle custom startup functions"""
+
+import os
+
 from django.apps import AppConfig
+from home.src.config import AppConfig as ArchivistConfig
+from home.src.helper import RedisArchivist
+from home.src.index_management import index_check
+
+
+def make_folders():
+    """make needed cache folders here so docker doesn't mess it up"""
+    folders = ["download", "channels", "videos", "import", "backup"]
+    config = ArchivistConfig().config
+    cache_dir = config["application"]["cache_dir"]
+    for folder in folders:
+        folder_path = os.path.join(cache_dir, folder)
+        try:
+            os.makedirs(folder_path)
+        except FileExistsError:
+            continue
+
+
+def release_lock():
+    """make sure there are no leftover locks set in redis on container start"""
+    all_locks = ["manual_import", "downloading", "dl_queue", "dl_queue_id"]
+    for lock in all_locks:
+        response = RedisArchivist().del_message(lock)
+        if response:
+            print("deleted leftover key from redis: " + lock)
 
 
 class HomeConfig(AppConfig):
+    """call startup funcs"""
+
     default_auto_field = "django.db.models.BigAutoField"
     name = "home"
+
+    def ready(self):
+        release_lock()
+        index_check()
+        make_folders()

tubearchivist/home/config.json

@@ -1,17 +1,20 @@
 {
     "archive": {
         "sort": "published",
-        "hide_watched": false,
-        "show_subed_only": false,
         "page_size": 12
     },
+    "default_view": {
+        "home": "grid",
+        "channel": "list",
+        "downloads": "list"
+    },
     "subscriptions": {
         "auto_search": false,
         "auto_download": false,
         "channel_size": 50
     },
     "downloads": {
-        "limit_count": 5,
+        "limit_count": false,
         "limit_speed": false,
         "sleep_interval": 3,
         "format": false,

tubearchivist/home/src/config.py

@@ -8,7 +8,7 @@ Functionality:
 import json
 import os
 
-from home.src.helper import get_message, set_message
+from home.src.helper import RedisArchivist
 
 
 class AppConfig:
@@ -51,7 +51,7 @@ class AppConfig:
     @staticmethod
     def get_config_redis():
         """read config json set from redis to overwrite defaults"""
-        config = get_message("config")
+        config = RedisArchivist().get_message("config")
         if not list(config.values())[0]:
             return False
 
@@ -73,7 +73,7 @@ class AppConfig:
                 config_dict, config_value = key.split(".")
                 config[config_dict][config_value] = to_write
 
-        set_message("config", config, expire=False)
+        RedisArchivist().set_message("config", config, expire=False)
 
     def load_new_defaults(self):
         """check config.json for missing defaults"""
@@ -100,4 +100,4 @@ class AppConfig:
                     needs_update = True
 
         if needs_update:
-            set_message("config", redis_config, expire=False)
+            RedisArchivist().set_message("config", redis_config, expire=False)

tubearchivist/home/src/download.py

@@ -14,7 +14,13 @@ from time import sleep
 import requests
 import yt_dlp as youtube_dl
 from home.src.config import AppConfig
-from home.src.helper import DurationConverter, clean_string, set_message
+from home.src.helper import (
+    DurationConverter,
+    RedisArchivist,
+    RedisQueue,
+    clean_string,
+    ignore_filelist,
+)
 from home.src.index import YoutubeChannel, index_new_video
 
 
@@ -37,7 +43,7 @@ class PendingList:
                 "title": "Adding to download queue.",
                 "message": "Extracting lists",
             }
-            set_message("progress:download", mess_dict)
+            RedisArchivist().set_message("progress:download", mess_dict)
             # extract
             url = entry["url"]
             url_type = entry["type"]
@@ -92,7 +98,7 @@ class PendingList:
                 "title": "Adding to download queue.",
                 "message": "Processing IDs...",
             }
-            set_message("progress:download", mess_dict)
+            RedisArchivist().set_message("progress:download", mess_dict)
         # add last newline
         bulk_list.append("\n")
         query_str = "\n".join(bulk_list)
@@ -147,7 +153,7 @@ class PendingList:
             "size": 50,
             "query": {"match_all": {}},
             "pit": {"id": pit_id, "keep_alive": "1m"},
-            "sort": [{"timestamp": {"order": "desc"}}],
+            "sort": [{"timestamp": {"order": "asc"}}],
         }
         query_str = json.dumps(data)
         url = self.ES_URL + "/_search"
@@ -214,11 +220,13 @@ class PendingList:
 
     def get_all_downloaded(self):
         """get a list of all videos in archive"""
-        all_channel_folders = os.listdir(self.VIDEOS)
+        channel_folders = os.listdir(self.VIDEOS)
+        all_channel_folders = ignore_filelist(channel_folders)
         all_downloaded = []
         for channel_folder in all_channel_folders:
             channel_path = os.path.join(self.VIDEOS, channel_folder)
-            all_videos = os.listdir(channel_path)
+            videos = os.listdir(channel_path)
+            all_videos = ignore_filelist(videos)
             youtube_vids = [i[9:20] for i in all_videos]
             for youtube_id in youtube_vids:
                 all_downloaded.append(youtube_id)
@@ -256,7 +264,7 @@ class PendingList:
             "title": "Added to ignore list",
             "message": "",
         }
-        set_message("progress:download", mess_dict)
+        RedisArchivist().set_message("progress:download", mess_dict)
         if not request.ok:
             print(request)
 
@@ -342,7 +350,7 @@ class ChannelSubscription:
         for channel in all_channels:
             channel_id = channel["channel_id"]
             last_videos = self.get_last_youtube_videos(channel_id)
-            set_message(
+            RedisArchivist().set_message(
                 "progress:download",
                 {
                     "status": "rescan",
@@ -400,19 +408,28 @@ def playlist_extractor(playlist_id):
 
 
 class VideoDownloader:
-    """handle the video download functionality"""
+    """
+    handle the video download functionality
+    if not initiated with list, take from queue
+    """
 
-    def __init__(self, youtube_id_list):
+    def __init__(self, youtube_id_list=False):
         self.youtube_id_list = youtube_id_list
         self.config = AppConfig().config
 
-    def download_list(self):
-        """download the list of youtube_ids"""
-        limit_count = self.config["downloads"]["limit_count"]
-        if limit_count:
-            self.youtube_id_list = self.youtube_id_list[:limit_count]
+    def run_queue(self):
+        """setup download queue in redis loop until no more items"""
+        queue = RedisQueue("dl_queue")
+
+        limit_queue = self.config["downloads"]["limit_count"]
+        if limit_queue:
+            queue.trim(limit_queue - 1)
+
+        while True:
+            youtube_id = queue.get_next()
+            if not youtube_id:
+                break
 
-        for youtube_id in self.youtube_id_list:
             try:
                 self.dl_single_vid(youtube_id)
             except youtube_dl.utils.DownloadError:
@@ -421,8 +438,14 @@ class VideoDownloader:
             vid_dict = index_new_video(youtube_id)
             self.move_to_archive(vid_dict)
             self.delete_from_pending(youtube_id)
-            if self.config["downloads"]["sleep_interval"]:
-                sleep(self.config["downloads"]["sleep_interval"])
+
+    @staticmethod
+    def add_pending():
+        """add pending videos to download queue"""
+        all_pending, _ = PendingList().get_all_pending()
+        to_add = [i["youtube_id"] for i in all_pending]
+        queue = RedisQueue("dl_queue")
+        queue.add_list(to_add)
 
     @staticmethod
     def progress_hook(response):
@@ -445,7 +468,7 @@ class VideoDownloader:
             "title": title,
             "message": message,
         }
-        set_message("progress:download", mess_dict)
+        RedisArchivist().set_message("progress:download", mess_dict)
 
     def dl_single_vid(self, youtube_id):
         """download single video"""
@@ -486,7 +509,8 @@ class VideoDownloader:
 
         # check if already in cache to continue from there
         cache_dir = self.config["application"]["cache_dir"]
-        all_cached = os.listdir(cache_dir + "/download/")
+        cached = os.listdir(cache_dir + "/download/")
+        all_cached = ignore_filelist(cached)
         for file_name in all_cached:
             if youtube_id in file_name:
                 obs["outtmpl"] = cache_dir + "/download/" + file_name
@@ -511,7 +535,9 @@ class VideoDownloader:
         os.makedirs(new_folder, exist_ok=True)
         # find real filename
         cache_dir = self.config["application"]["cache_dir"]
-        for file_str in os.listdir(cache_dir + "/download"):
+        cached = os.listdir(cache_dir + "/download/")
+        all_cached = ignore_filelist(cached)
+        for file_str in all_cached:
             if youtube_id in file_str:
                 old_file = file_str
         old_file_path = os.path.join(cache_dir, "download", old_file)

tubearchivist/home/src/helper.py

@@ -13,8 +13,6 @@ import unicodedata
 import redis
 import requests
 
-REDIS_HOST = os.environ.get("REDIS_HOST")
-
 
 def get_total_hits(index, es_url, match_field):
     """get total hits from index"""
@@ -40,12 +38,28 @@ def clean_string(file_name):
     return cleaned
 
 
+def ignore_filelist(filelist):
+    """ignore temp files for os.listdir sanitizer"""
+    to_ignore = ["Icon\r\r", "Temporary Items", "Network Trash Folder"]
+    cleaned = []
+    for file_name in filelist:
+        if file_name.startswith(".") or file_name in to_ignore:
+            continue
+
+        cleaned.append(file_name)
+
+    return cleaned
+
+
 def process_url_list(url_str):
     """parse url_list to find valid youtube video or channel ids"""
     to_replace = ["watch?v=", "playlist?list="]
     url_list = re.split("\n+", url_str[0])
     youtube_ids = []
     for url in url_list:
+        if "/c/" in url or "/user/" in url:
+            raise ValueError("user name is not unique, use channel ID")
+
         url_clean = url.strip().strip("/").split("/")[-1]
         for i in to_replace:
             url_clean = url_clean.replace(i, "")
@@ -66,62 +80,133 @@ def process_url_list(url_str):
     return youtube_ids
 
 
-def set_message(key, message, expire=True):
-    """write new message to redis"""
-    redis_connection = redis.Redis(host=REDIS_HOST)
-    redis_connection.execute_command("JSON.SET", key, ".", json.dumps(message))
-    if expire:
-        redis_connection.execute_command("EXPIRE", key, 20)
+class RedisArchivist:
+    """collection of methods to interact with redis"""
+
+    REDIS_HOST = os.environ.get("REDIS_HOST")
+    REDIS_PORT = os.environ.get("REDIS_PORT")
+
+    if not REDIS_PORT:
+        REDIS_PORT = 6379
+
+    def __init__(self):
+        self.redis_connection = redis.Redis(
+            host=self.REDIS_HOST, port=self.REDIS_PORT
+        )
+
+    def set_message(self, key, message, expire=True):
+        """write new message to redis"""
+        self.redis_connection.execute_command(
+            "JSON.SET", key, ".", json.dumps(message)
+        )
+
+        if expire:
+            self.redis_connection.execute_command("EXPIRE", key, 20)
+
+    def get_message(self, key):
+        """get message dict from redis"""
+        reply = self.redis_connection.execute_command("JSON.GET", key)
+        if reply:
+            json_str = json.loads(reply)
+        else:
+            json_str = {"status": False}
+
+        return json_str
+
+    def del_message(self, key):
+        """delete key from redis"""
+        response = self.redis_connection.execute_command("DEL", key)
+        return response
+
+    def get_lock(self, lock_key):
+        """handle lock for task management"""
+        redis_lock = self.redis_connection.lock(lock_key)
+        return redis_lock
+
+    def get_dl_message(self, cache_dir):
+        """get latest download progress message if available"""
+        reply = self.redis_connection.execute_command(
+            "JSON.GET", "progress:download"
+        )
+        if reply:
+            json_str = json.loads(reply)
+        elif json_str := self.monitor_cache_dir(cache_dir):
+            json_str = self.monitor_cache_dir(cache_dir)
+        else:
+            json_str = {"status": False}
+
+        return json_str
+
+    @staticmethod
+    def monitor_cache_dir(cache_dir):
+        """
+        look at download cache dir directly as alternative progress info
+        """
+        dl_cache = os.path.join(cache_dir, "download")
+        all_cache_file = os.listdir(dl_cache)
+        cache_file = ignore_filelist(all_cache_file)
+        if cache_file:
+            filename = cache_file[0][12:].replace("_", " ").split(".")[0]
+            mess_dict = {
+                "status": "downloading",
+                "level": "info",
+                "title": "Downloading: " + filename,
+                "message": "",
+            }
+        else:
+            return False
+
+        return mess_dict
 
 
-def get_message(key):
-    """get any message from JSON key"""
-    redis_connection = redis.Redis(host=REDIS_HOST)
-    reply = redis_connection.execute_command("JSON.GET", key)
-    if reply:
-        json_str = json.loads(reply)
-    else:
-        json_str = {"status": False}
-    return json_str
+class RedisQueue:
+    """dynamically interact with the download queue in redis"""
 
+    REDIS_HOST = os.environ.get("REDIS_HOST")
+    REDIS_PORT = os.environ.get("REDIS_PORT")
 
-def get_dl_message(cache_dir):
-    """get latest message if available"""
-    redis_connection = redis.Redis(host=REDIS_HOST)
-    reply = redis_connection.execute_command("JSON.GET", "progress:download")
-    if reply:
-        json_str = json.loads(reply)
-    elif json_str := monitor_cache_dir(cache_dir):
-        json_str = monitor_cache_dir(cache_dir)
-    else:
-        json_str = {"status": False}
-    return json_str
+    if not REDIS_PORT:
+        REDIS_PORT = 6379
 
+    def __init__(self, key):
+        self.key = key
+        self.conn = redis.Redis(host=self.REDIS_HOST, port=self.REDIS_PORT)
 
-def get_lock(lock_key):
-    """handle lock for task management"""
-    redis_lock = redis.Redis(host=REDIS_HOST).lock(lock_key)
-    return redis_lock
+    def get_all(self):
+        """return all elements in list"""
+        result = self.conn.execute_command("LRANGE", self.key, 0, -1)
+        all_elements = [i.decode() for i in result]
+        return all_elements
 
+    def add_list(self, to_add):
+        """add list to queue"""
+        self.conn.execute_command("RPUSH", self.key, *to_add)
 
-def monitor_cache_dir(cache_dir):
-    """
-    look at download cache dir directly as alternative progress info
-    """
-    dl_cache = os.path.join(cache_dir, "download")
-    cache_file = os.listdir(dl_cache)
-    if cache_file:
-        filename = cache_file[0][12:].replace("_", " ").split(".")[0]
-        mess_dict = {
-            "status": "downloading",
-            "level": "info",
-            "title": "Downloading: " + filename,
-            "message": "",
-        }
-    else:
-        return False
+    def add_priority(self, to_add):
+        """add single video to front of queue"""
+        self.clear_item(to_add)
+        self.conn.execute_command("LPUSH", self.key, to_add)
 
-    return mess_dict
+    def get_next(self):
+        """return next element in the queue, False if none"""
+        result = self.conn.execute_command("LPOP", self.key)
+        if not result:
+            return False
+
+        next_element = result.decode()
+        return next_element
+
+    def clear(self):
+        """delete list from redis"""
+        self.conn.execute_command("DEL", self.key)
+
+    def clear_item(self, to_clear):
+        """remove single item from list if it's there"""
+        self.conn.execute_command("LREM", self.key, 0, to_clear)
+
+    def trim(self, size):
+        """trim the queue based on settings amount"""
+        self.conn.execute_command("LTRIM", self.key, 0, size)
 
 
 class DurationConverter:

tubearchivist/home/src/index_management.py

@@ -13,6 +13,7 @@ from datetime import datetime
 
 import requests
 from home.src.config import AppConfig
+from home.src.helper import ignore_filelist
 
 # expected mapping and settings
 INDEX_CONFIG = [
@@ -433,9 +434,11 @@ class ElasticBackup:
         """extract backup zip and return filelist"""
         cache_dir = self.config["application"]["cache_dir"]
         backup_dir = os.path.join(cache_dir, "backup")
+        backup_files = os.listdir(backup_dir)
+        all_backup_files = ignore_filelist(backup_files)
         all_available_backups = [
             i
-            for i in os.listdir(backup_dir)
+            for i in all_backup_files
             if i.startswith("ta_") and i.endswith(".zip")
         ]
         all_available_backups.sort()

tubearchivist/home/src/reindex.py

@@ -18,10 +18,10 @@ import requests
 from home.src.config import AppConfig
 from home.src.download import ChannelSubscription, PendingList, VideoDownloader
 from home.src.helper import (
+    RedisArchivist,
     clean_string,
-    get_message,
     get_total_hits,
-    set_message,
+    ignore_filelist,
 )
 from home.src.index import YoutubeChannel, YoutubeVideo, index_new_video
 
@@ -127,7 +127,7 @@ class Reindex:
                 "title": "Scraping all youtube channels",
                 "message": message,
             }
-            set_message("progress:download", mess_dict)
+            RedisArchivist().set_message("progress:download", mess_dict)
             channel_index = YoutubeChannel(channel_id)
             subscribed = channel_index.channel_dict["channel_subscribed"]
             channel_index.channel_dict = channel_index.build_channel_dict(
@@ -209,12 +209,15 @@ class FilesystemScanner:
 
     def get_all_downloaded(self):
         """get a list of all video files downloaded"""
-        all_channels = os.listdir(self.VIDEOS)
+        channels = os.listdir(self.VIDEOS)
+        all_channels = ignore_filelist(channels)
         all_channels.sort()
         all_downloaded = []
         for channel_name in all_channels:
             channel_path = os.path.join(self.VIDEOS, channel_name)
-            for video in os.listdir(channel_path):
+            videos = os.listdir(channel_path)
+            all_videos = ignore_filelist(videos)
+            for video in all_videos:
                 youtube_id = video[9:20]
                 all_downloaded.append((channel_name, video, youtube_id))
 
@@ -339,8 +342,8 @@ class ManualImport:
 
     def import_folder_parser(self):
         """detect files in import folder"""
-
-        to_import = os.listdir(self.IMPORT_DIR)
+        import_files = os.listdir(self.IMPORT_DIR)
+        to_import = ignore_filelist(import_files)
         to_import.sort()
         video_files = [i for i in to_import if not i.endswith(".json")]
 
@@ -468,7 +471,7 @@ def reindex_old_documents():
     """daily refresh of old documents"""
     # check needed last run
     now = int(datetime.now().strftime("%s"))
-    last_reindex = get_message("last_reindex")
+    last_reindex = RedisArchivist().get_message("last_reindex")
     if isinstance(last_reindex, int) and now - last_reindex < 60 * 60 * 24:
         return
     # continue if needed
@@ -476,4 +479,4 @@ def reindex_old_documents():
     reindex_handler.check_outdated()
     reindex_handler.reindex()
     # set timestamp
-    set_message("last_reindex", now, expire=False)
+    RedisArchivist().set_message("last_reindex", now, expire=False)

tubearchivist/home/src/searching.py

@@ -13,6 +13,7 @@ from datetime import datetime
 
 import requests
 from home.src.config import AppConfig
+from home.src.helper import ignore_filelist
 from PIL import Image
 
 
@@ -105,7 +106,8 @@ class SearchHandler:
     def cache_dl_vids(self, all_videos):
         """video thumbs links for cache"""
         vid_cache = os.path.join(self.CACHE_DIR, "videos")
-        all_vid_cached = os.listdir(vid_cache)
+        vid_cached = os.listdir(vid_cache)
+        all_vid_cached = ignore_filelist(vid_cached)
         # videos
         for video_dict in all_videos:
             youtube_id = video_dict["youtube_id"]
@@ -124,7 +126,8 @@ class SearchHandler:
     def cache_dl_chan(self, all_channels):
         """download channel thumbs"""
         chan_cache = os.path.join(self.CACHE_DIR, "channels")
-        all_chan_cached = os.listdir(chan_cache)
+        chan_cached = os.listdir(chan_cache)
+        all_chan_cached = ignore_filelist(chan_cached)
         for channel_dict in all_channels:
             channel_id_cache = channel_dict["channel_id"]
             channel_banner_url = channel_dict["chan_banner"]

tubearchivist/home/tasks.py

@@ -9,15 +9,19 @@ import os
 from celery import Celery, shared_task
 from home.src.config import AppConfig
 from home.src.download import ChannelSubscription, PendingList, VideoDownloader
-from home.src.helper import get_lock
+from home.src.helper import RedisArchivist, RedisQueue
 from home.src.index_management import backup_all_indexes, restore_from_backup
 from home.src.reindex import ManualImport, reindex_old_documents
 
 CONFIG = AppConfig().config
-REDIS_HOST = CONFIG["application"]["REDIS_HOST"]
+REDIS_HOST = os.environ.get("REDIS_HOST")
+REDIS_PORT = os.environ.get("REDIS_PORT")
+
+if not REDIS_PORT:
+    REDIS_PORT = 6379
 
 os.environ.setdefault("DJANGO_SETTINGS_MODULE", "home.settings")
-app = Celery("tasks", broker="redis://" + REDIS_HOST)
+app = Celery("tasks", broker=f"redis://{REDIS_HOST}:{REDIS_PORT}")
 app.config_from_object("django.conf:settings", namespace="CELERY")
 app.autodiscover_tasks()
 
@@ -37,20 +41,47 @@ def update_subscribed():
 @shared_task
 def download_pending():
     """download latest pending videos"""
-    pending_handler = PendingList()
-    pending_vids = pending_handler.get_all_pending()[0]
-    to_download = [i["youtube_id"] for i in pending_vids]
-    to_download.reverse()
-    if to_download:
-        download_handler = VideoDownloader(to_download)
-        download_handler.download_list()
+
+    have_lock = False
+    my_lock = RedisArchivist().get_lock("downloading")
+
+    try:
+        have_lock = my_lock.acquire(blocking=False)
+        if have_lock:
+            downloader = VideoDownloader()
+            downloader.add_pending()
+            downloader.run_queue()
+        else:
+            print("Did not acquire download lock.")
+
+    finally:
+        if have_lock:
+            my_lock.release()
 
 
 @shared_task
 def download_single(youtube_id):
     """start download single video now"""
-    download_handler = VideoDownloader([youtube_id])
-    download_handler.download_list()
+
+    queue = RedisQueue("dl_queue")
+    queue.add_priority(youtube_id)
+    print("Added to queue with priority: " + youtube_id)
+
+    # start queue if needed
+    have_lock = False
+    my_lock = RedisArchivist().get_lock("downloading")
+
+    try:
+        have_lock = my_lock.acquire(blocking=False)
+        if have_lock:
+            VideoDownloader().run_queue()
+        else:
+            print("Download queue already running.")
+
+    finally:
+        # release if only single run
+        if have_lock and not queue.get_next():
+            my_lock.release()
 
 
 @shared_task
@@ -73,7 +104,7 @@ def run_manual_import():
 
     print("starting media file import")
     have_lock = False
-    my_lock = get_lock("manual_import")
+    my_lock = RedisArchivist().get_lock("manual_import")
 
     try:
         have_lock = my_lock.acquire(blocking=False)
@@ -101,3 +132,25 @@ def run_restore_backup():
     """called from settings page, dump backup to zip file"""
     restore_from_backup()
     print("index restore finished")
+
+
+def kill_dl(task_id):
+    """kill download worker task by ID"""
+    app.control.revoke(task_id, terminate=True)
+    _ = RedisArchivist().del_message("dl_queue_id")
+    RedisQueue("dl_queue").clear()
+
+    # clear cache
+    cache_dir = os.path.join(CONFIG["application"]["cache_dir"], "download")
+    for cached in os.listdir(cache_dir):
+        to_delete = os.path.join(cache_dir, cached)
+        os.remove(to_delete)
+
+    # notify
+    mess_dict = {
+        "status": "downloading",
+        "level": "error",
+        "title": "Brutally killing download queue",
+        "message": "",
+    }
+    RedisArchivist().set_message("progress:download", mess_dict)

tubearchivist/home/templates/home/about.html

@@ -5,35 +5,13 @@
     <h1>About The Tube Archivist</h1>
 </div>
 <div class="about-section">
-    <h2>Getting started</h2>
-    <ol>
-        <li>Before adding some videos to the download queue, take a look at the download format settings on your <a href="{% url 'settings' %}#format">settings</a> page and set your desired download quality.</li>
-        <li>While at the settings page also set the value for the Subscriptions <a href="{% url 'settings' %}#subscriptions">page size</a>. This will define the max amount of videos that will get added once you run <i>Rescan Subscriptions</i>. By default, yt-dlp uses a 50 video page size. Any value higher than that will significantly slow down the scanning process.</li>
-        <li>Subscribe to some of your favorite YouTube channels in the <a href="{% url 'channel' %}">channel</a> page. To subscribe to a channel you have a few options:
-            <ul>
-                <li>Enter the YouTube channel ID, a 25 character alphanumeric string. For example <span class="settings-current">UCBa659QWEk1AI4Tg--mrJ2A</span></li>
-                <li>Enter the URL to the channel page on YouTube. For example <span class="settings-current">https://www.youtube.com/channel/UCBa659QWEk1AI4Tg--mrJ2A</span></li>
-                <li>Enter the video URL for any video and let <i>Tube Archivist</i> extract the channel ID for you. For example <span class="settings-current">https://www.youtube.com/watch?v=2tdiKTSdE9Y</span></li>
-            </ul>
-        </li>
-        <li>After that head over to the <a href="{% url 'downloads' %}">Downloads</a> Page. Once you click on the <i>Rescan Subscriptions</i> button, <b>Tube Archivist</b> will go through all your subscribed channels and look for new videos. Initially these are the latest videos depending on the page size as described above.</li>
-        <li>Click on <span class="settings-current">ignore</span> for any video you don't want to download or on <span class="settings-current">Download now</span> for any video you want to start downloading now independently from the queue.</li>
-        <li>Click on the <span class="settings-current">Download queue</span> button to start downloading. </b>Tube Archivist</b> will then start the queue from the bottom and download as many videos as defined in <a href="{% url 'settings' %}#downloads">download limit</a>. There is currently no good way to cancel the download process.</li>
-    </ol>
+    <h2>Useful Links</h2>
+    <p>This project is in active and constant development, take a look at the <a href="https://github.com/bbilly1/tubearchivist#roadmap" target="_blank">roadmap</a> for a overview.</p>
+    <p>For any questions on what a button or a function does, You can find the up-to-date user documentation on <a href="https://github.com/bbilly1/tubearchivist/wiki" target="_blank">Github</a>.</p>
+    <p>All contributions are welcome: Open an <a href="https://github.com/bbilly1/tubearchivist/issues" target="_blank">issue</a> for any bugs and errors, start a <a href="https://github.com/bbilly1/tubearchivist/discussions" target="_blank">discussion</a> for anything that will require a more indepth look. The <a href="https://github.com/bbilly1/tubearchivist/blob/master/CONTRIBUTING.md" target="_blank">contributing</a> page is a good place to get started.</p>
 </div>
 <div class="about-section">
-    <h2>Additional</h2>
-    <ul>
-        <li>You can manually add videos to the queue on the <a href="{% url 'downloads' %}">download page</a> by clicking on <i>Add to download queue</i> and then entering a list of videos. Add one link per line. You have a few options:
-            <ul>
-                <li>Add a link to a YouTube video. For example <span class="settings-current">https://www.youtube.com/watch?v=2tdiKTSdE9Y</span>.</li>
-                <li>Add a YouTube video ID. For example <span class="settings-current">2tdiKTSdE9Y</span>.</li>
-                <li>Add a link to a YouTube video by providing the shortened URL, for example <span class="settings-current">https://youtu.be/2tdiKTSdE9Y</span>.</li>
-                <li>Add a Channel ID or Channel URL to add every available video to the download queue. This will ignore the page size as described above. You can still ignore selected videos before starting the download.</li>
-                <li>Add a playlist ID or URL to add every available video in the list to the download queue, for example <span class="settings-current">https://www.youtube.com/playlist?list=PL96C35uN7xGLLeET0dOWaKHkAlPsrkcha</span> or <span class="settings-current">PL96C35uN7xGLLeET0dOWaKHkAlPsrkcha</span>. Note that when you add a link to a video in a playlist, <b>Tube Archivist</b> assumes you want to download only the specific video and not the whole playlist, for example <span class="settings-current">https://www.youtube.com/watch?v=CINVwWHlzTY&list=PL96C35uN7xGLLeET0dOWaKHkAlPsrkcha</span> will only add one video <span class="settings-current">CINVwWHlzTY</span> to the queue.</li>
-            </ul>
-        </li>
-        <li>As you watch videos, mark them as watched by clicking on the unseen icon <span class="about-icon"><img src="{% static 'img/icon-unseen.svg' %}" alt="unseen-icon" class="unseen-icon"></span>. Once marked as watched, the icon will change to <span class="about-icon"><img src="{% static 'img/icon-seen.svg' %}" alt="seen-icon" class="seen-icon"></span>.</li>
-    </ul>
+    <h2>Donate</h2>
+    <p>Here are <a href="https://github.com/bbilly1/tubearchivist#donate" target="_blank">some links</a>, if you want to buy the developer a coffee. Thank you for your support!</p>
 </div>
 {% endblock content %}

tubearchivist/home/templates/home/base.html

@@ -96,7 +96,7 @@
     </div>
     <div class="footer">
         <div class="boxed-content">
-            <span>© 2021 The Tube Archivist v0.0.3 | <a href="https://github.com/bbilly1/tubearchivist" target="_blank">Github</a> | <a href="https://hub.docker.com/r/bbilly1/tubearchivist" target="_blank">Docker Hub</a></span>
+            <span>© 2021 The Tube Archivist v0.0.5 | <a href="https://github.com/bbilly1/tubearchivist" target="_blank">Github</a> | <a href="https://hub.docker.com/r/bbilly1/tubearchivist" target="_blank">Docker Hub</a></span>
         </div>
     </div>
 </body>

tubearchivist/home/templates/home/channel.html

@@ -30,27 +30,38 @@
         </form>
     </div>
 </div>
-<div class="padding-box">
-    <h2>Total matching channels: {{ max_hits }}</h2>
-    <span>Change show / hide subscribed only </span><span class="settings-current">{{ show_subed_only }}</span>
-    <select name="watched" id="watched" onchange="showSubscribedOnly(this.value)">
-        <option value="" disabled selected> -- change -- </option>
-        <option value="0">show all channels</option>
-        <option value="1">show subscribed channels only</option>
-    </select>
+<div class="view-controls">
+    <div class="toggle">
+        <span>Show only subscribed channels:</span>
+        <div class="toggleBox">
+            <input 
+                id="show_subed_only" onclick="toggleCheckbox(this)" type="checkbox"
+                {% if show_subed_only %}
+                    checked
+                {% endif %}
+                >
+            <label for="" class="onbtn">On</label>
+            <label for="" class="ofbtn">Off</label>
+        </div>
+    </div>
+    <div class="view-icons">
+        <img src="{% static 'img/icon-gridview.svg' %}" onclick="changeView(this)" data-origin="channel" data-value="grid" alt="grid view">
+        <img src="{% static 'img/icon-listview.svg' %}" onclick="changeView(this)" data-origin="channel" data-value="list" alt="list view">
+    </div>
 </div>
-<div>
+<h2>Total matching channels: {{ max_hits }}</h2>
+<div class="channel-list {{ view_style }}">
     {% if channels %}
         {% for channel in channels %}
-        <div class="channel-item">
+        <div class="channel-item {{ view_style }}">
             {% if channel.source.channel_banner_url %}
-                <div class="channel-banner">
+                <div class="channel-banner {{ view_style }}">
                     <a href="{% url 'channel_id' channel.source.channel_id %}">
                         <img src="/cache/channels/{{ channel.source.channel_id }}_banner.jpg" alt="{{ channel.source.channel_id }}-banner">
                     </a>
                 </div>
             {% endif %}
-            <div class="info-box info-box-2">
+            <div class="info-box info-box-2 {{ view_style }}">
                 <div class="info-box-item">
                     <div class="round-img">
                         <a href="{% url 'channel_id' channel.source.channel_id %}">

tubearchivist/home/templates/home/channel_id.html

@@ -10,7 +10,7 @@
         <a href="/channel/{{ channel_info.channel_id }}/"><img src="/cache/channels/{{ channel_info.channel_id }}_banner.jpg" alt="channel_banner"></a>
     {% endif %}
 </div>
-<div class="info-box info-box-3 padding-box">
+<div class="info-box info-box-3">
     <div class="info-box-item">
         <div class="round-img">
             <a href="{% url 'channel_id' channel_info.channel_id %}">
@@ -61,14 +61,18 @@
         </div>
     </div>
 {% endif %}
+<div class="view-icons">
+    <img src="{% static 'img/icon-gridview.svg' %}" onclick="changeView(this)" data-origin="home" data-value="grid" alt="grid view">
+    <img src="{% static 'img/icon-listview.svg' %}" onclick="changeView(this)" data-origin="home" data-value="list" alt="list view">
+</div>
 <div id="player" class="video-player"></div>
 <h2>Videos</h2>
-<div class="video-list">
+<div class="video-list {{ view_style }}">
     {% if videos %}
         {% for video in videos %}
-            <div class="video-item">
+            <div class="video-item {{ view_style }}">
                 <a href="#player" data-src="/media/{{ video.source.media_url }}" data-thumb="/cache/videos/{{ video.source.youtube_id }}.jpg" data-title="{{ video.source.title }}" data-channel="{{ video.source.channel.channel_name }}" data-id="{{ video.source.youtube_id }}" onclick="createPlayer(this)">
-                    <div class="video-thumb-wrap">
+                    <div class="video-thumb-wrap {{ view_style }}">
                         <div class="video-thumb">
                             <img src="/cache/videos/{{ video.source.youtube_id }}.jpg" alt="video-thumb">
                         </div>
@@ -77,7 +81,7 @@
                         </div>
                     </div>
                 </a>
-                <div class="video-desc">
+                <div class="video-desc {{ view_style }}">
                     <div class="video-desc-player" id="video-info-{{ video.source.youtube_id }}">
                         {% if video.source.player.watched %}
                             <img src="{% static 'img/icon-seen.svg' %}" alt="seen-icon" id="{{ video.source.youtube_id }}" class="seen-icon">

tubearchivist/home/templates/home/downloads.html

@@ -5,14 +5,15 @@
     <h1>Downloads</h1>
 </div>
 <div id="downloadMessage"></div>
-<div class="info-box info-box-3 padding-box">
+<div id="downloadControl"></div>
+<div class="info-box info-box-3">
     <div class="icon-text">
         <img id="rescan-icon" onclick="rescanPending()" src="{% static 'img/icon-rescan.svg' %}" alt="rescan-icon">
         <p>Rescan subscriptions</p>
     </div>
     <div class="icon-text">
         <img id="download-icon" onclick="dlPending()" src="{% static 'img/icon-download.svg' %}" alt="download-icon">
-        <p>Download queue</p>
+        <p>Start download</p>
     </div>
     <div class="icon-text">
         <img id="add-icon" onclick="showForm()" src="{% static 'img/icon-add.svg' %}" alt="add-icon">
@@ -26,30 +27,60 @@
         </div>
     </div>
 </div>
-<h2>Download queue</h2>
-<div>
-    {% if pending %}
-        <h3>Total pending downloads: {{ max_hits }}</h3>
-        {% for video in pending %}
-            <div class="dl-item" id="dl-{{ video.youtube_id }}">
-                <div class="dl-thumb">
+<div class="view-controls">
+    <div class="toggle">
+        <span>Show only ignored videos:</span>
+        <div class="toggleBox">
+            <input 
+                id="show_ignored_only" onclick="toggleCheckbox(this)" type="checkbox"
+                {% if show_ignored_only %}
+                    checked
+                {% endif %}
+                >
+            <label for="" class="onbtn">On</label>
+            <label for="" class="ofbtn">Off</label>
+        </div>
+    </div>
+    <div class="view-icons">
+        <img src="{% static 'img/icon-gridview.svg' %}" onclick="changeView(this)" data-origin="downloads" data-value="grid" alt="grid view">
+        <img src="{% static 'img/icon-listview.svg' %}" onclick="changeView(this)" data-origin="downloads" data-value="list" alt="list view">
+    </div>
+</div>
+{% if show_ignored_only %}
+    <h2>Ignored from download</h2>
+{% else %}
+    <h2>Download queue</h2>
+{% endif %}
+<h3>Total videos: {{ max_hits }}</h3>
+<div class="dl-list {{ view_style }}">
+    {% if all_video_hits %}
+        {% for video in all_video_hits %}
+            <div class="dl-item {{ view_style }}" id="dl-{{ video.youtube_id }}">
+                <div class="dl-thumb {{ view_style }}">
                     <img src="{{ video.vid_thumb_url }}" alt="video_thumb">
                 </div>
-                <div class="dl-desc">
-                    <h3>{{ video.title }}</h3>
+                <div class="dl-desc {{ view_style }}">
+                    {% if show_ignored_only %}
+                        <h3>Ignore: {{ video.title }}</h3>
+                    {% else %}
+                        <h3>Download: {{ video.title }}</h3>
+                    {% endif %}
                     {% if video.channel_indexed %}
                         <a href="{% url 'channel_id' video.channel_id %}">{{ video.channel_name }}</a>
                     {% else %}
                         <span>{{ video.channel_name }}</span>
                     {% endif %}
                     <p>Published: {{ video.published }} | Duration: {{ video.duration }} | {{ video.youtube_id }}</p>
-                    <button data-id="{{ video.youtube_id }}" onclick="toIgnore(this)">Ignore</button>
-                    <button id="{{ video.youtube_id }}" data-id="{{ video.youtube_id }}" onclick="downloadNow(this)">Download now</button>
+                    {% if show_ignored_only %}
+                        <button data-id="{{ video.youtube_id }}" onclick="forgetIgnore(this)">Forget</button>
+                        <button data-id="{{ video.youtube_id }}" onclick="addSingle(this)">Add to queue</button>
+                    {% else %}
+                        <button data-id="{{ video.youtube_id }}" onclick="toIgnore(this)">Ignore</button>
+                        <button id="{{ video.youtube_id }}" data-id="{{ video.youtube_id }}" onclick="downloadNow(this)">Download now</button>
+                    {% endif %}
                 </div>
             </div>
         {% endfor %}
-    {% else %}
-        <h3>No pending downloads</h3>
     {% endif %}
 </div>
 <script type="text/javascript" src="{% static 'progress.js' %}"></script>

tubearchivist/home/templates/home/home.html

@@ -13,13 +13,6 @@
                 <option value="downloaded">date downloaded</option>
             </select>
         </p>
-        <p>Hide watched videos <span class="settings-current">{{ hide_watched }}</span>
-            <select name="watched" id="watched" onchange="hideWatched(this.value)">
-                <option value="" disabled selected> -- change hide watched -- </option>
-                <option value="0">show watched videos</option>
-                <option value="1">hide watched videos</option>
-            </select>
-        </p>
     </div>
     <div class="search-form icon-text">
         <div class="search-icon">
@@ -31,13 +24,32 @@
         </form>
     </div>
 </div>
+<div class="view-controls">
+    <div class="toggle">
+        <span>Hide watched videos:</span>
+        <div class="toggleBox">
+            <input 
+                id="hide_watched" onclick="toggleCheckbox(this)" type="checkbox"
+                {% if hide_watched %}
+                    checked
+                {% endif %}
+                >
+            <label for="" class="onbtn">On</label>
+            <label for="" class="ofbtn">Off</label>
+        </div>
+    </div>
+    <div class="view-icons">
+        <img src="{% static 'img/icon-gridview.svg' %}" onclick="changeView(this)" data-origin="home" data-value="grid" alt="grid view">
+        <img src="{% static 'img/icon-listview.svg' %}" onclick="changeView(this)" data-origin="home" data-value="list" alt="list view">
+    </div>
+</div>
 <div id="player" class="video-player"></div>
-<div class="video-list">
+<div class="video-list {{ view_style }}">
     {% if videos %}
         {% for video in videos %}
-            <div class="video-item">
+            <div class="video-item {{ view_style }}">
                 <a href="#player" data-src="/media/{{ video.source.media_url }}" data-thumb="/cache/videos/{{ video.source.youtube_id }}.jpg" data-title="{{ video.source.title }}" data-channel="{{ video.source.channel.channel_name }}" data-id="{{ video.source.youtube_id }}" onclick="createPlayer(this)">
-                    <div class="video-thumb-wrap">
+                    <div class="video-thumb-wrap {{ view_style }}">
                         <div class="video-thumb">
                             <img src="/cache/videos/{{ video.source.youtube_id }}.jpg" alt="video-thumb">
                         </div>
@@ -46,7 +58,7 @@
                         </div>
                     </div>
                 </a>
-                <div class="video-desc">
+                <div class="video-desc {{ view_style }}">
                     <div class="video-desc-player" id="video-info-{{ video.source.youtube_id }}">
                         {% if video.source.player.watched %}
                             <img src="{% static 'img/icon-seen.svg' %}" alt="seen-icon" id="{{ video.source.youtube_id }}" class="seen-icon">

tubearchivist/home/templates/home/settings.html

@@ -19,33 +19,6 @@
     </div>
     <div class="settings-group">
         <h2>Archive View</h2>
-        <div class="settings-item">
-            <p>Current default Sort: <span class="settings-current">{{ config.archive.sort }}</span></p>
-            <i>Change how the home view and channels view sorts by defaults.</i><br>
-            <select name="archive.sort" id="archive.sort"">
-                <option value="" disabled selected> -- change sort order -- </option>
-                <option value="published">date published</option>
-                <option value="downloaded">date downloaded</option>
-            </select>
-        </div>
-        <div class="settings-item">
-            <p>Current default hide watched: <span class="settings-current">{{ config.archive.hide_watched }}</span></p>
-            <i>Show or hide watched videos by default.</i><br>
-            <select name="archive.hide_watched" id="archive.hide_watched"">
-                <option value="" disabled selected> -- change visibility -- </option>
-                <option value="0">show watched</option>
-                <option value="1">hide watched</option>
-            </select>
-        </div>
-        <div class="settings-item">
-            <p>Current default show subscribed channels only: <span class="settings-current">{{ config.archive.show_subed_only }}</span></p>
-            <i>Show subscribed channels only by default.</i><br>
-            <select name="archive.show_subed_only" id="archive.show_subed_only"">
-                <option value="" disabled selected> -- change visibility -- </option>
-                <option value="0">show subscribed only</option>
-                <option value="1">hide not subscribed</option>
-            </select>
-        </div>
         <div class="settings-item">
             <p>Current page size: <span class="settings-current">{{ config.archive.page_size }}</span></p>
             <i>Result of videos showing in archive page</i><br>

tubearchivist/home/views.py

@@ -14,18 +14,14 @@ from django.utils.http import urlencode
 from django.views import View
 from home.src.config import AppConfig
 from home.src.download import ChannelSubscription, PendingList
-from home.src.helper import (
-    get_dl_message,
-    get_message,
-    process_url_list,
-    set_message,
-)
+from home.src.helper import RedisArchivist, RedisQueue, process_url_list
 from home.src.index import WatchState
 from home.src.searching import Pagination, SearchForm, SearchHandler
 from home.tasks import (
     download_pending,
     download_single,
     extrac_dl,
+    kill_dl,
     run_backup,
     run_manual_import,
     run_restore_backup,
@@ -43,7 +39,7 @@ class HomeView(View):
 
     def get(self, request):
         """return home search results"""
-        colors, sort_order, hide_watched = self.read_config()
+        colors, view_style, sort_order, hide_watched = self.read_config()
         # handle search
         search_get = request.GET.get("search", False)
         if search_get:
@@ -70,6 +66,7 @@ class HomeView(View):
             "sortorder": sort_order,
             "hide_watched": hide_watched,
             "colors": colors,
+            "view_style": view_style,
         }
         return render(request, "home/home.html", context)
 
@@ -112,9 +109,12 @@ class HomeView(View):
         """read needed values from redis"""
         config_handler = AppConfig().config
         colors = config_handler["application"]["colors"]
-        sort_order = get_message("sort_order")
-        hide_watched = get_message("hide_watched")
-        return colors, sort_order, hide_watched
+        view_style = config_handler["default_view"]["home"]
+        sort_order = RedisArchivist().get_message("sort_order")
+        if not sort_order:
+            sort_order = "published"
+        hide_watched = RedisArchivist().get_message("hide_watched")
+        return colors, view_style, sort_order, hide_watched
 
     @staticmethod
     def post(request):
@@ -148,44 +148,54 @@ class DownloadView(View):
         """handle get requests"""
         config = AppConfig().config
         colors = config["application"]["colors"]
+        view_style = config["default_view"]["downloads"]
+        ignored = RedisArchivist().get_message("show_ignored_only")
+        show_ignored_only = ignored["status"]
 
         page_get = int(request.GET.get("page", 0))
         pagination_handler = Pagination(page_get)
 
         url = config["application"]["es_url"] + "/ta_download/_search"
-        data = self.build_data(pagination_handler)
+        data = self.build_data(pagination_handler, show_ignored_only)
         search = SearchHandler(url, data, cache=False)
 
         videos_hits = search.get_data()
         max_hits = search.max_hits
 
         if videos_hits:
-            all_pending = [i["source"] for i in videos_hits]
+            all_video_hits = [i["source"] for i in videos_hits]
             pagination_handler.validate(max_hits)
             pagination = pagination_handler.pagination
         else:
-            all_pending = False
+            all_video_hits = False
             pagination = False
 
         context = {
-            "pending": all_pending,
+            "all_video_hits": all_video_hits,
             "max_hits": max_hits,
             "pagination": pagination,
             "title": "Downloads",
             "colors": colors,
+            "show_ignored_only": show_ignored_only,
+            "view_style": view_style,
         }
         return render(request, "home/downloads.html", context)
 
     @staticmethod
-    def build_data(pagination_handler):
+    def build_data(pagination_handler, show_ignored_only):
         """build data dict for search"""
         page_size = pagination_handler.pagination["page_size"]
         page_from = pagination_handler.pagination["page_from"]
+        if show_ignored_only:
+            filter_view = "ignore"
+        else:
+            filter_view = "pending"
+
         data = {
             "size": page_size,
             "from": page_from,
-            "query": {"term": {"status": {"value": "pending"}}},
-            "sort": [{"timestamp": {"order": "desc"}}],
+            "query": {"term": {"status": {"value": filter_view}}},
+            "sort": [{"timestamp": {"order": "asc"}}],
         }
         return data
 
@@ -195,18 +205,18 @@ class DownloadView(View):
         download_post = dict(request.POST)
         if "vid-url" in download_post.keys():
             url_str = download_post["vid-url"]
-            print("adding to queue")
-            youtube_ids = process_url_list(url_str)
-            if not youtube_ids:
+            try:
+                youtube_ids = process_url_list(url_str)
+            except ValueError:
                 # failed to process
-                print(url_str)
+                print(f"failed to parse: {url_str}")
                 mess_dict = {
                     "status": "downloading",
                     "level": "error",
                     "title": "Failed to extract links.",
-                    "message": "",
+                    "message": "Not a video, channel or playlist ID or URL",
                 }
-                set_message("progress:download", mess_dict)
+                RedisArchivist().set_message("progress:download", mess_dict)
                 return redirect("downloads")
 
             print(youtube_ids)
@@ -223,9 +233,9 @@ class ChannelIdView(View):
 
     def get(self, request, channel_id_detail):
         """get method"""
-        es_url, colors = self.read_config()
+        es_url, colors, view_style = self.read_config()
         context = self.get_channel_videos(request, channel_id_detail, es_url)
-        context.update({"colors": colors})
+        context.update({"colors": colors, "view_style": view_style})
         return render(request, "home/channel_id.html", context)
 
     @staticmethod
@@ -234,7 +244,8 @@ class ChannelIdView(View):
         config = AppConfig().config
         es_url = config["application"]["es_url"]
         colors = config["application"]["colors"]
-        return es_url, colors
+        view_style = config["default_view"]["home"]
+        return es_url, colors, view_style
 
     def get_channel_videos(self, request, channel_id_detail, es_url):
         """get channel from video index"""
@@ -307,7 +318,7 @@ class ChannelView(View):
 
     def get(self, request):
         """handle http get requests"""
-        es_url, colors = self.read_config()
+        es_url, colors, view_style = self.read_config()
         page_get = int(request.GET.get("page", 0))
         pagination_handler = Pagination(page_get)
         page_size = pagination_handler.pagination["page_size"]
@@ -320,7 +331,7 @@ class ChannelView(View):
             "query": {"match_all": {}},
             "sort": [{"channel_name.keyword": {"order": "asc"}}],
         }
-        show_subed_only = get_message("show_subed_only")
+        show_subed_only = RedisArchivist().get_message("show_subed_only")
         if show_subed_only:
             data["query"] = {"term": {"channel_subscribed": {"value": True}}}
         search = SearchHandler(url, data)
@@ -334,6 +345,7 @@ class ChannelView(View):
             "show_subed_only": show_subed_only,
             "title": "Channels",
             "colors": colors,
+            "view_style": view_style,
         }
         return render(request, "home/channel.html", context)
 
@@ -343,7 +355,8 @@ class ChannelView(View):
         config = AppConfig().config
         es_url = config["application"]["es_url"]
         colors = config["application"]["colors"]
-        return es_url, colors
+        view_style = config["default_view"]["channel"]
+        return es_url, colors, view_style
 
     def post(self, request):
         """handle http post requests"""
@@ -440,7 +453,7 @@ def progress(request):
     """endpoint for download progress ajax calls"""
     config = AppConfig().config
     cache_dir = config["application"]["cache_dir"]
-    json_data = get_dl_message(cache_dir)
+    json_data = RedisArchivist().get_dl_message(cache_dir)
     return JsonResponse(json_data)
 
 
@@ -476,14 +489,19 @@ class PostData:
         """map dict key and return function to execute"""
         exec_map = {
             "watched": self.watched,
+            "change_view": self.change_view,
             "rescan_pending": self.rescan_pending,
             "ignore": self.ignore,
             "dl_pending": self.dl_pending,
+            "queue": self.queue_handler,
             "unsubscribe": self.unsubscribe,
             "sort_order": self.sort_order,
             "hide_watched": self.hide_watched,
             "show_subed_only": self.show_subed_only,
             "dlnow": self.dlnow,
+            "show_ignored_only": self.show_ignored_only,
+            "forgetIgnore": self.forget_ignore,
+            "addSingle": self.add_single,
             "manual-import": self.manual_import,
             "db-backup": self.db_backup,
             "db-restore": self.db_restore,
@@ -497,6 +515,14 @@ class PostData:
         WatchState(self.exec_val).mark_as_watched()
         return {"success": True}
 
+    def change_view(self):
+        """process view changes in home, channel, and downloads"""
+        origin, new_view = self.exec_val.split(":")
+        print(f"change view on page {origin} to {new_view}")
+        update_dict = {f"default_view.{origin}": [new_view]}
+        AppConfig().update_config(update_dict)
+        return {"success": True}
+
     @staticmethod
     def rescan_pending():
         """look for new items in subscribed channels"""
@@ -506,17 +532,35 @@ class PostData:
 
     def ignore(self):
         """ignore from download queue"""
-        print("ignore video")
+        id_to_ignore = self.exec_val
+        print("ignore video " + id_to_ignore)
         handler = PendingList()
-        ignore_list = self.exec_val
-        handler.ignore_from_pending([ignore_list])
+        handler.ignore_from_pending([id_to_ignore])
+        # also clear from redis queue
+        RedisQueue("dl_queue").clear_item(id_to_ignore)
         return {"success": True}
 
     @staticmethod
     def dl_pending():
         """start the download queue"""
         print("download pending")
-        download_pending.delay()
+        running = download_pending.delay()
+        task_id = running.id
+        print("set task id: " + task_id)
+        RedisArchivist().set_message("dl_queue_id", task_id, expire=False)
+        return {"success": True}
+
+    def queue_handler(self):
+        """queue controls from frontend"""
+        to_execute = self.exec_val
+        if to_execute == "stop":
+            print("stopping download queue")
+            RedisQueue("dl_queue").clear()
+        elif to_execute == "kill":
+            task_id = RedisArchivist().get_message("dl_queue_id")
+            print("brutally killing " + task_id)
+            kill_dl(task_id)
+
         return {"success": True}
 
     def unsubscribe(self):
@@ -531,28 +575,60 @@ class PostData:
     def sort_order(self):
         """change the sort between published to downloaded"""
         sort_order = self.exec_val
-        set_message("sort_order", sort_order, expire=False)
+        RedisArchivist().set_message("sort_order", sort_order, expire=False)
         return {"success": True}
 
     def hide_watched(self):
         """toggle if to show watched vids or not"""
         hide_watched = bool(int(self.exec_val))
         print(f"hide watched: {hide_watched}")
-        set_message("hide_watched", hide_watched, expire=False)
+        RedisArchivist().set_message(
+            "hide_watched", hide_watched, expire=False
+        )
         return {"success": True}
 
     def show_subed_only(self):
         """show or hide subscribed channels only on channels page"""
         show_subed_only = bool(int(self.exec_val))
         print(f"show subed only: {show_subed_only}")
-        set_message("show_subed_only", show_subed_only, expire=False)
+        RedisArchivist().set_message(
+            "show_subed_only", show_subed_only, expire=False
+        )
         return {"success": True}
 
     def dlnow(self):
         """start downloading single vid now"""
         youtube_id = self.exec_val
         print("downloading: " + youtube_id)
-        download_single.delay(youtube_id=youtube_id)
+        running = download_single.delay(youtube_id=youtube_id)
+        task_id = running.id
+        print("set task id: " + task_id)
+        RedisArchivist().set_message("dl_queue_id", task_id, expire=False)
+        return {"success": True}
+
+    def show_ignored_only(self):
+        """switch view on /downloads/ to show ignored only"""
+        show_value = self.exec_val
+        print(f"Filter download view ignored only: {show_value}")
+        RedisArchivist().set_message(
+            "show_ignored_only", {"status": show_value}, expire=False
+        )
+        return {"success": True}
+
+    def forget_ignore(self):
+        """delete from ta_download index"""
+        youtube_id = self.exec_val
+        print("forgetting from download index: " + youtube_id)
+        PendingList().delete_from_pending(youtube_id)
+        return {"success": True}
+
+    def add_single(self):
+        """add single youtube_id to download queue"""
+        youtube_id = self.exec_val
+        print("add vid to dl queue: " + youtube_id)
+        PendingList().delete_from_pending(youtube_id)
+        youtube_ids = process_url_list([youtube_id])
+        extrac_dl.delay(youtube_ids)
         return {"success": True}
 
     @staticmethod

tubearchivist/requirements.txt

@@ -6,4 +6,4 @@ redis==3.5.3
 requests==2.26.0
 uWSGI==2.0.19.1
 whitenoise==5.3.0
-yt_dlp==2021.9.2
+yt_dlp==2021.9.25

tubearchivist/static/css/dark.css

@@ -7,4 +7,5 @@
     --accent-font-dark: #259485;
     --accent-font-light: #97d4c8;
     --img-filter: invert(50%) sepia(9%) saturate(2940%) hue-rotate(122deg) brightness(94%) contrast(90%);
+    --img-filter-error: invert(16%) sepia(60%) saturate(3717%) hue-rotate(349deg) brightness(86%) contrast(120%);
 }

tubearchivist/static/css/light.css

@@ -7,4 +7,5 @@
     --accent-font-dark: #259485;
     --accent-font-light: #35b399;
     --img-filter: invert(50%) sepia(9%) saturate(2940%) hue-rotate(122deg) brightness(94%) contrast(90%);
+    --img-filter-error: invert(83%) sepia(35%) saturate(1238%) hue-rotate(297deg) brightness(103%) contrast(97%);
 }

tubearchivist/static/css/style.css

@@ -138,6 +138,73 @@ button:hover {
     text-decoration: underline;
 }
 
+/* toggle on-off */
+.toggle {
+    display: flex;
+    align-items: center;
+}
+
+.toggleBox > input[type="checkbox"] {
+    position: relative;
+    width: 70px;
+    height: 30px;
+    background-color: var(--accent-font-dark);
+    border-color: var(--accent-font-dark);
+    appearance: none;
+    border-radius: 15px;
+    transition: 0.4s;
+    box-shadow: inset 0 0 5px rgba(0, 0, 0, 0.2);
+    cursor: pointer;
+}
+
+.toggleBox > input:checked[type="checkbox"] {
+    background-color: var(--accent-font-light);
+    border-color: var(--accent-font-light);
+}
+
+.toggleBox > input[type="checkbox"]::before {
+    z-index: 2;
+    position: absolute;
+    content: "";
+    left: 0;
+    top: 0;
+    width: 30px;
+    height: 30px;
+    background-color: white;
+    border-radius: 50%;
+    transform: scale(1.1);
+    transition: 0.4s;
+}
+
+.toggleBox > input:checked[type="checkbox"]::before {
+    left: 40px;
+}
+
+.toggleBox {
+    margin-left: 10px;
+    position: relative;
+    display: inline;
+}
+
+.toggleBox > label {
+    position: absolute;
+    color: var(--main-font);
+    pointer-events: none;
+}
+
+.toggleBox > .onbtn {
+    bottom: 15px;
+    left: 15px;
+    font-family: Sen-Regular, sans-serif;
+}
+
+.toggleBox > .ofbtn {
+    bottom: 15px;
+    right: 15px;
+    font-family: Sen-Regular, sans-serif;
+    color: var(--main-font);
+}
+
 /* navigation */
 .top-nav {
     display: block;
@@ -176,7 +243,7 @@ button:hover {
 
 /* top of page */
 .title-bar {
-    padding: 25px 0;
+    padding: 20px 0;
 }
 
 .sort {
@@ -211,6 +278,26 @@ button:hover {
     filter: var(--img-filter);
 }
 
+.view-controls {
+    display: grid;
+    grid-template-columns: 1fr 1fr;
+    border-bottom: 2px solid;
+    border-color: var(--accent-font-dark);
+    margin: 15px 0;
+}
+
+.view-icons {
+    display: flex;
+    justify-content: end;
+}
+
+.view-icons img {
+    width: 30px;
+    margin: 5px 10px;
+    cursor: pointer;
+    filter: var(--img-filter);
+}
+
 #search-box {
     display: none;
     flex: auto;
@@ -261,16 +348,27 @@ button:hover {
 
 
 /* video list */
-.video-list {
+.video-list.grid {
     display: grid;
     grid-template-columns: 1fr 1fr 1fr;
     grid-gap: 1rem;
 }
 
+.video-list.list {
+    display: grid;
+    grid-template-columns: unset;
+    grid-gap: 1rem;
+}
+
 .video-item {
     overflow: hidden;
 }
 
+.video-item.list {
+    display: grid;
+    grid-template-columns: 25% auto;
+}
+
 .video-thumb img {
     width: 100%;
 }
@@ -301,12 +399,25 @@ button:hover {
     padding: 8px;
 }
 
-.video-desc {
+.video-desc.grid {
     padding: 10px;
     height: 100%;
     background-color: var(--highlight-bg);
 }
 
+.video-desc.list {
+    padding: 10px;
+    height: unset;
+    background-color: var(--highlight-bg);
+    display: flex;
+    flex-wrap: wrap-reverse;
+    align-content: center;
+}
+
+.video-desc > div {
+    width: 100%;
+}
+
 .video-desc img {
     width: 20px;
     margin-right: 10px;
@@ -362,6 +473,7 @@ button:hover {
 .info-box {
     display: grid;
     grid-gap: 1rem;
+    margin: 1rem 0;
 }
 
 .info-box-3 {
@@ -417,14 +529,41 @@ button:hover {
 }
 
 /* channel overview page */
-.channel-item {
+.channel-list.list {
+    display: block;
+}
+
+.channel-list.grid {
+    display: grid;
+    grid-template-columns: 1fr 1fr 1fr;
+    gap: 1rem;
+}
+
+.channel-item.list {
     padding: 20px 0;
 }
 
+.channel-item.grid > .info-box {
+    display: block;
+}
+
 .channel-banner img {
     width: 100%;
 }
 
+.channel-banner.grid {
+    overflow: hidden;
+}
+
+.channel-banner.list img {
+    width: 100%;
+}
+
+.channel-banner.grid img {
+    width: 250%;
+    transform: translateX(-30%);
+}
+
 
 /* download page */
 .icon-text {
@@ -438,30 +577,74 @@ button:hover {
     cursor: pointer;
 }
 
-.dl-item {
+.dl-list.list {
+    display: block;
+}
+
+.dl-list.grid {
+    display: grid;
+    grid-template-columns: 1fr 1fr 1fr;
+    grid-gap: 1rem;
+}
+
+.dl-item.list {
     display: flex;
     margin: 15px 0;
     align-items: center;
     background-color: var(--highlight-bg);
 }
 
+.dl-item.grid {
+    display: flex;
+    flex-wrap: wrap;
+    background-color: var(--highlight-bg);
+}
+
 .dl-check {
     width: 30px;
 }
 
-.dl-thumb {
+.dl-thumb.list {
     width: 25%;
 }
 
+.dl-thumb.grid {
+    width: 100%;
+}
+
 .dl-item img {
     width: 100%;
 }
 
-.dl-desc {
-    padding-left: 15px;
+.dl-desc.list {
+    padding: 0 15px;
     width: 75%;
 }
 
+.dl-desc.grid {
+    padding: 15px;
+    width: 100%;
+}
+
+.dl-control-icons {
+    display: flex;
+    justify-content: center;
+    padding: 10px 0;
+}
+
+.dl-control-icons img {
+    width: 30px;
+    cursor: pointer;
+    margin: 5px;
+}
+
+#stop-icon {
+    filter: var(--img-filter);
+}
+
+#kill-icon {
+    filter: var(--img-filter-error);
+}
 
 /* status message */
 .download-progress {
@@ -567,9 +750,18 @@ button:hover {
     .boxed-content {
         width: 90%;
     }
-    .video-list {
+    .video-list.grid,
+    .dl-list.grid,
+    .channel-list.grid {
         grid-template-columns: 1fr 1fr;
     }
+    .dl-thumb.list {
+        width: 35%;
+    }
+    .video-item.list {
+        display: grid;
+        grid-template-columns: 35% auto;
+    }
     .two-col {
         display: block;
     }
@@ -583,15 +775,26 @@ button:hover {
     * {
         word-wrap: anywhere;
     }
-    .video-list {
+    .video-list.grid,
+    .dl-list.grid,
+    .channel-list.grid,
+    .video-item.list {
         grid-template-columns: 1fr;
     }
+    .video-desc.grid {
+        height: unset;
+        display: flex;
+        flex-wrap: wrap-reverse;
+    }
     .boxed-content {
         width: 95%;
     }
     .footer {
         text-align: center;
     }
+    .toggle {
+        flex-wrap: wrap;
+    }
     .top-nav {
         flex-wrap: wrap-reverse;
         display: flex;

tubearchivist/static/img/icon-gridview.svg

@@ -0,0 +1,122 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<svg
+   xmlns:dc="http://purl.org/dc/elements/1.1/"
+   xmlns:cc="http://creativecommons.org/ns#"
+   xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
+   xmlns:svg="http://www.w3.org/2000/svg"
+   xmlns="http://www.w3.org/2000/svg"
+   xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+   xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+   width="2000"
+   height="2000"
+   viewBox="0 0 529.16666 529.16735"
+   version="1.1"
+   id="svg8"
+   inkscape:version="0.92.4 (5da689c313, 2019-01-14)"
+   sodipodi:docname="Gridview.svg">
+  <defs
+     id="defs2" />
+  <sodipodi:namedview
+     id="base"
+     pagecolor="#ffffff"
+     bordercolor="#666666"
+     borderopacity="1.0"
+     inkscape:pageopacity="0.0"
+     inkscape:pageshadow="2"
+     inkscape:zoom="0.35729063"
+     inkscape:cx="901.7564"
+     inkscape:cy="1021.9111"
+     inkscape:document-units="mm"
+     inkscape:current-layer="layer1"
+     showgrid="false"
+     units="px"
+     showguides="true"
+     inkscape:guide-bbox="true"
+     inkscape:window-width="1920"
+     inkscape:window-height="1017"
+     inkscape:window-x="-8"
+     inkscape:window-y="-8"
+     inkscape:window-maximized="1">
+    <sodipodi:guide
+       position="247.25932,291.92959"
+       orientation="1,0"
+       id="guide853"
+       inkscape:locked="false" />
+    <sodipodi:guide
+       position="337.22901,167.98535"
+       orientation="0,1"
+       id="guide855"
+       inkscape:locked="false" />
+    <sodipodi:guide
+       position="266.76325,305.4565"
+       orientation="0,1"
+       id="guide857"
+       inkscape:locked="false" />
+    <sodipodi:guide
+       position="257.79774,279.50371"
+       orientation="0,1"
+       id="guide861"
+       inkscape:locked="false" />
+  </sodipodi:namedview>
+  <metadata
+     id="metadata5">
+    <rdf:RDF>
+      <cc:Work
+         rdf:about="">
+        <dc:format>image/svg+xml</dc:format>
+        <dc:type
+           rdf:resource="http://purl.org/dc/dcmitype/StillImage" />
+        <dc:title></dc:title>
+      </cc:Work>
+    </rdf:RDF>
+  </metadata>
+  <g
+     inkscape:label="Ebene 1"
+     inkscape:groupmode="layer"
+     id="layer1"
+     transform="translate(0,232.16736)">
+    <g
+       id="g873"
+       transform="matrix(1.3431799,0,0,1.3431799,-84.854433,26.13855)"
+       style="stroke:none">
+      <rect
+         ry="7.445024"
+         rx="7.445024"
+         y="-121.39048"
+         x="79.903137"
+         height="113.24854"
+         width="167.35619"
+         id="rect815"
+         style="opacity:1;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.56300002;stroke-linecap:round;stroke-linejoin:bevel;stroke-miterlimit:4;stroke-dasharray:1.12600005, 0.56300002999999998;stroke-dashoffset:0;stroke-opacity:0.22508042;paint-order:markers fill stroke" />
+      <rect
+         ry="7.445024"
+         rx="7.445024"
+         y="-121.7837"
+         x="273.05484"
+         height="113.24854"
+         width="167.35619"
+         id="rect815-4"
+         style="opacity:1;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.56300002;stroke-linecap:round;stroke-linejoin:bevel;stroke-miterlimit:4;stroke-dasharray:1.12600006, 0.56300004000000003;stroke-dashoffset:0;stroke-opacity:0.22508042;paint-order:markers fill stroke" />
+      <rect
+         ry="7.445024"
+         rx="7.445024"
+         y="17.882772"
+         x="79.903137"
+         height="113.24854"
+         width="167.35619"
+         id="rect815-2"
+         style="opacity:1;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.56300002;stroke-linecap:round;stroke-linejoin:bevel;stroke-miterlimit:4;stroke-dasharray:1.12600006, 0.56300004000000003;stroke-dashoffset:0;stroke-opacity:0.22508042;paint-order:markers fill stroke" />
+      <rect
+         ry="7.445024"
+         rx="7.445024"
+         y="17.453594"
+         x="273.05484"
+         height="113.24854"
+         width="167.35619"
+         id="rect815-7"
+         style="opacity:1;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.56300002;stroke-linecap:round;stroke-linejoin:bevel;stroke-miterlimit:4;stroke-dasharray:1.12600006, 0.56300004000000003;stroke-dashoffset:0;stroke-opacity:0.22508042;paint-order:markers fill stroke" />
+    </g>
+  </g>
+</svg>

tubearchivist/static/img/icon-listview.svg

@@ -0,0 +1,122 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<svg
+   xmlns:dc="http://purl.org/dc/elements/1.1/"
+   xmlns:cc="http://creativecommons.org/ns#"
+   xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
+   xmlns:svg="http://www.w3.org/2000/svg"
+   xmlns="http://www.w3.org/2000/svg"
+   xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+   xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+   width="2000"
+   height="2000"
+   viewBox="0 0 529.16666 529.16735"
+   version="1.1"
+   id="svg8"
+   inkscape:version="0.92.4 (5da689c313, 2019-01-14)"
+   sodipodi:docname="Listview.svg">
+  <defs
+     id="defs2" />
+  <sodipodi:namedview
+     id="base"
+     pagecolor="#ffffff"
+     bordercolor="#666666"
+     borderopacity="1.0"
+     inkscape:pageopacity="0.0"
+     inkscape:pageshadow="2"
+     inkscape:zoom="0.42053519"
+     inkscape:cx="851.82064"
+     inkscape:cy="1105.7974"
+     inkscape:document-units="mm"
+     inkscape:current-layer="g873"
+     showgrid="false"
+     units="px"
+     showguides="true"
+     inkscape:guide-bbox="true"
+     inkscape:window-width="1920"
+     inkscape:window-height="1017"
+     inkscape:window-x="-8"
+     inkscape:window-y="-8"
+     inkscape:window-maximized="1">
+    <sodipodi:guide
+       position="247.25932,291.92959"
+       orientation="1,0"
+       id="guide853"
+       inkscape:locked="false" />
+    <sodipodi:guide
+       position="337.22901,167.98535"
+       orientation="0,1"
+       id="guide855"
+       inkscape:locked="false" />
+    <sodipodi:guide
+       position="266.76325,305.4565"
+       orientation="0,1"
+       id="guide857"
+       inkscape:locked="false" />
+    <sodipodi:guide
+       position="257.79774,279.50371"
+       orientation="0,1"
+       id="guide861"
+       inkscape:locked="false" />
+    <sodipodi:guide
+       position="22.413775,336.67894"
+       orientation="1,0"
+       id="guide926"
+       inkscape:locked="false" />
+    <sodipodi:guide
+       position="503.71355,217.50031"
+       orientation="1,0"
+       id="guide928"
+       inkscape:locked="false" />
+  </sodipodi:namedview>
+  <metadata
+     id="metadata5">
+    <rdf:RDF>
+      <cc:Work
+         rdf:about="">
+        <dc:format>image/svg+xml</dc:format>
+        <dc:type
+           rdf:resource="http://purl.org/dc/dcmitype/StillImage" />
+        <dc:title></dc:title>
+      </cc:Work>
+    </rdf:RDF>
+  </metadata>
+  <g
+     inkscape:label="Ebene 1"
+     inkscape:groupmode="layer"
+     id="layer1"
+     transform="translate(0,232.16736)">
+    <g
+       id="g873"
+       transform="matrix(1.3431799,0,0,1.3431799,-84.854433,26.13855)">
+      <rect
+         ry="7.445024"
+         rx="7.445024"
+         y="-121.34892"
+         x="79.944702"
+         height="70.107315"
+         width="358.24551"
+         id="rect815"
+         style="opacity:1;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.64810181;stroke-linecap:round;stroke-linejoin:bevel;stroke-miterlimit:4;stroke-dasharray:1.29620369, 0.64810185;stroke-dashoffset:0;stroke-opacity:0.22508042;paint-order:markers fill stroke" />
+      <rect
+         ry="7.4450235"
+         rx="7.4450235"
+         y="-31.167784"
+         x="79.861153"
+         height="70.107315"
+         width="358.32907"
+         id="rect815-20"
+         style="opacity:1;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.64817739;stroke-linecap:round;stroke-linejoin:bevel;stroke-miterlimit:4;stroke-dasharray:1.29635485, 0.64817743;stroke-dashoffset:0;stroke-opacity:0.22508042;paint-order:markers fill stroke" />
+      <rect
+         ry="7.4450235"
+         rx="7.4450231"
+         y="60.024326"
+         x="79.908524"
+         height="70.106117"
+         width="358.28171"
+         id="rect815-8"
+         style="opacity:1;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.64812905;stroke-linecap:round;stroke-linejoin:bevel;stroke-miterlimit:4;stroke-dasharray:1.29625813, 0.64812907;stroke-dashoffset:0;stroke-opacity:0.22508042;paint-order:markers fill stroke" />
+    </g>
+  </g>
+</svg>

tubearchivist/static/img/icon-stop.svg

@@ -0,0 +1,67 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<svg
+   xmlns:dc="http://purl.org/dc/elements/1.1/"
+   xmlns:cc="http://creativecommons.org/ns#"
+   xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
+   xmlns:svg="http://www.w3.org/2000/svg"
+   xmlns="http://www.w3.org/2000/svg"
+   xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+   xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+   width="500"
+   height="500"
+   viewBox="0 0 132.29197 132.29167"
+   version="1.1"
+   id="svg1303"
+   inkscape:version="0.92.4 (5da689c313, 2019-01-14)"
+   sodipodi:docname="Icons_stop.svg">
+  <defs
+     id="defs1297" />
+  <sodipodi:namedview
+     id="base"
+     pagecolor="#ffffff"
+     bordercolor="#666666"
+     borderopacity="1.0"
+     inkscape:pageopacity="0.0"
+     inkscape:pageshadow="2"
+     inkscape:zoom="1.0105705"
+     inkscape:cx="43.182711"
+     inkscape:cy="168.09972"
+     inkscape:document-units="mm"
+     inkscape:current-layer="layer1"
+     showgrid="false"
+     units="px"
+     inkscape:window-width="1920"
+     inkscape:window-height="1017"
+     inkscape:window-x="-8"
+     inkscape:window-y="-8"
+     inkscape:window-maximized="1" />
+  <metadata
+     id="metadata1300">
+    <rdf:RDF>
+      <cc:Work
+         rdf:about="">
+        <dc:format>image/svg+xml</dc:format>
+        <dc:type
+           rdf:resource="http://purl.org/dc/dcmitype/StillImage" />
+        <dc:title></dc:title>
+      </cc:Work>
+    </rdf:RDF>
+  </metadata>
+  <g
+     inkscape:label="Ebene 1"
+     inkscape:groupmode="layer"
+     id="layer1"
+     transform="translate(0,-164.70764)">
+    <rect
+       style="opacity:1;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0;stroke-linecap:round;stroke-linejoin:bevel;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;paint-order:markers fill stroke"
+       id="rect836"
+       width="118.86465"
+       height="118.86465"
+       x="6.7136617"
+       y="171.42116"
+       rx="10.00003"
+       ry="10.00003" />
+  </g>
+</svg>

tubearchivist/static/progress.js

@@ -15,9 +15,13 @@ function checkMessage() {
     req.open('GET', '/downloads/progress', true);
     req.onload = function() {
         var dlProgress = req.response;
+        // var dlStatus = dlProgress['status'];
         if (dlProgress['status']) {
             buildDownloadMessage(dlProgress);
             handleInterval();
+            // if (dlStatus == 'downloading') {
+            //     buildDownloadIcons();
+            // };
         };
     };
     req.send();
@@ -70,4 +74,33 @@ function buildDownloadMessage(dlProgress) {
     message.appendChild(title);
     message.appendChild(messageText);
     box.appendChild(message);
+    if (dlStatus == 'downloading' && dlLevel != 'error') {
+        box.appendChild(buildDownloadIcons());
+    };
 };
+
+
+// add dl control icons
+function buildDownloadIcons() {
+    var iconBox = document.createElement('div');
+    iconBox.classList = 'dl-control-icons';
+    // stop icon
+    var stopIcon = document.createElement('img');
+    stopIcon.setAttribute('id', "stop-icon");
+    stopIcon.setAttribute('title', "Stop Download Queue");
+    stopIcon.setAttribute('src', "/static/img/icon-stop.svg");
+    stopIcon.setAttribute('alt', "stop icon");
+    stopIcon.setAttribute('onclick', 'stopQueue()');
+    // kill icon
+    var killIcon = document.createElement('img');
+    killIcon.setAttribute('id', "kill-icon");
+    killIcon.setAttribute('title', "Kill Download Queue");
+    killIcon.setAttribute('src', "/static/img/icon-close.svg");
+    killIcon.setAttribute('alt', "kill icon");
+    killIcon.setAttribute('onclick', 'killQueue()');
+    // stich together
+    iconBox.appendChild(stopIcon);
+    iconBox.appendChild(killIcon);
+    
+    return iconBox
+}

tubearchivist/static/script.js

@@ -8,24 +8,6 @@ function sortChange(sortValue) {
     }, 500);
 }
 
-function hideWatched(hideValue) {
-    var payload = JSON.stringify({'hide_watched': hideValue});
-    sendPost(payload);
-    setTimeout(function(){
-        location.reload();
-        return false;
-    }, 500);
-}
-
-function showSubscribedOnly(showValue) {
-    var payload = JSON.stringify({'show_subed_only': showValue});
-    sendPost(payload);
-    setTimeout(function(){
-        location.reload();
-        return false;
-    }, 500);
-}
-
 function isWatched(youtube_id) {
     var payload = JSON.stringify({'watched': youtube_id});
     sendPost(payload);
@@ -43,6 +25,32 @@ function unsubscribe(channel_id) {
     document.getElementById(channel_id).remove();
 }
 
+function changeView(image) {
+    var sourcePage = image.getAttribute("data-origin");
+    var newView = image.getAttribute("data-value");
+    var payload = JSON.stringify({'change_view': sourcePage + ":" + newView});
+    sendPost(payload);
+    setTimeout(function(){
+        location.reload();
+        return false;
+    }, 500);
+}
+
+function toggleCheckbox(checkbox) {
+    // pass checkbox id as key and checkbox.checked as value
+    var toggleId = checkbox.id;
+    var toggleVal = checkbox.checked;
+    var payloadDict = {};
+    payloadDict[toggleId] = toggleVal;
+    var payload = JSON.stringify(payloadDict);
+    sendPost(payload);
+    setTimeout(function(){
+        var currPage = window.location.pathname;
+        window.location.replace(currPage);
+        return false;
+    }, 500);
+}
+
 // download page buttons
 function rescanPending() {
     var payload = JSON.stringify({'rescan_pending': true});
@@ -79,6 +87,35 @@ function downloadNow(button) {
     }, 500);
 }
 
+function forgetIgnore(button) {
+    var youtube_id = button.getAttribute('data-id');
+    var payload = JSON.stringify({'forgetIgnore': youtube_id});
+    sendPost(payload);
+    document.getElementById("dl-" + youtube_id).remove();
+}
+
+function addSingle(button) {
+    var youtube_id = button.getAttribute('data-id');
+    var payload = JSON.stringify({'addSingle': youtube_id});
+    sendPost(payload);
+    document.getElementById("dl-" + youtube_id).remove();
+    setTimeout(function(){
+        handleInterval();
+    }, 500);
+}
+
+function stopQueue() {
+    var payload = JSON.stringify({'queue': 'stop'});
+    sendPost(payload);
+    document.getElementById('stop-icon').remove();
+}
+
+function killQueue() {
+    var payload = JSON.stringify({'queue': 'kill'});
+    sendPost(payload);
+    document.getElementById('kill-icon').remove();
+}
+
 // settings page buttons
 function manualImport() {
     var payload = JSON.stringify({'manual-import': true});