Merge development into master

.github/workflows/ci.yml

@@ -79,7 +79,7 @@ jobs:
           fetch-depth: 1
 
       - name: Set up Python 3.8
-        uses: actions/setup-python@v3
+        uses: actions/setup-python@v4
         with:
           python-version: "3.8"
 

.github/workflows/test_bazarr_execution.yml

@@ -35,7 +35,7 @@ jobs:
         working-directory: ${{ env.UI_DIRECTORY }}
 
       - name: Set up Python 3.8
-        uses: actions/setup-python@v3
+        uses: actions/setup-python@v4
         with:
           python-version: "3.8"
 

README.md

@@ -1,4 +1,5 @@
 # bazarr
+
 Bazarr is a companion application to Sonarr and Radarr. It manages and downloads subtitles based on your requirements. You define your preferences by TV show or movie and Bazarr takes care of everything for you.
 
 Be aware that Bazarr doesn't scan disk to detect series and movies: It only takes care of the series and movies that are indexed in Sonarr and Radarr.
@@ -6,10 +7,12 @@ Be aware that Bazarr doesn't scan disk to detect series and movies: It only take
 Thanks to the folks at OpenSubtitles for their logo that was an inspiration for ours.
 
 ## Support on Paypal
+
 At the request of some, here is a way to demonstrate your appreciation for the efforts made in the development of Bazarr:
 [![Donate](https://img.shields.io/badge/Donate-PayPal-green.svg)](https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=XHHRWXT9YB7WE&source=url)
 
 # Status
+
 [![GitHub issues](https://img.shields.io/github/issues/morpheus65535/bazarr.svg?style=flat-square)](https://github.com/morpheus65535/bazarr/issues)
 [![GitHub stars](https://img.shields.io/github/stars/morpheus65535/bazarr.svg?style=flat-square)](https://github.com/morpheus65535/bazarr/stargazers)
 [![Docker Pulls](https://img.shields.io/docker/pulls/linuxserver/bazarr.svg?style=flat-square)](https://hub.docker.com/r/linuxserver/bazarr/)
@@ -17,6 +20,7 @@ At the request of some, here is a way to demonstrate your appreciation for the e
 [![Discord](https://img.shields.io/badge/discord-chat-MH2e2eb.svg?style=flat-square)](https://discord.gg/MH2e2eb)
 
 # Support
+
 For installation and configuration instructions, see [wiki](https://wiki.bazarr.media).
 
 You can reach us for support on [Discord](https://discord.gg/MH2e2eb).
@@ -24,64 +28,69 @@ You can reach us for support on [Discord](https://discord.gg/MH2e2eb).
 If you find a bug, please open an issue on [Github](https://github.com/morpheus65535/bazarr/issues).
 
 # Feature Requests
+
 If you need something that is not already part of Bazarr, feel free to create a feature request on [Feature Upvote](http://features.bazarr.media).
 
 ## Major Features Include:
 
-* Support for major platforms: Windows, Linux, macOS, Raspberry Pi, etc.
-* Automatically add new series and episodes from Sonarr
-* Automatically add new movies from Radarr
-* Series or movies based configuration for subtitles languages
-* Scan your existing library for internal and external subtitles and download any missing
-* Keep history of what was downloaded from where and when
-* Manual search so you can download subtitles on demand
-* Upgrade subtitles previously downloaded when a better one is found
-* Ability to delete external subtitles from disk
-* Currently support 184 subtitles languages with support for forced/foreign subtitles (depending of providers)
-* And a beautiful UI based on Sonarr
+- Support for major platforms: Windows, Linux, macOS, Raspberry Pi, etc.
+- Automatically add new series and episodes from Sonarr
+- Automatically add new movies from Radarr
+- Series or movies based configuration for subtitles languages
+- Scan your existing library for internal and external subtitles and download any missing
+- Keep history of what was downloaded from where and when
+- Manual search so you can download subtitles on demand
+- Upgrade subtitles previously downloaded when a better one is found
+- Ability to delete external subtitles from disk
+- Currently support 184 subtitles languages with support for forced/foreign subtitles (depending of providers)
+- And a beautiful UI based on Sonarr
 
 ## Supported subtitles providers:
-* Addic7ed
-* Argenteam
-* Assrt
-* BetaSeries
-* BSplayer
-* Embedded Subtitles
-* GreekSubtitles
-* Hosszupuska
-* LegendasDivx
-* LegendasTV
-* Ktuvit (Get `hashed_password` using method described [here](https://github.com/XBMCil/service.subtitles.ktuvit))
-* Napiprojekt
-* Napisy24
-* Nekur
-* OpenSubtitles.org
-* Podnapisi
-* RegieLive
-* Sous-Titres.eu
-* Subdivx
-* Subs.sab.bz
-* Subs4Free
-* Subs4Series
-* Subscene
-* Subscenter
-* Subsunacs.net
-* SubSynchro
-* Subtitrari-noi.ro
-* subtitri.id.lv
-* Subtitulamos.tv
-* Sucha
-* Supersubtitles
-* Titlovi
-* Titrari.ro
-* Titulky.com
-* TuSubtitulo
-* TVSubtitles
-* Wizdom
-* XSubs
-* Yavka.net
-* YIFY Subtitles
-* Zimuku
+
+- Addic7ed
+- Argenteam
+- Assrt
+- BetaSeries
+- BSplayer
+- Embedded Subtitles
+- Gestdown.info
+- GreekSubtitles
+- Hosszupuska
+- LegendasDivx
+- LegendasTV
+- Karagarga.in
+- Ktuvit (Get `hashed_password` using method described [here](https://github.com/XBMCil/service.subtitles.ktuvit))
+- Napiprojekt
+- Napisy24
+- Nekur
+- OpenSubtitles.org
+- Podnapisi
+- RegieLive
+- Sous-Titres.eu
+- Subdivx
+- subf2m.co
+- Subs.sab.bz
+- Subs4Free
+- Subs4Series
+- Subscene
+- Subscenter
+- Subsunacs.net
+- SubSynchro
+- Subtitrari-noi.ro
+- subtitri.id.lv
+- Subtitulamos.tv
+- Sucha
+- Supersubtitles
+- Titlovi
+- Titrari.ro
+- Titulky.com
+- TuSubtitulo
+- TVSubtitles
+- Wizdom
+- XSubs
+- Yavka.net
+- YIFY Subtitles
+- Zimuku
 
 ## Screenshot
 
@@ -89,5 +98,5 @@ If you need something that is not already part of Bazarr, feel free to create a
 
 ### License
 
-* [GNU GPL v3](http://www.gnu.org/licenses/gpl.html)
-* Copyright 2010-2019
+- [GNU GPL v3](http://www.gnu.org/licenses/gpl.html)
+- Copyright 2010-2019

bazarr.py

@@ -8,7 +8,7 @@ import sys
 import time
 import atexit
 
-from bazarr.get_args import args
+from bazarr.app.get_args import args
 
 
 def check_python_version():
@@ -30,6 +30,21 @@ def check_python_version():
         sys.exit(1)
 
 
+def get_python_path():
+    if sys.platform == "darwin":
+        # Do not run Python from within macOS framework bundle.
+        python_bundle_path = os.path.join(sys.base_exec_prefix, "Resources", "Python.app", "Contents", "MacOS", "Python")
+        if os.path.exists(python_bundle_path):
+            import tempfile
+
+            python_path = os.path.join(tempfile.mkdtemp(), "python")
+            os.symlink(python_bundle_path, python_path)
+
+            return python_path
+
+    return sys.executable
+
+
 check_python_version()
 
 dir_name = os.path.dirname(__file__)
@@ -49,7 +64,7 @@ def terminate_child_process(ep):
 
 
 def start_bazarr():
-    script = [sys.executable, "-u", os.path.normcase(os.path.join(dir_name, 'bazarr', 'main.py'))] + sys.argv[1:]
+    script = [get_python_path(), "-u", os.path.normcase(os.path.join(dir_name, 'bazarr', 'main.py'))] + sys.argv[1:]
     ep = subprocess.Popen(script, stdout=None, stderr=None, stdin=subprocess.DEVNULL)
     atexit.register(end_child_process, ep=ep)
     signal.signal(signal.SIGTERM, lambda signal_no, frame: terminate_child_process(ep))

bazarr/api/badges/badges.py

@@ -1,14 +1,14 @@
 # coding=utf-8
 
+import operator
+
+from functools import reduce
 from flask import jsonify
 from flask_restful import Resource
 
-import operator
-from functools import reduce
-
-from database import get_exclusion_clause, TableEpisodes, TableShows, TableMovies
-from get_providers import get_throttled_providers
-from utils import get_health_issues
+from app.database import get_exclusion_clause, TableEpisodes, TableShows, TableMovies
+from app.get_providers import get_throttled_providers
+from utilities.health import get_health_issues
 
 from ..utils import authenticate
 

bazarr/api/episodes/blacklist.py

@@ -6,12 +6,14 @@ import pretty
 from flask import request, jsonify
 from flask_restful import Resource
 
-from database import TableEpisodes, TableShows, TableBlacklist
+from app.database import TableEpisodes, TableShows, TableBlacklist
+from subtitles.tools.delete import delete_subtitles
+from sonarr.blacklist import blacklist_log, blacklist_delete_all, blacklist_delete
+from utilities.path_mappings import path_mappings
+from subtitles.mass_download import episode_download_subtitles
+from app.event_handler import event_stream
+
 from ..utils import authenticate, postprocessEpisode
-from utils import blacklist_log, delete_subtitles, blacklist_delete_all, blacklist_delete
-from helper import path_mappings
-from get_subtitle.mass_download import episode_download_subtitles
-from event_handler import event_stream
 
 
 # GET: get blacklist

bazarr/api/episodes/episodes.py

@@ -3,7 +3,8 @@
 from flask import request, jsonify
 from flask_restful import Resource
 
-from database import TableEpisodes
+from app.database import TableEpisodes
+
 from ..utils import authenticate, postprocessEpisode
 
 

bazarr/api/episodes/episodes_subtitles.py

@@ -7,16 +7,18 @@ from flask import request
 from flask_restful import Resource
 from subliminal_patch.core import SUBTITLE_EXTENSIONS
 
-from database import TableShows, TableEpisodes, get_audio_profile_languages, get_profile_id
+from app.database import TableShows, TableEpisodes, get_audio_profile_languages, get_profile_id
+from utilities.path_mappings import path_mappings
+from subtitles.upload import manual_upload_subtitle
+from subtitles.download import generate_subtitles
+from subtitles.tools.delete import delete_subtitles
+from sonarr.history import history_log
+from app.notifier import send_notifications
+from subtitles.indexer.series import store_subtitles
+from app.event_handler import event_stream
+from app.config import settings
+
 from ..utils import authenticate
-from helper import path_mappings
-from get_subtitle.upload import manual_upload_subtitle
-from get_subtitle.download import generate_subtitles
-from utils import history_log, delete_subtitles
-from notifier import send_notifications
-from list_subtitles import store_subtitles
-from event_handler import event_stream
-from config import settings
 
 
 # PATCH: Download Subtitles

bazarr/api/episodes/history.py

@@ -11,10 +11,11 @@ from functools import reduce
 from peewee import fn
 from datetime import timedelta
 
-from database import get_exclusion_clause, TableEpisodes, TableShows, TableHistory, TableBlacklist
+from app.database import get_exclusion_clause, TableEpisodes, TableShows, TableHistory, TableBlacklist
+from app.config import settings
+from utilities.path_mappings import path_mappings
+
 from ..utils import authenticate, postprocessEpisode
-from config import settings
-from helper import path_mappings
 
 
 class EpisodesHistory(Resource):

bazarr/api/episodes/wanted.py

@@ -6,7 +6,8 @@ from flask import request, jsonify
 from flask_restful import Resource
 from functools import reduce
 
-from database import get_exclusion_clause, TableEpisodes, TableShows
+from app.database import get_exclusion_clause, TableEpisodes, TableShows
+
 from ..utils import authenticate, postprocessEpisode
 
 

bazarr/api/files/files.py

@@ -3,7 +3,7 @@
 from flask import request, jsonify
 from flask_restful import Resource
 
-from filesystem import browse_bazarr_filesystem
+from utilities.filesystem import browse_bazarr_filesystem
 
 from ..utils import authenticate
 

bazarr/api/files/files_radarr.py

@@ -3,7 +3,7 @@
 from flask import request, jsonify
 from flask_restful import Resource
 
-from filesystem import browse_radarr_filesystem
+from radarr.filesystem import browse_radarr_filesystem
 
 from ..utils import authenticate
 

bazarr/api/files/files_sonarr.py

@@ -3,7 +3,7 @@
 from flask import request, jsonify
 from flask_restful import Resource
 
-from filesystem import browse_sonarr_filesystem
+from sonarr.filesystem import browse_sonarr_filesystem
 
 from ..utils import authenticate
 

bazarr/api/history/stats.py

@@ -10,7 +10,7 @@ from flask_restful import Resource
 from functools import reduce
 from peewee import fn
 
-from database import TableHistory, TableHistoryMovie
+from app.database import TableHistory, TableHistoryMovie
 
 from ..utils import authenticate
 

bazarr/api/movies/blacklist.py

@@ -6,12 +6,14 @@ import pretty
 from flask import request, jsonify
 from flask_restful import Resource
 
-from database import TableMovies, TableBlacklistMovie
+from app.database import TableMovies, TableBlacklistMovie
+from subtitles.tools.delete import delete_subtitles
+from radarr.blacklist import blacklist_log_movie, blacklist_delete_all_movie, blacklist_delete_movie
+from utilities.path_mappings import path_mappings
+from subtitles.mass_download import movies_download_subtitles
+from app.event_handler import event_stream
+
 from ..utils import authenticate, postprocessMovie
-from utils import blacklist_log_movie, delete_subtitles, blacklist_delete_all_movie, blacklist_delete_movie
-from helper import path_mappings
-from get_subtitle.mass_download import movies_download_subtitles
-from event_handler import event_stream
 
 
 # GET: get blacklist

bazarr/api/movies/history.py

@@ -11,10 +11,11 @@ from functools import reduce
 from peewee import fn
 from datetime import timedelta
 
-from database import get_exclusion_clause, TableMovies, TableHistoryMovie, TableBlacklistMovie
+from app.database import get_exclusion_clause, TableMovies, TableHistoryMovie, TableBlacklistMovie
+from app.config import settings
+from utilities.path_mappings import path_mappings
+
 from ..utils import authenticate, postprocessMovie
-from config import settings
-from helper import path_mappings
 
 
 class MoviesHistory(Resource):

bazarr/api/movies/movies.py

@@ -3,12 +3,13 @@
 from flask import request, jsonify
 from flask_restful import Resource
 
-from database import TableMovies
+from app.database import TableMovies
+from subtitles.indexer.movies import list_missing_subtitles_movies, movies_scan_subtitles
+from app.event_handler import event_stream
+from subtitles.wanted import wanted_search_missing_subtitles_movies
+from subtitles.mass_download import movies_download_subtitles
+
 from ..utils import authenticate, postprocessMovie, None_Keys
-from list_subtitles import list_missing_subtitles_movies, movies_scan_subtitles
-from event_handler import event_stream
-from get_subtitle.wanted import wanted_search_missing_subtitles_movies
-from get_subtitle.mass_download import movies_download_subtitles
 
 
 class Movies(Resource):

bazarr/api/movies/movies_subtitles.py

@@ -7,16 +7,18 @@ from flask import request
 from flask_restful import Resource
 from subliminal_patch.core import SUBTITLE_EXTENSIONS
 
-from database import TableMovies, get_audio_profile_languages, get_profile_id
+from app.database import TableMovies, get_audio_profile_languages, get_profile_id
+from utilities.path_mappings import path_mappings
+from subtitles.upload import manual_upload_subtitle
+from subtitles.download import generate_subtitles
+from subtitles.tools.delete import delete_subtitles
+from radarr.history import history_log_movie
+from app.notifier import send_notifications_movie
+from subtitles.indexer.movies import store_subtitles_movie
+from app.event_handler import event_stream
+from app.config import settings
+
 from ..utils import authenticate
-from helper import path_mappings
-from get_subtitle.upload import manual_upload_subtitle
-from get_subtitle.download import generate_subtitles
-from utils import history_log_movie, delete_subtitles
-from notifier import send_notifications_movie
-from list_subtitles import store_subtitles_movie
-from event_handler import event_stream
-from config import settings
 
 
 # PATCH: Download Subtitles

bazarr/api/movies/wanted.py

@@ -6,7 +6,8 @@ from flask import request, jsonify
 from flask_restful import Resource
 from functools import reduce
 
-from database import get_exclusion_clause, TableMovies
+from app.database import get_exclusion_clause, TableMovies
+
 from ..utils import authenticate, postprocessMovie
 
 

bazarr/api/providers/providers.py

@@ -4,8 +4,9 @@ from flask import request, jsonify
 from flask_restful import Resource
 from operator import itemgetter
 
-from database import TableHistory, TableHistoryMovie
-from get_providers import list_throttled_providers, reset_throttled_providers
+from app.database import TableHistory, TableHistoryMovie
+from app.get_providers import list_throttled_providers, reset_throttled_providers
+
 from ..utils import authenticate, False_Keys
 
 

bazarr/api/providers/providers_episodes.py

@@ -3,14 +3,14 @@
 from flask import request, jsonify
 from flask_restful import Resource
 
-from database import TableEpisodes, TableShows, get_audio_profile_languages, get_profile_id
-from helper import path_mappings
-from get_providers import get_providers
-from get_subtitle.manual import manual_search, manual_download_subtitle
-from utils import history_log
-from config import settings
-from notifier import send_notifications
-from list_subtitles import store_subtitles
+from app.database import TableEpisodes, TableShows, get_audio_profile_languages, get_profile_id
+from utilities.path_mappings import path_mappings
+from app.get_providers import get_providers
+from subtitles.manual import manual_search, manual_download_subtitle
+from sonarr.history import history_log
+from app.config import settings
+from app.notifier import send_notifications
+from subtitles.indexer.series import store_subtitles
 
 from ..utils import authenticate
 

bazarr/api/providers/providers_movies.py

@@ -1,19 +1,20 @@
 # coding=utf-8
 
+import logging
+
 from flask import request, jsonify
 from flask_restful import Resource
 
-from database import TableMovies, get_audio_profile_languages, get_profile_id
-from helper import path_mappings
-from get_providers import get_providers
-from get_subtitle.manual import manual_search, manual_download_subtitle
-from utils import history_log_movie
-from config import settings
-from notifier import send_notifications_movie
-from list_subtitles import store_subtitles_movie
+from app.database import TableMovies, get_audio_profile_languages, get_profile_id
+from utilities.path_mappings import path_mappings
+from app.get_providers import get_providers
+from subtitles.manual import manual_search, manual_download_subtitle
+from radarr.history import history_log_movie
+from app.config import settings
+from app.notifier import send_notifications_movie
+from subtitles.indexer.movies import store_subtitles_movie
 
 from ..utils import authenticate
-import logging
 
 
 class ProviderMovies(Resource):

bazarr/api/series/series.py

@@ -1,17 +1,18 @@
 # coding=utf-8
 
+import operator
+
 from flask import request, jsonify
 from flask_restful import Resource
-
-import operator
 from functools import reduce
 
-from database import get_exclusion_clause, TableEpisodes, TableShows
-from list_subtitles import list_missing_subtitles, series_scan_subtitles
-from get_subtitle.mass_download import series_download_subtitles
-from get_subtitle.wanted import wanted_search_missing_subtitles_series
+from app.database import get_exclusion_clause, TableEpisodes, TableShows
+from subtitles.indexer.series import list_missing_subtitles, series_scan_subtitles
+from subtitles.mass_download import series_download_subtitles
+from subtitles.wanted import wanted_search_missing_subtitles_series
+from app.event_handler import event_stream
+
 from ..utils import authenticate, postprocessSeries, None_Keys
-from event_handler import event_stream
 
 
 class Series(Resource):

bazarr/api/subtitles/subtitles.py

@@ -7,13 +7,17 @@ import gc
 from flask import request
 from flask_restful import Resource
 
-from database import TableEpisodes, TableMovies
-from helper import path_mappings
+from app.database import TableEpisodes, TableMovies
+from utilities.path_mappings import path_mappings
+from subtitles.tools.subsyncer import SubSyncer
+from subtitles.tools.translate import translate_subtitles_file
+from subtitles.tools.mods import subtitles_apply_mods
+from subtitles.indexer.series import store_subtitles
+from subtitles.indexer.movies import store_subtitles_movie
+from app.config import settings
+from app.event_handler import event_stream
+
 from ..utils import authenticate
-from subsyncer import SubSyncer
-from utils import translate_subtitles_file, subtitles_apply_mods
-from list_subtitles import store_subtitles, store_subtitles_movie
-from config import settings
 
 
 class Subtitles(Resource):
@@ -27,7 +31,6 @@ class Subtitles(Resource):
         id = request.form.get('id')
 
         if media_type == 'episode':
-            subtitles_path = path_mappings.path_replace(subtitles_path)
             metadata = TableEpisodes.select(TableEpisodes.path, TableEpisodes.sonarrSeriesId)\
                 .where(TableEpisodes.sonarrEpisodeId == id)\
                 .dicts()\
@@ -38,7 +41,6 @@ class Subtitles(Resource):
 
             video_path = path_mappings.path_replace(metadata['path'])
         else:
-            subtitles_path = path_mappings.path_replace_movie(subtitles_path)
             metadata = TableMovies.select(TableMovies.path).where(TableMovies.radarrId == id).dicts().get_or_none()
 
             if not metadata:
@@ -58,20 +60,16 @@ class Subtitles(Resource):
             del subsync
             gc.collect()
         elif action == 'translate':
+            from_language = os.path.splitext(subtitles_path)[0].rsplit(".", 1)[1].replace('_', '-')
             dest_language = language
             forced = True if request.form.get('forced') == 'true' else False
             hi = True if request.form.get('hi') == 'true' else False
-            result = translate_subtitles_file(video_path=video_path, source_srt_file=subtitles_path,
-                                              to_lang=dest_language,
-                                              forced=forced, hi=hi)
-            if result:
-                if media_type == 'episode':
-                    store_subtitles(path_mappings.path_replace_reverse(video_path), video_path)
-                else:
-                    store_subtitles_movie(path_mappings.path_replace_reverse_movie(video_path), video_path)
-                return '', 200
-            else:
-                return '', 404
+            translate_subtitles_file(video_path=video_path, source_srt_file=subtitles_path,
+                                     from_lang=from_language, to_lang=dest_language, forced=forced, hi=hi,
+                                     media_type="series" if media_type == "episode" else "movies",
+                                     sonarr_series_id=metadata.get('sonarrSeriesId'),
+                                     sonarr_episode_id=int(id),
+                                     radarr_id=id)
         else:
             use_original_format = True if request.form.get('original_format') == 'true' else False
             subtitles_apply_mods(language, subtitles_path, [action], use_original_format)
@@ -82,4 +80,12 @@ class Subtitles(Resource):
         if chmod:
             os.chmod(subtitles_path, chmod)
 
+        if media_type == 'episode':
+            store_subtitles(path_mappings.path_replace_reverse(video_path), video_path)
+            event_stream(type='series', payload=metadata['sonarrSeriesId'])
+            event_stream(type='episode', payload=int(id))
+        else:
+            store_subtitles_movie(path_mappings.path_replace_reverse_movie(video_path), video_path)
+            event_stream(type='movie', payload=int(id))
+
         return '', 204

bazarr/api/system/account.py

@@ -5,8 +5,8 @@ import gc
 from flask import request, session
 from flask_restful import Resource
 
-from config import settings
-from utils import check_credentials
+from app.config import settings
+from utilities.helper import check_credentials
 
 
 class SystemAccount(Resource):

bazarr/api/system/backups.py

@@ -3,8 +3,9 @@
 from flask import jsonify, request
 from flask_restful import Resource
 
+from utilities.backup import get_backup_files, prepare_restore, delete_backup_file, backup_to_zip
+
 from ..utils import authenticate
-from backup import get_backup_files, prepare_restore, delete_backup_file, backup_to_zip
 
 
 class SystemBackups(Resource):

bazarr/api/system/health.py

@@ -3,8 +3,9 @@
 from flask import jsonify
 from flask_restful import Resource
 
+from utilities.health import get_health_issues
+
 from ..utils import authenticate
-from utils import get_health_issues
 
 
 class SystemHealth(Resource):

bazarr/api/system/languages.py

@@ -2,12 +2,12 @@
 
 from flask import request, jsonify
 from flask_restful import Resource
-
 from operator import itemgetter
 
+from app.database import TableHistory, TableHistoryMovie, TableSettingsLanguages
+from languages.get_languages import alpha2_from_alpha3, language_from_alpha2
+
 from ..utils import authenticate, False_Keys
-from database import TableHistory, TableHistoryMovie, TableSettingsLanguages
-from get_languages import alpha2_from_alpha3, language_from_alpha2
 
 
 class Languages(Resource):

bazarr/api/system/languages_profiles.py

@@ -3,8 +3,9 @@
 from flask import jsonify
 from flask_restful import Resource
 
+from app.database import get_profiles_list
+
 from ..utils import authenticate
-from database import get_profiles_list
 
 
 class LanguagesProfiles(Resource):

bazarr/api/system/logs.py

@@ -6,9 +6,10 @@ import os
 from flask import jsonify
 from flask_restful import Resource
 
+from app.logger import empty_log
+from app.get_args import args
+
 from ..utils import authenticate
-from logger import empty_log
-from get_args import args
 
 
 class SystemLogs(Resource):

bazarr/api/system/releases.py

@@ -8,9 +8,10 @@ import logging
 from flask import jsonify
 from flask_restful import Resource
 
+from app.config import settings
+from app.get_args import args
+
 from ..utils import authenticate
-from config import settings
-from get_args import args
 
 
 class SystemReleases(Resource):

bazarr/api/system/searches.py

@@ -3,9 +3,10 @@
 from flask import request, jsonify
 from flask_restful import Resource
 
+from app.config import settings
+from app.database import TableShows, TableMovies
+
 from ..utils import authenticate
-from config import settings
-from database import TableShows, TableMovies
 
 
 class Searches(Resource):

bazarr/api/system/settings.py

@@ -5,13 +5,15 @@ import json
 from flask import request, jsonify
 from flask_restful import Resource
 
+from app.database import TableLanguagesProfiles, TableSettingsLanguages, TableShows, TableMovies, \
+    TableSettingsNotifier
+from app.event_handler import event_stream
+from app.config import settings, save_settings, get_settings
+from app.scheduler import scheduler
+from subtitles.indexer.series import list_missing_subtitles
+from subtitles.indexer.movies import list_missing_subtitles_movies
+
 from ..utils import authenticate
-from database import TableLanguagesProfiles, TableSettingsLanguages, TableShows, TableMovies, TableSettingsNotifier, \
-    update_profile_id_list
-from event_handler import event_stream
-from config import settings, save_settings, get_settings
-from scheduler import scheduler
-from list_subtitles import list_missing_subtitles, list_missing_subtitles_movies
 
 
 class SystemSettings(Resource):
@@ -86,7 +88,6 @@ class SystemSettings(Resource):
                 # Remove deleted profiles
                 TableLanguagesProfiles.delete().where(TableLanguagesProfiles.profileId == profileId).execute()
 
-            update_profile_id_list()
             event_stream("languages")
 
             if settings.general.getboolean('use_sonarr'):

bazarr/api/system/status.py

@@ -2,14 +2,18 @@
 
 import os
 import platform
+import logging
 
 from flask import jsonify
 from flask_restful import Resource
+from tzlocal import get_localzone_name
+
+from radarr.info import get_radarr_info
+from sonarr.info import get_sonarr_info
+from app.get_args import args
+from init import startTime
 
 from ..utils import authenticate
-from utils import get_sonarr_info, get_radarr_info
-from get_args import args
-from init import startTime
 
 
 class SystemStatus(Resource):
@@ -21,6 +25,12 @@ class SystemStatus(Resource):
         if 'BAZARR_PACKAGE_AUTHOR' in os.environ and os.environ['BAZARR_PACKAGE_AUTHOR'] != '':
             package_version = f'{package_version} by {os.environ["BAZARR_PACKAGE_AUTHOR"]}'
 
+        try:
+            timezone = get_localzone_name() or "Undefined"
+        except Exception:
+            timezone = "Exception while getting time zone name."
+            logging.exception("BAZARR is unable to get configured time zone name.")
+
         system_status = {}
         system_status.update({'bazarr_version': os.environ["BAZARR_VERSION"]})
         system_status.update({'package_version': package_version})
@@ -32,5 +42,6 @@ class SystemStatus(Resource):
             os.path.dirname(__file__))))})
         system_status.update({'bazarr_config_directory': args.config_dir})
         system_status.update({'start_time': startTime})
+        system_status.update({'timezone': timezone})
 
         return jsonify(data=system_status)

bazarr/api/system/system.py

@@ -9,7 +9,7 @@ from ..utils import authenticate
 class System(Resource):
     @authenticate
     def post(self):
-        from server import webserver
+        from app.server import webserver
         action = request.args.get('action')
         if action == "shutdown":
             webserver.shutdown()

bazarr/api/system/tasks.py

@@ -3,8 +3,9 @@
 from flask import request, jsonify
 from flask_restful import Resource
 
+from app.scheduler import scheduler
+
 from ..utils import authenticate
-from scheduler import scheduler
 
 
 class SystemTasks(Resource):

bazarr/api/utils.py

@@ -6,10 +6,10 @@ from functools import wraps
 from flask import request, abort
 from operator import itemgetter
 
-from config import settings, base_url
-from get_languages import language_from_alpha2, alpha3_from_alpha2
-from database import get_audio_profile_languages, get_desired_languages
-from helper import path_mappings
+from app.config import settings, base_url
+from languages.get_languages import language_from_alpha2, alpha3_from_alpha2
+from app.database import get_audio_profile_languages, get_desired_languages
+from utilities.path_mappings import path_mappings
 
 None_Keys = ['null', 'undefined', '', None]
 

bazarr/api/webhooks/plex.py

@@ -9,8 +9,9 @@ from flask import request
 from flask_restful import Resource
 from bs4 import BeautifulSoup as bso
 
-from database import TableEpisodes, TableShows, TableMovies
-from get_subtitle.mass_download import episode_download_subtitles, movies_download_subtitles
+from app.database import TableEpisodes, TableShows, TableMovies
+from subtitles.mass_download import episode_download_subtitles, movies_download_subtitles
+
 from ..utils import authenticate
 
 

bazarr/api/webhooks/radarr.py

@@ -3,10 +3,11 @@
 from flask import request
 from flask_restful import Resource
 
-from database import TableMovies
-from get_subtitle.mass_download import movies_download_subtitles
-from list_subtitles import store_subtitles_movie
-from helper import path_mappings
+from app.database import TableMovies
+from subtitles.mass_download import movies_download_subtitles
+from subtitles.indexer.movies import store_subtitles_movie
+from utilities.path_mappings import path_mappings
+
 from ..utils import authenticate
 
 

bazarr/api/webhooks/sonarr.py

@@ -3,10 +3,11 @@
 from flask import request
 from flask_restful import Resource
 
-from database import TableEpisodes, TableShows
-from get_subtitle.mass_download import episode_download_subtitles
-from list_subtitles import store_subtitles
-from helper import path_mappings
+from app.database import TableEpisodes, TableShows
+from subtitles.mass_download import episode_download_subtitles
+from subtitles.indexer.series import store_subtitles
+from utilities.path_mappings import path_mappings
+
 from ..utils import authenticate
 
 

bazarr/app/__init__.py

@@ -0,0 +1 @@
+# coding=utf-8

bazarr/app/app.py

@@ -1,23 +1,18 @@
 # coding=utf-8
 
-from flask import Flask
+from flask import Flask, redirect
 from flask_socketio import SocketIO
-import os
 
-from get_args import args
-from config import settings, base_url
+from .get_args import args
+from .config import settings, base_url
 
 socketio = SocketIO()
 
 
 def create_app():
     # Flask Setup
-    app = Flask(__name__,
-                template_folder=os.path.join(os.path.dirname(__file__), '..', 'frontend', 'build'),
-                static_folder=os.path.join(os.path.dirname(__file__), '..', 'frontend', 'build', 'static'),
-                static_url_path=base_url.rstrip('/') + '/static')
+    app = Flask(__name__)
     app.wsgi_app = ReverseProxied(app.wsgi_app)
-    app.route = prefix_route(app.route, base_url.rstrip('/'))
 
     app.config["SECRET_KEY"] = settings.general.flask_secret_key
     app.config['JSONIFY_PRETTYPRINT_REGULAR'] = True
@@ -30,19 +25,14 @@ def create_app():
 
     socketio.init_app(app, path=base_url.rstrip('/')+'/api/socket.io', cors_allowed_origins='*',
                       async_mode='threading', allow_upgrades=False, transports='polling')
+
+    @app.errorhandler(404)
+    def page_not_found(_):
+        return redirect(base_url, code=302)
+
     return app
 
 
-def prefix_route(route_function, prefix='', mask='{0}{1}'):
-    # Defines a new route function with a prefix.
-    # The mask argument is a `format string` formatted with, in that order: prefix, route
-    def newroute(route, *args, **kwargs):
-        # New function to prefix the route
-        return route_function(mask.format(prefix, route), *args, **kwargs)
-
-    return newroute
-
-
 class ReverseProxied(object):
     def __init__(self, app):
         self.app = app

bazarr/app/check_update.py

@@ -6,11 +6,12 @@ import logging
 import json
 import requests
 import semver
+
 from shutil import rmtree
 from zipfile import ZipFile
 
-from get_args import args
-from config import settings
+from .get_args import args
+from .config import settings
 
 
 def check_releases():
@@ -120,8 +121,8 @@ def apply_update():
     is_updated = False
     update_dir = os.path.join(args.config_dir, 'update')
     bazarr_zip = os.path.join(update_dir, 'bazarr.zip')
-    bazarr_dir = os.path.dirname(os.path.dirname(__file__))
-    build_dir = os.path.join(os.path.dirname(__file__), 'frontend', 'build')
+    bazarr_dir = os.path.dirname(os.path.dirname(os.path.dirname(__file__)))
+    build_dir = os.path.join(bazarr_dir, 'frontend', 'build')
 
     if os.path.isdir(update_dir):
         if os.path.isfile(bazarr_zip):
@@ -168,7 +169,7 @@ def apply_update():
 
     if is_updated:
         logging.debug('BAZARR new release have been installed, now we restart')
-        from server import webserver
+        from .server import webserver
         webserver.restart()
 
 

bazarr/app/config.py

@@ -5,12 +5,10 @@ import os
 import ast
 
 from urllib.parse import quote_plus
-
 from subliminal.cache import region
-
 from simpleconfigparser import simpleconfigparser, configparser, NoOptionError
 
-from get_args import args
+from .get_args import args
 
 
 class SimpleConfigParser(simpleconfigparser):
@@ -202,17 +200,19 @@ defaults = {
     'titulky': {
         'username': '',
         'password': '',
-        'skip_wrong_fps': 'False',
-        'approved_only': 'False',
-        'multithreading': 'True'
+        'approved_only': 'False'
     },
     'embeddedsubtitles': {
-        'include_ass': 'True',
-        'include_srt': 'True',
+        'included_codecs': '[]',
         'hi_fallback': 'False',
-        'mergerfs_mode': 'False',
         'timeout': '600',
     },
+    'karagarga': {
+        'username': '',
+        'password': '',
+        'f_username': '',
+        'f_password': '',
+    },
     'subsync': {
         'use_subsync': 'False',
         'use_subsync_threshold': 'False',
@@ -265,6 +265,7 @@ raw_keys = ['movie_default_forced', 'serie_default_forced']
 
 array_keys = ['excluded_tags',
               'exclude',
+              'included_codecs',
               'subzero_mods',
               'excluded_series_types',
               'enabled_providers',
@@ -498,41 +499,41 @@ def save_settings(settings_items):
 
     # Reconfigure Bazarr to reflect changes
     if configure_debug:
-        from logger import configure_logging
+        from .logger import configure_logging
         configure_logging(settings.general.getboolean('debug') or args.debug)
 
     if configure_captcha:
         configure_captcha_func()
 
     if update_schedule:
-        from scheduler import scheduler
-        from event_handler import event_stream
+        from .scheduler import scheduler
+        from .event_handler import event_stream
         scheduler.update_configurable_tasks()
         event_stream(type='task')
 
     if sonarr_changed:
-        from signalr_client import sonarr_signalr_client
+        from .signalr_client import sonarr_signalr_client
         try:
             sonarr_signalr_client.restart()
         except Exception:
             pass
 
     if radarr_changed:
-        from signalr_client import radarr_signalr_client
+        from .signalr_client import radarr_signalr_client
         try:
             radarr_signalr_client.restart()
         except Exception:
             pass
 
     if update_path_map:
-        from helper import path_mappings
+        from utilities.path_mappings import path_mappings
         path_mappings.update()
 
     if configure_proxy:
         configure_proxy_func()
 
     if exclusion_updated:
-        from event_handler import event_stream
+        from .event_handler import event_stream
         event_stream(type='badges')
         if sonarr_exclusion_updated:
             event_stream(type='reset-episode-wanted')
@@ -540,76 +541,6 @@ def save_settings(settings_items):
             event_stream(type='reset-movie-wanted')
 
 
-def url_sonarr():
-    if settings.sonarr.getboolean('ssl'):
-        protocol_sonarr = "https"
-    else:
-        protocol_sonarr = "http"
-
-    if settings.sonarr.base_url == '':
-        settings.sonarr.base_url = "/"
-    if not settings.sonarr.base_url.startswith("/"):
-        settings.sonarr.base_url = "/" + settings.sonarr.base_url
-    if settings.sonarr.base_url.endswith("/"):
-        settings.sonarr.base_url = settings.sonarr.base_url[:-1]
-
-    if settings.sonarr.port in empty_values:
-        port = ""
-    else:
-        port = f":{settings.sonarr.port}"
-
-    return f"{protocol_sonarr}://{settings.sonarr.ip}{port}{settings.sonarr.base_url}"
-
-
-def url_sonarr_short():
-    if settings.sonarr.getboolean('ssl'):
-        protocol_sonarr = "https"
-    else:
-        protocol_sonarr = "http"
-
-    if settings.sonarr.port in empty_values:
-        port = ""
-    else:
-        port = f":{settings.sonarr.port}"
-
-    return f"{protocol_sonarr}://{settings.sonarr.ip}{port}"
-
-
-def url_radarr():
-    if settings.radarr.getboolean('ssl'):
-        protocol_radarr = "https"
-    else:
-        protocol_radarr = "http"
-
-    if settings.radarr.base_url == '':
-        settings.radarr.base_url = "/"
-    if not settings.radarr.base_url.startswith("/"):
-        settings.radarr.base_url = "/" + settings.radarr.base_url
-    if settings.radarr.base_url.endswith("/"):
-        settings.radarr.base_url = settings.radarr.base_url[:-1]
-
-    if settings.radarr.port in empty_values:
-        port = ""
-    else:
-        port = f":{settings.radarr.port}"
-
-    return f"{protocol_radarr}://{settings.radarr.ip}{port}{settings.radarr.base_url}"
-
-
-def url_radarr_short():
-    if settings.radarr.getboolean('ssl'):
-        protocol_radarr = "https"
-    else:
-        protocol_radarr = "http"
-
-    if settings.radarr.port in empty_values:
-        port = ""
-    else:
-        port = f":{settings.radarr.port}"
-
-    return f"{protocol_radarr}://{settings.radarr.ip}{port}"
-
-
 def get_array_from(property):
     if property:
         if '[' in property:

bazarr/app/database.py

@@ -1,16 +1,20 @@
+# -*- coding: utf-8 -*-
+
 import os
 import atexit
 import json
 import ast
 import time
+
 from peewee import Model, AutoField, TextField, IntegerField, ForeignKeyField, BlobField, BooleanField
 from playhouse.sqliteq import SqliteQueueDatabase
 from playhouse.migrate import SqliteMigrator, migrate
 from playhouse.sqlite_ext import RowIDField
 
-from helper import path_mappings
-from config import settings, get_array_from
-from get_args import args
+from utilities.path_mappings import path_mappings
+
+from .config import settings, get_array_from
+from .get_args import args
 
 database = SqliteQueueDatabase(os.path.join(args.config_dir, 'db', 'bazarr.db'),
                                use_gevent=False,
@@ -476,7 +480,7 @@ def get_profile_cutoff(profile_id):
 
 
 def get_audio_profile_languages(series_id=None, episode_id=None, movie_id=None):
-    from get_languages import alpha2_from_language, alpha3_from_language
+    from languages.get_languages import alpha2_from_language, alpha3_from_language
     audio_languages = []
 
     if series_id:

bazarr/app/event_handler.py

@@ -1,6 +1,6 @@
 # coding=utf-8
 
-from app import socketio
+from .app import socketio
 
 
 def event_stream(type, action="update", payload=None):

bazarr/app/get_args.py

@@ -0,0 +1,38 @@
+# coding=utf-8
+
+import os
+import argparse
+
+from distutils.util import strtobool
+
+no_update = os.environ.get("NO_UPDATE", "false").strip() == "true"
+no_cli = os.environ.get("NO_CLI", "false").strip() == "true"
+
+parser = argparse.ArgumentParser()
+
+parser.register('type', bool, strtobool)
+
+config_dir = os.path.realpath(os.path.join(os.path.dirname(os.path.dirname(__file__)), '..', 'data'))
+parser.add_argument('-c', '--config', default=config_dir, type=str, metavar="DIR",
+                    dest="config_dir", help="Directory containing the configuration (default: %s)" % config_dir)
+parser.add_argument('-p', '--port', type=int, metavar="PORT", dest="port",
+                    help="Port number (default: 6767)")
+if not no_update:
+    parser.add_argument('--no-update', default=False, type=bool, const=True, metavar="BOOL", nargs="?",
+                        help="Disable update functionality (default: False)")
+parser.add_argument('--debug', default=False, type=bool, const=True, metavar="BOOL", nargs="?",
+                    help="Enable console debugging (default: False)")
+parser.add_argument('--release-update', default=False, type=bool, const=True, metavar="BOOL", nargs="?",
+                    help="Enable file based updater (default: False)")
+parser.add_argument('--dev', default=False, type=bool, const=True, metavar="BOOL", nargs="?",
+                    help="Enable developer mode (default: False)")
+parser.add_argument('--no-tasks', default=False, type=bool, const=True, metavar="BOOL", nargs="?",
+                    help="Disable all tasks (default: False)")
+
+
+if not no_cli:
+    args = parser.parse_args()
+    if no_update:
+        args.no_update = True
+else:
+    args = parser.parse_args(["-c", config_dir, "--no-update"])

bazarr/app/get_providers.py

@@ -1,4 +1,5 @@
 # coding=utf-8
+
 import os
 import datetime
 import pytz
@@ -8,12 +9,7 @@ import pretty
 import time
 import socket
 import requests
-import tzlocal
 
-from get_args import args
-from config import settings, get_array_from
-from event_handler import event_stream
-from utils import get_binary, blacklist_log, blacklist_log_movie
 from subliminal_patch.exceptions import TooManyRequests, APIThrottled, ParseResponseError, IPAddressBlocked, \
     MustGetBlacklisted, SearchLimitReached
 from subliminal.providers.opensubtitles import DownloadLimitReached
@@ -21,6 +17,13 @@ from subliminal.exceptions import DownloadLimitExceeded, ServiceUnavailable
 from subliminal import region as subliminal_cache_region
 from subliminal_patch.extensions import provider_registry
 
+from app.get_args import args
+from app.config import settings, get_array_from
+from app.event_handler import event_stream
+from utilities.binaries import get_binary
+from radarr.blacklist import blacklist_log_movie
+from sonarr.blacklist import blacklist_log
+
 
 def time_until_midnight(timezone):
     # type: (datetime.datetime) -> datetime.timedelta
@@ -35,62 +38,69 @@ def time_until_midnight(timezone):
 
 # Titulky resets its download limits at the start of a new day from its perspective - the Europe/Prague timezone
 # Needs to convert to offset-naive dt
-titulky_limit_reset_timedelta = time_until_midnight(timezone=pytz.timezone('Europe/Prague'))
+def titulky_limit_reset_timedelta():
+    return time_until_midnight(timezone=pytz.timezone('Europe/Prague'))
 
-# LegendasDivx reset its searches limit at approximately midnight, Lisbon time, everyday.
-legendasdivx_limit_reset_timedelta = time_until_midnight(timezone=pytz.timezone('Europe/Lisbon')) + \
-                                    datetime.timedelta(minutes=15)
 
-hours_until_end_of_day = time_until_midnight(timezone=tzlocal.get_localzone()).days + 1
+# LegendasDivx reset its searches limit at approximately midnight, Lisbon time, every day. We wait 1 more hours just
+# to be sure.
+def legendasdivx_limit_reset_timedelta():
+    return time_until_midnight(timezone=pytz.timezone('Europe/Lisbon')) + datetime.timedelta(minutes=60)
+
 
 VALID_THROTTLE_EXCEPTIONS = (TooManyRequests, DownloadLimitExceeded, ServiceUnavailable, APIThrottled,
                              ParseResponseError, IPAddressBlocked)
 VALID_COUNT_EXCEPTIONS = ('TooManyRequests', 'ServiceUnavailable', 'APIThrottled', requests.exceptions.Timeout,
                           requests.exceptions.ConnectTimeout, requests.exceptions.ReadTimeout, socket.timeout)
 
-PROVIDER_THROTTLE_MAP = {
-    "default": {
-        TooManyRequests: (datetime.timedelta(hours=1), "1 hour"),
-        DownloadLimitExceeded: (datetime.timedelta(hours=3), "3 hours"),
-        ServiceUnavailable: (datetime.timedelta(minutes=20), "20 minutes"),
-        APIThrottled: (datetime.timedelta(minutes=10), "10 minutes"),
-        ParseResponseError: (datetime.timedelta(hours=6), "6 hours"),
-        requests.exceptions.Timeout: (datetime.timedelta(hours=1), "1 hour"),
-        socket.timeout: (datetime.timedelta(hours=1), "1 hour"),
-        requests.exceptions.ConnectTimeout: (datetime.timedelta(hours=1), "1 hour"),
-        requests.exceptions.ReadTimeout: (datetime.timedelta(hours=1), "1 hour"),
-    },
-    "opensubtitles": {
-        TooManyRequests: (datetime.timedelta(hours=3), "3 hours"),
-        DownloadLimitExceeded: (datetime.timedelta(hours=6), "6 hours"),
-        DownloadLimitReached: (datetime.timedelta(hours=6), "6 hours"),
-        APIThrottled: (datetime.timedelta(seconds=15), "15 seconds"),
-    },
-    "opensubtitlescom": {
-        TooManyRequests: (datetime.timedelta(minutes=1), "1 minute"),
-        DownloadLimitExceeded: (datetime.timedelta(hours=24), "24 hours"),
-    },
-    "addic7ed": {
-        DownloadLimitExceeded: (datetime.timedelta(hours=3), "3 hours"),
-        TooManyRequests: (datetime.timedelta(minutes=5), "5 minutes"),
-        IPAddressBlocked: (datetime.timedelta(hours=1), "1 hours"),
-    },
-    "titulky": {
-        DownloadLimitExceeded: (titulky_limit_reset_timedelta, f"{titulky_limit_reset_timedelta.seconds // 3600 + 1} hours")
-    },
-    "legendasdivx": {
-        TooManyRequests: (datetime.timedelta(hours=3), "3 hours"),
-        DownloadLimitExceeded: (
-            legendasdivx_limit_reset_timedelta,
-            f"{legendasdivx_limit_reset_timedelta.seconds // 3600 + 1} hours"),
-        IPAddressBlocked: (
-            legendasdivx_limit_reset_timedelta,
-            f"{legendasdivx_limit_reset_timedelta.seconds // 3600 + 1} hours"),
-        SearchLimitReached: (
-            legendasdivx_limit_reset_timedelta,
-            f"{legendasdivx_limit_reset_timedelta.seconds // 3600 + 1} hours"),
+
+def provider_throttle_map():
+    return {
+        "default": {
+            TooManyRequests: (datetime.timedelta(hours=1), "1 hour"),
+            DownloadLimitExceeded: (datetime.timedelta(hours=3), "3 hours"),
+            ServiceUnavailable: (datetime.timedelta(minutes=20), "20 minutes"),
+            APIThrottled: (datetime.timedelta(minutes=10), "10 minutes"),
+            ParseResponseError: (datetime.timedelta(hours=6), "6 hours"),
+            requests.exceptions.Timeout: (datetime.timedelta(hours=1), "1 hour"),
+            socket.timeout: (datetime.timedelta(hours=1), "1 hour"),
+            requests.exceptions.ConnectTimeout: (datetime.timedelta(hours=1), "1 hour"),
+            requests.exceptions.ReadTimeout: (datetime.timedelta(hours=1), "1 hour"),
+        },
+        "opensubtitles": {
+            TooManyRequests: (datetime.timedelta(hours=3), "3 hours"),
+            DownloadLimitExceeded: (datetime.timedelta(hours=6), "6 hours"),
+            DownloadLimitReached: (datetime.timedelta(hours=6), "6 hours"),
+            APIThrottled: (datetime.timedelta(seconds=15), "15 seconds"),
+        },
+        "opensubtitlescom": {
+            TooManyRequests: (datetime.timedelta(minutes=1), "1 minute"),
+            DownloadLimitExceeded: (datetime.timedelta(hours=24), "24 hours"),
+        },
+        "addic7ed": {
+            DownloadLimitExceeded: (datetime.timedelta(hours=3), "3 hours"),
+            TooManyRequests: (datetime.timedelta(minutes=5), "5 minutes"),
+            IPAddressBlocked: (datetime.timedelta(hours=1), "1 hours"),
+        },
+        "titulky": {
+            DownloadLimitExceeded: (
+                titulky_limit_reset_timedelta(),
+                f"{titulky_limit_reset_timedelta().seconds // 3600 + 1} hours")
+        },
+        "legendasdivx": {
+            TooManyRequests: (datetime.timedelta(hours=3), "3 hours"),
+            DownloadLimitExceeded: (
+                legendasdivx_limit_reset_timedelta(),
+                f"{legendasdivx_limit_reset_timedelta().seconds // 3600 + 1} hours"),
+            IPAddressBlocked: (
+                legendasdivx_limit_reset_timedelta(),
+                f"{legendasdivx_limit_reset_timedelta().seconds // 3600 + 1} hours"),
+            SearchLimitReached: (
+                legendasdivx_limit_reset_timedelta(),
+                f"{legendasdivx_limit_reset_timedelta().seconds // 3600 + 1} hours"),
+        }
     }
-}
+
 
 PROVIDERS_FORCED_OFF = ["addic7ed", "tvsubtitles", "legendasdivx", "legendastv", "napiprojekt", "shooter",
                         "hosszupuska", "supersubtitles", "titlovi", "argenteam", "assrt", "subscene"]
@@ -205,9 +215,7 @@ def get_providers_auth():
         'titulky': {
             'username': settings.titulky.username,
             'password': settings.titulky.password,
-            'skip_wrong_fps': settings.titulky.getboolean('skip_wrong_fps'),
             'approved_only': settings.titulky.getboolean('approved_only'),
-            'multithreading': settings.titulky.getboolean('multithreading'),
         },
         'titlovi': {
             'username': settings.titlovi.username,
@@ -218,21 +226,25 @@ def get_providers_auth():
             'hashed_password': settings.ktuvit.hashed_password,
         },
         'embeddedsubtitles': {
-            'include_ass': settings.embeddedsubtitles.getboolean('include_ass'),
-            'include_srt': settings.embeddedsubtitles.getboolean('include_srt'),
+            'included_codecs': get_array_from(settings.embeddedsubtitles.included_codecs),
             'hi_fallback': settings.embeddedsubtitles.getboolean('hi_fallback'),
-            'mergerfs_mode': settings.embeddedsubtitles.getboolean('mergerfs_mode'),
             'cache_dir': os.path.join(args.config_dir, "cache"),
             'ffprobe_path': _FFPROBE_BINARY,
             'ffmpeg_path': _FFMPEG_BINARY,
             'timeout': settings.embeddedsubtitles.timeout,
-        }
+        },
+        'karagarga': {
+            'username': settings.karagarga.username,
+            'password': settings.karagarga.password,
+            'f_username': settings.karagarga.f_username,
+            'f_password': settings.karagarga.f_password,
+        },
     }
 
 
 def _handle_mgb(name, exception):
     # There's no way to get Radarr/Sonarr IDs from subliminal_patch. Blacklisted subtitles
-    # will not appear on fronted but they will work with utils.get_blacklist
+    # will not appear on fronted but they will work with get_blacklist
     if exception.media_type == "series":
         blacklist_log("", "", name, exception.id, "")
     else:
@@ -250,8 +262,8 @@ def provider_throttle(name, exception):
             if isinstance(cls, valid_cls):
                 cls = valid_cls
 
-    throttle_data = PROVIDER_THROTTLE_MAP.get(name, PROVIDER_THROTTLE_MAP["default"]).get(cls, None) or \
-        PROVIDER_THROTTLE_MAP["default"].get(cls, None)
+    throttle_data = provider_throttle_map().get(name, provider_throttle_map()["default"]).get(cls, None) or \
+        provider_throttle_map()["default"].get(cls, None)
 
     if throttle_data:
         throttle_delta, throttle_description = throttle_data

bazarr/app/libs.py

@@ -2,11 +2,12 @@
 
 import os
 import sys
+
 from shutil import rmtree
 
 
 def clean_libs():
-    libs_dir = os.path.join(os.path.dirname(os.path.dirname(__file__)), 'libs')
+    libs_dir = os.path.join(os.path.dirname(os.path.dirname(os.path.dirname(__file__))), 'libs')
 
     # Delete the old module almost empty directory compatible only with Python 2.7.x that cause bad magic number error
     # if they are present in Python 3.x.
@@ -17,7 +18,7 @@ def clean_libs():
 
 
 def set_libs():
-    sys.path.insert(0, os.path.join(os.path.dirname(__file__), '../libs/'))
+    sys.path.insert(0, os.path.join(os.path.dirname(os.path.dirname(__file__)), '../libs/'))
 
 
 clean_libs()

bazarr/app/logger.py

@@ -10,8 +10,8 @@ import warnings
 from logging.handlers import TimedRotatingFileHandler
 from pytz_deprecation_shim import PytzUsageWarning
 
-from get_args import args
-from config import settings
+from .get_args import args
+from .config import settings
 
 
 logger = logging.getLogger()
@@ -146,7 +146,7 @@ class PatchedTimedRotatingFileHandler(TimedRotatingFileHandler):
     def __init__(self, filename, when='h', interval=1, backupCount=0, encoding=None, delay=False, utc=False,
                  atTime=None, errors=None):
         super(PatchedTimedRotatingFileHandler, self).__init__(filename, when, interval, backupCount, encoding, delay, utc,
-                                                       atTime, errors)
+                                                              atTime, errors)
 
     def getFilesToDelete(self):
         """

bazarr/app/notifier.py

@@ -3,7 +3,7 @@
 import apprise
 import logging
 
-from database import TableSettingsNotifier, TableEpisodes, TableShows, TableMovies
+from .database import TableSettingsNotifier, TableEpisodes, TableShows, TableMovies
 
 
 def update_notifier():
@@ -84,6 +84,8 @@ def get_movie(radarr_id):
 
 def send_notifications(sonarr_series_id, sonarr_episode_id, message):
     providers = get_notifier_providers()
+    if not len(providers):
+        return
     series = get_series(sonarr_series_id)
     if not series:
         return
@@ -114,6 +116,8 @@ def send_notifications(sonarr_series_id, sonarr_episode_id, message):
 
 def send_notifications_movie(radarr_id, message):
     providers = get_notifier_providers()
+    if not len(providers):
+        return
     movie = get_movie(radarr_id)
     if not movie:
         return

bazarr/app/scheduler.py

@@ -13,20 +13,23 @@ from datetime import datetime, timedelta
 from calendar import day_name
 from random import randrange
 
-from get_episodes import sync_episodes, update_all_episodes
-from get_movies import update_movies, update_all_movies
-from get_series import update_series
-from config import settings
-from get_subtitle.wanted import wanted_search_missing_subtitles_series, wanted_search_missing_subtitles_movies
-from get_subtitle.upgrade import upgrade_subtitles
-from utils import cache_maintenance, check_health
-from get_args import args
+from sonarr.sync.series import update_series
+from sonarr.sync.episodes import sync_episodes, update_all_episodes
+from radarr.sync.movies import update_movies, update_all_movies
+from subtitles.wanted import wanted_search_missing_subtitles_series, wanted_search_missing_subtitles_movies
+from subtitles.upgrade import upgrade_subtitles
+from utilities.cache import cache_maintenance
+from utilities.health import check_health
+from utilities.backup import backup_to_zip
+
+from .config import settings
+from .get_args import args
+from .event_handler import event_stream
+
 if not args.no_update:
-    from check_update import check_if_new_update, check_releases
+    from .check_update import check_if_new_update, check_releases
 else:
-    from check_update import check_releases
-from event_handler import event_stream
-from backup import backup_to_zip
+    from .check_update import check_releases
 
 
 class Scheduler:

bazarr/app/server.py

@@ -4,18 +4,22 @@ import warnings
 import logging
 import os
 import io
+
 from waitress.server import create_server
 
-from get_args import args
-from config import settings, base_url
-from database import database
+from api import api_bp_list
+from .ui import ui_bp
+from .get_args import args
+from .config import settings, base_url
+from .database import database
+from .app import create_app
 
-from app import create_app
 app = create_app()
 
-from api import api_bp_list  # noqa E402
 for item in api_bp_list:
-    app.register_blueprint(item, url_prefix=base_url.rstrip('/') + '/api')
+    ui_bp.register_blueprint(item, url_prefix='/api')
+
+app.register_blueprint(ui_bp, url_prefix=base_url.rstrip('/'))
 
 
 class Server:
@@ -37,7 +41,10 @@ class Server:
             logging.info(
                 'BAZARR is started and waiting for request on http://' + str(settings.general.ip) + ':' + (str(
                     args.port) if args.port else str(settings.general.port)) + str(base_url))
-            self.server.run()
+            try:
+                self.server.run()
+            except Exception:
+                pass
         except KeyboardInterrupt:
             self.shutdown()
 
@@ -51,7 +58,7 @@ class Server:
             try:
                 stop_file = io.open(os.path.join(args.config_dir, "bazarr.stop"), "w", encoding='UTF-8')
             except Exception as e:
-                logging.error('BAZARR Cannot create bazarr.stop file: ' + repr(e))
+                logging.error('BAZARR Cannot create stop file: ' + repr(e))
             else:
                 logging.info('Bazarr is being shutdown...')
                 stop_file.write(str(''))
@@ -68,7 +75,7 @@ class Server:
             try:
                 restart_file = io.open(os.path.join(args.config_dir, "bazarr.restart"), "w", encoding='UTF-8')
             except Exception as e:
-                logging.error('BAZARR Cannot create bazarr.restart file: ' + repr(e))
+                logging.error('BAZARR Cannot create restart file: ' + repr(e))
             else:
                 logging.info('Bazarr is being restarted...')
                 restart_file.write(str(''))

bazarr/app/signalr_client.py

@@ -1,25 +1,24 @@
 # coding=utf-8
 
 import logging
-
 import json
-import os
 import time
+
 from requests import Session
 from signalr import Connection
 from requests.exceptions import ConnectionError
 from signalrcore.hub_connection_builder import HubConnectionBuilder
 
-from config import settings, url_sonarr, url_radarr
-from get_episodes import sync_episodes, sync_one_episode
-from get_series import update_series, update_one_series
-from get_movies import update_movies, update_one_movie
-from scheduler import scheduler
-from utils import get_sonarr_info
-from get_args import args
+from constants import headers
+from sonarr.sync.episodes import sync_episodes, sync_one_episode
+from sonarr.sync.series import update_series, update_one_series
+from radarr.sync.movies import update_movies, update_one_movie
+from sonarr.info import get_sonarr_info, url_sonarr
+from radarr.info import url_radarr
 
-
-headers = {"User-Agent": os.environ["SZ_USER_AGENT"]}
+from .config import settings
+from .scheduler import scheduler
+from .get_args import args
 
 
 class SonarrSignalrClientLegacy:

bazarr/app/ui.py

@@ -0,0 +1,153 @@
+# coding=utf-8
+
+import os
+import requests
+
+from flask import request, abort, render_template, Response, session, send_file, stream_with_context, Blueprint
+from functools import wraps
+from urllib.parse import unquote
+
+from constants import headers
+from sonarr.info import get_sonarr_info, url_sonarr
+from radarr.info import get_radarr_info, url_radarr
+from utilities.helper import check_credentials
+
+from .config import settings, base_url
+from .database import System
+from .get_args import args
+
+ui_bp = Blueprint('ui', __name__,
+                  template_folder=os.path.join(os.path.dirname(os.path.dirname(os.path.dirname(__file__))),
+                                               'frontend', 'build'),
+                  static_folder=os.path.join(os.path.dirname(os.path.dirname(os.path.dirname(__file__))), 'frontend',
+                                             'build', 'assets'),
+                  static_url_path='/assets')
+
+static_bp = Blueprint('images', __name__,
+                      static_folder=os.path.join(os.path.dirname(os.path.dirname(os.path.dirname(__file__))),
+                                                 'frontend', 'build', 'images'), static_url_path='/images')
+
+ui_bp.register_blueprint(static_bp)
+
+
+def check_login(actual_method):
+    @wraps(actual_method)
+    def wrapper(*args, **kwargs):
+        if settings.auth.type == 'basic':
+            auth = request.authorization
+            if not (auth and check_credentials(request.authorization.username, request.authorization.password)):
+                return ('Unauthorized', 401, {
+                    'WWW-Authenticate': 'Basic realm="Login Required"'
+                })
+        elif settings.auth.type == 'form':
+            if 'logged_in' not in session:
+                return abort(401, message="Unauthorized")
+        actual_method(*args, **kwargs)
+
+
+@ui_bp.route('/', defaults={'path': ''})
+@ui_bp.route('/<path:path>')
+def catch_all(path):
+    auth = True
+    if settings.auth.type == 'basic':
+        auth = request.authorization
+        if not (auth and check_credentials(request.authorization.username, request.authorization.password)):
+            return ('Unauthorized', 401, {
+                'WWW-Authenticate': 'Basic realm="Login Required"'
+            })
+    elif settings.auth.type == 'form':
+        if 'logged_in' not in session:
+            auth = False
+
+    try:
+        updated = System.get().updated
+    except Exception:
+        updated = '0'
+
+    inject = dict()
+    inject["baseUrl"] = base_url
+    inject["canUpdate"] = not args.no_update
+    inject["hasUpdate"] = updated != '0'
+
+    if auth:
+        inject["apiKey"] = settings.auth.apikey
+
+    template_url = base_url
+    if not template_url.endswith("/"):
+        template_url += "/"
+
+    return render_template("index.html", BAZARR_SERVER_INJECT=inject, baseUrl=template_url)
+
+
+@check_login
+@ui_bp.route('/bazarr.log')
+def download_log():
+    return send_file(os.path.join(args.config_dir, 'log', 'bazarr.log'), cache_timeout=0, as_attachment=True)
+
+
+@check_login
+@ui_bp.route('/images/series/<path:url>', methods=['GET'])
+def series_images(url):
+    url = url.strip("/")
+    apikey = settings.sonarr.apikey
+    baseUrl = settings.sonarr.base_url
+    if get_sonarr_info.is_legacy():
+        url_image = (url_sonarr() + '/api/' + url.lstrip(baseUrl) + '?apikey=' +
+                     apikey).replace('poster-250', 'poster-500')
+    else:
+        url_image = (url_sonarr() + '/api/v3/' + url.lstrip(baseUrl) + '?apikey=' +
+                     apikey).replace('poster-250', 'poster-500')
+    try:
+        req = requests.get(url_image, stream=True, timeout=15, verify=False, headers=headers)
+    except Exception:
+        return '', 404
+    else:
+        return Response(stream_with_context(req.iter_content(2048)), content_type=req.headers['content-type'])
+
+
+@check_login
+@ui_bp.route('/images/movies/<path:url>', methods=['GET'])
+def movies_images(url):
+    apikey = settings.radarr.apikey
+    baseUrl = settings.radarr.base_url
+    if get_radarr_info.is_legacy():
+        url_image = url_radarr() + '/api/' + url.lstrip(baseUrl) + '?apikey=' + apikey
+    else:
+        url_image = url_radarr() + '/api/v3/' + url.lstrip(baseUrl) + '?apikey=' + apikey
+    try:
+        req = requests.get(url_image, stream=True, timeout=15, verify=False, headers=headers)
+    except Exception:
+        return '', 404
+    else:
+        return Response(stream_with_context(req.iter_content(2048)), content_type=req.headers['content-type'])
+
+
+def configured():
+    System.update({System.configured: '1'}).execute()
+
+
+@check_login
+@ui_bp.route('/test', methods=['GET'])
+@ui_bp.route('/test/<protocol>/<path:url>', methods=['GET'])
+def proxy(protocol, url):
+    url = protocol + '://' + unquote(url)
+    params = request.args
+    try:
+        result = requests.get(url, params, allow_redirects=False, verify=False, timeout=5, headers=headers)
+    except Exception as e:
+        return dict(status=False, error=repr(e))
+    else:
+        if result.status_code == 200:
+            try:
+                version = result.json()['version']
+                return dict(status=True, version=version)
+            except Exception:
+                return dict(status=False, error='Error Occurred. Check your settings.')
+        elif result.status_code == 401:
+            return dict(status=False, error='Access Denied. Check API key.')
+        elif result.status_code == 404:
+            return dict(status=False, error='Cannot get version. Maybe unsupported legacy API call?')
+        elif 300 <= result.status_code <= 399:
+            return dict(status=False, error='Wrong URL Base.')
+        else:
+            return dict(status=False, error=result.raise_for_status())

bazarr/constants.py

@@ -0,0 +1,10 @@
+# coding=utf-8
+
+import os
+import re
+
+# set Bazarr user-agent used to make requests
+headers = {"User-Agent": os.environ["SZ_USER_AGENT"]}
+
+# hearing-impaired detection regex
+hi_regex = re.compile(r'[*¶♫♪].{3,}[*¶♫♪]|[\[\(\{].{3,}[\]\)\}](?<!{\\an\d})')

bazarr/filesystem.py

@@ -1,106 +0,0 @@
-# coding=utf-8
-
-import os
-import requests
-import logging
-import string
-
-from config import settings, url_sonarr, url_radarr
-from utils import get_sonarr_info, get_radarr_info
-
-headers = {"User-Agent": os.environ["SZ_USER_AGENT"]}
-
-
-def browse_bazarr_filesystem(path='#'):
-    if path == '#' or path == '/' or path == '':
-        if os.name == 'nt':
-            dir_list = []
-            for drive in string.ascii_uppercase:
-                drive_letter = drive + ':\\'
-                if os.path.exists(drive_letter):
-                    dir_list.append(drive_letter)
-        else:
-            path = "/"
-            dir_list = [f for f in os.listdir(path) if os.path.isdir(os.path.join(path, f))]
-    else:
-        dir_list = [f for f in os.listdir(path) if os.path.isdir(os.path.join(path, f))]
-
-    data = []
-    for item in dir_list:
-        full_path = os.path.join(path, item, '')
-        item = {
-            "name": item,
-            "path": full_path
-        }
-        data.append(item)
-
-    parent = os.path.dirname(path)
-
-    result = {'directories': sorted(data, key=lambda i: i['name'])}
-    if path == '#':
-        result.update({'parent': '#'})
-    else:
-        result.update({'parent': parent})
-
-    return result
-
-
-def browse_sonarr_filesystem(path='#'):
-    if path == '#':
-        path = ''
-    if get_sonarr_info.is_legacy():
-        url_sonarr_api_filesystem = url_sonarr() + "/api/filesystem?path=" + path + \
-                                    "&allowFoldersWithoutTrailingSlashes=true&includeFiles=false&apikey=" + \
-                                    settings.sonarr.apikey
-    else:
-        url_sonarr_api_filesystem = url_sonarr() + "/api/v3/filesystem?path=" + path + \
-                                    "&allowFoldersWithoutTrailingSlashes=true&includeFiles=false&apikey=" + \
-                                    settings.sonarr.apikey
-    try:
-        r = requests.get(url_sonarr_api_filesystem, timeout=60, verify=False, headers=headers)
-        r.raise_for_status()
-    except requests.exceptions.HTTPError:
-        logging.exception("BAZARR Error trying to get series from Sonarr. Http error.")
-        return
-    except requests.exceptions.ConnectionError:
-        logging.exception("BAZARR Error trying to get series from Sonarr. Connection Error.")
-        return
-    except requests.exceptions.Timeout:
-        logging.exception("BAZARR Error trying to get series from Sonarr. Timeout Error.")
-        return
-    except requests.exceptions.RequestException:
-        logging.exception("BAZARR Error trying to get series from Sonarr.")
-        return
-
-    return r.json()
-
-
-def browse_radarr_filesystem(path='#'):
-    if path == '#':
-        path = ''
-
-    if get_radarr_info.is_legacy():
-        url_radarr_api_filesystem = url_radarr() + "/api/filesystem?path=" + path + \
-                                    "&allowFoldersWithoutTrailingSlashes=true&includeFiles=false&apikey=" + \
-                                    settings.radarr.apikey
-    else:
-        url_radarr_api_filesystem = url_radarr() + "/api/v3/filesystem?path=" + path + \
-                                    "&allowFoldersWithoutTrailingSlashes=true&includeFiles=false&apikey=" + \
-                                    settings.radarr.apikey
-    try:
-        r = requests.get(url_radarr_api_filesystem, timeout=60, verify=False, headers=headers)
-        r.raise_for_status()
-    except requests.exceptions.HTTPError:
-        logging.exception("BAZARR Error trying to get series from Radarr. Http error.")
-        return
-    except requests.exceptions.ConnectionError:
-        logging.exception("BAZARR Error trying to get series from Radarr. Connection Error.")
-        return
-    except requests.exceptions.Timeout:
-        logging.exception("BAZARR Error trying to get series from Radarr. Timeout Error.")
-        return
-    except requests.exceptions.RequestException:
-        logging.exception("BAZARR Error trying to get series from Radarr.")
-        return
-
-    return r.json()

bazarr/get_args.py

@@ -1,37 +1,4 @@
 # coding=utf-8
 
-import os
-import argparse
-
-from distutils.util import strtobool
-
-no_update = bool(os.environ.get("NO_UPDATE", False))
-parser = argparse.ArgumentParser()
-
-
-def get_args():
-    parser.register('type', bool, strtobool)
-
-    config_dir = os.path.realpath(os.path.join(os.path.dirname(__file__), '..', 'data'))
-    parser.add_argument('-c', '--config', default=config_dir, type=str, metavar="DIR",
-                        dest="config_dir", help="Directory containing the configuration (default: %s)" % config_dir)
-    parser.add_argument('-p', '--port', type=int, metavar="PORT", dest="port",
-                        help="Port number (default: 6767)")
-    if not no_update:
-        parser.add_argument('--no-update', default=False, type=bool, const=True, metavar="BOOL", nargs="?",
-                            help="Disable update functionality (default: False)")
-    parser.add_argument('--debug', default=False, type=bool, const=True, metavar="BOOL", nargs="?",
-                        help="Enable console debugging (default: False)")
-    parser.add_argument('--release-update', default=False, type=bool, const=True, metavar="BOOL", nargs="?",
-                        help="Enable file based updater (default: False)")
-    parser.add_argument('--dev', default=False, type=bool, const=True, metavar="BOOL", nargs="?",
-                        help="Enable developer mode (default: False)")
-    parser.add_argument('--no-tasks', default=False, type=bool, const=True, metavar="BOOL", nargs="?",
-                        help="Disable all tasks (default: False)")
-
-    return parser.parse_args()
-
-
-args = get_args()
-if no_update:
-    args.no_update = True
+# This is required to prevent daemon (bazarr.py) from raising an ImportError Exception after upgrading from 1.0.4
+from .app.get_args import args  # noqa: W0611

bazarr/get_movies.py

@@ -1,586 +0,0 @@
-# coding=utf-8
-
-import os
-import requests
-import logging
-from peewee import IntegrityError
-
-from config import settings, url_radarr
-from helper import path_mappings
-from utils import get_radarr_info
-from list_subtitles import store_subtitles_movie, movies_full_scan_subtitles
-from get_rootfolder import check_radarr_rootfolder
-
-from get_subtitle.mass_download import movies_download_subtitles
-from database import TableMovies
-from event_handler import event_stream, show_progress, hide_progress
-from get_languages import language_from_alpha2
-
-headers = {"User-Agent": os.environ["SZ_USER_AGENT"]}
-
-
-def update_all_movies():
-    movies_full_scan_subtitles()
-    logging.info('BAZARR All existing movie subtitles indexed from disk.')
-
-
-def update_movies(send_event=True):
-    check_radarr_rootfolder()
-    logging.debug('BAZARR Starting movie sync from Radarr.')
-    apikey_radarr = settings.radarr.apikey
-
-    movie_default_enabled = settings.general.getboolean('movie_default_enabled')
-
-    if movie_default_enabled is True:
-        movie_default_profile = settings.general.movie_default_profile
-        if movie_default_profile == '':
-            movie_default_profile = None
-    else:
-        movie_default_profile = None
-
-    if apikey_radarr is None:
-        pass
-    else:
-        audio_profiles = get_profile_list()
-        tagsDict = get_tags()
-
-        # Get movies data from radarr
-        movies = get_movies_from_radarr_api(url=url_radarr(), apikey_radarr=apikey_radarr)
-        if not movies:
-            return
-        else:
-            # Get current movies in DB
-            current_movies_db = TableMovies.select(TableMovies.tmdbId, TableMovies.path, TableMovies.radarrId).dicts()
-
-            current_movies_db_list = [x['tmdbId'] for x in current_movies_db]
-
-            current_movies_radarr = []
-            movies_to_update = []
-            movies_to_add = []
-            altered_movies = []
-
-            # Build new and updated movies
-            movies_count = len(movies)
-            for i, movie in enumerate(movies):
-                if send_event:
-                    show_progress(id='movies_progress',
-                                  header='Syncing movies...',
-                                  name=movie['title'],
-                                  value=i,
-                                  count=movies_count)
-
-                if movie['hasFile'] is True:
-                    if 'movieFile' in movie:
-                        if movie['movieFile']['size'] > 20480:
-                            # Add movies in radarr to current movies list
-                            current_movies_radarr.append(str(movie['tmdbId']))
-
-                            if str(movie['tmdbId']) in current_movies_db_list:
-                                movies_to_update.append(movieParser(movie, action='update',
-                                                                    tags_dict=tagsDict,
-                                                                    movie_default_profile=movie_default_profile,
-                                                                    audio_profiles=audio_profiles))
-                            else:
-                                movies_to_add.append(movieParser(movie, action='insert',
-                                                                 tags_dict=tagsDict,
-                                                                 movie_default_profile=movie_default_profile,
-                                                                 audio_profiles=audio_profiles))
-
-            if send_event:
-                hide_progress(id='movies_progress')
-
-            # Remove old movies from DB
-            removed_movies = list(set(current_movies_db_list) - set(current_movies_radarr))
-
-            for removed_movie in removed_movies:
-                try:
-                    TableMovies.delete().where(TableMovies.tmdbId == removed_movie).execute()
-                except Exception as e:
-                    logging.error(f"BAZARR cannot remove movie tmdbId {removed_movie} because of {e}")
-                    continue
-
-            # Update movies in DB
-            movies_in_db_list = []
-            movies_in_db = TableMovies.select(TableMovies.radarrId,
-                                              TableMovies.title,
-                                              TableMovies.path,
-                                              TableMovies.tmdbId,
-                                              TableMovies.overview,
-                                              TableMovies.poster,
-                                              TableMovies.fanart,
-                                              TableMovies.audio_language,
-                                              TableMovies.sceneName,
-                                              TableMovies.monitored,
-                                              TableMovies.sortTitle,
-                                              TableMovies.year,
-                                              TableMovies.alternativeTitles,
-                                              TableMovies.format,
-                                              TableMovies.resolution,
-                                              TableMovies.video_codec,
-                                              TableMovies.audio_codec,
-                                              TableMovies.imdbId,
-                                              TableMovies.movie_file_id,
-                                              TableMovies.tags,
-                                              TableMovies.file_size).dicts()
-
-            for item in movies_in_db:
-                movies_in_db_list.append(item)
-
-            movies_to_update_list = [i for i in movies_to_update if i not in movies_in_db_list]
-
-            for updated_movie in movies_to_update_list:
-                try:
-                    TableMovies.update(updated_movie).where(TableMovies.tmdbId == updated_movie['tmdbId']).execute()
-                except IntegrityError as e:
-                    logging.error(f"BAZARR cannot update movie {updated_movie['path']} because of {e}")
-                    continue
-                else:
-                    altered_movies.append([updated_movie['tmdbId'],
-                                           updated_movie['path'],
-                                           updated_movie['radarrId'],
-                                           updated_movie['monitored']])
-
-            # Insert new movies in DB
-            for added_movie in movies_to_add:
-                try:
-                    result = TableMovies.insert(added_movie).on_conflict(action='IGNORE').execute()
-                except IntegrityError as e:
-                    logging.error(f"BAZARR cannot insert movie {added_movie['path']} because of {e}")
-                    continue
-                else:
-                    if result > 0:
-                        altered_movies.append([added_movie['tmdbId'],
-                                               added_movie['path'],
-                                               added_movie['radarrId'],
-                                               added_movie['monitored']])
-                        if send_event:
-                            event_stream(type='movie', action='update', payload=int(added_movie['radarrId']))
-                    else:
-                        logging.debug('BAZARR unable to insert this movie into the database:',
-                                      path_mappings.path_replace_movie(added_movie['path']))
-
-            # Store subtitles for added or modified movies
-            for i, altered_movie in enumerate(altered_movies, 1):
-                store_subtitles_movie(altered_movie[1], path_mappings.path_replace_movie(altered_movie[1]))
-
-            logging.debug('BAZARR All movies synced from Radarr into database.')
-
-
-def update_one_movie(movie_id, action, defer_search=False):
-    logging.debug('BAZARR syncing this specific movie from Radarr: {}'.format(movie_id))
-
-    # Check if there's a row in database for this movie ID
-    existing_movie = TableMovies.select(TableMovies.path)\
-        .where(TableMovies.radarrId == movie_id)\
-        .dicts()\
-        .get_or_none()
-
-    # Remove movie from DB
-    if action == 'deleted':
-        if existing_movie:
-            try:
-                TableMovies.delete().where(TableMovies.radarrId == movie_id).execute()
-            except Exception as e:
-                logging.error(f"BAZARR cannot delete movie {existing_movie['path']} because of {e}")
-            else:
-                event_stream(type='movie', action='delete', payload=int(movie_id))
-                logging.debug('BAZARR deleted this movie from the database:{}'.format(path_mappings.path_replace_movie(
-                    existing_movie['path'])))
-        return
-
-    movie_default_enabled = settings.general.getboolean('movie_default_enabled')
-
-    if movie_default_enabled is True:
-        movie_default_profile = settings.general.movie_default_profile
-        if movie_default_profile == '':
-            movie_default_profile = None
-    else:
-        movie_default_profile = None
-
-    audio_profiles = get_profile_list()
-    tagsDict = get_tags()
-
-    try:
-        # Get movie data from radarr api
-        movie = None
-        movie_data = get_movies_from_radarr_api(url=url_radarr(), apikey_radarr=settings.radarr.apikey,
-                                                radarr_id=movie_id)
-        if not movie_data:
-            return
-        else:
-            if action == 'updated' and existing_movie:
-                movie = movieParser(movie_data, action='update', tags_dict=tagsDict,
-                                    movie_default_profile=movie_default_profile, audio_profiles=audio_profiles)
-            elif action == 'updated' and not existing_movie:
-                movie = movieParser(movie_data, action='insert', tags_dict=tagsDict,
-                                    movie_default_profile=movie_default_profile, audio_profiles=audio_profiles)
-    except Exception:
-        logging.debug('BAZARR cannot get movie returned by SignalR feed from Radarr API.')
-        return
-
-    # Drop useless events
-    if not movie and not existing_movie:
-        return
-
-    # Remove movie from DB
-    if not movie and existing_movie:
-        try:
-            TableMovies.delete().where(TableMovies.radarrId == movie_id).execute()
-        except Exception as e:
-            logging.error(f"BAZARR cannot insert episode {existing_movie['path']} because of {e}")
-        else:
-            event_stream(type='movie', action='delete', payload=int(movie_id))
-            logging.debug('BAZARR deleted this movie from the database:{}'.format(path_mappings.path_replace_movie(
-                existing_movie['path'])))
-            return
-
-    # Update existing movie in DB
-    elif movie and existing_movie:
-        try:
-            TableMovies.update(movie).where(TableMovies.radarrId == movie['radarrId']).execute()
-        except IntegrityError as e:
-            logging.error(f"BAZARR cannot insert episode {movie['path']} because of {e}")
-        else:
-            event_stream(type='movie', action='update', payload=int(movie_id))
-            logging.debug('BAZARR updated this movie into the database:{}'.format(path_mappings.path_replace_movie(
-                movie['path'])))
-
-    # Insert new movie in DB
-    elif movie and not existing_movie:
-        try:
-            TableMovies.insert(movie).on_conflict(action='IGNORE').execute()
-        except IntegrityError as e:
-            logging.error(f"BAZARR cannot insert movie {movie['path']} because of {e}")
-        else:
-            event_stream(type='movie', action='update', payload=int(movie_id))
-            logging.debug('BAZARR inserted this movie into the database:{}'.format(path_mappings.path_replace_movie(
-                movie['path'])))
-
-    # Storing existing subtitles
-    logging.debug('BAZARR storing subtitles for this movie: {}'.format(path_mappings.path_replace_movie(
-            movie['path'])))
-    store_subtitles_movie(movie['path'], path_mappings.path_replace_movie(movie['path']))
-
-    # Downloading missing subtitles
-    if defer_search:
-        logging.debug('BAZARR searching for missing subtitles is deferred until scheduled task execution for this '
-                      'movie: {}'.format(path_mappings.path_replace_movie(movie['path'])))
-    else:
-        logging.debug('BAZARR downloading missing subtitles for this movie: {}'.format(path_mappings.path_replace_movie(
-            movie['path'])))
-        movies_download_subtitles(movie_id)
-
-
-def get_profile_list():
-    apikey_radarr = settings.radarr.apikey
-    profiles_list = []
-    # Get profiles data from radarr
-    if get_radarr_info.is_legacy():
-        url_radarr_api_movies = url_radarr() + "/api/profile?apikey=" + apikey_radarr
-    else:
-        url_radarr_api_movies = url_radarr() + "/api/v3/qualityprofile?apikey=" + apikey_radarr
-
-    try:
-        profiles_json = requests.get(url_radarr_api_movies, timeout=60, verify=False, headers=headers)
-    except requests.exceptions.ConnectionError:
-        logging.exception("BAZARR Error trying to get profiles from Radarr. Connection Error.")
-    except requests.exceptions.Timeout:
-        logging.exception("BAZARR Error trying to get profiles from Radarr. Timeout Error.")
-    except requests.exceptions.RequestException:
-        logging.exception("BAZARR Error trying to get profiles from Radarr.")
-    else:
-        # Parsing data returned from radarr
-        if get_radarr_info.is_legacy():
-            for profile in profiles_json.json():
-                profiles_list.append([profile['id'], profile['language'].capitalize()])
-        else:
-            for profile in profiles_json.json():
-                profiles_list.append([profile['id'], profile['language']['name'].capitalize()])
-
-        return profiles_list
-
-    return None
-
-
-def profile_id_to_language(id, profiles):
-    for profile in profiles:
-        profiles_to_return = []
-        if id == profile[0]:
-            profiles_to_return.append(profile[1])
-    return profiles_to_return
-
-
-def RadarrFormatAudioCodec(audioFormat, audioCodecID, audioProfile, audioAdditionalFeatures):
-    if type(audioFormat) is not str:
-        return audioFormat
-    else:
-        if audioFormat == "AC-3":
-            return "AC3"
-        elif audioFormat == "E-AC-3":
-            return "EAC3"
-        elif audioFormat == "AAC":
-            if audioCodecID == "A_AAC/MPEG4/LC/SBR":
-                return "HE-AAC"
-            else:
-                return "AAC"
-        elif audioFormat.strip() == "mp3":
-            return "MP3"
-        elif audioFormat == "MPEG Audio":
-            if audioCodecID == "55" or audioCodecID == "A_MPEG/L3" or audioProfile == "Layer 3":
-                return "MP3"
-            if audioCodecID == "A_MPEG/L2" or audioProfile == "Layer 2":
-                return "MP2"
-        elif audioFormat == "MLP FBA":
-            if audioAdditionalFeatures == "16-ch":
-                return "TrueHD Atmos"
-            else:
-                return "TrueHD"
-        else:
-            return audioFormat
-
-
-def RadarrFormatVideoCodec(videoFormat, videoCodecID, videoCodecLibrary):
-    if type(videoFormat) is not str:
-        return videoFormat
-    else:
-        if videoFormat == "x264":
-            return "h264"
-        elif videoFormat == "AVC" or videoFormat == "V.MPEG4/ISO/AVC":
-            return "h264"
-        elif videoCodecLibrary and (videoFormat == "HEVC" or videoFormat == "V_MPEGH/ISO/HEVC"):
-            if videoCodecLibrary.startswith("x265"):
-                return "h265"
-        elif videoCodecID and videoFormat == "MPEG Video":
-            if videoCodecID == "2" or videoCodecID == "V_MPEG2":
-                return "Mpeg2"
-            else:
-                return "Mpeg"
-        elif videoFormat == "MPEG-1 Video":
-            return "Mpeg"
-        elif videoFormat == "MPEG-2 Video":
-            return "Mpeg2"
-        elif videoCodecLibrary and videoCodecID and videoFormat == "MPEG-4 Visual":
-            if videoCodecID.endswith("XVID") or videoCodecLibrary.startswith("XviD"):
-                return "XviD"
-            if videoCodecID.endswith("DIV3") or videoCodecID.endswith("DIVX") or videoCodecID.endswith(
-                    "DX50") or videoCodecLibrary.startswith("DivX"):
-                return "DivX"
-        elif videoFormat == "VC-1":
-            return "VC1"
-        elif videoFormat == "WMV2":
-            return "WMV"
-        elif videoFormat == "DivX" or videoFormat == "div3":
-            return "DivX"
-        else:
-            return videoFormat
-
-
-def get_tags():
-    apikey_radarr = settings.radarr.apikey
-    tagsDict = []
-
-    # Get tags data from Radarr
-    if get_radarr_info.is_legacy():
-        url_radarr_api_series = url_radarr() + "/api/tag?apikey=" + apikey_radarr
-    else:
-        url_radarr_api_series = url_radarr() + "/api/v3/tag?apikey=" + apikey_radarr
-
-    try:
-        tagsDict = requests.get(url_radarr_api_series, timeout=60, verify=False, headers=headers)
-    except requests.exceptions.ConnectionError:
-        logging.exception("BAZARR Error trying to get tags from Radarr. Connection Error.")
-        return []
-    except requests.exceptions.Timeout:
-        logging.exception("BAZARR Error trying to get tags from Radarr. Timeout Error.")
-        return []
-    except requests.exceptions.RequestException:
-        logging.exception("BAZARR Error trying to get tags from Radarr.")
-        return []
-    except requests.exceptions.HTTPError:
-        logging.exception("BAZARR Exception while trying to get tags from Radarr.")
-        return []
-    else:
-        try:
-            return tagsDict.json()
-        except Exception:
-            return []
-
-
-def movieParser(movie, action, tags_dict, movie_default_profile, audio_profiles):
-    if 'movieFile' in movie:
-        # Detect file separator
-        if movie['path'][0] == "/":
-            separator = "/"
-        else:
-            separator = "\\"
-
-        try:
-            overview = str(movie['overview'])
-        except Exception:
-            overview = ""
-        try:
-            poster_big = movie['images'][0]['url']
-            poster = os.path.splitext(poster_big)[0] + '-500' + os.path.splitext(poster_big)[1]
-        except Exception:
-            poster = ""
-        try:
-            fanart = movie['images'][1]['url']
-        except Exception:
-            fanart = ""
-
-        if 'sceneName' in movie['movieFile']:
-            sceneName = movie['movieFile']['sceneName']
-        else:
-            sceneName = None
-
-        alternativeTitles = None
-        if get_radarr_info.is_legacy():
-            if 'alternativeTitles' in movie:
-                alternativeTitles = str([item['title'] for item in movie['alternativeTitles']])
-        else:
-            if 'alternateTitles' in movie:
-                alternativeTitles = str([item['title'] for item in movie['alternateTitles']])
-
-        if 'imdbId' in movie:
-            imdbId = movie['imdbId']
-        else:
-            imdbId = None
-
-        try:
-            format, resolution = movie['movieFile']['quality']['quality']['name'].split('-')
-        except Exception:
-            format = movie['movieFile']['quality']['quality']['name']
-            try:
-                resolution = str(movie['movieFile']['quality']['quality']['resolution']) + 'p'
-            except Exception:
-                resolution = None
-
-        if 'mediaInfo' in movie['movieFile']:
-            videoFormat = videoCodecID = videoCodecLibrary = None
-            if get_radarr_info.is_legacy():
-                if 'videoFormat' in movie['movieFile']['mediaInfo']:
-                    videoFormat = movie['movieFile']['mediaInfo']['videoFormat']
-            else:
-                if 'videoCodec' in movie['movieFile']['mediaInfo']:
-                    videoFormat = movie['movieFile']['mediaInfo']['videoCodec']
-            if 'videoCodecID' in movie['movieFile']['mediaInfo']:
-                videoCodecID = movie['movieFile']['mediaInfo']['videoCodecID']
-            if 'videoCodecLibrary' in movie['movieFile']['mediaInfo']:
-                videoCodecLibrary = movie['movieFile']['mediaInfo']['videoCodecLibrary']
-            videoCodec = RadarrFormatVideoCodec(videoFormat, videoCodecID, videoCodecLibrary)
-
-            audioFormat = audioCodecID = audioProfile = audioAdditionalFeatures = None
-            if get_radarr_info.is_legacy():
-                if 'audioFormat' in movie['movieFile']['mediaInfo']:
-                    audioFormat = movie['movieFile']['mediaInfo']['audioFormat']
-            else:
-                if 'audioCodec' in movie['movieFile']['mediaInfo']:
-                    audioFormat = movie['movieFile']['mediaInfo']['audioCodec']
-            if 'audioCodecID' in movie['movieFile']['mediaInfo']:
-                audioCodecID = movie['movieFile']['mediaInfo']['audioCodecID']
-            if 'audioProfile' in movie['movieFile']['mediaInfo']:
-                audioProfile = movie['movieFile']['mediaInfo']['audioProfile']
-            if 'audioAdditionalFeatures' in movie['movieFile']['mediaInfo']:
-                audioAdditionalFeatures = movie['movieFile']['mediaInfo']['audioAdditionalFeatures']
-            audioCodec = RadarrFormatAudioCodec(audioFormat, audioCodecID, audioProfile, audioAdditionalFeatures)
-        else:
-            videoCodec = None
-            audioCodec = None
-
-        audio_language = []
-        if get_radarr_info.is_legacy():
-            if 'mediaInfo' in movie['movieFile']:
-                if 'audioLanguages' in movie['movieFile']['mediaInfo']:
-                    audio_languages_list = movie['movieFile']['mediaInfo']['audioLanguages'].split('/')
-                    if len(audio_languages_list):
-                        for audio_language_list in audio_languages_list:
-                            audio_language.append(audio_language_list.strip())
-            if not audio_language:
-                audio_language = profile_id_to_language(movie['qualityProfileId'], audio_profiles)
-        else:
-            if 'languages' in movie['movieFile'] and len(movie['movieFile']['languages']):
-                for item in movie['movieFile']['languages']:
-                    if isinstance(item, dict):
-                        if 'name' in item:
-                            language = item['name']
-                            if item['name'] == 'Portuguese (Brazil)':
-                                language = language_from_alpha2('pb')
-                            audio_language.append(language)
-
-        tags = [d['label'] for d in tags_dict if d['id'] in movie['tags']]
-
-        if action == 'update':
-            return {'radarrId': int(movie["id"]),
-                    'title': movie["title"],
-                    'path': movie["path"] + separator + movie['movieFile']['relativePath'],
-                    'tmdbId': str(movie["tmdbId"]),
-                    'poster': poster,
-                    'fanart': fanart,
-                    'audio_language': str(audio_language),
-                    'sceneName': sceneName,
-                    'monitored': str(bool(movie['monitored'])),
-                    'year': str(movie['year']),
-                    'sortTitle': movie['sortTitle'],
-                    'alternativeTitles': alternativeTitles,
-                    'format': format,
-                    'resolution': resolution,
-                    'video_codec': videoCodec,
-                    'audio_codec': audioCodec,
-                    'overview': overview,
-                    'imdbId': imdbId,
-                    'movie_file_id': int(movie['movieFile']['id']),
-                    'tags': str(tags),
-                    'file_size': movie['movieFile']['size']}
-        else:
-            return {'radarrId': int(movie["id"]),
-                    'title': movie["title"],
-                    'path': movie["path"] + separator + movie['movieFile']['relativePath'],
-                    'tmdbId': str(movie["tmdbId"]),
-                    'subtitles': '[]',
-                    'overview': overview,
-                    'poster': poster,
-                    'fanart': fanart,
-                    'audio_language': str(audio_language),
-                    'sceneName': sceneName,
-                    'monitored': str(bool(movie['monitored'])),
-                    'sortTitle': movie['sortTitle'],
-                    'year': str(movie['year']),
-                    'alternativeTitles': alternativeTitles,
-                    'format': format,
-                    'resolution': resolution,
-                    'video_codec': videoCodec,
-                    'audio_codec': audioCodec,
-                    'imdbId': imdbId,
-                    'movie_file_id': int(movie['movieFile']['id']),
-                    'tags': str(tags),
-                    'profileId': movie_default_profile,
-                    'file_size': movie['movieFile']['size']}
-
-
-def get_movies_from_radarr_api(url, apikey_radarr, radarr_id=None):
-    if get_radarr_info.is_legacy():
-        url_radarr_api_movies = url + "/api/movie" + ("/{}".format(radarr_id) if radarr_id else "") + "?apikey=" + \
-                                apikey_radarr
-    else:
-        url_radarr_api_movies = url + "/api/v3/movie" + ("/{}".format(radarr_id) if radarr_id else "") + "?apikey=" + \
-                                apikey_radarr
-
-    try:
-        r = requests.get(url_radarr_api_movies, timeout=60, verify=False, headers=headers)
-        if r.status_code == 404:
-            return
-        r.raise_for_status()
-    except requests.exceptions.HTTPError:
-        logging.exception("BAZARR Error trying to get movies from Radarr. Http error.")
-        return
-    except requests.exceptions.ConnectionError:
-        logging.exception("BAZARR Error trying to get movies from Radarr. Connection Error.")
-        return
-    except requests.exceptions.Timeout:
-        logging.exception("BAZARR Error trying to get movies from Radarr. Timeout Error.")
-        return
-    except requests.exceptions.RequestException:
-        logging.exception("BAZARR Error trying to get movies from Radarr.")
-        return
-    else:
-        return r.json()

bazarr/get_rootfolder.py

@@ -1,159 +0,0 @@
-# coding=utf-8
-
-import os
-import requests
-import logging
-
-from config import settings, url_sonarr, url_radarr
-from helper import path_mappings
-from database import TableShowsRootfolder, TableMoviesRootfolder, TableShows, TableMovies
-from utils import get_sonarr_info, get_radarr_info
-
-headers = {"User-Agent": os.environ["SZ_USER_AGENT"]}
-
-
-def get_sonarr_rootfolder():
-    apikey_sonarr = settings.sonarr.apikey
-    sonarr_rootfolder = []
-
-    # Get root folder data from Sonarr
-    if get_sonarr_info.is_legacy():
-        url_sonarr_api_rootfolder = url_sonarr() + "/api/rootfolder?apikey=" + apikey_sonarr
-    else:
-        url_sonarr_api_rootfolder = url_sonarr() + "/api/v3/rootfolder?apikey=" + apikey_sonarr
-
-    try:
-        rootfolder = requests.get(url_sonarr_api_rootfolder, timeout=60, verify=False, headers=headers)
-    except requests.exceptions.ConnectionError:
-        logging.exception("BAZARR Error trying to get rootfolder from Sonarr. Connection Error.")
-        return []
-    except requests.exceptions.Timeout:
-        logging.exception("BAZARR Error trying to get rootfolder from Sonarr. Timeout Error.")
-        return []
-    except requests.exceptions.RequestException:
-        logging.exception("BAZARR Error trying to get rootfolder from Sonarr.")
-        return []
-    else:
-        sonarr_movies_paths = list(TableShows.select(TableShows.path).dicts())
-        for folder in rootfolder.json():
-            if any(item['path'].startswith(folder['path']) for item in sonarr_movies_paths):
-                sonarr_rootfolder.append({'id': folder['id'], 'path': folder['path']})
-        db_rootfolder = TableShowsRootfolder.select(TableShowsRootfolder.id, TableShowsRootfolder.path).dicts()
-        rootfolder_to_remove = [x for x in db_rootfolder if not
-                                next((item for item in sonarr_rootfolder if item['id'] == x['id']), False)]
-        rootfolder_to_update = [x for x in sonarr_rootfolder if
-                                next((item for item in db_rootfolder if item['id'] == x['id']), False)]
-        rootfolder_to_insert = [x for x in sonarr_rootfolder if not
-                                next((item for item in db_rootfolder if item['id'] == x['id']), False)]
-
-        for item in rootfolder_to_remove:
-            TableShowsRootfolder.delete().where(TableShowsRootfolder.id == item['id']).execute()
-        for item in rootfolder_to_update:
-            TableShowsRootfolder.update({TableShowsRootfolder.path: item['path']})\
-                .where(TableShowsRootfolder.id == item['id'])\
-                .execute()
-        for item in rootfolder_to_insert:
-            TableShowsRootfolder.insert({TableShowsRootfolder.id: item['id'], TableShowsRootfolder.path: item['path']})\
-                .execute()
-
-
-def check_sonarr_rootfolder():
-    get_sonarr_rootfolder()
-    rootfolder = TableShowsRootfolder.select(TableShowsRootfolder.id, TableShowsRootfolder.path).dicts()
-    for item in rootfolder:
-        root_path = item['path']
-        if not root_path.endswith(('/', '\\')):
-            if root_path.startswith('/'):
-                root_path += '/'
-            else:
-                root_path += '\\'
-        if not os.path.isdir(path_mappings.path_replace(root_path)):
-            TableShowsRootfolder.update({TableShowsRootfolder.accessible: 0,
-                                         TableShowsRootfolder.error: 'This Sonarr root directory does not seems to '
-                                                                     'be accessible by Bazarr. Please check path '
-                                                                     'mapping.'})\
-                .where(TableShowsRootfolder.id == item['id'])\
-                .execute()
-        elif not os.access(path_mappings.path_replace(root_path), os.W_OK):
-            TableShowsRootfolder.update({TableShowsRootfolder.accessible: 0,
-                                         TableShowsRootfolder.error: 'Bazarr cannot write to this directory.'}) \
-                .where(TableShowsRootfolder.id == item['id']) \
-                .execute()
-        else:
-            TableShowsRootfolder.update({TableShowsRootfolder.accessible: 1,
-                                         TableShowsRootfolder.error: ''}) \
-                .where(TableShowsRootfolder.id == item['id']) \
-                .execute()
-
-
-def get_radarr_rootfolder():
-    apikey_radarr = settings.radarr.apikey
-    radarr_rootfolder = []
-
-    # Get root folder data from Radarr
-    if get_radarr_info.is_legacy():
-        url_radarr_api_rootfolder = url_radarr() + "/api/rootfolder?apikey=" + apikey_radarr
-    else:
-        url_radarr_api_rootfolder = url_radarr() + "/api/v3/rootfolder?apikey=" + apikey_radarr
-
-    try:
-        rootfolder = requests.get(url_radarr_api_rootfolder, timeout=60, verify=False, headers=headers)
-    except requests.exceptions.ConnectionError:
-        logging.exception("BAZARR Error trying to get rootfolder from Radarr. Connection Error.")
-        return []
-    except requests.exceptions.Timeout:
-        logging.exception("BAZARR Error trying to get rootfolder from Radarr. Timeout Error.")
-        return []
-    except requests.exceptions.RequestException:
-        logging.exception("BAZARR Error trying to get rootfolder from Radarr.")
-        return []
-    else:
-        radarr_movies_paths = list(TableMovies.select(TableMovies.path).dicts())
-        for folder in rootfolder.json():
-            if any(item['path'].startswith(folder['path']) for item in radarr_movies_paths):
-                radarr_rootfolder.append({'id': folder['id'], 'path': folder['path']})
-        db_rootfolder = TableMoviesRootfolder.select(TableMoviesRootfolder.id, TableMoviesRootfolder.path).dicts()
-        rootfolder_to_remove = [x for x in db_rootfolder if not
-                                next((item for item in radarr_rootfolder if item['id'] == x['id']), False)]
-        rootfolder_to_update = [x for x in radarr_rootfolder if
-                                next((item for item in db_rootfolder if item['id'] == x['id']), False)]
-        rootfolder_to_insert = [x for x in radarr_rootfolder if not
-                                next((item for item in db_rootfolder if item['id'] == x['id']), False)]
-
-        for item in rootfolder_to_remove:
-            TableMoviesRootfolder.delete().where(TableMoviesRootfolder.id == item['id']).execute()
-        for item in rootfolder_to_update:
-            TableMoviesRootfolder.update({TableMoviesRootfolder.path: item['path']})\
-                .where(TableMoviesRootfolder.id == item['id']).execute()
-        for item in rootfolder_to_insert:
-            TableMoviesRootfolder.insert({TableMoviesRootfolder.id: item['id'],
-                                          TableMoviesRootfolder.path: item['path']}).execute()
-
-
-def check_radarr_rootfolder():
-    get_radarr_rootfolder()
-    rootfolder = TableMoviesRootfolder.select(TableMoviesRootfolder.id, TableMoviesRootfolder.path).dicts()
-    for item in rootfolder:
-        root_path = item['path']
-        if not root_path.endswith(('/', '\\')):
-            if root_path.startswith('/'):
-                root_path += '/'
-            else:
-                root_path += '\\'
-        if not os.path.isdir(path_mappings.path_replace_movie(root_path)):
-            TableMoviesRootfolder.update({TableMoviesRootfolder.accessible: 0,
-                                         TableMoviesRootfolder.error: 'This Radarr root directory does not seems to '
-                                                                      'be accessible by Bazarr. Please check path '
-                                                                      'mapping.'}) \
-                .where(TableMoviesRootfolder.id == item['id']) \
-                .execute()
-        elif not os.access(path_mappings.path_replace_movie(root_path), os.W_OK):
-            TableMoviesRootfolder.update({TableMoviesRootfolder.accessible: 0,
-                                          TableMoviesRootfolder.error: 'Bazarr cannot write to this directory'}) \
-                .where(TableMoviesRootfolder.id == item['id']) \
-                .execute()
-        else:
-            TableMoviesRootfolder.update({TableMoviesRootfolder.accessible: 1,
-                                          TableMoviesRootfolder.error: ''}) \
-                .where(TableMoviesRootfolder.id == item['id']) \
-                .execute()

bazarr/helper.py

@@ -1,173 +0,0 @@
-# coding=utf-8
-
-import os
-import re
-import logging
-
-from charamel import Detector
-from bs4 import UnicodeDammit
-
-from config import settings, get_array_from
-
-
-class PathMappings:
-    def __init__(self):
-        self.path_mapping_series = []
-        self.path_mapping_movies = []
-
-    def update(self):
-        self.path_mapping_series = [x for x in get_array_from(settings.general.path_mappings) if x[0] != x[1]]
-        self.path_mapping_movies = [x for x in get_array_from(settings.general.path_mappings_movie) if x[0] != x[1]]
-
-    def path_replace(self, path):
-        if path is None:
-            return None
-
-        for path_mapping in self.path_mapping_series:
-            if path_mapping[0] == path_mapping[1]:
-                continue
-            if '' in path_mapping:
-                continue
-            if path_mapping[0] in path:
-                path = path.replace(path_mapping[0], path_mapping[1])
-                if path.startswith('\\\\') or re.match(r'^[a-zA-Z]:\\', path):
-                    path = path.replace('/', '\\')
-                elif path.startswith('/'):
-                    path = path.replace('\\', '/')
-                break
-        return path
-
-    def path_replace_reverse(self, path):
-        if path is None:
-            return None
-
-        for path_mapping in self.path_mapping_series:
-            if path_mapping[0] == path_mapping[1]:
-                continue
-            if '' in path_mapping:
-                continue
-            if path_mapping[1] in path:
-                path = path.replace(path_mapping[1], path_mapping[0])
-                if path.startswith('\\\\') or re.match(r'^[a-zA-Z]:\\', path):
-                    path = path.replace('/', '\\')
-                elif path.startswith('/'):
-                    path = path.replace('\\', '/')
-                break
-        return path
-
-    def path_replace_movie(self, path):
-        if path is None:
-            return None
-
-        for path_mapping in self.path_mapping_movies:
-            if path_mapping[0] == path_mapping[1]:
-                continue
-            if '' in path_mapping:
-                continue
-            if path_mapping[0] in path:
-                path = path.replace(path_mapping[0], path_mapping[1])
-                if path.startswith('\\\\') or re.match(r'^[a-zA-Z]:\\', path):
-                    path = path.replace('/', '\\')
-                elif path.startswith('/'):
-                    path = path.replace('\\', '/')
-                break
-        return path
-
-    def path_replace_reverse_movie(self, path):
-        if path is None:
-            return None
-
-        for path_mapping in self.path_mapping_movies:
-            if path_mapping[0] == path_mapping[1]:
-                continue
-            if '' in path_mapping:
-                continue
-            if path_mapping[1] in path:
-                path = path.replace(path_mapping[1], path_mapping[0])
-                if path.startswith('\\\\') or re.match(r'^[a-zA-Z]:\\', path):
-                    path = path.replace('/', '\\')
-                elif path.startswith('/'):
-                    path = path.replace('\\', '/')
-                break
-        return path
-
-
-path_mappings = PathMappings()
-
-
-def pp_replace(pp_command, episode, subtitles, language, language_code2, language_code3, episode_language, episode_language_code2, episode_language_code3, forced, score, subtitle_id, provider, series_id, episode_id, hi):
-    pp_command = pp_command.replace('{{directory}}', os.path.dirname(episode))
-    pp_command = pp_command.replace('{{episode}}', episode)
-    pp_command = pp_command.replace('{{episode_name}}', os.path.splitext(os.path.basename(episode))[0])
-    pp_command = pp_command.replace('{{subtitles}}', str(subtitles))
-    pp_command = pp_command.replace('{{subtitles_language}}', str(language))
-    pp_command = pp_command.replace('{{subtitles_language_code2}}', str(language_code2))
-    pp_command = pp_command.replace('{{subtitles_language_code3}}', str(language_code3))
-    pp_command = pp_command.replace('{{subtitles_language_code2_dot}}', str(language_code2).replace(':', '.'))
-    pp_command = pp_command.replace('{{subtitles_language_code3_dot}}', str(language_code3).replace(':', '.'))
-    pp_command = pp_command.replace('{{episode_language}}', str(episode_language))
-    pp_command = pp_command.replace('{{episode_language_code2}}', str(episode_language_code2))
-    pp_command = pp_command.replace('{{episode_language_code3}}', str(episode_language_code3))
-    pp_command = pp_command.replace('{{score}}', str(score))
-    pp_command = pp_command.replace('{{subtitle_id}}', str(subtitle_id))
-    pp_command = pp_command.replace('{{provider}}', str(provider))
-    pp_command = pp_command.replace('{{series_id}}', str(series_id))
-    pp_command = pp_command.replace('{{episode_id}}', str(episode_id))
-    return pp_command
-
-
-def get_subtitle_destination_folder():
-    fld_custom = str(settings.general.subfolder_custom).strip() if (settings.general.subfolder_custom and
-                                                                    settings.general.subfolder != 'current') else None
-    return fld_custom
-
-
-def get_target_folder(file_path):
-    subfolder = settings.general.subfolder
-    fld_custom = str(settings.general.subfolder_custom).strip() \
-        if settings.general.subfolder_custom else None
-
-    if subfolder != "current" and fld_custom:
-        # specific subFolder requested, create it if it doesn't exist
-        fld_base = os.path.split(file_path)[0]
-
-        if subfolder == "absolute":
-            # absolute folder
-            fld = fld_custom
-        elif subfolder == "relative":
-            fld = os.path.join(fld_base, fld_custom)
-        else:
-            fld = None
-
-        fld = force_unicode(fld)
-
-        if not os.path.isdir(fld):
-            try:
-                os.makedirs(fld)
-            except Exception:
-                logging.error('BAZARR is unable to create directory to save subtitles: ' + fld)
-                fld = None
-    else:
-        fld = None
-
-    return fld
-
-
-def force_unicode(s):
-    """
-    Ensure a string is unicode, not encoded; used for enforcing file paths to be unicode upon saving a subtitle,
-    to prevent encoding issues when saving a subtitle to a non-ascii path.
-    :param s: string
-    :return: unicode string
-    """
-    if not isinstance(s, str):
-        try:
-            s = s.decode("utf-8")
-        except UnicodeDecodeError:
-            detector = Detector()
-            t = detector.detect(s)
-            try:
-                s = s.decode(t)
-            except UnicodeDecodeError:
-                s = UnicodeDammit(s).unicode_markup
-    return s

bazarr/init.py

@@ -2,20 +2,21 @@
 
 import os
 import io
-import rarfile
 import sys
 import subprocess
-
-from config import settings, configure_captcha_func
-from get_args import args
-from logger import configure_logging
-from helper import path_mappings
-from backup import restore_from_backup
-
-from dogpile.cache.region import register_backend as register_cache_backend
 import subliminal
 import datetime
 import time
+import rarfile
+
+from dogpile.cache.region import register_backend as register_cache_backend
+
+from app.config import settings, configure_captcha_func
+from app.get_args import args
+from app.logger import configure_logging
+from utilities.binaries import get_binary, BinaryNotFound
+from utilities.path_mappings import path_mappings
+from utilities.backup import restore_from_backup
 
 # set start time global variable as epoch
 global startTime
@@ -27,12 +28,6 @@ restore_from_backup()
 # set subliminal_patch user agent
 os.environ["SZ_USER_AGENT"] = "Bazarr/{}".format(os.environ["BAZARR_VERSION"])
 
-# set subliminal_patch hearing-impaired extension to use when naming subtitles
-os.environ["SZ_HI_EXTENSION"] = settings.general.hi_extension
-
-# set anti-captcha provider and key
-configure_captcha_func()
-
 # Check if args.config_dir exist
 if not os.path.exists(args.config_dir):
     # Create config_dir directory tree
@@ -50,9 +45,18 @@ if not os.path.exists(os.path.join(args.config_dir, 'log')):
     os.mkdir(os.path.join(args.config_dir, 'log'))
 if not os.path.exists(os.path.join(args.config_dir, 'cache')):
     os.mkdir(os.path.join(args.config_dir, 'cache'))
+if not os.path.exists(os.path.join(args.config_dir, 'backup')):
+    os.mkdir(os.path.join(args.config_dir, 'backup'))
 if not os.path.exists(os.path.join(args.config_dir, 'restore')):
     os.mkdir(os.path.join(args.config_dir, 'restore'))
 
+# set subliminal_patch hearing-impaired extension to use when naming subtitles
+os.environ["SZ_HI_EXTENSION"] = settings.general.hi_extension
+
+# set anti-captcha provider and key
+configure_captcha_func()
+
+# configure logging
 configure_logging(settings.general.getboolean('debug') or args.debug)
 import logging  # noqa E402
 
@@ -95,7 +99,7 @@ if not args.no_update:
                 try:
                     restart_file = io.open(os.path.join(args.config_dir, "bazarr.restart"), "w", encoding='UTF-8')
                 except Exception as e:
-                    logging.error('BAZARR Cannot create bazarr.restart file: ' + repr(e))
+                    logging.error('BAZARR Cannot create restart file: ' + repr(e))
                 else:
                     logging.info('Bazarr is being restarted...')
                     restart_file.write(str(''))
@@ -163,11 +167,12 @@ if os.path.isfile(package_info_file):
 # Configure dogpile file caching for Subliminal request
 register_cache_backend("subzero.cache.file", "subzero.cache_backends.file", "SZFileBackend")
 subliminal.region.configure('subzero.cache.file', expiration_time=datetime.timedelta(days=30),
-                            arguments={'appname': "sz_cache", 'app_cache_dir': args.config_dir})
+                            arguments={'appname': "sz_cache", 'app_cache_dir': args.config_dir},
+                            replace_existing_backend=True)
 subliminal.region.backend.sync()
 
 if not os.path.exists(os.path.join(args.config_dir, 'config', 'releases.txt')):
-    from check_update import check_releases
+    from app.check_update import check_releases
     check_releases()
     logging.debug("BAZARR Created releases file")
 
@@ -189,23 +194,30 @@ with open(os.path.normpath(os.path.join(args.config_dir, 'config', 'config.ini')
 
 
 def init_binaries():
-    from utils import get_binary
-    exe = get_binary("unrar")
-
-    rarfile.UNRAR_TOOL = exe
-    rarfile.ORIG_UNRAR_TOOL = exe
     try:
-        rarfile.custom_check([rarfile.UNRAR_TOOL], True)
-    except Exception:
-        logging.debug("custom check failed for: %s", exe)
-
-    logging.debug("Using UnRAR from: %s", exe)
-    unrar = exe
-
-    return unrar
+        exe = get_binary("unar")
+        rarfile.UNAR_TOOL = exe
+        rarfile.UNRAR_TOOL = None
+        rarfile.tool_setup(unrar=False, unar=True, bsdtar=False, force=True)
+    except (BinaryNotFound, rarfile.RarCannotExec):
+        try:
+            exe = get_binary("unrar")
+            rarfile.UNRAR_TOOL = exe
+            rarfile.UNAR_TOOL = None
+            rarfile.tool_setup(unrar=True, unar=False, bsdtar=False, force=True)
+        except (BinaryNotFound, rarfile.RarCannotExec):
+            logging.exception("BAZARR requires a rar archive extraction utilities (unrar, unar) and it can't be found.")
+            raise BinaryNotFound
+        else:
+            logging.debug("Using UnRAR from: %s", exe)
+            return exe
+    else:
+        logging.debug("Using unar from: %s", exe)
+        return exe
 
 
-from database import init_db, migrate_db  # noqa E402
+# keep this import at the end to prevent peewee.OperationalError: unable to open database file
+from app.database import init_db, migrate_db  # noqa E402
 init_db()
 migrate_db()
 init_binaries()

bazarr/languages/__init__.py

@@ -0,0 +1 @@
+# coding=utf-8

bazarr/languages/get_languages.py

@@ -3,8 +3,9 @@
 import pycountry
 
 from subzero.language import Language
-from custom_lang import CustomLanguage
-from database import TableSettingsLanguages
+
+from .custom_lang import CustomLanguage
+from app.database import TableSettingsLanguages
 
 
 def load_language_in_db():
@@ -91,7 +92,3 @@ def get_language_set():
             language_set.add(custom.subzero_language())
 
     return language_set
-
-
-if __name__ == '__main__':
-    load_language_in_db()

bazarr/list_subtitles.py

@@ -1,649 +0,0 @@
-# coding=utf-8
-
-import gc
-import os
-import logging
-import ast
-import re
-from guess_language import guess_language
-from subliminal_patch import core, search_external_subtitles
-from subzero.language import Language
-
-from custom_lang import CustomLanguage
-from database import get_profiles_list, get_profile_cutoff, TableEpisodes, TableShows, TableMovies
-from get_languages import alpha2_from_alpha3, language_from_alpha2, get_language_set
-from config import settings
-from helper import path_mappings, get_subtitle_destination_folder
-
-from embedded_subs_reader import embedded_subs_reader
-from event_handler import event_stream, show_progress, hide_progress
-from charamel import Detector
-
-gc.enable()
-
-global hi_regex
-hi_regex = re.compile(r'[*¶♫♪].{3,}[*¶♫♪]|[\[\(\{].{3,}[\]\)\}](?<!{\\an\d})')
-
-
-def store_subtitles(original_path, reversed_path, use_cache=True):
-    logging.debug('BAZARR started subtitles indexing for this file: ' + reversed_path)
-    actual_subtitles = []
-    if os.path.exists(reversed_path):
-        if settings.general.getboolean('use_embedded_subs'):
-            logging.debug("BAZARR is trying to index embedded subtitles.")
-            item = TableEpisodes.select(TableEpisodes.episode_file_id, TableEpisodes.file_size)\
-                .where(TableEpisodes.path == original_path)\
-                .dicts()\
-                .get_or_none()
-            if not item:
-                logging.exception(f"BAZARR error when trying to select this episode from database: {reversed_path}")
-            else:
-                try:
-                    subtitle_languages = embedded_subs_reader(reversed_path,
-                                                              file_size=item['file_size'],
-                                                              episode_file_id=item['episode_file_id'],
-                                                              use_cache=use_cache)
-                    for subtitle_language, subtitle_forced, subtitle_hi, subtitle_codec in subtitle_languages:
-                        try:
-                            if (settings.general.getboolean("ignore_pgs_subs") and subtitle_codec.lower() == "pgs") or \
-                                    (settings.general.getboolean("ignore_vobsub_subs") and subtitle_codec.lower() ==
-                                     "vobsub") or \
-                                    (settings.general.getboolean("ignore_ass_subs") and subtitle_codec.lower() ==
-                                     "ass"):
-                                logging.debug("BAZARR skipping %s sub for language: %s" % (subtitle_codec, alpha2_from_alpha3(subtitle_language)))
-                                continue
-
-                            if alpha2_from_alpha3(subtitle_language) is not None:
-                                lang = str(alpha2_from_alpha3(subtitle_language))
-                                if subtitle_forced:
-                                    lang = lang + ":forced"
-                                if subtitle_hi:
-                                    lang = lang + ":hi"
-                                logging.debug("BAZARR embedded subtitles detected: " + lang)
-                                actual_subtitles.append([lang, None])
-                        except Exception as error:
-                            logging.debug("BAZARR unable to index this unrecognized language: %s (%s)", subtitle_language, error)
-                except Exception:
-                    logging.exception(
-                        "BAZARR error when trying to analyze this %s file: %s" % (os.path.splitext(reversed_path)[1],
-                                                                                  reversed_path))
-                    pass
-        try:
-            dest_folder = get_subtitle_destination_folder()
-            core.CUSTOM_PATHS = [dest_folder] if dest_folder else []
-            subtitles = search_external_subtitles(reversed_path, languages=get_language_set(),
-                                                  only_one=settings.general.getboolean('single_language'))
-            full_dest_folder_path = os.path.dirname(reversed_path)
-            if dest_folder:
-                if settings.general.subfolder == "absolute":
-                    full_dest_folder_path = dest_folder
-                elif settings.general.subfolder == "relative":
-                    full_dest_folder_path = os.path.join(os.path.dirname(reversed_path), dest_folder)
-            subtitles = guess_external_subtitles(full_dest_folder_path, subtitles)
-        except Exception:
-            logging.exception("BAZARR unable to index external subtitles.")
-        else:
-            for subtitle, language in subtitles.items():
-                valid_language = False
-                if language:
-                    if hasattr(language, 'alpha3'):
-                        valid_language = alpha2_from_alpha3(language.alpha3)
-                else:
-                    logging.debug(f"Skipping subtitles because we are unable to define language: {subtitle}")
-                    continue
-
-                if not valid_language:
-                    logging.debug(f'{language.alpha3} is an unsupported language code.')
-                    continue
-
-                subtitle_path = get_external_subtitles_path(reversed_path, subtitle)
-
-                custom = CustomLanguage.found_external(subtitle, subtitle_path)
-                if custom is not None:
-                    actual_subtitles.append([custom, path_mappings.path_replace_reverse(subtitle_path)])
-
-                elif str(language) != 'und':
-                    if language.forced:
-                        language_str = str(language)
-                    elif language.hi:
-                        language_str = str(language) + ':hi'
-                    else:
-                        language_str = str(language)
-                    logging.debug("BAZARR external subtitles detected: " + language_str)
-                    actual_subtitles.append([language_str, path_mappings.path_replace_reverse(subtitle_path)])
-
-        TableEpisodes.update({TableEpisodes.subtitles: str(actual_subtitles)})\
-            .where(TableEpisodes.path == original_path)\
-            .execute()
-        matching_episodes = TableEpisodes.select(TableEpisodes.sonarrEpisodeId, TableEpisodes.sonarrSeriesId)\
-            .where(TableEpisodes.path == original_path)\
-            .dicts()
-
-        for episode in matching_episodes:
-            if episode:
-                logging.debug("BAZARR storing those languages to DB: " + str(actual_subtitles))
-                list_missing_subtitles(epno=episode['sonarrEpisodeId'])
-            else:
-                logging.debug("BAZARR haven't been able to update existing subtitles to DB : " + str(actual_subtitles))
-    else:
-        logging.debug("BAZARR this file doesn't seems to exist or isn't accessible.")
-
-    logging.debug('BAZARR ended subtitles indexing for this file: ' + reversed_path)
-
-    return actual_subtitles
-
-
-def store_subtitles_movie(original_path, reversed_path, use_cache=True):
-    logging.debug('BAZARR started subtitles indexing for this file: ' + reversed_path)
-    actual_subtitles = []
-    if os.path.exists(reversed_path):
-        if settings.general.getboolean('use_embedded_subs'):
-            logging.debug("BAZARR is trying to index embedded subtitles.")
-            item = TableMovies.select(TableMovies.movie_file_id, TableMovies.file_size)\
-                .where(TableMovies.path == original_path)\
-                .dicts()\
-                .get_or_none()
-            if not item:
-                logging.exception(f"BAZARR error when trying to select this movie from database: {reversed_path}")
-            else:
-                try:
-                    subtitle_languages = embedded_subs_reader(reversed_path,
-                                                              file_size=item['file_size'],
-                                                              movie_file_id=item['movie_file_id'],
-                                                              use_cache=use_cache)
-                    for subtitle_language, subtitle_forced, subtitle_hi, subtitle_codec in subtitle_languages:
-                        try:
-                            if (settings.general.getboolean("ignore_pgs_subs") and subtitle_codec.lower() == "pgs") or \
-                                    (settings.general.getboolean("ignore_vobsub_subs") and subtitle_codec.lower() ==
-                                     "vobsub") or \
-                                    (settings.general.getboolean("ignore_ass_subs") and subtitle_codec.lower() ==
-                                     "ass"):
-                                logging.debug("BAZARR skipping %s sub for language: %s" % (subtitle_codec, alpha2_from_alpha3(subtitle_language)))
-                                continue
-
-                            if alpha2_from_alpha3(subtitle_language) is not None:
-                                lang = str(alpha2_from_alpha3(subtitle_language))
-                                if subtitle_forced:
-                                    lang = lang + ':forced'
-                                if subtitle_hi:
-                                    lang = lang + ':hi'
-                                logging.debug("BAZARR embedded subtitles detected: " + lang)
-                                actual_subtitles.append([lang, None])
-                        except Exception:
-                            logging.debug("BAZARR unable to index this unrecognized language: " + subtitle_language)
-                            pass
-                except Exception:
-                    logging.exception(
-                        "BAZARR error when trying to analyze this %s file: %s" % (os.path.splitext(reversed_path)[1],
-                                                                                  reversed_path))
-                    pass
-
-        try:
-            dest_folder = get_subtitle_destination_folder() or ''
-            core.CUSTOM_PATHS = [dest_folder] if dest_folder else []
-            subtitles = search_external_subtitles(reversed_path, languages=get_language_set())
-            full_dest_folder_path = os.path.dirname(reversed_path)
-            if dest_folder:
-                if settings.general.subfolder == "absolute":
-                    full_dest_folder_path = dest_folder
-                elif settings.general.subfolder == "relative":
-                    full_dest_folder_path = os.path.join(os.path.dirname(reversed_path), dest_folder)
-            subtitles = guess_external_subtitles(full_dest_folder_path, subtitles)
-        except Exception:
-            logging.exception("BAZARR unable to index external subtitles.")
-            pass
-        else:
-            for subtitle, language in subtitles.items():
-                valid_language = False
-                if language:
-                    if hasattr(language, 'alpha3'):
-                        valid_language = alpha2_from_alpha3(language.alpha3)
-                else:
-                    logging.debug(f"Skipping subtitles because we are unable to define language: {subtitle}")
-                    continue
-
-                if not valid_language:
-                    logging.debug(f'{language.alpha3} is an unsupported language code.')
-                    continue
-
-                subtitle_path = get_external_subtitles_path(reversed_path, subtitle)
-                custom = CustomLanguage.found_external(subtitle, subtitle_path)
-
-                if custom is not None:
-                    actual_subtitles.append([custom, path_mappings.path_replace_reverse_movie(subtitle_path)])
-
-                elif str(language.basename) != 'und':
-                    if language.forced:
-                        language_str = str(language)
-                    elif language.hi:
-                        language_str = str(language) + ':hi'
-                    else:
-                        language_str = str(language)
-                    logging.debug("BAZARR external subtitles detected: " + language_str)
-                    actual_subtitles.append([language_str, path_mappings.path_replace_reverse_movie(subtitle_path)])
-
-        TableMovies.update({TableMovies.subtitles: str(actual_subtitles)})\
-            .where(TableMovies.path == original_path)\
-            .execute()
-        matching_movies = TableMovies.select(TableMovies.radarrId).where(TableMovies.path == original_path).dicts()
-
-        for movie in matching_movies:
-            if movie:
-                logging.debug("BAZARR storing those languages to DB: " + str(actual_subtitles))
-                list_missing_subtitles_movies(no=movie['radarrId'])
-            else:
-                logging.debug("BAZARR haven't been able to update existing subtitles to DB : " + str(actual_subtitles))
-    else:
-        logging.debug("BAZARR this file doesn't seems to exist or isn't accessible.")
-
-    logging.debug('BAZARR ended subtitles indexing for this file: ' + reversed_path)
-
-    return actual_subtitles
-
-
-def list_missing_subtitles(no=None, epno=None, send_event=True):
-    if epno is not None:
-        episodes_subtitles_clause = (TableEpisodes.sonarrEpisodeId == epno)
-    elif no is not None:
-        episodes_subtitles_clause = (TableEpisodes.sonarrSeriesId == no)
-    else:
-        episodes_subtitles_clause = None
-    episodes_subtitles = TableEpisodes.select(TableShows.sonarrSeriesId,
-                                              TableEpisodes.sonarrEpisodeId,
-                                              TableEpisodes.subtitles,
-                                              TableShows.profileId,
-                                              TableEpisodes.audio_language)\
-        .join(TableShows, on=(TableEpisodes.sonarrSeriesId == TableShows.sonarrSeriesId))\
-        .where(episodes_subtitles_clause)\
-        .dicts()
-    if isinstance(episodes_subtitles, str):
-        logging.error("BAZARR list missing subtitles query to DB returned this instead of rows: " + episodes_subtitles)
-        return
-
-    use_embedded_subs = settings.general.getboolean('use_embedded_subs')
-
-    for episode_subtitles in episodes_subtitles:
-        missing_subtitles_text = '[]'
-        if episode_subtitles['profileId']:
-            # get desired subtitles
-            desired_subtitles_temp = get_profiles_list(profile_id=episode_subtitles['profileId'])
-            desired_subtitles_list = []
-            if desired_subtitles_temp:
-                for language in desired_subtitles_temp['items']:
-                    if language['audio_exclude'] == "True":
-                        if language_from_alpha2(language['language']) in ast.literal_eval(
-                                episode_subtitles['audio_language']):
-                            continue
-                    desired_subtitles_list.append([language['language'], language['forced'], language['hi']])
-
-            # get existing subtitles
-            actual_subtitles_list = []
-            if episode_subtitles['subtitles'] is not None:
-                if use_embedded_subs:
-                    actual_subtitles_temp = ast.literal_eval(episode_subtitles['subtitles'])
-                else:
-                    actual_subtitles_temp = [x for x in ast.literal_eval(episode_subtitles['subtitles']) if x[1]]
-
-                for subtitles in actual_subtitles_temp:
-                    subtitles = subtitles[0].split(':')
-                    lang = subtitles[0]
-                    forced = False
-                    hi = False
-                    if len(subtitles) > 1:
-                        if subtitles[1] == 'forced':
-                            forced = True
-                            hi = False
-                        elif subtitles[1] == 'hi':
-                            forced = False
-                            hi = True
-                    actual_subtitles_list.append([lang, str(forced), str(hi)])
-
-            # check if cutoff is reached and skip any further check
-            cutoff_met = False
-            cutoff_temp_list = get_profile_cutoff(profile_id=episode_subtitles['profileId'])
-
-            if cutoff_temp_list:
-                for cutoff_temp in cutoff_temp_list:
-                    cutoff_language = [cutoff_temp['language'], cutoff_temp['forced'], cutoff_temp['hi']]
-                    if cutoff_temp['audio_exclude'] == 'True' and language_from_alpha2(cutoff_temp['language']) in \
-                            ast.literal_eval(episode_subtitles['audio_language']):
-                        cutoff_met = True
-                    elif cutoff_language in actual_subtitles_list:
-                        cutoff_met = True
-                    elif cutoff_language and [cutoff_language[0], 'True', 'False'] in actual_subtitles_list:
-                        cutoff_met = True
-                    elif cutoff_language and [cutoff_language[0], 'False', 'True'] in actual_subtitles_list:
-                        cutoff_met = True
-
-            if cutoff_met:
-                missing_subtitles_text = str([])
-            else:
-                # if cutoff isn't met or None, we continue
-
-                # get difference between desired and existing subtitles
-                missing_subtitles_list = []
-                for item in desired_subtitles_list:
-                    if item not in actual_subtitles_list:
-                        missing_subtitles_list.append(item)
-
-                # remove missing that have hi subtitles for this language in existing
-                for item in actual_subtitles_list:
-                    if item[2] == 'True':
-                        try:
-                            missing_subtitles_list.remove([item[0], 'False', 'False'])
-                        except ValueError:
-                            pass
-
-                # make the missing languages list looks like expected
-                missing_subtitles_output_list = []
-                for item in missing_subtitles_list:
-                    lang = item[0]
-                    if item[1] == 'True':
-                        lang += ':forced'
-                    elif item[2] == 'True':
-                        lang += ':hi'
-                    missing_subtitles_output_list.append(lang)
-
-                missing_subtitles_text = str(missing_subtitles_output_list)
-
-        TableEpisodes.update({TableEpisodes.missing_subtitles: missing_subtitles_text})\
-            .where(TableEpisodes.sonarrEpisodeId == episode_subtitles['sonarrEpisodeId'])\
-            .execute()
-
-        if send_event:
-            event_stream(type='episode', payload=episode_subtitles['sonarrEpisodeId'])
-            event_stream(type='badges')
-
-
-def list_missing_subtitles_movies(no=None, send_event=True):
-    movies_subtitles = TableMovies.select(TableMovies.radarrId,
-                                          TableMovies.subtitles,
-                                          TableMovies.profileId,
-                                          TableMovies.audio_language)\
-        .where((TableMovies.radarrId == no) if no else None)\
-        .dicts()
-    if isinstance(movies_subtitles, str):
-        logging.error("BAZARR list missing subtitles query to DB returned this instead of rows: " + movies_subtitles)
-        return
-
-    use_embedded_subs = settings.general.getboolean('use_embedded_subs')
-
-    for movie_subtitles in movies_subtitles:
-        missing_subtitles_text = '[]'
-        if movie_subtitles['profileId']:
-            # get desired subtitles
-            desired_subtitles_temp = get_profiles_list(profile_id=movie_subtitles['profileId'])
-            desired_subtitles_list = []
-            if desired_subtitles_temp:
-                for language in desired_subtitles_temp['items']:
-                    if language['audio_exclude'] == "True":
-                        if language_from_alpha2(language['language']) in ast.literal_eval(
-                                movie_subtitles['audio_language']):
-                            continue
-                    desired_subtitles_list.append([language['language'], language['forced'], language['hi']])
-
-            # get existing subtitles
-            actual_subtitles_list = []
-            if movie_subtitles['subtitles'] is not None:
-                if use_embedded_subs:
-                    actual_subtitles_temp = ast.literal_eval(movie_subtitles['subtitles'])
-                else:
-                    actual_subtitles_temp = [x for x in ast.literal_eval(movie_subtitles['subtitles']) if x[1]]
-
-                for subtitles in actual_subtitles_temp:
-                    subtitles = subtitles[0].split(':')
-                    lang = subtitles[0]
-                    forced = False
-                    hi = False
-                    if len(subtitles) > 1:
-                        if subtitles[1] == 'forced':
-                            forced = True
-                            hi = False
-                        elif subtitles[1] == 'hi':
-                            forced = False
-                            hi = True
-                    actual_subtitles_list.append([lang, str(forced), str(hi)])
-
-            # check if cutoff is reached and skip any further check
-            cutoff_met = False
-            cutoff_temp_list = get_profile_cutoff(profile_id=movie_subtitles['profileId'])
-
-            if cutoff_temp_list:
-                for cutoff_temp in cutoff_temp_list:
-                    cutoff_language = [cutoff_temp['language'], cutoff_temp['forced'], cutoff_temp['hi']]
-                    if cutoff_temp['audio_exclude'] == 'True' and language_from_alpha2(cutoff_temp['language']) in \
-                            ast.literal_eval(movie_subtitles['audio_language']):
-                        cutoff_met = True
-                    elif cutoff_language in actual_subtitles_list:
-                        cutoff_met = True
-                    elif cutoff_language and [cutoff_language[0], 'True', 'False'] in actual_subtitles_list:
-                        cutoff_met = True
-                    elif cutoff_language and [cutoff_language[0], 'False', 'True'] in actual_subtitles_list:
-                        cutoff_met = True
-
-            if cutoff_met:
-                missing_subtitles_text = str([])
-            else:
-                # get difference between desired and existing subtitles
-                missing_subtitles_list = []
-                for item in desired_subtitles_list:
-                    if item not in actual_subtitles_list:
-                        missing_subtitles_list.append(item)
-
-                # remove missing that have forced or hi subtitles for this language in existing
-                for item in actual_subtitles_list:
-                    if item[2] == 'True':
-                        try:
-                            missing_subtitles_list.remove([item[0], 'False', 'False'])
-                        except ValueError:
-                            pass
-
-                # make the missing languages list looks like expected
-                missing_subtitles_output_list = []
-                for item in missing_subtitles_list:
-                    lang = item[0]
-                    if item[1] == 'True':
-                        lang += ':forced'
-                    elif item[2] == 'True':
-                        lang += ':hi'
-                    missing_subtitles_output_list.append(lang)
-
-                missing_subtitles_text = str(missing_subtitles_output_list)
-
-        TableMovies.update({TableMovies.missing_subtitles: missing_subtitles_text})\
-            .where(TableMovies.radarrId == movie_subtitles['radarrId'])\
-            .execute()
-
-        if send_event:
-            event_stream(type='movie', payload=movie_subtitles['radarrId'])
-            event_stream(type='badges')
-
-
-def series_full_scan_subtitles():
-    use_ffprobe_cache = settings.sonarr.getboolean('use_ffprobe_cache')
-
-    episodes = TableEpisodes.select(TableEpisodes.path).dicts()
-
-    count_episodes = len(episodes)
-    for i, episode in enumerate(episodes):
-        show_progress(id='episodes_disk_scan',
-                      header='Full disk scan...',
-                      name='Episodes subtitles',
-                      value=i,
-                      count=count_episodes)
-        store_subtitles(episode['path'], path_mappings.path_replace(episode['path']), use_cache=use_ffprobe_cache)
-
-    hide_progress(id='episodes_disk_scan')
-
-    gc.collect()
-
-
-def movies_full_scan_subtitles():
-    use_ffprobe_cache = settings.radarr.getboolean('use_ffprobe_cache')
-
-    movies = TableMovies.select(TableMovies.path).dicts()
-
-    count_movies = len(movies)
-    for i, movie in enumerate(movies):
-        show_progress(id='movies_disk_scan',
-                      header='Full disk scan...',
-                      name='Movies subtitles',
-                      value=i,
-                      count=count_movies)
-        store_subtitles_movie(movie['path'], path_mappings.path_replace_movie(movie['path']),
-                              use_cache=use_ffprobe_cache)
-
-    hide_progress(id='movies_disk_scan')
-
-    gc.collect()
-
-
-def series_scan_subtitles(no):
-    episodes = TableEpisodes.select(TableEpisodes.path)\
-        .where(TableEpisodes.sonarrSeriesId == no)\
-        .order_by(TableEpisodes.sonarrEpisodeId)\
-        .dicts()
-
-    for episode in episodes:
-        store_subtitles(episode['path'], path_mappings.path_replace(episode['path']), use_cache=False)
-
-
-def movies_scan_subtitles(no):
-    movies = TableMovies.select(TableMovies.path)\
-        .where(TableMovies.radarrId == no)\
-        .order_by(TableMovies.radarrId)\
-        .dicts()
-
-    for movie in movies:
-        store_subtitles_movie(movie['path'], path_mappings.path_replace_movie(movie['path']), use_cache=False)
-
-
-def get_external_subtitles_path(file, subtitle):
-    fld = os.path.dirname(file)
-
-    if settings.general.subfolder == "current":
-        path = os.path.join(fld, subtitle)
-    elif settings.general.subfolder == "absolute":
-        custom_fld = settings.general.subfolder_custom
-        if os.path.exists(os.path.join(fld, subtitle)):
-            path = os.path.join(fld, subtitle)
-        elif os.path.exists(os.path.join(custom_fld, subtitle)):
-            path = os.path.join(custom_fld, subtitle)
-        else:
-            path = None
-    elif settings.general.subfolder == "relative":
-        custom_fld = os.path.join(fld, settings.general.subfolder_custom)
-        if os.path.exists(os.path.join(fld, subtitle)):
-            path = os.path.join(fld, subtitle)
-        elif os.path.exists(os.path.join(custom_fld, subtitle)):
-            path = os.path.join(custom_fld, subtitle)
-        else:
-            path = None
-    else:
-        path = None
-
-    return path
-
-
-def guess_external_subtitles(dest_folder, subtitles):
-    for subtitle, language in subtitles.items():
-        if not language:
-            subtitle_path = os.path.join(dest_folder, subtitle)
-            if os.path.exists(subtitle_path) and os.path.splitext(subtitle_path)[1] in core.SUBTITLE_EXTENSIONS:
-                logging.debug("BAZARR falling back to file content analysis to detect language.")
-                detected_language = None
-
-                # to improve performance, skip detection of files larger that 1M
-                if os.path.getsize(subtitle_path) > 1*1024*1024:
-                    logging.debug("BAZARR subtitles file is too large to be text based. Skipping this file: " +
-                                  subtitle_path)
-                    continue
-
-                with open(subtitle_path, 'rb') as f:
-                    text = f.read()
-
-                try:
-                    text = text.decode('utf-8')
-                    detected_language = guess_language(text)
-                    # add simplified and traditional chinese detection
-                    if detected_language == 'zh':
-                        traditional_chinese_fuzzy = [u"繁", u"雙語"]
-                        traditional_chinese = [".cht", ".tc", ".zh-tw", ".zht", ".zh-hant", ".zhhant", ".zh_hant",
-                                               ".hant", ".big5", ".traditional"]
-                        if str(os.path.splitext(subtitle)[0]).lower().endswith(tuple(traditional_chinese)) or (str(subtitle_path).lower())[:-5] in traditional_chinese_fuzzy:
-                            detected_language == 'zt'
-                except UnicodeDecodeError:
-                    detector = Detector()
-                    try:
-                        guess = detector.detect(text)
-                    except Exception:
-                        logging.debug("BAZARR skipping this subtitles because we can't guess the encoding. "
-                                      "It's probably a binary file: " + subtitle_path)
-                        continue
-                    else:
-                        logging.debug('BAZARR detected encoding %r', guess)
-                        try:
-                            text = text.decode(guess)
-                        except Exception:
-                            logging.debug(
-                                "BAZARR skipping this subtitles because we can't decode the file using the "
-                                "guessed encoding. It's probably a binary file: " + subtitle_path)
-                            continue
-                    detected_language = guess_language(text)
-                except Exception:
-                    logging.debug('BAZARR was unable to detect encoding for this subtitles file: %r', subtitle_path)
-                finally:
-                    if detected_language:
-                        logging.debug("BAZARR external subtitles detected and guessed this language: " + str(
-                            detected_language))
-                        try:
-                            subtitles[subtitle] = Language.rebuild(Language.fromietf(detected_language), forced=False,
-                                                                   hi=False)
-                        except Exception:
-                            pass
-
-        # If language is still None (undetected), skip it
-        if not language:
-            pass
-
-        # Skip HI detection if forced
-        elif language.forced:
-            pass
-
-        # Detect hearing-impaired external subtitles not identified in filename
-        elif not subtitles[subtitle].hi:
-            subtitle_path = os.path.join(dest_folder, subtitle)
-
-            # check if file exist:
-            if os.path.exists(subtitle_path) and os.path.splitext(subtitle_path)[1] in core.SUBTITLE_EXTENSIONS:
-                # to improve performance, skip detection of files larger that 1M
-                if os.path.getsize(subtitle_path) > 1 * 1024 * 1024:
-                    logging.debug("BAZARR subtitles file is too large to be text based. Skipping this file: " +
-                                  subtitle_path)
-                    continue
-
-                with open(subtitle_path, 'rb') as f:
-                    text = f.read()
-
-                try:
-                    text = text.decode('utf-8')
-                except UnicodeDecodeError:
-                    detector = Detector()
-                    try:
-                        guess = detector.detect(text)
-                    except Exception:
-                        logging.debug("BAZARR skipping this subtitles because we can't guess the encoding. "
-                                      "It's probably a binary file: " + subtitle_path)
-                        continue
-                    else:
-                        logging.debug('BAZARR detected encoding %r', guess)
-                        try:
-                            text = text.decode(guess)
-                        except Exception:
-                            logging.debug("BAZARR skipping this subtitles because we can't decode the file using the "
-                                          "guessed encoding. It's probably a binary file: " + subtitle_path)
-                            continue
-
-                if bool(re.search(hi_regex, text)):
-                    subtitles[subtitle] = Language.rebuild(subtitles[subtitle], forced=False, hi=True)
-    return subtitles

bazarr/main.py

@@ -12,32 +12,19 @@ if os.path.isfile(version_file):
 
 os.environ["BAZARR_VERSION"] = bazarr_version.lstrip('v')
 
-import libs  # noqa W0611
+import app.libs  # noqa W0611
 
-from get_args import args  # noqa E402
-from config import settings, url_sonarr, url_radarr, configure_proxy_func, base_url  # noqa E402
-
-from init import *  # noqa E402
-from database import System  # noqa E402
-
-from notifier import update_notifier  # noqa E402
-
-from urllib.parse import unquote  # noqa E402
-from get_languages import load_language_in_db  # noqa E402
-from flask import request, redirect, abort, render_template, Response, session, send_file, stream_with_context, \
-    send_from_directory
 from threading import Thread  # noqa E402
-import requests  # noqa E402
 
-from get_series import *  # noqa E402
-from get_episodes import *  # noqa E402
-from get_movies import *  # noqa E402
-from signalr_client import sonarr_signalr_client, radarr_signalr_client  # noqa E402
-
-from check_update import apply_update, check_releases  # noqa E402
-from server import app, webserver  # noqa E402
-from functools import wraps  # noqa E402
-from utils import check_credentials, get_sonarr_info, get_radarr_info  # noqa E402
+from app.get_args import args  # noqa E402
+from app.config import settings, configure_proxy_func, base_url  # noqa E402
+from init import *  # noqa E402
+from app.database import System  # noqa E402
+from app.notifier import update_notifier  # noqa E402
+from languages.get_languages import load_language_in_db  # noqa E402
+from app.signalr_client import sonarr_signalr_client, radarr_signalr_client  # noqa E402
+from app.check_update import apply_update, check_releases  # noqa E402
+from app.server import webserver  # noqa E402
 
 # Install downloaded update
 if bazarr_version != '':
@@ -56,152 +43,6 @@ login_auth = settings.auth.type
 
 update_notifier()
 
-headers = {"User-Agent": os.environ["SZ_USER_AGENT"]}
-
-
-def check_login(actual_method):
-    @wraps(actual_method)
-    def wrapper(*args, **kwargs):
-        if settings.auth.type == 'basic':
-            auth = request.authorization
-            if not (auth and check_credentials(request.authorization.username, request.authorization.password)):
-                return ('Unauthorized', 401, {
-                    'WWW-Authenticate': 'Basic realm="Login Required"'
-                })
-        elif settings.auth.type == 'form':
-            if 'logged_in' not in session:
-                return abort(401, message="Unauthorized")
-        actual_method(*args, **kwargs)
-
-
-@app.errorhandler(404)
-def page_not_found(e):
-    return redirect(base_url, code=302)
-
-
-@app.route('/', defaults={'path': ''})
-@app.route('/<path:path>')
-def catch_all(path):
-    auth = True
-    if settings.auth.type == 'basic':
-        auth = request.authorization
-        if not (auth and check_credentials(request.authorization.username, request.authorization.password)):
-            return ('Unauthorized', 401, {
-                'WWW-Authenticate': 'Basic realm="Login Required"'
-            })
-    elif settings.auth.type == 'form':
-        if 'logged_in' not in session:
-            auth = False
-
-    try:
-        updated = System.get().updated
-    except Exception:
-        updated = '0'
-
-    inject = dict()
-    inject["baseUrl"] = base_url
-    inject["canUpdate"] = not args.no_update
-    inject["hasUpdate"] = updated != '0'
-
-    if auth:
-        inject["apiKey"] = settings.auth.apikey
-
-    template_url = base_url
-    if not template_url.endswith("/"):
-        template_url += "/"
-
-    return render_template("index.html", BAZARR_SERVER_INJECT=inject, baseUrl=template_url)
-
-
-@app.route('/assets/<path:filename>')
-def web_assets(filename):
-    path = os.path.join(os.path.dirname(__file__), '..', 'frontend', 'build', 'assets')
-    return send_from_directory(path, filename)
-
-
-@check_login
-@app.route('/bazarr.log')
-def download_log():
-    return send_file(os.path.join(args.config_dir, 'log', 'bazarr.log'), cache_timeout=0, as_attachment=True)
-
-
-@check_login
-@app.route('/images/series/<path:url>', methods=['GET'])
-def series_images(url):
-    url = url.strip("/")
-    apikey = settings.sonarr.apikey
-    baseUrl = settings.sonarr.base_url
-    if get_sonarr_info.is_legacy():
-        url_image = (url_sonarr() + '/api/' + url.lstrip(baseUrl) + '?apikey=' +
-                     apikey).replace('poster-250', 'poster-500')
-    else:
-        url_image = (url_sonarr() + '/api/v3/' + url.lstrip(baseUrl) + '?apikey=' +
-                     apikey).replace('poster-250', 'poster-500')
-    try:
-        req = requests.get(url_image, stream=True, timeout=15, verify=False, headers=headers)
-    except Exception:
-        return '', 404
-    else:
-        return Response(stream_with_context(req.iter_content(2048)), content_type=req.headers['content-type'])
-
-
-@check_login
-@app.route('/images/movies/<path:url>', methods=['GET'])
-def movies_images(url):
-    apikey = settings.radarr.apikey
-    baseUrl = settings.radarr.base_url
-    if get_radarr_info.is_legacy():
-        url_image = url_radarr() + '/api/' + url.lstrip(baseUrl) + '?apikey=' + apikey
-    else:
-        url_image = url_radarr() + '/api/v3/' + url.lstrip(baseUrl) + '?apikey=' + apikey
-    try:
-        req = requests.get(url_image, stream=True, timeout=15, verify=False, headers=headers)
-    except Exception:
-        return '', 404
-    else:
-        return Response(stream_with_context(req.iter_content(2048)), content_type=req.headers['content-type'])
-
-
-# @app.route('/check_update')
-# @authenticate
-# def check_update():
-#     if not args.no_update:
-#         check_and_apply_update()
-
-#     return '', 200
-
-
-def configured():
-    System.update({System.configured: '1'}).execute()
-
-
-@check_login
-@app.route('/test', methods=['GET'])
-@app.route('/test/<protocol>/<path:url>', methods=['GET'])
-def proxy(protocol, url):
-    url = protocol + '://' + unquote(url)
-    params = request.args
-    try:
-        result = requests.get(url, params, allow_redirects=False, verify=False, timeout=5, headers=headers)
-    except Exception as e:
-        return dict(status=False, error=repr(e))
-    else:
-        if result.status_code == 200:
-            try:
-                version = result.json()['version']
-                return dict(status=True, version=version)
-            except Exception:
-                return dict(status=False, error='Error Occurred. Check your settings.')
-        elif result.status_code == 401:
-            return dict(status=False, error='Access Denied. Check API key.')
-        elif result.status_code == 404:
-            return dict(status=False, error='Cannot get version. Maybe unsupported legacy API call?')
-        elif 300 <= result.status_code <= 399:
-            return dict(status=False, error='Wrong URL Base.')
-        else:
-            return dict(status=False, error=result.raise_for_status())
-
-
 if settings.general.getboolean('use_sonarr'):
     Thread(target=sonarr_signalr_client.start).start()
 if settings.general.getboolean('use_radarr'):

bazarr/radarr/__init__.py

@@ -0,0 +1 @@
+# coding=utf-8

bazarr/radarr/blacklist.py

@@ -0,0 +1,39 @@
+# coding=utf-8
+
+import time
+
+from app.database import TableBlacklistMovie
+from app.event_handler import event_stream
+
+
+def get_blacklist_movie():
+    blacklist_db = TableBlacklistMovie.select(TableBlacklistMovie.provider, TableBlacklistMovie.subs_id).dicts()
+
+    blacklist_list = []
+    for item in blacklist_db:
+        blacklist_list.append((item['provider'], item['subs_id']))
+
+    return blacklist_list
+
+
+def blacklist_log_movie(radarr_id, provider, subs_id, language):
+    TableBlacklistMovie.insert({
+        TableBlacklistMovie.radarr_id: radarr_id,
+        TableBlacklistMovie.timestamp: time.time(),
+        TableBlacklistMovie.provider: provider,
+        TableBlacklistMovie.subs_id: subs_id,
+        TableBlacklistMovie.language: language
+    }).execute()
+    event_stream(type='movie-blacklist')
+
+
+def blacklist_delete_movie(provider, subs_id):
+    TableBlacklistMovie.delete().where((TableBlacklistMovie.provider == provider) and
+                                       (TableBlacklistMovie.subs_id == subs_id))\
+        .execute()
+    event_stream(type='movie-blacklist', action='delete')
+
+
+def blacklist_delete_all_movie():
+    TableBlacklistMovie.delete().execute()
+    event_stream(type='movie-blacklist', action='delete')

bazarr/radarr/filesystem.py

@@ -0,0 +1,39 @@
+# coding=utf-8
+
+import requests
+import logging
+
+from app.config import settings
+from radarr.info import get_radarr_info, url_radarr
+from constants import headers
+
+
+def browse_radarr_filesystem(path='#'):
+    if path == '#':
+        path = ''
+
+    if get_radarr_info.is_legacy():
+        url_radarr_api_filesystem = url_radarr() + "/api/filesystem?path=" + path + \
+                                    "&allowFoldersWithoutTrailingSlashes=true&includeFiles=false&apikey=" + \
+                                    settings.radarr.apikey
+    else:
+        url_radarr_api_filesystem = url_radarr() + "/api/v3/filesystem?path=" + path + \
+                                    "&allowFoldersWithoutTrailingSlashes=true&includeFiles=false&apikey=" + \
+                                    settings.radarr.apikey
+    try:
+        r = requests.get(url_radarr_api_filesystem, timeout=60, verify=False, headers=headers)
+        r.raise_for_status()
+    except requests.exceptions.HTTPError:
+        logging.exception("BAZARR Error trying to get series from Radarr. Http error.")
+        return
+    except requests.exceptions.ConnectionError:
+        logging.exception("BAZARR Error trying to get series from Radarr. Connection Error.")
+        return
+    except requests.exceptions.Timeout:
+        logging.exception("BAZARR Error trying to get series from Radarr. Timeout Error.")
+        return
+    except requests.exceptions.RequestException:
+        logging.exception("BAZARR Error trying to get series from Radarr.")
+        return
+
+    return r.json()

bazarr/radarr/history.py

@@ -0,0 +1,23 @@
+# coding=utf-8
+
+import time
+
+from app.database import TableHistoryMovie
+from app.event_handler import event_stream
+
+
+def history_log_movie(action, radarr_id, description, video_path=None, language=None, provider=None, score=None,
+                      subs_id=None, subtitles_path=None):
+    TableHistoryMovie.insert({
+        TableHistoryMovie.action: action,
+        TableHistoryMovie.radarrId: radarr_id,
+        TableHistoryMovie.timestamp: time.time(),
+        TableHistoryMovie.description: description,
+        TableHistoryMovie.video_path: video_path,
+        TableHistoryMovie.language: language,
+        TableHistoryMovie.provider: provider,
+        TableHistoryMovie.score: score,
+        TableHistoryMovie.subs_id: subs_id,
+        TableHistoryMovie.subtitles_path: subtitles_path
+    }).execute()
+    event_stream(type='movie-history')

bazarr/radarr/info.py

@@ -0,0 +1,84 @@
+# coding=utf-8
+
+import logging
+import requests
+import datetime
+import json
+
+from dogpile.cache import make_region
+
+from app.config import settings, empty_values
+from constants import headers
+
+region = make_region().configure('dogpile.cache.memory')
+
+
+class GetRadarrInfo:
+    @staticmethod
+    def version():
+        """
+        Call system/status API endpoint and get the Radarr version
+        @return: str
+        """
+        radarr_version = region.get("radarr_version", expiration_time=datetime.timedelta(seconds=60).total_seconds())
+        if radarr_version:
+            region.set("radarr_version", radarr_version)
+            return radarr_version
+        else:
+            radarr_version = ''
+        if settings.general.getboolean('use_radarr'):
+            try:
+                rv = url_radarr() + "/api/system/status?apikey=" + settings.radarr.apikey
+                radarr_json = requests.get(rv, timeout=60, verify=False, headers=headers).json()
+                if 'version' in radarr_json:
+                    radarr_version = radarr_json['version']
+                else:
+                    raise json.decoder.JSONDecodeError
+            except json.decoder.JSONDecodeError:
+                try:
+                    rv = url_radarr() + "/api/v3/system/status?apikey=" + settings.radarr.apikey
+                    radarr_version = requests.get(rv, timeout=60, verify=False, headers=headers).json()['version']
+                except json.decoder.JSONDecodeError:
+                    logging.debug('BAZARR cannot get Radarr version')
+                    radarr_version = 'unknown'
+            except Exception:
+                logging.debug('BAZARR cannot get Radarr version')
+                radarr_version = 'unknown'
+        logging.debug('BAZARR got this Radarr version from its API: {}'.format(radarr_version))
+        region.set("radarr_version", radarr_version)
+        return radarr_version
+
+    def is_legacy(self):
+        """
+        Call self.version() and parse the result to determine if it's a legacy version of Radarr
+        @return: bool
+        """
+        radarr_version = self.version()
+        if radarr_version.startswith('0.'):
+            return True
+        else:
+            return False
+
+
+get_radarr_info = GetRadarrInfo()
+
+
+def url_radarr():
+    if settings.radarr.getboolean('ssl'):
+        protocol_radarr = "https"
+    else:
+        protocol_radarr = "http"
+
+    if settings.radarr.base_url == '':
+        settings.radarr.base_url = "/"
+    if not settings.radarr.base_url.startswith("/"):
+        settings.radarr.base_url = "/" + settings.radarr.base_url
+    if settings.radarr.base_url.endswith("/"):
+        settings.radarr.base_url = settings.radarr.base_url[:-1]
+
+    if settings.radarr.port in empty_values:
+        port = ""
+    else:
+        port = f":{settings.radarr.port}"
+
+    return f"{protocol_radarr}://{settings.radarr.ip}{port}{settings.radarr.base_url}"

bazarr/radarr/notify.py

@@ -0,0 +1,23 @@
+# coding=utf-8
+
+import logging
+import requests
+
+from app.config import settings
+from radarr.info import get_radarr_info, url_radarr
+from constants import headers
+
+
+def notify_radarr(radarr_id):
+    try:
+        if get_radarr_info.is_legacy():
+            url = url_radarr() + "/api/command?apikey=" + settings.radarr.apikey
+        else:
+            url = url_radarr() + "/api/v3/command?apikey=" + settings.radarr.apikey
+        data = {
+            'name': 'RescanMovie',
+            'movieId': int(radarr_id)
+        }
+        requests.post(url, json=data, timeout=60, verify=False, headers=headers)
+    except Exception:
+        logging.exception('BAZARR cannot notify Radarr')

bazarr/radarr/rootfolder.py

@@ -0,0 +1,84 @@
+# coding=utf-8
+
+import os
+import requests
+import logging
+
+from app.config import settings
+from utilities.path_mappings import path_mappings
+from app.database import TableMoviesRootfolder, TableMovies
+from radarr.info import get_radarr_info, url_radarr
+from constants import headers
+
+
+def get_radarr_rootfolder():
+    apikey_radarr = settings.radarr.apikey
+    radarr_rootfolder = []
+
+    # Get root folder data from Radarr
+    if get_radarr_info.is_legacy():
+        url_radarr_api_rootfolder = url_radarr() + "/api/rootfolder?apikey=" + apikey_radarr
+    else:
+        url_radarr_api_rootfolder = url_radarr() + "/api/v3/rootfolder?apikey=" + apikey_radarr
+
+    try:
+        rootfolder = requests.get(url_radarr_api_rootfolder, timeout=60, verify=False, headers=headers)
+    except requests.exceptions.ConnectionError:
+        logging.exception("BAZARR Error trying to get rootfolder from Radarr. Connection Error.")
+        return []
+    except requests.exceptions.Timeout:
+        logging.exception("BAZARR Error trying to get rootfolder from Radarr. Timeout Error.")
+        return []
+    except requests.exceptions.RequestException:
+        logging.exception("BAZARR Error trying to get rootfolder from Radarr.")
+        return []
+    else:
+        radarr_movies_paths = list(TableMovies.select(TableMovies.path).dicts())
+        for folder in rootfolder.json():
+            if any(item['path'].startswith(folder['path']) for item in radarr_movies_paths):
+                radarr_rootfolder.append({'id': folder['id'], 'path': folder['path']})
+        db_rootfolder = TableMoviesRootfolder.select(TableMoviesRootfolder.id, TableMoviesRootfolder.path).dicts()
+        rootfolder_to_remove = [x for x in db_rootfolder if not
+                                next((item for item in radarr_rootfolder if item['id'] == x['id']), False)]
+        rootfolder_to_update = [x for x in radarr_rootfolder if
+                                next((item for item in db_rootfolder if item['id'] == x['id']), False)]
+        rootfolder_to_insert = [x for x in radarr_rootfolder if not
+                                next((item for item in db_rootfolder if item['id'] == x['id']), False)]
+
+        for item in rootfolder_to_remove:
+            TableMoviesRootfolder.delete().where(TableMoviesRootfolder.id == item['id']).execute()
+        for item in rootfolder_to_update:
+            TableMoviesRootfolder.update({TableMoviesRootfolder.path: item['path']})\
+                .where(TableMoviesRootfolder.id == item['id']).execute()
+        for item in rootfolder_to_insert:
+            TableMoviesRootfolder.insert({TableMoviesRootfolder.id: item['id'],
+                                          TableMoviesRootfolder.path: item['path']}).execute()
+
+
+def check_radarr_rootfolder():
+    get_radarr_rootfolder()
+    rootfolder = TableMoviesRootfolder.select(TableMoviesRootfolder.id, TableMoviesRootfolder.path).dicts()
+    for item in rootfolder:
+        root_path = item['path']
+        if not root_path.endswith(('/', '\\')):
+            if root_path.startswith('/'):
+                root_path += '/'
+            else:
+                root_path += '\\'
+        if not os.path.isdir(path_mappings.path_replace_movie(root_path)):
+            TableMoviesRootfolder.update({TableMoviesRootfolder.accessible: 0,
+                                         TableMoviesRootfolder.error: 'This Radarr root directory does not seems to '
+                                                                      'be accessible by  Please check path '
+                                                                      'mapping.'}) \
+                .where(TableMoviesRootfolder.id == item['id']) \
+                .execute()
+        elif not os.access(path_mappings.path_replace_movie(root_path), os.W_OK):
+            TableMoviesRootfolder.update({TableMoviesRootfolder.accessible: 0,
+                                          TableMoviesRootfolder.error: 'Bazarr cannot write to this directory'}) \
+                .where(TableMoviesRootfolder.id == item['id']) \
+                .execute()
+        else:
+            TableMoviesRootfolder.update({TableMoviesRootfolder.accessible: 1,
+                                          TableMoviesRootfolder.error: ''}) \
+                .where(TableMoviesRootfolder.id == item['id']) \
+                .execute()

bazarr/radarr/sync/__init__.py

@@ -0,0 +1 @@
+# coding=utf-8

bazarr/radarr/sync/converter.py

@@ -0,0 +1,65 @@
+# coding=utf-8
+
+def RadarrFormatAudioCodec(audioFormat, audioCodecID, audioProfile, audioAdditionalFeatures):
+    if type(audioFormat) is not str:
+        return audioFormat
+    else:
+        if audioFormat == "AC-3":
+            return "AC3"
+        elif audioFormat == "E-AC-3":
+            return "EAC3"
+        elif audioFormat == "AAC":
+            if audioCodecID == "A_AAC/MPEG4/LC/SBR":
+                return "HE-AAC"
+            else:
+                return "AAC"
+        elif audioFormat.strip() == "mp3":
+            return "MP3"
+        elif audioFormat == "MPEG Audio":
+            if audioCodecID == "55" or audioCodecID == "A_MPEG/L3" or audioProfile == "Layer 3":
+                return "MP3"
+            if audioCodecID == "A_MPEG/L2" or audioProfile == "Layer 2":
+                return "MP2"
+        elif audioFormat == "MLP FBA":
+            if audioAdditionalFeatures == "16-ch":
+                return "TrueHD Atmos"
+            else:
+                return "TrueHD"
+        else:
+            return audioFormat
+
+
+def RadarrFormatVideoCodec(videoFormat, videoCodecID, videoCodecLibrary):
+    if type(videoFormat) is not str:
+        return videoFormat
+    else:
+        if videoFormat == "x264":
+            return "h264"
+        elif videoFormat == "AVC" or videoFormat == "V.MPEG4/ISO/AVC":
+            return "h264"
+        elif videoCodecLibrary and (videoFormat == "HEVC" or videoFormat == "V_MPEGH/ISO/HEVC"):
+            if videoCodecLibrary.startswith("x265"):
+                return "h265"
+        elif videoCodecID and videoFormat == "MPEG Video":
+            if videoCodecID == "2" or videoCodecID == "V_MPEG2":
+                return "Mpeg2"
+            else:
+                return "Mpeg"
+        elif videoFormat == "MPEG-1 Video":
+            return "Mpeg"
+        elif videoFormat == "MPEG-2 Video":
+            return "Mpeg2"
+        elif videoCodecLibrary and videoCodecID and videoFormat == "MPEG-4 Visual":
+            if videoCodecID.endswith("XVID") or videoCodecLibrary.startswith("XviD"):
+                return "XviD"
+            if videoCodecID.endswith("DIV3") or videoCodecID.endswith("DIVX") or videoCodecID.endswith(
+                    "DX50") or videoCodecLibrary.startswith("DivX"):
+                return "DivX"
+        elif videoFormat == "VC-1":
+            return "VC1"
+        elif videoFormat == "WMV2":
+            return "WMV"
+        elif videoFormat == "DivX" or videoFormat == "div3":
+            return "DivX"
+        else:
+            return videoFormat

bazarr/radarr/sync/movies.py

@@ -0,0 +1,269 @@
+# coding=utf-8
+
+import logging
+
+from peewee import IntegrityError
+
+from app.config import settings
+from radarr.info import url_radarr
+from utilities.path_mappings import path_mappings
+from subtitles.indexer.movies import store_subtitles_movie, movies_full_scan_subtitles
+from radarr.rootfolder import check_radarr_rootfolder
+from subtitles.mass_download import movies_download_subtitles
+from app.database import TableMovies
+from app.event_handler import event_stream, show_progress, hide_progress
+
+from .utils import get_profile_list, get_tags, get_movies_from_radarr_api
+from .parser import movieParser
+
+
+def update_all_movies():
+    movies_full_scan_subtitles()
+    logging.info('BAZARR All existing movie subtitles indexed from disk.')
+
+
+def update_movies(send_event=True):
+    check_radarr_rootfolder()
+    logging.debug('BAZARR Starting movie sync from Radarr.')
+    apikey_radarr = settings.radarr.apikey
+
+    movie_default_enabled = settings.general.getboolean('movie_default_enabled')
+
+    if movie_default_enabled is True:
+        movie_default_profile = settings.general.movie_default_profile
+        if movie_default_profile == '':
+            movie_default_profile = None
+    else:
+        movie_default_profile = None
+
+    if apikey_radarr is None:
+        pass
+    else:
+        audio_profiles = get_profile_list()
+        tagsDict = get_tags()
+
+        # Get movies data from radarr
+        movies = get_movies_from_radarr_api(url=url_radarr(), apikey_radarr=apikey_radarr)
+        if not movies:
+            return
+        else:
+            # Get current movies in DB
+            current_movies_db = TableMovies.select(TableMovies.tmdbId, TableMovies.path, TableMovies.radarrId).dicts()
+
+            current_movies_db_list = [x['tmdbId'] for x in current_movies_db]
+
+            current_movies_radarr = []
+            movies_to_update = []
+            movies_to_add = []
+            altered_movies = []
+
+            # Build new and updated movies
+            movies_count = len(movies)
+            for i, movie in enumerate(movies):
+                if send_event:
+                    show_progress(id='movies_progress',
+                                  header='Syncing movies...',
+                                  name=movie['title'],
+                                  value=i,
+                                  count=movies_count)
+
+                if movie['hasFile'] is True:
+                    if 'movieFile' in movie:
+                        if movie['movieFile']['size'] > 20480:
+                            # Add movies in radarr to current movies list
+                            current_movies_radarr.append(str(movie['tmdbId']))
+
+                            if str(movie['tmdbId']) in current_movies_db_list:
+                                movies_to_update.append(movieParser(movie, action='update',
+                                                                    tags_dict=tagsDict,
+                                                                    movie_default_profile=movie_default_profile,
+                                                                    audio_profiles=audio_profiles))
+                            else:
+                                movies_to_add.append(movieParser(movie, action='insert',
+                                                                 tags_dict=tagsDict,
+                                                                 movie_default_profile=movie_default_profile,
+                                                                 audio_profiles=audio_profiles))
+
+            if send_event:
+                hide_progress(id='movies_progress')
+
+            # Remove old movies from DB
+            removed_movies = list(set(current_movies_db_list) - set(current_movies_radarr))
+
+            for removed_movie in removed_movies:
+                try:
+                    TableMovies.delete().where(TableMovies.tmdbId == removed_movie).execute()
+                except Exception as e:
+                    logging.error(f"BAZARR cannot remove movie tmdbId {removed_movie} because of {e}")
+                    continue
+
+            # Update movies in DB
+            movies_in_db_list = []
+            movies_in_db = TableMovies.select(TableMovies.radarrId,
+                                              TableMovies.title,
+                                              TableMovies.path,
+                                              TableMovies.tmdbId,
+                                              TableMovies.overview,
+                                              TableMovies.poster,
+                                              TableMovies.fanart,
+                                              TableMovies.audio_language,
+                                              TableMovies.sceneName,
+                                              TableMovies.monitored,
+                                              TableMovies.sortTitle,
+                                              TableMovies.year,
+                                              TableMovies.alternativeTitles,
+                                              TableMovies.format,
+                                              TableMovies.resolution,
+                                              TableMovies.video_codec,
+                                              TableMovies.audio_codec,
+                                              TableMovies.imdbId,
+                                              TableMovies.movie_file_id,
+                                              TableMovies.tags,
+                                              TableMovies.file_size).dicts()
+
+            for item in movies_in_db:
+                movies_in_db_list.append(item)
+
+            movies_to_update_list = [i for i in movies_to_update if i not in movies_in_db_list]
+
+            for updated_movie in movies_to_update_list:
+                try:
+                    TableMovies.update(updated_movie).where(TableMovies.tmdbId == updated_movie['tmdbId']).execute()
+                except IntegrityError as e:
+                    logging.error(f"BAZARR cannot update movie {updated_movie['path']} because of {e}")
+                    continue
+                else:
+                    altered_movies.append([updated_movie['tmdbId'],
+                                           updated_movie['path'],
+                                           updated_movie['radarrId'],
+                                           updated_movie['monitored']])
+
+            # Insert new movies in DB
+            for added_movie in movies_to_add:
+                try:
+                    result = TableMovies.insert(added_movie).on_conflict(action='IGNORE').execute()
+                except IntegrityError as e:
+                    logging.error(f"BAZARR cannot insert movie {added_movie['path']} because of {e}")
+                    continue
+                else:
+                    if result > 0:
+                        altered_movies.append([added_movie['tmdbId'],
+                                               added_movie['path'],
+                                               added_movie['radarrId'],
+                                               added_movie['monitored']])
+                        if send_event:
+                            event_stream(type='movie', action='update', payload=int(added_movie['radarrId']))
+                    else:
+                        logging.debug('BAZARR unable to insert this movie into the database:',
+                                      path_mappings.path_replace_movie(added_movie['path']))
+
+            # Store subtitles for added or modified movies
+            for i, altered_movie in enumerate(altered_movies, 1):
+                store_subtitles_movie(altered_movie[1], path_mappings.path_replace_movie(altered_movie[1]))
+
+            logging.debug('BAZARR All movies synced from Radarr into database.')
+
+
+def update_one_movie(movie_id, action, defer_search=False):
+    logging.debug('BAZARR syncing this specific movie from Radarr: {}'.format(movie_id))
+
+    # Check if there's a row in database for this movie ID
+    existing_movie = TableMovies.select(TableMovies.path)\
+        .where(TableMovies.radarrId == movie_id)\
+        .dicts()\
+        .get_or_none()
+
+    # Remove movie from DB
+    if action == 'deleted':
+        if existing_movie:
+            try:
+                TableMovies.delete().where(TableMovies.radarrId == movie_id).execute()
+            except Exception as e:
+                logging.error(f"BAZARR cannot delete movie {existing_movie['path']} because of {e}")
+            else:
+                event_stream(type='movie', action='delete', payload=int(movie_id))
+                logging.debug('BAZARR deleted this movie from the database:{}'.format(path_mappings.path_replace_movie(
+                    existing_movie['path'])))
+        return
+
+    movie_default_enabled = settings.general.getboolean('movie_default_enabled')
+
+    if movie_default_enabled is True:
+        movie_default_profile = settings.general.movie_default_profile
+        if movie_default_profile == '':
+            movie_default_profile = None
+    else:
+        movie_default_profile = None
+
+    audio_profiles = get_profile_list()
+    tagsDict = get_tags()
+
+    try:
+        # Get movie data from radarr api
+        movie = None
+        movie_data = get_movies_from_radarr_api(url=url_radarr(), apikey_radarr=settings.radarr.apikey,
+                                                radarr_id=movie_id)
+        if not movie_data:
+            return
+        else:
+            if action == 'updated' and existing_movie:
+                movie = movieParser(movie_data, action='update', tags_dict=tagsDict,
+                                    movie_default_profile=movie_default_profile, audio_profiles=audio_profiles)
+            elif action == 'updated' and not existing_movie:
+                movie = movieParser(movie_data, action='insert', tags_dict=tagsDict,
+                                    movie_default_profile=movie_default_profile, audio_profiles=audio_profiles)
+    except Exception:
+        logging.debug('BAZARR cannot get movie returned by SignalR feed from Radarr API.')
+        return
+
+    # Drop useless events
+    if not movie and not existing_movie:
+        return
+
+    # Remove movie from DB
+    if not movie and existing_movie:
+        try:
+            TableMovies.delete().where(TableMovies.radarrId == movie_id).execute()
+        except Exception as e:
+            logging.error(f"BAZARR cannot insert episode {existing_movie['path']} because of {e}")
+        else:
+            event_stream(type='movie', action='delete', payload=int(movie_id))
+            logging.debug('BAZARR deleted this movie from the database:{}'.format(path_mappings.path_replace_movie(
+                existing_movie['path'])))
+            return
+
+    # Update existing movie in DB
+    elif movie and existing_movie:
+        try:
+            TableMovies.update(movie).where(TableMovies.radarrId == movie['radarrId']).execute()
+        except IntegrityError as e:
+            logging.error(f"BAZARR cannot insert episode {movie['path']} because of {e}")
+        else:
+            event_stream(type='movie', action='update', payload=int(movie_id))
+            logging.debug('BAZARR updated this movie into the database:{}'.format(path_mappings.path_replace_movie(
+                movie['path'])))
+
+    # Insert new movie in DB
+    elif movie and not existing_movie:
+        try:
+            TableMovies.insert(movie).on_conflict(action='IGNORE').execute()
+        except IntegrityError as e:
+            logging.error(f"BAZARR cannot insert movie {movie['path']} because of {e}")
+        else:
+            event_stream(type='movie', action='update', payload=int(movie_id))
+            logging.debug('BAZARR inserted this movie into the database:{}'.format(path_mappings.path_replace_movie(
+                movie['path'])))
+
+    # Storing existing subtitles
+    logging.debug('BAZARR storing subtitles for this movie: {}'.format(path_mappings.path_replace_movie(
+            movie['path'])))
+    store_subtitles_movie(movie['path'], path_mappings.path_replace_movie(movie['path']))
+
+    # Downloading missing subtitles
+    if defer_search:
+        logging.debug('BAZARR searching for missing subtitles is deferred until scheduled task execution for this '
+                      'movie: {}'.format(path_mappings.path_replace_movie(movie['path'])))
+    else:
+        logging.debug('BAZARR downloading missing subtitles for this movie: {}'.format(path_mappings.path_replace_movie(
+            movie['path'])))
+        movies_download_subtitles(movie_id)

bazarr/radarr/sync/parser.py

@@ -0,0 +1,167 @@
+# coding=utf-8
+
+import os
+
+from radarr.info import get_radarr_info
+from languages.get_languages import language_from_alpha2
+
+from .converter import RadarrFormatAudioCodec, RadarrFormatVideoCodec
+
+
+def movieParser(movie, action, tags_dict, movie_default_profile, audio_profiles):
+    if 'movieFile' in movie:
+        # Detect file separator
+        if movie['path'][0] == "/":
+            separator = "/"
+        else:
+            separator = "\\"
+
+        try:
+            overview = str(movie['overview'])
+        except Exception:
+            overview = ""
+        try:
+            poster_big = movie['images'][0]['url']
+            poster = os.path.splitext(poster_big)[0] + '-500' + os.path.splitext(poster_big)[1]
+        except Exception:
+            poster = ""
+        try:
+            fanart = movie['images'][1]['url']
+        except Exception:
+            fanart = ""
+
+        if 'sceneName' in movie['movieFile']:
+            sceneName = movie['movieFile']['sceneName']
+        else:
+            sceneName = None
+
+        alternativeTitles = None
+        if get_radarr_info.is_legacy():
+            if 'alternativeTitles' in movie:
+                alternativeTitles = str([item['title'] for item in movie['alternativeTitles']])
+        else:
+            if 'alternateTitles' in movie:
+                alternativeTitles = str([item['title'] for item in movie['alternateTitles']])
+
+        if 'imdbId' in movie:
+            imdbId = movie['imdbId']
+        else:
+            imdbId = None
+
+        try:
+            format, resolution = movie['movieFile']['quality']['quality']['name'].split('-')
+        except Exception:
+            format = movie['movieFile']['quality']['quality']['name']
+            try:
+                resolution = str(movie['movieFile']['quality']['quality']['resolution']) + 'p'
+            except Exception:
+                resolution = None
+
+        if 'mediaInfo' in movie['movieFile']:
+            videoFormat = videoCodecID = videoCodecLibrary = None
+            if get_radarr_info.is_legacy():
+                if 'videoFormat' in movie['movieFile']['mediaInfo']:
+                    videoFormat = movie['movieFile']['mediaInfo']['videoFormat']
+            else:
+                if 'videoCodec' in movie['movieFile']['mediaInfo']:
+                    videoFormat = movie['movieFile']['mediaInfo']['videoCodec']
+            if 'videoCodecID' in movie['movieFile']['mediaInfo']:
+                videoCodecID = movie['movieFile']['mediaInfo']['videoCodecID']
+            if 'videoCodecLibrary' in movie['movieFile']['mediaInfo']:
+                videoCodecLibrary = movie['movieFile']['mediaInfo']['videoCodecLibrary']
+            videoCodec = RadarrFormatVideoCodec(videoFormat, videoCodecID, videoCodecLibrary)
+
+            audioFormat = audioCodecID = audioProfile = audioAdditionalFeatures = None
+            if get_radarr_info.is_legacy():
+                if 'audioFormat' in movie['movieFile']['mediaInfo']:
+                    audioFormat = movie['movieFile']['mediaInfo']['audioFormat']
+            else:
+                if 'audioCodec' in movie['movieFile']['mediaInfo']:
+                    audioFormat = movie['movieFile']['mediaInfo']['audioCodec']
+            if 'audioCodecID' in movie['movieFile']['mediaInfo']:
+                audioCodecID = movie['movieFile']['mediaInfo']['audioCodecID']
+            if 'audioProfile' in movie['movieFile']['mediaInfo']:
+                audioProfile = movie['movieFile']['mediaInfo']['audioProfile']
+            if 'audioAdditionalFeatures' in movie['movieFile']['mediaInfo']:
+                audioAdditionalFeatures = movie['movieFile']['mediaInfo']['audioAdditionalFeatures']
+            audioCodec = RadarrFormatAudioCodec(audioFormat, audioCodecID, audioProfile, audioAdditionalFeatures)
+        else:
+            videoCodec = None
+            audioCodec = None
+
+        audio_language = []
+        if get_radarr_info.is_legacy():
+            if 'mediaInfo' in movie['movieFile']:
+                if 'audioLanguages' in movie['movieFile']['mediaInfo']:
+                    audio_languages_list = movie['movieFile']['mediaInfo']['audioLanguages'].split('/')
+                    if len(audio_languages_list):
+                        for audio_language_list in audio_languages_list:
+                            audio_language.append(audio_language_list.strip())
+            if not audio_language:
+                audio_language = profile_id_to_language(movie['qualityProfileId'], audio_profiles)
+        else:
+            if 'languages' in movie['movieFile'] and len(movie['movieFile']['languages']):
+                for item in movie['movieFile']['languages']:
+                    if isinstance(item, dict):
+                        if 'name' in item:
+                            language = item['name']
+                            if item['name'] == 'Portuguese (Brazil)':
+                                language = language_from_alpha2('pb')
+                            audio_language.append(language)
+
+        tags = [d['label'] for d in tags_dict if d['id'] in movie['tags']]
+
+        if action == 'update':
+            return {'radarrId': int(movie["id"]),
+                    'title': movie["title"],
+                    'path': movie["path"] + separator + movie['movieFile']['relativePath'],
+                    'tmdbId': str(movie["tmdbId"]),
+                    'poster': poster,
+                    'fanart': fanart,
+                    'audio_language': str(audio_language),
+                    'sceneName': sceneName,
+                    'monitored': str(bool(movie['monitored'])),
+                    'year': str(movie['year']),
+                    'sortTitle': movie['sortTitle'],
+                    'alternativeTitles': alternativeTitles,
+                    'format': format,
+                    'resolution': resolution,
+                    'video_codec': videoCodec,
+                    'audio_codec': audioCodec,
+                    'overview': overview,
+                    'imdbId': imdbId,
+                    'movie_file_id': int(movie['movieFile']['id']),
+                    'tags': str(tags),
+                    'file_size': movie['movieFile']['size']}
+        else:
+            return {'radarrId': int(movie["id"]),
+                    'title': movie["title"],
+                    'path': movie["path"] + separator + movie['movieFile']['relativePath'],
+                    'tmdbId': str(movie["tmdbId"]),
+                    'subtitles': '[]',
+                    'overview': overview,
+                    'poster': poster,
+                    'fanart': fanart,
+                    'audio_language': str(audio_language),
+                    'sceneName': sceneName,
+                    'monitored': str(bool(movie['monitored'])),
+                    'sortTitle': movie['sortTitle'],
+                    'year': str(movie['year']),
+                    'alternativeTitles': alternativeTitles,
+                    'format': format,
+                    'resolution': resolution,
+                    'video_codec': videoCodec,
+                    'audio_codec': audioCodec,
+                    'imdbId': imdbId,
+                    'movie_file_id': int(movie['movieFile']['id']),
+                    'tags': str(tags),
+                    'profileId': movie_default_profile,
+                    'file_size': movie['movieFile']['size']}
+
+
+def profile_id_to_language(id, profiles):
+    for profile in profiles:
+        profiles_to_return = []
+        if id == profile[0]:
+            profiles_to_return.append(profile[1])
+    return profiles_to_return

bazarr/radarr/sync/utils.py

@@ -0,0 +1,99 @@
+# coding=utf-8
+
+import requests
+import logging
+
+from app.config import settings
+from radarr.info import get_radarr_info, url_radarr
+from constants import headers
+
+
+def get_profile_list():
+    apikey_radarr = settings.radarr.apikey
+    profiles_list = []
+    # Get profiles data from radarr
+    if get_radarr_info.is_legacy():
+        url_radarr_api_movies = url_radarr() + "/api/profile?apikey=" + apikey_radarr
+    else:
+        url_radarr_api_movies = url_radarr() + "/api/v3/qualityprofile?apikey=" + apikey_radarr
+
+    try:
+        profiles_json = requests.get(url_radarr_api_movies, timeout=60, verify=False, headers=headers)
+    except requests.exceptions.ConnectionError:
+        logging.exception("BAZARR Error trying to get profiles from Radarr. Connection Error.")
+    except requests.exceptions.Timeout:
+        logging.exception("BAZARR Error trying to get profiles from Radarr. Timeout Error.")
+    except requests.exceptions.RequestException:
+        logging.exception("BAZARR Error trying to get profiles from Radarr.")
+    else:
+        # Parsing data returned from radarr
+        if get_radarr_info.is_legacy():
+            for profile in profiles_json.json():
+                profiles_list.append([profile['id'], profile['language'].capitalize()])
+        else:
+            for profile in profiles_json.json():
+                profiles_list.append([profile['id'], profile['language']['name'].capitalize()])
+
+        return profiles_list
+
+    return None
+
+
+def get_tags():
+    apikey_radarr = settings.radarr.apikey
+    tagsDict = []
+
+    # Get tags data from Radarr
+    if get_radarr_info.is_legacy():
+        url_radarr_api_series = url_radarr() + "/api/tag?apikey=" + apikey_radarr
+    else:
+        url_radarr_api_series = url_radarr() + "/api/v3/tag?apikey=" + apikey_radarr
+
+    try:
+        tagsDict = requests.get(url_radarr_api_series, timeout=60, verify=False, headers=headers)
+    except requests.exceptions.ConnectionError:
+        logging.exception("BAZARR Error trying to get tags from Radarr. Connection Error.")
+        return []
+    except requests.exceptions.Timeout:
+        logging.exception("BAZARR Error trying to get tags from Radarr. Timeout Error.")
+        return []
+    except requests.exceptions.RequestException:
+        logging.exception("BAZARR Error trying to get tags from Radarr.")
+        return []
+    except requests.exceptions.HTTPError:
+        logging.exception("BAZARR Exception while trying to get tags from Radarr.")
+        return []
+    else:
+        try:
+            return tagsDict.json()
+        except Exception:
+            return []
+
+
+def get_movies_from_radarr_api(url, apikey_radarr, radarr_id=None):
+    if get_radarr_info.is_legacy():
+        url_radarr_api_movies = url + "/api/movie" + ("/{}".format(radarr_id) if radarr_id else "") + "?apikey=" + \
+                                apikey_radarr
+    else:
+        url_radarr_api_movies = url + "/api/v3/movie" + ("/{}".format(radarr_id) if radarr_id else "") + "?apikey=" + \
+                                apikey_radarr
+
+    try:
+        r = requests.get(url_radarr_api_movies, timeout=60, verify=False, headers=headers)
+        if r.status_code == 404:
+            return
+        r.raise_for_status()
+    except requests.exceptions.HTTPError:
+        logging.exception("BAZARR Error trying to get movies from Radarr. Http error.")
+        return
+    except requests.exceptions.ConnectionError:
+        logging.exception("BAZARR Error trying to get movies from Radarr. Connection Error.")
+        return
+    except requests.exceptions.Timeout:
+        logging.exception("BAZARR Error trying to get movies from Radarr. Timeout Error.")
+        return
+    except requests.exceptions.RequestException:
+        logging.exception("BAZARR Error trying to get movies from Radarr.")
+        return
+    else:
+        return r.json()

bazarr/sonarr/__init__.py

@@ -0,0 +1 @@
+# coding=utf-8

bazarr/sonarr/blacklist.py

@@ -0,0 +1,40 @@
+# coding=utf-8
+
+import time
+
+from app.database import TableBlacklist
+from app.event_handler import event_stream
+
+
+def get_blacklist():
+    blacklist_db = TableBlacklist.select(TableBlacklist.provider, TableBlacklist.subs_id).dicts()
+
+    blacklist_list = []
+    for item in blacklist_db:
+        blacklist_list.append((item['provider'], item['subs_id']))
+
+    return blacklist_list
+
+
+def blacklist_log(sonarr_series_id, sonarr_episode_id, provider, subs_id, language):
+    TableBlacklist.insert({
+        TableBlacklist.sonarr_series_id: sonarr_series_id,
+        TableBlacklist.sonarr_episode_id: sonarr_episode_id,
+        TableBlacklist.timestamp: time.time(),
+        TableBlacklist.provider: provider,
+        TableBlacklist.subs_id: subs_id,
+        TableBlacklist.language: language
+    }).execute()
+    event_stream(type='episode-blacklist')
+
+
+def blacklist_delete(provider, subs_id):
+    TableBlacklist.delete().where((TableBlacklist.provider == provider) and
+                                  (TableBlacklist.subs_id == subs_id))\
+        .execute()
+    event_stream(type='episode-blacklist', action='delete')
+
+
+def blacklist_delete_all():
+    TableBlacklist.delete().execute()
+    event_stream(type='episode-blacklist', action='delete')

bazarr/sonarr/filesystem.py

@@ -0,0 +1,38 @@
+# coding=utf-8
+
+import requests
+import logging
+
+from app.config import settings
+from sonarr.info import get_sonarr_info, url_sonarr
+from constants import headers
+
+
+def browse_sonarr_filesystem(path='#'):
+    if path == '#':
+        path = ''
+    if get_sonarr_info.is_legacy():
+        url_sonarr_api_filesystem = url_sonarr() + "/api/filesystem?path=" + path + \
+                                    "&allowFoldersWithoutTrailingSlashes=true&includeFiles=false&apikey=" + \
+                                    settings.sonarr.apikey
+    else:
+        url_sonarr_api_filesystem = url_sonarr() + "/api/v3/filesystem?path=" + path + \
+                                    "&allowFoldersWithoutTrailingSlashes=true&includeFiles=false&apikey=" + \
+                                    settings.sonarr.apikey
+    try:
+        r = requests.get(url_sonarr_api_filesystem, timeout=60, verify=False, headers=headers)
+        r.raise_for_status()
+    except requests.exceptions.HTTPError:
+        logging.exception("BAZARR Error trying to get series from Sonarr. Http error.")
+        return
+    except requests.exceptions.ConnectionError:
+        logging.exception("BAZARR Error trying to get series from Sonarr. Connection Error.")
+        return
+    except requests.exceptions.Timeout:
+        logging.exception("BAZARR Error trying to get series from Sonarr. Timeout Error.")
+        return
+    except requests.exceptions.RequestException:
+        logging.exception("BAZARR Error trying to get series from Sonarr.")
+        return
+
+    return r.json()

bazarr/sonarr/history.py

@@ -0,0 +1,24 @@
+# coding=utf-8
+
+import time
+
+from app.database import TableHistory
+from app.event_handler import event_stream
+
+
+def history_log(action, sonarr_series_id, sonarr_episode_id, description, video_path=None, language=None, provider=None,
+                score=None, subs_id=None, subtitles_path=None):
+    TableHistory.insert({
+        TableHistory.action: action,
+        TableHistory.sonarrSeriesId: sonarr_series_id,
+        TableHistory.sonarrEpisodeId: sonarr_episode_id,
+        TableHistory.timestamp: time.time(),
+        TableHistory.description: description,
+        TableHistory.video_path: video_path,
+        TableHistory.language: language,
+        TableHistory.provider: provider,
+        TableHistory.score: score,
+        TableHistory.subs_id: subs_id,
+        TableHistory.subtitles_path: subtitles_path
+    }).execute()
+    event_stream(type='episode-history')

bazarr/sonarr/info.py

@@ -0,0 +1,84 @@
+# coding=utf-8
+
+import logging
+import requests
+import datetime
+import json
+
+from dogpile.cache import make_region
+
+from app.config import settings, empty_values
+from constants import headers
+
+region = make_region().configure('dogpile.cache.memory')
+
+
+class GetSonarrInfo:
+    @staticmethod
+    def version():
+        """
+        Call system/status API endpoint and get the Sonarr version
+        @return: str
+        """
+        sonarr_version = region.get("sonarr_version", expiration_time=datetime.timedelta(seconds=60).total_seconds())
+        if sonarr_version:
+            region.set("sonarr_version", sonarr_version)
+            return sonarr_version
+        else:
+            sonarr_version = ''
+        if settings.general.getboolean('use_sonarr'):
+            try:
+                sv = url_sonarr() + "/api/system/status?apikey=" + settings.sonarr.apikey
+                sonarr_json = requests.get(sv, timeout=60, verify=False, headers=headers).json()
+                if 'version' in sonarr_json:
+                    sonarr_version = sonarr_json['version']
+                else:
+                    raise json.decoder.JSONDecodeError
+            except json.decoder.JSONDecodeError:
+                try:
+                    sv = url_sonarr() + "/api/v3/system/status?apikey=" + settings.sonarr.apikey
+                    sonarr_version = requests.get(sv, timeout=60, verify=False, headers=headers).json()['version']
+                except json.decoder.JSONDecodeError:
+                    logging.debug('BAZARR cannot get Sonarr version')
+                    sonarr_version = 'unknown'
+            except Exception:
+                logging.debug('BAZARR cannot get Sonarr version')
+                sonarr_version = 'unknown'
+        logging.debug('BAZARR got this Sonarr version from its API: {}'.format(sonarr_version))
+        region.set("sonarr_version", sonarr_version)
+        return sonarr_version
+
+    def is_legacy(self):
+        """
+        Call self.version() and parse the result to determine if it's a legacy version of Sonarr API
+        @return: bool
+        """
+        sonarr_version = self.version()
+        if sonarr_version.startswith(('0.', '2.')):
+            return True
+        else:
+            return False
+
+
+get_sonarr_info = GetSonarrInfo()
+
+
+def url_sonarr():
+    if settings.sonarr.getboolean('ssl'):
+        protocol_sonarr = "https"
+    else:
+        protocol_sonarr = "http"
+
+    if settings.sonarr.base_url == '':
+        settings.sonarr.base_url = "/"
+    if not settings.sonarr.base_url.startswith("/"):
+        settings.sonarr.base_url = "/" + settings.sonarr.base_url
+    if settings.sonarr.base_url.endswith("/"):
+        settings.sonarr.base_url = settings.sonarr.base_url[:-1]
+
+    if settings.sonarr.port in empty_values:
+        port = ""
+    else:
+        port = f":{settings.sonarr.port}"
+
+    return f"{protocol_sonarr}://{settings.sonarr.ip}{port}{settings.sonarr.base_url}"

bazarr/sonarr/notify.py

@@ -0,0 +1,23 @@
+# coding=utf-8
+
+import logging
+import requests
+
+from app.config import settings
+from sonarr.info import get_sonarr_info, url_sonarr
+from constants import headers
+
+
+def notify_sonarr(sonarr_series_id):
+    try:
+        if get_sonarr_info.is_legacy():
+            url = url_sonarr() + "/api/command?apikey=" + settings.sonarr.apikey
+        else:
+            url = url_sonarr() + "/api/v3/command?apikey=" + settings.sonarr.apikey
+        data = {
+            'name': 'RescanSeries',
+            'seriesId': int(sonarr_series_id)
+        }
+        requests.post(url, json=data, timeout=60, verify=False, headers=headers)
+    except Exception:
+        logging.exception('BAZARR cannot notify Sonarr')

bazarr/sonarr/rootfolder.py

@@ -0,0 +1,85 @@
+# coding=utf-8
+
+import os
+import requests
+import logging
+
+from app.config import settings
+from app.database import TableShowsRootfolder, TableShows
+from utilities.path_mappings import path_mappings
+from sonarr.info import get_sonarr_info, url_sonarr
+from constants import headers
+
+
+def get_sonarr_rootfolder():
+    apikey_sonarr = settings.sonarr.apikey
+    sonarr_rootfolder = []
+
+    # Get root folder data from Sonarr
+    if get_sonarr_info.is_legacy():
+        url_sonarr_api_rootfolder = url_sonarr() + "/api/rootfolder?apikey=" + apikey_sonarr
+    else:
+        url_sonarr_api_rootfolder = url_sonarr() + "/api/v3/rootfolder?apikey=" + apikey_sonarr
+
+    try:
+        rootfolder = requests.get(url_sonarr_api_rootfolder, timeout=60, verify=False, headers=headers)
+    except requests.exceptions.ConnectionError:
+        logging.exception("BAZARR Error trying to get rootfolder from Sonarr. Connection Error.")
+        return []
+    except requests.exceptions.Timeout:
+        logging.exception("BAZARR Error trying to get rootfolder from Sonarr. Timeout Error.")
+        return []
+    except requests.exceptions.RequestException:
+        logging.exception("BAZARR Error trying to get rootfolder from Sonarr.")
+        return []
+    else:
+        sonarr_movies_paths = list(TableShows.select(TableShows.path).dicts())
+        for folder in rootfolder.json():
+            if any(item['path'].startswith(folder['path']) for item in sonarr_movies_paths):
+                sonarr_rootfolder.append({'id': folder['id'], 'path': folder['path']})
+        db_rootfolder = TableShowsRootfolder.select(TableShowsRootfolder.id, TableShowsRootfolder.path).dicts()
+        rootfolder_to_remove = [x for x in db_rootfolder if not
+                                next((item for item in sonarr_rootfolder if item['id'] == x['id']), False)]
+        rootfolder_to_update = [x for x in sonarr_rootfolder if
+                                next((item for item in db_rootfolder if item['id'] == x['id']), False)]
+        rootfolder_to_insert = [x for x in sonarr_rootfolder if not
+                                next((item for item in db_rootfolder if item['id'] == x['id']), False)]
+
+        for item in rootfolder_to_remove:
+            TableShowsRootfolder.delete().where(TableShowsRootfolder.id == item['id']).execute()
+        for item in rootfolder_to_update:
+            TableShowsRootfolder.update({TableShowsRootfolder.path: item['path']})\
+                .where(TableShowsRootfolder.id == item['id'])\
+                .execute()
+        for item in rootfolder_to_insert:
+            TableShowsRootfolder.insert({TableShowsRootfolder.id: item['id'], TableShowsRootfolder.path: item['path']})\
+                .execute()
+
+
+def check_sonarr_rootfolder():
+    get_sonarr_rootfolder()
+    rootfolder = TableShowsRootfolder.select(TableShowsRootfolder.id, TableShowsRootfolder.path).dicts()
+    for item in rootfolder:
+        root_path = item['path']
+        if not root_path.endswith(('/', '\\')):
+            if root_path.startswith('/'):
+                root_path += '/'
+            else:
+                root_path += '\\'
+        if not os.path.isdir(path_mappings.path_replace(root_path)):
+            TableShowsRootfolder.update({TableShowsRootfolder.accessible: 0,
+                                         TableShowsRootfolder.error: 'This Sonarr root directory does not seems to '
+                                                                     'be accessible by  Please check path '
+                                                                     'mapping.'})\
+                .where(TableShowsRootfolder.id == item['id'])\
+                .execute()
+        elif not os.access(path_mappings.path_replace(root_path), os.W_OK):
+            TableShowsRootfolder.update({TableShowsRootfolder.accessible: 0,
+                                         TableShowsRootfolder.error: 'Bazarr cannot write to this directory.'}) \
+                .where(TableShowsRootfolder.id == item['id']) \
+                .execute()
+        else:
+            TableShowsRootfolder.update({TableShowsRootfolder.accessible: 1,
+                                         TableShowsRootfolder.error: ''}) \
+                .where(TableShowsRootfolder.id == item['id']) \
+                .execute()

bazarr/sonarr/sync/__init__.py

@@ -0,0 +1 @@
+# coding=utf-8

bazarr/sonarr/sync/converter.py

@@ -0,0 +1,46 @@
+# coding=utf-8
+
+def SonarrFormatAudioCodec(audio_codec):
+    if type(audio_codec) is not str:
+        return audio_codec
+    else:
+        if audio_codec == 'AC-3':
+            return 'AC3'
+        elif audio_codec == 'E-AC-3':
+            return 'EAC3'
+        elif audio_codec == 'MPEG Audio':
+            return 'MP3'
+        else:
+            return audio_codec
+
+
+def SonarrFormatVideoCodec(video_codec):
+    if type(video_codec) is not str:
+        return video_codec
+    else:
+        if video_codec == 'x264' or video_codec == 'AVC':
+            return 'h264'
+        elif video_codec == 'x265' or video_codec == 'HEVC':
+            return 'h265'
+        elif video_codec.startswith('XviD'):
+            return 'XviD'
+        elif video_codec.startswith('DivX'):
+            return 'DivX'
+        elif video_codec == 'MPEG-1 Video':
+            return 'Mpeg'
+        elif video_codec == 'MPEG-2 Video':
+            return 'Mpeg2'
+        elif video_codec == 'MPEG-4 Video':
+            return 'Mpeg4'
+        elif video_codec == 'VC-1':
+            return 'VC1'
+        elif video_codec.endswith('VP6'):
+            return 'VP6'
+        elif video_codec.endswith('VP7'):
+            return 'VP7'
+        elif video_codec.endswith('VP8'):
+            return 'VP8'
+        elif video_codec.endswith('VP9'):
+            return 'VP9'
+        else:
+            return video_codec

bazarr/sonarr/sync/episodes.py

@@ -1,19 +1,19 @@
 # coding=utf-8
 
-import os
-import requests
 import logging
+
 from peewee import IntegrityError
 
-from database import TableEpisodes, TableShows
-from config import settings, url_sonarr
-from helper import path_mappings
-from list_subtitles import store_subtitles, series_full_scan_subtitles
-from get_subtitle.mass_download import episode_download_subtitles
-from event_handler import event_stream, show_progress, hide_progress
-from utils import get_sonarr_info
+from app.database import TableEpisodes
+from app.config import settings
+from utilities.path_mappings import path_mappings
+from subtitles.indexer.series import store_subtitles, series_full_scan_subtitles
+from subtitles.mass_download import episode_download_subtitles
+from app.event_handler import event_stream, show_progress, hide_progress
+from sonarr.info import get_sonarr_info, url_sonarr
 
-headers = {"User-Agent": os.environ["SZ_USER_AGENT"]}
+from .parser import episodeParser
+from .utils import get_series_from_sonarr_api, get_episodes_from_sonarr_api, get_episodesFiles_from_sonarr_api
 
 
 def update_all_episodes():
@@ -40,7 +40,7 @@ def sync_episodes(series_id=None, send_event=True):
     altered_episodes = []
 
     # Get sonarrId for each series from database
-    seriesIdList = get_series_from_sonarr_api(series_id=series_id, url=url_sonarr(), apikey_sonarr=apikey_sonarr,)
+    seriesIdList = get_series_from_sonarr_api(url=url_sonarr(), apikey_sonarr=apikey_sonarr, sonarr_series_id=series_id)
 
     series_count = len(seriesIdList)
     for i, seriesId in enumerate(seriesIdList):
@@ -53,14 +53,14 @@ def sync_episodes(series_id=None, send_event=True):
 
         # Get episodes data for a series from Sonarr
         episodes = get_episodes_from_sonarr_api(url=url_sonarr(), apikey_sonarr=apikey_sonarr,
-                                                series_id=seriesId['sonarrSeriesId'])
+                                                series_id=seriesId['id'])
         if not episodes:
             continue
         else:
             # For Sonarr v3, we need to update episodes to integrate the episodeFile API endpoint results
             if not get_sonarr_info.is_legacy():
                 episodeFiles = get_episodesFiles_from_sonarr_api(url=url_sonarr(), apikey_sonarr=apikey_sonarr,
-                                                                 series_id=seriesId['sonarrSeriesId'])
+                                                                 series_id=seriesId['id'])
                 for episode in episodes:
                     if episode['hasFile']:
                         item = [x for x in episodeFiles if x['id'] == episode['episodeFileId']]
@@ -246,203 +246,3 @@ def sync_one_episode(episode_id, defer_search=False):
         logging.debug('BAZARR downloading missing subtitles for this episode: {}'.format(path_mappings.path_replace(
             episode['path'])))
         episode_download_subtitles(episode_id)
-
-
-def SonarrFormatAudioCodec(audio_codec):
-    if type(audio_codec) is not str:
-        return audio_codec
-    else:
-        if audio_codec == 'AC-3':
-            return 'AC3'
-        elif audio_codec == 'E-AC-3':
-            return 'EAC3'
-        elif audio_codec == 'MPEG Audio':
-            return 'MP3'
-        else:
-            return audio_codec
-
-
-def SonarrFormatVideoCodec(video_codec):
-    if type(video_codec) is not str:
-        return video_codec
-    else:
-        if video_codec == 'x264' or video_codec == 'AVC':
-            return 'h264'
-        elif video_codec == 'x265' or video_codec == 'HEVC':
-            return 'h265'
-        elif video_codec.startswith('XviD'):
-            return 'XviD'
-        elif video_codec.startswith('DivX'):
-            return 'DivX'
-        elif video_codec == 'MPEG-1 Video':
-            return 'Mpeg'
-        elif video_codec == 'MPEG-2 Video':
-            return 'Mpeg2'
-        elif video_codec == 'MPEG-4 Video':
-            return 'Mpeg4'
-        elif video_codec == 'VC-1':
-            return 'VC1'
-        elif video_codec.endswith('VP6'):
-            return 'VP6'
-        elif video_codec.endswith('VP7'):
-            return 'VP7'
-        elif video_codec.endswith('VP8'):
-            return 'VP8'
-        elif video_codec.endswith('VP9'):
-            return 'VP9'
-        else:
-            return video_codec
-
-
-def episodeParser(episode):
-    if 'hasFile' in episode:
-        if episode['hasFile'] is True:
-            if 'episodeFile' in episode:
-                if episode['episodeFile']['size'] > 20480:
-                    if 'sceneName' in episode['episodeFile']:
-                        sceneName = episode['episodeFile']['sceneName']
-                    else:
-                        sceneName = None
-
-                    audio_language = []
-                    if 'language' in episode['episodeFile'] and len(episode['episodeFile']['language']):
-                        item = episode['episodeFile']['language']
-                        if isinstance(item, dict):
-                            if 'name' in item:
-                                audio_language.append(item['name'])
-                    else:
-                        audio_language = TableShows.get(TableShows.sonarrSeriesId == episode['seriesId']).audio_language
-
-                    if 'mediaInfo' in episode['episodeFile']:
-                        if 'videoCodec' in episode['episodeFile']['mediaInfo']:
-                            videoCodec = episode['episodeFile']['mediaInfo']['videoCodec']
-                            videoCodec = SonarrFormatVideoCodec(videoCodec)
-                        else:
-                            videoCodec = None
-
-                        if 'audioCodec' in episode['episodeFile']['mediaInfo']:
-                            audioCodec = episode['episodeFile']['mediaInfo']['audioCodec']
-                            audioCodec = SonarrFormatAudioCodec(audioCodec)
-                        else:
-                            audioCodec = None
-                    else:
-                        videoCodec = None
-                        audioCodec = None
-
-                    try:
-                        video_format, video_resolution = episode['episodeFile']['quality']['quality']['name'].split('-')
-                    except Exception:
-                        video_format = episode['episodeFile']['quality']['quality']['name']
-                        try:
-                            video_resolution = str(episode['episodeFile']['quality']['quality']['resolution']) + 'p'
-                        except Exception:
-                            video_resolution = None
-
-                    return {'sonarrSeriesId': episode['seriesId'],
-                            'sonarrEpisodeId': episode['id'],
-                            'title': episode['title'],
-                            'path': episode['episodeFile']['path'],
-                            'season': episode['seasonNumber'],
-                            'episode': episode['episodeNumber'],
-                            'scene_name': sceneName,
-                            'monitored': str(bool(episode['monitored'])),
-                            'format': video_format,
-                            'resolution': video_resolution,
-                            'video_codec': videoCodec,
-                            'audio_codec': audioCodec,
-                            'episode_file_id': episode['episodeFile']['id'],
-                            'audio_language': str(audio_language),
-                            'file_size': episode['episodeFile']['size']}
-
-
-def get_series_from_sonarr_api(series_id, url, apikey_sonarr):
-    if series_id:
-        url_sonarr_api_series = url + "/api/{0}series/{1}?apikey={2}".format(
-            '' if get_sonarr_info.is_legacy() else 'v3/', series_id, apikey_sonarr)
-    else:
-        url_sonarr_api_series = url + "/api/{0}series?apikey={1}".format(
-            '' if get_sonarr_info.is_legacy() else 'v3/', apikey_sonarr)
-    try:
-        r = requests.get(url_sonarr_api_series, timeout=60, verify=False, headers=headers)
-        r.raise_for_status()
-    except requests.exceptions.HTTPError as e:
-        if e.response.status_code:
-            raise requests.exceptions.HTTPError
-        logging.exception("BAZARR Error trying to get series from Sonarr. Http error.")
-        return
-    except requests.exceptions.ConnectionError:
-        logging.exception("BAZARR Error trying to get series from Sonarr. Connection Error.")
-        return
-    except requests.exceptions.Timeout:
-        logging.exception("BAZARR Error trying to get series from Sonarr. Timeout Error.")
-        return
-    except requests.exceptions.RequestException:
-        logging.exception("BAZARR Error trying to get series from Sonarr.")
-        return
-    else:
-        series_json = []
-        if series_id:
-            series_json.append(r.json())
-        else:
-            series_json = r.json()
-        series_list = []
-        for series in series_json:
-            series_list.append({'sonarrSeriesId': series['id'], 'title': series['title']})
-        return series_list
-
-
-def get_episodes_from_sonarr_api(url, apikey_sonarr, series_id=None, episode_id=None):
-    if series_id:
-        url_sonarr_api_episode = url + "/api/{0}episode?seriesId={1}&apikey={2}".format(
-            '' if get_sonarr_info.is_legacy() else 'v3/', series_id, apikey_sonarr)
-    elif episode_id:
-        url_sonarr_api_episode = url + "/api/{0}episode/{1}?apikey={2}".format(
-            '' if get_sonarr_info.is_legacy() else 'v3/', episode_id, apikey_sonarr)
-    else:
-        return
-
-    try:
-        r = requests.get(url_sonarr_api_episode, timeout=60, verify=False, headers=headers)
-        r.raise_for_status()
-    except requests.exceptions.HTTPError:
-        logging.exception("BAZARR Error trying to get episodes from Sonarr. Http error.")
-        return
-    except requests.exceptions.ConnectionError:
-        logging.exception("BAZARR Error trying to get episodes from Sonarr. Connection Error.")
-        return
-    except requests.exceptions.Timeout:
-        logging.exception("BAZARR Error trying to get episodes from Sonarr. Timeout Error.")
-        return
-    except requests.exceptions.RequestException:
-        logging.exception("BAZARR Error trying to get episodes from Sonarr.")
-        return
-    else:
-        return r.json()
-
-
-def get_episodesFiles_from_sonarr_api(url, apikey_sonarr, series_id=None, episode_file_id=None):
-    if series_id:
-        url_sonarr_api_episodeFiles = url + "/api/v3/episodeFile?seriesId={0}&apikey={1}".format(series_id,
-                                                                                                 apikey_sonarr)
-    elif episode_file_id:
-        url_sonarr_api_episodeFiles = url + "/api/v3/episodeFile/{0}?apikey={1}".format(episode_file_id, apikey_sonarr)
-    else:
-        return
-
-    try:
-        r = requests.get(url_sonarr_api_episodeFiles, timeout=60, verify=False, headers=headers)
-        r.raise_for_status()
-    except requests.exceptions.HTTPError:
-        logging.exception("BAZARR Error trying to get episodeFiles from Sonarr. Http error.")
-        return
-    except requests.exceptions.ConnectionError:
-        logging.exception("BAZARR Error trying to get episodeFiles from Sonarr. Connection Error.")
-        return
-    except requests.exceptions.Timeout:
-        logging.exception("BAZARR Error trying to get episodeFiles from Sonarr. Timeout Error.")
-        return
-    except requests.exceptions.RequestException:
-        logging.exception("BAZARR Error trying to get episodeFiles from Sonarr.")
-        return
-    else:
-        return r.json()

bazarr/sonarr/sync/parser.py

@@ -0,0 +1,136 @@
+# coding=utf-8
+
+import os
+
+from app.database import TableShows
+from sonarr.info import get_sonarr_info
+
+from .converter import SonarrFormatVideoCodec, SonarrFormatAudioCodec
+
+
+def seriesParser(show, action, tags_dict, serie_default_profile, audio_profiles):
+    overview = show['overview'] if 'overview' in show else ''
+    poster = ''
+    fanart = ''
+    for image in show['images']:
+        if image['coverType'] == 'poster':
+            poster_big = image['url'].split('?')[0]
+            poster = os.path.splitext(poster_big)[0] + '-250' + os.path.splitext(poster_big)[1]
+
+        if image['coverType'] == 'fanart':
+            fanart = image['url'].split('?')[0]
+
+    alternate_titles = None
+    if show['alternateTitles'] is not None:
+        alternate_titles = str([item['title'] for item in show['alternateTitles']])
+
+    audio_language = []
+    if get_sonarr_info.is_legacy():
+        audio_language = profile_id_to_language(show['qualityProfileId'], audio_profiles)
+    else:
+        audio_language = profile_id_to_language(show['languageProfileId'], audio_profiles)
+
+    tags = [d['label'] for d in tags_dict if d['id'] in show['tags']]
+
+    imdbId = show['imdbId'] if 'imdbId' in show else None
+
+    if action == 'update':
+        return {'title': show["title"],
+                'path': show["path"],
+                'tvdbId': int(show["tvdbId"]),
+                'sonarrSeriesId': int(show["id"]),
+                'overview': overview,
+                'poster': poster,
+                'fanart': fanart,
+                'audio_language': str(audio_language),
+                'sortTitle': show['sortTitle'],
+                'year': str(show['year']),
+                'alternateTitles': alternate_titles,
+                'tags': str(tags),
+                'seriesType': show['seriesType'],
+                'imdbId': imdbId}
+    else:
+        return {'title': show["title"],
+                'path': show["path"],
+                'tvdbId': show["tvdbId"],
+                'sonarrSeriesId': show["id"],
+                'overview': overview,
+                'poster': poster,
+                'fanart': fanart,
+                'audio_language': str(audio_language),
+                'sortTitle': show['sortTitle'],
+                'year': str(show['year']),
+                'alternateTitles': alternate_titles,
+                'tags': str(tags),
+                'seriesType': show['seriesType'],
+                'imdbId': imdbId,
+                'profileId': serie_default_profile}
+
+
+def profile_id_to_language(id_, profiles):
+    profiles_to_return = []
+    for profile in profiles:
+        if id_ == profile[0]:
+            profiles_to_return.append(profile[1])
+    return profiles_to_return
+
+
+def episodeParser(episode):
+    if 'hasFile' in episode:
+        if episode['hasFile'] is True:
+            if 'episodeFile' in episode:
+                if episode['episodeFile']['size'] > 20480:
+                    if 'sceneName' in episode['episodeFile']:
+                        sceneName = episode['episodeFile']['sceneName']
+                    else:
+                        sceneName = None
+
+                    audio_language = []
+                    if 'language' in episode['episodeFile'] and len(episode['episodeFile']['language']):
+                        item = episode['episodeFile']['language']
+                        if isinstance(item, dict):
+                            if 'name' in item:
+                                audio_language.append(item['name'])
+                    else:
+                        audio_language = TableShows.get(TableShows.sonarrSeriesId == episode['seriesId']).audio_language
+
+                    if 'mediaInfo' in episode['episodeFile']:
+                        if 'videoCodec' in episode['episodeFile']['mediaInfo']:
+                            videoCodec = episode['episodeFile']['mediaInfo']['videoCodec']
+                            videoCodec = SonarrFormatVideoCodec(videoCodec)
+                        else:
+                            videoCodec = None
+
+                        if 'audioCodec' in episode['episodeFile']['mediaInfo']:
+                            audioCodec = episode['episodeFile']['mediaInfo']['audioCodec']
+                            audioCodec = SonarrFormatAudioCodec(audioCodec)
+                        else:
+                            audioCodec = None
+                    else:
+                        videoCodec = None
+                        audioCodec = None
+
+                    try:
+                        video_format, video_resolution = episode['episodeFile']['quality']['quality']['name'].split('-')
+                    except Exception:
+                        video_format = episode['episodeFile']['quality']['quality']['name']
+                        try:
+                            video_resolution = str(episode['episodeFile']['quality']['quality']['resolution']) + 'p'
+                        except Exception:
+                            video_resolution = None
+
+                    return {'sonarrSeriesId': episode['seriesId'],
+                            'sonarrEpisodeId': episode['id'],
+                            'title': episode['title'],
+                            'path': episode['episodeFile']['path'],
+                            'season': episode['seasonNumber'],
+                            'episode': episode['episodeNumber'],
+                            'scene_name': sceneName,
+                            'monitored': str(bool(episode['monitored'])),
+                            'format': video_format,
+                            'resolution': video_resolution,
+                            'video_codec': videoCodec,
+                            'audio_codec': audioCodec,
+                            'episode_file_id': episode['episodeFile']['id'],
+                            'audio_language': str(audio_language),
+                            'file_size': episode['episodeFile']['size']}

bazarr/sonarr/sync/series.py

@@ -1,20 +1,20 @@
 # coding=utf-8
 
-import os
-import requests
 import logging
+
 from peewee import IntegrityError
 
-from config import settings, url_sonarr
-from list_subtitles import list_missing_subtitles
-from get_rootfolder import check_sonarr_rootfolder
-from database import TableShows, TableEpisodes
-from get_episodes import sync_episodes
-from utils import get_sonarr_info
-from helper import path_mappings
-from event_handler import event_stream, show_progress, hide_progress
+from app.config import settings
+from sonarr.info import url_sonarr
+from subtitles.indexer.series import list_missing_subtitles
+from sonarr.rootfolder import check_sonarr_rootfolder
+from app.database import TableShows, TableEpisodes
+from utilities.path_mappings import path_mappings
+from app.event_handler import event_stream, show_progress, hide_progress
 
-headers = {"User-Agent": os.environ["SZ_USER_AGENT"]}
+from .episodes import sync_episodes
+from .parser import seriesParser
+from .utils import get_profile_list, get_tags, get_series_from_sonarr_api
 
 
 def update_series(send_event=True):
@@ -214,152 +214,3 @@ def update_one_series(series_id, action):
             event_stream(type='series', action='update', payload=int(series_id))
             logging.debug('BAZARR inserted this series into the database:{}'.format(path_mappings.path_replace(
                 series['path'])))
-
-
-def get_profile_list():
-    apikey_sonarr = settings.sonarr.apikey
-    profiles_list = []
-
-    # Get profiles data from Sonarr
-    if get_sonarr_info.is_legacy():
-        url_sonarr_api_series = url_sonarr() + "/api/profile?apikey=" + apikey_sonarr
-    else:
-        url_sonarr_api_series = url_sonarr() + "/api/v3/languageprofile?apikey=" + apikey_sonarr
-
-    try:
-        profiles_json = requests.get(url_sonarr_api_series, timeout=60, verify=False, headers=headers)
-    except requests.exceptions.ConnectionError:
-        logging.exception("BAZARR Error trying to get profiles from Sonarr. Connection Error.")
-        return None
-    except requests.exceptions.Timeout:
-        logging.exception("BAZARR Error trying to get profiles from Sonarr. Timeout Error.")
-        return None
-    except requests.exceptions.RequestException:
-        logging.exception("BAZARR Error trying to get profiles from Sonarr.")
-        return None
-
-    # Parsing data returned from Sonarr
-    if get_sonarr_info.is_legacy():
-        for profile in profiles_json.json():
-            profiles_list.append([profile['id'], profile['language'].capitalize()])
-    else:
-        for profile in profiles_json.json():
-            profiles_list.append([profile['id'], profile['name'].capitalize()])
-
-    return profiles_list
-
-
-def profile_id_to_language(id_, profiles):
-    profiles_to_return = []
-    for profile in profiles:
-        if id_ == profile[0]:
-            profiles_to_return.append(profile[1])
-    return profiles_to_return
-
-
-def get_tags():
-    apikey_sonarr = settings.sonarr.apikey
-    tagsDict = []
-
-    # Get tags data from Sonarr
-    if get_sonarr_info.is_legacy():
-        url_sonarr_api_series = url_sonarr() + "/api/tag?apikey=" + apikey_sonarr
-    else:
-        url_sonarr_api_series = url_sonarr() + "/api/v3/tag?apikey=" + apikey_sonarr
-
-    try:
-        tagsDict = requests.get(url_sonarr_api_series, timeout=60, verify=False, headers=headers)
-    except requests.exceptions.ConnectionError:
-        logging.exception("BAZARR Error trying to get tags from Sonarr. Connection Error.")
-        return []
-    except requests.exceptions.Timeout:
-        logging.exception("BAZARR Error trying to get tags from Sonarr. Timeout Error.")
-        return []
-    except requests.exceptions.RequestException:
-        logging.exception("BAZARR Error trying to get tags from Sonarr.")
-        return []
-    else:
-        return tagsDict.json()
-
-
-def seriesParser(show, action, tags_dict, serie_default_profile, audio_profiles):
-    overview = show['overview'] if 'overview' in show else ''
-    poster = ''
-    fanart = ''
-    for image in show['images']:
-        if image['coverType'] == 'poster':
-            poster_big = image['url'].split('?')[0]
-            poster = os.path.splitext(poster_big)[0] + '-250' + os.path.splitext(poster_big)[1]
-
-        if image['coverType'] == 'fanart':
-            fanart = image['url'].split('?')[0]
-
-    alternate_titles = None
-    if show['alternateTitles'] is not None:
-        alternate_titles = str([item['title'] for item in show['alternateTitles']])
-
-    audio_language = []
-    if get_sonarr_info.is_legacy():
-        audio_language = profile_id_to_language(show['qualityProfileId'], audio_profiles)
-    else:
-        audio_language = profile_id_to_language(show['languageProfileId'], audio_profiles)
-
-    tags = [d['label'] for d in tags_dict if d['id'] in show['tags']]
-
-    imdbId = show['imdbId'] if 'imdbId' in show else None
-
-    if action == 'update':
-        return {'title': show["title"],
-                'path': show["path"],
-                'tvdbId': int(show["tvdbId"]),
-                'sonarrSeriesId': int(show["id"]),
-                'overview': overview,
-                'poster': poster,
-                'fanart': fanart,
-                'audio_language': str(audio_language),
-                'sortTitle': show['sortTitle'],
-                'year': str(show['year']),
-                'alternateTitles': alternate_titles,
-                'tags': str(tags),
-                'seriesType': show['seriesType'],
-                'imdbId': imdbId}
-    else:
-        return {'title': show["title"],
-                'path': show["path"],
-                'tvdbId': show["tvdbId"],
-                'sonarrSeriesId': show["id"],
-                'overview': overview,
-                'poster': poster,
-                'fanart': fanart,
-                'audio_language': str(audio_language),
-                'sortTitle': show['sortTitle'],
-                'year': str(show['year']),
-                'alternateTitles': alternate_titles,
-                'tags': str(tags),
-                'seriesType': show['seriesType'],
-                'imdbId': imdbId,
-                'profileId': serie_default_profile}
-
-
-def get_series_from_sonarr_api(url, apikey_sonarr, sonarr_series_id=None):
-    url_sonarr_api_series = url + "/api/{0}series/{1}?apikey={2}".format(
-        '' if get_sonarr_info.is_legacy() else 'v3/', sonarr_series_id if sonarr_series_id else "", apikey_sonarr)
-    try:
-        r = requests.get(url_sonarr_api_series, timeout=60, verify=False, headers=headers)
-        r.raise_for_status()
-    except requests.exceptions.HTTPError as e:
-        if e.response.status_code:
-            raise requests.exceptions.HTTPError
-        logging.exception("BAZARR Error trying to get series from Sonarr. Http error.")
-        return
-    except requests.exceptions.ConnectionError:
-        logging.exception("BAZARR Error trying to get series from Sonarr. Connection Error.")
-        return
-    except requests.exceptions.Timeout:
-        logging.exception("BAZARR Error trying to get series from Sonarr. Timeout Error.")
-        return
-    except requests.exceptions.RequestException:
-        logging.exception("BAZARR Error trying to get series from Sonarr.")
-        return
-    else:
-        return r.json()

bazarr/sonarr/sync/utils.py

@@ -0,0 +1,147 @@
+# coding=utf-8
+
+import requests
+import logging
+
+from app.config import settings
+from sonarr.info import get_sonarr_info, url_sonarr
+from constants import headers
+
+
+def get_profile_list():
+    apikey_sonarr = settings.sonarr.apikey
+    profiles_list = []
+
+    # Get profiles data from Sonarr
+    if get_sonarr_info.is_legacy():
+        url_sonarr_api_series = url_sonarr() + "/api/profile?apikey=" + apikey_sonarr
+    else:
+        url_sonarr_api_series = url_sonarr() + "/api/v3/languageprofile?apikey=" + apikey_sonarr
+
+    try:
+        profiles_json = requests.get(url_sonarr_api_series, timeout=60, verify=False, headers=headers)
+    except requests.exceptions.ConnectionError:
+        logging.exception("BAZARR Error trying to get profiles from Sonarr. Connection Error.")
+        return None
+    except requests.exceptions.Timeout:
+        logging.exception("BAZARR Error trying to get profiles from Sonarr. Timeout Error.")
+        return None
+    except requests.exceptions.RequestException:
+        logging.exception("BAZARR Error trying to get profiles from Sonarr.")
+        return None
+
+    # Parsing data returned from Sonarr
+    if get_sonarr_info.is_legacy():
+        for profile in profiles_json.json():
+            profiles_list.append([profile['id'], profile['language'].capitalize()])
+    else:
+        for profile in profiles_json.json():
+            profiles_list.append([profile['id'], profile['name'].capitalize()])
+
+    return profiles_list
+
+
+def get_tags():
+    apikey_sonarr = settings.sonarr.apikey
+    tagsDict = []
+
+    # Get tags data from Sonarr
+    if get_sonarr_info.is_legacy():
+        url_sonarr_api_series = url_sonarr() + "/api/tag?apikey=" + apikey_sonarr
+    else:
+        url_sonarr_api_series = url_sonarr() + "/api/v3/tag?apikey=" + apikey_sonarr
+
+    try:
+        tagsDict = requests.get(url_sonarr_api_series, timeout=60, verify=False, headers=headers)
+    except requests.exceptions.ConnectionError:
+        logging.exception("BAZARR Error trying to get tags from Sonarr. Connection Error.")
+        return []
+    except requests.exceptions.Timeout:
+        logging.exception("BAZARR Error trying to get tags from Sonarr. Timeout Error.")
+        return []
+    except requests.exceptions.RequestException:
+        logging.exception("BAZARR Error trying to get tags from Sonarr.")
+        return []
+    else:
+        return tagsDict.json()
+
+
+def get_series_from_sonarr_api(url, apikey_sonarr, sonarr_series_id=None):
+    url_sonarr_api_series = url + "/api/{0}series/{1}?apikey={2}".format(
+        '' if get_sonarr_info.is_legacy() else 'v3/', sonarr_series_id if sonarr_series_id else "", apikey_sonarr)
+    try:
+        r = requests.get(url_sonarr_api_series, timeout=60, verify=False, headers=headers)
+        r.raise_for_status()
+    except requests.exceptions.HTTPError as e:
+        if e.response.status_code:
+            raise requests.exceptions.HTTPError
+        logging.exception("BAZARR Error trying to get series from Sonarr. Http error.")
+        return
+    except requests.exceptions.ConnectionError:
+        logging.exception("BAZARR Error trying to get series from Sonarr. Connection Error.")
+        return
+    except requests.exceptions.Timeout:
+        logging.exception("BAZARR Error trying to get series from Sonarr. Timeout Error.")
+        return
+    except requests.exceptions.RequestException:
+        logging.exception("BAZARR Error trying to get series from Sonarr.")
+        return
+    else:
+        return r.json()
+
+
+def get_episodes_from_sonarr_api(url, apikey_sonarr, series_id=None, episode_id=None):
+    if series_id:
+        url_sonarr_api_episode = url + "/api/{0}episode?seriesId={1}&apikey={2}".format(
+            '' if get_sonarr_info.is_legacy() else 'v3/', series_id, apikey_sonarr)
+    elif episode_id:
+        url_sonarr_api_episode = url + "/api/{0}episode/{1}?apikey={2}".format(
+            '' if get_sonarr_info.is_legacy() else 'v3/', episode_id, apikey_sonarr)
+    else:
+        return
+
+    try:
+        r = requests.get(url_sonarr_api_episode, timeout=60, verify=False, headers=headers)
+        r.raise_for_status()
+    except requests.exceptions.HTTPError:
+        logging.exception("BAZARR Error trying to get episodes from Sonarr. Http error.")
+        return
+    except requests.exceptions.ConnectionError:
+        logging.exception("BAZARR Error trying to get episodes from Sonarr. Connection Error.")
+        return
+    except requests.exceptions.Timeout:
+        logging.exception("BAZARR Error trying to get episodes from Sonarr. Timeout Error.")
+        return
+    except requests.exceptions.RequestException:
+        logging.exception("BAZARR Error trying to get episodes from Sonarr.")
+        return
+    else:
+        return r.json()
+
+
+def get_episodesFiles_from_sonarr_api(url, apikey_sonarr, series_id=None, episode_file_id=None):
+    if series_id:
+        url_sonarr_api_episodeFiles = url + "/api/v3/episodeFile?seriesId={0}&apikey={1}".format(series_id,
+                                                                                                 apikey_sonarr)
+    elif episode_file_id:
+        url_sonarr_api_episodeFiles = url + "/api/v3/episodeFile/{0}?apikey={1}".format(episode_file_id, apikey_sonarr)
+    else:
+        return
+
+    try:
+        r = requests.get(url_sonarr_api_episodeFiles, timeout=60, verify=False, headers=headers)
+        r.raise_for_status()
+    except requests.exceptions.HTTPError:
+        logging.exception("BAZARR Error trying to get episodeFiles from Sonarr. Http error.")
+        return
+    except requests.exceptions.ConnectionError:
+        logging.exception("BAZARR Error trying to get episodeFiles from Sonarr. Connection Error.")
+        return
+    except requests.exceptions.Timeout:
+        logging.exception("BAZARR Error trying to get episodeFiles from Sonarr. Timeout Error.")
+        return
+    except requests.exceptions.RequestException:
+        logging.exception("BAZARR Error trying to get episodeFiles from Sonarr.")
+        return
+    else:
+        return r.json()

bazarr/subtitles/adaptive_searching.py

@@ -6,7 +6,7 @@ import logging
 
 from datetime import datetime, timedelta
 
-from config import settings
+from app.config import settings
 
 
 def is_search_active(desired_language, attempt_string):

bazarr/subtitles/download.py

@@ -11,10 +11,11 @@ from subliminal_patch.core import save_subtitles
 from subliminal_patch.core_persistent import download_best_subtitles
 from subliminal_patch.score import compute_score
 
-from config import settings, get_array_from
-from helper import get_target_folder, force_unicode
-from get_languages import alpha3_from_alpha2
-from score import movie_score, series_score
+from app.config import settings, get_array_from
+from utilities.helper import get_target_folder, force_unicode
+from languages.get_languages import alpha3_from_alpha2
+from subtitles.tools.score import movie_score, series_score
+
 from .pool import update_pools, _get_pool
 from .utils import get_video, _get_lang_obj, _get_scores
 from .processing import process_subtitle
@@ -142,11 +143,13 @@ def _get_language_obj(languages):
 
 
 def _set_forced_providers(forced_required, pool):
+    # TODO: maybe a separate pool for forced configs? only_foreign is hardcoded
+    # in get_providers and this causes updating the pool on every call
     if forced_required:
-        pool.provider_configs['podnapisi']['only_foreign'] = True
-        pool.provider_configs['subscene']['only_foreign'] = True
-        pool.provider_configs['opensubtitles']['only_foreign'] = True
-    else:
-        pool.provider_configs['podnapisi']['only_foreign'] = False
-        pool.provider_configs['subscene']['only_foreign'] = False
-        pool.provider_configs['opensubtitles']['only_foreign'] = False
+        pool.provider_configs.update(
+            {
+                "podnapisi": {"only_foreign": True},
+                "subscene": {"only_foreign": True},
+                "opensubtitles": {"only_foreign": True}
+             }
+        )

bazarr/subtitles/indexer/__init__.py

@@ -0,0 +1 @@
+# coding=utf-8

bazarr/subtitles/indexer/movies.py

@@ -0,0 +1,261 @@
+# coding=utf-8
+
+import gc
+import os
+import logging
+import ast
+
+from subliminal_patch import core, search_external_subtitles
+
+from languages.custom_lang import CustomLanguage
+from app.database import get_profiles_list, get_profile_cutoff, TableMovies
+from languages.get_languages import alpha2_from_alpha3, language_from_alpha2, get_language_set
+from app.config import settings
+from utilities.helper import get_subtitle_destination_folder
+from utilities.path_mappings import path_mappings
+from subtitles.tools.embedded_subs_reader import embedded_subs_reader
+from app.event_handler import event_stream, show_progress, hide_progress
+from subtitles.indexer.utils import guess_external_subtitles, get_external_subtitles_path
+
+gc.enable()
+
+
+def store_subtitles_movie(original_path, reversed_path, use_cache=True):
+    logging.debug('BAZARR started subtitles indexing for this file: ' + reversed_path)
+    actual_subtitles = []
+    if os.path.exists(reversed_path):
+        if settings.general.getboolean('use_embedded_subs'):
+            logging.debug("BAZARR is trying to index embedded subtitles.")
+            item = TableMovies.select(TableMovies.movie_file_id, TableMovies.file_size)\
+                .where(TableMovies.path == original_path)\
+                .dicts()\
+                .get_or_none()
+            if not item:
+                logging.exception(f"BAZARR error when trying to select this movie from database: {reversed_path}")
+            else:
+                try:
+                    subtitle_languages = embedded_subs_reader(reversed_path,
+                                                              file_size=item['file_size'],
+                                                              movie_file_id=item['movie_file_id'],
+                                                              use_cache=use_cache)
+                    for subtitle_language, subtitle_forced, subtitle_hi, subtitle_codec in subtitle_languages:
+                        try:
+                            if (settings.general.getboolean("ignore_pgs_subs") and subtitle_codec.lower() == "pgs") or \
+                                    (settings.general.getboolean("ignore_vobsub_subs") and subtitle_codec.lower() ==
+                                     "vobsub") or \
+                                    (settings.general.getboolean("ignore_ass_subs") and subtitle_codec.lower() ==
+                                     "ass"):
+                                logging.debug("BAZARR skipping %s sub for language: %s" % (subtitle_codec, alpha2_from_alpha3(subtitle_language)))
+                                continue
+
+                            if alpha2_from_alpha3(subtitle_language) is not None:
+                                lang = str(alpha2_from_alpha3(subtitle_language))
+                                if subtitle_forced:
+                                    lang = lang + ':forced'
+                                if subtitle_hi:
+                                    lang = lang + ':hi'
+                                logging.debug("BAZARR embedded subtitles detected: " + lang)
+                                actual_subtitles.append([lang, None])
+                        except Exception:
+                            logging.debug("BAZARR unable to index this unrecognized language: " + subtitle_language)
+                            pass
+                except Exception:
+                    logging.exception(
+                        "BAZARR error when trying to analyze this %s file: %s" % (os.path.splitext(reversed_path)[1],
+                                                                                  reversed_path))
+                    pass
+
+        try:
+            dest_folder = get_subtitle_destination_folder() or ''
+            core.CUSTOM_PATHS = [dest_folder] if dest_folder else []
+            subtitles = search_external_subtitles(reversed_path, languages=get_language_set())
+            full_dest_folder_path = os.path.dirname(reversed_path)
+            if dest_folder:
+                if settings.general.subfolder == "absolute":
+                    full_dest_folder_path = dest_folder
+                elif settings.general.subfolder == "relative":
+                    full_dest_folder_path = os.path.join(os.path.dirname(reversed_path), dest_folder)
+            subtitles = guess_external_subtitles(full_dest_folder_path, subtitles)
+        except Exception:
+            logging.exception("BAZARR unable to index external subtitles.")
+            pass
+        else:
+            for subtitle, language in subtitles.items():
+                valid_language = False
+                if language:
+                    if hasattr(language, 'alpha3'):
+                        valid_language = alpha2_from_alpha3(language.alpha3)
+                else:
+                    logging.debug(f"Skipping subtitles because we are unable to define language: {subtitle}")
+                    continue
+
+                if not valid_language:
+                    logging.debug(f'{language.alpha3} is an unsupported language code.')
+                    continue
+
+                subtitle_path = get_external_subtitles_path(reversed_path, subtitle)
+                custom = CustomLanguage.found_external(subtitle, subtitle_path)
+
+                if custom is not None:
+                    actual_subtitles.append([custom, path_mappings.path_replace_reverse_movie(subtitle_path)])
+
+                elif str(language.basename) != 'und':
+                    if language.forced:
+                        language_str = str(language)
+                    elif language.hi:
+                        language_str = str(language) + ':hi'
+                    else:
+                        language_str = str(language)
+                    logging.debug("BAZARR external subtitles detected: " + language_str)
+                    actual_subtitles.append([language_str, path_mappings.path_replace_reverse_movie(subtitle_path)])
+
+        TableMovies.update({TableMovies.subtitles: str(actual_subtitles)})\
+            .where(TableMovies.path == original_path)\
+            .execute()
+        matching_movies = TableMovies.select(TableMovies.radarrId).where(TableMovies.path == original_path).dicts()
+
+        for movie in matching_movies:
+            if movie:
+                logging.debug("BAZARR storing those languages to DB: " + str(actual_subtitles))
+                list_missing_subtitles_movies(no=movie['radarrId'])
+            else:
+                logging.debug("BAZARR haven't been able to update existing subtitles to DB : " + str(actual_subtitles))
+    else:
+        logging.debug("BAZARR this file doesn't seems to exist or isn't accessible.")
+
+    logging.debug('BAZARR ended subtitles indexing for this file: ' + reversed_path)
+
+    return actual_subtitles
+
+
+def list_missing_subtitles_movies(no=None, send_event=True):
+    movies_subtitles = TableMovies.select(TableMovies.radarrId,
+                                          TableMovies.subtitles,
+                                          TableMovies.profileId,
+                                          TableMovies.audio_language)\
+        .where((TableMovies.radarrId == no) if no else None)\
+        .dicts()
+    if isinstance(movies_subtitles, str):
+        logging.error("BAZARR list missing subtitles query to DB returned this instead of rows: " + movies_subtitles)
+        return
+
+    use_embedded_subs = settings.general.getboolean('use_embedded_subs')
+
+    for movie_subtitles in movies_subtitles:
+        missing_subtitles_text = '[]'
+        if movie_subtitles['profileId']:
+            # get desired subtitles
+            desired_subtitles_temp = get_profiles_list(profile_id=movie_subtitles['profileId'])
+            desired_subtitles_list = []
+            if desired_subtitles_temp:
+                for language in desired_subtitles_temp['items']:
+                    if language['audio_exclude'] == "True":
+                        if language_from_alpha2(language['language']) in ast.literal_eval(
+                                movie_subtitles['audio_language']):
+                            continue
+                    desired_subtitles_list.append([language['language'], language['forced'], language['hi']])
+
+            # get existing subtitles
+            actual_subtitles_list = []
+            if movie_subtitles['subtitles'] is not None:
+                if use_embedded_subs:
+                    actual_subtitles_temp = ast.literal_eval(movie_subtitles['subtitles'])
+                else:
+                    actual_subtitles_temp = [x for x in ast.literal_eval(movie_subtitles['subtitles']) if x[1]]
+
+                for subtitles in actual_subtitles_temp:
+                    subtitles = subtitles[0].split(':')
+                    lang = subtitles[0]
+                    forced = False
+                    hi = False
+                    if len(subtitles) > 1:
+                        if subtitles[1] == 'forced':
+                            forced = True
+                            hi = False
+                        elif subtitles[1] == 'hi':
+                            forced = False
+                            hi = True
+                    actual_subtitles_list.append([lang, str(forced), str(hi)])
+
+            # check if cutoff is reached and skip any further check
+            cutoff_met = False
+            cutoff_temp_list = get_profile_cutoff(profile_id=movie_subtitles['profileId'])
+
+            if cutoff_temp_list:
+                for cutoff_temp in cutoff_temp_list:
+                    cutoff_language = [cutoff_temp['language'], cutoff_temp['forced'], cutoff_temp['hi']]
+                    if cutoff_temp['audio_exclude'] == 'True' and language_from_alpha2(cutoff_temp['language']) in \
+                            ast.literal_eval(movie_subtitles['audio_language']):
+                        cutoff_met = True
+                    elif cutoff_language in actual_subtitles_list:
+                        cutoff_met = True
+                    # HI is considered as good as normal
+                    elif cutoff_language and [cutoff_language[0], 'False', 'True'] in actual_subtitles_list:
+                        cutoff_met = True
+
+            if cutoff_met:
+                missing_subtitles_text = str([])
+            else:
+                # get difference between desired and existing subtitles
+                missing_subtitles_list = []
+                for item in desired_subtitles_list:
+                    if item not in actual_subtitles_list:
+                        missing_subtitles_list.append(item)
+
+                # remove missing that have forced or hi subtitles for this language in existing
+                for item in actual_subtitles_list:
+                    if item[2] == 'True':
+                        try:
+                            missing_subtitles_list.remove([item[0], 'False', 'False'])
+                        except ValueError:
+                            pass
+
+                # make the missing languages list looks like expected
+                missing_subtitles_output_list = []
+                for item in missing_subtitles_list:
+                    lang = item[0]
+                    if item[1] == 'True':
+                        lang += ':forced'
+                    elif item[2] == 'True':
+                        lang += ':hi'
+                    missing_subtitles_output_list.append(lang)
+
+                missing_subtitles_text = str(missing_subtitles_output_list)
+
+        TableMovies.update({TableMovies.missing_subtitles: missing_subtitles_text})\
+            .where(TableMovies.radarrId == movie_subtitles['radarrId'])\
+            .execute()
+
+        if send_event:
+            event_stream(type='movie', payload=movie_subtitles['radarrId'])
+            event_stream(type='badges')
+
+
+def movies_full_scan_subtitles():
+    use_ffprobe_cache = settings.radarr.getboolean('use_ffprobe_cache')
+
+    movies = TableMovies.select(TableMovies.path).dicts()
+
+    count_movies = len(movies)
+    for i, movie in enumerate(movies):
+        show_progress(id='movies_disk_scan',
+                      header='Full disk scan...',
+                      name='Movies subtitles',
+                      value=i,
+                      count=count_movies)
+        store_subtitles_movie(movie['path'], path_mappings.path_replace_movie(movie['path']),
+                              use_cache=use_ffprobe_cache)
+
+    hide_progress(id='movies_disk_scan')
+
+    gc.collect()
+
+
+def movies_scan_subtitles(no):
+    movies = TableMovies.select(TableMovies.path)\
+        .where(TableMovies.radarrId == no)\
+        .order_by(TableMovies.radarrId)\
+        .dicts()
+
+    for movie in movies:
+        store_subtitles_movie(movie['path'], path_mappings.path_replace_movie(movie['path']), use_cache=False)

bazarr/subtitles/indexer/series.py

@@ -0,0 +1,270 @@
+# coding=utf-8
+
+import gc
+import os
+import logging
+import ast
+
+from subliminal_patch import core, search_external_subtitles
+
+from languages.custom_lang import CustomLanguage
+from app.database import get_profiles_list, get_profile_cutoff, TableEpisodes, TableShows
+from languages.get_languages import alpha2_from_alpha3, language_from_alpha2, get_language_set
+from app.config import settings
+from utilities.helper import get_subtitle_destination_folder
+from utilities.path_mappings import path_mappings
+from subtitles.tools.embedded_subs_reader import embedded_subs_reader
+from app.event_handler import event_stream, show_progress, hide_progress
+from subtitles.indexer.utils import guess_external_subtitles, get_external_subtitles_path
+
+gc.enable()
+
+
+def store_subtitles(original_path, reversed_path, use_cache=True):
+    logging.debug('BAZARR started subtitles indexing for this file: ' + reversed_path)
+    actual_subtitles = []
+    if os.path.exists(reversed_path):
+        if settings.general.getboolean('use_embedded_subs'):
+            logging.debug("BAZARR is trying to index embedded subtitles.")
+            item = TableEpisodes.select(TableEpisodes.episode_file_id, TableEpisodes.file_size)\
+                .where(TableEpisodes.path == original_path)\
+                .dicts()\
+                .get_or_none()
+            if not item:
+                logging.exception(f"BAZARR error when trying to select this episode from database: {reversed_path}")
+            else:
+                try:
+                    subtitle_languages = embedded_subs_reader(reversed_path,
+                                                              file_size=item['file_size'],
+                                                              episode_file_id=item['episode_file_id'],
+                                                              use_cache=use_cache)
+                    for subtitle_language, subtitle_forced, subtitle_hi, subtitle_codec in subtitle_languages:
+                        try:
+                            if (settings.general.getboolean("ignore_pgs_subs") and subtitle_codec.lower() == "pgs") or \
+                                    (settings.general.getboolean("ignore_vobsub_subs") and subtitle_codec.lower() ==
+                                     "vobsub") or \
+                                    (settings.general.getboolean("ignore_ass_subs") and subtitle_codec.lower() ==
+                                     "ass"):
+                                logging.debug("BAZARR skipping %s sub for language: %s" % (subtitle_codec, alpha2_from_alpha3(subtitle_language)))
+                                continue
+
+                            if alpha2_from_alpha3(subtitle_language) is not None:
+                                lang = str(alpha2_from_alpha3(subtitle_language))
+                                if subtitle_forced:
+                                    lang = lang + ":forced"
+                                if subtitle_hi:
+                                    lang = lang + ":hi"
+                                logging.debug("BAZARR embedded subtitles detected: " + lang)
+                                actual_subtitles.append([lang, None])
+                        except Exception as error:
+                            logging.debug("BAZARR unable to index this unrecognized language: %s (%s)", subtitle_language, error)
+                except Exception:
+                    logging.exception(
+                        "BAZARR error when trying to analyze this %s file: %s" % (os.path.splitext(reversed_path)[1],
+                                                                                  reversed_path))
+                    pass
+        try:
+            dest_folder = get_subtitle_destination_folder()
+            core.CUSTOM_PATHS = [dest_folder] if dest_folder else []
+            subtitles = search_external_subtitles(reversed_path, languages=get_language_set(),
+                                                  only_one=settings.general.getboolean('single_language'))
+            full_dest_folder_path = os.path.dirname(reversed_path)
+            if dest_folder:
+                if settings.general.subfolder == "absolute":
+                    full_dest_folder_path = dest_folder
+                elif settings.general.subfolder == "relative":
+                    full_dest_folder_path = os.path.join(os.path.dirname(reversed_path), dest_folder)
+            subtitles = guess_external_subtitles(full_dest_folder_path, subtitles)
+        except Exception:
+            logging.exception("BAZARR unable to index external subtitles.")
+        else:
+            for subtitle, language in subtitles.items():
+                valid_language = False
+                if language:
+                    if hasattr(language, 'alpha3'):
+                        valid_language = alpha2_from_alpha3(language.alpha3)
+                else:
+                    logging.debug(f"Skipping subtitles because we are unable to define language: {subtitle}")
+                    continue
+
+                if not valid_language:
+                    logging.debug(f'{language.alpha3} is an unsupported language code.')
+                    continue
+
+                subtitle_path = get_external_subtitles_path(reversed_path, subtitle)
+
+                custom = CustomLanguage.found_external(subtitle, subtitle_path)
+                if custom is not None:
+                    actual_subtitles.append([custom, path_mappings.path_replace_reverse(subtitle_path)])
+
+                elif str(language) != 'und':
+                    if language.forced:
+                        language_str = str(language)
+                    elif language.hi:
+                        language_str = str(language) + ':hi'
+                    else:
+                        language_str = str(language)
+                    logging.debug("BAZARR external subtitles detected: " + language_str)
+                    actual_subtitles.append([language_str, path_mappings.path_replace_reverse(subtitle_path)])
+
+        TableEpisodes.update({TableEpisodes.subtitles: str(actual_subtitles)})\
+            .where(TableEpisodes.path == original_path)\
+            .execute()
+        matching_episodes = TableEpisodes.select(TableEpisodes.sonarrEpisodeId, TableEpisodes.sonarrSeriesId)\
+            .where(TableEpisodes.path == original_path)\
+            .dicts()
+
+        for episode in matching_episodes:
+            if episode:
+                logging.debug("BAZARR storing those languages to DB: " + str(actual_subtitles))
+                list_missing_subtitles(epno=episode['sonarrEpisodeId'])
+            else:
+                logging.debug("BAZARR haven't been able to update existing subtitles to DB : " + str(actual_subtitles))
+    else:
+        logging.debug("BAZARR this file doesn't seems to exist or isn't accessible.")
+
+    logging.debug('BAZARR ended subtitles indexing for this file: ' + reversed_path)
+
+    return actual_subtitles
+
+
+def list_missing_subtitles(no=None, epno=None, send_event=True):
+    if epno is not None:
+        episodes_subtitles_clause = (TableEpisodes.sonarrEpisodeId == epno)
+    elif no is not None:
+        episodes_subtitles_clause = (TableEpisodes.sonarrSeriesId == no)
+    else:
+        episodes_subtitles_clause = None
+    episodes_subtitles = TableEpisodes.select(TableShows.sonarrSeriesId,
+                                              TableEpisodes.sonarrEpisodeId,
+                                              TableEpisodes.subtitles,
+                                              TableShows.profileId,
+                                              TableEpisodes.audio_language)\
+        .join(TableShows, on=(TableEpisodes.sonarrSeriesId == TableShows.sonarrSeriesId))\
+        .where(episodes_subtitles_clause)\
+        .dicts()
+    if isinstance(episodes_subtitles, str):
+        logging.error("BAZARR list missing subtitles query to DB returned this instead of rows: " + episodes_subtitles)
+        return
+
+    use_embedded_subs = settings.general.getboolean('use_embedded_subs')
+
+    for episode_subtitles in episodes_subtitles:
+        missing_subtitles_text = '[]'
+        if episode_subtitles['profileId']:
+            # get desired subtitles
+            desired_subtitles_temp = get_profiles_list(profile_id=episode_subtitles['profileId'])
+            desired_subtitles_list = []
+            if desired_subtitles_temp:
+                for language in desired_subtitles_temp['items']:
+                    if language['audio_exclude'] == "True":
+                        if language_from_alpha2(language['language']) in ast.literal_eval(
+                                episode_subtitles['audio_language']):
+                            continue
+                    desired_subtitles_list.append([language['language'], language['forced'], language['hi']])
+
+            # get existing subtitles
+            actual_subtitles_list = []
+            if episode_subtitles['subtitles'] is not None:
+                if use_embedded_subs:
+                    actual_subtitles_temp = ast.literal_eval(episode_subtitles['subtitles'])
+                else:
+                    actual_subtitles_temp = [x for x in ast.literal_eval(episode_subtitles['subtitles']) if x[1]]
+
+                for subtitles in actual_subtitles_temp:
+                    subtitles = subtitles[0].split(':')
+                    lang = subtitles[0]
+                    forced = False
+                    hi = False
+                    if len(subtitles) > 1:
+                        if subtitles[1] == 'forced':
+                            forced = True
+                            hi = False
+                        elif subtitles[1] == 'hi':
+                            forced = False
+                            hi = True
+                    actual_subtitles_list.append([lang, str(forced), str(hi)])
+
+            # check if cutoff is reached and skip any further check
+            cutoff_met = False
+            cutoff_temp_list = get_profile_cutoff(profile_id=episode_subtitles['profileId'])
+
+            if cutoff_temp_list:
+                for cutoff_temp in cutoff_temp_list:
+                    cutoff_language = [cutoff_temp['language'], cutoff_temp['forced'], cutoff_temp['hi']]
+                    if cutoff_temp['audio_exclude'] == 'True' and language_from_alpha2(cutoff_temp['language']) in \
+                            ast.literal_eval(episode_subtitles['audio_language']):
+                        cutoff_met = True
+                    elif cutoff_language in actual_subtitles_list:
+                        cutoff_met = True
+                    # HI is considered as good as normal
+                    elif [cutoff_language[0], 'False', 'True'] in actual_subtitles_list:
+                        cutoff_met = True
+
+            if cutoff_met:
+                missing_subtitles_text = str([])
+            else:
+                # if cutoff isn't met or None, we continue
+
+                # get difference between desired and existing subtitles
+                missing_subtitles_list = []
+                for item in desired_subtitles_list:
+                    if item not in actual_subtitles_list:
+                        missing_subtitles_list.append(item)
+
+                # remove missing that have hi subtitles for this language in existing
+                for item in actual_subtitles_list:
+                    if item[2] == 'True':
+                        try:
+                            missing_subtitles_list.remove([item[0], 'False', 'False'])
+                        except ValueError:
+                            pass
+
+                # make the missing languages list looks like expected
+                missing_subtitles_output_list = []
+                for item in missing_subtitles_list:
+                    lang = item[0]
+                    if item[1] == 'True':
+                        lang += ':forced'
+                    elif item[2] == 'True':
+                        lang += ':hi'
+                    missing_subtitles_output_list.append(lang)
+
+                missing_subtitles_text = str(missing_subtitles_output_list)
+
+        TableEpisodes.update({TableEpisodes.missing_subtitles: missing_subtitles_text})\
+            .where(TableEpisodes.sonarrEpisodeId == episode_subtitles['sonarrEpisodeId'])\
+            .execute()
+
+        if send_event:
+            event_stream(type='episode', payload=episode_subtitles['sonarrEpisodeId'])
+            event_stream(type='badges')
+
+
+def series_full_scan_subtitles():
+    use_ffprobe_cache = settings.sonarr.getboolean('use_ffprobe_cache')
+
+    episodes = TableEpisodes.select(TableEpisodes.path).dicts()
+
+    count_episodes = len(episodes)
+    for i, episode in enumerate(episodes):
+        show_progress(id='episodes_disk_scan',
+                      header='Full disk scan...',
+                      name='Episodes subtitles',
+                      value=i,
+                      count=count_episodes)
+        store_subtitles(episode['path'], path_mappings.path_replace(episode['path']), use_cache=use_ffprobe_cache)
+
+    hide_progress(id='episodes_disk_scan')
+
+    gc.collect()
+
+
+def series_scan_subtitles(no):
+    episodes = TableEpisodes.select(TableEpisodes.path)\
+        .where(TableEpisodes.sonarrSeriesId == no)\
+        .order_by(TableEpisodes.sonarrEpisodeId)\
+        .dicts()
+
+    for episode in episodes:
+        store_subtitles(episode['path'], path_mappings.path_replace(episode['path']), use_cache=False)

bazarr/subtitles/indexer/utils.py

@@ -0,0 +1,144 @@
+# coding=utf-8
+
+import os
+import logging
+import re
+
+from guess_language import guess_language
+from subliminal_patch import core
+from subzero.language import Language
+from charamel import Detector
+
+from app.config import settings
+from constants import hi_regex
+
+
+def get_external_subtitles_path(file, subtitle):
+    fld = os.path.dirname(file)
+
+    if settings.general.subfolder == "current":
+        path = os.path.join(fld, subtitle)
+    elif settings.general.subfolder == "absolute":
+        custom_fld = settings.general.subfolder_custom
+        if os.path.exists(os.path.join(fld, subtitle)):
+            path = os.path.join(fld, subtitle)
+        elif os.path.exists(os.path.join(custom_fld, subtitle)):
+            path = os.path.join(custom_fld, subtitle)
+        else:
+            path = None
+    elif settings.general.subfolder == "relative":
+        custom_fld = os.path.join(fld, settings.general.subfolder_custom)
+        if os.path.exists(os.path.join(fld, subtitle)):
+            path = os.path.join(fld, subtitle)
+        elif os.path.exists(os.path.join(custom_fld, subtitle)):
+            path = os.path.join(custom_fld, subtitle)
+        else:
+            path = None
+    else:
+        path = None
+
+    return path
+
+
+def guess_external_subtitles(dest_folder, subtitles):
+    for subtitle, language in subtitles.items():
+        if not language:
+            subtitle_path = os.path.join(dest_folder, subtitle)
+            if os.path.exists(subtitle_path) and os.path.splitext(subtitle_path)[1] in core.SUBTITLE_EXTENSIONS:
+                logging.debug("BAZARR falling back to file content analysis to detect language.")
+                detected_language = None
+
+                # to improve performance, skip detection of files larger that 1M
+                if os.path.getsize(subtitle_path) > 1*1024*1024:
+                    logging.debug("BAZARR subtitles file is too large to be text based. Skipping this file: " +
+                                  subtitle_path)
+                    continue
+
+                with open(subtitle_path, 'rb') as f:
+                    text = f.read()
+
+                try:
+                    text = text.decode('utf-8')
+                    detected_language = guess_language(text)
+                    # add simplified and traditional chinese detection
+                    if detected_language == 'zh':
+                        traditional_chinese_fuzzy = [u"繁", u"雙語"]
+                        traditional_chinese = [".cht", ".tc", ".zh-tw", ".zht", ".zh-hant", ".zhhant", ".zh_hant",
+                                               ".hant", ".big5", ".traditional"]
+                        if str(os.path.splitext(subtitle)[0]).lower().endswith(tuple(traditional_chinese)) or (str(subtitle_path).lower())[:-5] in traditional_chinese_fuzzy:
+                            detected_language == 'zt'
+                except UnicodeDecodeError:
+                    detector = Detector()
+                    try:
+                        guess = detector.detect(text)
+                    except Exception:
+                        logging.debug("BAZARR skipping this subtitles because we can't guess the encoding. "
+                                      "It's probably a binary file: " + subtitle_path)
+                        continue
+                    else:
+                        logging.debug('BAZARR detected encoding %r', guess)
+                        try:
+                            text = text.decode(guess)
+                        except Exception:
+                            logging.debug(
+                                "BAZARR skipping this subtitles because we can't decode the file using the "
+                                "guessed encoding. It's probably a binary file: " + subtitle_path)
+                            continue
+                    detected_language = guess_language(text)
+                except Exception:
+                    logging.debug('BAZARR was unable to detect encoding for this subtitles file: %r', subtitle_path)
+                finally:
+                    if detected_language:
+                        logging.debug("BAZARR external subtitles detected and guessed this language: " + str(
+                            detected_language))
+                        try:
+                            subtitles[subtitle] = Language.rebuild(Language.fromietf(detected_language), forced=False,
+                                                                   hi=False)
+                        except Exception:
+                            pass
+
+        # If language is still None (undetected), skip it
+        if not language:
+            pass
+
+        # Skip HI detection if forced
+        elif language.forced:
+            pass
+
+        # Detect hearing-impaired external subtitles not identified in filename
+        elif not subtitles[subtitle].hi:
+            subtitle_path = os.path.join(dest_folder, subtitle)
+
+            # check if file exist:
+            if os.path.exists(subtitle_path) and os.path.splitext(subtitle_path)[1] in core.SUBTITLE_EXTENSIONS:
+                # to improve performance, skip detection of files larger that 1M
+                if os.path.getsize(subtitle_path) > 1 * 1024 * 1024:
+                    logging.debug("BAZARR subtitles file is too large to be text based. Skipping this file: " +
+                                  subtitle_path)
+                    continue
+
+                with open(subtitle_path, 'rb') as f:
+                    text = f.read()
+
+                try:
+                    text = text.decode('utf-8')
+                except UnicodeDecodeError:
+                    detector = Detector()
+                    try:
+                        guess = detector.detect(text)
+                    except Exception:
+                        logging.debug("BAZARR skipping this subtitles because we can't guess the encoding. "
+                                      "It's probably a binary file: " + subtitle_path)
+                        continue
+                    else:
+                        logging.debug('BAZARR detected encoding %r', guess)
+                        try:
+                            text = text.decode(guess)
+                        except Exception:
+                            logging.debug("BAZARR skipping this subtitles because we can't decode the file using the "
+                                          "guessed encoding. It's probably a binary file: " + subtitle_path)
+                            continue
+
+                if bool(re.search(hi_regex, text)):
+                    subtitles[subtitle] = Language.rebuild(subtitles[subtitle], forced=False, hi=True)
+    return subtitles

bazarr/subtitles/manual.py

@@ -13,11 +13,12 @@ from subliminal_patch.core import save_subtitles
 from subliminal_patch.core_persistent import list_all_subtitles, download_subtitles
 from subliminal_patch.score import compute_score
 
-from get_languages import alpha3_from_alpha2
-from config import settings, get_array_from
-from helper import get_target_folder, force_unicode
-from database import get_profiles_list
-from score import movie_score, series_score
+from languages.get_languages import alpha3_from_alpha2
+from app.config import settings, get_array_from
+from utilities.helper import get_target_folder, force_unicode
+from app.database import get_profiles_list
+from subtitles.tools.score import movie_score, series_score
+
 from .pool import update_pools, _get_pool, _init_pool
 from .utils import get_video, _get_lang_obj, _get_scores
 from .processing import process_subtitle
@@ -55,10 +56,9 @@ def manual_search(path, profile_id, providers, sceneName, title, media_type):
                         if language.forced:
                             subscene_language_set.add(language)
                     if len(subscene_language_set):
-                        s_pool.provider_configs['subscene'] = {}
-                        s_pool.provider_configs['subscene']['only_foreign'] = True
+                        s_pool.provider_configs.update({"subscene": {"only_foreign": True}})
                         subtitles_subscene = list_all_subtitles([video], subscene_language_set, s_pool)
-                        s_pool.provider_configs['subscene']['only_foreign'] = False
+                        s_pool.provider_configs.update({"subscene": {"only_foreign": False}})
                         subtitles[video] += subtitles_subscene[video]
             else:
                 subtitles = []
@@ -154,7 +154,7 @@ def manual_search(path, profile_id, providers, sceneName, title, media_type):
 
 @update_pools
 def manual_download_subtitle(path, audio_language, hi, forced, subtitle, provider, sceneName, title, media_type,
-    use_original_format, profile_id):
+                             use_original_format, profile_id):
     logging.debug('BAZARR Manually downloading Subtitles for this file: ' + path)
 
     if settings.general.getboolean('utf8_encode'):
@@ -268,5 +268,4 @@ def _get_language_obj(profile_id):
 
 def _set_forced_providers(also_forced, pool):
     if also_forced:
-        pool.provider_configs['podnapisi']['also_foreign'] = True
-        pool.provider_configs['opensubtitles']['also_foreign'] = True
+        pool.provider_configs.update({'podnapisi': {'also_foreign': True}, 'opensubtitles': {'also_foreign': True}})