2012-09-13 15:27:34 +00:00
# This file is part of Mylar.
#
# Mylar is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# Mylar is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with Mylar. If not, see <http://www.gnu.org/licenses/>.
import time
2013-01-07 19:26:10 +00:00
import datetime
2012-09-13 15:27:34 +00:00
from xml . dom . minidom import parseString
import urllib2
import shlex
IMP:New scheduler tab (Manage / Activity) where you can see job current run status', next runtime, and prev runtimes as well as force/pause a job, FIX: Disabling
torrents will now properly hide torrent information, IMP: Specified daemon port for deluge as an on-screen tip for more detail, IMP: Added 100,200,ALL as viewable
watchlist views, FIX: When viewing pullist and annual integration enabled, if annual was present would incorrectly link to invalid annual series instead of the
actual series itself, IMP: Added more detail error messages to metatagging errors and better handling of stranded files during cleanup, IMP: Improved some handling for weekly pull-list one-off's and refactored the
nzb/oneoff post-processing into a seperate function for future callables, Moved all the main url locations for public torrent sites to the init module so that it
can be cascaded down for use in other modules instead as a global, IMP: Added a 'deep_search_32p' variable in the config.ini for specific usage with 32p, where
if there is more than one result will dig deeper into each result to try and figure out if there are series matches, as opposed to the default where it will only use ref32p
table if available or just the first hit in a multiple series search results and ignore the remainder, FIX:Fixed some unknown characters appearing in the pullist due
to unicode-related conversion problems, FIX: fixed some special cases of file parsing errors due to Volume label being named different than expected, FIX: Added a
3s pause between experimental searches to try and not hit their frequency limitation, IMP: Weekly Pullist One-off's will now show status of Snatched/Downloaded as
required, FIX: Fixed some deluge parameter problems when using auto-snatch torrent script/option, IMP: Changed the downlocation in the auto-snatch option to an
env variable instead of being passed to avoid unicode-related problems, FIX: Fixed some magnet-related issues for torrents when using a watchdir + TPSE, FIX: Added
more verbose error message for rtorrent connection issues, FIX: Could not connect to rtorrent client if no username/password were provided, IMP: Set the db updater to run
every 5 minutes on the watchlist, automatically refreshing the oldest updated series each time that is more than 5 hours old (force db update from the activity/job schedulers page will run the db updater
against the entire watchlist in sequence), IMP: Attempt to handle long paths in windows (ie. > 256c) by prepending the unicode windows api character to the import
a directory path (windows only), IMP: When manual metatagging a series, will update the series after all the metatagging has been completed as opposed to after each
issue, IMP: Will now display available inkdrops on Config/Search Providers tab when using 32P (future will utilize/indicate inkdrop threshold when downloading)
2017-09-10 15:50:08 +00:00
import operator
2015-05-22 08:32:51 +00:00
import re
2012-10-30 10:43:01 +00:00
import os
2013-08-06 03:20:09 +00:00
import itertools
2017-09-21 19:01:11 +00:00
import sys
import exceptions
2012-09-13 15:27:34 +00:00
import mylar
from mylar import db , logger , helpers , filechecker
IMP:New scheduler tab (Manage / Activity) where you can see job current run status', next runtime, and prev runtimes as well as force/pause a job, FIX: Disabling
torrents will now properly hide torrent information, IMP: Specified daemon port for deluge as an on-screen tip for more detail, IMP: Added 100,200,ALL as viewable
watchlist views, FIX: When viewing pullist and annual integration enabled, if annual was present would incorrectly link to invalid annual series instead of the
actual series itself, IMP: Added more detail error messages to metatagging errors and better handling of stranded files during cleanup, IMP: Improved some handling for weekly pull-list one-off's and refactored the
nzb/oneoff post-processing into a seperate function for future callables, Moved all the main url locations for public torrent sites to the init module so that it
can be cascaded down for use in other modules instead as a global, IMP: Added a 'deep_search_32p' variable in the config.ini for specific usage with 32p, where
if there is more than one result will dig deeper into each result to try and figure out if there are series matches, as opposed to the default where it will only use ref32p
table if available or just the first hit in a multiple series search results and ignore the remainder, FIX:Fixed some unknown characters appearing in the pullist due
to unicode-related conversion problems, FIX: fixed some special cases of file parsing errors due to Volume label being named different than expected, FIX: Added a
3s pause between experimental searches to try and not hit their frequency limitation, IMP: Weekly Pullist One-off's will now show status of Snatched/Downloaded as
required, FIX: Fixed some deluge parameter problems when using auto-snatch torrent script/option, IMP: Changed the downlocation in the auto-snatch option to an
env variable instead of being passed to avoid unicode-related problems, FIX: Fixed some magnet-related issues for torrents when using a watchdir + TPSE, FIX: Added
more verbose error message for rtorrent connection issues, FIX: Could not connect to rtorrent client if no username/password were provided, IMP: Set the db updater to run
every 5 minutes on the watchlist, automatically refreshing the oldest updated series each time that is more than 5 hours old (force db update from the activity/job schedulers page will run the db updater
against the entire watchlist in sequence), IMP: Attempt to handle long paths in windows (ie. > 256c) by prepending the unicode windows api character to the import
a directory path (windows only), IMP: When manual metatagging a series, will update the series after all the metatagging has been completed as opposed to after each
issue, IMP: Will now display available inkdrops on Config/Search Providers tab when using 32P (future will utilize/indicate inkdrop threshold when downloading)
2017-09-10 15:50:08 +00:00
def dbUpdate ( ComicIDList = None , calledfrom = None , sched = False ) :
FIX: included version of comictagger should now work with both Windows and *nix based OS' again, IMP: Global Copy/Move option available when performing post-processing, IMP: Added a verbose file-checking option (FOLDER_SCAN_LOG_VERBOSE) - when enabled will log as it currently does during manual post-processing/file-checking runs, when disabled it will not spam the log nearly as much resulting in more readable log files, IMP: Added Verbose debug logging both via startup option(-v), as well as toggle button in Log GUI (from headphones), as well as per-page loading of log file(s) in GUI, FIX: When doing manual post-processing on issues that were in story arcs, will now indicate X story-arc issues were post-processed for better visibility, FIX: Fixed an issue with deleting from the nzblog table when story arc issues were post-processed, IMP: Added WEEKFOLDER_LOC to the config.ini to allow for specification of where the weekly download directories will default to (as opposed to off of ComicLocation root), IMP: Better handling of some special character references in series titles when looking for series on the auto-wanted list, IMP: 32P will now auto-disable provider if logon returns invalid credentials, FIX: When using alt_pull on weekly pull list, xA0 unicode character caused error, FIX: If title had invalid character in filename that was replaced with a character that already existed in the title, would not scan in during file-checking, FIX: When searching for a series (weeklypull-list/add a series), if the title contained 'and' or '&' would return really mixed up results, FIX: When Post-Processing, if filename being processed had special characters (ie. comma) and was different than nzbname, in some cases would fail to find/move issues, IMP: Utilize internal comictagger to convert from cbr/cbz, IMP: Added more checks when post-processing to ensure files are handled correctly, IMP: Added meta-tag reading when importing series/issues - if previously tagged with CT, will reverse look-up the provided IssueID to reference the correct ComicID, IMP: If scanned directory during import contins cvinfo file, use that and force the ComicID to entire directory when importing a series, IMP: Manual meta-tagging issues will no longer create temporary directories and/or create files in the Comic Location root causing problems for some users, FIX: Annuals weren't properly sorted upon loading of comic details page for some series, IMP: Added some extra checks when validating/creating directories, FIX: Fixed a problem when displaying some covers of .cbz files on the comic details page
2016-01-26 07:49:56 +00:00
if mylar . IMPORTLOCK :
logger . info ( ' Import is currently running - deferring this until the next scheduled run sequence. ' )
return
2012-09-13 15:27:34 +00:00
myDB = db . DBConnection ( )
2013-07-01 05:19:15 +00:00
if ComicIDList is None :
2017-10-25 18:33:56 +00:00
if mylar . CONFIG . UPDATE_ENDED :
2015-04-28 02:15:53 +00:00
logger . info ( ' Updating only Continuing Series (option enabled) - this might cause problems with the pull-list matching for rebooted series ' )
comiclist = [ ]
FIX: if 32p was disabled due to inability to signon, would still attempt to retrieve cached result when match occured which would then possibly result in excessive attempted logins, IMP: Added Force Type option to comicdetails page which will allow the type of series to be forced to TPB/GN format if Mylar is unable to decipher it under normal circumstances (doing this will affect how files are parsed for the given series), IMP: TPB/GN File Parser will now account for mutiple issues of the same TPB series within the given directory, IMP: When enabling/disabling Force Type, will rename existing folder to accomodate new naming folder convention if required, IMP: Started to move some of the file rename/manipulation modules into a filer.py module for more consistency, IMP: Added a to Folder Format which when triggered will use the designated Series Type, IMP:(#2130) When importing, option now available for the final series destination path to use the imported path location instead of defaulting to the Comic Location base folder - status will be no longer be Archived in such cases. Will also not show previous imported series on subsequent import runs if the import directory has not changed (thnx @CuddleBear92), IMP: Added the codebase for a Rename Preview option into the mix - GUI available, but not linked, FIX: Removed DEM from RSS scans when Public Torrents option is enabled, FIX:(#2137) When adding series, if series indicated date in a different format other than just numeric, would cause error
2018-12-10 17:27:53 +00:00
completelist = myDB . select ( ' SELECT LatestDate, ComicPublished, ForceContinuing, NewPublish, LastUpdated, ComicID, ComicName, Corrected_SeriesYear, Corrected_Type, ComicYear from comics WHERE Status= " Active " or Status= " Loading " order by LastUpdated DESC, LatestDate ASC ' )
2015-04-28 02:15:53 +00:00
for comlist in completelist :
if comlist [ ' LatestDate ' ] is None :
recentstatus = ' Loading '
elif comlist [ ' ComicPublished ' ] is None or comlist [ ' ComicPublished ' ] == ' ' or comlist [ ' LatestDate ' ] is None :
recentstatus = ' Unknown '
elif comlist [ ' ForceContinuing ' ] == 1 :
recentstatus = ' Continuing '
2015-05-22 08:32:51 +00:00
elif ' present ' in comlist [ ' ComicPublished ' ] . lower ( ) or ( helpers . today ( ) [ : 4 ] in comlist [ ' LatestDate ' ] ) :
2015-04-28 02:15:53 +00:00
latestdate = comlist [ ' LatestDate ' ]
2015-05-22 08:32:51 +00:00
c_date = datetime . date ( int ( latestdate [ : 4 ] ) , int ( latestdate [ 5 : 7 ] ) , 1 )
2015-04-28 02:15:53 +00:00
n_date = datetime . date . today ( )
recentchk = ( n_date - c_date ) . days
if comlist [ ' NewPublish ' ] :
recentstatus = ' Continuing '
else :
if recentchk < 55 :
recentstatus = ' Continuing '
else :
recentstatus = ' Ended '
else :
recentstatus = ' Ended '
if recentstatus == ' Continuing ' :
FIX:(#1431) Fixed post-processing cleanup when using SAB + files resided in a sub-folder of the path sent from SAB, IMP: Added hardlink/softlink option for non-windows users (note that metatagging is forcibly disabled when these options are enabled), IMP: Added Enforce Permissions option to force Mylar to either force / ensure permissions are as set within Mylar, or to ignore permissions and just try to perform the given action (this is for adding series, post-processing, importing, arcs, etc), FIX: Fixed a problem where if local watchdir was set and a torrent downloader was selected, both would attempt to be used, but only the watchdir was actually utilized (now it's one or the other, never both), FIX: Mylar will now remove issues that are in Failed status that have incorrect Series Data attached, FIX:(#1438) When file-checking/post-processing if issue date in Mylar was current year and date in filename was the following year (for end-of-year issues), would cause traceback error, FIX: If series year polled from CV is incorrect or not provided, option is available in Edit Settings of series detail page that allows user to specify the correct series year, FIX:(#1427) Fixed a typo when the Scan Library option is used and ComicID's were able to be parsed from the files, FIX: When issues were archived, depending on if series had issues in a Downloaded status or not, would give different results to the Have totals, IMP: Manage tab / Manage Comics now has metatag option available to allow metatagging multiple series in a batch mode, IMP:(#1419) When 'write cvinfo' option is enabled, will now also write cvinfo into story arc directories when added / refreshed, FIX: Fixed a problem when group metatag was being used and causing errors, IMP: Manage Failed page now will link back to the series for easier reference, and the 'link' column was removed due to invalid references, FIX: Removed the 'issue not updated' from the series detail page as was not accurate, IMP: Added a Select Filtered Issues button to the series detail page to allow for selecting of all filtered issues (Show ALL MUST be enabled for it to work properly)
2016-11-02 15:00:56 +00:00
comiclist . append ( { " LatestDate " : comlist [ ' LatestDate ' ] ,
" LastUpdated " : comlist [ ' LastUpdated ' ] ,
" ComicID " : comlist [ ' ComicID ' ] ,
" ComicName " : comlist [ ' ComicName ' ] ,
FIX:(#1466) When post-processing if filename did not have issue year or volume, would error, IMP:(#1418) Separate copy/move/hardlink/symlink option for Story arcs, IMP:(#1417) Added story arc format folder to story arc options - can now specify folder format for story arc naming, IMP:(#1374) Added rtorrent_authentication to config.ini allowing user to specify digest/basic authentication, as well as the rpc_url (rtorrent_rpc_url) and ssl (rtorrent_ssl). Rtorrent_host no longer requires protocol in field, IMP: Instead of hitting the intermediary/32P site to get the torrent group, will now retain specific id for torrent group to speed up searches, FIX: (#1472) Fixed some problems with series being searched against 32P which had special characters and would return no results, FIX: Added some specific exception catches when attempting to retrieve search results for nzb providers, FIX: Fixed publisher retaining problem for story arcs (note story arcs will have to be refreshed), FIX: Fixed a problem with the pull-list matching up to the correct series when the issue was an annual for a given series with the alt_pull 2 method (would create an invalid link on the Wanted tab), IMP:(#1470) Added config.ini option to specify folder-formatting for weekly folder numbering, 0=YYYY-WN, 1=YYYY-MM-DD, IMP: (#1471) Added quick check to ensure that newznab host entries do not contain leading/trailing spaces and will strip accordingly, FIX:(#1476) Annuals now have option to add to reading list via the series detail page, IMP: Fixed password fields to allow for special characters within password, IMP: Weekly pull will now correctly show status for previous/future weeks for watchlisted series, IMP: Download option on pullist will work for previous weeks, FIX: Fixed some issues with one-off and story arc post-processing using different file operations, FIX: Fixed error when performing an 'Force Update Active Comics' from the Manage tab (or the scheduled updating of all active comics), IMP: Storyarc issues that are not on watchlist will now retain value for volume to help when searching and file-checking/post-processing, FIX:(#1473) Changed pullist dates on pullist page to unicode to allow for non-English presentation
2016-12-02 17:45:49 +00:00
" ComicYear " : comlist [ ' ComicYear ' ] ,
FIX: if 32p was disabled due to inability to signon, would still attempt to retrieve cached result when match occured which would then possibly result in excessive attempted logins, IMP: Added Force Type option to comicdetails page which will allow the type of series to be forced to TPB/GN format if Mylar is unable to decipher it under normal circumstances (doing this will affect how files are parsed for the given series), IMP: TPB/GN File Parser will now account for mutiple issues of the same TPB series within the given directory, IMP: When enabling/disabling Force Type, will rename existing folder to accomodate new naming folder convention if required, IMP: Started to move some of the file rename/manipulation modules into a filer.py module for more consistency, IMP: Added a to Folder Format which when triggered will use the designated Series Type, IMP:(#2130) When importing, option now available for the final series destination path to use the imported path location instead of defaulting to the Comic Location base folder - status will be no longer be Archived in such cases. Will also not show previous imported series on subsequent import runs if the import directory has not changed (thnx @CuddleBear92), IMP: Added the codebase for a Rename Preview option into the mix - GUI available, but not linked, FIX: Removed DEM from RSS scans when Public Torrents option is enabled, FIX:(#2137) When adding series, if series indicated date in a different format other than just numeric, would cause error
2018-12-10 17:27:53 +00:00
" Corrected_SeriesYear " : comlist [ ' Corrected_SeriesYear ' ] ,
" Corrected_Type " : comlist [ ' Corrected_Type ' ] } )
2015-04-28 02:15:53 +00:00
else :
FIX: if 32p was disabled due to inability to signon, would still attempt to retrieve cached result when match occured which would then possibly result in excessive attempted logins, IMP: Added Force Type option to comicdetails page which will allow the type of series to be forced to TPB/GN format if Mylar is unable to decipher it under normal circumstances (doing this will affect how files are parsed for the given series), IMP: TPB/GN File Parser will now account for mutiple issues of the same TPB series within the given directory, IMP: When enabling/disabling Force Type, will rename existing folder to accomodate new naming folder convention if required, IMP: Started to move some of the file rename/manipulation modules into a filer.py module for more consistency, IMP: Added a to Folder Format which when triggered will use the designated Series Type, IMP:(#2130) When importing, option now available for the final series destination path to use the imported path location instead of defaulting to the Comic Location base folder - status will be no longer be Archived in such cases. Will also not show previous imported series on subsequent import runs if the import directory has not changed (thnx @CuddleBear92), IMP: Added the codebase for a Rename Preview option into the mix - GUI available, but not linked, FIX: Removed DEM from RSS scans when Public Torrents option is enabled, FIX:(#2137) When adding series, if series indicated date in a different format other than just numeric, would cause error
2018-12-10 17:27:53 +00:00
comiclist = myDB . select ( ' SELECT LatestDate, LastUpdated, ComicID, ComicName, ComicYear, Corrected_SeriesYear, Corrected_Type from comics WHERE Status= " Active " or Status= " Loading " order by LastUpdated DESC, latestDate ASC ' )
2013-07-01 05:19:15 +00:00
else :
FIX:(#1431) Fixed post-processing cleanup when using SAB + files resided in a sub-folder of the path sent from SAB, IMP: Added hardlink/softlink option for non-windows users (note that metatagging is forcibly disabled when these options are enabled), IMP: Added Enforce Permissions option to force Mylar to either force / ensure permissions are as set within Mylar, or to ignore permissions and just try to perform the given action (this is for adding series, post-processing, importing, arcs, etc), FIX: Fixed a problem where if local watchdir was set and a torrent downloader was selected, both would attempt to be used, but only the watchdir was actually utilized (now it's one or the other, never both), FIX: Mylar will now remove issues that are in Failed status that have incorrect Series Data attached, FIX:(#1438) When file-checking/post-processing if issue date in Mylar was current year and date in filename was the following year (for end-of-year issues), would cause traceback error, FIX: If series year polled from CV is incorrect or not provided, option is available in Edit Settings of series detail page that allows user to specify the correct series year, FIX:(#1427) Fixed a typo when the Scan Library option is used and ComicID's were able to be parsed from the files, FIX: When issues were archived, depending on if series had issues in a Downloaded status or not, would give different results to the Have totals, IMP: Manage tab / Manage Comics now has metatag option available to allow metatagging multiple series in a batch mode, IMP:(#1419) When 'write cvinfo' option is enabled, will now also write cvinfo into story arc directories when added / refreshed, FIX: Fixed a problem when group metatag was being used and causing errors, IMP: Manage Failed page now will link back to the series for easier reference, and the 'link' column was removed due to invalid references, FIX: Removed the 'issue not updated' from the series detail page as was not accurate, IMP: Added a Select Filtered Issues button to the series detail page to allow for selecting of all filtered issues (Show ALL MUST be enabled for it to work properly)
2016-11-02 15:00:56 +00:00
comiclist = [ ]
comiclisting = ComicIDList
for cl in comiclisting :
FIX: if 32p was disabled due to inability to signon, would still attempt to retrieve cached result when match occured which would then possibly result in excessive attempted logins, IMP: Added Force Type option to comicdetails page which will allow the type of series to be forced to TPB/GN format if Mylar is unable to decipher it under normal circumstances (doing this will affect how files are parsed for the given series), IMP: TPB/GN File Parser will now account for mutiple issues of the same TPB series within the given directory, IMP: When enabling/disabling Force Type, will rename existing folder to accomodate new naming folder convention if required, IMP: Started to move some of the file rename/manipulation modules into a filer.py module for more consistency, IMP: Added a to Folder Format which when triggered will use the designated Series Type, IMP:(#2130) When importing, option now available for the final series destination path to use the imported path location instead of defaulting to the Comic Location base folder - status will be no longer be Archived in such cases. Will also not show previous imported series on subsequent import runs if the import directory has not changed (thnx @CuddleBear92), IMP: Added the codebase for a Rename Preview option into the mix - GUI available, but not linked, FIX: Removed DEM from RSS scans when Public Torrents option is enabled, FIX:(#2137) When adding series, if series indicated date in a different format other than just numeric, would cause error
2018-12-10 17:27:53 +00:00
comiclist + = myDB . select ( ' SELECT ComicID, ComicName, ComicYear, Corrected_SeriesYear, Corrected_Type, LastUpdated from comics WHERE ComicID=? order by LastUpdated DESC, LatestDate ASC ' , [ cl ] )
2012-09-13 15:27:34 +00:00
IMP:New scheduler tab (Manage / Activity) where you can see job current run status', next runtime, and prev runtimes as well as force/pause a job, FIX: Disabling
torrents will now properly hide torrent information, IMP: Specified daemon port for deluge as an on-screen tip for more detail, IMP: Added 100,200,ALL as viewable
watchlist views, FIX: When viewing pullist and annual integration enabled, if annual was present would incorrectly link to invalid annual series instead of the
actual series itself, IMP: Added more detail error messages to metatagging errors and better handling of stranded files during cleanup, IMP: Improved some handling for weekly pull-list one-off's and refactored the
nzb/oneoff post-processing into a seperate function for future callables, Moved all the main url locations for public torrent sites to the init module so that it
can be cascaded down for use in other modules instead as a global, IMP: Added a 'deep_search_32p' variable in the config.ini for specific usage with 32p, where
if there is more than one result will dig deeper into each result to try and figure out if there are series matches, as opposed to the default where it will only use ref32p
table if available or just the first hit in a multiple series search results and ignore the remainder, FIX:Fixed some unknown characters appearing in the pullist due
to unicode-related conversion problems, FIX: fixed some special cases of file parsing errors due to Volume label being named different than expected, FIX: Added a
3s pause between experimental searches to try and not hit their frequency limitation, IMP: Weekly Pullist One-off's will now show status of Snatched/Downloaded as
required, FIX: Fixed some deluge parameter problems when using auto-snatch torrent script/option, IMP: Changed the downlocation in the auto-snatch option to an
env variable instead of being passed to avoid unicode-related problems, FIX: Fixed some magnet-related issues for torrents when using a watchdir + TPSE, FIX: Added
more verbose error message for rtorrent connection issues, FIX: Could not connect to rtorrent client if no username/password were provided, IMP: Set the db updater to run
every 5 minutes on the watchlist, automatically refreshing the oldest updated series each time that is more than 5 hours old (force db update from the activity/job schedulers page will run the db updater
against the entire watchlist in sequence), IMP: Attempt to handle long paths in windows (ie. > 256c) by prepending the unicode windows api character to the import
a directory path (windows only), IMP: When manual metatagging a series, will update the series after all the metatagging has been completed as opposed to after each
issue, IMP: Will now display available inkdrops on Config/Search Providers tab when using 32P (future will utilize/indicate inkdrop threshold when downloading)
2017-09-10 15:50:08 +00:00
if all ( [ sched is False , calledfrom is None ] ) :
2014-08-18 19:54:34 +00:00
logger . info ( ' Starting update for %i active comics ' % len ( comiclist ) )
2015-05-22 08:32:51 +00:00
IMP: Cleaned up interface for StoryArcs / Story Arc Details, IMP: Cleaned up interface for Reading list Management, IMP: Added better reading list management - new status (added, downloaded, read), IMP: Added sync option for use with another device for reading list transfer (ie. tablet) Android only, IMP: Autopopulate new weekly pull releases to reading list, IMP: 'Watch' option in weekly pull list now fully functional. Will watch CV for series' that do not have any series data yet as they are new starting series. Will auto-add once available, IMP: Auto-watch check is run after every refresh/recreate of the weeklypull list, IMP: Improved the Add a Series option where it will now look for issues that are 'new' or 'wanted' during add sequence, IMP: Main page interface now has coloured have/total bars to denote series completion, IMP: New scheduler / threading locks in place in an attempt to avoid database locks, FIX: Removed some erroneous locking that was going on when importing a directory was being run, IMP: Stat counter now present when post-processing multiple issues in sequence, FIX: for issue number error when post-processing and issue number was a non-alphanumeric, FIX: for metatagging: when original file was .cbz, would try to convert and fail, FIX: for issues that were negative and were preceeded by a # in the filename (filechecker), FIX: for publisher having non-alphanumeric character in name when attempting to determine publisher, FIX: if annuals enabled, would incorrectly show as being 'already in library' when viewing search results if results constained annuals, FIX:(#944) for incorrect nzbname being used when post-processing was being performed from an nzb client (experimental mainly), IMP: Turned off logging for ComicVine API counter, FIX: Added retry attempts when connecting to ComicVine in order to avoid errors when adding a series, IMP:(#963) Added ability to add snatched to filter when viewing Wanted issues on Wanted tab, FIX: When importing and then selecting a series to import via the select screen, will now flip back to the importresults and add the selected series in the background, IMP:(#952) Main page is now sorted in ascending order by Continuing/Ended status (and subbed by whether is Active/Paused).Custom sorting is still available, FIX: Dupecheck will now automatically assume existing 0-byte files are to be overwritten when performing post-processing, FIX: If publication date for series contained a '?' (usually with brand new series) will force to 'Present' to allow for pull-list comparisons to take place, FIX: Mylar will now disallow search results which have 'covers only' or 'variant' in the filename, IMP: Better nzbname generation/retrieval (will check inside nzb for possible names) to be used when post-processing, IMP: DB Update will now perform update to all active comics in descending order by Latest Date (instead of random order), FIX: Enforce the 5hr limit rule when running DB update (will only update series that haven't been updated in >5 hours), FIX: Annuals will now have/retain the proper status upon doing DB Update, FIX: Have totals will now be updated when doing a recheck files (sometimes wouldn't get updated depending on various states of status'), FIX:(#966) Added urllib2.URLError exeception trap when attempting to check Git for updates, IMP: Removed the individual sqlite calls for weeklypull, and brought them into line with using the db module (which will minimize concurrent access, which seemed to be causing db locks), IMP: Cleaned up some code and shuffled some functions so they are in more appropriate locations
2015-03-27 17:27:59 +00:00
cnt = 1
IMP: Added Choose specific Download option to manually select from result list on Upcoming/Details/Weekly tabs, IMP: Added Wanted storyarcs to the overall Wanted section so now will search as per global options (storyarc issues can be displayed optionally on Wanted tab), IMP: Added custom url option for image banners/posters for storyarcs, IMP: updated Cherrypy, FIX: Fixed ComicRN not working when forms authentication used - in conjunction updated autoProcessComics to 2.0 which now uses apikey instead of user/pass, IMP: Alternate Series Covers option for alternating existing series image to the most current issue image, IMP: Added overall series total to series page for reference, IMP: Search workflow completely changed to accomodate more than one indexer - now will simultaneously sumbit initial request to each provider, wait 30s submit additional as required at 30s intervals, FIX: Removed TPSE as an option and relabelled to just Public Torrents, IMP: Added direct backlog search to WWT option (pack support will follow), FIX: Removed line about configparser being required for ComicTagger usage, IMP: Test code in place for newzab testing, FIX: Fixed layout problem with torrents that are in auto-snatch status on weekly tab, IMP: backend code improvements to allow for better alias usage and annual linking directly from WS, IMP: Updated systemd init-scripts with read.me, IMP: When post-processing, will now check for available destination free space before actually moving files, IMP: Will copy during metatagging to cache folder instead of move being an option so cleanup is cleaner if something fails, FIX: Changed readinglist table to storyarcs for clarity, IMP: When post-processing issues, will now only update the one issue status and adjust totals accordingly (instead of doing a complete rescan of the series), FIX: Clear out empty ID's from the Failed DB on startup, IMP: Initial code-run at REST API interface, FIX: Fixed some matching problems with 32p due to case, IMP: removed apikeys from log entries that were accidentally logging, IMP: When searching 32p, if items get packed up - will now delete the cached reference so new items of the same can be located, IMP: ForceSearch option switched to scheduler so simultaneous runs should not occur, FIX: Fixed manual metatagging error that would occur if multiple destination directories existed
2018-02-16 19:57:01 +00:00
if sched is True :
logger . fdebug ( ' Refresh sequence set to fire every %s minutes for %s day(s) ' % ( mylar . DBUPDATE_INTERVAL , mylar . CONFIG . REFRESH_CACHE ) )
2017-10-25 18:33:56 +00:00
IMP:New scheduler tab (Manage / Activity) where you can see job current run status', next runtime, and prev runtimes as well as force/pause a job, FIX: Disabling
torrents will now properly hide torrent information, IMP: Specified daemon port for deluge as an on-screen tip for more detail, IMP: Added 100,200,ALL as viewable
watchlist views, FIX: When viewing pullist and annual integration enabled, if annual was present would incorrectly link to invalid annual series instead of the
actual series itself, IMP: Added more detail error messages to metatagging errors and better handling of stranded files during cleanup, IMP: Improved some handling for weekly pull-list one-off's and refactored the
nzb/oneoff post-processing into a seperate function for future callables, Moved all the main url locations for public torrent sites to the init module so that it
can be cascaded down for use in other modules instead as a global, IMP: Added a 'deep_search_32p' variable in the config.ini for specific usage with 32p, where
if there is more than one result will dig deeper into each result to try and figure out if there are series matches, as opposed to the default where it will only use ref32p
table if available or just the first hit in a multiple series search results and ignore the remainder, FIX:Fixed some unknown characters appearing in the pullist due
to unicode-related conversion problems, FIX: fixed some special cases of file parsing errors due to Volume label being named different than expected, FIX: Added a
3s pause between experimental searches to try and not hit their frequency limitation, IMP: Weekly Pullist One-off's will now show status of Snatched/Downloaded as
required, FIX: Fixed some deluge parameter problems when using auto-snatch torrent script/option, IMP: Changed the downlocation in the auto-snatch option to an
env variable instead of being passed to avoid unicode-related problems, FIX: Fixed some magnet-related issues for torrents when using a watchdir + TPSE, FIX: Added
more verbose error message for rtorrent connection issues, FIX: Could not connect to rtorrent client if no username/password were provided, IMP: Set the db updater to run
every 5 minutes on the watchlist, automatically refreshing the oldest updated series each time that is more than 5 hours old (force db update from the activity/job schedulers page will run the db updater
against the entire watchlist in sequence), IMP: Attempt to handle long paths in windows (ie. > 256c) by prepending the unicode windows api character to the import
a directory path (windows only), IMP: When manual metatagging a series, will update the series after all the metatagging has been completed as opposed to after each
issue, IMP: Will now display available inkdrops on Config/Search Providers tab when using 32P (future will utilize/indicate inkdrop threshold when downloading)
2017-09-10 15:50:08 +00:00
for comic in sorted ( comiclist , key = operator . itemgetter ( ' LastUpdated ' ) , reverse = True ) :
FIX:(#1466) When post-processing if filename did not have issue year or volume, would error, IMP:(#1418) Separate copy/move/hardlink/symlink option for Story arcs, IMP:(#1417) Added story arc format folder to story arc options - can now specify folder format for story arc naming, IMP:(#1374) Added rtorrent_authentication to config.ini allowing user to specify digest/basic authentication, as well as the rpc_url (rtorrent_rpc_url) and ssl (rtorrent_ssl). Rtorrent_host no longer requires protocol in field, IMP: Instead of hitting the intermediary/32P site to get the torrent group, will now retain specific id for torrent group to speed up searches, FIX: (#1472) Fixed some problems with series being searched against 32P which had special characters and would return no results, FIX: Added some specific exception catches when attempting to retrieve search results for nzb providers, FIX: Fixed publisher retaining problem for story arcs (note story arcs will have to be refreshed), FIX: Fixed a problem with the pull-list matching up to the correct series when the issue was an annual for a given series with the alt_pull 2 method (would create an invalid link on the Wanted tab), IMP:(#1470) Added config.ini option to specify folder-formatting for weekly folder numbering, 0=YYYY-WN, 1=YYYY-MM-DD, IMP: (#1471) Added quick check to ensure that newznab host entries do not contain leading/trailing spaces and will strip accordingly, FIX:(#1476) Annuals now have option to add to reading list via the series detail page, IMP: Fixed password fields to allow for special characters within password, IMP: Weekly pull will now correctly show status for previous/future weeks for watchlisted series, IMP: Download option on pullist will work for previous weeks, FIX: Fixed some issues with one-off and story arc post-processing using different file operations, FIX: Fixed error when performing an 'Force Update Active Comics' from the Manage tab (or the scheduled updating of all active comics), IMP: Storyarc issues that are not on watchlist will now retain value for volume to help when searching and file-checking/post-processing, FIX:(#1473) Changed pullist dates on pullist page to unicode to allow for non-English presentation
2016-12-02 17:45:49 +00:00
dspyear = comic [ ' ComicYear ' ]
csyear = None
FIX: if 32p was disabled due to inability to signon, would still attempt to retrieve cached result when match occured which would then possibly result in excessive attempted logins, IMP: Added Force Type option to comicdetails page which will allow the type of series to be forced to TPB/GN format if Mylar is unable to decipher it under normal circumstances (doing this will affect how files are parsed for the given series), IMP: TPB/GN File Parser will now account for mutiple issues of the same TPB series within the given directory, IMP: When enabling/disabling Force Type, will rename existing folder to accomodate new naming folder convention if required, IMP: Started to move some of the file rename/manipulation modules into a filer.py module for more consistency, IMP: Added a to Folder Format which when triggered will use the designated Series Type, IMP:(#2130) When importing, option now available for the final series destination path to use the imported path location instead of defaulting to the Comic Location base folder - status will be no longer be Archived in such cases. Will also not show previous imported series on subsequent import runs if the import directory has not changed (thnx @CuddleBear92), IMP: Added the codebase for a Rename Preview option into the mix - GUI available, but not linked, FIX: Removed DEM from RSS scans when Public Torrents option is enabled, FIX:(#2137) When adding series, if series indicated date in a different format other than just numeric, would cause error
2018-12-10 17:27:53 +00:00
fixed_type = None
if comic [ ' Corrected_Type ' ] is not None :
fixed_type = comic [ ' Corrected_Type ' ]
FIX:(#1466) When post-processing if filename did not have issue year or volume, would error, IMP:(#1418) Separate copy/move/hardlink/symlink option for Story arcs, IMP:(#1417) Added story arc format folder to story arc options - can now specify folder format for story arc naming, IMP:(#1374) Added rtorrent_authentication to config.ini allowing user to specify digest/basic authentication, as well as the rpc_url (rtorrent_rpc_url) and ssl (rtorrent_ssl). Rtorrent_host no longer requires protocol in field, IMP: Instead of hitting the intermediary/32P site to get the torrent group, will now retain specific id for torrent group to speed up searches, FIX: (#1472) Fixed some problems with series being searched against 32P which had special characters and would return no results, FIX: Added some specific exception catches when attempting to retrieve search results for nzb providers, FIX: Fixed publisher retaining problem for story arcs (note story arcs will have to be refreshed), FIX: Fixed a problem with the pull-list matching up to the correct series when the issue was an annual for a given series with the alt_pull 2 method (would create an invalid link on the Wanted tab), IMP:(#1470) Added config.ini option to specify folder-formatting for weekly folder numbering, 0=YYYY-WN, 1=YYYY-MM-DD, IMP: (#1471) Added quick check to ensure that newznab host entries do not contain leading/trailing spaces and will strip accordingly, FIX:(#1476) Annuals now have option to add to reading list via the series detail page, IMP: Fixed password fields to allow for special characters within password, IMP: Weekly pull will now correctly show status for previous/future weeks for watchlisted series, IMP: Download option on pullist will work for previous weeks, FIX: Fixed some issues with one-off and story arc post-processing using different file operations, FIX: Fixed error when performing an 'Force Update Active Comics' from the Manage tab (or the scheduled updating of all active comics), IMP: Storyarc issues that are not on watchlist will now retain value for volume to help when searching and file-checking/post-processing, FIX:(#1473) Changed pullist dates on pullist page to unicode to allow for non-English presentation
2016-12-02 17:45:49 +00:00
if comic [ ' Corrected_SeriesYear ' ] is not None :
csyear = comic [ ' Corrected_SeriesYear ' ]
if int ( csyear ) != int ( comic [ ' ComicYear ' ] ) :
comic [ ' ComicYear ' ] = csyear
dspyear = csyear
2013-07-01 05:19:15 +00:00
if ComicIDList is None :
2015-04-28 02:15:53 +00:00
ComicID = comic [ ' ComicID ' ]
ComicName = comic [ ' ComicName ' ]
c_date = comic [ ' LastUpdated ' ]
IMP: Cleaned up interface for StoryArcs / Story Arc Details, IMP: Cleaned up interface for Reading list Management, IMP: Added better reading list management - new status (added, downloaded, read), IMP: Added sync option for use with another device for reading list transfer (ie. tablet) Android only, IMP: Autopopulate new weekly pull releases to reading list, IMP: 'Watch' option in weekly pull list now fully functional. Will watch CV for series' that do not have any series data yet as they are new starting series. Will auto-add once available, IMP: Auto-watch check is run after every refresh/recreate of the weeklypull list, IMP: Improved the Add a Series option where it will now look for issues that are 'new' or 'wanted' during add sequence, IMP: Main page interface now has coloured have/total bars to denote series completion, IMP: New scheduler / threading locks in place in an attempt to avoid database locks, FIX: Removed some erroneous locking that was going on when importing a directory was being run, IMP: Stat counter now present when post-processing multiple issues in sequence, FIX: for issue number error when post-processing and issue number was a non-alphanumeric, FIX: for metatagging: when original file was .cbz, would try to convert and fail, FIX: for issues that were negative and were preceeded by a # in the filename (filechecker), FIX: for publisher having non-alphanumeric character in name when attempting to determine publisher, FIX: if annuals enabled, would incorrectly show as being 'already in library' when viewing search results if results constained annuals, FIX:(#944) for incorrect nzbname being used when post-processing was being performed from an nzb client (experimental mainly), IMP: Turned off logging for ComicVine API counter, FIX: Added retry attempts when connecting to ComicVine in order to avoid errors when adding a series, IMP:(#963) Added ability to add snatched to filter when viewing Wanted issues on Wanted tab, FIX: When importing and then selecting a series to import via the select screen, will now flip back to the importresults and add the selected series in the background, IMP:(#952) Main page is now sorted in ascending order by Continuing/Ended status (and subbed by whether is Active/Paused).Custom sorting is still available, FIX: Dupecheck will now automatically assume existing 0-byte files are to be overwritten when performing post-processing, FIX: If publication date for series contained a '?' (usually with brand new series) will force to 'Present' to allow for pull-list comparisons to take place, FIX: Mylar will now disallow search results which have 'covers only' or 'variant' in the filename, IMP: Better nzbname generation/retrieval (will check inside nzb for possible names) to be used when post-processing, IMP: DB Update will now perform update to all active comics in descending order by Latest Date (instead of random order), FIX: Enforce the 5hr limit rule when running DB update (will only update series that haven't been updated in >5 hours), FIX: Annuals will now have/retain the proper status upon doing DB Update, FIX: Have totals will now be updated when doing a recheck files (sometimes wouldn't get updated depending on various states of status'), FIX:(#966) Added urllib2.URLError exeception trap when attempting to check Git for updates, IMP: Removed the individual sqlite calls for weeklypull, and brought them into line with using the db module (which will minimize concurrent access, which seemed to be causing db locks), IMP: Cleaned up some code and shuffled some functions so they are in more appropriate locations
2015-03-27 17:27:59 +00:00
if c_date is None :
logger . error ( ComicName + ' failed during a previous add /refresh as it has no Last Update timestamp. Forcing refresh now. ' )
else :
c_obj_date = datetime . datetime . strptime ( c_date , " % Y- % m- %d % H: % M: % S " )
n_date = datetime . datetime . now ( )
absdiff = abs ( n_date - c_obj_date )
hours = ( absdiff . days * 24 * 60 * 60 + absdiff . seconds ) / 3600.0
2017-10-25 18:33:56 +00:00
cache_hours = mylar . CONFIG . REFRESH_CACHE * 24
if hours < cache_hours :
#logger.fdebug('%s [%s] Was refreshed less than %s hours ago. Skipping Refresh at this time.' % (ComicName, ComicID, cache_hours))
IMP: Cleaned up interface for StoryArcs / Story Arc Details, IMP: Cleaned up interface for Reading list Management, IMP: Added better reading list management - new status (added, downloaded, read), IMP: Added sync option for use with another device for reading list transfer (ie. tablet) Android only, IMP: Autopopulate new weekly pull releases to reading list, IMP: 'Watch' option in weekly pull list now fully functional. Will watch CV for series' that do not have any series data yet as they are new starting series. Will auto-add once available, IMP: Auto-watch check is run after every refresh/recreate of the weeklypull list, IMP: Improved the Add a Series option where it will now look for issues that are 'new' or 'wanted' during add sequence, IMP: Main page interface now has coloured have/total bars to denote series completion, IMP: New scheduler / threading locks in place in an attempt to avoid database locks, FIX: Removed some erroneous locking that was going on when importing a directory was being run, IMP: Stat counter now present when post-processing multiple issues in sequence, FIX: for issue number error when post-processing and issue number was a non-alphanumeric, FIX: for metatagging: when original file was .cbz, would try to convert and fail, FIX: for issues that were negative and were preceeded by a # in the filename (filechecker), FIX: for publisher having non-alphanumeric character in name when attempting to determine publisher, FIX: if annuals enabled, would incorrectly show as being 'already in library' when viewing search results if results constained annuals, FIX:(#944) for incorrect nzbname being used when post-processing was being performed from an nzb client (experimental mainly), IMP: Turned off logging for ComicVine API counter, FIX: Added retry attempts when connecting to ComicVine in order to avoid errors when adding a series, IMP:(#963) Added ability to add snatched to filter when viewing Wanted issues on Wanted tab, FIX: When importing and then selecting a series to import via the select screen, will now flip back to the importresults and add the selected series in the background, IMP:(#952) Main page is now sorted in ascending order by Continuing/Ended status (and subbed by whether is Active/Paused).Custom sorting is still available, FIX: Dupecheck will now automatically assume existing 0-byte files are to be overwritten when performing post-processing, FIX: If publication date for series contained a '?' (usually with brand new series) will force to 'Present' to allow for pull-list comparisons to take place, FIX: Mylar will now disallow search results which have 'covers only' or 'variant' in the filename, IMP: Better nzbname generation/retrieval (will check inside nzb for possible names) to be used when post-processing, IMP: DB Update will now perform update to all active comics in descending order by Latest Date (instead of random order), FIX: Enforce the 5hr limit rule when running DB update (will only update series that haven't been updated in >5 hours), FIX: Annuals will now have/retain the proper status upon doing DB Update, FIX: Have totals will now be updated when doing a recheck files (sometimes wouldn't get updated depending on various states of status'), FIX:(#966) Added urllib2.URLError exeception trap when attempting to check Git for updates, IMP: Removed the individual sqlite calls for weeklypull, and brought them into line with using the db module (which will minimize concurrent access, which seemed to be causing db locks), IMP: Cleaned up some code and shuffled some functions so they are in more appropriate locations
2015-03-27 17:27:59 +00:00
cnt + = 1
continue
2019-02-02 21:18:57 +00:00
logger . info ( ' [ %s / %s ] Refreshing : %s ( %s ) [ %s ] ' % ( cnt , len ( comiclist ) , ComicName , dspyear , ComicID ) )
2013-07-01 05:19:15 +00:00
else :
FIX:(#1431) Fixed post-processing cleanup when using SAB + files resided in a sub-folder of the path sent from SAB, IMP: Added hardlink/softlink option for non-windows users (note that metatagging is forcibly disabled when these options are enabled), IMP: Added Enforce Permissions option to force Mylar to either force / ensure permissions are as set within Mylar, or to ignore permissions and just try to perform the given action (this is for adding series, post-processing, importing, arcs, etc), FIX: Fixed a problem where if local watchdir was set and a torrent downloader was selected, both would attempt to be used, but only the watchdir was actually utilized (now it's one or the other, never both), FIX: Mylar will now remove issues that are in Failed status that have incorrect Series Data attached, FIX:(#1438) When file-checking/post-processing if issue date in Mylar was current year and date in filename was the following year (for end-of-year issues), would cause traceback error, FIX: If series year polled from CV is incorrect or not provided, option is available in Edit Settings of series detail page that allows user to specify the correct series year, FIX:(#1427) Fixed a typo when the Scan Library option is used and ComicID's were able to be parsed from the files, FIX: When issues were archived, depending on if series had issues in a Downloaded status or not, would give different results to the Have totals, IMP: Manage tab / Manage Comics now has metatag option available to allow metatagging multiple series in a batch mode, IMP:(#1419) When 'write cvinfo' option is enabled, will now also write cvinfo into story arc directories when added / refreshed, FIX: Fixed a problem when group metatag was being used and causing errors, IMP: Manage Failed page now will link back to the series for easier reference, and the 'link' column was removed due to invalid references, FIX: Removed the 'issue not updated' from the series detail page as was not accurate, IMP: Added a Select Filtered Issues button to the series detail page to allow for selecting of all filtered issues (Show ALL MUST be enabled for it to work properly)
2016-11-02 15:00:56 +00:00
ComicID = comic [ ' ComicID ' ]
ComicName = comic [ ' ComicName ' ]
IMP:New scheduler tab (Manage / Activity) where you can see job current run status', next runtime, and prev runtimes as well as force/pause a job, FIX: Disabling
torrents will now properly hide torrent information, IMP: Specified daemon port for deluge as an on-screen tip for more detail, IMP: Added 100,200,ALL as viewable
watchlist views, FIX: When viewing pullist and annual integration enabled, if annual was present would incorrectly link to invalid annual series instead of the
actual series itself, IMP: Added more detail error messages to metatagging errors and better handling of stranded files during cleanup, IMP: Improved some handling for weekly pull-list one-off's and refactored the
nzb/oneoff post-processing into a seperate function for future callables, Moved all the main url locations for public torrent sites to the init module so that it
can be cascaded down for use in other modules instead as a global, IMP: Added a 'deep_search_32p' variable in the config.ini for specific usage with 32p, where
if there is more than one result will dig deeper into each result to try and figure out if there are series matches, as opposed to the default where it will only use ref32p
table if available or just the first hit in a multiple series search results and ignore the remainder, FIX:Fixed some unknown characters appearing in the pullist due
to unicode-related conversion problems, FIX: fixed some special cases of file parsing errors due to Volume label being named different than expected, FIX: Added a
3s pause between experimental searches to try and not hit their frequency limitation, IMP: Weekly Pullist One-off's will now show status of Snatched/Downloaded as
required, FIX: Fixed some deluge parameter problems when using auto-snatch torrent script/option, IMP: Changed the downlocation in the auto-snatch option to an
env variable instead of being passed to avoid unicode-related problems, FIX: Fixed some magnet-related issues for torrents when using a watchdir + TPSE, FIX: Added
more verbose error message for rtorrent connection issues, FIX: Could not connect to rtorrent client if no username/password were provided, IMP: Set the db updater to run
every 5 minutes on the watchlist, automatically refreshing the oldest updated series each time that is more than 5 hours old (force db update from the activity/job schedulers page will run the db updater
against the entire watchlist in sequence), IMP: Attempt to handle long paths in windows (ie. > 256c) by prepending the unicode windows api character to the import
a directory path (windows only), IMP: When manual metatagging a series, will update the series after all the metatagging has been completed as opposed to after each
issue, IMP: Will now display available inkdrops on Config/Search Providers tab when using 32P (future will utilize/indicate inkdrop threshold when downloading)
2017-09-10 15:50:08 +00:00
2019-02-02 21:18:57 +00:00
logger . info ( ' Refreshing/Updating: %s ( %s ) [ %s ] ' % ( ComicName , dspyear , ComicID ) )
FIX:(#1431) Fixed post-processing cleanup when using SAB + files resided in a sub-folder of the path sent from SAB, IMP: Added hardlink/softlink option for non-windows users (note that metatagging is forcibly disabled when these options are enabled), IMP: Added Enforce Permissions option to force Mylar to either force / ensure permissions are as set within Mylar, or to ignore permissions and just try to perform the given action (this is for adding series, post-processing, importing, arcs, etc), FIX: Fixed a problem where if local watchdir was set and a torrent downloader was selected, both would attempt to be used, but only the watchdir was actually utilized (now it's one or the other, never both), FIX: Mylar will now remove issues that are in Failed status that have incorrect Series Data attached, FIX:(#1438) When file-checking/post-processing if issue date in Mylar was current year and date in filename was the following year (for end-of-year issues), would cause traceback error, FIX: If series year polled from CV is incorrect or not provided, option is available in Edit Settings of series detail page that allows user to specify the correct series year, FIX:(#1427) Fixed a typo when the Scan Library option is used and ComicID's were able to be parsed from the files, FIX: When issues were archived, depending on if series had issues in a Downloaded status or not, would give different results to the Have totals, IMP: Manage tab / Manage Comics now has metatag option available to allow metatagging multiple series in a batch mode, IMP:(#1419) When 'write cvinfo' option is enabled, will now also write cvinfo into story arc directories when added / refreshed, FIX: Fixed a problem when group metatag was being used and causing errors, IMP: Manage Failed page now will link back to the series for easier reference, and the 'link' column was removed due to invalid references, FIX: Removed the 'issue not updated' from the series detail page as was not accurate, IMP: Added a Select Filtered Issues button to the series detail page to allow for selecting of all filtered issues (Show ALL MUST be enabled for it to work properly)
2016-11-02 15:00:56 +00:00
2013-02-17 10:31:18 +00:00
mismatch = " no "
2017-10-25 18:33:56 +00:00
if not mylar . CONFIG . CV_ONLY or ComicID [ : 1 ] == " G " :
2014-05-25 19:08:15 +00:00
CV_EXcomicid = myDB . selectone ( " SELECT * from exceptions WHERE ComicID=? " , [ ComicID ] ) . fetchone ( )
FIX:(#304) Index out of range on recheck, FIX:(#303) Comicvine link updated on details page, FIX:(#302) Query rate (Search Delay) Added as a configuration option, FIX:(#300) Version number error on searching when no version number, FIX:(#297) Manual Rename of Files working, FIX:(#294) 'AU' issue problems should be resolved now, FIX:(#290) V#(year) and Vol#(year) added to filechecking, IMP: ComicVine data use only now as a hidden option (cv_only = 1 in config.ini), IMP: added as options for file naming, IMP: Rough drafting of Annuals (annuals_on = 1 in config.ini), Other fixes..
2013-04-06 09:43:18 +00:00
if CV_EXcomicid is None : pass
else :
if CV_EXcomicid [ ' variloop ' ] == ' 99 ' :
mismatch = " yes "
2015-06-22 03:38:10 +00:00
if ComicID [ : 1 ] == " G " :
mylar . importer . GCDimport ( ComicID )
else :
cchk = importer . addComictoDB ( ComicID , mismatch )
2013-02-17 10:31:18 +00:00
else :
2017-10-25 18:33:56 +00:00
if mylar . CONFIG . CV_ONETIMER == 1 :
IMP:New scheduler tab (Manage / Activity) where you can see job current run status', next runtime, and prev runtimes as well as force/pause a job, FIX: Disabling
torrents will now properly hide torrent information, IMP: Specified daemon port for deluge as an on-screen tip for more detail, IMP: Added 100,200,ALL as viewable
watchlist views, FIX: When viewing pullist and annual integration enabled, if annual was present would incorrectly link to invalid annual series instead of the
actual series itself, IMP: Added more detail error messages to metatagging errors and better handling of stranded files during cleanup, IMP: Improved some handling for weekly pull-list one-off's and refactored the
nzb/oneoff post-processing into a seperate function for future callables, Moved all the main url locations for public torrent sites to the init module so that it
can be cascaded down for use in other modules instead as a global, IMP: Added a 'deep_search_32p' variable in the config.ini for specific usage with 32p, where
if there is more than one result will dig deeper into each result to try and figure out if there are series matches, as opposed to the default where it will only use ref32p
table if available or just the first hit in a multiple series search results and ignore the remainder, FIX:Fixed some unknown characters appearing in the pullist due
to unicode-related conversion problems, FIX: fixed some special cases of file parsing errors due to Volume label being named different than expected, FIX: Added a
3s pause between experimental searches to try and not hit their frequency limitation, IMP: Weekly Pullist One-off's will now show status of Snatched/Downloaded as
required, FIX: Fixed some deluge parameter problems when using auto-snatch torrent script/option, IMP: Changed the downlocation in the auto-snatch option to an
env variable instead of being passed to avoid unicode-related problems, FIX: Fixed some magnet-related issues for torrents when using a watchdir + TPSE, FIX: Added
more verbose error message for rtorrent connection issues, FIX: Could not connect to rtorrent client if no username/password were provided, IMP: Set the db updater to run
every 5 minutes on the watchlist, automatically refreshing the oldest updated series each time that is more than 5 hours old (force db update from the activity/job schedulers page will run the db updater
against the entire watchlist in sequence), IMP: Attempt to handle long paths in windows (ie. > 256c) by prepending the unicode windows api character to the import
a directory path (windows only), IMP: When manual metatagging a series, will update the series after all the metatagging has been completed as opposed to after each
issue, IMP: Will now display available inkdrops on Config/Search Providers tab when using 32P (future will utilize/indicate inkdrop threshold when downloading)
2017-09-10 15:50:08 +00:00
if sched is True :
helpers . job_management ( write = True , job = ' DB Updater ' , current_run = helpers . utctimestamp ( ) , status = ' Running ' )
mylar . UPDATER_STATUS = ' Running '
2014-05-25 19:08:15 +00:00
logger . fdebug ( " CV_OneTimer option enabled... " )
#in order to update to JUST CV_ONLY, we need to delete the issues for a given series so it's a clea$
logger . fdebug ( " Gathering the status of all issues for the series. " )
issues = myDB . select ( ' SELECT * FROM issues WHERE ComicID=? ' , [ ComicID ] )
FIX: (#746) updated autoProcessComics.py / ComicRN.py's which will now send proper completion messeages to clients, FIX: (#752) refresh series will now test if a series is 'out of whack' with it's numerical issue count (ie. 5/4) or it has no issue data due to a bad refresh / api maxing out and will adjust it's processing to accomodate either, IMP: (#750) Added ComicVine API Checker which will check API counts at regular intervals to inform/warn users of usage, as well as adding a screen-time display of API hits / mins used at the bottom of every page (refreshing/reloading pages will update counts), FIX: (#747)EOL normalization (dos2unix) on search.py - removed classes & exceptions as not being used, IMP: (#747) Skip processing issues with an invalid store date & issue date (thnx rupaschomaker), FIX: Removed strings when searching/logging torrents as was causing ascii errors especially with KAT, IMP: Added [META-TAGGING] to logging for meta-tagging module, IMP: Added ability in GUI to select CR or Cbl tags (or both) when writing metadata to cbz files, IMP: Improved support/usage with ComicTagger v1.1.15 which allows for personal CV API Key usage - if supplied to Mylar, will use when tagging with ComicTagger, IMP: Added Manual Search option to allow for individual searches of issues without changing initial status.
2014-06-18 19:58:19 +00:00
if not issues :
#if issues are None it's probably a bad refresh/maxed out API that resulted in the issue data
#getting wiped out and not refreshed. Setting whack=True will force a complete refresh.
2014-08-18 19:54:34 +00:00
logger . fdebug ( ' No issue data available. This is Whack. ' )
FIX: (#746) updated autoProcessComics.py / ComicRN.py's which will now send proper completion messeages to clients, FIX: (#752) refresh series will now test if a series is 'out of whack' with it's numerical issue count (ie. 5/4) or it has no issue data due to a bad refresh / api maxing out and will adjust it's processing to accomodate either, IMP: (#750) Added ComicVine API Checker which will check API counts at regular intervals to inform/warn users of usage, as well as adding a screen-time display of API hits / mins used at the bottom of every page (refreshing/reloading pages will update counts), FIX: (#747)EOL normalization (dos2unix) on search.py - removed classes & exceptions as not being used, IMP: (#747) Skip processing issues with an invalid store date & issue date (thnx rupaschomaker), FIX: Removed strings when searching/logging torrents as was causing ascii errors especially with KAT, IMP: Added [META-TAGGING] to logging for meta-tagging module, IMP: Added ability in GUI to select CR or Cbl tags (or both) when writing metadata to cbz files, IMP: Improved support/usage with ComicTagger v1.1.15 which allows for personal CV API Key usage - if supplied to Mylar, will use when tagging with ComicTagger, IMP: Added Manual Search option to allow for individual searches of issues without changing initial status.
2014-06-18 19:58:19 +00:00
whack = True
else :
#check for series that are numerically out of whack (ie. 5/4)
2014-08-18 19:54:34 +00:00
logger . fdebug ( ' Checking how out of whack the series is. ' )
FIX: (#746) updated autoProcessComics.py / ComicRN.py's which will now send proper completion messeages to clients, FIX: (#752) refresh series will now test if a series is 'out of whack' with it's numerical issue count (ie. 5/4) or it has no issue data due to a bad refresh / api maxing out and will adjust it's processing to accomodate either, IMP: (#750) Added ComicVine API Checker which will check API counts at regular intervals to inform/warn users of usage, as well as adding a screen-time display of API hits / mins used at the bottom of every page (refreshing/reloading pages will update counts), FIX: (#747)EOL normalization (dos2unix) on search.py - removed classes & exceptions as not being used, IMP: (#747) Skip processing issues with an invalid store date & issue date (thnx rupaschomaker), FIX: Removed strings when searching/logging torrents as was causing ascii errors especially with KAT, IMP: Added [META-TAGGING] to logging for meta-tagging module, IMP: Added ability in GUI to select CR or Cbl tags (or both) when writing metadata to cbz files, IMP: Improved support/usage with ComicTagger v1.1.15 which allows for personal CV API Key usage - if supplied to Mylar, will use when tagging with ComicTagger, IMP: Added Manual Search option to allow for individual searches of issues without changing initial status.
2014-06-18 19:58:19 +00:00
whack = helpers . havetotals ( refreshit = ComicID )
2014-08-18 19:54:34 +00:00
if calledfrom == ' weekly ' :
if whack == True :
logger . info ( ' Series is out of whack. Forcibly refreshing series to ensure everything is in order. ' )
return True
else :
return False
2014-05-25 19:08:15 +00:00
annload = [ ] #initiate the list here so we don't error out below.
2013-04-07 19:18:26 +00:00
2017-10-25 18:33:56 +00:00
if mylar . CONFIG . ANNUALS_ON :
2014-05-25 19:08:15 +00:00
#now we load the annuals into memory to pass through to importer when refreshing so that it can
#refresh even the manually added annuals.
annual_load = myDB . select ( ' SELECT * FROM annuals WHERE ComicID=? ' , [ ComicID ] )
logger . fdebug ( ' checking annual db ' )
for annthis in annual_load :
if not any ( d [ ' ReleaseComicID ' ] == annthis [ ' ReleaseComicID ' ] for d in annload ) :
#print 'matched on annual'
annload . append ( {
' ReleaseComicID ' : annthis [ ' ReleaseComicID ' ] ,
' ReleaseComicName ' : annthis [ ' ReleaseComicName ' ] ,
' ComicID ' : annthis [ ' ComicID ' ] ,
' ComicName ' : annthis [ ' ComicName ' ]
} )
#print 'added annual'
issues + = annual_load #myDB.select('SELECT * FROM annuals WHERE ComicID=?', [ComicID])
#store the issues' status for a given comicid, after deleting and readding, flip the status back to$
2017-09-14 18:49:24 +00:00
#logger.fdebug("Deleting all issue data.")
#myDB.action('DELETE FROM issues WHERE ComicID=?', [ComicID])
#myDB.action('DELETE FROM annuals WHERE ComicID=?', [ComicID])
2014-05-25 19:08:15 +00:00
logger . fdebug ( " Refreshing the series and pulling in new data using only CV. " )
FIX: (#746) updated autoProcessComics.py / ComicRN.py's which will now send proper completion messeages to clients, FIX: (#752) refresh series will now test if a series is 'out of whack' with it's numerical issue count (ie. 5/4) or it has no issue data due to a bad refresh / api maxing out and will adjust it's processing to accomodate either, IMP: (#750) Added ComicVine API Checker which will check API counts at regular intervals to inform/warn users of usage, as well as adding a screen-time display of API hits / mins used at the bottom of every page (refreshing/reloading pages will update counts), FIX: (#747)EOL normalization (dos2unix) on search.py - removed classes & exceptions as not being used, IMP: (#747) Skip processing issues with an invalid store date & issue date (thnx rupaschomaker), FIX: Removed strings when searching/logging torrents as was causing ascii errors especially with KAT, IMP: Added [META-TAGGING] to logging for meta-tagging module, IMP: Added ability in GUI to select CR or Cbl tags (or both) when writing metadata to cbz files, IMP: Improved support/usage with ComicTagger v1.1.15 which allows for personal CV API Key usage - if supplied to Mylar, will use when tagging with ComicTagger, IMP: Added Manual Search option to allow for individual searches of issues without changing initial status.
2014-06-18 19:58:19 +00:00
if whack == False :
FIX: if 32p was disabled due to inability to signon, would still attempt to retrieve cached result when match occured which would then possibly result in excessive attempted logins, IMP: Added Force Type option to comicdetails page which will allow the type of series to be forced to TPB/GN format if Mylar is unable to decipher it under normal circumstances (doing this will affect how files are parsed for the given series), IMP: TPB/GN File Parser will now account for mutiple issues of the same TPB series within the given directory, IMP: When enabling/disabling Force Type, will rename existing folder to accomodate new naming folder convention if required, IMP: Started to move some of the file rename/manipulation modules into a filer.py module for more consistency, IMP: Added a to Folder Format which when triggered will use the designated Series Type, IMP:(#2130) When importing, option now available for the final series destination path to use the imported path location instead of defaulting to the Comic Location base folder - status will be no longer be Archived in such cases. Will also not show previous imported series on subsequent import runs if the import directory has not changed (thnx @CuddleBear92), IMP: Added the codebase for a Rename Preview option into the mix - GUI available, but not linked, FIX: Removed DEM from RSS scans when Public Torrents option is enabled, FIX:(#2137) When adding series, if series indicated date in a different format other than just numeric, would cause error
2018-12-10 17:27:53 +00:00
chkstatus = mylar . importer . addComictoDB ( ComicID , mismatch , calledfrom = ' dbupdate ' , annload = annload , csyear = csyear , fixed_type = fixed_type )
2017-09-19 15:41:48 +00:00
if chkstatus [ ' status ' ] == ' complete ' :
2017-09-14 18:49:24 +00:00
#delete the data here if it's all valid.
logger . fdebug ( " Deleting all old issue data to make sure new data is clean... " )
myDB . action ( ' DELETE FROM issues WHERE ComicID=? ' , [ ComicID ] )
myDB . action ( ' DELETE FROM annuals WHERE ComicID=? ' , [ ComicID ] )
2017-09-19 15:41:48 +00:00
mylar . importer . issue_collection ( chkstatus [ ' issuedata ' ] , nostatus = ' True ' )
2017-09-18 20:07:21 +00:00
#need to update annuals at this point too....
2017-09-19 15:41:48 +00:00
if chkstatus [ ' anndata ' ] is not None :
mylar . importer . manualAnnual ( annchk = chkstatus [ ' anndata ' ] )
2017-09-14 18:49:24 +00:00
else :
logger . warn ( ' There was an error when refreshing this series - Make sure directories are writable/exist, etc ' )
return
FIX: (#746) updated autoProcessComics.py / ComicRN.py's which will now send proper completion messeages to clients, FIX: (#752) refresh series will now test if a series is 'out of whack' with it's numerical issue count (ie. 5/4) or it has no issue data due to a bad refresh / api maxing out and will adjust it's processing to accomodate either, IMP: (#750) Added ComicVine API Checker which will check API counts at regular intervals to inform/warn users of usage, as well as adding a screen-time display of API hits / mins used at the bottom of every page (refreshing/reloading pages will update counts), FIX: (#747)EOL normalization (dos2unix) on search.py - removed classes & exceptions as not being used, IMP: (#747) Skip processing issues with an invalid store date & issue date (thnx rupaschomaker), FIX: Removed strings when searching/logging torrents as was causing ascii errors especially with KAT, IMP: Added [META-TAGGING] to logging for meta-tagging module, IMP: Added ability in GUI to select CR or Cbl tags (or both) when writing metadata to cbz files, IMP: Improved support/usage with ComicTagger v1.1.15 which allows for personal CV API Key usage - if supplied to Mylar, will use when tagging with ComicTagger, IMP: Added Manual Search option to allow for individual searches of issues without changing initial status.
2014-06-18 19:58:19 +00:00
issues_new = myDB . select ( ' SELECT * FROM issues WHERE ComicID=? ' , [ ComicID ] )
annuals = [ ]
ann_list = [ ]
2017-09-14 18:49:24 +00:00
#reload the annuals here.
2017-10-25 18:33:56 +00:00
if mylar . CONFIG . ANNUALS_ON :
FIX: (#746) updated autoProcessComics.py / ComicRN.py's which will now send proper completion messeages to clients, FIX: (#752) refresh series will now test if a series is 'out of whack' with it's numerical issue count (ie. 5/4) or it has no issue data due to a bad refresh / api maxing out and will adjust it's processing to accomodate either, IMP: (#750) Added ComicVine API Checker which will check API counts at regular intervals to inform/warn users of usage, as well as adding a screen-time display of API hits / mins used at the bottom of every page (refreshing/reloading pages will update counts), FIX: (#747)EOL normalization (dos2unix) on search.py - removed classes & exceptions as not being used, IMP: (#747) Skip processing issues with an invalid store date & issue date (thnx rupaschomaker), FIX: Removed strings when searching/logging torrents as was causing ascii errors especially with KAT, IMP: Added [META-TAGGING] to logging for meta-tagging module, IMP: Added ability in GUI to select CR or Cbl tags (or both) when writing metadata to cbz files, IMP: Improved support/usage with ComicTagger v1.1.15 which allows for personal CV API Key usage - if supplied to Mylar, will use when tagging with ComicTagger, IMP: Added Manual Search option to allow for individual searches of issues without changing initial status.
2014-06-18 19:58:19 +00:00
annuals_list = myDB . select ( ' SELECT * FROM annuals WHERE ComicID=? ' , [ ComicID ] )
ann_list + = annuals_list
issues_new + = annuals_list
logger . fdebug ( " Attempting to put the Status ' back how they were. " )
icount = 0
#the problem - the loop below will not match on NEW issues that have been refreshed that weren't present in the
#db before (ie. you left Mylar off for abit, and when you started it up it pulled down new issue information)
#need to test if issuenew['Status'] is None, but in a seperate loop below.
fndissue = [ ]
for issue in issues :
for issuenew in issues_new :
2014-08-27 02:20:16 +00:00
#logger.fdebug(str(issue['Issue_Number']) + ' - issuenew:' + str(issuenew['IssueID']) + ' : ' + str(issuenew['Status']))
#logger.fdebug(str(issue['Issue_Number']) + ' - issue:' + str(issue['IssueID']) + ' : ' + str(issue['Status']))
FIX: undefined on filter box on startup, IMP: Added Meta-Tagging options on a series / issue basis on comic details screen, IMP: Issue Information is now available per issue and is extracted currently from the cbz file to display (if no cbz is present, the option isn't available), IMP: Failed Download handling is implemented and available in GUI - required to replace existing autoProcessComics.py and ComicRN.py scripts, IMP: Added ability to specify post-processing script instead of ComicRN.py, IMP: Added abilty to edit the issue date for a given issue by simply clicking on it - this will help to avoid dates that have incorrect or 0000-00-00, IMP: Story Arc searching is working (not adding yet), IMP: Added Archived/Ignored options to Upcoming/Wanted tab, IMP: Fixed some alignment and display issues on the Upcoming section, IMP: Added better directory handling for Updating Comic Location when it needs to get changed for all existing series (locmove in config.ini), IMP: Added better handling for unicode characters in series titles when searching / filechecking, IMP: When adding a new series with no data, Mylar would error out (now will add and just retain 0 issues), FIX: When year was fuzzied, search would still attempt to do a date-check comparison and say everything failed, IMP: Better handling of nzb names when retaining for post-processing comparisons, IMP: Future Upcoming now will use actual shipping date of issue if available in order to see if issue is available, FIX: If annuals were enabled, refreshing a series would put some issues into an Archived status because the actual counts would be off, IMP: When file checking, Alternate Naming would be searched last which resulted in matching incorrectly to the series title or other alternate naming for the given series, now will check the Alternate Naming first for a match, then drop back down to the series name itself otherwise, IMP: Improved Annual detection when integrated with a given series, IMP: Improved the checking of the future Upcoming list for issues marked as Wanted but not available yet and then auto-adding, IMP: Improved upon story arc checking for missing issues / searching for wanted, IMP: Enabling Annuals support now within Configuration GUI, bunch of other things....
2014-07-28 19:28:09 +00:00
try :
2017-09-21 19:01:11 +00:00
if issuenew [ ' IssueID ' ] == issue [ ' IssueID ' ] :
newVAL = None
FIX: undefined on filter box on startup, IMP: Added Meta-Tagging options on a series / issue basis on comic details screen, IMP: Issue Information is now available per issue and is extracted currently from the cbz file to display (if no cbz is present, the option isn't available), IMP: Failed Download handling is implemented and available in GUI - required to replace existing autoProcessComics.py and ComicRN.py scripts, IMP: Added ability to specify post-processing script instead of ComicRN.py, IMP: Added abilty to edit the issue date for a given issue by simply clicking on it - this will help to avoid dates that have incorrect or 0000-00-00, IMP: Story Arc searching is working (not adding yet), IMP: Added Archived/Ignored options to Upcoming/Wanted tab, IMP: Fixed some alignment and display issues on the Upcoming section, IMP: Added better directory handling for Updating Comic Location when it needs to get changed for all existing series (locmove in config.ini), IMP: Added better handling for unicode characters in series titles when searching / filechecking, IMP: When adding a new series with no data, Mylar would error out (now will add and just retain 0 issues), FIX: When year was fuzzied, search would still attempt to do a date-check comparison and say everything failed, IMP: Better handling of nzb names when retaining for post-processing comparisons, IMP: Future Upcoming now will use actual shipping date of issue if available in order to see if issue is available, FIX: If annuals were enabled, refreshing a series would put some issues into an Archived status because the actual counts would be off, IMP: When file checking, Alternate Naming would be searched last which resulted in matching incorrectly to the series title or other alternate naming for the given series, now will check the Alternate Naming first for a match, then drop back down to the series name itself otherwise, IMP: Improved Annual detection when integrated with a given series, IMP: Improved the checking of the future Upcoming list for issues marked as Wanted but not available yet and then auto-adding, IMP: Improved upon story arc checking for missing issues / searching for wanted, IMP: Enabling Annuals support now within Configuration GUI, bunch of other things....
2014-07-28 19:28:09 +00:00
ctrlVAL = { " IssueID " : issue [ ' IssueID ' ] }
2017-09-21 19:01:11 +00:00
if any ( [ issuenew [ ' Status ' ] != issue [ ' Status ' ] , issue [ ' IssueDate_Edit ' ] is not None ] ) :
#if the status is None and the original status is either Downloaded / Archived, keep status & stats
if issuenew [ ' Status ' ] == None and ( issue [ ' Status ' ] == ' Downloaded ' or issue [ ' Status ' ] == ' Archived ' ) :
newVAL = { " Location " : issue [ ' Location ' ] ,
" ComicSize " : issue [ ' ComicSize ' ] ,
" Status " : issue [ ' Status ' ] }
#if the status is now Downloaded/Snatched, keep status & stats (downloaded only)
elif issuenew [ ' Status ' ] == ' Downloaded ' or issue [ ' Status ' ] == ' Snatched ' :
newVAL = { " Location " : issue [ ' Location ' ] ,
" ComicSize " : issue [ ' ComicSize ' ] }
if issuenew [ ' Status ' ] == ' Downloaded ' :
newVAL [ ' Status ' ] = issuenew [ ' Status ' ]
else :
newVAL [ ' Status ' ] = issue [ ' Status ' ]
elif issue [ ' Status ' ] == ' Archived ' :
newVAL = { " Status " : issue [ ' Status ' ] ,
" Location " : issue [ ' Location ' ] ,
" ComicSize " : issue [ ' ComicSize ' ] }
FIX: undefined on filter box on startup, IMP: Added Meta-Tagging options on a series / issue basis on comic details screen, IMP: Issue Information is now available per issue and is extracted currently from the cbz file to display (if no cbz is present, the option isn't available), IMP: Failed Download handling is implemented and available in GUI - required to replace existing autoProcessComics.py and ComicRN.py scripts, IMP: Added ability to specify post-processing script instead of ComicRN.py, IMP: Added abilty to edit the issue date for a given issue by simply clicking on it - this will help to avoid dates that have incorrect or 0000-00-00, IMP: Story Arc searching is working (not adding yet), IMP: Added Archived/Ignored options to Upcoming/Wanted tab, IMP: Fixed some alignment and display issues on the Upcoming section, IMP: Added better directory handling for Updating Comic Location when it needs to get changed for all existing series (locmove in config.ini), IMP: Added better handling for unicode characters in series titles when searching / filechecking, IMP: When adding a new series with no data, Mylar would error out (now will add and just retain 0 issues), FIX: When year was fuzzied, search would still attempt to do a date-check comparison and say everything failed, IMP: Better handling of nzb names when retaining for post-processing comparisons, IMP: Future Upcoming now will use actual shipping date of issue if available in order to see if issue is available, FIX: If annuals were enabled, refreshing a series would put some issues into an Archived status because the actual counts would be off, IMP: When file checking, Alternate Naming would be searched last which resulted in matching incorrectly to the series title or other alternate naming for the given series, now will check the Alternate Naming first for a match, then drop back down to the series name itself otherwise, IMP: Improved Annual detection when integrated with a given series, IMP: Improved the checking of the future Upcoming list for issues marked as Wanted but not available yet and then auto-adding, IMP: Improved upon story arc checking for missing issues / searching for wanted, IMP: Enabling Annuals support now within Configuration GUI, bunch of other things....
2014-07-28 19:28:09 +00:00
else :
2017-09-21 19:01:11 +00:00
#change the status to the previous status
newVAL = { " Status " : issue [ ' Status ' ] }
2017-09-24 17:35:45 +00:00
if all ( [ issuenew [ ' Status ' ] == None , issue [ ' Status ' ] == ' Skipped ' ] ) :
2017-10-25 18:33:56 +00:00
if issuenew [ ' ReleaseDate ' ] == ' 0000-00-00 ' :
2017-09-29 00:26:50 +00:00
dk = re . sub ( ' - ' , ' ' , issue [ ' IssueDate ' ] ) . strip ( )
IMP: Cleaned up interface for StoryArcs / Story Arc Details, IMP: Cleaned up interface for Reading list Management, IMP: Added better reading list management - new status (added, downloaded, read), IMP: Added sync option for use with another device for reading list transfer (ie. tablet) Android only, IMP: Autopopulate new weekly pull releases to reading list, IMP: 'Watch' option in weekly pull list now fully functional. Will watch CV for series' that do not have any series data yet as they are new starting series. Will auto-add once available, IMP: Auto-watch check is run after every refresh/recreate of the weeklypull list, IMP: Improved the Add a Series option where it will now look for issues that are 'new' or 'wanted' during add sequence, IMP: Main page interface now has coloured have/total bars to denote series completion, IMP: New scheduler / threading locks in place in an attempt to avoid database locks, FIX: Removed some erroneous locking that was going on when importing a directory was being run, IMP: Stat counter now present when post-processing multiple issues in sequence, FIX: for issue number error when post-processing and issue number was a non-alphanumeric, FIX: for metatagging: when original file was .cbz, would try to convert and fail, FIX: for issues that were negative and were preceeded by a # in the filename (filechecker), FIX: for publisher having non-alphanumeric character in name when attempting to determine publisher, FIX: if annuals enabled, would incorrectly show as being 'already in library' when viewing search results if results constained annuals, FIX:(#944) for incorrect nzbname being used when post-processing was being performed from an nzb client (experimental mainly), IMP: Turned off logging for ComicVine API counter, FIX: Added retry attempts when connecting to ComicVine in order to avoid errors when adding a series, IMP:(#963) Added ability to add snatched to filter when viewing Wanted issues on Wanted tab, FIX: When importing and then selecting a series to import via the select screen, will now flip back to the importresults and add the selected series in the background, IMP:(#952) Main page is now sorted in ascending order by Continuing/Ended status (and subbed by whether is Active/Paused).Custom sorting is still available, FIX: Dupecheck will now automatically assume existing 0-byte files are to be overwritten when performing post-processing, FIX: If publication date for series contained a '?' (usually with brand new series) will force to 'Present' to allow for pull-list comparisons to take place, FIX: Mylar will now disallow search results which have 'covers only' or 'variant' in the filename, IMP: Better nzbname generation/retrieval (will check inside nzb for possible names) to be used when post-processing, IMP: DB Update will now perform update to all active comics in descending order by Latest Date (instead of random order), FIX: Enforce the 5hr limit rule when running DB update (will only update series that haven't been updated in >5 hours), FIX: Annuals will now have/retain the proper status upon doing DB Update, FIX: Have totals will now be updated when doing a recheck files (sometimes wouldn't get updated depending on various states of status'), FIX:(#966) Added urllib2.URLError exeception trap when attempting to check Git for updates, IMP: Removed the individual sqlite calls for weeklypull, and brought them into line with using the db module (which will minimize concurrent access, which seemed to be causing db locks), IMP: Cleaned up some code and shuffled some functions so they are in more appropriate locations
2015-03-27 17:27:59 +00:00
else :
2017-09-29 00:26:50 +00:00
dk = re . sub ( ' - ' , ' ' , issuenew [ ' ReleaseDate ' ] ) . strip ( ) # converts date to 20140718 format
2017-10-25 18:33:56 +00:00
if dk == ' 0000-00-00 ' :
2017-09-29 00:26:50 +00:00
logger . warn ( ' Issue Data is invalid for Issue Number %s . Marking this issue as Skipped ' % issue [ ' Issue_Number ' ] )
IMP: Cleaned up interface for StoryArcs / Story Arc Details, IMP: Cleaned up interface for Reading list Management, IMP: Added better reading list management - new status (added, downloaded, read), IMP: Added sync option for use with another device for reading list transfer (ie. tablet) Android only, IMP: Autopopulate new weekly pull releases to reading list, IMP: 'Watch' option in weekly pull list now fully functional. Will watch CV for series' that do not have any series data yet as they are new starting series. Will auto-add once available, IMP: Auto-watch check is run after every refresh/recreate of the weeklypull list, IMP: Improved the Add a Series option where it will now look for issues that are 'new' or 'wanted' during add sequence, IMP: Main page interface now has coloured have/total bars to denote series completion, IMP: New scheduler / threading locks in place in an attempt to avoid database locks, FIX: Removed some erroneous locking that was going on when importing a directory was being run, IMP: Stat counter now present when post-processing multiple issues in sequence, FIX: for issue number error when post-processing and issue number was a non-alphanumeric, FIX: for metatagging: when original file was .cbz, would try to convert and fail, FIX: for issues that were negative and were preceeded by a # in the filename (filechecker), FIX: for publisher having non-alphanumeric character in name when attempting to determine publisher, FIX: if annuals enabled, would incorrectly show as being 'already in library' when viewing search results if results constained annuals, FIX:(#944) for incorrect nzbname being used when post-processing was being performed from an nzb client (experimental mainly), IMP: Turned off logging for ComicVine API counter, FIX: Added retry attempts when connecting to ComicVine in order to avoid errors when adding a series, IMP:(#963) Added ability to add snatched to filter when viewing Wanted issues on Wanted tab, FIX: When importing and then selecting a series to import via the select screen, will now flip back to the importresults and add the selected series in the background, IMP:(#952) Main page is now sorted in ascending order by Continuing/Ended status (and subbed by whether is Active/Paused).Custom sorting is still available, FIX: Dupecheck will now automatically assume existing 0-byte files are to be overwritten when performing post-processing, FIX: If publication date for series contained a '?' (usually with brand new series) will force to 'Present' to allow for pull-list comparisons to take place, FIX: Mylar will now disallow search results which have 'covers only' or 'variant' in the filename, IMP: Better nzbname generation/retrieval (will check inside nzb for possible names) to be used when post-processing, IMP: DB Update will now perform update to all active comics in descending order by Latest Date (instead of random order), FIX: Enforce the 5hr limit rule when running DB update (will only update series that haven't been updated in >5 hours), FIX: Annuals will now have/retain the proper status upon doing DB Update, FIX: Have totals will now be updated when doing a recheck files (sometimes wouldn't get updated depending on various states of status'), FIX:(#966) Added urllib2.URLError exeception trap when attempting to check Git for updates, IMP: Removed the individual sqlite calls for weeklypull, and brought them into line with using the db module (which will minimize concurrent access, which seemed to be causing db locks), IMP: Cleaned up some code and shuffled some functions so they are in more appropriate locations
2015-03-27 17:27:59 +00:00
newVAL = { " Status " : " Skipped " }
2017-09-29 00:26:50 +00:00
else :
datechk = datetime . datetime . strptime ( dk , " % Y % m %d " )
nowdate = datetime . datetime . now ( )
now_week = datetime . datetime . strftime ( nowdate , " % Y % U " )
issue_week = datetime . datetime . strftime ( datechk , " % Y % U " )
2017-10-25 18:33:56 +00:00
if mylar . CONFIG . AUTOWANT_ALL :
2017-09-29 00:26:50 +00:00
newVAL = { " Status " : " Wanted " }
elif issue_week > = now_week :
2017-10-25 18:33:56 +00:00
logger . fdebug ( ' Issue_week: %s -- now_week: %s ' % ( issue_week , now_week ) )
2017-09-29 00:26:50 +00:00
logger . fdebug ( ' Issue date [ %s ] is in/beyond current week - marking as Wanted. ' % dk )
newVAL = { " Status " : " Wanted " }
else :
newVAL = { " Status " : " Skipped " }
FIX: undefined on filter box on startup, IMP: Added Meta-Tagging options on a series / issue basis on comic details screen, IMP: Issue Information is now available per issue and is extracted currently from the cbz file to display (if no cbz is present, the option isn't available), IMP: Failed Download handling is implemented and available in GUI - required to replace existing autoProcessComics.py and ComicRN.py scripts, IMP: Added ability to specify post-processing script instead of ComicRN.py, IMP: Added abilty to edit the issue date for a given issue by simply clicking on it - this will help to avoid dates that have incorrect or 0000-00-00, IMP: Story Arc searching is working (not adding yet), IMP: Added Archived/Ignored options to Upcoming/Wanted tab, IMP: Fixed some alignment and display issues on the Upcoming section, IMP: Added better directory handling for Updating Comic Location when it needs to get changed for all existing series (locmove in config.ini), IMP: Added better handling for unicode characters in series titles when searching / filechecking, IMP: When adding a new series with no data, Mylar would error out (now will add and just retain 0 issues), FIX: When year was fuzzied, search would still attempt to do a date-check comparison and say everything failed, IMP: Better handling of nzb names when retaining for post-processing comparisons, IMP: Future Upcoming now will use actual shipping date of issue if available in order to see if issue is available, FIX: If annuals were enabled, refreshing a series would put some issues into an Archived status because the actual counts would be off, IMP: When file checking, Alternate Naming would be searched last which resulted in matching incorrectly to the series title or other alternate naming for the given series, now will check the Alternate Naming first for a match, then drop back down to the series name itself otherwise, IMP: Improved Annual detection when integrated with a given series, IMP: Improved the checking of the future Upcoming list for issues marked as Wanted but not available yet and then auto-adding, IMP: Improved upon story arc checking for missing issues / searching for wanted, IMP: Enabling Annuals support now within Configuration GUI, bunch of other things....
2014-07-28 19:28:09 +00:00
2017-09-21 19:01:11 +00:00
if newVAL is not None :
if issue [ ' IssueDate_Edit ' ] :
logger . fdebug ( ' [# ' + str ( issue [ ' Issue_Number ' ] ) + ' ] detected manually edited Issue Date. ' )
logger . fdebug ( ' new value : ' + str ( issue [ ' IssueDate ' ] ) + ' ... cv value : ' + str ( issuenew [ ' IssueDate ' ] ) )
newVAL [ ' IssueDate ' ] = issue [ ' IssueDate ' ]
newVAL [ ' IssueDate_Edit ' ] = issue [ ' IssueDate_Edit ' ]
if any ( d [ ' IssueID ' ] == str ( issue [ ' IssueID ' ] ) for d in ann_list ) :
logger . fdebug ( " annual detected for " + str ( issue [ ' IssueID ' ] ) + " #: " + str ( issue [ ' Issue_Number ' ] ) )
myDB . upsert ( " Annuals " , newVAL , ctrlVAL )
else :
#logger.fdebug('#' + str(issue['Issue_Number']) + ' writing issuedata: ' + str(newVAL))
myDB . upsert ( " Issues " , newVAL , ctrlVAL )
fndissue . append ( { " IssueID " : issue [ ' IssueID ' ] } )
icount + = 1
break
2017-09-29 00:26:50 +00:00
except ( RuntimeError , TypeError , ValueError , OSError ) as e :
logger . warn ( ' Something is out of whack somewhere with the series: %s ' % e )
FIX: undefined on filter box on startup, IMP: Added Meta-Tagging options on a series / issue basis on comic details screen, IMP: Issue Information is now available per issue and is extracted currently from the cbz file to display (if no cbz is present, the option isn't available), IMP: Failed Download handling is implemented and available in GUI - required to replace existing autoProcessComics.py and ComicRN.py scripts, IMP: Added ability to specify post-processing script instead of ComicRN.py, IMP: Added abilty to edit the issue date for a given issue by simply clicking on it - this will help to avoid dates that have incorrect or 0000-00-00, IMP: Story Arc searching is working (not adding yet), IMP: Added Archived/Ignored options to Upcoming/Wanted tab, IMP: Fixed some alignment and display issues on the Upcoming section, IMP: Added better directory handling for Updating Comic Location when it needs to get changed for all existing series (locmove in config.ini), IMP: Added better handling for unicode characters in series titles when searching / filechecking, IMP: When adding a new series with no data, Mylar would error out (now will add and just retain 0 issues), FIX: When year was fuzzied, search would still attempt to do a date-check comparison and say everything failed, IMP: Better handling of nzb names when retaining for post-processing comparisons, IMP: Future Upcoming now will use actual shipping date of issue if available in order to see if issue is available, FIX: If annuals were enabled, refreshing a series would put some issues into an Archived status because the actual counts would be off, IMP: When file checking, Alternate Naming would be searched last which resulted in matching incorrectly to the series title or other alternate naming for the given series, now will check the Alternate Naming first for a match, then drop back down to the series name itself otherwise, IMP: Improved Annual detection when integrated with a given series, IMP: Improved the checking of the future Upcoming list for issues marked as Wanted but not available yet and then auto-adding, IMP: Improved upon story arc checking for missing issues / searching for wanted, IMP: Enabling Annuals support now within Configuration GUI, bunch of other things....
2014-07-28 19:28:09 +00:00
#if it's an annual (ie. deadpool-2011 ) on a refresh will throw index errors for some reason.
2017-09-29 00:26:50 +00:00
except :
logger . warn ( ' Unexpected Error: %s ' % sys . exc_info ( ) [ 0 ] )
raise
FIX: undefined on filter box on startup, IMP: Added Meta-Tagging options on a series / issue basis on comic details screen, IMP: Issue Information is now available per issue and is extracted currently from the cbz file to display (if no cbz is present, the option isn't available), IMP: Failed Download handling is implemented and available in GUI - required to replace existing autoProcessComics.py and ComicRN.py scripts, IMP: Added ability to specify post-processing script instead of ComicRN.py, IMP: Added abilty to edit the issue date for a given issue by simply clicking on it - this will help to avoid dates that have incorrect or 0000-00-00, IMP: Story Arc searching is working (not adding yet), IMP: Added Archived/Ignored options to Upcoming/Wanted tab, IMP: Fixed some alignment and display issues on the Upcoming section, IMP: Added better directory handling for Updating Comic Location when it needs to get changed for all existing series (locmove in config.ini), IMP: Added better handling for unicode characters in series titles when searching / filechecking, IMP: When adding a new series with no data, Mylar would error out (now will add and just retain 0 issues), FIX: When year was fuzzied, search would still attempt to do a date-check comparison and say everything failed, IMP: Better handling of nzb names when retaining for post-processing comparisons, IMP: Future Upcoming now will use actual shipping date of issue if available in order to see if issue is available, FIX: If annuals were enabled, refreshing a series would put some issues into an Archived status because the actual counts would be off, IMP: When file checking, Alternate Naming would be searched last which resulted in matching incorrectly to the series title or other alternate naming for the given series, now will check the Alternate Naming first for a match, then drop back down to the series name itself otherwise, IMP: Improved Annual detection when integrated with a given series, IMP: Improved the checking of the future Upcoming list for issues marked as Wanted but not available yet and then auto-adding, IMP: Improved upon story arc checking for missing issues / searching for wanted, IMP: Enabling Annuals support now within Configuration GUI, bunch of other things....
2014-07-28 19:28:09 +00:00
FIX: (#746) updated autoProcessComics.py / ComicRN.py's which will now send proper completion messeages to clients, FIX: (#752) refresh series will now test if a series is 'out of whack' with it's numerical issue count (ie. 5/4) or it has no issue data due to a bad refresh / api maxing out and will adjust it's processing to accomodate either, IMP: (#750) Added ComicVine API Checker which will check API counts at regular intervals to inform/warn users of usage, as well as adding a screen-time display of API hits / mins used at the bottom of every page (refreshing/reloading pages will update counts), FIX: (#747)EOL normalization (dos2unix) on search.py - removed classes & exceptions as not being used, IMP: (#747) Skip processing issues with an invalid store date & issue date (thnx rupaschomaker), FIX: Removed strings when searching/logging torrents as was causing ascii errors especially with KAT, IMP: Added [META-TAGGING] to logging for meta-tagging module, IMP: Added ability in GUI to select CR or Cbl tags (or both) when writing metadata to cbz files, IMP: Improved support/usage with ComicTagger v1.1.15 which allows for personal CV API Key usage - if supplied to Mylar, will use when tagging with ComicTagger, IMP: Added Manual Search option to allow for individual searches of issues without changing initial status.
2014-06-18 19:58:19 +00:00
logger . info ( " In the process of converting the data to CV, I changed the status of " + str ( icount ) + " issues. " )
IMP: Cleaned up interface for StoryArcs / Story Arc Details, IMP: Cleaned up interface for Reading list Management, IMP: Added better reading list management - new status (added, downloaded, read), IMP: Added sync option for use with another device for reading list transfer (ie. tablet) Android only, IMP: Autopopulate new weekly pull releases to reading list, IMP: 'Watch' option in weekly pull list now fully functional. Will watch CV for series' that do not have any series data yet as they are new starting series. Will auto-add once available, IMP: Auto-watch check is run after every refresh/recreate of the weeklypull list, IMP: Improved the Add a Series option where it will now look for issues that are 'new' or 'wanted' during add sequence, IMP: Main page interface now has coloured have/total bars to denote series completion, IMP: New scheduler / threading locks in place in an attempt to avoid database locks, FIX: Removed some erroneous locking that was going on when importing a directory was being run, IMP: Stat counter now present when post-processing multiple issues in sequence, FIX: for issue number error when post-processing and issue number was a non-alphanumeric, FIX: for metatagging: when original file was .cbz, would try to convert and fail, FIX: for issues that were negative and were preceeded by a # in the filename (filechecker), FIX: for publisher having non-alphanumeric character in name when attempting to determine publisher, FIX: if annuals enabled, would incorrectly show as being 'already in library' when viewing search results if results constained annuals, FIX:(#944) for incorrect nzbname being used when post-processing was being performed from an nzb client (experimental mainly), IMP: Turned off logging for ComicVine API counter, FIX: Added retry attempts when connecting to ComicVine in order to avoid errors when adding a series, IMP:(#963) Added ability to add snatched to filter when viewing Wanted issues on Wanted tab, FIX: When importing and then selecting a series to import via the select screen, will now flip back to the importresults and add the selected series in the background, IMP:(#952) Main page is now sorted in ascending order by Continuing/Ended status (and subbed by whether is Active/Paused).Custom sorting is still available, FIX: Dupecheck will now automatically assume existing 0-byte files are to be overwritten when performing post-processing, FIX: If publication date for series contained a '?' (usually with brand new series) will force to 'Present' to allow for pull-list comparisons to take place, FIX: Mylar will now disallow search results which have 'covers only' or 'variant' in the filename, IMP: Better nzbname generation/retrieval (will check inside nzb for possible names) to be used when post-processing, IMP: DB Update will now perform update to all active comics in descending order by Latest Date (instead of random order), FIX: Enforce the 5hr limit rule when running DB update (will only update series that haven't been updated in >5 hours), FIX: Annuals will now have/retain the proper status upon doing DB Update, FIX: Have totals will now be updated when doing a recheck files (sometimes wouldn't get updated depending on various states of status'), FIX:(#966) Added urllib2.URLError exeception trap when attempting to check Git for updates, IMP: Removed the individual sqlite calls for weeklypull, and brought them into line with using the db module (which will minimize concurrent access, which seemed to be causing db locks), IMP: Cleaned up some code and shuffled some functions so they are in more appropriate locations
2015-03-27 17:27:59 +00:00
issuesnew = myDB . select ( ' SELECT * FROM issues WHERE ComicID=? AND Status is NULL ' , [ ComicID ] )
FIX: (#746) updated autoProcessComics.py / ComicRN.py's which will now send proper completion messeages to clients, FIX: (#752) refresh series will now test if a series is 'out of whack' with it's numerical issue count (ie. 5/4) or it has no issue data due to a bad refresh / api maxing out and will adjust it's processing to accomodate either, IMP: (#750) Added ComicVine API Checker which will check API counts at regular intervals to inform/warn users of usage, as well as adding a screen-time display of API hits / mins used at the bottom of every page (refreshing/reloading pages will update counts), FIX: (#747)EOL normalization (dos2unix) on search.py - removed classes & exceptions as not being used, IMP: (#747) Skip processing issues with an invalid store date & issue date (thnx rupaschomaker), FIX: Removed strings when searching/logging torrents as was causing ascii errors especially with KAT, IMP: Added [META-TAGGING] to logging for meta-tagging module, IMP: Added ability in GUI to select CR or Cbl tags (or both) when writing metadata to cbz files, IMP: Improved support/usage with ComicTagger v1.1.15 which allows for personal CV API Key usage - if supplied to Mylar, will use when tagging with ComicTagger, IMP: Added Manual Search option to allow for individual searches of issues without changing initial status.
2014-06-18 19:58:19 +00:00
2017-10-25 18:33:56 +00:00
if mylar . CONFIG . AUTOWANT_UPCOMING :
FIX: (#746) updated autoProcessComics.py / ComicRN.py's which will now send proper completion messeages to clients, FIX: (#752) refresh series will now test if a series is 'out of whack' with it's numerical issue count (ie. 5/4) or it has no issue data due to a bad refresh / api maxing out and will adjust it's processing to accomodate either, IMP: (#750) Added ComicVine API Checker which will check API counts at regular intervals to inform/warn users of usage, as well as adding a screen-time display of API hits / mins used at the bottom of every page (refreshing/reloading pages will update counts), FIX: (#747)EOL normalization (dos2unix) on search.py - removed classes & exceptions as not being used, IMP: (#747) Skip processing issues with an invalid store date & issue date (thnx rupaschomaker), FIX: Removed strings when searching/logging torrents as was causing ascii errors especially with KAT, IMP: Added [META-TAGGING] to logging for meta-tagging module, IMP: Added ability in GUI to select CR or Cbl tags (or both) when writing metadata to cbz files, IMP: Improved support/usage with ComicTagger v1.1.15 which allows for personal CV API Key usage - if supplied to Mylar, will use when tagging with ComicTagger, IMP: Added Manual Search option to allow for individual searches of issues without changing initial status.
2014-06-18 19:58:19 +00:00
newstatus = " Wanted "
else :
newstatus = " Skipped "
IMP: Cleaned up interface for StoryArcs / Story Arc Details, IMP: Cleaned up interface for Reading list Management, IMP: Added better reading list management - new status (added, downloaded, read), IMP: Added sync option for use with another device for reading list transfer (ie. tablet) Android only, IMP: Autopopulate new weekly pull releases to reading list, IMP: 'Watch' option in weekly pull list now fully functional. Will watch CV for series' that do not have any series data yet as they are new starting series. Will auto-add once available, IMP: Auto-watch check is run after every refresh/recreate of the weeklypull list, IMP: Improved the Add a Series option where it will now look for issues that are 'new' or 'wanted' during add sequence, IMP: Main page interface now has coloured have/total bars to denote series completion, IMP: New scheduler / threading locks in place in an attempt to avoid database locks, FIX: Removed some erroneous locking that was going on when importing a directory was being run, IMP: Stat counter now present when post-processing multiple issues in sequence, FIX: for issue number error when post-processing and issue number was a non-alphanumeric, FIX: for metatagging: when original file was .cbz, would try to convert and fail, FIX: for issues that were negative and were preceeded by a # in the filename (filechecker), FIX: for publisher having non-alphanumeric character in name when attempting to determine publisher, FIX: if annuals enabled, would incorrectly show as being 'already in library' when viewing search results if results constained annuals, FIX:(#944) for incorrect nzbname being used when post-processing was being performed from an nzb client (experimental mainly), IMP: Turned off logging for ComicVine API counter, FIX: Added retry attempts when connecting to ComicVine in order to avoid errors when adding a series, IMP:(#963) Added ability to add snatched to filter when viewing Wanted issues on Wanted tab, FIX: When importing and then selecting a series to import via the select screen, will now flip back to the importresults and add the selected series in the background, IMP:(#952) Main page is now sorted in ascending order by Continuing/Ended status (and subbed by whether is Active/Paused).Custom sorting is still available, FIX: Dupecheck will now automatically assume existing 0-byte files are to be overwritten when performing post-processing, FIX: If publication date for series contained a '?' (usually with brand new series) will force to 'Present' to allow for pull-list comparisons to take place, FIX: Mylar will now disallow search results which have 'covers only' or 'variant' in the filename, IMP: Better nzbname generation/retrieval (will check inside nzb for possible names) to be used when post-processing, IMP: DB Update will now perform update to all active comics in descending order by Latest Date (instead of random order), FIX: Enforce the 5hr limit rule when running DB update (will only update series that haven't been updated in >5 hours), FIX: Annuals will now have/retain the proper status upon doing DB Update, FIX: Have totals will now be updated when doing a recheck files (sometimes wouldn't get updated depending on various states of status'), FIX:(#966) Added urllib2.URLError exeception trap when attempting to check Git for updates, IMP: Removed the individual sqlite calls for weeklypull, and brought them into line with using the db module (which will minimize concurrent access, which seemed to be causing db locks), IMP: Cleaned up some code and shuffled some functions so they are in more appropriate locations
2015-03-27 17:27:59 +00:00
IMP: 32P legacy mode (rss feed only) and Auth mode (user/pass) available now. With Auth mode, will monitor all of personal notification feeds, IMP: Changed access methods to 32P for retrieving links, FIX:(#1007) Fixed error for one-off downloading via weekly pull list, FIX: (#1004) Fix for post-processing errors (missing files), FIX: (#1001) Extra scripts & pre-scripts will now accept bash scripts, FIX: Annuals were being incorrectly tallied in Have total as well as not having status correctly updated, IMP: Fixed some discrepencies with the provider_order generation line, IMP: Made some additional adjustments to the file-checking to account for some additional variations in naming, FIX: Fixed traceback errors when folder monitor was running and wouldn't be able to complete the post-processing properly, FIX: Cleanup code and some other small fixes....
2015-05-09 02:39:28 +00:00
newiss = [ ]
IMP: Cleaned up interface for StoryArcs / Story Arc Details, IMP: Cleaned up interface for Reading list Management, IMP: Added better reading list management - new status (added, downloaded, read), IMP: Added sync option for use with another device for reading list transfer (ie. tablet) Android only, IMP: Autopopulate new weekly pull releases to reading list, IMP: 'Watch' option in weekly pull list now fully functional. Will watch CV for series' that do not have any series data yet as they are new starting series. Will auto-add once available, IMP: Auto-watch check is run after every refresh/recreate of the weeklypull list, IMP: Improved the Add a Series option where it will now look for issues that are 'new' or 'wanted' during add sequence, IMP: Main page interface now has coloured have/total bars to denote series completion, IMP: New scheduler / threading locks in place in an attempt to avoid database locks, FIX: Removed some erroneous locking that was going on when importing a directory was being run, IMP: Stat counter now present when post-processing multiple issues in sequence, FIX: for issue number error when post-processing and issue number was a non-alphanumeric, FIX: for metatagging: when original file was .cbz, would try to convert and fail, FIX: for issues that were negative and were preceeded by a # in the filename (filechecker), FIX: for publisher having non-alphanumeric character in name when attempting to determine publisher, FIX: if annuals enabled, would incorrectly show as being 'already in library' when viewing search results if results constained annuals, FIX:(#944) for incorrect nzbname being used when post-processing was being performed from an nzb client (experimental mainly), IMP: Turned off logging for ComicVine API counter, FIX: Added retry attempts when connecting to ComicVine in order to avoid errors when adding a series, IMP:(#963) Added ability to add snatched to filter when viewing Wanted issues on Wanted tab, FIX: When importing and then selecting a series to import via the select screen, will now flip back to the importresults and add the selected series in the background, IMP:(#952) Main page is now sorted in ascending order by Continuing/Ended status (and subbed by whether is Active/Paused).Custom sorting is still available, FIX: Dupecheck will now automatically assume existing 0-byte files are to be overwritten when performing post-processing, FIX: If publication date for series contained a '?' (usually with brand new series) will force to 'Present' to allow for pull-list comparisons to take place, FIX: Mylar will now disallow search results which have 'covers only' or 'variant' in the filename, IMP: Better nzbname generation/retrieval (will check inside nzb for possible names) to be used when post-processing, IMP: DB Update will now perform update to all active comics in descending order by Latest Date (instead of random order), FIX: Enforce the 5hr limit rule when running DB update (will only update series that haven't been updated in >5 hours), FIX: Annuals will now have/retain the proper status upon doing DB Update, FIX: Have totals will now be updated when doing a recheck files (sometimes wouldn't get updated depending on various states of status'), FIX:(#966) Added urllib2.URLError exeception trap when attempting to check Git for updates, IMP: Removed the individual sqlite calls for weeklypull, and brought them into line with using the db module (which will minimize concurrent access, which seemed to be causing db locks), IMP: Cleaned up some code and shuffled some functions so they are in more appropriate locations
2015-03-27 17:27:59 +00:00
for iss in issuesnew :
newiss . append ( { " IssueID " : iss [ ' IssueID ' ] ,
" Status " : newstatus ,
" Annual " : False } )
IMP: 32P legacy mode (rss feed only) and Auth mode (user/pass) available now. With Auth mode, will monitor all of personal notification feeds, IMP: Changed access methods to 32P for retrieving links, FIX:(#1007) Fixed error for one-off downloading via weekly pull list, FIX: (#1004) Fix for post-processing errors (missing files), FIX: (#1001) Extra scripts & pre-scripts will now accept bash scripts, FIX: Annuals were being incorrectly tallied in Have total as well as not having status correctly updated, IMP: Fixed some discrepencies with the provider_order generation line, IMP: Made some additional adjustments to the file-checking to account for some additional variations in naming, FIX: Fixed traceback errors when folder monitor was running and wouldn't be able to complete the post-processing properly, FIX: Cleanup code and some other small fixes....
2015-05-09 02:39:28 +00:00
2017-10-25 18:33:56 +00:00
if mylar . CONFIG . ANNUALS_ON :
IMP: 32P legacy mode (rss feed only) and Auth mode (user/pass) available now. With Auth mode, will monitor all of personal notification feeds, IMP: Changed access methods to 32P for retrieving links, FIX:(#1007) Fixed error for one-off downloading via weekly pull list, FIX: (#1004) Fix for post-processing errors (missing files), FIX: (#1001) Extra scripts & pre-scripts will now accept bash scripts, FIX: Annuals were being incorrectly tallied in Have total as well as not having status correctly updated, IMP: Fixed some discrepencies with the provider_order generation line, IMP: Made some additional adjustments to the file-checking to account for some additional variations in naming, FIX: Fixed traceback errors when folder monitor was running and wouldn't be able to complete the post-processing properly, FIX: Cleanup code and some other small fixes....
2015-05-09 02:39:28 +00:00
annualsnew = myDB . select ( ' SELECT * FROM annuals WHERE ComicID=? AND Status is NULL ' , [ ComicID ] )
for ann in annualsnew :
newiss . append ( { " IssueID " : iss [ ' IssueID ' ] ,
" Status " : newstatus ,
" Annual " : True } )
IMP: Cleaned up interface for StoryArcs / Story Arc Details, IMP: Cleaned up interface for Reading list Management, IMP: Added better reading list management - new status (added, downloaded, read), IMP: Added sync option for use with another device for reading list transfer (ie. tablet) Android only, IMP: Autopopulate new weekly pull releases to reading list, IMP: 'Watch' option in weekly pull list now fully functional. Will watch CV for series' that do not have any series data yet as they are new starting series. Will auto-add once available, IMP: Auto-watch check is run after every refresh/recreate of the weeklypull list, IMP: Improved the Add a Series option where it will now look for issues that are 'new' or 'wanted' during add sequence, IMP: Main page interface now has coloured have/total bars to denote series completion, IMP: New scheduler / threading locks in place in an attempt to avoid database locks, FIX: Removed some erroneous locking that was going on when importing a directory was being run, IMP: Stat counter now present when post-processing multiple issues in sequence, FIX: for issue number error when post-processing and issue number was a non-alphanumeric, FIX: for metatagging: when original file was .cbz, would try to convert and fail, FIX: for issues that were negative and were preceeded by a # in the filename (filechecker), FIX: for publisher having non-alphanumeric character in name when attempting to determine publisher, FIX: if annuals enabled, would incorrectly show as being 'already in library' when viewing search results if results constained annuals, FIX:(#944) for incorrect nzbname being used when post-processing was being performed from an nzb client (experimental mainly), IMP: Turned off logging for ComicVine API counter, FIX: Added retry attempts when connecting to ComicVine in order to avoid errors when adding a series, IMP:(#963) Added ability to add snatched to filter when viewing Wanted issues on Wanted tab, FIX: When importing and then selecting a series to import via the select screen, will now flip back to the importresults and add the selected series in the background, IMP:(#952) Main page is now sorted in ascending order by Continuing/Ended status (and subbed by whether is Active/Paused).Custom sorting is still available, FIX: Dupecheck will now automatically assume existing 0-byte files are to be overwritten when performing post-processing, FIX: If publication date for series contained a '?' (usually with brand new series) will force to 'Present' to allow for pull-list comparisons to take place, FIX: Mylar will now disallow search results which have 'covers only' or 'variant' in the filename, IMP: Better nzbname generation/retrieval (will check inside nzb for possible names) to be used when post-processing, IMP: DB Update will now perform update to all active comics in descending order by Latest Date (instead of random order), FIX: Enforce the 5hr limit rule when running DB update (will only update series that haven't been updated in >5 hours), FIX: Annuals will now have/retain the proper status upon doing DB Update, FIX: Have totals will now be updated when doing a recheck files (sometimes wouldn't get updated depending on various states of status'), FIX:(#966) Added urllib2.URLError exeception trap when attempting to check Git for updates, IMP: Removed the individual sqlite calls for weeklypull, and brought them into line with using the db module (which will minimize concurrent access, which seemed to be causing db locks), IMP: Cleaned up some code and shuffled some functions so they are in more appropriate locations
2015-03-27 17:27:59 +00:00
FIX: (#746) updated autoProcessComics.py / ComicRN.py's which will now send proper completion messeages to clients, FIX: (#752) refresh series will now test if a series is 'out of whack' with it's numerical issue count (ie. 5/4) or it has no issue data due to a bad refresh / api maxing out and will adjust it's processing to accomodate either, IMP: (#750) Added ComicVine API Checker which will check API counts at regular intervals to inform/warn users of usage, as well as adding a screen-time display of API hits / mins used at the bottom of every page (refreshing/reloading pages will update counts), FIX: (#747)EOL normalization (dos2unix) on search.py - removed classes & exceptions as not being used, IMP: (#747) Skip processing issues with an invalid store date & issue date (thnx rupaschomaker), FIX: Removed strings when searching/logging torrents as was causing ascii errors especially with KAT, IMP: Added [META-TAGGING] to logging for meta-tagging module, IMP: Added ability in GUI to select CR or Cbl tags (or both) when writing metadata to cbz files, IMP: Improved support/usage with ComicTagger v1.1.15 which allows for personal CV API Key usage - if supplied to Mylar, will use when tagging with ComicTagger, IMP: Added Manual Search option to allow for individual searches of issues without changing initial status.
2014-06-18 19:58:19 +00:00
if len ( newiss ) > 0 :
for newi in newiss :
ctrlVAL = { " IssueID " : newi [ ' IssueID ' ] }
newVAL = { " Status " : newi [ ' Status ' ] }
IMP: Added Choose specific Download option to manually select from result list on Upcoming/Details/Weekly tabs, IMP: Added Wanted storyarcs to the overall Wanted section so now will search as per global options (storyarc issues can be displayed optionally on Wanted tab), IMP: Added custom url option for image banners/posters for storyarcs, IMP: updated Cherrypy, FIX: Fixed ComicRN not working when forms authentication used - in conjunction updated autoProcessComics to 2.0 which now uses apikey instead of user/pass, IMP: Alternate Series Covers option for alternating existing series image to the most current issue image, IMP: Added overall series total to series page for reference, IMP: Search workflow completely changed to accomodate more than one indexer - now will simultaneously sumbit initial request to each provider, wait 30s submit additional as required at 30s intervals, FIX: Removed TPSE as an option and relabelled to just Public Torrents, IMP: Added direct backlog search to WWT option (pack support will follow), FIX: Removed line about configparser being required for ComicTagger usage, IMP: Test code in place for newzab testing, FIX: Fixed layout problem with torrents that are in auto-snatch status on weekly tab, IMP: backend code improvements to allow for better alias usage and annual linking directly from WS, IMP: Updated systemd init-scripts with read.me, IMP: When post-processing, will now check for available destination free space before actually moving files, IMP: Will copy during metatagging to cache folder instead of move being an option so cleanup is cleaner if something fails, FIX: Changed readinglist table to storyarcs for clarity, IMP: When post-processing issues, will now only update the one issue status and adjust totals accordingly (instead of doing a complete rescan of the series), FIX: Clear out empty ID's from the Failed DB on startup, IMP: Initial code-run at REST API interface, FIX: Fixed some matching problems with 32p due to case, IMP: removed apikeys from log entries that were accidentally logging, IMP: When searching 32p, if items get packed up - will now delete the cached reference so new items of the same can be located, IMP: ForceSearch option switched to scheduler so simultaneous runs should not occur, FIX: Fixed manual metatagging error that would occur if multiple destination directories existed
2018-02-16 19:57:01 +00:00
#logger.fdebug('writing issuedata: ' + str(newVAL))
IMP: Cleaned up interface for StoryArcs / Story Arc Details, IMP: Cleaned up interface for Reading list Management, IMP: Added better reading list management - new status (added, downloaded, read), IMP: Added sync option for use with another device for reading list transfer (ie. tablet) Android only, IMP: Autopopulate new weekly pull releases to reading list, IMP: 'Watch' option in weekly pull list now fully functional. Will watch CV for series' that do not have any series data yet as they are new starting series. Will auto-add once available, IMP: Auto-watch check is run after every refresh/recreate of the weeklypull list, IMP: Improved the Add a Series option where it will now look for issues that are 'new' or 'wanted' during add sequence, IMP: Main page interface now has coloured have/total bars to denote series completion, IMP: New scheduler / threading locks in place in an attempt to avoid database locks, FIX: Removed some erroneous locking that was going on when importing a directory was being run, IMP: Stat counter now present when post-processing multiple issues in sequence, FIX: for issue number error when post-processing and issue number was a non-alphanumeric, FIX: for metatagging: when original file was .cbz, would try to convert and fail, FIX: for issues that were negative and were preceeded by a # in the filename (filechecker), FIX: for publisher having non-alphanumeric character in name when attempting to determine publisher, FIX: if annuals enabled, would incorrectly show as being 'already in library' when viewing search results if results constained annuals, FIX:(#944) for incorrect nzbname being used when post-processing was being performed from an nzb client (experimental mainly), IMP: Turned off logging for ComicVine API counter, FIX: Added retry attempts when connecting to ComicVine in order to avoid errors when adding a series, IMP:(#963) Added ability to add snatched to filter when viewing Wanted issues on Wanted tab, FIX: When importing and then selecting a series to import via the select screen, will now flip back to the importresults and add the selected series in the background, IMP:(#952) Main page is now sorted in ascending order by Continuing/Ended status (and subbed by whether is Active/Paused).Custom sorting is still available, FIX: Dupecheck will now automatically assume existing 0-byte files are to be overwritten when performing post-processing, FIX: If publication date for series contained a '?' (usually with brand new series) will force to 'Present' to allow for pull-list comparisons to take place, FIX: Mylar will now disallow search results which have 'covers only' or 'variant' in the filename, IMP: Better nzbname generation/retrieval (will check inside nzb for possible names) to be used when post-processing, IMP: DB Update will now perform update to all active comics in descending order by Latest Date (instead of random order), FIX: Enforce the 5hr limit rule when running DB update (will only update series that haven't been updated in >5 hours), FIX: Annuals will now have/retain the proper status upon doing DB Update, FIX: Have totals will now be updated when doing a recheck files (sometimes wouldn't get updated depending on various states of status'), FIX:(#966) Added urllib2.URLError exeception trap when attempting to check Git for updates, IMP: Removed the individual sqlite calls for weeklypull, and brought them into line with using the db module (which will minimize concurrent access, which seemed to be causing db locks), IMP: Cleaned up some code and shuffled some functions so they are in more appropriate locations
2015-03-27 17:27:59 +00:00
if newi [ ' Annual ' ] == True :
myDB . upsert ( " Annuals " , newVAL , ctrlVAL )
else :
myDB . upsert ( " Issues " , newVAL , ctrlVAL )
FIX: (#746) updated autoProcessComics.py / ComicRN.py's which will now send proper completion messeages to clients, FIX: (#752) refresh series will now test if a series is 'out of whack' with it's numerical issue count (ie. 5/4) or it has no issue data due to a bad refresh / api maxing out and will adjust it's processing to accomodate either, IMP: (#750) Added ComicVine API Checker which will check API counts at regular intervals to inform/warn users of usage, as well as adding a screen-time display of API hits / mins used at the bottom of every page (refreshing/reloading pages will update counts), FIX: (#747)EOL normalization (dos2unix) on search.py - removed classes & exceptions as not being used, IMP: (#747) Skip processing issues with an invalid store date & issue date (thnx rupaschomaker), FIX: Removed strings when searching/logging torrents as was causing ascii errors especially with KAT, IMP: Added [META-TAGGING] to logging for meta-tagging module, IMP: Added ability in GUI to select CR or Cbl tags (or both) when writing metadata to cbz files, IMP: Improved support/usage with ComicTagger v1.1.15 which allows for personal CV API Key usage - if supplied to Mylar, will use when tagging with ComicTagger, IMP: Added Manual Search option to allow for individual searches of issues without changing initial status.
2014-06-18 19:58:19 +00:00
logger . info ( ' I have added ' + str ( len ( newiss ) ) + ' new issues for this series that were not present before. ' )
2014-08-18 19:54:34 +00:00
forceRescan ( ComicID )
2015-05-22 08:32:51 +00:00
FIX: (#746) updated autoProcessComics.py / ComicRN.py's which will now send proper completion messeages to clients, FIX: (#752) refresh series will now test if a series is 'out of whack' with it's numerical issue count (ie. 5/4) or it has no issue data due to a bad refresh / api maxing out and will adjust it's processing to accomodate either, IMP: (#750) Added ComicVine API Checker which will check API counts at regular intervals to inform/warn users of usage, as well as adding a screen-time display of API hits / mins used at the bottom of every page (refreshing/reloading pages will update counts), FIX: (#747)EOL normalization (dos2unix) on search.py - removed classes & exceptions as not being used, IMP: (#747) Skip processing issues with an invalid store date & issue date (thnx rupaschomaker), FIX: Removed strings when searching/logging torrents as was causing ascii errors especially with KAT, IMP: Added [META-TAGGING] to logging for meta-tagging module, IMP: Added ability in GUI to select CR or Cbl tags (or both) when writing metadata to cbz files, IMP: Improved support/usage with ComicTagger v1.1.15 which allows for personal CV API Key usage - if supplied to Mylar, will use when tagging with ComicTagger, IMP: Added Manual Search option to allow for individual searches of issues without changing initial status.
2014-06-18 19:58:19 +00:00
else :
2017-09-19 15:41:48 +00:00
chkstatus = mylar . importer . addComictoDB ( ComicID , mismatch , annload = annload , csyear = csyear )
#if cchk:
# #delete the data here if it's all valid.
# #logger.fdebug("Deleting all old issue data to make sure new data is clean...")
# myDB.action('DELETE FROM issues WHERE ComicID=?', [ComicID])
# myDB.action('DELETE FROM annuals WHERE ComicID=?', [ComicID])
# mylar.importer.issue_collection(cchk, nostatus='True')
# #need to update annuals at this point too....
# if annchk:
# mylar.importer.manualAnnual(annchk=annchk)
2014-05-25 19:08:15 +00:00
else :
2017-09-19 15:41:48 +00:00
chkstatus = mylar . importer . addComictoDB ( ComicID , mismatch )
IMP: Added ForceRSS Check and Test SABnzbd Connection buttons in Config, FIX: If Annuals not enabled, would error on home screen, IMP: updated mylar.init.d (thnx Kalinon), FIX: Manual Post-Processing fix for Manual Run (thnx Kalinon), IMP: Library Monitor working (check folder every X minutes and Post-Process), IMP: Future Upcoming introduction, IMP: Experimental search better handling of year inclusions, FIX: Filechecker will now pick up series with years in the series title accordingly, FIX: Torrent seedbox sending would lockup occassionally when attempting to send torrent file, FIX: malformed image url on some series, IMP: Moved issue updating to a seperate function, IMP: When series was refreshed, would download the last issue (or few issues depending on date), regardless of status, IMP: When series is volume 1 or volume label doesn't exist, either assume V1 or remove volume requirements to improve matching hits, IMP: StoryArcs will now check in StoryArc folder for existing issues and change status in StoryArc accordingly...
2013-11-28 15:48:59 +00:00
2017-11-18 01:00:49 +00:00
cnt + = 1
IMP: Added Choose specific Download option to manually select from result list on Upcoming/Details/Weekly tabs, IMP: Added Wanted storyarcs to the overall Wanted section so now will search as per global options (storyarc issues can be displayed optionally on Wanted tab), IMP: Added custom url option for image banners/posters for storyarcs, IMP: updated Cherrypy, FIX: Fixed ComicRN not working when forms authentication used - in conjunction updated autoProcessComics to 2.0 which now uses apikey instead of user/pass, IMP: Alternate Series Covers option for alternating existing series image to the most current issue image, IMP: Added overall series total to series page for reference, IMP: Search workflow completely changed to accomodate more than one indexer - now will simultaneously sumbit initial request to each provider, wait 30s submit additional as required at 30s intervals, FIX: Removed TPSE as an option and relabelled to just Public Torrents, IMP: Added direct backlog search to WWT option (pack support will follow), FIX: Removed line about configparser being required for ComicTagger usage, IMP: Test code in place for newzab testing, FIX: Fixed layout problem with torrents that are in auto-snatch status on weekly tab, IMP: backend code improvements to allow for better alias usage and annual linking directly from WS, IMP: Updated systemd init-scripts with read.me, IMP: When post-processing, will now check for available destination free space before actually moving files, IMP: Will copy during metatagging to cache folder instead of move being an option so cleanup is cleaner if something fails, FIX: Changed readinglist table to storyarcs for clarity, IMP: When post-processing issues, will now only update the one issue status and adjust totals accordingly (instead of doing a complete rescan of the series), FIX: Clear out empty ID's from the Failed DB on startup, IMP: Initial code-run at REST API interface, FIX: Fixed some matching problems with 32p due to case, IMP: removed apikeys from log entries that were accidentally logging, IMP: When searching 32p, if items get packed up - will now delete the cached reference so new items of the same can be located, IMP: ForceSearch option switched to scheduler so simultaneous runs should not occur, FIX: Fixed manual metatagging error that would occur if multiple destination directories existed
2018-02-16 19:57:01 +00:00
if all ( [ sched is False , calledfrom != ' refresh ' ] ) :
IMP:New scheduler tab (Manage / Activity) where you can see job current run status', next runtime, and prev runtimes as well as force/pause a job, FIX: Disabling
torrents will now properly hide torrent information, IMP: Specified daemon port for deluge as an on-screen tip for more detail, IMP: Added 100,200,ALL as viewable
watchlist views, FIX: When viewing pullist and annual integration enabled, if annual was present would incorrectly link to invalid annual series instead of the
actual series itself, IMP: Added more detail error messages to metatagging errors and better handling of stranded files during cleanup, IMP: Improved some handling for weekly pull-list one-off's and refactored the
nzb/oneoff post-processing into a seperate function for future callables, Moved all the main url locations for public torrent sites to the init module so that it
can be cascaded down for use in other modules instead as a global, IMP: Added a 'deep_search_32p' variable in the config.ini for specific usage with 32p, where
if there is more than one result will dig deeper into each result to try and figure out if there are series matches, as opposed to the default where it will only use ref32p
table if available or just the first hit in a multiple series search results and ignore the remainder, FIX:Fixed some unknown characters appearing in the pullist due
to unicode-related conversion problems, FIX: fixed some special cases of file parsing errors due to Volume label being named different than expected, FIX: Added a
3s pause between experimental searches to try and not hit their frequency limitation, IMP: Weekly Pullist One-off's will now show status of Snatched/Downloaded as
required, FIX: Fixed some deluge parameter problems when using auto-snatch torrent script/option, IMP: Changed the downlocation in the auto-snatch option to an
env variable instead of being passed to avoid unicode-related problems, FIX: Fixed some magnet-related issues for torrents when using a watchdir + TPSE, FIX: Added
more verbose error message for rtorrent connection issues, FIX: Could not connect to rtorrent client if no username/password were provided, IMP: Set the db updater to run
every 5 minutes on the watchlist, automatically refreshing the oldest updated series each time that is more than 5 hours old (force db update from the activity/job schedulers page will run the db updater
against the entire watchlist in sequence), IMP: Attempt to handle long paths in windows (ie. > 256c) by prepending the unicode windows api character to the import
a directory path (windows only), IMP: When manual metatagging a series, will update the series after all the metatagging has been completed as opposed to after each
issue, IMP: Will now display available inkdrops on Config/Search Providers tab when using 32P (future will utilize/indicate inkdrop threshold when downloading)
2017-09-10 15:50:08 +00:00
time . sleep ( 15 ) #pause for 15 secs so dont hammer CV and get 500 error
else :
break
2017-11-26 01:03:43 +00:00
helpers . job_management ( write = True , job = ' DB Updater ' , last_run_completed = helpers . utctimestamp ( ) , status = ' Waiting ' )
mylar . UPDATER_STATUS = ' Waiting '
2012-09-13 15:27:34 +00:00
logger . info ( ' Update complete ' )
def latest_update ( ComicID , LatestIssue , LatestDate ) :
# here we add to comics.latest
2017-01-20 19:15:15 +00:00
logger . fdebug ( str ( ComicID ) + ' - updating latest_date to : ' + str ( LatestDate ) )
2012-09-13 15:27:34 +00:00
myDB = db . DBConnection ( )
2012-10-16 08:16:29 +00:00
latestCTRLValueDict = { " ComicID " : ComicID }
newlatestDict = { " LatestIssue " : str ( LatestIssue ) ,
" LatestDate " : str ( LatestDate ) }
myDB . upsert ( " comics " , newlatestDict , latestCTRLValueDict )
2012-09-13 15:27:34 +00:00
IMP: Added ALT_PULL=2 method for weeklypull management. Will now update against an external site and pull down already populated ComicID/IssueID's for the series that exist on the pullist. Alternate Search Names aren't needed with this option to match on pullist, as well as CV API Hits are also not used since it is all populated. Also allows for future viewing of pullists (up to 4 weeks in advance), FIX: Alternate search names now will be searched against when doing manual post-processing, FIX: When manually post-processing, if series volume wasn't specified would fail to match against v1 (by default), IMP:(#1309) Https_chain option now allowed within config.ini, IMP: 32P pack support on a per series basis (will search individual torrents first before packs), IMP: When pack is snatched, will mark all issues within pack that are not in a Downloaded status as Snatched within Mylar (annuals currently don't work), IMP: Removed unnecessary config spamming on startup when verbose mode was enabled, IMP: Allow for searching on 32p against series+publisher for some titles that distinguish between series by different publisher, IMP: Better series matching when trying to find series matches on 32P, FIX: When metatagging, if volume label is not provided within Mylar would default to None (now will be v1), IMP:(#1304) Attempt at better file parsing when utf-8 filenames are being parsed, FIX: Proper handling of Infinity issue number when file-checking, FIX: When adding series and annuals were enabled, if annual was a new release, would not be shown on the annual subset table for the given series (and subsequently wouldn't be auto-marked as Wanted), FIX:(#1306) Correct handling of the imported value when doing an import and moving files was selected (would previously error out during moving for some imports), FIX: When cbz files were being imported and were attempted to being auto-imported, would fail due to improper handling of the imported variable, FIX: Manage issues will now default the dropdown to the correct selected option, FIX: Manage Comics - fixed dropdown options for multiple selection of series - delete/pause/resume, IMP: Added 'delete stragglers' option to Story Arcs when deleting an arc to ensure that all traces of the arc are removed from the db, FIX: Manual/group metatagging would not tag properly if the START_YEAR_AS_VOLUME option was enabled, FIX: (#1313) NzbHydra wouldn't set the nzbid properly when using Failed Download handling/Retrying
2016-07-10 22:28:14 +00:00
def upcoming_update ( ComicID , ComicName , IssueNumber , IssueDate , forcecheck = None , futurepull = None , altissuenumber = None , weekinfo = None ) :
2012-09-13 15:27:34 +00:00
# here we add to upcoming table...
myDB = db . DBConnection ( )
2013-07-30 04:57:37 +00:00
dspComicName = ComicName #to make sure that the word 'annual' will be displayed on screen
2015-05-22 08:32:51 +00:00
if ' annual ' in ComicName . lower ( ) :
2013-07-30 04:57:37 +00:00
adjComicName = re . sub ( " \\ bannual \\ b " , " " , ComicName . lower ( ) ) # for use with comparisons.
2013-08-19 06:14:47 +00:00
logger . fdebug ( ' annual detected - adjusting name to : ' + adjComicName )
2013-07-30 04:57:37 +00:00
else :
adjComicName = ComicName
2012-09-13 15:27:34 +00:00
controlValue = { " ComicID " : ComicID }
2013-07-30 04:57:37 +00:00
newValue = { " ComicName " : adjComicName ,
2012-10-16 08:16:29 +00:00
" IssueNumber " : str ( IssueNumber ) ,
2013-07-30 04:57:37 +00:00
" DisplayComicName " : dspComicName ,
2012-10-16 08:16:29 +00:00
" IssueDate " : str ( IssueDate ) }
2012-10-21 15:30:26 +00:00
IMP: Added ForceRSS Check and Test SABnzbd Connection buttons in Config, FIX: If Annuals not enabled, would error on home screen, IMP: updated mylar.init.d (thnx Kalinon), FIX: Manual Post-Processing fix for Manual Run (thnx Kalinon), IMP: Library Monitor working (check folder every X minutes and Post-Process), IMP: Future Upcoming introduction, IMP: Experimental search better handling of year inclusions, FIX: Filechecker will now pick up series with years in the series title accordingly, FIX: Torrent seedbox sending would lockup occassionally when attempting to send torrent file, FIX: malformed image url on some series, IMP: Moved issue updating to a seperate function, IMP: When series was refreshed, would download the last issue (or few issues depending on date), regardless of status, IMP: When series is volume 1 or volume label doesn't exist, either assume V1 or remove volume requirements to improve matching hits, IMP: StoryArcs will now check in StoryArc folder for existing issues and change status in StoryArc accordingly...
2013-11-28 15:48:59 +00:00
#let's refresh the series here just to make sure if an issue is available/not.
2013-01-01 20:09:28 +00:00
mismatch = " no "
IMP: Ability to now specify search provider order (regardless of torrents or nzb) within the config.ini, IMP: (#667) Changed the db module to try to accomodate db locking errors and lowering the amount of actual write transactions that were committed along with a new scheduler system, IMP: Changed sabnzbd directory to post-processing, and included subdirs for sabnzbd & nzbget ComicRN scripts, IMP: NZBGet Post-Processing ComicRN.py script (updated for use with nzbget v11.0+)added & updated in post-processing/nzbget directory (thnx ministoat), FIX: If Issue Location was None, and status was Downloaded would cause error in GUI and break series, IMP: (#689) Minimum # of seeders added (will work with KAT), IMP: (#680) Added Boxcar 2 IO Notifications, IMP: Added PushBullet Notifications, IMP: Cleaned up some notification messages so it's not so cluttered, IMP: Added Clickable series link in History tab, IMP: Added Post-Processed as a status to History tab to show manually post-processed items, IMP: Removed log level dropdown from Logs page & added 'ThreadName' as a column, IMP: Added Force Check Availability & View Future Pull-list to Upcoming sub-tabs, IMP: Added '--safe' option to startup options which will redirect directly to Manage Comics screen incase things are broken, FIX: Added proper month conversions for manual post-processing when doing comparitive issue analysis for matches, FIX: (#613) Allow for negative issue numbers in post-processing when renaming and issue padding is enabled, FIX: File Permissions on post-processing would stop post-processing if couldn't change, now will just log the error and continue, IMP: Added Scheduler (from sickbeard) to allow for threadnaming and better scheduling, IMP: Filenames in the format of ' () ' will now get scanned in, IMP: During manual post-processing will now stop looking for matches upon a successful match, IMP: A Refresh/Weeklypull series check will now just scan in issue data, instead of series info,etc, IMP: Removed some legacy GCD code that is no longer in use, IMP: Exception/traceback handling will now be logged, FIX: Unable to grab torrents from KAT due to content-encoding detection failing, IMP: Added universal date-time conversion to allow for non-english based dates to be properly compared when checking search results against publication dates, FIX: Annuals will now get proper notification (prior was leaving out the word 'annual' from notification/logs), IMP: Improved future pull-list detection and increased retension (now ~5 months), IMP: Will now mark new issues as Wanted on a Refresh Series if autowant upcoming is enabled (was reverting to a status of None previously), IMP: Cannot change status to Downloaded if current status is Skipped or Wanted, FIX: (#704) UnSkipped will now work (X in options column on comic details page), IMP: future_check will check upcoming future issues (future pull-list) that have no series data yet (ie. #1's) and auto-add them to watchlist when the data is available and auto-want accordingly, IMP: (#706) Downloading issues to local machine (via comicdetails screen) with special characters in filename now will work, IMP: improved comparison checks during weekly pull list and improved speed abit since only refreshing issue data now instead of entire series, Other Referenced issues: (#670)(#690) and some others....
2014-05-25 18:32:11 +00:00
CV_EXcomicid = myDB . selectone ( " SELECT * from exceptions WHERE ComicID=? " , [ ComicID ] ) . fetchone ( )
IMP: Added ALT_PULL=2 method for weeklypull management. Will now update against an external site and pull down already populated ComicID/IssueID's for the series that exist on the pullist. Alternate Search Names aren't needed with this option to match on pullist, as well as CV API Hits are also not used since it is all populated. Also allows for future viewing of pullists (up to 4 weeks in advance), FIX: Alternate search names now will be searched against when doing manual post-processing, FIX: When manually post-processing, if series volume wasn't specified would fail to match against v1 (by default), IMP:(#1309) Https_chain option now allowed within config.ini, IMP: 32P pack support on a per series basis (will search individual torrents first before packs), IMP: When pack is snatched, will mark all issues within pack that are not in a Downloaded status as Snatched within Mylar (annuals currently don't work), IMP: Removed unnecessary config spamming on startup when verbose mode was enabled, IMP: Allow for searching on 32p against series+publisher for some titles that distinguish between series by different publisher, IMP: Better series matching when trying to find series matches on 32P, FIX: When metatagging, if volume label is not provided within Mylar would default to None (now will be v1), IMP:(#1304) Attempt at better file parsing when utf-8 filenames are being parsed, FIX: Proper handling of Infinity issue number when file-checking, FIX: When adding series and annuals were enabled, if annual was a new release, would not be shown on the annual subset table for the given series (and subsequently wouldn't be auto-marked as Wanted), FIX:(#1306) Correct handling of the imported value when doing an import and moving files was selected (would previously error out during moving for some imports), FIX: When cbz files were being imported and were attempted to being auto-imported, would fail due to improper handling of the imported variable, FIX: Manage issues will now default the dropdown to the correct selected option, FIX: Manage Comics - fixed dropdown options for multiple selection of series - delete/pause/resume, IMP: Added 'delete stragglers' option to Story Arcs when deleting an arc to ensure that all traces of the arc are removed from the db, FIX: Manual/group metatagging would not tag properly if the START_YEAR_AS_VOLUME option was enabled, FIX: (#1313) NzbHydra wouldn't set the nzbid properly when using Failed Download handling/Retrying
2016-07-10 22:28:14 +00:00
if CV_EXcomicid is None :
pass
2013-01-01 20:09:28 +00:00
else :
if CV_EXcomicid [ ' variloop ' ] == ' 99 ' :
mismatch = " yes "
2017-10-25 18:33:56 +00:00
if mylar . CONFIG . ALT_PULL != 2 or mylar . PULLBYFILE is True :
IMP: Added ALT_PULL=2 method for weeklypull management. Will now update against an external site and pull down already populated ComicID/IssueID's for the series that exist on the pullist. Alternate Search Names aren't needed with this option to match on pullist, as well as CV API Hits are also not used since it is all populated. Also allows for future viewing of pullists (up to 4 weeks in advance), FIX: Alternate search names now will be searched against when doing manual post-processing, FIX: When manually post-processing, if series volume wasn't specified would fail to match against v1 (by default), IMP:(#1309) Https_chain option now allowed within config.ini, IMP: 32P pack support on a per series basis (will search individual torrents first before packs), IMP: When pack is snatched, will mark all issues within pack that are not in a Downloaded status as Snatched within Mylar (annuals currently don't work), IMP: Removed unnecessary config spamming on startup when verbose mode was enabled, IMP: Allow for searching on 32p against series+publisher for some titles that distinguish between series by different publisher, IMP: Better series matching when trying to find series matches on 32P, FIX: When metatagging, if volume label is not provided within Mylar would default to None (now will be v1), IMP:(#1304) Attempt at better file parsing when utf-8 filenames are being parsed, FIX: Proper handling of Infinity issue number when file-checking, FIX: When adding series and annuals were enabled, if annual was a new release, would not be shown on the annual subset table for the given series (and subsequently wouldn't be auto-marked as Wanted), FIX:(#1306) Correct handling of the imported value when doing an import and moving files was selected (would previously error out during moving for some imports), FIX: When cbz files were being imported and were attempted to being auto-imported, would fail due to improper handling of the imported variable, FIX: Manage issues will now default the dropdown to the correct selected option, FIX: Manage Comics - fixed dropdown options for multiple selection of series - delete/pause/resume, IMP: Added 'delete stragglers' option to Story Arcs when deleting an arc to ensure that all traces of the arc are removed from the db, FIX: Manual/group metatagging would not tag properly if the START_YEAR_AS_VOLUME option was enabled, FIX: (#1313) NzbHydra wouldn't set the nzbid properly when using Failed Download handling/Retrying
2016-07-10 22:28:14 +00:00
lastupdatechk = myDB . selectone ( " SELECT * FROM comics WHERE ComicID=? " , [ ComicID ] ) . fetchone ( )
if lastupdatechk is None :
pullupd = " yes "
else :
c_date = lastupdatechk [ ' LastUpdated ' ]
if c_date is None :
logger . error ( lastupdatechk [ ' ComicName ' ] + ' failed during a previous add /refresh. Please either delete and readd the series, or try a refresh of the series. ' )
return
c_obj_date = datetime . datetime . strptime ( c_date , " % Y- % m- %d % H: % M: % S " )
n_date = datetime . datetime . now ( )
absdiff = abs ( n_date - c_obj_date )
hours = ( absdiff . days * 24 * 60 * 60 + absdiff . seconds ) / 3600.0
2013-01-07 19:26:10 +00:00
else :
IMP: Added ALT_PULL=2 method for weeklypull management. Will now update against an external site and pull down already populated ComicID/IssueID's for the series that exist on the pullist. Alternate Search Names aren't needed with this option to match on pullist, as well as CV API Hits are also not used since it is all populated. Also allows for future viewing of pullists (up to 4 weeks in advance), FIX: Alternate search names now will be searched against when doing manual post-processing, FIX: When manually post-processing, if series volume wasn't specified would fail to match against v1 (by default), IMP:(#1309) Https_chain option now allowed within config.ini, IMP: 32P pack support on a per series basis (will search individual torrents first before packs), IMP: When pack is snatched, will mark all issues within pack that are not in a Downloaded status as Snatched within Mylar (annuals currently don't work), IMP: Removed unnecessary config spamming on startup when verbose mode was enabled, IMP: Allow for searching on 32p against series+publisher for some titles that distinguish between series by different publisher, IMP: Better series matching when trying to find series matches on 32P, FIX: When metatagging, if volume label is not provided within Mylar would default to None (now will be v1), IMP:(#1304) Attempt at better file parsing when utf-8 filenames are being parsed, FIX: Proper handling of Infinity issue number when file-checking, FIX: When adding series and annuals were enabled, if annual was a new release, would not be shown on the annual subset table for the given series (and subsequently wouldn't be auto-marked as Wanted), FIX:(#1306) Correct handling of the imported value when doing an import and moving files was selected (would previously error out during moving for some imports), FIX: When cbz files were being imported and were attempted to being auto-imported, would fail due to improper handling of the imported variable, FIX: Manage issues will now default the dropdown to the correct selected option, FIX: Manage Comics - fixed dropdown options for multiple selection of series - delete/pause/resume, IMP: Added 'delete stragglers' option to Story Arcs when deleting an arc to ensure that all traces of the arc are removed from the db, FIX: Manual/group metatagging would not tag properly if the START_YEAR_AS_VOLUME option was enabled, FIX: (#1313) NzbHydra wouldn't set the nzbid properly when using Failed Download handling/Retrying
2016-07-10 22:28:14 +00:00
#if it's at this point and the refresh is None, odds are very good that it's already up-to-date so let it flow thru
2017-10-25 18:33:56 +00:00
if mylar . CONFIG . PULL_REFRESH is None :
2018-03-18 02:25:24 +00:00
mylar . CONFIG . PULL_REFRESH = datetime . datetime . today ( ) . replace ( second = 0 , microsecond = 0 )
2016-08-20 21:56:30 +00:00
#update the PULL_REFRESH
2017-10-25 18:33:56 +00:00
#mylar.config_write()
logger . fdebug ( ' pull_refresh: ' + str ( mylar . CONFIG . PULL_REFRESH ) )
c_obj_date = datetime . datetime . strptime ( str ( mylar . CONFIG . PULL_REFRESH ) , " % Y- % m- %d % H: % M: % S " )
IMP: Added ALT_PULL=2 method for weeklypull management. Will now update against an external site and pull down already populated ComicID/IssueID's for the series that exist on the pullist. Alternate Search Names aren't needed with this option to match on pullist, as well as CV API Hits are also not used since it is all populated. Also allows for future viewing of pullists (up to 4 weeks in advance), FIX: Alternate search names now will be searched against when doing manual post-processing, FIX: When manually post-processing, if series volume wasn't specified would fail to match against v1 (by default), IMP:(#1309) Https_chain option now allowed within config.ini, IMP: 32P pack support on a per series basis (will search individual torrents first before packs), IMP: When pack is snatched, will mark all issues within pack that are not in a Downloaded status as Snatched within Mylar (annuals currently don't work), IMP: Removed unnecessary config spamming on startup when verbose mode was enabled, IMP: Allow for searching on 32p against series+publisher for some titles that distinguish between series by different publisher, IMP: Better series matching when trying to find series matches on 32P, FIX: When metatagging, if volume label is not provided within Mylar would default to None (now will be v1), IMP:(#1304) Attempt at better file parsing when utf-8 filenames are being parsed, FIX: Proper handling of Infinity issue number when file-checking, FIX: When adding series and annuals were enabled, if annual was a new release, would not be shown on the annual subset table for the given series (and subsequently wouldn't be auto-marked as Wanted), FIX:(#1306) Correct handling of the imported value when doing an import and moving files was selected (would previously error out during moving for some imports), FIX: When cbz files were being imported and were attempted to being auto-imported, would fail due to improper handling of the imported variable, FIX: Manage issues will now default the dropdown to the correct selected option, FIX: Manage Comics - fixed dropdown options for multiple selection of series - delete/pause/resume, IMP: Added 'delete stragglers' option to Story Arcs when deleting an arc to ensure that all traces of the arc are removed from the db, FIX: Manual/group metatagging would not tag properly if the START_YEAR_AS_VOLUME option was enabled, FIX: (#1313) NzbHydra wouldn't set the nzbid properly when using Failed Download handling/Retrying
2016-07-10 22:28:14 +00:00
#logger.fdebug('c_obj_date: ' + str(c_obj_date))
2013-01-07 19:26:10 +00:00
n_date = datetime . datetime . now ( )
IMP: Added ALT_PULL=2 method for weeklypull management. Will now update against an external site and pull down already populated ComicID/IssueID's for the series that exist on the pullist. Alternate Search Names aren't needed with this option to match on pullist, as well as CV API Hits are also not used since it is all populated. Also allows for future viewing of pullists (up to 4 weeks in advance), FIX: Alternate search names now will be searched against when doing manual post-processing, FIX: When manually post-processing, if series volume wasn't specified would fail to match against v1 (by default), IMP:(#1309) Https_chain option now allowed within config.ini, IMP: 32P pack support on a per series basis (will search individual torrents first before packs), IMP: When pack is snatched, will mark all issues within pack that are not in a Downloaded status as Snatched within Mylar (annuals currently don't work), IMP: Removed unnecessary config spamming on startup when verbose mode was enabled, IMP: Allow for searching on 32p against series+publisher for some titles that distinguish between series by different publisher, IMP: Better series matching when trying to find series matches on 32P, FIX: When metatagging, if volume label is not provided within Mylar would default to None (now will be v1), IMP:(#1304) Attempt at better file parsing when utf-8 filenames are being parsed, FIX: Proper handling of Infinity issue number when file-checking, FIX: When adding series and annuals were enabled, if annual was a new release, would not be shown on the annual subset table for the given series (and subsequently wouldn't be auto-marked as Wanted), FIX:(#1306) Correct handling of the imported value when doing an import and moving files was selected (would previously error out during moving for some imports), FIX: When cbz files were being imported and were attempted to being auto-imported, would fail due to improper handling of the imported variable, FIX: Manage issues will now default the dropdown to the correct selected option, FIX: Manage Comics - fixed dropdown options for multiple selection of series - delete/pause/resume, IMP: Added 'delete stragglers' option to Story Arcs when deleting an arc to ensure that all traces of the arc are removed from the db, FIX: Manual/group metatagging would not tag properly if the START_YEAR_AS_VOLUME option was enabled, FIX: (#1313) NzbHydra wouldn't set the nzbid properly when using Failed Download handling/Retrying
2016-07-10 22:28:14 +00:00
#logger.fdebug('n_date: ' + str(n_date))
2013-01-07 19:26:10 +00:00
absdiff = abs ( n_date - c_obj_date )
IMP: Added ALT_PULL=2 method for weeklypull management. Will now update against an external site and pull down already populated ComicID/IssueID's for the series that exist on the pullist. Alternate Search Names aren't needed with this option to match on pullist, as well as CV API Hits are also not used since it is all populated. Also allows for future viewing of pullists (up to 4 weeks in advance), FIX: Alternate search names now will be searched against when doing manual post-processing, FIX: When manually post-processing, if series volume wasn't specified would fail to match against v1 (by default), IMP:(#1309) Https_chain option now allowed within config.ini, IMP: 32P pack support on a per series basis (will search individual torrents first before packs), IMP: When pack is snatched, will mark all issues within pack that are not in a Downloaded status as Snatched within Mylar (annuals currently don't work), IMP: Removed unnecessary config spamming on startup when verbose mode was enabled, IMP: Allow for searching on 32p against series+publisher for some titles that distinguish between series by different publisher, IMP: Better series matching when trying to find series matches on 32P, FIX: When metatagging, if volume label is not provided within Mylar would default to None (now will be v1), IMP:(#1304) Attempt at better file parsing when utf-8 filenames are being parsed, FIX: Proper handling of Infinity issue number when file-checking, FIX: When adding series and annuals were enabled, if annual was a new release, would not be shown on the annual subset table for the given series (and subsequently wouldn't be auto-marked as Wanted), FIX:(#1306) Correct handling of the imported value when doing an import and moving files was selected (would previously error out during moving for some imports), FIX: When cbz files were being imported and were attempted to being auto-imported, would fail due to improper handling of the imported variable, FIX: Manage issues will now default the dropdown to the correct selected option, FIX: Manage Comics - fixed dropdown options for multiple selection of series - delete/pause/resume, IMP: Added 'delete stragglers' option to Story Arcs when deleting an arc to ensure that all traces of the arc are removed from the db, FIX: Manual/group metatagging would not tag properly if the START_YEAR_AS_VOLUME option was enabled, FIX: (#1313) NzbHydra wouldn't set the nzbid properly when using Failed Download handling/Retrying
2016-07-10 22:28:14 +00:00
#logger.fdebug('absdiff: ' + str(absdiff))
2013-01-07 19:26:10 +00:00
hours = ( absdiff . days * 24 * 60 * 60 + absdiff . seconds ) / 3600.0
IMP: Added ALT_PULL=2 method for weeklypull management. Will now update against an external site and pull down already populated ComicID/IssueID's for the series that exist on the pullist. Alternate Search Names aren't needed with this option to match on pullist, as well as CV API Hits are also not used since it is all populated. Also allows for future viewing of pullists (up to 4 weeks in advance), FIX: Alternate search names now will be searched against when doing manual post-processing, FIX: When manually post-processing, if series volume wasn't specified would fail to match against v1 (by default), IMP:(#1309) Https_chain option now allowed within config.ini, IMP: 32P pack support on a per series basis (will search individual torrents first before packs), IMP: When pack is snatched, will mark all issues within pack that are not in a Downloaded status as Snatched within Mylar (annuals currently don't work), IMP: Removed unnecessary config spamming on startup when verbose mode was enabled, IMP: Allow for searching on 32p against series+publisher for some titles that distinguish between series by different publisher, IMP: Better series matching when trying to find series matches on 32P, FIX: When metatagging, if volume label is not provided within Mylar would default to None (now will be v1), IMP:(#1304) Attempt at better file parsing when utf-8 filenames are being parsed, FIX: Proper handling of Infinity issue number when file-checking, FIX: When adding series and annuals were enabled, if annual was a new release, would not be shown on the annual subset table for the given series (and subsequently wouldn't be auto-marked as Wanted), FIX:(#1306) Correct handling of the imported value when doing an import and moving files was selected (would previously error out during moving for some imports), FIX: When cbz files were being imported and were attempted to being auto-imported, would fail due to improper handling of the imported variable, FIX: Manage issues will now default the dropdown to the correct selected option, FIX: Manage Comics - fixed dropdown options for multiple selection of series - delete/pause/resume, IMP: Added 'delete stragglers' option to Story Arcs when deleting an arc to ensure that all traces of the arc are removed from the db, FIX: Manual/group metatagging would not tag properly if the START_YEAR_AS_VOLUME option was enabled, FIX: (#1313) NzbHydra wouldn't set the nzbid properly when using Failed Download handling/Retrying
2016-07-10 22:28:14 +00:00
#logger.fdebug('hours: ' + str(hours))
IMP: Updated CT to a newer version - changed to use rarfile, fixes size invalid errors with some cbr's encountered during conversions, IMP: Search and Post-Processing Queues now added in order to queue up all searches/post-processing in sequence instead of loading lists and iterating through it and encountering various lock-related errors, IMP: In most cases, will now post-process directly against comicid/issueid/storyarcid instead of relying on relationship via nzb/torrent, FIX: realigned some checkboxes on series detail page, FIX: in configuration, indicated that ComicRN cannot be used when Completed Download Handling is enabled for a client now, FIX: Fix for issues named as 'special' that are part of the annuals section, but would not work in various portions (also will allow for naming differences in the future hopefully), FIX: Will now trap error properly when file cannot be located during dupecheck so that post-processing can continue without error, FIX: Fixed newznab test option on configuration screen so that it will return the correct response (previous was an invalid api endpoint), FIX: When retrieving image for storyarcs would error when no image was present and stop the process, FIX: accounted for some incorrect newznab responses that would return a non-api endpoint when retrieving via rss, which resulted in invalid nzbs, FIX: in some cases, weekly pull snatches would not update properly on the weekly page due to some db values not being written
2018-06-14 17:07:39 +00:00
if any ( [ ' annual ' in ComicName . lower ( ) , ' special ' in ComicName . lower ( ) ] ) :
2017-10-25 18:33:56 +00:00
if mylar . CONFIG . ANNUALS_ON :
FIX:(#1466) When post-processing if filename did not have issue year or volume, would error, IMP:(#1418) Separate copy/move/hardlink/symlink option for Story arcs, IMP:(#1417) Added story arc format folder to story arc options - can now specify folder format for story arc naming, IMP:(#1374) Added rtorrent_authentication to config.ini allowing user to specify digest/basic authentication, as well as the rpc_url (rtorrent_rpc_url) and ssl (rtorrent_ssl). Rtorrent_host no longer requires protocol in field, IMP: Instead of hitting the intermediary/32P site to get the torrent group, will now retain specific id for torrent group to speed up searches, FIX: (#1472) Fixed some problems with series being searched against 32P which had special characters and would return no results, FIX: Added some specific exception catches when attempting to retrieve search results for nzb providers, FIX: Fixed publisher retaining problem for story arcs (note story arcs will have to be refreshed), FIX: Fixed a problem with the pull-list matching up to the correct series when the issue was an annual for a given series with the alt_pull 2 method (would create an invalid link on the Wanted tab), IMP:(#1470) Added config.ini option to specify folder-formatting for weekly folder numbering, 0=YYYY-WN, 1=YYYY-MM-DD, IMP: (#1471) Added quick check to ensure that newznab host entries do not contain leading/trailing spaces and will strip accordingly, FIX:(#1476) Annuals now have option to add to reading list via the series detail page, IMP: Fixed password fields to allow for special characters within password, IMP: Weekly pull will now correctly show status for previous/future weeks for watchlisted series, IMP: Download option on pullist will work for previous weeks, FIX: Fixed some issues with one-off and story arc post-processing using different file operations, FIX: Fixed error when performing an 'Force Update Active Comics' from the Manage tab (or the scheduled updating of all active comics), IMP: Storyarc issues that are not on watchlist will now retain value for volume to help when searching and file-checking/post-processing, FIX:(#1473) Changed pullist dates on pullist page to unicode to allow for non-English presentation
2016-12-02 17:45:49 +00:00
logger . info ( ' checking: ' + str ( ComicID ) + ' -- issue#: ' + str ( IssueNumber ) )
IMP: Ability to now specify search provider order (regardless of torrents or nzb) within the config.ini, IMP: (#667) Changed the db module to try to accomodate db locking errors and lowering the amount of actual write transactions that were committed along with a new scheduler system, IMP: Changed sabnzbd directory to post-processing, and included subdirs for sabnzbd & nzbget ComicRN scripts, IMP: NZBGet Post-Processing ComicRN.py script (updated for use with nzbget v11.0+)added & updated in post-processing/nzbget directory (thnx ministoat), FIX: If Issue Location was None, and status was Downloaded would cause error in GUI and break series, IMP: (#689) Minimum # of seeders added (will work with KAT), IMP: (#680) Added Boxcar 2 IO Notifications, IMP: Added PushBullet Notifications, IMP: Cleaned up some notification messages so it's not so cluttered, IMP: Added Clickable series link in History tab, IMP: Added Post-Processed as a status to History tab to show manually post-processed items, IMP: Removed log level dropdown from Logs page & added 'ThreadName' as a column, IMP: Added Force Check Availability & View Future Pull-list to Upcoming sub-tabs, IMP: Added '--safe' option to startup options which will redirect directly to Manage Comics screen incase things are broken, FIX: Added proper month conversions for manual post-processing when doing comparitive issue analysis for matches, FIX: (#613) Allow for negative issue numbers in post-processing when renaming and issue padding is enabled, FIX: File Permissions on post-processing would stop post-processing if couldn't change, now will just log the error and continue, IMP: Added Scheduler (from sickbeard) to allow for threadnaming and better scheduling, IMP: Filenames in the format of ' () ' will now get scanned in, IMP: During manual post-processing will now stop looking for matches upon a successful match, IMP: A Refresh/Weeklypull series check will now just scan in issue data, instead of series info,etc, IMP: Removed some legacy GCD code that is no longer in use, IMP: Exception/traceback handling will now be logged, FIX: Unable to grab torrents from KAT due to content-encoding detection failing, IMP: Added universal date-time conversion to allow for non-english based dates to be properly compared when checking search results against publication dates, FIX: Annuals will now get proper notification (prior was leaving out the word 'annual' from notification/logs), IMP: Improved future pull-list detection and increased retension (now ~5 months), IMP: Will now mark new issues as Wanted on a Refresh Series if autowant upcoming is enabled (was reverting to a status of None previously), IMP: Cannot change status to Downloaded if current status is Skipped or Wanted, FIX: (#704) UnSkipped will now work (X in options column on comic details page), IMP: future_check will check upcoming future issues (future pull-list) that have no series data yet (ie. #1's) and auto-add them to watchlist when the data is available and auto-want accordingly, IMP: (#706) Downloading issues to local machine (via comicdetails screen) with special characters in filename now will work, IMP: improved comparison checks during weekly pull list and improved speed abit since only refreshing issue data now instead of entire series, Other Referenced issues: (#670)(#690) and some others....
2014-05-25 18:32:11 +00:00
issuechk = myDB . selectone ( " SELECT * FROM annuals WHERE ComicID=? AND Issue_Number=? " , [ ComicID , IssueNumber ] ) . fetchone ( )
2013-07-31 06:34:07 +00:00
else :
IMP: Updated CT to a newer version - changed to use rarfile, fixes size invalid errors with some cbr's encountered during conversions, IMP: Search and Post-Processing Queues now added in order to queue up all searches/post-processing in sequence instead of loading lists and iterating through it and encountering various lock-related errors, IMP: In most cases, will now post-process directly against comicid/issueid/storyarcid instead of relying on relationship via nzb/torrent, FIX: realigned some checkboxes on series detail page, FIX: in configuration, indicated that ComicRN cannot be used when Completed Download Handling is enabled for a client now, FIX: Fix for issues named as 'special' that are part of the annuals section, but would not work in various portions (also will allow for naming differences in the future hopefully), FIX: Will now trap error properly when file cannot be located during dupecheck so that post-processing can continue without error, FIX: Fixed newznab test option on configuration screen so that it will return the correct response (previous was an invalid api endpoint), FIX: When retrieving image for storyarcs would error when no image was present and stop the process, FIX: accounted for some incorrect newznab responses that would return a non-api endpoint when retrieving via rss, which resulted in invalid nzbs, FIX: in some cases, weekly pull snatches would not update properly on the weekly page due to some db values not being written
2018-06-14 17:07:39 +00:00
logger . fdebug ( ' Non-standard issue detected (annual/special/etc), but Annual Integration is not enabled. Ignoring result. ' )
2013-07-31 06:34:07 +00:00
return
2013-07-30 04:57:37 +00:00
else :
IMP: Ability to now specify search provider order (regardless of torrents or nzb) within the config.ini, IMP: (#667) Changed the db module to try to accomodate db locking errors and lowering the amount of actual write transactions that were committed along with a new scheduler system, IMP: Changed sabnzbd directory to post-processing, and included subdirs for sabnzbd & nzbget ComicRN scripts, IMP: NZBGet Post-Processing ComicRN.py script (updated for use with nzbget v11.0+)added & updated in post-processing/nzbget directory (thnx ministoat), FIX: If Issue Location was None, and status was Downloaded would cause error in GUI and break series, IMP: (#689) Minimum # of seeders added (will work with KAT), IMP: (#680) Added Boxcar 2 IO Notifications, IMP: Added PushBullet Notifications, IMP: Cleaned up some notification messages so it's not so cluttered, IMP: Added Clickable series link in History tab, IMP: Added Post-Processed as a status to History tab to show manually post-processed items, IMP: Removed log level dropdown from Logs page & added 'ThreadName' as a column, IMP: Added Force Check Availability & View Future Pull-list to Upcoming sub-tabs, IMP: Added '--safe' option to startup options which will redirect directly to Manage Comics screen incase things are broken, FIX: Added proper month conversions for manual post-processing when doing comparitive issue analysis for matches, FIX: (#613) Allow for negative issue numbers in post-processing when renaming and issue padding is enabled, FIX: File Permissions on post-processing would stop post-processing if couldn't change, now will just log the error and continue, IMP: Added Scheduler (from sickbeard) to allow for threadnaming and better scheduling, IMP: Filenames in the format of ' () ' will now get scanned in, IMP: During manual post-processing will now stop looking for matches upon a successful match, IMP: A Refresh/Weeklypull series check will now just scan in issue data, instead of series info,etc, IMP: Removed some legacy GCD code that is no longer in use, IMP: Exception/traceback handling will now be logged, FIX: Unable to grab torrents from KAT due to content-encoding detection failing, IMP: Added universal date-time conversion to allow for non-english based dates to be properly compared when checking search results against publication dates, FIX: Annuals will now get proper notification (prior was leaving out the word 'annual' from notification/logs), IMP: Improved future pull-list detection and increased retension (now ~5 months), IMP: Will now mark new issues as Wanted on a Refresh Series if autowant upcoming is enabled (was reverting to a status of None previously), IMP: Cannot change status to Downloaded if current status is Skipped or Wanted, FIX: (#704) UnSkipped will now work (X in options column on comic details page), IMP: future_check will check upcoming future issues (future pull-list) that have no series data yet (ie. #1's) and auto-add them to watchlist when the data is available and auto-want accordingly, IMP: (#706) Downloading issues to local machine (via comicdetails screen) with special characters in filename now will work, IMP: improved comparison checks during weekly pull list and improved speed abit since only refreshing issue data now instead of entire series, Other Referenced issues: (#670)(#690) and some others....
2014-05-25 18:32:11 +00:00
issuechk = myDB . selectone ( " SELECT * FROM issues WHERE ComicID=? AND Issue_Number=? " , [ ComicID , IssueNumber ] ) . fetchone ( )
IMP: Removed unneeded code from several modules, IMP:(#659) Subdirectories now will be scanned in properly when in series directories, FIX:(#635)(#658) Moved code from html into module to allow for better integration and further templating needs, IMP: Added ability for mylar to auto-grab SABnzbd API when provided with user/pass, IMP: Test SABnzbd button works again - will auto-rollback to NZBKey usage, IMP: Added counts to Upcoming page, IMP: Added some more detailed parsing to the ImportResults, IMP: ImportResults will now show proper series title, IMP: Improved some logic when determining if a new weekly issue is on a watchlist if a rebooted series, IMP: A bunch of smaller fixes, and some various code tweaks...
2014-04-02 19:08:59 +00:00
if issuechk is None and altissuenumber is not None :
logger . info ( ' altissuenumber is : ' + str ( altissuenumber ) )
IMP: Ability to now specify search provider order (regardless of torrents or nzb) within the config.ini, IMP: (#667) Changed the db module to try to accomodate db locking errors and lowering the amount of actual write transactions that were committed along with a new scheduler system, IMP: Changed sabnzbd directory to post-processing, and included subdirs for sabnzbd & nzbget ComicRN scripts, IMP: NZBGet Post-Processing ComicRN.py script (updated for use with nzbget v11.0+)added & updated in post-processing/nzbget directory (thnx ministoat), FIX: If Issue Location was None, and status was Downloaded would cause error in GUI and break series, IMP: (#689) Minimum # of seeders added (will work with KAT), IMP: (#680) Added Boxcar 2 IO Notifications, IMP: Added PushBullet Notifications, IMP: Cleaned up some notification messages so it's not so cluttered, IMP: Added Clickable series link in History tab, IMP: Added Post-Processed as a status to History tab to show manually post-processed items, IMP: Removed log level dropdown from Logs page & added 'ThreadName' as a column, IMP: Added Force Check Availability & View Future Pull-list to Upcoming sub-tabs, IMP: Added '--safe' option to startup options which will redirect directly to Manage Comics screen incase things are broken, FIX: Added proper month conversions for manual post-processing when doing comparitive issue analysis for matches, FIX: (#613) Allow for negative issue numbers in post-processing when renaming and issue padding is enabled, FIX: File Permissions on post-processing would stop post-processing if couldn't change, now will just log the error and continue, IMP: Added Scheduler (from sickbeard) to allow for threadnaming and better scheduling, IMP: Filenames in the format of ' () ' will now get scanned in, IMP: During manual post-processing will now stop looking for matches upon a successful match, IMP: A Refresh/Weeklypull series check will now just scan in issue data, instead of series info,etc, IMP: Removed some legacy GCD code that is no longer in use, IMP: Exception/traceback handling will now be logged, FIX: Unable to grab torrents from KAT due to content-encoding detection failing, IMP: Added universal date-time conversion to allow for non-english based dates to be properly compared when checking search results against publication dates, FIX: Annuals will now get proper notification (prior was leaving out the word 'annual' from notification/logs), IMP: Improved future pull-list detection and increased retension (now ~5 months), IMP: Will now mark new issues as Wanted on a Refresh Series if autowant upcoming is enabled (was reverting to a status of None previously), IMP: Cannot change status to Downloaded if current status is Skipped or Wanted, FIX: (#704) UnSkipped will now work (X in options column on comic details page), IMP: future_check will check upcoming future issues (future pull-list) that have no series data yet (ie. #1's) and auto-add them to watchlist when the data is available and auto-want accordingly, IMP: (#706) Downloading issues to local machine (via comicdetails screen) with special characters in filename now will work, IMP: improved comparison checks during weekly pull list and improved speed abit since only refreshing issue data now instead of entire series, Other Referenced issues: (#670)(#690) and some others....
2014-05-25 18:32:11 +00:00
issuechk = myDB . selectone ( " SELECT * FROM issues WHERE ComicID=? AND Int_IssueNumber=? " , [ ComicID , helpers . issuedigits ( altissuenumber ) ] ) . fetchone ( )
2012-12-31 16:52:16 +00:00
if issuechk is None :
IMP: Added ForceRSS Check and Test SABnzbd Connection buttons in Config, FIX: If Annuals not enabled, would error on home screen, IMP: updated mylar.init.d (thnx Kalinon), FIX: Manual Post-Processing fix for Manual Run (thnx Kalinon), IMP: Library Monitor working (check folder every X minutes and Post-Process), IMP: Future Upcoming introduction, IMP: Experimental search better handling of year inclusions, FIX: Filechecker will now pick up series with years in the series title accordingly, FIX: Torrent seedbox sending would lockup occassionally when attempting to send torrent file, FIX: malformed image url on some series, IMP: Moved issue updating to a seperate function, IMP: When series was refreshed, would download the last issue (or few issues depending on date), regardless of status, IMP: When series is volume 1 or volume label doesn't exist, either assume V1 or remove volume requirements to improve matching hits, IMP: StoryArcs will now check in StoryArc folder for existing issues and change status in StoryArc accordingly...
2013-11-28 15:48:59 +00:00
if futurepull is None :
2015-02-09 18:44:10 +00:00
og_status = None
2017-10-25 18:33:56 +00:00
if mylar . CONFIG . ALT_PULL != 2 or mylar . PULLBYFILE is True :
IMP: Added ALT_PULL=2 method for weeklypull management. Will now update against an external site and pull down already populated ComicID/IssueID's for the series that exist on the pullist. Alternate Search Names aren't needed with this option to match on pullist, as well as CV API Hits are also not used since it is all populated. Also allows for future viewing of pullists (up to 4 weeks in advance), FIX: Alternate search names now will be searched against when doing manual post-processing, FIX: When manually post-processing, if series volume wasn't specified would fail to match against v1 (by default), IMP:(#1309) Https_chain option now allowed within config.ini, IMP: 32P pack support on a per series basis (will search individual torrents first before packs), IMP: When pack is snatched, will mark all issues within pack that are not in a Downloaded status as Snatched within Mylar (annuals currently don't work), IMP: Removed unnecessary config spamming on startup when verbose mode was enabled, IMP: Allow for searching on 32p against series+publisher for some titles that distinguish between series by different publisher, IMP: Better series matching when trying to find series matches on 32P, FIX: When metatagging, if volume label is not provided within Mylar would default to None (now will be v1), IMP:(#1304) Attempt at better file parsing when utf-8 filenames are being parsed, FIX: Proper handling of Infinity issue number when file-checking, FIX: When adding series and annuals were enabled, if annual was a new release, would not be shown on the annual subset table for the given series (and subsequently wouldn't be auto-marked as Wanted), FIX:(#1306) Correct handling of the imported value when doing an import and moving files was selected (would previously error out during moving for some imports), FIX: When cbz files were being imported and were attempted to being auto-imported, would fail due to improper handling of the imported variable, FIX: Manage issues will now default the dropdown to the correct selected option, FIX: Manage Comics - fixed dropdown options for multiple selection of series - delete/pause/resume, IMP: Added 'delete stragglers' option to Story Arcs when deleting an arc to ensure that all traces of the arc are removed from the db, FIX: Manual/group metatagging would not tag properly if the START_YEAR_AS_VOLUME option was enabled, FIX: (#1313) NzbHydra wouldn't set the nzbid properly when using Failed Download handling/Retrying
2016-07-10 22:28:14 +00:00
logger . fdebug ( adjComicName + ' Issue: ' + str ( IssueNumber ) + ' not present in listings to mark for download...updating comic and adding to Upcoming Wanted Releases. ' )
# we need to either decrease the total issue count, OR indicate that an issue is upcoming.
upco_results = myDB . select ( " SELECT COUNT(*) FROM UPCOMING WHERE ComicID=? " , [ ComicID ] )
upco_iss = upco_results [ 0 ] [ 0 ]
#logger.info("upco_iss: " + str(upco_iss))
if int ( upco_iss ) > 0 :
#logger.info("There is " + str(upco_iss) + " of " + str(ComicName) + " that's not accounted for")
newKey = { " ComicID " : ComicID }
newVal = { " not_updated_db " : str ( upco_iss ) }
myDB . upsert ( " comics " , newVal , newKey )
elif int ( upco_iss ) < = 0 and lastupdatechk [ ' not_updated_db ' ] :
#if not_updated_db has a value, and upco_iss is > 0, let's zero it back out cause it's updated now.
newKey = { " ComicID " : ComicID }
newVal = { " not_updated_db " : " " }
myDB . upsert ( " comics " , newVal , newKey )
if hours > 5 or forcecheck == ' yes ' :
pullupd = " yes "
logger . fdebug ( ' Now Refreshing comic ' + ComicName + ' to make sure it is up-to-date ' )
if ComicID [ : 1 ] == " G " :
mylar . importer . GCDimport ( ComicID , pullupd )
else :
2017-09-20 23:51:00 +00:00
cchk = mylar . importer . updateissuedata ( ComicID , ComicName , calledfrom = ' weeklycheck ' ) #mylar.importer.addComictoDB(ComicID,mismatch,pullupd)
IMP: Added ALT_PULL=2 method for weeklypull management. Will now update against an external site and pull down already populated ComicID/IssueID's for the series that exist on the pullist. Alternate Search Names aren't needed with this option to match on pullist, as well as CV API Hits are also not used since it is all populated. Also allows for future viewing of pullists (up to 4 weeks in advance), FIX: Alternate search names now will be searched against when doing manual post-processing, FIX: When manually post-processing, if series volume wasn't specified would fail to match against v1 (by default), IMP:(#1309) Https_chain option now allowed within config.ini, IMP: 32P pack support on a per series basis (will search individual torrents first before packs), IMP: When pack is snatched, will mark all issues within pack that are not in a Downloaded status as Snatched within Mylar (annuals currently don't work), IMP: Removed unnecessary config spamming on startup when verbose mode was enabled, IMP: Allow for searching on 32p against series+publisher for some titles that distinguish between series by different publisher, IMP: Better series matching when trying to find series matches on 32P, FIX: When metatagging, if volume label is not provided within Mylar would default to None (now will be v1), IMP:(#1304) Attempt at better file parsing when utf-8 filenames are being parsed, FIX: Proper handling of Infinity issue number when file-checking, FIX: When adding series and annuals were enabled, if annual was a new release, would not be shown on the annual subset table for the given series (and subsequently wouldn't be auto-marked as Wanted), FIX:(#1306) Correct handling of the imported value when doing an import and moving files was selected (would previously error out during moving for some imports), FIX: When cbz files were being imported and were attempted to being auto-imported, would fail due to improper handling of the imported variable, FIX: Manage issues will now default the dropdown to the correct selected option, FIX: Manage Comics - fixed dropdown options for multiple selection of series - delete/pause/resume, IMP: Added 'delete stragglers' option to Story Arcs when deleting an arc to ensure that all traces of the arc are removed from the db, FIX: Manual/group metatagging would not tag properly if the START_YEAR_AS_VOLUME option was enabled, FIX: (#1313) NzbHydra wouldn't set the nzbid properly when using Failed Download handling/Retrying
2016-07-10 22:28:14 +00:00
else :
logger . fdebug ( ' It has not been longer than 5 hours since we last did this...we will wait so we do not hammer things. ' )
IMP: Added ForceRSS Check and Test SABnzbd Connection buttons in Config, FIX: If Annuals not enabled, would error on home screen, IMP: updated mylar.init.d (thnx Kalinon), FIX: Manual Post-Processing fix for Manual Run (thnx Kalinon), IMP: Library Monitor working (check folder every X minutes and Post-Process), IMP: Future Upcoming introduction, IMP: Experimental search better handling of year inclusions, FIX: Filechecker will now pick up series with years in the series title accordingly, FIX: Torrent seedbox sending would lockup occassionally when attempting to send torrent file, FIX: malformed image url on some series, IMP: Moved issue updating to a seperate function, IMP: When series was refreshed, would download the last issue (or few issues depending on date), regardless of status, IMP: When series is volume 1 or volume label doesn't exist, either assume V1 or remove volume requirements to improve matching hits, IMP: StoryArcs will now check in StoryArc folder for existing issues and change status in StoryArc accordingly...
2013-11-28 15:48:59 +00:00
else :
IMP: Added ALT_PULL=2 method for weeklypull management. Will now update against an external site and pull down already populated ComicID/IssueID's for the series that exist on the pullist. Alternate Search Names aren't needed with this option to match on pullist, as well as CV API Hits are also not used since it is all populated. Also allows for future viewing of pullists (up to 4 weeks in advance), FIX: Alternate search names now will be searched against when doing manual post-processing, FIX: When manually post-processing, if series volume wasn't specified would fail to match against v1 (by default), IMP:(#1309) Https_chain option now allowed within config.ini, IMP: 32P pack support on a per series basis (will search individual torrents first before packs), IMP: When pack is snatched, will mark all issues within pack that are not in a Downloaded status as Snatched within Mylar (annuals currently don't work), IMP: Removed unnecessary config spamming on startup when verbose mode was enabled, IMP: Allow for searching on 32p against series+publisher for some titles that distinguish between series by different publisher, IMP: Better series matching when trying to find series matches on 32P, FIX: When metatagging, if volume label is not provided within Mylar would default to None (now will be v1), IMP:(#1304) Attempt at better file parsing when utf-8 filenames are being parsed, FIX: Proper handling of Infinity issue number when file-checking, FIX: When adding series and annuals were enabled, if annual was a new release, would not be shown on the annual subset table for the given series (and subsequently wouldn't be auto-marked as Wanted), FIX:(#1306) Correct handling of the imported value when doing an import and moving files was selected (would previously error out during moving for some imports), FIX: When cbz files were being imported and were attempted to being auto-imported, would fail due to improper handling of the imported variable, FIX: Manage issues will now default the dropdown to the correct selected option, FIX: Manage Comics - fixed dropdown options for multiple selection of series - delete/pause/resume, IMP: Added 'delete stragglers' option to Story Arcs when deleting an arc to ensure that all traces of the arc are removed from the db, FIX: Manual/group metatagging would not tag properly if the START_YEAR_AS_VOLUME option was enabled, FIX: (#1313) NzbHydra wouldn't set the nzbid properly when using Failed Download handling/Retrying
2016-07-10 22:28:14 +00:00
logger . fdebug ( ' [WEEKLY-PULL] Walksoftly has been enabled. ComicID/IssueID control given to the ninja to monitor. ' )
IMP:New scheduler tab (Manage / Activity) where you can see job current run status', next runtime, and prev runtimes as well as force/pause a job, FIX: Disabling
torrents will now properly hide torrent information, IMP: Specified daemon port for deluge as an on-screen tip for more detail, IMP: Added 100,200,ALL as viewable
watchlist views, FIX: When viewing pullist and annual integration enabled, if annual was present would incorrectly link to invalid annual series instead of the
actual series itself, IMP: Added more detail error messages to metatagging errors and better handling of stranded files during cleanup, IMP: Improved some handling for weekly pull-list one-off's and refactored the
nzb/oneoff post-processing into a seperate function for future callables, Moved all the main url locations for public torrent sites to the init module so that it
can be cascaded down for use in other modules instead as a global, IMP: Added a 'deep_search_32p' variable in the config.ini for specific usage with 32p, where
if there is more than one result will dig deeper into each result to try and figure out if there are series matches, as opposed to the default where it will only use ref32p
table if available or just the first hit in a multiple series search results and ignore the remainder, FIX:Fixed some unknown characters appearing in the pullist due
to unicode-related conversion problems, FIX: fixed some special cases of file parsing errors due to Volume label being named different than expected, FIX: Added a
3s pause between experimental searches to try and not hit their frequency limitation, IMP: Weekly Pullist One-off's will now show status of Snatched/Downloaded as
required, FIX: Fixed some deluge parameter problems when using auto-snatch torrent script/option, IMP: Changed the downlocation in the auto-snatch option to an
env variable instead of being passed to avoid unicode-related problems, FIX: Fixed some magnet-related issues for torrents when using a watchdir + TPSE, FIX: Added
more verbose error message for rtorrent connection issues, FIX: Could not connect to rtorrent client if no username/password were provided, IMP: Set the db updater to run
every 5 minutes on the watchlist, automatically refreshing the oldest updated series each time that is more than 5 hours old (force db update from the activity/job schedulers page will run the db updater
against the entire watchlist in sequence), IMP: Attempt to handle long paths in windows (ie. > 256c) by prepending the unicode windows api character to the import
a directory path (windows only), IMP: When manual metatagging a series, will update the series after all the metatagging has been completed as opposed to after each
issue, IMP: Will now display available inkdrops on Config/Search Providers tab when using 32P (future will utilize/indicate inkdrop threshold when downloading)
2017-09-10 15:50:08 +00:00
logger . fdebug ( ' hours: ' + str ( hours ) + ' -- forcecheck: ' + str ( forcecheck ) )
IMP: Added ALT_PULL=2 method for weeklypull management. Will now update against an external site and pull down already populated ComicID/IssueID's for the series that exist on the pullist. Alternate Search Names aren't needed with this option to match on pullist, as well as CV API Hits are also not used since it is all populated. Also allows for future viewing of pullists (up to 4 weeks in advance), FIX: Alternate search names now will be searched against when doing manual post-processing, FIX: When manually post-processing, if series volume wasn't specified would fail to match against v1 (by default), IMP:(#1309) Https_chain option now allowed within config.ini, IMP: 32P pack support on a per series basis (will search individual torrents first before packs), IMP: When pack is snatched, will mark all issues within pack that are not in a Downloaded status as Snatched within Mylar (annuals currently don't work), IMP: Removed unnecessary config spamming on startup when verbose mode was enabled, IMP: Allow for searching on 32p against series+publisher for some titles that distinguish between series by different publisher, IMP: Better series matching when trying to find series matches on 32P, FIX: When metatagging, if volume label is not provided within Mylar would default to None (now will be v1), IMP:(#1304) Attempt at better file parsing when utf-8 filenames are being parsed, FIX: Proper handling of Infinity issue number when file-checking, FIX: When adding series and annuals were enabled, if annual was a new release, would not be shown on the annual subset table for the given series (and subsequently wouldn't be auto-marked as Wanted), FIX:(#1306) Correct handling of the imported value when doing an import and moving files was selected (would previously error out during moving for some imports), FIX: When cbz files were being imported and were attempted to being auto-imported, would fail due to improper handling of the imported variable, FIX: Manage issues will now default the dropdown to the correct selected option, FIX: Manage Comics - fixed dropdown options for multiple selection of series - delete/pause/resume, IMP: Added 'delete stragglers' option to Story Arcs when deleting an arc to ensure that all traces of the arc are removed from the db, FIX: Manual/group metatagging would not tag properly if the START_YEAR_AS_VOLUME option was enabled, FIX: (#1313) NzbHydra wouldn't set the nzbid properly when using Failed Download handling/Retrying
2016-07-10 22:28:14 +00:00
if hours > 2 or forcecheck == ' yes ' :
logger . fdebug ( ' weekinfo: ' + str ( weekinfo ) )
2018-03-18 02:25:24 +00:00
mylar . CONFIG . PULL_REFRESH = datetime . datetime . today ( ) . replace ( second = 0 , microsecond = 0 )
2016-08-20 21:56:30 +00:00
#update the PULL_REFRESH
2017-10-25 18:33:56 +00:00
#mylar.config_write()
IMP: Added ALT_PULL=2 method for weeklypull management. Will now update against an external site and pull down already populated ComicID/IssueID's for the series that exist on the pullist. Alternate Search Names aren't needed with this option to match on pullist, as well as CV API Hits are also not used since it is all populated. Also allows for future viewing of pullists (up to 4 weeks in advance), FIX: Alternate search names now will be searched against when doing manual post-processing, FIX: When manually post-processing, if series volume wasn't specified would fail to match against v1 (by default), IMP:(#1309) Https_chain option now allowed within config.ini, IMP: 32P pack support on a per series basis (will search individual torrents first before packs), IMP: When pack is snatched, will mark all issues within pack that are not in a Downloaded status as Snatched within Mylar (annuals currently don't work), IMP: Removed unnecessary config spamming on startup when verbose mode was enabled, IMP: Allow for searching on 32p against series+publisher for some titles that distinguish between series by different publisher, IMP: Better series matching when trying to find series matches on 32P, FIX: When metatagging, if volume label is not provided within Mylar would default to None (now will be v1), IMP:(#1304) Attempt at better file parsing when utf-8 filenames are being parsed, FIX: Proper handling of Infinity issue number when file-checking, FIX: When adding series and annuals were enabled, if annual was a new release, would not be shown on the annual subset table for the given series (and subsequently wouldn't be auto-marked as Wanted), FIX:(#1306) Correct handling of the imported value when doing an import and moving files was selected (would previously error out during moving for some imports), FIX: When cbz files were being imported and were attempted to being auto-imported, would fail due to improper handling of the imported variable, FIX: Manage issues will now default the dropdown to the correct selected option, FIX: Manage Comics - fixed dropdown options for multiple selection of series - delete/pause/resume, IMP: Added 'delete stragglers' option to Story Arcs when deleting an arc to ensure that all traces of the arc are removed from the db, FIX: Manual/group metatagging would not tag properly if the START_YEAR_AS_VOLUME option was enabled, FIX: (#1313) NzbHydra wouldn't set the nzbid properly when using Failed Download handling/Retrying
2016-07-10 22:28:14 +00:00
chkitout = mylar . locg . locg ( weeknumber = str ( weekinfo [ ' weeknumber ' ] ) , year = str ( weekinfo [ ' year ' ] ) )
IMP:New scheduler tab (Manage / Activity) where you can see job current run status', next runtime, and prev runtimes as well as force/pause a job, FIX: Disabling
torrents will now properly hide torrent information, IMP: Specified daemon port for deluge as an on-screen tip for more detail, IMP: Added 100,200,ALL as viewable
watchlist views, FIX: When viewing pullist and annual integration enabled, if annual was present would incorrectly link to invalid annual series instead of the
actual series itself, IMP: Added more detail error messages to metatagging errors and better handling of stranded files during cleanup, IMP: Improved some handling for weekly pull-list one-off's and refactored the
nzb/oneoff post-processing into a seperate function for future callables, Moved all the main url locations for public torrent sites to the init module so that it
can be cascaded down for use in other modules instead as a global, IMP: Added a 'deep_search_32p' variable in the config.ini for specific usage with 32p, where
if there is more than one result will dig deeper into each result to try and figure out if there are series matches, as opposed to the default where it will only use ref32p
table if available or just the first hit in a multiple series search results and ignore the remainder, FIX:Fixed some unknown characters appearing in the pullist due
to unicode-related conversion problems, FIX: fixed some special cases of file parsing errors due to Volume label being named different than expected, FIX: Added a
3s pause between experimental searches to try and not hit their frequency limitation, IMP: Weekly Pullist One-off's will now show status of Snatched/Downloaded as
required, FIX: Fixed some deluge parameter problems when using auto-snatch torrent script/option, IMP: Changed the downlocation in the auto-snatch option to an
env variable instead of being passed to avoid unicode-related problems, FIX: Fixed some magnet-related issues for torrents when using a watchdir + TPSE, FIX: Added
more verbose error message for rtorrent connection issues, FIX: Could not connect to rtorrent client if no username/password were provided, IMP: Set the db updater to run
every 5 minutes on the watchlist, automatically refreshing the oldest updated series each time that is more than 5 hours old (force db update from the activity/job schedulers page will run the db updater
against the entire watchlist in sequence), IMP: Attempt to handle long paths in windows (ie. > 256c) by prepending the unicode windows api character to the import
a directory path (windows only), IMP: When manual metatagging a series, will update the series after all the metatagging has been completed as opposed to after each
issue, IMP: Will now display available inkdrops on Config/Search Providers tab when using 32P (future will utilize/indicate inkdrop threshold when downloading)
2017-09-10 15:50:08 +00:00
IMP: Added ALT_PULL=2 method for weeklypull management. Will now update against an external site and pull down already populated ComicID/IssueID's for the series that exist on the pullist. Alternate Search Names aren't needed with this option to match on pullist, as well as CV API Hits are also not used since it is all populated. Also allows for future viewing of pullists (up to 4 weeks in advance), FIX: Alternate search names now will be searched against when doing manual post-processing, FIX: When manually post-processing, if series volume wasn't specified would fail to match against v1 (by default), IMP:(#1309) Https_chain option now allowed within config.ini, IMP: 32P pack support on a per series basis (will search individual torrents first before packs), IMP: When pack is snatched, will mark all issues within pack that are not in a Downloaded status as Snatched within Mylar (annuals currently don't work), IMP: Removed unnecessary config spamming on startup when verbose mode was enabled, IMP: Allow for searching on 32p against series+publisher for some titles that distinguish between series by different publisher, IMP: Better series matching when trying to find series matches on 32P, FIX: When metatagging, if volume label is not provided within Mylar would default to None (now will be v1), IMP:(#1304) Attempt at better file parsing when utf-8 filenames are being parsed, FIX: Proper handling of Infinity issue number when file-checking, FIX: When adding series and annuals were enabled, if annual was a new release, would not be shown on the annual subset table for the given series (and subsequently wouldn't be auto-marked as Wanted), FIX:(#1306) Correct handling of the imported value when doing an import and moving files was selected (would previously error out during moving for some imports), FIX: When cbz files were being imported and were attempted to being auto-imported, would fail due to improper handling of the imported variable, FIX: Manage issues will now default the dropdown to the correct selected option, FIX: Manage Comics - fixed dropdown options for multiple selection of series - delete/pause/resume, IMP: Added 'delete stragglers' option to Story Arcs when deleting an arc to ensure that all traces of the arc are removed from the db, FIX: Manual/group metatagging would not tag properly if the START_YEAR_AS_VOLUME option was enabled, FIX: (#1313) NzbHydra wouldn't set the nzbid properly when using Failed Download handling/Retrying
2016-07-10 22:28:14 +00:00
logger . fdebug ( ' linking ComicID to Pull-list to reflect status. ' )
downstats = { " ComicID " : ComicID ,
" IssueID " : None ,
" Status " : None }
return downstats
2013-01-07 19:26:10 +00:00
else :
IMP: Added ForceRSS Check and Test SABnzbd Connection buttons in Config, FIX: If Annuals not enabled, would error on home screen, IMP: updated mylar.init.d (thnx Kalinon), FIX: Manual Post-Processing fix for Manual Run (thnx Kalinon), IMP: Library Monitor working (check folder every X minutes and Post-Process), IMP: Future Upcoming introduction, IMP: Experimental search better handling of year inclusions, FIX: Filechecker will now pick up series with years in the series title accordingly, FIX: Torrent seedbox sending would lockup occassionally when attempting to send torrent file, FIX: malformed image url on some series, IMP: Moved issue updating to a seperate function, IMP: When series was refreshed, would download the last issue (or few issues depending on date), regardless of status, IMP: When series is volume 1 or volume label doesn't exist, either assume V1 or remove volume requirements to improve matching hits, IMP: StoryArcs will now check in StoryArc folder for existing issues and change status in StoryArc accordingly...
2013-11-28 15:48:59 +00:00
# if futurepull is not None, let's just update the status and ComicID
2014-04-14 17:42:42 +00:00
# NOTE: THIS IS CREATING EMPTY ENTRIES IN THE FUTURE TABLE. ???
IMP: Added ForceRSS Check and Test SABnzbd Connection buttons in Config, FIX: If Annuals not enabled, would error on home screen, IMP: updated mylar.init.d (thnx Kalinon), FIX: Manual Post-Processing fix for Manual Run (thnx Kalinon), IMP: Library Monitor working (check folder every X minutes and Post-Process), IMP: Future Upcoming introduction, IMP: Experimental search better handling of year inclusions, FIX: Filechecker will now pick up series with years in the series title accordingly, FIX: Torrent seedbox sending would lockup occassionally when attempting to send torrent file, FIX: malformed image url on some series, IMP: Moved issue updating to a seperate function, IMP: When series was refreshed, would download the last issue (or few issues depending on date), regardless of status, IMP: When series is volume 1 or volume label doesn't exist, either assume V1 or remove volume requirements to improve matching hits, IMP: StoryArcs will now check in StoryArc folder for existing issues and change status in StoryArc accordingly...
2013-11-28 15:48:59 +00:00
nKey = { " ComicID " : ComicID }
nVal = { " Status " : " Wanted " }
myDB . upsert ( " future " , nVal , nKey )
2015-02-09 18:44:10 +00:00
return
IMP: Removed unneeded code from several modules, IMP:(#659) Subdirectories now will be scanned in properly when in series directories, FIX:(#635)(#658) Moved code from html into module to allow for better integration and further templating needs, IMP: Added ability for mylar to auto-grab SABnzbd API when provided with user/pass, IMP: Test SABnzbd button works again - will auto-rollback to NZBKey usage, IMP: Added counts to Upcoming page, IMP: Added some more detailed parsing to the ImportResults, IMP: ImportResults will now show proper series title, IMP: Improved some logic when determining if a new weekly issue is on a watchlist if a rebooted series, IMP: A bunch of smaller fixes, and some various code tweaks...
2014-04-02 19:08:59 +00:00
if issuechk is not None :
if issuechk [ ' Issue_Number ' ] == IssueNumber or issuechk [ ' Issue_Number ' ] == altissuenumber :
2015-02-09 18:44:10 +00:00
og_status = issuechk [ ' Status ' ]
2014-08-18 19:54:34 +00:00
#check for 'out-of-whack' series here.
IMP:New scheduler tab (Manage / Activity) where you can see job current run status', next runtime, and prev runtimes as well as force/pause a job, FIX: Disabling
torrents will now properly hide torrent information, IMP: Specified daemon port for deluge as an on-screen tip for more detail, IMP: Added 100,200,ALL as viewable
watchlist views, FIX: When viewing pullist and annual integration enabled, if annual was present would incorrectly link to invalid annual series instead of the
actual series itself, IMP: Added more detail error messages to metatagging errors and better handling of stranded files during cleanup, IMP: Improved some handling for weekly pull-list one-off's and refactored the
nzb/oneoff post-processing into a seperate function for future callables, Moved all the main url locations for public torrent sites to the init module so that it
can be cascaded down for use in other modules instead as a global, IMP: Added a 'deep_search_32p' variable in the config.ini for specific usage with 32p, where
if there is more than one result will dig deeper into each result to try and figure out if there are series matches, as opposed to the default where it will only use ref32p
table if available or just the first hit in a multiple series search results and ignore the remainder, FIX:Fixed some unknown characters appearing in the pullist due
to unicode-related conversion problems, FIX: fixed some special cases of file parsing errors due to Volume label being named different than expected, FIX: Added a
3s pause between experimental searches to try and not hit their frequency limitation, IMP: Weekly Pullist One-off's will now show status of Snatched/Downloaded as
required, FIX: Fixed some deluge parameter problems when using auto-snatch torrent script/option, IMP: Changed the downlocation in the auto-snatch option to an
env variable instead of being passed to avoid unicode-related problems, FIX: Fixed some magnet-related issues for torrents when using a watchdir + TPSE, FIX: Added
more verbose error message for rtorrent connection issues, FIX: Could not connect to rtorrent client if no username/password were provided, IMP: Set the db updater to run
every 5 minutes on the watchlist, automatically refreshing the oldest updated series each time that is more than 5 hours old (force db update from the activity/job schedulers page will run the db updater
against the entire watchlist in sequence), IMP: Attempt to handle long paths in windows (ie. > 256c) by prepending the unicode windows api character to the import
a directory path (windows only), IMP: When manual metatagging a series, will update the series after all the metatagging has been completed as opposed to after each
issue, IMP: Will now display available inkdrops on Config/Search Providers tab when using 32P (future will utilize/indicate inkdrop threshold when downloading)
2017-09-10 15:50:08 +00:00
whackness = dbUpdate ( [ ComicID ] , calledfrom = ' weekly ' , sched = False )
2017-09-21 00:19:26 +00:00
if any ( [ whackness == True , og_status is None ] ) :
2015-05-22 08:32:51 +00:00
if any ( [ issuechk [ ' Status ' ] == ' Downloaded ' , issuechk [ ' Status ' ] == ' Archived ' , issuechk [ ' Status ' ] == ' Snatched ' ] ) :
2015-02-09 18:44:10 +00:00
logger . fdebug ( ' Forcibly maintaining status of : ' + og_status + ' for # ' + issuechk [ ' Issue_Number ' ] + ' to ensure integrity. ' )
2014-08-18 19:54:34 +00:00
logger . fdebug ( ' Comic series has an incorrect total count. Forcily refreshing series to ensure data is current. ' )
dbUpdate ( [ ComicID ] )
issuechk = myDB . selectone ( " SELECT * FROM issues WHERE ComicID=? AND Int_IssueNumber=? " , [ ComicID , helpers . issuedigits ( IssueNumber ) ] ) . fetchone ( )
2015-02-09 18:44:10 +00:00
if issuechk [ ' Status ' ] != og_status and ( issuechk [ ' Status ' ] != ' Downloaded ' or issuechk [ ' Status ' ] != ' Archived ' or issuechk [ ' Status ' ] != ' Snatched ' ) :
2017-09-21 00:32:53 +00:00
logger . fdebug ( ' Forcibly changing status of %s back to %s for # %s to stop repeated downloads. ' % ( issuechk [ ' Status ' ] , og_status , issuechk [ ' Issue_Number ' ] ) )
2015-02-09 18:44:10 +00:00
else :
2017-09-21 00:32:53 +00:00
logger . fdebug ( ' [ %s ] / [ %s ] Status has not changed during refresh or is marked as being Wanted/Skipped correctly. ' % ( issuechk [ ' Status ' ] , og_status ) )
2015-02-09 18:44:10 +00:00
og_status = issuechk [ ' Status ' ]
2014-08-18 19:54:34 +00:00
else :
logger . fdebug ( ' Comic series already up-to-date ... no need to refresh at this time. ' )
IMP: Removed unneeded code from several modules, IMP:(#659) Subdirectories now will be scanned in properly when in series directories, FIX:(#635)(#658) Moved code from html into module to allow for better integration and further templating needs, IMP: Added ability for mylar to auto-grab SABnzbd API when provided with user/pass, IMP: Test SABnzbd button works again - will auto-rollback to NZBKey usage, IMP: Added counts to Upcoming page, IMP: Added some more detailed parsing to the ImportResults, IMP: ImportResults will now show proper series title, IMP: Improved some logic when determining if a new weekly issue is on a watchlist if a rebooted series, IMP: A bunch of smaller fixes, and some various code tweaks...
2014-04-02 19:08:59 +00:00
logger . fdebug ( ' Available to be marked for download - checking... ' + adjComicName + ' Issue: ' + str ( issuechk [ ' Issue_Number ' ] ) )
2015-02-09 18:44:10 +00:00
logger . fdebug ( ' ...Existing status: ' + og_status )
IMP: Removed unneeded code from several modules, IMP:(#659) Subdirectories now will be scanned in properly when in series directories, FIX:(#635)(#658) Moved code from html into module to allow for better integration and further templating needs, IMP: Added ability for mylar to auto-grab SABnzbd API when provided with user/pass, IMP: Test SABnzbd button works again - will auto-rollback to NZBKey usage, IMP: Added counts to Upcoming page, IMP: Added some more detailed parsing to the ImportResults, IMP: ImportResults will now show proper series title, IMP: Improved some logic when determining if a new weekly issue is on a watchlist if a rebooted series, IMP: A bunch of smaller fixes, and some various code tweaks...
2014-04-02 19:08:59 +00:00
control = { " IssueID " : issuechk [ ' IssueID ' ] }
newValue [ ' IssueID ' ] = issuechk [ ' IssueID ' ]
2015-02-09 18:44:10 +00:00
if og_status == " Snatched " :
2015-05-22 08:32:51 +00:00
values = { " Status " : " Snatched " }
IMP: Removed unneeded code from several modules, IMP:(#659) Subdirectories now will be scanned in properly when in series directories, FIX:(#635)(#658) Moved code from html into module to allow for better integration and further templating needs, IMP: Added ability for mylar to auto-grab SABnzbd API when provided with user/pass, IMP: Test SABnzbd button works again - will auto-rollback to NZBKey usage, IMP: Added counts to Upcoming page, IMP: Added some more detailed parsing to the ImportResults, IMP: ImportResults will now show proper series title, IMP: Improved some logic when determining if a new weekly issue is on a watchlist if a rebooted series, IMP: A bunch of smaller fixes, and some various code tweaks...
2014-04-02 19:08:59 +00:00
newValue [ ' Status ' ] = " Snatched "
2015-02-09 18:44:10 +00:00
elif og_status == " Downloaded " :
2015-05-22 08:32:51 +00:00
values = { " Status " : " Downloaded " }
IMP: Removed unneeded code from several modules, IMP:(#659) Subdirectories now will be scanned in properly when in series directories, FIX:(#635)(#658) Moved code from html into module to allow for better integration and further templating needs, IMP: Added ability for mylar to auto-grab SABnzbd API when provided with user/pass, IMP: Test SABnzbd button works again - will auto-rollback to NZBKey usage, IMP: Added counts to Upcoming page, IMP: Added some more detailed parsing to the ImportResults, IMP: ImportResults will now show proper series title, IMP: Improved some logic when determining if a new weekly issue is on a watchlist if a rebooted series, IMP: A bunch of smaller fixes, and some various code tweaks...
2014-04-02 19:08:59 +00:00
newValue [ ' Status ' ] = " Downloaded "
#if the status is Downloaded and it's on the pullist - let's mark it so everyone can bask in the glory
2015-02-09 18:44:10 +00:00
elif og_status == " Wanted " :
2015-05-22 08:32:51 +00:00
values = { " Status " : " Wanted " }
newValue [ ' Status ' ] = " Wanted "
2015-02-09 18:44:10 +00:00
elif og_status == " Archived " :
2015-05-22 08:32:51 +00:00
values = { " Status " : " Archived " }
IMP: Removed unneeded code from several modules, IMP:(#659) Subdirectories now will be scanned in properly when in series directories, FIX:(#635)(#658) Moved code from html into module to allow for better integration and further templating needs, IMP: Added ability for mylar to auto-grab SABnzbd API when provided with user/pass, IMP: Test SABnzbd button works again - will auto-rollback to NZBKey usage, IMP: Added counts to Upcoming page, IMP: Added some more detailed parsing to the ImportResults, IMP: ImportResults will now show proper series title, IMP: Improved some logic when determining if a new weekly issue is on a watchlist if a rebooted series, IMP: A bunch of smaller fixes, and some various code tweaks...
2014-04-02 19:08:59 +00:00
newValue [ ' Status ' ] = " Archived "
2015-02-09 18:44:10 +00:00
elif og_status == ' Failed ' :
2017-10-25 18:33:56 +00:00
if mylar . CONFIG . FAILED_DOWNLOAD_HANDLING :
if mylar . CONFIG . FAILED_AUTO :
2015-05-22 08:32:51 +00:00
values = { " Status " : " Wanted " }
IMP: 32P legacy mode (rss feed only) and Auth mode (user/pass) available now. With Auth mode, will monitor all of personal notification feeds, IMP: Changed access methods to 32P for retrieving links, FIX:(#1007) Fixed error for one-off downloading via weekly pull list, FIX: (#1004) Fix for post-processing errors (missing files), FIX: (#1001) Extra scripts & pre-scripts will now accept bash scripts, FIX: Annuals were being incorrectly tallied in Have total as well as not having status correctly updated, IMP: Fixed some discrepencies with the provider_order generation line, IMP: Made some additional adjustments to the file-checking to account for some additional variations in naming, FIX: Fixed traceback errors when folder monitor was running and wouldn't be able to complete the post-processing properly, FIX: Cleanup code and some other small fixes....
2015-05-09 02:39:28 +00:00
newValue [ ' Status ' ] = " Wanted "
2015-02-09 18:44:10 +00:00
else :
2015-05-22 08:32:51 +00:00
values = { " Status " : " Failed " }
IMP: 32P legacy mode (rss feed only) and Auth mode (user/pass) available now. With Auth mode, will monitor all of personal notification feeds, IMP: Changed access methods to 32P for retrieving links, FIX:(#1007) Fixed error for one-off downloading via weekly pull list, FIX: (#1004) Fix for post-processing errors (missing files), FIX: (#1001) Extra scripts & pre-scripts will now accept bash scripts, FIX: Annuals were being incorrectly tallied in Have total as well as not having status correctly updated, IMP: Fixed some discrepencies with the provider_order generation line, IMP: Made some additional adjustments to the file-checking to account for some additional variations in naming, FIX: Fixed traceback errors when folder monitor was running and wouldn't be able to complete the post-processing properly, FIX: Cleanup code and some other small fixes....
2015-05-09 02:39:28 +00:00
newValue [ ' Status ' ] = " Failed "
2015-02-09 18:44:10 +00:00
else :
2015-05-22 08:32:51 +00:00
values = { " Status " : " Skipped " }
IMP: 32P legacy mode (rss feed only) and Auth mode (user/pass) available now. With Auth mode, will monitor all of personal notification feeds, IMP: Changed access methods to 32P for retrieving links, FIX:(#1007) Fixed error for one-off downloading via weekly pull list, FIX: (#1004) Fix for post-processing errors (missing files), FIX: (#1001) Extra scripts & pre-scripts will now accept bash scripts, FIX: Annuals were being incorrectly tallied in Have total as well as not having status correctly updated, IMP: Fixed some discrepencies with the provider_order generation line, IMP: Made some additional adjustments to the file-checking to account for some additional variations in naming, FIX: Fixed traceback errors when folder monitor was running and wouldn't be able to complete the post-processing properly, FIX: Cleanup code and some other small fixes....
2015-05-09 02:39:28 +00:00
newValue [ ' Status ' ] = " Skipped "
IMP: Removed unneeded code from several modules, IMP:(#659) Subdirectories now will be scanned in properly when in series directories, FIX:(#635)(#658) Moved code from html into module to allow for better integration and further templating needs, IMP: Added ability for mylar to auto-grab SABnzbd API when provided with user/pass, IMP: Test SABnzbd button works again - will auto-rollback to NZBKey usage, IMP: Added counts to Upcoming page, IMP: Added some more detailed parsing to the ImportResults, IMP: ImportResults will now show proper series title, IMP: Improved some logic when determining if a new weekly issue is on a watchlist if a rebooted series, IMP: A bunch of smaller fixes, and some various code tweaks...
2014-04-02 19:08:59 +00:00
else :
2015-05-22 08:32:51 +00:00
values = { " Status " : " Skipped " }
IMP: Removed unneeded code from several modules, IMP:(#659) Subdirectories now will be scanned in properly when in series directories, FIX:(#635)(#658) Moved code from html into module to allow for better integration and further templating needs, IMP: Added ability for mylar to auto-grab SABnzbd API when provided with user/pass, IMP: Test SABnzbd button works again - will auto-rollback to NZBKey usage, IMP: Added counts to Upcoming page, IMP: Added some more detailed parsing to the ImportResults, IMP: ImportResults will now show proper series title, IMP: Improved some logic when determining if a new weekly issue is on a watchlist if a rebooted series, IMP: A bunch of smaller fixes, and some various code tweaks...
2014-04-02 19:08:59 +00:00
newValue [ ' Status ' ] = " Skipped "
#was in wrong place :(
2012-10-21 15:30:26 +00:00
else :
IMP:New scheduler tab (Manage / Activity) where you can see job current run status', next runtime, and prev runtimes as well as force/pause a job, FIX: Disabling
torrents will now properly hide torrent information, IMP: Specified daemon port for deluge as an on-screen tip for more detail, IMP: Added 100,200,ALL as viewable
watchlist views, FIX: When viewing pullist and annual integration enabled, if annual was present would incorrectly link to invalid annual series instead of the
actual series itself, IMP: Added more detail error messages to metatagging errors and better handling of stranded files during cleanup, IMP: Improved some handling for weekly pull-list one-off's and refactored the
nzb/oneoff post-processing into a seperate function for future callables, Moved all the main url locations for public torrent sites to the init module so that it
can be cascaded down for use in other modules instead as a global, IMP: Added a 'deep_search_32p' variable in the config.ini for specific usage with 32p, where
if there is more than one result will dig deeper into each result to try and figure out if there are series matches, as opposed to the default where it will only use ref32p
table if available or just the first hit in a multiple series search results and ignore the remainder, FIX:Fixed some unknown characters appearing in the pullist due
to unicode-related conversion problems, FIX: fixed some special cases of file parsing errors due to Volume label being named different than expected, FIX: Added a
3s pause between experimental searches to try and not hit their frequency limitation, IMP: Weekly Pullist One-off's will now show status of Snatched/Downloaded as
required, FIX: Fixed some deluge parameter problems when using auto-snatch torrent script/option, IMP: Changed the downlocation in the auto-snatch option to an
env variable instead of being passed to avoid unicode-related problems, FIX: Fixed some magnet-related issues for torrents when using a watchdir + TPSE, FIX: Added
more verbose error message for rtorrent connection issues, FIX: Could not connect to rtorrent client if no username/password were provided, IMP: Set the db updater to run
every 5 minutes on the watchlist, automatically refreshing the oldest updated series each time that is more than 5 hours old (force db update from the activity/job schedulers page will run the db updater
against the entire watchlist in sequence), IMP: Attempt to handle long paths in windows (ie. > 256c) by prepending the unicode windows api character to the import
a directory path (windows only), IMP: When manual metatagging a series, will update the series after all the metatagging has been completed as opposed to after each
issue, IMP: Will now display available inkdrops on Config/Search Providers tab when using 32P (future will utilize/indicate inkdrop threshold when downloading)
2017-09-10 15:50:08 +00:00
logger . fdebug ( ' Issues do not match for some reason...weekly new issue: %s ' % IssueNumber )
IMP: Removed unneeded code from several modules, IMP:(#659) Subdirectories now will be scanned in properly when in series directories, FIX:(#635)(#658) Moved code from html into module to allow for better integration and further templating needs, IMP: Added ability for mylar to auto-grab SABnzbd API when provided with user/pass, IMP: Test SABnzbd button works again - will auto-rollback to NZBKey usage, IMP: Added counts to Upcoming page, IMP: Added some more detailed parsing to the ImportResults, IMP: ImportResults will now show proper series title, IMP: Improved some logic when determining if a new weekly issue is on a watchlist if a rebooted series, IMP: A bunch of smaller fixes, and some various code tweaks...
2014-04-02 19:08:59 +00:00
return
2013-09-05 02:36:41 +00:00
2017-10-25 18:33:56 +00:00
if mylar . CONFIG . AUTOWANT_UPCOMING :
2013-01-07 18:26:15 +00:00
#for issues not in db - to be added to Upcoming table.
2015-02-09 18:44:10 +00:00
if og_status is None :
2013-01-07 18:26:15 +00:00
newValue [ ' Status ' ] = " Wanted "
2013-08-19 06:14:47 +00:00
logger . fdebug ( ' ...Changing Status to Wanted and throwing it in the Upcoming section since it is not published yet. ' )
2015-05-22 08:32:51 +00:00
#this works for issues existing in DB...
2015-02-09 18:44:10 +00:00
elif og_status == " Skipped " :
2013-01-07 18:26:15 +00:00
newValue [ ' Status ' ] = " Wanted "
2013-07-01 05:19:15 +00:00
values = { " Status " : " Wanted " }
2013-08-19 06:14:47 +00:00
logger . fdebug ( ' ...New status of Wanted ' )
2015-02-09 18:44:10 +00:00
elif og_status == " Wanted " :
2013-08-19 06:14:47 +00:00
logger . fdebug ( ' ...Status already Wanted .. not changing. ' )
2013-01-07 18:26:15 +00:00
else :
2015-02-09 18:44:10 +00:00
logger . fdebug ( ' ...Already have issue - keeping existing status of : ' + og_status )
2013-09-05 02:36:41 +00:00
if issuechk is None :
myDB . upsert ( " upcoming " , newValue , controlValue )
2015-04-28 02:15:53 +00:00
logger . fdebug ( ' updating Pull-list to reflect status. ' )
downstats = { " Status " : newValue [ ' Status ' ] ,
" ComicID " : ComicID ,
" IssueID " : None }
return downstats
2013-01-03 10:26:39 +00:00
else :
2013-09-05 02:36:41 +00:00
logger . fdebug ( ' --attempt to find errant adds to Wanted list ' )
logger . fdebug ( ' UpcomingNewValue: ' + str ( newValue ) )
logger . fdebug ( ' UpcomingcontrolValue: ' + str ( controlValue ) )
if issuechk [ ' IssueDate ' ] == ' 0000-00-00 ' and newValue [ ' IssueDate ' ] != ' 0000-00-00 ' :
logger . fdebug ( ' Found a 0000-00-00 issue - force updating series to try and get it proper. ' )
dateVal = { " IssueDate " : newValue [ ' IssueDate ' ] ,
" ComicName " : issuechk [ ' ComicName ' ] ,
" Status " : newValue [ ' Status ' ] ,
" IssueNumber " : issuechk [ ' Issue_Number ' ] }
2015-05-21 15:50:15 +00:00
logger . fdebug ( ' updating date in upcoming table to : ' + str ( newValue [ ' IssueDate ' ] ) + ' [ ' + newValue [ ' Status ' ] + ' ] ' )
2013-09-05 02:36:41 +00:00
logger . fdebug ( ' ComicID: ' + str ( controlValue ) )
myDB . upsert ( " upcoming " , dateVal , controlValue )
logger . fdebug ( ' Temporarily putting the Issue Date for ' + str ( issuechk [ ' Issue_Number ' ] ) + ' to ' + str ( newValue [ ' IssueDate ' ] ) )
values = { " IssueDate " : newValue [ ' IssueDate ' ] }
#if ComicID[:1] == "G": mylar.importer.GCDimport(ComicID,pullupd='yes')
#else: mylar.importer.addComictoDB(ComicID,mismatch,pullupd='yes')
FIX: (#2006)(#1986) Auto-Want series were not being auto-added, and even after manually added would result in count error after sucessful post-processing, FIX: When filechecking, if a 'special' was included as part of the annuals of a given series, it would not be detected, IMP: (#1982) When ComicRN & Completed Download Handling (CDH) are both enabled, will now check to see if ComicRN is active for the given category in the respective client. If it is, CDH will not post-process and issue a warning and let ComicRN handle the post-processing, FIX: Fixed a utf-8 encoding error when 'Run script AFTER snatch' was enabled, FIX: (#2004) Possible fix for 500 error on Wanted tab, when series had an annual/special attached but was located in the incorrect table, FIX: Removed some logging lines when CDH is monitoring downloads
2018-06-30 18:35:52 +00:00
if any ( [ ' annual ' in ComicName . lower ( ) , ' special ' in ComicName . lower ( ) ] ) :
2013-09-05 02:36:41 +00:00
myDB . upsert ( " annuals " , values , control )
else :
myDB . upsert ( " issues " , values , control )
2015-05-22 08:32:51 +00:00
if any ( [ og_status == ' Downloaded ' , og_status == ' Archived ' , og_status == ' Snatched ' , og_status == ' Wanted ' , newValue [ ' Status ' ] == ' Wanted ' ] ) :
2015-05-21 15:50:15 +00:00
logger . fdebug ( ' updating Pull-list to reflect status change: ' + og_status + ' [ ' + newValue [ ' Status ' ] + ' ] ' )
2015-05-22 08:32:51 +00:00
if og_status != ' Skipped ' :
2015-05-21 15:50:15 +00:00
downstats = { " Status " : og_status ,
" ComicID " : issuechk [ ' ComicID ' ] ,
" IssueID " : issuechk [ ' IssueID ' ] }
else :
downstats = { " Status " : newValue [ ' Status ' ] ,
" ComicID " : issuechk [ ' ComicID ' ] ,
" IssueID " : issuechk [ ' IssueID ' ] }
2013-09-05 02:36:41 +00:00
return downstats
2012-10-16 08:16:29 +00:00
2012-09-13 15:27:34 +00:00
IMP: Added ALT_PULL=2 method for weeklypull management. Will now update against an external site and pull down already populated ComicID/IssueID's for the series that exist on the pullist. Alternate Search Names aren't needed with this option to match on pullist, as well as CV API Hits are also not used since it is all populated. Also allows for future viewing of pullists (up to 4 weeks in advance), FIX: Alternate search names now will be searched against when doing manual post-processing, FIX: When manually post-processing, if series volume wasn't specified would fail to match against v1 (by default), IMP:(#1309) Https_chain option now allowed within config.ini, IMP: 32P pack support on a per series basis (will search individual torrents first before packs), IMP: When pack is snatched, will mark all issues within pack that are not in a Downloaded status as Snatched within Mylar (annuals currently don't work), IMP: Removed unnecessary config spamming on startup when verbose mode was enabled, IMP: Allow for searching on 32p against series+publisher for some titles that distinguish between series by different publisher, IMP: Better series matching when trying to find series matches on 32P, FIX: When metatagging, if volume label is not provided within Mylar would default to None (now will be v1), IMP:(#1304) Attempt at better file parsing when utf-8 filenames are being parsed, FIX: Proper handling of Infinity issue number when file-checking, FIX: When adding series and annuals were enabled, if annual was a new release, would not be shown on the annual subset table for the given series (and subsequently wouldn't be auto-marked as Wanted), FIX:(#1306) Correct handling of the imported value when doing an import and moving files was selected (would previously error out during moving for some imports), FIX: When cbz files were being imported and were attempted to being auto-imported, would fail due to improper handling of the imported variable, FIX: Manage issues will now default the dropdown to the correct selected option, FIX: Manage Comics - fixed dropdown options for multiple selection of series - delete/pause/resume, IMP: Added 'delete stragglers' option to Story Arcs when deleting an arc to ensure that all traces of the arc are removed from the db, FIX: Manual/group metatagging would not tag properly if the START_YEAR_AS_VOLUME option was enabled, FIX: (#1313) NzbHydra wouldn't set the nzbid properly when using Failed Download handling/Retrying
2016-07-10 22:28:14 +00:00
def weekly_update ( ComicName , IssueNumber , CStatus , CID , weeknumber , year , altissuenumber = None ) :
logger . fdebug ( ' Weekly Update for week ' + str ( weeknumber ) + ' - ' + str ( year ) + ' : ' + str ( ComicName ) + ' # ' + str ( IssueNumber ) + ' to a status of ' + str ( CStatus ) )
IMP: Removed unneeded code from several modules, IMP:(#659) Subdirectories now will be scanned in properly when in series directories, FIX:(#635)(#658) Moved code from html into module to allow for better integration and further templating needs, IMP: Added ability for mylar to auto-grab SABnzbd API when provided with user/pass, IMP: Test SABnzbd button works again - will auto-rollback to NZBKey usage, IMP: Added counts to Upcoming page, IMP: Added some more detailed parsing to the ImportResults, IMP: ImportResults will now show proper series title, IMP: Improved some logic when determining if a new weekly issue is on a watchlist if a rebooted series, IMP: A bunch of smaller fixes, and some various code tweaks...
2014-04-02 19:08:59 +00:00
if altissuenumber :
logger . fdebug ( ' weekly_update of table : ' + str ( ComicName ) + ' (Alternate Issue #): ' + str ( altissuenumber ) + ' to a status of ' + str ( CStatus ) )
2012-09-13 15:27:34 +00:00
# here we update status of weekly table...
2013-04-26 03:45:02 +00:00
# added Issue to stop false hits on series' that have multiple releases in a week
# added CStatus to update status flags on Pullist screen
2012-09-13 15:27:34 +00:00
myDB = db . DBConnection ( )
2017-01-06 18:06:58 +00:00
issuecheck = myDB . selectone ( " SELECT * FROM weekly WHERE COMIC=? AND ISSUE=? and WEEKNUMBER=? AND YEAR=? " , [ ComicName , IssueNumber , int ( weeknumber ) , year ] ) . fetchone ( )
IMP: Added ALT_PULL=2 method for weeklypull management. Will now update against an external site and pull down already populated ComicID/IssueID's for the series that exist on the pullist. Alternate Search Names aren't needed with this option to match on pullist, as well as CV API Hits are also not used since it is all populated. Also allows for future viewing of pullists (up to 4 weeks in advance), FIX: Alternate search names now will be searched against when doing manual post-processing, FIX: When manually post-processing, if series volume wasn't specified would fail to match against v1 (by default), IMP:(#1309) Https_chain option now allowed within config.ini, IMP: 32P pack support on a per series basis (will search individual torrents first before packs), IMP: When pack is snatched, will mark all issues within pack that are not in a Downloaded status as Snatched within Mylar (annuals currently don't work), IMP: Removed unnecessary config spamming on startup when verbose mode was enabled, IMP: Allow for searching on 32p against series+publisher for some titles that distinguish between series by different publisher, IMP: Better series matching when trying to find series matches on 32P, FIX: When metatagging, if volume label is not provided within Mylar would default to None (now will be v1), IMP:(#1304) Attempt at better file parsing when utf-8 filenames are being parsed, FIX: Proper handling of Infinity issue number when file-checking, FIX: When adding series and annuals were enabled, if annual was a new release, would not be shown on the annual subset table for the given series (and subsequently wouldn't be auto-marked as Wanted), FIX:(#1306) Correct handling of the imported value when doing an import and moving files was selected (would previously error out during moving for some imports), FIX: When cbz files were being imported and were attempted to being auto-imported, would fail due to improper handling of the imported variable, FIX: Manage issues will now default the dropdown to the correct selected option, FIX: Manage Comics - fixed dropdown options for multiple selection of series - delete/pause/resume, IMP: Added 'delete stragglers' option to Story Arcs when deleting an arc to ensure that all traces of the arc are removed from the db, FIX: Manual/group metatagging would not tag properly if the START_YEAR_AS_VOLUME option was enabled, FIX: (#1313) NzbHydra wouldn't set the nzbid properly when using Failed Download handling/Retrying
2016-07-10 22:28:14 +00:00
2013-04-26 03:45:02 +00:00
if issuecheck is not None :
2015-05-22 08:32:51 +00:00
controlValue = { " COMIC " : str ( ComicName ) ,
IMP: Added ALT_PULL=2 method for weeklypull management. Will now update against an external site and pull down already populated ComicID/IssueID's for the series that exist on the pullist. Alternate Search Names aren't needed with this option to match on pullist, as well as CV API Hits are also not used since it is all populated. Also allows for future viewing of pullists (up to 4 weeks in advance), FIX: Alternate search names now will be searched against when doing manual post-processing, FIX: When manually post-processing, if series volume wasn't specified would fail to match against v1 (by default), IMP:(#1309) Https_chain option now allowed within config.ini, IMP: 32P pack support on a per series basis (will search individual torrents first before packs), IMP: When pack is snatched, will mark all issues within pack that are not in a Downloaded status as Snatched within Mylar (annuals currently don't work), IMP: Removed unnecessary config spamming on startup when verbose mode was enabled, IMP: Allow for searching on 32p against series+publisher for some titles that distinguish between series by different publisher, IMP: Better series matching when trying to find series matches on 32P, FIX: When metatagging, if volume label is not provided within Mylar would default to None (now will be v1), IMP:(#1304) Attempt at better file parsing when utf-8 filenames are being parsed, FIX: Proper handling of Infinity issue number when file-checking, FIX: When adding series and annuals were enabled, if annual was a new release, would not be shown on the annual subset table for the given series (and subsequently wouldn't be auto-marked as Wanted), FIX:(#1306) Correct handling of the imported value when doing an import and moving files was selected (would previously error out during moving for some imports), FIX: When cbz files were being imported and were attempted to being auto-imported, would fail due to improper handling of the imported variable, FIX: Manage issues will now default the dropdown to the correct selected option, FIX: Manage Comics - fixed dropdown options for multiple selection of series - delete/pause/resume, IMP: Added 'delete stragglers' option to Story Arcs when deleting an arc to ensure that all traces of the arc are removed from the db, FIX: Manual/group metatagging would not tag properly if the START_YEAR_AS_VOLUME option was enabled, FIX: (#1313) NzbHydra wouldn't set the nzbid properly when using Failed Download handling/Retrying
2016-07-10 22:28:14 +00:00
" ISSUE " : str ( IssueNumber ) ,
2017-01-06 18:06:58 +00:00
" WEEKNUMBER " : int ( weeknumber ) ,
IMP: Added ALT_PULL=2 method for weeklypull management. Will now update against an external site and pull down already populated ComicID/IssueID's for the series that exist on the pullist. Alternate Search Names aren't needed with this option to match on pullist, as well as CV API Hits are also not used since it is all populated. Also allows for future viewing of pullists (up to 4 weeks in advance), FIX: Alternate search names now will be searched against when doing manual post-processing, FIX: When manually post-processing, if series volume wasn't specified would fail to match against v1 (by default), IMP:(#1309) Https_chain option now allowed within config.ini, IMP: 32P pack support on a per series basis (will search individual torrents first before packs), IMP: When pack is snatched, will mark all issues within pack that are not in a Downloaded status as Snatched within Mylar (annuals currently don't work), IMP: Removed unnecessary config spamming on startup when verbose mode was enabled, IMP: Allow for searching on 32p against series+publisher for some titles that distinguish between series by different publisher, IMP: Better series matching when trying to find series matches on 32P, FIX: When metatagging, if volume label is not provided within Mylar would default to None (now will be v1), IMP:(#1304) Attempt at better file parsing when utf-8 filenames are being parsed, FIX: Proper handling of Infinity issue number when file-checking, FIX: When adding series and annuals were enabled, if annual was a new release, would not be shown on the annual subset table for the given series (and subsequently wouldn't be auto-marked as Wanted), FIX:(#1306) Correct handling of the imported value when doing an import and moving files was selected (would previously error out during moving for some imports), FIX: When cbz files were being imported and were attempted to being auto-imported, would fail due to improper handling of the imported variable, FIX: Manage issues will now default the dropdown to the correct selected option, FIX: Manage Comics - fixed dropdown options for multiple selection of series - delete/pause/resume, IMP: Added 'delete stragglers' option to Story Arcs when deleting an arc to ensure that all traces of the arc are removed from the db, FIX: Manual/group metatagging would not tag properly if the START_YEAR_AS_VOLUME option was enabled, FIX: (#1313) NzbHydra wouldn't set the nzbid properly when using Failed Download handling/Retrying
2016-07-10 22:28:14 +00:00
" YEAR " : year }
2015-04-28 02:15:53 +00:00
IMP: Added ALT_PULL=2 method for weeklypull management. Will now update against an external site and pull down already populated ComicID/IssueID's for the series that exist on the pullist. Alternate Search Names aren't needed with this option to match on pullist, as well as CV API Hits are also not used since it is all populated. Also allows for future viewing of pullists (up to 4 weeks in advance), FIX: Alternate search names now will be searched against when doing manual post-processing, FIX: When manually post-processing, if series volume wasn't specified would fail to match against v1 (by default), IMP:(#1309) Https_chain option now allowed within config.ini, IMP: 32P pack support on a per series basis (will search individual torrents first before packs), IMP: When pack is snatched, will mark all issues within pack that are not in a Downloaded status as Snatched within Mylar (annuals currently don't work), IMP: Removed unnecessary config spamming on startup when verbose mode was enabled, IMP: Allow for searching on 32p against series+publisher for some titles that distinguish between series by different publisher, IMP: Better series matching when trying to find series matches on 32P, FIX: When metatagging, if volume label is not provided within Mylar would default to None (now will be v1), IMP:(#1304) Attempt at better file parsing when utf-8 filenames are being parsed, FIX: Proper handling of Infinity issue number when file-checking, FIX: When adding series and annuals were enabled, if annual was a new release, would not be shown on the annual subset table for the given series (and subsequently wouldn't be auto-marked as Wanted), FIX:(#1306) Correct handling of the imported value when doing an import and moving files was selected (would previously error out during moving for some imports), FIX: When cbz files were being imported and were attempted to being auto-imported, would fail due to improper handling of the imported variable, FIX: Manage issues will now default the dropdown to the correct selected option, FIX: Manage Comics - fixed dropdown options for multiple selection of series - delete/pause/resume, IMP: Added 'delete stragglers' option to Story Arcs when deleting an arc to ensure that all traces of the arc are removed from the db, FIX: Manual/group metatagging would not tag properly if the START_YEAR_AS_VOLUME option was enabled, FIX: (#1313) NzbHydra wouldn't set the nzbid properly when using Failed Download handling/Retrying
2016-07-10 22:28:14 +00:00
logger . info ( ' controlValue: ' + str ( controlValue ) )
2015-04-28 02:15:53 +00:00
try :
if CID [ ' IssueID ' ] :
cidissueid = CID [ ' IssueID ' ]
else :
cidissueid = None
except :
cidissueid = None
IMP: Added ALT_PULL=2 method for weeklypull management. Will now update against an external site and pull down already populated ComicID/IssueID's for the series that exist on the pullist. Alternate Search Names aren't needed with this option to match on pullist, as well as CV API Hits are also not used since it is all populated. Also allows for future viewing of pullists (up to 4 weeks in advance), FIX: Alternate search names now will be searched against when doing manual post-processing, FIX: When manually post-processing, if series volume wasn't specified would fail to match against v1 (by default), IMP:(#1309) Https_chain option now allowed within config.ini, IMP: 32P pack support on a per series basis (will search individual torrents first before packs), IMP: When pack is snatched, will mark all issues within pack that are not in a Downloaded status as Snatched within Mylar (annuals currently don't work), IMP: Removed unnecessary config spamming on startup when verbose mode was enabled, IMP: Allow for searching on 32p against series+publisher for some titles that distinguish between series by different publisher, IMP: Better series matching when trying to find series matches on 32P, FIX: When metatagging, if volume label is not provided within Mylar would default to None (now will be v1), IMP:(#1304) Attempt at better file parsing when utf-8 filenames are being parsed, FIX: Proper handling of Infinity issue number when file-checking, FIX: When adding series and annuals were enabled, if annual was a new release, would not be shown on the annual subset table for the given series (and subsequently wouldn't be auto-marked as Wanted), FIX:(#1306) Correct handling of the imported value when doing an import and moving files was selected (would previously error out during moving for some imports), FIX: When cbz files were being imported and were attempted to being auto-imported, would fail due to improper handling of the imported variable, FIX: Manage issues will now default the dropdown to the correct selected option, FIX: Manage Comics - fixed dropdown options for multiple selection of series - delete/pause/resume, IMP: Added 'delete stragglers' option to Story Arcs when deleting an arc to ensure that all traces of the arc are removed from the db, FIX: Manual/group metatagging would not tag properly if the START_YEAR_AS_VOLUME option was enabled, FIX: (#1313) NzbHydra wouldn't set the nzbid properly when using Failed Download handling/Retrying
2016-07-10 22:28:14 +00:00
logger . info ( ' CStatus: ' + str ( CStatus ) )
2013-04-26 03:45:02 +00:00
if CStatus :
2015-04-28 02:15:53 +00:00
newValue = { " STATUS " : CStatus }
2013-04-26 03:45:02 +00:00
else :
2017-10-25 18:33:56 +00:00
if mylar . CONFIG . AUTOWANT_UPCOMING :
2015-04-28 02:15:53 +00:00
newValue = { " STATUS " : " Wanted " }
2013-04-26 03:45:02 +00:00
else :
2015-04-28 02:15:53 +00:00
newValue = { " STATUS " : " Skipped " }
#setting this here regardless, as it will be a match for a watchlist hit at this point anyways - so link it here what's availalbe.
newValue [ ' ComicID ' ] = CID [ ' ComicID ' ]
newValue [ ' IssueID ' ] = cidissueid
2013-04-26 03:45:02 +00:00
IMP: Added ALT_PULL=2 method for weeklypull management. Will now update against an external site and pull down already populated ComicID/IssueID's for the series that exist on the pullist. Alternate Search Names aren't needed with this option to match on pullist, as well as CV API Hits are also not used since it is all populated. Also allows for future viewing of pullists (up to 4 weeks in advance), FIX: Alternate search names now will be searched against when doing manual post-processing, FIX: When manually post-processing, if series volume wasn't specified would fail to match against v1 (by default), IMP:(#1309) Https_chain option now allowed within config.ini, IMP: 32P pack support on a per series basis (will search individual torrents first before packs), IMP: When pack is snatched, will mark all issues within pack that are not in a Downloaded status as Snatched within Mylar (annuals currently don't work), IMP: Removed unnecessary config spamming on startup when verbose mode was enabled, IMP: Allow for searching on 32p against series+publisher for some titles that distinguish between series by different publisher, IMP: Better series matching when trying to find series matches on 32P, FIX: When metatagging, if volume label is not provided within Mylar would default to None (now will be v1), IMP:(#1304) Attempt at better file parsing when utf-8 filenames are being parsed, FIX: Proper handling of Infinity issue number when file-checking, FIX: When adding series and annuals were enabled, if annual was a new release, would not be shown on the annual subset table for the given series (and subsequently wouldn't be auto-marked as Wanted), FIX:(#1306) Correct handling of the imported value when doing an import and moving files was selected (would previously error out during moving for some imports), FIX: When cbz files were being imported and were attempted to being auto-imported, would fail due to improper handling of the imported variable, FIX: Manage issues will now default the dropdown to the correct selected option, FIX: Manage Comics - fixed dropdown options for multiple selection of series - delete/pause/resume, IMP: Added 'delete stragglers' option to Story Arcs when deleting an arc to ensure that all traces of the arc are removed from the db, FIX: Manual/group metatagging would not tag properly if the START_YEAR_AS_VOLUME option was enabled, FIX: (#1313) NzbHydra wouldn't set the nzbid properly when using Failed Download handling/Retrying
2016-07-10 22:28:14 +00:00
logger . info ( ' newValue: ' + str ( newValue ) )
myDB . upsert ( " weekly " , newValue , controlValue )
2012-09-13 15:27:34 +00:00
2014-04-14 17:42:42 +00:00
def newpullcheck ( ComicName , ComicID , issue = None ) :
2012-09-24 05:17:29 +00:00
# When adding a new comic, let's check for new issues on this week's pullist and update.
2017-10-25 18:33:56 +00:00
if mylar . CONFIG . ALT_PULL != 2 or mylar . PULLBYFILE is True :
2016-09-30 20:10:08 +00:00
mylar . weeklypull . pullitcheck ( comic1off_name = ComicName , comic1off_id = ComicID , issue = issue )
else :
mylar . weeklypull . new_pullcheck ( weeknumber = mylar . CURRENT_WEEKNUMBER , pullyear = mylar . CURRENT_YEAR , comic1off_name = ComicName , comic1off_id = ComicID , issue = issue )
2012-09-24 05:17:29 +00:00
return
2012-09-17 05:12:40 +00:00
def no_searchresults ( ComicID ) :
# when there's a mismatch between CV & GCD - let's change the status to
# something other than 'Loaded'
myDB = db . DBConnection ( )
2015-05-22 08:32:51 +00:00
controlValue = { " ComicID " : ComicID }
2013-01-11 21:20:51 +00:00
newValue = { " Status " : " Error " ,
2013-03-14 08:33:16 +00:00
" LatestDate " : " Error " ,
2015-05-22 08:32:51 +00:00
" LatestIssue " : " Error " }
2012-09-17 05:12:40 +00:00
myDB . upsert ( " comics " , newValue , controlValue )
FIX:(#1630) One-off issues will now be post-processed according to grab-bag dir location, IMP: One-off pull-list issues will now have the status marked accordingly on weekly pullist for tracking purposes, FIX: Fix for utorrent not being able to update status properly after initial snatch, FIX: Fix for some incorrect variable references due to different snatch methods when trying to run on-snatch scripts, FIX: Fix for auto-torrent-snatcher trying to snatch a torrent with an '&' in the filename, FIX:(#1637) Fix for telegram not saving on snatch toggle properly, IMP: added additional logging for metatagging when attempting to create temp folder in cache location, IMP: will now clean up any left over metagtagging directories that fail during normal post-processing instead of leaving and continuing, FIX: Will notify on one-off grab's/storyarc's being post-processed, FIX: fix for displaying cbz issue details (the 'i' on the comic details page) in which the series title contained non-alphanumeric characters, IMP: Added ability to change status to Wanted on Upcoming tab (will perform an immediate search on selected issues), IMP: Threaded the one-off search call from the weeklypull
2017-06-02 14:05:06 +00:00
def nzblog ( IssueID , NZBName , ComicName , SARC = None , IssueArcID = None , id = None , prov = None , alt_nzbname = None , oneoff = False ) :
2012-10-30 10:43:01 +00:00
myDB = db . DBConnection ( )
2013-05-25 06:18:00 +00:00
2015-02-18 19:33:18 +00:00
newValue = { ' NZBName ' : NZBName }
2013-05-25 06:18:00 +00:00
2015-05-11 08:14:47 +00:00
if SARC :
2016-11-11 16:52:24 +00:00
logger . fdebug ( " Story Arc (SARC) detected as: " + str ( SARC ) )
2015-05-11 08:14:47 +00:00
IssueID = ' S ' + str ( IssueArcID )
newValue [ ' SARC ' ] = SARC
FIX:(#1630) One-off issues will now be post-processed according to grab-bag dir location, IMP: One-off pull-list issues will now have the status marked accordingly on weekly pullist for tracking purposes, FIX: Fix for utorrent not being able to update status properly after initial snatch, FIX: Fix for some incorrect variable references due to different snatch methods when trying to run on-snatch scripts, FIX: Fix for auto-torrent-snatcher trying to snatch a torrent with an '&' in the filename, FIX:(#1637) Fix for telegram not saving on snatch toggle properly, IMP: added additional logging for metatagging when attempting to create temp folder in cache location, IMP: will now clean up any left over metagtagging directories that fail during normal post-processing instead of leaving and continuing, FIX: Will notify on one-off grab's/storyarc's being post-processed, FIX: fix for displaying cbz issue details (the 'i' on the comic details page) in which the series title contained non-alphanumeric characters, IMP: Added ability to change status to Wanted on Upcoming tab (will perform an immediate search on selected issues), IMP: Threaded the one-off search call from the weeklypull
2017-06-02 14:05:06 +00:00
if oneoff is True :
logger . fdebug ( ' One-Off download detected when updating - crossing the t \' s and dotting the i \' s so things work... ' )
newValue [ ' OneOff ' ] = True
2013-04-12 02:14:27 +00:00
if IssueID is None or IssueID == ' None ' :
#if IssueID is None, it's a one-off download from the pull-list.
#give it a generic ID above the last one so it doesn't throw an error later.
2017-10-25 18:33:56 +00:00
if any ( [ mylar . CONFIG . HIGHCOUNT == 0 , mylar . CONFIG . HIGHCOUNT is None ] ) :
mylar . CONFIG . HIGHCOUNT = 900000
2015-05-22 08:32:51 +00:00
else :
2017-10-25 18:33:56 +00:00
mylar . CONFIG . HIGHCOUNT + = 1
2017-01-20 19:15:15 +00:00
2017-10-25 18:33:56 +00:00
IssueID = mylar . CONFIG . HIGHCOUNT
#mylar.config_write()
2015-05-22 08:32:51 +00:00
FIX: undefined on filter box on startup, IMP: Added Meta-Tagging options on a series / issue basis on comic details screen, IMP: Issue Information is now available per issue and is extracted currently from the cbz file to display (if no cbz is present, the option isn't available), IMP: Failed Download handling is implemented and available in GUI - required to replace existing autoProcessComics.py and ComicRN.py scripts, IMP: Added ability to specify post-processing script instead of ComicRN.py, IMP: Added abilty to edit the issue date for a given issue by simply clicking on it - this will help to avoid dates that have incorrect or 0000-00-00, IMP: Story Arc searching is working (not adding yet), IMP: Added Archived/Ignored options to Upcoming/Wanted tab, IMP: Fixed some alignment and display issues on the Upcoming section, IMP: Added better directory handling for Updating Comic Location when it needs to get changed for all existing series (locmove in config.ini), IMP: Added better handling for unicode characters in series titles when searching / filechecking, IMP: When adding a new series with no data, Mylar would error out (now will add and just retain 0 issues), FIX: When year was fuzzied, search would still attempt to do a date-check comparison and say everything failed, IMP: Better handling of nzb names when retaining for post-processing comparisons, IMP: Future Upcoming now will use actual shipping date of issue if available in order to see if issue is available, FIX: If annuals were enabled, refreshing a series would put some issues into an Archived status because the actual counts would be off, IMP: When file checking, Alternate Naming would be searched last which resulted in matching incorrectly to the series title or other alternate naming for the given series, now will check the Alternate Naming first for a match, then drop back down to the series name itself otherwise, IMP: Improved Annual detection when integrated with a given series, IMP: Improved the checking of the future Upcoming list for issues marked as Wanted but not available yet and then auto-adding, IMP: Improved upon story arc checking for missing issues / searching for wanted, IMP: Enabling Annuals support now within Configuration GUI, bunch of other things....
2014-07-28 19:28:09 +00:00
controlValue = { " IssueID " : IssueID ,
" Provider " : prov }
if id :
logger . info ( ' setting the nzbid for this download grabbed by ' + prov + ' in the nzblog to : ' + str ( id ) )
2015-02-18 19:33:18 +00:00
newValue [ ' ID ' ] = id
FIX: undefined on filter box on startup, IMP: Added Meta-Tagging options on a series / issue basis on comic details screen, IMP: Issue Information is now available per issue and is extracted currently from the cbz file to display (if no cbz is present, the option isn't available), IMP: Failed Download handling is implemented and available in GUI - required to replace existing autoProcessComics.py and ComicRN.py scripts, IMP: Added ability to specify post-processing script instead of ComicRN.py, IMP: Added abilty to edit the issue date for a given issue by simply clicking on it - this will help to avoid dates that have incorrect or 0000-00-00, IMP: Story Arc searching is working (not adding yet), IMP: Added Archived/Ignored options to Upcoming/Wanted tab, IMP: Fixed some alignment and display issues on the Upcoming section, IMP: Added better directory handling for Updating Comic Location when it needs to get changed for all existing series (locmove in config.ini), IMP: Added better handling for unicode characters in series titles when searching / filechecking, IMP: When adding a new series with no data, Mylar would error out (now will add and just retain 0 issues), FIX: When year was fuzzied, search would still attempt to do a date-check comparison and say everything failed, IMP: Better handling of nzb names when retaining for post-processing comparisons, IMP: Future Upcoming now will use actual shipping date of issue if available in order to see if issue is available, FIX: If annuals were enabled, refreshing a series would put some issues into an Archived status because the actual counts would be off, IMP: When file checking, Alternate Naming would be searched last which resulted in matching incorrectly to the series title or other alternate naming for the given series, now will check the Alternate Naming first for a match, then drop back down to the series name itself otherwise, IMP: Improved Annual detection when integrated with a given series, IMP: Improved the checking of the future Upcoming list for issues marked as Wanted but not available yet and then auto-adding, IMP: Improved upon story arc checking for missing issues / searching for wanted, IMP: Enabling Annuals support now within Configuration GUI, bunch of other things....
2014-07-28 19:28:09 +00:00
2015-02-18 19:33:18 +00:00
if alt_nzbname :
logger . info ( ' setting the alternate nzbname for this download grabbed by ' + prov + ' in the nzblog to : ' + alt_nzbname )
newValue [ ' AltNZBName ' ] = alt_nzbname
2012-10-30 10:43:01 +00:00
2015-02-18 19:33:18 +00:00
#check if it exists already in the log.
chkd = myDB . selectone ( ' SELECT * FROM nzblog WHERE IssueID=? and Provider=? ' , [ IssueID , prov ] ) . fetchone ( )
if chkd is None :
pass
else :
FIX: if 32p was disabled due to inability to signon, would still attempt to retrieve cached result when match occured which would then possibly result in excessive attempted logins, IMP: Added Force Type option to comicdetails page which will allow the type of series to be forced to TPB/GN format if Mylar is unable to decipher it under normal circumstances (doing this will affect how files are parsed for the given series), IMP: TPB/GN File Parser will now account for mutiple issues of the same TPB series within the given directory, IMP: When enabling/disabling Force Type, will rename existing folder to accomodate new naming folder convention if required, IMP: Started to move some of the file rename/manipulation modules into a filer.py module for more consistency, IMP: Added a to Folder Format which when triggered will use the designated Series Type, IMP:(#2130) When importing, option now available for the final series destination path to use the imported path location instead of defaulting to the Comic Location base folder - status will be no longer be Archived in such cases. Will also not show previous imported series on subsequent import runs if the import directory has not changed (thnx @CuddleBear92), IMP: Added the codebase for a Rename Preview option into the mix - GUI available, but not linked, FIX: Removed DEM from RSS scans when Public Torrents option is enabled, FIX:(#2137) When adding series, if series indicated date in a different format other than just numeric, would cause error
2018-12-10 17:27:53 +00:00
altnames = chkd [ ' AltNZBName ' ]
IMP: Added 'Force to TPB/GN' option for printed editions and 'Force to Print' option for TPB/GN's (this WILL impact how files are scanned in - so make sure you have files named appropriately), IMP: Added One-Shot as a Type, FIX: Fixed incorrect variable reference to seriesvolume during post-processing of TPB series, FIX: Fixed refresh to fire off every 5 minutes, FIX: When post-processing and metatagging a cbz file with rename files not enabled, would indicate the new filename as being a copy of the existing file (ie. (1).cbz) which would carry over to a final filename, IMP: Allow Mylar to populate the alias list on add/refresh of a series if the aliases don't already exist in the Alternate Series field for the series, FIX: Fixed a problem with single quotes in series titles when using experimental search,
IMP: When searching for a one-off series, will accept either issues with an issue number or those with no issue numbers as acceptable search results when searching, IMP: Updated search code to utilize the existing filename parser within Mylar when parsing the search results for more consistency
2018-12-22 22:48:14 +00:00
if any ( [ altnames is None , altnames == ' ' ] ) :
2015-02-18 19:33:18 +00:00
#we need to wipe the entry so we can re-update with the alt-nzbname if required
myDB . action ( ' DELETE FROM nzblog WHERE IssueID=? and Provider=? ' , [ IssueID , prov ] )
2015-05-22 08:32:51 +00:00
logger . fdebug ( ' Deleted stale entry from nzblog for IssueID: ' + str ( IssueID ) + ' [ ' + prov + ' ] ' )
2015-02-18 19:33:18 +00:00
myDB . upsert ( " nzblog " , newValue , controlValue )
2015-05-22 08:32:51 +00:00
FIX: undefined on filter box on startup, IMP: Added Meta-Tagging options on a series / issue basis on comic details screen, IMP: Issue Information is now available per issue and is extracted currently from the cbz file to display (if no cbz is present, the option isn't available), IMP: Failed Download handling is implemented and available in GUI - required to replace existing autoProcessComics.py and ComicRN.py scripts, IMP: Added ability to specify post-processing script instead of ComicRN.py, IMP: Added abilty to edit the issue date for a given issue by simply clicking on it - this will help to avoid dates that have incorrect or 0000-00-00, IMP: Story Arc searching is working (not adding yet), IMP: Added Archived/Ignored options to Upcoming/Wanted tab, IMP: Fixed some alignment and display issues on the Upcoming section, IMP: Added better directory handling for Updating Comic Location when it needs to get changed for all existing series (locmove in config.ini), IMP: Added better handling for unicode characters in series titles when searching / filechecking, IMP: When adding a new series with no data, Mylar would error out (now will add and just retain 0 issues), FIX: When year was fuzzied, search would still attempt to do a date-check comparison and say everything failed, IMP: Better handling of nzb names when retaining for post-processing comparisons, IMP: Future Upcoming now will use actual shipping date of issue if available in order to see if issue is available, FIX: If annuals were enabled, refreshing a series would put some issues into an Archived status because the actual counts would be off, IMP: When file checking, Alternate Naming would be searched last which resulted in matching incorrectly to the series title or other alternate naming for the given series, now will check the Alternate Naming first for a match, then drop back down to the series name itself otherwise, IMP: Improved Annual detection when integrated with a given series, IMP: Improved the checking of the future Upcoming list for issues marked as Wanted but not available yet and then auto-adding, IMP: Improved upon story arc checking for missing issues / searching for wanted, IMP: Enabling Annuals support now within Configuration GUI, bunch of other things....
2014-07-28 19:28:09 +00:00
FIX:(#1630) One-off issues will now be post-processed according to grab-bag dir location, IMP: One-off pull-list issues will now have the status marked accordingly on weekly pullist for tracking purposes, FIX: Fix for utorrent not being able to update status properly after initial snatch, FIX: Fix for some incorrect variable references due to different snatch methods when trying to run on-snatch scripts, FIX: Fix for auto-torrent-snatcher trying to snatch a torrent with an '&' in the filename, FIX:(#1637) Fix for telegram not saving on snatch toggle properly, IMP: added additional logging for metatagging when attempting to create temp folder in cache location, IMP: will now clean up any left over metagtagging directories that fail during normal post-processing instead of leaving and continuing, FIX: Will notify on one-off grab's/storyarc's being post-processed, FIX: fix for displaying cbz issue details (the 'i' on the comic details page) in which the series title contained non-alphanumeric characters, IMP: Added ability to change status to Wanted on Upcoming tab (will perform an immediate search on selected issues), IMP: Threaded the one-off search call from the weeklypull
2017-06-02 14:05:06 +00:00
def foundsearch ( ComicID , IssueID , mode = None , down = None , provider = None , SARC = None , IssueArcID = None , module = None , hash = None , crc = None , comicname = None , issuenumber = None , pullinfo = None ) :
2013-01-06 08:51:44 +00:00
# When doing a Force Search (Wanted tab), the resulting search calls this to update.
# this is all redudant code that forceRescan already does.
2015-05-22 08:32:51 +00:00
# should be redone at some point so that instead of rescanning entire
2013-01-06 08:51:44 +00:00
# series directory, it just scans for the issue it just downloaded and
# and change the status to Snatched accordingly. It is not to increment the have count
# at this stage as it's not downloaded - just the .nzb has been snatched and sent to SAB.
IMP:(#753) Better handling of cbz files when using metatagging option, IMP: Added option for MetaTagging to enable/disable overwriting of existing metadata on cbz files, IMP: Post-Processing will now delete folders that are identical to filenames after successful post-processing, FIX:(#756) Fix for Manual Search for an Issue which resulted in traceback errors upon not finding any matches (Windows), IMP: Added module-name to logging for some modules to help troubleshooting (eventually all will be done), IMP: If a filename contained repeated characters for spaces (ie. ___ or ...), it would fail on post-processing due to the nzbname being off, IMP: Monitoring of CV API will now be persitent on restarts/shutdowns/etc, IMP: Added a maximum CV API global so that it can be adjusted for monitoring more easily should API requirements change, IMP: When Adding new series, will now only mark issues that are actually upcoming as Wanted (based on Store Date)
2014-06-22 08:32:45 +00:00
if module is None :
module = ' '
module + = ' [UPDATER] '
2013-09-07 06:05:32 +00:00
myDB = db . DBConnection ( )
IMP: (#844)Annuals now deleted from table on a Delete Series command, IMP: (#837)Better volume detection when searching for issues, IMP: (#842)(#808) Added some handling for issue titles when searching (it's not perfect, but it works for most), IMP: (#836)Added .cb7 for filechecking purposes only, IMP: (#830)issue numbers that are -1 can now be renamed, FIX: 'bi-annual' now fixed along with some other improvements for annual handling, IMP: Added version check for autoProcessComics and ComicRN - in order to help warn users of changes and the need to update these files since they are usually outside of the mylar git directory when being used, FIX:(#840) Boxcar2 should be working again, FIX: (#845) If search returned a filename that had no distinction between issue number and issue title and/or extra information, would error out if any of the words contained NOW, IMP: (#823) Added some better detection for NZBGet parameters when using ComicRN.py
2014-10-06 18:10:36 +00:00
modcomicname = False
2013-09-07 06:05:32 +00:00
IMP:(#753) Better handling of cbz files when using metatagging option, IMP: Added option for MetaTagging to enable/disable overwriting of existing metadata on cbz files, IMP: Post-Processing will now delete folders that are identical to filenames after successful post-processing, FIX:(#756) Fix for Manual Search for an Issue which resulted in traceback errors upon not finding any matches (Windows), IMP: Added module-name to logging for some modules to help troubleshooting (eventually all will be done), IMP: If a filename contained repeated characters for spaces (ie. ___ or ...), it would fail on post-processing due to the nzbname being off, IMP: Monitoring of CV API will now be persitent on restarts/shutdowns/etc, IMP: Added a maximum CV API global so that it can be adjusted for monitoring more easily should API requirements change, IMP: When Adding new series, will now only mark issues that are actually upcoming as Wanted (based on Store Date)
2014-06-22 08:32:45 +00:00
logger . fdebug ( module + ' comicid: ' + str ( ComicID ) )
logger . fdebug ( module + ' issueid: ' + str ( IssueID ) )
FIX:(#1630) One-off issues will now be post-processed according to grab-bag dir location, IMP: One-off pull-list issues will now have the status marked accordingly on weekly pullist for tracking purposes, FIX: Fix for utorrent not being able to update status properly after initial snatch, FIX: Fix for some incorrect variable references due to different snatch methods when trying to run on-snatch scripts, FIX: Fix for auto-torrent-snatcher trying to snatch a torrent with an '&' in the filename, FIX:(#1637) Fix for telegram not saving on snatch toggle properly, IMP: added additional logging for metatagging when attempting to create temp folder in cache location, IMP: will now clean up any left over metagtagging directories that fail during normal post-processing instead of leaving and continuing, FIX: Will notify on one-off grab's/storyarc's being post-processed, FIX: fix for displaying cbz issue details (the 'i' on the comic details page) in which the series title contained non-alphanumeric characters, IMP: Added ability to change status to Wanted on Upcoming tab (will perform an immediate search on selected issues), IMP: Threaded the one-off search call from the weeklypull
2017-06-02 14:05:06 +00:00
if mode != ' pullwant ' :
if mode != ' story_arc ' :
comic = myDB . selectone ( ' SELECT * FROM comics WHERE ComicID=? ' , [ ComicID ] ) . fetchone ( )
ComicName = comic [ ' ComicName ' ]
if mode == ' want_ann ' :
issue = myDB . selectone ( ' SELECT * FROM annuals WHERE IssueID=? ' , [ IssueID ] ) . fetchone ( )
if ComicName != issue [ ' ReleaseComicName ' ] + ' Annual ' :
ComicName = issue [ ' ReleaseComicName ' ]
modcomicname = True
else :
issue = myDB . selectone ( ' SELECT * FROM issues WHERE IssueID=? ' , [ IssueID ] ) . fetchone ( )
CYear = issue [ ' IssueDate ' ] [ : 4 ]
IssueNum = issue [ ' Issue_Number ' ]
2013-09-07 06:05:32 +00:00
FIX:(#1630) One-off issues will now be post-processed according to grab-bag dir location, IMP: One-off pull-list issues will now have the status marked accordingly on weekly pullist for tracking purposes, FIX: Fix for utorrent not being able to update status properly after initial snatch, FIX: Fix for some incorrect variable references due to different snatch methods when trying to run on-snatch scripts, FIX: Fix for auto-torrent-snatcher trying to snatch a torrent with an '&' in the filename, FIX:(#1637) Fix for telegram not saving on snatch toggle properly, IMP: added additional logging for metatagging when attempting to create temp folder in cache location, IMP: will now clean up any left over metagtagging directories that fail during normal post-processing instead of leaving and continuing, FIX: Will notify on one-off grab's/storyarc's being post-processed, FIX: fix for displaying cbz issue details (the 'i' on the comic details page) in which the series title contained non-alphanumeric characters, IMP: Added ability to change status to Wanted on Upcoming tab (will perform an immediate search on selected issues), IMP: Threaded the one-off search call from the weeklypull
2017-06-02 14:05:06 +00:00
else :
IMP: Added Choose specific Download option to manually select from result list on Upcoming/Details/Weekly tabs, IMP: Added Wanted storyarcs to the overall Wanted section so now will search as per global options (storyarc issues can be displayed optionally on Wanted tab), IMP: Added custom url option for image banners/posters for storyarcs, IMP: updated Cherrypy, FIX: Fixed ComicRN not working when forms authentication used - in conjunction updated autoProcessComics to 2.0 which now uses apikey instead of user/pass, IMP: Alternate Series Covers option for alternating existing series image to the most current issue image, IMP: Added overall series total to series page for reference, IMP: Search workflow completely changed to accomodate more than one indexer - now will simultaneously sumbit initial request to each provider, wait 30s submit additional as required at 30s intervals, FIX: Removed TPSE as an option and relabelled to just Public Torrents, IMP: Added direct backlog search to WWT option (pack support will follow), FIX: Removed line about configparser being required for ComicTagger usage, IMP: Test code in place for newzab testing, FIX: Fixed layout problem with torrents that are in auto-snatch status on weekly tab, IMP: backend code improvements to allow for better alias usage and annual linking directly from WS, IMP: Updated systemd init-scripts with read.me, IMP: When post-processing, will now check for available destination free space before actually moving files, IMP: Will copy during metatagging to cache folder instead of move being an option so cleanup is cleaner if something fails, FIX: Changed readinglist table to storyarcs for clarity, IMP: When post-processing issues, will now only update the one issue status and adjust totals accordingly (instead of doing a complete rescan of the series), FIX: Clear out empty ID's from the Failed DB on startup, IMP: Initial code-run at REST API interface, FIX: Fixed some matching problems with 32p due to case, IMP: removed apikeys from log entries that were accidentally logging, IMP: When searching 32p, if items get packed up - will now delete the cached reference so new items of the same can be located, IMP: ForceSearch option switched to scheduler so simultaneous runs should not occur, FIX: Fixed manual metatagging error that would occur if multiple destination directories existed
2018-02-16 19:57:01 +00:00
issue = myDB . selectone ( ' SELECT * FROM storyarcs WHERE IssueArcID=? ' , [ IssueArcID ] ) . fetchone ( )
FIX:(#1630) One-off issues will now be post-processed according to grab-bag dir location, IMP: One-off pull-list issues will now have the status marked accordingly on weekly pullist for tracking purposes, FIX: Fix for utorrent not being able to update status properly after initial snatch, FIX: Fix for some incorrect variable references due to different snatch methods when trying to run on-snatch scripts, FIX: Fix for auto-torrent-snatcher trying to snatch a torrent with an '&' in the filename, FIX:(#1637) Fix for telegram not saving on snatch toggle properly, IMP: added additional logging for metatagging when attempting to create temp folder in cache location, IMP: will now clean up any left over metagtagging directories that fail during normal post-processing instead of leaving and continuing, FIX: Will notify on one-off grab's/storyarc's being post-processed, FIX: fix for displaying cbz issue details (the 'i' on the comic details page) in which the series title contained non-alphanumeric characters, IMP: Added ability to change status to Wanted on Upcoming tab (will perform an immediate search on selected issues), IMP: Threaded the one-off search call from the weeklypull
2017-06-02 14:05:06 +00:00
ComicName = issue [ ' ComicName ' ]
CYear = issue [ ' IssueYEAR ' ]
IssueNum = issue [ ' IssueNumber ' ]
2013-07-01 05:19:15 +00:00
else :
FIX:(#1630) One-off issues will now be post-processed according to grab-bag dir location, IMP: One-off pull-list issues will now have the status marked accordingly on weekly pullist for tracking purposes, FIX: Fix for utorrent not being able to update status properly after initial snatch, FIX: Fix for some incorrect variable references due to different snatch methods when trying to run on-snatch scripts, FIX: Fix for auto-torrent-snatcher trying to snatch a torrent with an '&' in the filename, FIX:(#1637) Fix for telegram not saving on snatch toggle properly, IMP: added additional logging for metatagging when attempting to create temp folder in cache location, IMP: will now clean up any left over metagtagging directories that fail during normal post-processing instead of leaving and continuing, FIX: Will notify on one-off grab's/storyarc's being post-processed, FIX: fix for displaying cbz issue details (the 'i' on the comic details page) in which the series title contained non-alphanumeric characters, IMP: Added ability to change status to Wanted on Upcoming tab (will perform an immediate search on selected issues), IMP: Threaded the one-off search call from the weeklypull
2017-06-02 14:05:06 +00:00
oneinfo = myDB . selectone ( ' SELECT * FROM weekly WHERE IssueID=? ' , [ IssueID ] ) . fetchone ( )
if oneinfo is None :
ComicName = comicname
IssueNum = issuenumber
onefail = True
else :
ComicName = oneinfo [ ' COMIC ' ]
IssueNum = oneinfo [ ' ISSUE ' ]
onefail = False
2012-09-13 15:27:34 +00:00
2013-05-15 09:04:26 +00:00
if down is None :
# update the status to Snatched (so it won't keep on re-downloading!)
IMP:(#753) Better handling of cbz files when using metatagging option, IMP: Added option for MetaTagging to enable/disable overwriting of existing metadata on cbz files, IMP: Post-Processing will now delete folders that are identical to filenames after successful post-processing, FIX:(#756) Fix for Manual Search for an Issue which resulted in traceback errors upon not finding any matches (Windows), IMP: Added module-name to logging for some modules to help troubleshooting (eventually all will be done), IMP: If a filename contained repeated characters for spaces (ie. ___ or ...), it would fail on post-processing due to the nzbname being off, IMP: Monitoring of CV API will now be persitent on restarts/shutdowns/etc, IMP: Added a maximum CV API global so that it can be adjusted for monitoring more easily should API requirements change, IMP: When Adding new series, will now only mark issues that are actually upcoming as Wanted (based on Store Date)
2014-06-22 08:32:45 +00:00
logger . info ( module + ' Updating status to snatched ' )
logger . fdebug ( module + ' Provider is ' + provider )
2017-01-06 18:41:13 +00:00
if hash :
logger . fdebug ( module + ' Hash set to : ' + hash )
2013-05-15 09:04:26 +00:00
newValue = { " Status " : " Snatched " }
2013-09-07 06:05:32 +00:00
if mode == ' story_arc ' :
cValue = { " IssueArcID " : IssueArcID }
snatchedupdate = { " IssueArcID " : IssueArcID }
IMP: Added Choose specific Download option to manually select from result list on Upcoming/Details/Weekly tabs, IMP: Added Wanted storyarcs to the overall Wanted section so now will search as per global options (storyarc issues can be displayed optionally on Wanted tab), IMP: Added custom url option for image banners/posters for storyarcs, IMP: updated Cherrypy, FIX: Fixed ComicRN not working when forms authentication used - in conjunction updated autoProcessComics to 2.0 which now uses apikey instead of user/pass, IMP: Alternate Series Covers option for alternating existing series image to the most current issue image, IMP: Added overall series total to series page for reference, IMP: Search workflow completely changed to accomodate more than one indexer - now will simultaneously sumbit initial request to each provider, wait 30s submit additional as required at 30s intervals, FIX: Removed TPSE as an option and relabelled to just Public Torrents, IMP: Added direct backlog search to WWT option (pack support will follow), FIX: Removed line about configparser being required for ComicTagger usage, IMP: Test code in place for newzab testing, FIX: Fixed layout problem with torrents that are in auto-snatch status on weekly tab, IMP: backend code improvements to allow for better alias usage and annual linking directly from WS, IMP: Updated systemd init-scripts with read.me, IMP: When post-processing, will now check for available destination free space before actually moving files, IMP: Will copy during metatagging to cache folder instead of move being an option so cleanup is cleaner if something fails, FIX: Changed readinglist table to storyarcs for clarity, IMP: When post-processing issues, will now only update the one issue status and adjust totals accordingly (instead of doing a complete rescan of the series), FIX: Clear out empty ID's from the Failed DB on startup, IMP: Initial code-run at REST API interface, FIX: Fixed some matching problems with 32p due to case, IMP: removed apikeys from log entries that were accidentally logging, IMP: When searching 32p, if items get packed up - will now delete the cached reference so new items of the same can be located, IMP: ForceSearch option switched to scheduler so simultaneous runs should not occur, FIX: Fixed manual metatagging error that would occur if multiple destination directories existed
2018-02-16 19:57:01 +00:00
myDB . upsert ( " storyarcs " , newValue , cValue )
2013-09-07 06:05:32 +00:00
# update the snatched DB
snatchedupdate = { " IssueID " : IssueArcID ,
" Status " : " Snatched " ,
" Provider " : provider
}
2013-05-15 09:04:26 +00:00
2013-07-01 05:19:15 +00:00
else :
2013-09-07 06:05:32 +00:00
if mode == ' want_ann ' :
controlValue = { " IssueID " : IssueID }
myDB . upsert ( " annuals " , newValue , controlValue )
else :
controlValue = { " IssueID " : IssueID }
FIX:(#1630) One-off issues will now be post-processed according to grab-bag dir location, IMP: One-off pull-list issues will now have the status marked accordingly on weekly pullist for tracking purposes, FIX: Fix for utorrent not being able to update status properly after initial snatch, FIX: Fix for some incorrect variable references due to different snatch methods when trying to run on-snatch scripts, FIX: Fix for auto-torrent-snatcher trying to snatch a torrent with an '&' in the filename, FIX:(#1637) Fix for telegram not saving on snatch toggle properly, IMP: added additional logging for metatagging when attempting to create temp folder in cache location, IMP: will now clean up any left over metagtagging directories that fail during normal post-processing instead of leaving and continuing, FIX: Will notify on one-off grab's/storyarc's being post-processed, FIX: fix for displaying cbz issue details (the 'i' on the comic details page) in which the series title contained non-alphanumeric characters, IMP: Added ability to change status to Wanted on Upcoming tab (will perform an immediate search on selected issues), IMP: Threaded the one-off search call from the weeklypull
2017-06-02 14:05:06 +00:00
if mode != ' pullwant ' :
myDB . upsert ( " issues " , newValue , controlValue )
2013-09-07 06:05:32 +00:00
# update the snatched DB
snatchedupdate = { " IssueID " : IssueID ,
" Status " : " Snatched " ,
" Provider " : provider
}
if mode == ' story_arc ' :
IssueNum = issue [ ' IssueNumber ' ]
newsnatchValues = { " ComicName " : ComicName ,
" ComicID " : ' None ' ,
" Issue_Number " : IssueNum ,
" DateAdded " : helpers . now ( ) ,
2017-01-06 18:41:13 +00:00
" Status " : " Snatched " ,
" Hash " : hash
2013-09-07 06:05:32 +00:00
}
FIX:(#1630) One-off issues will now be post-processed according to grab-bag dir location, IMP: One-off pull-list issues will now have the status marked accordingly on weekly pullist for tracking purposes, FIX: Fix for utorrent not being able to update status properly after initial snatch, FIX: Fix for some incorrect variable references due to different snatch methods when trying to run on-snatch scripts, FIX: Fix for auto-torrent-snatcher trying to snatch a torrent with an '&' in the filename, FIX:(#1637) Fix for telegram not saving on snatch toggle properly, IMP: added additional logging for metatagging when attempting to create temp folder in cache location, IMP: will now clean up any left over metagtagging directories that fail during normal post-processing instead of leaving and continuing, FIX: Will notify on one-off grab's/storyarc's being post-processed, FIX: fix for displaying cbz issue details (the 'i' on the comic details page) in which the series title contained non-alphanumeric characters, IMP: Added ability to change status to Wanted on Upcoming tab (will perform an immediate search on selected issues), IMP: Threaded the one-off search call from the weeklypull
2017-06-02 14:05:06 +00:00
myDB . upsert ( " snatched " , newsnatchValues , snatchedupdate )
2018-03-22 14:00:34 +00:00
elif mode != ' pullwant ' :
IMP: (#844)Annuals now deleted from table on a Delete Series command, IMP: (#837)Better volume detection when searching for issues, IMP: (#842)(#808) Added some handling for issue titles when searching (it's not perfect, but it works for most), IMP: (#836)Added .cb7 for filechecking purposes only, IMP: (#830)issue numbers that are -1 can now be renamed, FIX: 'bi-annual' now fixed along with some other improvements for annual handling, IMP: Added version check for autoProcessComics and ComicRN - in order to help warn users of changes and the need to update these files since they are usually outside of the mylar git directory when being used, FIX:(#840) Boxcar2 should be working again, FIX: (#845) If search returned a filename that had no distinction between issue number and issue title and/or extra information, would error out if any of the words contained NOW, IMP: (#823) Added some better detection for NZBGet parameters when using ComicRN.py
2014-10-06 18:10:36 +00:00
if modcomicname :
2013-09-07 06:05:32 +00:00
IssueNum = issue [ ' Issue_Number ' ]
IMP: (#844)Annuals now deleted from table on a Delete Series command, IMP: (#837)Better volume detection when searching for issues, IMP: (#842)(#808) Added some handling for issue titles when searching (it's not perfect, but it works for most), IMP: (#836)Added .cb7 for filechecking purposes only, IMP: (#830)issue numbers that are -1 can now be renamed, FIX: 'bi-annual' now fixed along with some other improvements for annual handling, IMP: Added version check for autoProcessComics and ComicRN - in order to help warn users of changes and the need to update these files since they are usually outside of the mylar git directory when being used, FIX:(#840) Boxcar2 should be working again, FIX: (#845) If search returned a filename that had no distinction between issue number and issue title and/or extra information, would error out if any of the words contained NOW, IMP: (#823) Added some better detection for NZBGet parameters when using ComicRN.py
2014-10-06 18:10:36 +00:00
else :
if mode == ' want_ann ' :
IssueNum = " Annual " + issue [ ' Issue_Number ' ]
else :
IssueNum = issue [ ' Issue_Number ' ]
2013-09-07 06:05:32 +00:00
newsnatchValues = { " ComicName " : ComicName ,
" ComicID " : ComicID ,
" Issue_Number " : IssueNum ,
" DateAdded " : helpers . now ( ) ,
2017-01-06 18:41:13 +00:00
" Status " : " Snatched " ,
" Hash " : hash
2013-09-07 06:05:32 +00:00
}
2015-05-21 15:50:15 +00:00
FIX:(#1630) One-off issues will now be post-processed according to grab-bag dir location, IMP: One-off pull-list issues will now have the status marked accordingly on weekly pullist for tracking purposes, FIX: Fix for utorrent not being able to update status properly after initial snatch, FIX: Fix for some incorrect variable references due to different snatch methods when trying to run on-snatch scripts, FIX: Fix for auto-torrent-snatcher trying to snatch a torrent with an '&' in the filename, FIX:(#1637) Fix for telegram not saving on snatch toggle properly, IMP: added additional logging for metatagging when attempting to create temp folder in cache location, IMP: will now clean up any left over metagtagging directories that fail during normal post-processing instead of leaving and continuing, FIX: Will notify on one-off grab's/storyarc's being post-processed, FIX: fix for displaying cbz issue details (the 'i' on the comic details page) in which the series title contained non-alphanumeric characters, IMP: Added ability to change status to Wanted on Upcoming tab (will perform an immediate search on selected issues), IMP: Threaded the one-off search call from the weeklypull
2017-06-02 14:05:06 +00:00
myDB . upsert ( " snatched " , newsnatchValues , snatchedupdate )
else :
#updating snatched table with one-off is abit difficult due to lack of complete information in some instances
#ie. alt_pull 2 not populated yet, alt_pull 0 method in general doesn't have enough info....
newsnatchValues = { " ComicName " : ComicName ,
" ComicID " : ComicID ,
" IssueID " : IssueID ,
" Issue_Number " : IssueNum ,
" DateAdded " : helpers . now ( ) ,
" Status " : " Snatched " ,
" Hash " : hash
}
myDB . upsert ( " snatched " , newsnatchValues , snatchedupdate )
#this will update the weeklypull list immediately after snatching to reflect the new status.
2015-05-21 15:50:15 +00:00
#-is ugly, should be linked directly to other table (IssueID should be populated in weekly pull at this point hopefully).
2015-05-22 08:32:51 +00:00
chkit = myDB . selectone ( " SELECT * FROM weekly WHERE ComicID=? AND IssueID=? " , [ ComicID , IssueID ] ) . fetchone ( )
2015-05-21 15:50:15 +00:00
if chkit is not None :
FIX:(#1630) One-off issues will now be post-processed according to grab-bag dir location, IMP: One-off pull-list issues will now have the status marked accordingly on weekly pullist for tracking purposes, FIX: Fix for utorrent not being able to update status properly after initial snatch, FIX: Fix for some incorrect variable references due to different snatch methods when trying to run on-snatch scripts, FIX: Fix for auto-torrent-snatcher trying to snatch a torrent with an '&' in the filename, FIX:(#1637) Fix for telegram not saving on snatch toggle properly, IMP: added additional logging for metatagging when attempting to create temp folder in cache location, IMP: will now clean up any left over metagtagging directories that fail during normal post-processing instead of leaving and continuing, FIX: Will notify on one-off grab's/storyarc's being post-processed, FIX: fix for displaying cbz issue details (the 'i' on the comic details page) in which the series title contained non-alphanumeric characters, IMP: Added ability to change status to Wanted on Upcoming tab (will perform an immediate search on selected issues), IMP: Threaded the one-off search call from the weeklypull
2017-06-02 14:05:06 +00:00
comicname = chkit [ ' COMIC ' ]
issue = chkit [ ' ISSUE ' ]
2015-05-21 15:50:15 +00:00
ctlVal = { " ComicID " : ComicID ,
" IssueID " : IssueID }
FIX:(#1630) One-off issues will now be post-processed according to grab-bag dir location, IMP: One-off pull-list issues will now have the status marked accordingly on weekly pullist for tracking purposes, FIX: Fix for utorrent not being able to update status properly after initial snatch, FIX: Fix for some incorrect variable references due to different snatch methods when trying to run on-snatch scripts, FIX: Fix for auto-torrent-snatcher trying to snatch a torrent with an '&' in the filename, FIX:(#1637) Fix for telegram not saving on snatch toggle properly, IMP: added additional logging for metatagging when attempting to create temp folder in cache location, IMP: will now clean up any left over metagtagging directories that fail during normal post-processing instead of leaving and continuing, FIX: Will notify on one-off grab's/storyarc's being post-processed, FIX: fix for displaying cbz issue details (the 'i' on the comic details page) in which the series title contained non-alphanumeric characters, IMP: Added ability to change status to Wanted on Upcoming tab (will perform an immediate search on selected issues), IMP: Threaded the one-off search call from the weeklypull
2017-06-02 14:05:06 +00:00
myDB . upsert ( " weekly " , newValue , ctlVal )
newValue [ ' IssueNumber ' ] = issue
newValue [ ' ComicName ' ] = comicname
newValue [ ' Status ' ] = " Snatched "
if pullinfo is not None :
newValue [ ' weeknumber ' ] = pullinfo [ ' weeknumber ' ]
newValue [ ' year ' ] = pullinfo [ ' year ' ]
IMP: Updated CT to a newer version - changed to use rarfile, fixes size invalid errors with some cbr's encountered during conversions, IMP: Search and Post-Processing Queues now added in order to queue up all searches/post-processing in sequence instead of loading lists and iterating through it and encountering various lock-related errors, IMP: In most cases, will now post-process directly against comicid/issueid/storyarcid instead of relying on relationship via nzb/torrent, FIX: realigned some checkboxes on series detail page, FIX: in configuration, indicated that ComicRN cannot be used when Completed Download Handling is enabled for a client now, FIX: Fix for issues named as 'special' that are part of the annuals section, but would not work in various portions (also will allow for naming differences in the future hopefully), FIX: Will now trap error properly when file cannot be located during dupecheck so that post-processing can continue without error, FIX: Fixed newznab test option on configuration screen so that it will return the correct response (previous was an invalid api endpoint), FIX: When retrieving image for storyarcs would error when no image was present and stop the process, FIX: accounted for some incorrect newznab responses that would return a non-api endpoint when retrieving via rss, which resulted in invalid nzbs, FIX: in some cases, weekly pull snatches would not update properly on the weekly page due to some db values not being written
2018-06-14 17:07:39 +00:00
else :
try :
newValue [ ' weeknumber ' ] = chkit [ ' weeknumber ' ]
newValue [ ' year ' ] = chkit [ ' year ' ]
except :
pass
FIX:(#1630) One-off issues will now be post-processed according to grab-bag dir location, IMP: One-off pull-list issues will now have the status marked accordingly on weekly pullist for tracking purposes, FIX: Fix for utorrent not being able to update status properly after initial snatch, FIX: Fix for some incorrect variable references due to different snatch methods when trying to run on-snatch scripts, FIX: Fix for auto-torrent-snatcher trying to snatch a torrent with an '&' in the filename, FIX:(#1637) Fix for telegram not saving on snatch toggle properly, IMP: added additional logging for metatagging when attempting to create temp folder in cache location, IMP: will now clean up any left over metagtagging directories that fail during normal post-processing instead of leaving and continuing, FIX: Will notify on one-off grab's/storyarc's being post-processed, FIX: fix for displaying cbz issue details (the 'i' on the comic details page) in which the series title contained non-alphanumeric characters, IMP: Added ability to change status to Wanted on Upcoming tab (will perform an immediate search on selected issues), IMP: Threaded the one-off search call from the weeklypull
2017-06-02 14:05:06 +00:00
myDB . upsert ( " oneoffhistory " , newValue , ctlVal )
2019-10-07 17:18:17 +00:00
logger . info ( ' %s Updated the status (Snatched) complete for %s Issue: %s ' % ( module , ComicName , IssueNum ) )
2013-05-15 09:04:26 +00:00
else :
IMP: Ability to now specify search provider order (regardless of torrents or nzb) within the config.ini, IMP: (#667) Changed the db module to try to accomodate db locking errors and lowering the amount of actual write transactions that were committed along with a new scheduler system, IMP: Changed sabnzbd directory to post-processing, and included subdirs for sabnzbd & nzbget ComicRN scripts, IMP: NZBGet Post-Processing ComicRN.py script (updated for use with nzbget v11.0+)added & updated in post-processing/nzbget directory (thnx ministoat), FIX: If Issue Location was None, and status was Downloaded would cause error in GUI and break series, IMP: (#689) Minimum # of seeders added (will work with KAT), IMP: (#680) Added Boxcar 2 IO Notifications, IMP: Added PushBullet Notifications, IMP: Cleaned up some notification messages so it's not so cluttered, IMP: Added Clickable series link in History tab, IMP: Added Post-Processed as a status to History tab to show manually post-processed items, IMP: Removed log level dropdown from Logs page & added 'ThreadName' as a column, IMP: Added Force Check Availability & View Future Pull-list to Upcoming sub-tabs, IMP: Added '--safe' option to startup options which will redirect directly to Manage Comics screen incase things are broken, FIX: Added proper month conversions for manual post-processing when doing comparitive issue analysis for matches, FIX: (#613) Allow for negative issue numbers in post-processing when renaming and issue padding is enabled, FIX: File Permissions on post-processing would stop post-processing if couldn't change, now will just log the error and continue, IMP: Added Scheduler (from sickbeard) to allow for threadnaming and better scheduling, IMP: Filenames in the format of ' () ' will now get scanned in, IMP: During manual post-processing will now stop looking for matches upon a successful match, IMP: A Refresh/Weeklypull series check will now just scan in issue data, instead of series info,etc, IMP: Removed some legacy GCD code that is no longer in use, IMP: Exception/traceback handling will now be logged, FIX: Unable to grab torrents from KAT due to content-encoding detection failing, IMP: Added universal date-time conversion to allow for non-english based dates to be properly compared when checking search results against publication dates, FIX: Annuals will now get proper notification (prior was leaving out the word 'annual' from notification/logs), IMP: Improved future pull-list detection and increased retension (now ~5 months), IMP: Will now mark new issues as Wanted on a Refresh Series if autowant upcoming is enabled (was reverting to a status of None previously), IMP: Cannot change status to Downloaded if current status is Skipped or Wanted, FIX: (#704) UnSkipped will now work (X in options column on comic details page), IMP: future_check will check upcoming future issues (future pull-list) that have no series data yet (ie. #1's) and auto-add them to watchlist when the data is available and auto-want accordingly, IMP: (#706) Downloading issues to local machine (via comicdetails screen) with special characters in filename now will work, IMP: improved comparison checks during weekly pull list and improved speed abit since only refreshing issue data now instead of entire series, Other Referenced issues: (#670)(#690) and some others....
2014-05-25 18:32:11 +00:00
if down == ' PP ' :
IMP:(#753) Better handling of cbz files when using metatagging option, IMP: Added option for MetaTagging to enable/disable overwriting of existing metadata on cbz files, IMP: Post-Processing will now delete folders that are identical to filenames after successful post-processing, FIX:(#756) Fix for Manual Search for an Issue which resulted in traceback errors upon not finding any matches (Windows), IMP: Added module-name to logging for some modules to help troubleshooting (eventually all will be done), IMP: If a filename contained repeated characters for spaces (ie. ___ or ...), it would fail on post-processing due to the nzbname being off, IMP: Monitoring of CV API will now be persitent on restarts/shutdowns/etc, IMP: Added a maximum CV API global so that it can be adjusted for monitoring more easily should API requirements change, IMP: When Adding new series, will now only mark issues that are actually upcoming as Wanted (based on Store Date)
2014-06-22 08:32:45 +00:00
logger . info ( module + ' Setting status to Post-Processed in history. ' )
IMP: Ability to now specify search provider order (regardless of torrents or nzb) within the config.ini, IMP: (#667) Changed the db module to try to accomodate db locking errors and lowering the amount of actual write transactions that were committed along with a new scheduler system, IMP: Changed sabnzbd directory to post-processing, and included subdirs for sabnzbd & nzbget ComicRN scripts, IMP: NZBGet Post-Processing ComicRN.py script (updated for use with nzbget v11.0+)added & updated in post-processing/nzbget directory (thnx ministoat), FIX: If Issue Location was None, and status was Downloaded would cause error in GUI and break series, IMP: (#689) Minimum # of seeders added (will work with KAT), IMP: (#680) Added Boxcar 2 IO Notifications, IMP: Added PushBullet Notifications, IMP: Cleaned up some notification messages so it's not so cluttered, IMP: Added Clickable series link in History tab, IMP: Added Post-Processed as a status to History tab to show manually post-processed items, IMP: Removed log level dropdown from Logs page & added 'ThreadName' as a column, IMP: Added Force Check Availability & View Future Pull-list to Upcoming sub-tabs, IMP: Added '--safe' option to startup options which will redirect directly to Manage Comics screen incase things are broken, FIX: Added proper month conversions for manual post-processing when doing comparitive issue analysis for matches, FIX: (#613) Allow for negative issue numbers in post-processing when renaming and issue padding is enabled, FIX: File Permissions on post-processing would stop post-processing if couldn't change, now will just log the error and continue, IMP: Added Scheduler (from sickbeard) to allow for threadnaming and better scheduling, IMP: Filenames in the format of ' () ' will now get scanned in, IMP: During manual post-processing will now stop looking for matches upon a successful match, IMP: A Refresh/Weeklypull series check will now just scan in issue data, instead of series info,etc, IMP: Removed some legacy GCD code that is no longer in use, IMP: Exception/traceback handling will now be logged, FIX: Unable to grab torrents from KAT due to content-encoding detection failing, IMP: Added universal date-time conversion to allow for non-english based dates to be properly compared when checking search results against publication dates, FIX: Annuals will now get proper notification (prior was leaving out the word 'annual' from notification/logs), IMP: Improved future pull-list detection and increased retension (now ~5 months), IMP: Will now mark new issues as Wanted on a Refresh Series if autowant upcoming is enabled (was reverting to a status of None previously), IMP: Cannot change status to Downloaded if current status is Skipped or Wanted, FIX: (#704) UnSkipped will now work (X in options column on comic details page), IMP: future_check will check upcoming future issues (future pull-list) that have no series data yet (ie. #1's) and auto-add them to watchlist when the data is available and auto-want accordingly, IMP: (#706) Downloading issues to local machine (via comicdetails screen) with special characters in filename now will work, IMP: improved comparison checks during weekly pull list and improved speed abit since only refreshing issue data now instead of entire series, Other Referenced issues: (#670)(#690) and some others....
2014-05-25 18:32:11 +00:00
downstatus = ' Post-Processed '
else :
IMP:(#753) Better handling of cbz files when using metatagging option, IMP: Added option for MetaTagging to enable/disable overwriting of existing metadata on cbz files, IMP: Post-Processing will now delete folders that are identical to filenames after successful post-processing, FIX:(#756) Fix for Manual Search for an Issue which resulted in traceback errors upon not finding any matches (Windows), IMP: Added module-name to logging for some modules to help troubleshooting (eventually all will be done), IMP: If a filename contained repeated characters for spaces (ie. ___ or ...), it would fail on post-processing due to the nzbname being off, IMP: Monitoring of CV API will now be persitent on restarts/shutdowns/etc, IMP: Added a maximum CV API global so that it can be adjusted for monitoring more easily should API requirements change, IMP: When Adding new series, will now only mark issues that are actually upcoming as Wanted (based on Store Date)
2014-06-22 08:32:45 +00:00
logger . info ( module + ' Setting status to Downloaded in history. ' )
IMP: Ability to now specify search provider order (regardless of torrents or nzb) within the config.ini, IMP: (#667) Changed the db module to try to accomodate db locking errors and lowering the amount of actual write transactions that were committed along with a new scheduler system, IMP: Changed sabnzbd directory to post-processing, and included subdirs for sabnzbd & nzbget ComicRN scripts, IMP: NZBGet Post-Processing ComicRN.py script (updated for use with nzbget v11.0+)added & updated in post-processing/nzbget directory (thnx ministoat), FIX: If Issue Location was None, and status was Downloaded would cause error in GUI and break series, IMP: (#689) Minimum # of seeders added (will work with KAT), IMP: (#680) Added Boxcar 2 IO Notifications, IMP: Added PushBullet Notifications, IMP: Cleaned up some notification messages so it's not so cluttered, IMP: Added Clickable series link in History tab, IMP: Added Post-Processed as a status to History tab to show manually post-processed items, IMP: Removed log level dropdown from Logs page & added 'ThreadName' as a column, IMP: Added Force Check Availability & View Future Pull-list to Upcoming sub-tabs, IMP: Added '--safe' option to startup options which will redirect directly to Manage Comics screen incase things are broken, FIX: Added proper month conversions for manual post-processing when doing comparitive issue analysis for matches, FIX: (#613) Allow for negative issue numbers in post-processing when renaming and issue padding is enabled, FIX: File Permissions on post-processing would stop post-processing if couldn't change, now will just log the error and continue, IMP: Added Scheduler (from sickbeard) to allow for threadnaming and better scheduling, IMP: Filenames in the format of ' () ' will now get scanned in, IMP: During manual post-processing will now stop looking for matches upon a successful match, IMP: A Refresh/Weeklypull series check will now just scan in issue data, instead of series info,etc, IMP: Removed some legacy GCD code that is no longer in use, IMP: Exception/traceback handling will now be logged, FIX: Unable to grab torrents from KAT due to content-encoding detection failing, IMP: Added universal date-time conversion to allow for non-english based dates to be properly compared when checking search results against publication dates, FIX: Annuals will now get proper notification (prior was leaving out the word 'annual' from notification/logs), IMP: Improved future pull-list detection and increased retension (now ~5 months), IMP: Will now mark new issues as Wanted on a Refresh Series if autowant upcoming is enabled (was reverting to a status of None previously), IMP: Cannot change status to Downloaded if current status is Skipped or Wanted, FIX: (#704) UnSkipped will now work (X in options column on comic details page), IMP: future_check will check upcoming future issues (future pull-list) that have no series data yet (ie. #1's) and auto-add them to watchlist when the data is available and auto-want accordingly, IMP: (#706) Downloading issues to local machine (via comicdetails screen) with special characters in filename now will work, IMP: improved comparison checks during weekly pull list and improved speed abit since only refreshing issue data now instead of entire series, Other Referenced issues: (#670)(#690) and some others....
2014-05-25 18:32:11 +00:00
downstatus = ' Downloaded '
2013-07-01 05:19:15 +00:00
if mode == ' want_ann ' :
FIX:(#1630) One-off issues will now be post-processed according to grab-bag dir location, IMP: One-off pull-list issues will now have the status marked accordingly on weekly pullist for tracking purposes, FIX: Fix for utorrent not being able to update status properly after initial snatch, FIX: Fix for some incorrect variable references due to different snatch methods when trying to run on-snatch scripts, FIX: Fix for auto-torrent-snatcher trying to snatch a torrent with an '&' in the filename, FIX:(#1637) Fix for telegram not saving on snatch toggle properly, IMP: added additional logging for metatagging when attempting to create temp folder in cache location, IMP: will now clean up any left over metagtagging directories that fail during normal post-processing instead of leaving and continuing, FIX: Will notify on one-off grab's/storyarc's being post-processed, FIX: fix for displaying cbz issue details (the 'i' on the comic details page) in which the series title contained non-alphanumeric characters, IMP: Added ability to change status to Wanted on Upcoming tab (will perform an immediate search on selected issues), IMP: Threaded the one-off search call from the weeklypull
2017-06-02 14:05:06 +00:00
if not modcomicname :
IssueNum = " Annual " + IssueNum
2013-09-07 06:05:32 +00:00
elif mode == ' story_arc ' :
IssueID = IssueArcID
2013-07-01 05:19:15 +00:00
2013-05-15 09:04:26 +00:00
snatchedupdate = { " IssueID " : IssueID ,
IMP: Ability to now specify search provider order (regardless of torrents or nzb) within the config.ini, IMP: (#667) Changed the db module to try to accomodate db locking errors and lowering the amount of actual write transactions that were committed along with a new scheduler system, IMP: Changed sabnzbd directory to post-processing, and included subdirs for sabnzbd & nzbget ComicRN scripts, IMP: NZBGet Post-Processing ComicRN.py script (updated for use with nzbget v11.0+)added & updated in post-processing/nzbget directory (thnx ministoat), FIX: If Issue Location was None, and status was Downloaded would cause error in GUI and break series, IMP: (#689) Minimum # of seeders added (will work with KAT), IMP: (#680) Added Boxcar 2 IO Notifications, IMP: Added PushBullet Notifications, IMP: Cleaned up some notification messages so it's not so cluttered, IMP: Added Clickable series link in History tab, IMP: Added Post-Processed as a status to History tab to show manually post-processed items, IMP: Removed log level dropdown from Logs page & added 'ThreadName' as a column, IMP: Added Force Check Availability & View Future Pull-list to Upcoming sub-tabs, IMP: Added '--safe' option to startup options which will redirect directly to Manage Comics screen incase things are broken, FIX: Added proper month conversions for manual post-processing when doing comparitive issue analysis for matches, FIX: (#613) Allow for negative issue numbers in post-processing when renaming and issue padding is enabled, FIX: File Permissions on post-processing would stop post-processing if couldn't change, now will just log the error and continue, IMP: Added Scheduler (from sickbeard) to allow for threadnaming and better scheduling, IMP: Filenames in the format of ' () ' will now get scanned in, IMP: During manual post-processing will now stop looking for matches upon a successful match, IMP: A Refresh/Weeklypull series check will now just scan in issue data, instead of series info,etc, IMP: Removed some legacy GCD code that is no longer in use, IMP: Exception/traceback handling will now be logged, FIX: Unable to grab torrents from KAT due to content-encoding detection failing, IMP: Added universal date-time conversion to allow for non-english based dates to be properly compared when checking search results against publication dates, FIX: Annuals will now get proper notification (prior was leaving out the word 'annual' from notification/logs), IMP: Improved future pull-list detection and increased retension (now ~5 months), IMP: Will now mark new issues as Wanted on a Refresh Series if autowant upcoming is enabled (was reverting to a status of None previously), IMP: Cannot change status to Downloaded if current status is Skipped or Wanted, FIX: (#704) UnSkipped will now work (X in options column on comic details page), IMP: future_check will check upcoming future issues (future pull-list) that have no series data yet (ie. #1's) and auto-add them to watchlist when the data is available and auto-want accordingly, IMP: (#706) Downloading issues to local machine (via comicdetails screen) with special characters in filename now will work, IMP: improved comparison checks during weekly pull list and improved speed abit since only refreshing issue data now instead of entire series, Other Referenced issues: (#670)(#690) and some others....
2014-05-25 18:32:11 +00:00
" Status " : downstatus ,
2013-07-30 04:57:37 +00:00
" Provider " : provider
}
2013-09-07 06:05:32 +00:00
newsnatchValues = { " ComicName " : ComicName ,
2013-05-15 09:04:26 +00:00
" ComicID " : ComicID ,
2013-07-01 05:19:15 +00:00
" Issue_Number " : IssueNum ,
2013-05-15 09:04:26 +00:00
" DateAdded " : helpers . now ( ) ,
2017-05-09 18:08:26 +00:00
" Status " : downstatus ,
" crc " : crc
2013-05-15 09:04:26 +00:00
}
myDB . upsert ( " snatched " , newsnatchValues , snatchedupdate )
2013-09-07 06:05:32 +00:00
if mode == ' story_arc ' :
cValue = { " IssueArcID " : IssueArcID }
nValue = { " Status " : " Downloaded " }
IMP: Added Choose specific Download option to manually select from result list on Upcoming/Details/Weekly tabs, IMP: Added Wanted storyarcs to the overall Wanted section so now will search as per global options (storyarc issues can be displayed optionally on Wanted tab), IMP: Added custom url option for image banners/posters for storyarcs, IMP: updated Cherrypy, FIX: Fixed ComicRN not working when forms authentication used - in conjunction updated autoProcessComics to 2.0 which now uses apikey instead of user/pass, IMP: Alternate Series Covers option for alternating existing series image to the most current issue image, IMP: Added overall series total to series page for reference, IMP: Search workflow completely changed to accomodate more than one indexer - now will simultaneously sumbit initial request to each provider, wait 30s submit additional as required at 30s intervals, FIX: Removed TPSE as an option and relabelled to just Public Torrents, IMP: Added direct backlog search to WWT option (pack support will follow), FIX: Removed line about configparser being required for ComicTagger usage, IMP: Test code in place for newzab testing, FIX: Fixed layout problem with torrents that are in auto-snatch status on weekly tab, IMP: backend code improvements to allow for better alias usage and annual linking directly from WS, IMP: Updated systemd init-scripts with read.me, IMP: When post-processing, will now check for available destination free space before actually moving files, IMP: Will copy during metatagging to cache folder instead of move being an option so cleanup is cleaner if something fails, FIX: Changed readinglist table to storyarcs for clarity, IMP: When post-processing issues, will now only update the one issue status and adjust totals accordingly (instead of doing a complete rescan of the series), FIX: Clear out empty ID's from the Failed DB on startup, IMP: Initial code-run at REST API interface, FIX: Fixed some matching problems with 32p due to case, IMP: removed apikeys from log entries that were accidentally logging, IMP: When searching 32p, if items get packed up - will now delete the cached reference so new items of the same can be located, IMP: ForceSearch option switched to scheduler so simultaneous runs should not occur, FIX: Fixed manual metatagging error that would occur if multiple destination directories existed
2018-02-16 19:57:01 +00:00
myDB . upsert ( " storyarcs " , nValue , cValue )
2013-07-01 05:19:15 +00:00
FIX:(#1630) One-off issues will now be post-processed according to grab-bag dir location, IMP: One-off pull-list issues will now have the status marked accordingly on weekly pullist for tracking purposes, FIX: Fix for utorrent not being able to update status properly after initial snatch, FIX: Fix for some incorrect variable references due to different snatch methods when trying to run on-snatch scripts, FIX: Fix for auto-torrent-snatcher trying to snatch a torrent with an '&' in the filename, FIX:(#1637) Fix for telegram not saving on snatch toggle properly, IMP: added additional logging for metatagging when attempting to create temp folder in cache location, IMP: will now clean up any left over metagtagging directories that fail during normal post-processing instead of leaving and continuing, FIX: Will notify on one-off grab's/storyarc's being post-processed, FIX: fix for displaying cbz issue details (the 'i' on the comic details page) in which the series title contained non-alphanumeric characters, IMP: Added ability to change status to Wanted on Upcoming tab (will perform an immediate search on selected issues), IMP: Threaded the one-off search call from the weeklypull
2017-06-02 14:05:06 +00:00
elif mode != ' pullwant ' :
2013-09-07 06:05:32 +00:00
controlValue = { " IssueID " : IssueID }
newValue = { " Status " : " Downloaded " }
IMP: Cleaned up interface for StoryArcs / Story Arc Details, IMP: Cleaned up interface for Reading list Management, IMP: Added better reading list management - new status (added, downloaded, read), IMP: Added sync option for use with another device for reading list transfer (ie. tablet) Android only, IMP: Autopopulate new weekly pull releases to reading list, IMP: 'Watch' option in weekly pull list now fully functional. Will watch CV for series' that do not have any series data yet as they are new starting series. Will auto-add once available, IMP: Auto-watch check is run after every refresh/recreate of the weeklypull list, IMP: Improved the Add a Series option where it will now look for issues that are 'new' or 'wanted' during add sequence, IMP: Main page interface now has coloured have/total bars to denote series completion, IMP: New scheduler / threading locks in place in an attempt to avoid database locks, FIX: Removed some erroneous locking that was going on when importing a directory was being run, IMP: Stat counter now present when post-processing multiple issues in sequence, FIX: for issue number error when post-processing and issue number was a non-alphanumeric, FIX: for metatagging: when original file was .cbz, would try to convert and fail, FIX: for issues that were negative and were preceeded by a # in the filename (filechecker), FIX: for publisher having non-alphanumeric character in name when attempting to determine publisher, FIX: if annuals enabled, would incorrectly show as being 'already in library' when viewing search results if results constained annuals, FIX:(#944) for incorrect nzbname being used when post-processing was being performed from an nzb client (experimental mainly), IMP: Turned off logging for ComicVine API counter, FIX: Added retry attempts when connecting to ComicVine in order to avoid errors when adding a series, IMP:(#963) Added ability to add snatched to filter when viewing Wanted issues on Wanted tab, FIX: When importing and then selecting a series to import via the select screen, will now flip back to the importresults and add the selected series in the background, IMP:(#952) Main page is now sorted in ascending order by Continuing/Ended status (and subbed by whether is Active/Paused).Custom sorting is still available, FIX: Dupecheck will now automatically assume existing 0-byte files are to be overwritten when performing post-processing, FIX: If publication date for series contained a '?' (usually with brand new series) will force to 'Present' to allow for pull-list comparisons to take place, FIX: Mylar will now disallow search results which have 'covers only' or 'variant' in the filename, IMP: Better nzbname generation/retrieval (will check inside nzb for possible names) to be used when post-processing, IMP: DB Update will now perform update to all active comics in descending order by Latest Date (instead of random order), FIX: Enforce the 5hr limit rule when running DB update (will only update series that haven't been updated in >5 hours), FIX: Annuals will now have/retain the proper status upon doing DB Update, FIX: Have totals will now be updated when doing a recheck files (sometimes wouldn't get updated depending on various states of status'), FIX:(#966) Added urllib2.URLError exeception trap when attempting to check Git for updates, IMP: Removed the individual sqlite calls for weeklypull, and brought them into line with using the db module (which will minimize concurrent access, which seemed to be causing db locks), IMP: Cleaned up some code and shuffled some functions so they are in more appropriate locations
2015-03-27 17:27:59 +00:00
if mode == ' want_ann ' :
myDB . upsert ( " annuals " , newValue , controlValue )
else :
myDB . upsert ( " issues " , newValue , controlValue )
2013-09-07 06:05:32 +00:00
2015-04-28 02:15:53 +00:00
#this will update the weeklypull list immediately after post-processing to reflect the new status.
2015-05-22 08:32:51 +00:00
chkit = myDB . selectone ( " SELECT * FROM weekly WHERE ComicID=? AND IssueID=? AND Status= ' Snatched ' " , [ ComicID , IssueID ] ) . fetchone ( )
2015-04-28 02:15:53 +00:00
if chkit is not None :
FIX:(#1630) One-off issues will now be post-processed according to grab-bag dir location, IMP: One-off pull-list issues will now have the status marked accordingly on weekly pullist for tracking purposes, FIX: Fix for utorrent not being able to update status properly after initial snatch, FIX: Fix for some incorrect variable references due to different snatch methods when trying to run on-snatch scripts, FIX: Fix for auto-torrent-snatcher trying to snatch a torrent with an '&' in the filename, FIX:(#1637) Fix for telegram not saving on snatch toggle properly, IMP: added additional logging for metatagging when attempting to create temp folder in cache location, IMP: will now clean up any left over metagtagging directories that fail during normal post-processing instead of leaving and continuing, FIX: Will notify on one-off grab's/storyarc's being post-processed, FIX: fix for displaying cbz issue details (the 'i' on the comic details page) in which the series title contained non-alphanumeric characters, IMP: Added ability to change status to Wanted on Upcoming tab (will perform an immediate search on selected issues), IMP: Threaded the one-off search call from the weeklypull
2017-06-02 14:05:06 +00:00
comicname = chkit [ ' COMIC ' ]
issue = chkit [ ' ISSUE ' ]
2015-05-22 08:32:51 +00:00
2015-04-28 02:15:53 +00:00
ctlVal = { " ComicID " : ComicID ,
" IssueID " : IssueID }
newVal = { " Status " : " Downloaded " }
myDB . upsert ( " weekly " , newVal , ctlVal )
FIX:(#1630) One-off issues will now be post-processed according to grab-bag dir location, IMP: One-off pull-list issues will now have the status marked accordingly on weekly pullist for tracking purposes, FIX: Fix for utorrent not being able to update status properly after initial snatch, FIX: Fix for some incorrect variable references due to different snatch methods when trying to run on-snatch scripts, FIX: Fix for auto-torrent-snatcher trying to snatch a torrent with an '&' in the filename, FIX:(#1637) Fix for telegram not saving on snatch toggle properly, IMP: added additional logging for metatagging when attempting to create temp folder in cache location, IMP: will now clean up any left over metagtagging directories that fail during normal post-processing instead of leaving and continuing, FIX: Will notify on one-off grab's/storyarc's being post-processed, FIX: fix for displaying cbz issue details (the 'i' on the comic details page) in which the series title contained non-alphanumeric characters, IMP: Added ability to change status to Wanted on Upcoming tab (will perform an immediate search on selected issues), IMP: Threaded the one-off search call from the weeklypull
2017-06-02 14:05:06 +00:00
newVal [ ' IssueNumber ' ] = issue
newVal [ ' ComicName ' ] = comicname
newVal [ ' Status ' ] = " Downloaded "
if pullinfo is not None :
newVal [ ' weeknumber ' ] = pullinfo [ ' weeknumber ' ]
newVal [ ' year ' ] = pullinfo [ ' year ' ]
myDB . upsert ( " oneoffhistory " , newVal , ctlVal )
2019-10-11 14:15:12 +00:00
logger . info ( ' %s Updating Status ( %s ) now completed for %s issue: %s ' % ( module , downstatus , ComicName , IssueNum ) )
2012-09-13 15:27:34 +00:00
return
FIX: (#2006)(#1986) Auto-Want series were not being auto-added, and even after manually added would result in count error after sucessful post-processing, FIX: When filechecking, if a 'special' was included as part of the annuals of a given series, it would not be detected, IMP: (#1982) When ComicRN & Completed Download Handling (CDH) are both enabled, will now check to see if ComicRN is active for the given category in the respective client. If it is, CDH will not post-process and issue a warning and let ComicRN handle the post-processing, FIX: Fixed a utf-8 encoding error when 'Run script AFTER snatch' was enabled, FIX: (#2004) Possible fix for 500 error on Wanted tab, when series had an annual/special attached but was located in the incorrect table, FIX: Removed some logging lines when CDH is monitoring downloads
2018-06-30 18:35:52 +00:00
def forceRescan ( ComicID , archive = None , module = None , recheck = False ) :
IMP:(#753) Better handling of cbz files when using metatagging option, IMP: Added option for MetaTagging to enable/disable overwriting of existing metadata on cbz files, IMP: Post-Processing will now delete folders that are identical to filenames after successful post-processing, FIX:(#756) Fix for Manual Search for an Issue which resulted in traceback errors upon not finding any matches (Windows), IMP: Added module-name to logging for some modules to help troubleshooting (eventually all will be done), IMP: If a filename contained repeated characters for spaces (ie. ___ or ...), it would fail on post-processing due to the nzbname being off, IMP: Monitoring of CV API will now be persitent on restarts/shutdowns/etc, IMP: Added a maximum CV API global so that it can be adjusted for monitoring more easily should API requirements change, IMP: When Adding new series, will now only mark issues that are actually upcoming as Wanted (based on Store Date)
2014-06-22 08:32:45 +00:00
if module is None :
module = ' '
module + = ' [FILE-RESCAN] '
2012-09-13 15:27:34 +00:00
myDB = db . DBConnection ( )
# file check to see if issue exists
IMP: Ability to now specify search provider order (regardless of torrents or nzb) within the config.ini, IMP: (#667) Changed the db module to try to accomodate db locking errors and lowering the amount of actual write transactions that were committed along with a new scheduler system, IMP: Changed sabnzbd directory to post-processing, and included subdirs for sabnzbd & nzbget ComicRN scripts, IMP: NZBGet Post-Processing ComicRN.py script (updated for use with nzbget v11.0+)added & updated in post-processing/nzbget directory (thnx ministoat), FIX: If Issue Location was None, and status was Downloaded would cause error in GUI and break series, IMP: (#689) Minimum # of seeders added (will work with KAT), IMP: (#680) Added Boxcar 2 IO Notifications, IMP: Added PushBullet Notifications, IMP: Cleaned up some notification messages so it's not so cluttered, IMP: Added Clickable series link in History tab, IMP: Added Post-Processed as a status to History tab to show manually post-processed items, IMP: Removed log level dropdown from Logs page & added 'ThreadName' as a column, IMP: Added Force Check Availability & View Future Pull-list to Upcoming sub-tabs, IMP: Added '--safe' option to startup options which will redirect directly to Manage Comics screen incase things are broken, FIX: Added proper month conversions for manual post-processing when doing comparitive issue analysis for matches, FIX: (#613) Allow for negative issue numbers in post-processing when renaming and issue padding is enabled, FIX: File Permissions on post-processing would stop post-processing if couldn't change, now will just log the error and continue, IMP: Added Scheduler (from sickbeard) to allow for threadnaming and better scheduling, IMP: Filenames in the format of ' () ' will now get scanned in, IMP: During manual post-processing will now stop looking for matches upon a successful match, IMP: A Refresh/Weeklypull series check will now just scan in issue data, instead of series info,etc, IMP: Removed some legacy GCD code that is no longer in use, IMP: Exception/traceback handling will now be logged, FIX: Unable to grab torrents from KAT due to content-encoding detection failing, IMP: Added universal date-time conversion to allow for non-english based dates to be properly compared when checking search results against publication dates, FIX: Annuals will now get proper notification (prior was leaving out the word 'annual' from notification/logs), IMP: Improved future pull-list detection and increased retension (now ~5 months), IMP: Will now mark new issues as Wanted on a Refresh Series if autowant upcoming is enabled (was reverting to a status of None previously), IMP: Cannot change status to Downloaded if current status is Skipped or Wanted, FIX: (#704) UnSkipped will now work (X in options column on comic details page), IMP: future_check will check upcoming future issues (future pull-list) that have no series data yet (ie. #1's) and auto-add them to watchlist when the data is available and auto-want accordingly, IMP: (#706) Downloading issues to local machine (via comicdetails screen) with special characters in filename now will work, IMP: improved comparison checks during weekly pull list and improved speed abit since only refreshing issue data now instead of entire series, Other Referenced issues: (#670)(#690) and some others....
2014-05-25 18:32:11 +00:00
rescan = myDB . selectone ( ' SELECT * FROM comics WHERE ComicID=? ' , [ ComicID ] ) . fetchone ( )
2015-05-22 08:32:51 +00:00
if rescan [ ' AlternateSearch ' ] is not None :
FIX: undefined on filter box on startup, IMP: Added Meta-Tagging options on a series / issue basis on comic details screen, IMP: Issue Information is now available per issue and is extracted currently from the cbz file to display (if no cbz is present, the option isn't available), IMP: Failed Download handling is implemented and available in GUI - required to replace existing autoProcessComics.py and ComicRN.py scripts, IMP: Added ability to specify post-processing script instead of ComicRN.py, IMP: Added abilty to edit the issue date for a given issue by simply clicking on it - this will help to avoid dates that have incorrect or 0000-00-00, IMP: Story Arc searching is working (not adding yet), IMP: Added Archived/Ignored options to Upcoming/Wanted tab, IMP: Fixed some alignment and display issues on the Upcoming section, IMP: Added better directory handling for Updating Comic Location when it needs to get changed for all existing series (locmove in config.ini), IMP: Added better handling for unicode characters in series titles when searching / filechecking, IMP: When adding a new series with no data, Mylar would error out (now will add and just retain 0 issues), FIX: When year was fuzzied, search would still attempt to do a date-check comparison and say everything failed, IMP: Better handling of nzb names when retaining for post-processing comparisons, IMP: Future Upcoming now will use actual shipping date of issue if available in order to see if issue is available, FIX: If annuals were enabled, refreshing a series would put some issues into an Archived status because the actual counts would be off, IMP: When file checking, Alternate Naming would be searched last which resulted in matching incorrectly to the series title or other alternate naming for the given series, now will check the Alternate Naming first for a match, then drop back down to the series name itself otherwise, IMP: Improved Annual detection when integrated with a given series, IMP: Improved the checking of the future Upcoming list for issues marked as Wanted but not available yet and then auto-adding, IMP: Improved upon story arc checking for missing issues / searching for wanted, IMP: Enabling Annuals support now within Configuration GUI, bunch of other things....
2014-07-28 19:28:09 +00:00
altnames = rescan [ ' AlternateSearch ' ] + ' ## '
else :
altnames = ' '
FIX: if 32p was disabled due to inability to signon, would still attempt to retrieve cached result when match occured which would then possibly result in excessive attempted logins, IMP: Added Force Type option to comicdetails page which will allow the type of series to be forced to TPB/GN format if Mylar is unable to decipher it under normal circumstances (doing this will affect how files are parsed for the given series), IMP: TPB/GN File Parser will now account for mutiple issues of the same TPB series within the given directory, IMP: When enabling/disabling Force Type, will rename existing folder to accomodate new naming folder convention if required, IMP: Started to move some of the file rename/manipulation modules into a filer.py module for more consistency, IMP: Added a to Folder Format which when triggered will use the designated Series Type, IMP:(#2130) When importing, option now available for the final series destination path to use the imported path location instead of defaulting to the Comic Location base folder - status will be no longer be Archived in such cases. Will also not show previous imported series on subsequent import runs if the import directory has not changed (thnx @CuddleBear92), IMP: Added the codebase for a Rename Preview option into the mix - GUI available, but not linked, FIX: Removed DEM from RSS scans when Public Torrents option is enabled, FIX:(#2137) When adding series, if series indicated date in a different format other than just numeric, would cause error
2018-12-10 17:27:53 +00:00
2019-01-04 15:12:02 +00:00
if ( all ( [ rescan [ ' Type ' ] != ' Print ' , rescan [ ' Type ' ] != ' Digital ' , rescan [ ' Type ' ] != ' None ' , rescan [ ' Type ' ] is not None ] ) and rescan [ ' Corrected_Type ' ] != ' Print ' ) or rescan [ ' Corrected_Type ' ] == ' TPB ' :
2019-01-14 15:54:54 +00:00
if rescan [ ' Type ' ] == ' One-Shot ' and rescan [ ' Corrected_Type ' ] is None :
booktype = ' One-Shot '
else :
booktype = ' TPB '
FIX: if 32p was disabled due to inability to signon, would still attempt to retrieve cached result when match occured which would then possibly result in excessive attempted logins, IMP: Added Force Type option to comicdetails page which will allow the type of series to be forced to TPB/GN format if Mylar is unable to decipher it under normal circumstances (doing this will affect how files are parsed for the given series), IMP: TPB/GN File Parser will now account for mutiple issues of the same TPB series within the given directory, IMP: When enabling/disabling Force Type, will rename existing folder to accomodate new naming folder convention if required, IMP: Started to move some of the file rename/manipulation modules into a filer.py module for more consistency, IMP: Added a to Folder Format which when triggered will use the designated Series Type, IMP:(#2130) When importing, option now available for the final series destination path to use the imported path location instead of defaulting to the Comic Location base folder - status will be no longer be Archived in such cases. Will also not show previous imported series on subsequent import runs if the import directory has not changed (thnx @CuddleBear92), IMP: Added the codebase for a Rename Preview option into the mix - GUI available, but not linked, FIX: Removed DEM from RSS scans when Public Torrents option is enabled, FIX:(#2137) When adding series, if series indicated date in a different format other than just numeric, would cause error
2018-12-10 17:27:53 +00:00
else :
booktype = None
FIX: undefined on filter box on startup, IMP: Added Meta-Tagging options on a series / issue basis on comic details screen, IMP: Issue Information is now available per issue and is extracted currently from the cbz file to display (if no cbz is present, the option isn't available), IMP: Failed Download handling is implemented and available in GUI - required to replace existing autoProcessComics.py and ComicRN.py scripts, IMP: Added ability to specify post-processing script instead of ComicRN.py, IMP: Added abilty to edit the issue date for a given issue by simply clicking on it - this will help to avoid dates that have incorrect or 0000-00-00, IMP: Story Arc searching is working (not adding yet), IMP: Added Archived/Ignored options to Upcoming/Wanted tab, IMP: Fixed some alignment and display issues on the Upcoming section, IMP: Added better directory handling for Updating Comic Location when it needs to get changed for all existing series (locmove in config.ini), IMP: Added better handling for unicode characters in series titles when searching / filechecking, IMP: When adding a new series with no data, Mylar would error out (now will add and just retain 0 issues), FIX: When year was fuzzied, search would still attempt to do a date-check comparison and say everything failed, IMP: Better handling of nzb names when retaining for post-processing comparisons, IMP: Future Upcoming now will use actual shipping date of issue if available in order to see if issue is available, FIX: If annuals were enabled, refreshing a series would put some issues into an Archived status because the actual counts would be off, IMP: When file checking, Alternate Naming would be searched last which resulted in matching incorrectly to the series title or other alternate naming for the given series, now will check the Alternate Naming first for a match, then drop back down to the series name itself otherwise, IMP: Improved Annual detection when integrated with a given series, IMP: Improved the checking of the future Upcoming list for issues marked as Wanted but not available yet and then auto-adding, IMP: Improved upon story arc checking for missing issues / searching for wanted, IMP: Enabling Annuals support now within Configuration GUI, bunch of other things....
2014-07-28 19:28:09 +00:00
annscan = myDB . select ( ' SELECT * FROM annuals WHERE ComicID=? ' , [ ComicID ] )
if annscan is None :
pass
else :
for ascan in annscan :
2014-07-30 05:11:19 +00:00
#logger.info('ReleaseComicName: ' + ascan['ReleaseComicName'])
FIX: undefined on filter box on startup, IMP: Added Meta-Tagging options on a series / issue basis on comic details screen, IMP: Issue Information is now available per issue and is extracted currently from the cbz file to display (if no cbz is present, the option isn't available), IMP: Failed Download handling is implemented and available in GUI - required to replace existing autoProcessComics.py and ComicRN.py scripts, IMP: Added ability to specify post-processing script instead of ComicRN.py, IMP: Added abilty to edit the issue date for a given issue by simply clicking on it - this will help to avoid dates that have incorrect or 0000-00-00, IMP: Story Arc searching is working (not adding yet), IMP: Added Archived/Ignored options to Upcoming/Wanted tab, IMP: Fixed some alignment and display issues on the Upcoming section, IMP: Added better directory handling for Updating Comic Location when it needs to get changed for all existing series (locmove in config.ini), IMP: Added better handling for unicode characters in series titles when searching / filechecking, IMP: When adding a new series with no data, Mylar would error out (now will add and just retain 0 issues), FIX: When year was fuzzied, search would still attempt to do a date-check comparison and say everything failed, IMP: Better handling of nzb names when retaining for post-processing comparisons, IMP: Future Upcoming now will use actual shipping date of issue if available in order to see if issue is available, FIX: If annuals were enabled, refreshing a series would put some issues into an Archived status because the actual counts would be off, IMP: When file checking, Alternate Naming would be searched last which resulted in matching incorrectly to the series title or other alternate naming for the given series, now will check the Alternate Naming first for a match, then drop back down to the series name itself otherwise, IMP: Improved Annual detection when integrated with a given series, IMP: Improved the checking of the future Upcoming list for issues marked as Wanted but not available yet and then auto-adding, IMP: Improved upon story arc checking for missing issues / searching for wanted, IMP: Enabling Annuals support now within Configuration GUI, bunch of other things....
2014-07-28 19:28:09 +00:00
if ascan [ ' ReleaseComicName ' ] not in altnames :
altnames + = ascan [ ' ReleaseComicName ' ] + ' !! ' + ascan [ ' ReleaseComicID ' ] + ' ## '
altnames = altnames [ : - 2 ]
2015-05-22 08:32:51 +00:00
logger . info ( module + ' Now checking files for ' + rescan [ ' ComicName ' ] + ' ( ' + str ( rescan [ ' ComicYear ' ] ) + ' ) in ' + rescan [ ' ComicLocation ' ] )
2014-11-21 17:16:28 +00:00
fca = [ ]
2013-02-17 10:31:18 +00:00
if archive is None :
FIX: When retrieving feeds from 32p and in Auth mode, personal notification feeds contained some invalid html entries that weren't removed properly resulting in no results when attempting to match for downloading, FIX: When searching 32P, if title had a '/' within the title - Mylar would mistakingly skip it due to some previous exceptions that were made for CBT, FIX: Main page would quickly display & hide the have% column instead of always being hidden, FIX: Adjusted some incorrect spacing for non-alphanumeric characters when comparing search results (should result in better matching hopefully), FIX: When adding a series and the most recent issue was present on the weekly-pull list, it would sometimes not mark it as Wanted and auto-attempt to search for it (if auto mark Upcoming enabled), FIX: Added Test Connection button for 32P where it will test logon credentials as well as if Captcha is present, IMP: If captcha is enabled for 32p and signon is required because keys are stale, will not send authentication information and will just bypass as a provider, IMP: Test Connection button added for SABnzbd, IMP: Added ability to directly add torrents to rtorrent and apply label + download directory options (config.ini only atm), FIX: If a search result had a 'vol.' label in it, depending on how the format of the label was mylar would refuse to remove the volume which resulted in failed matches (also fixed a similar issue with failing to remove the volume label when comparing search results), FIX: When filechecking, if a series had a - in the title, will now account for it properly, IMP: Completely redid the filecheck module which allows for integration into other modules as well as more detailed failure logs, IMP: Added Dynamic handder integration into filechecker and subsequent modules that use it which allows for special characters to be replaced with any other type of character, IMP: Manual post-processing speed improved greatly due to new usage of filecheck module, IMP: Importer backend code redone to include new filecheck module, IMP: Added status/counter to import process, IMP: Added force unlock option to importer for failed imports, IMP: Added new status to Import labelled as 'Manual Intervention' for imports that need the user to manually select an option from an available list, FIX: When import said there were search results to view, but none available - would blank screen, IMP: Added a failure log entry showing all the failed files that weren't able to be scanned in during an import (will be in GUI eventually), IMP: if only partial metadata is available during import, Mylar will attempt to use what's available from the metatagging instead of picking all of one/other, IMP: Better grouping of series/volumes when viewing the import results page as well as now indicating if annuals are present within the files, IMP: Added a file-icon beside each imported item on the import result page which allows the user to view the files that are associated with the given series grouping, IMP: Added a blacklisted_publishers option to config.ini which will blacklist specific publishers from being returned during search / import results, FIX: If duplicate dump folder had a value, but duplicate dump wasn't enabled - would still use the duplicate dump folder during post-processing runs, FIX: (#1194) Patch to allow for fixed H1 elements for title (thnx chazlarson), FIX: Removed UnRAR dependency checks in cmtagmylar since not being used anymore, FIX: Fixed a problem with non-ascii characters being recognized during a file-check in certain cases, IMP: Attmept by Mylar to grab an alternate jpg from file when viewing the issue details if it complies with the naming conventions, FIX: Fixed some metatagging issues with ComicBookLover tags not being handled properly if they didn't exist, IMP: Dupecheck now has a failback if it's comparing a cbr/cbr, cbz/cbz and cbr/cbz-priority is enabled, FIX: Quick check added for when adding/refreshing a comic that if a cover already existed, it would delete the cover prior to the attempt to retrieve it, IMP: Added some additional handling for when searching/adding fails, FIX: If a story arc didn't have proper issue dates (or invalid ones) would error out on loading the story arc main page - usually when arcs were imported using a cbl file.
2016-04-07 17:09:06 +00:00
tval = filechecker . FileChecker ( dir = rescan [ ' ComicLocation ' ] , watchcomic = rescan [ ' ComicName ' ] , Publisher = rescan [ ' ComicPublisher ' ] , AlternateSearch = altnames )
tmpval = tval . listFiles ( )
#tmpval = filechecker.listFiles(dir=rescan['ComicLocation'], watchcomic=rescan['ComicName'], Publisher=rescan['ComicPublisher'], AlternateSearch=altnames)
2014-11-21 17:16:28 +00:00
comiccnt = int ( tmpval [ ' comiccount ' ] )
2017-09-21 19:01:11 +00:00
#logger.fdebug(module + 'comiccnt is:' + str(comiccnt))
2014-11-21 17:16:28 +00:00
fca . append ( tmpval )
2017-10-28 16:33:59 +00:00
try :
if all ( [ mylar . CONFIG . MULTIPLE_DEST_DIRS is not None , mylar . CONFIG . MULTIPLE_DEST_DIRS != ' None ' , os . path . join ( mylar . CONFIG . MULTIPLE_DEST_DIRS , os . path . basename ( rescan [ ' ComicLocation ' ] ) ) != rescan [ ' ComicLocation ' ] , os . path . exists ( os . path . join ( mylar . CONFIG . MULTIPLE_DEST_DIRS , os . path . basename ( rescan [ ' ComicLocation ' ] ) ) ) ] ) :
logger . fdebug ( module + ' multiple_dest_dirs: ' + mylar . CONFIG . MULTIPLE_DEST_DIRS )
logger . fdebug ( module + ' dir: ' + rescan [ ' ComicLocation ' ] )
logger . fdebug ( module + ' os.path.basename: ' + os . path . basename ( rescan [ ' ComicLocation ' ] ) )
pathdir = os . path . join ( mylar . CONFIG . MULTIPLE_DEST_DIRS , os . path . basename ( rescan [ ' ComicLocation ' ] ) )
logger . info ( module + ' Now checking files for ' + rescan [ ' ComicName ' ] + ' ( ' + str ( rescan [ ' ComicYear ' ] ) + ' ) in : ' + pathdir )
mvals = filechecker . FileChecker ( dir = pathdir , watchcomic = rescan [ ' ComicName ' ] , Publisher = rescan [ ' ComicPublisher ' ] , AlternateSearch = altnames )
tmpv = mvals . listFiles ( )
#tmpv = filechecker.listFiles(dir=pathdir, watchcomic=rescan['ComicName'], Publisher=rescan['ComicPublisher'], AlternateSearch=altnames)
logger . fdebug ( module + ' tmpv filecount: ' + str ( tmpv [ ' comiccount ' ] ) )
comiccnt + = int ( tmpv [ ' comiccount ' ] )
fca . append ( tmpv )
except :
pass
2013-02-17 10:31:18 +00:00
else :
FIX: When retrieving feeds from 32p and in Auth mode, personal notification feeds contained some invalid html entries that weren't removed properly resulting in no results when attempting to match for downloading, FIX: When searching 32P, if title had a '/' within the title - Mylar would mistakingly skip it due to some previous exceptions that were made for CBT, FIX: Main page would quickly display & hide the have% column instead of always being hidden, FIX: Adjusted some incorrect spacing for non-alphanumeric characters when comparing search results (should result in better matching hopefully), FIX: When adding a series and the most recent issue was present on the weekly-pull list, it would sometimes not mark it as Wanted and auto-attempt to search for it (if auto mark Upcoming enabled), FIX: Added Test Connection button for 32P where it will test logon credentials as well as if Captcha is present, IMP: If captcha is enabled for 32p and signon is required because keys are stale, will not send authentication information and will just bypass as a provider, IMP: Test Connection button added for SABnzbd, IMP: Added ability to directly add torrents to rtorrent and apply label + download directory options (config.ini only atm), FIX: If a search result had a 'vol.' label in it, depending on how the format of the label was mylar would refuse to remove the volume which resulted in failed matches (also fixed a similar issue with failing to remove the volume label when comparing search results), FIX: When filechecking, if a series had a - in the title, will now account for it properly, IMP: Completely redid the filecheck module which allows for integration into other modules as well as more detailed failure logs, IMP: Added Dynamic handder integration into filechecker and subsequent modules that use it which allows for special characters to be replaced with any other type of character, IMP: Manual post-processing speed improved greatly due to new usage of filecheck module, IMP: Importer backend code redone to include new filecheck module, IMP: Added status/counter to import process, IMP: Added force unlock option to importer for failed imports, IMP: Added new status to Import labelled as 'Manual Intervention' for imports that need the user to manually select an option from an available list, FIX: When import said there were search results to view, but none available - would blank screen, IMP: Added a failure log entry showing all the failed files that weren't able to be scanned in during an import (will be in GUI eventually), IMP: if only partial metadata is available during import, Mylar will attempt to use what's available from the metatagging instead of picking all of one/other, IMP: Better grouping of series/volumes when viewing the import results page as well as now indicating if annuals are present within the files, IMP: Added a file-icon beside each imported item on the import result page which allows the user to view the files that are associated with the given series grouping, IMP: Added a blacklisted_publishers option to config.ini which will blacklist specific publishers from being returned during search / import results, FIX: If duplicate dump folder had a value, but duplicate dump wasn't enabled - would still use the duplicate dump folder during post-processing runs, FIX: (#1194) Patch to allow for fixed H1 elements for title (thnx chazlarson), FIX: Removed UnRAR dependency checks in cmtagmylar since not being used anymore, FIX: Fixed a problem with non-ascii characters being recognized during a file-check in certain cases, IMP: Attmept by Mylar to grab an alternate jpg from file when viewing the issue details if it complies with the naming conventions, FIX: Fixed some metatagging issues with ComicBookLover tags not being handled properly if they didn't exist, IMP: Dupecheck now has a failback if it's comparing a cbr/cbr, cbz/cbz and cbr/cbz-priority is enabled, FIX: Quick check added for when adding/refreshing a comic that if a cover already existed, it would delete the cover prior to the attempt to retrieve it, IMP: Added some additional handling for when searching/adding fails, FIX: If a story arc didn't have proper issue dates (or invalid ones) would error out on loading the story arc main page - usually when arcs were imported using a cbl file.
2016-04-07 17:09:06 +00:00
# files_arc = filechecker.listFiles(dir=archive, watchcomic=rescan['ComicName'], Publisher=rescan['ComicPublisher'], AlternateSearch=rescan['AlternateSearch'])
arcval = filechecker . FileChecker ( dir = archive , watchcomic = rescan [ ' ComicName ' ] , Publisher = rescan [ ' ComicPublisher ' ] , AlternateSearch = rescan [ ' AlternateSearch ' ] )
files_arc = arcval . listFiles ( )
IMP: (#1122) Added provider OMGWTFNZBS (results untested), FIX:(#1124) Experimental search results had a 1/9 (or simliar) parts as part of the header, would error out trying to create the nzb due to invalid path, FIX: When importing a series, if Move Files wasn't enabled would error out during the import process, FIX: When on Import page, if selecting Save Changes without Scanning Library would leave pop-up notification up forever, FIX: When searching for series (like A-Force), results would return 16000+ which would look like it locked up due to the excessive number of results - will now better handle such titles, and overall search results are limited to the first 1000 hits, FIX: When clicking on 'Add Series' from the pull-list, if a series was a new series with no issues published would return 0 results from CV even though the series would exist, FIX:(#1120) Story Arcs with special characters would cause error during manual post-processing run, FIX:(#1125) If year is fuzzied for a given series when searching, would error out with invalid date comparisons
2015-09-11 15:49:56 +00:00
fca . append ( files_arc )
comiccnt = int ( files_arc [ ' comiccount ' ] )
FIX: if 32p was disabled due to inability to signon, would still attempt to retrieve cached result when match occured which would then possibly result in excessive attempted logins, IMP: Added Force Type option to comicdetails page which will allow the type of series to be forced to TPB/GN format if Mylar is unable to decipher it under normal circumstances (doing this will affect how files are parsed for the given series), IMP: TPB/GN File Parser will now account for mutiple issues of the same TPB series within the given directory, IMP: When enabling/disabling Force Type, will rename existing folder to accomodate new naming folder convention if required, IMP: Started to move some of the file rename/manipulation modules into a filer.py module for more consistency, IMP: Added a to Folder Format which when triggered will use the designated Series Type, IMP:(#2130) When importing, option now available for the final series destination path to use the imported path location instead of defaulting to the Comic Location base folder - status will be no longer be Archived in such cases. Will also not show previous imported series on subsequent import runs if the import directory has not changed (thnx @CuddleBear92), IMP: Added the codebase for a Rename Preview option into the mix - GUI available, but not linked, FIX: Removed DEM from RSS scans when Public Torrents option is enabled, FIX:(#2137) When adding series, if series indicated date in a different format other than just numeric, would cause error
2018-12-10 17:27:53 +00:00
2014-11-21 17:16:28 +00:00
fcb = [ ]
fc = { }
FIX: if 32p was disabled due to inability to signon, would still attempt to retrieve cached result when match occured which would then possibly result in excessive attempted logins, IMP: Added Force Type option to comicdetails page which will allow the type of series to be forced to TPB/GN format if Mylar is unable to decipher it under normal circumstances (doing this will affect how files are parsed for the given series), IMP: TPB/GN File Parser will now account for mutiple issues of the same TPB series within the given directory, IMP: When enabling/disabling Force Type, will rename existing folder to accomodate new naming folder convention if required, IMP: Started to move some of the file rename/manipulation modules into a filer.py module for more consistency, IMP: Added a to Folder Format which when triggered will use the designated Series Type, IMP:(#2130) When importing, option now available for the final series destination path to use the imported path location instead of defaulting to the Comic Location base folder - status will be no longer be Archived in such cases. Will also not show previous imported series on subsequent import runs if the import directory has not changed (thnx @CuddleBear92), IMP: Added the codebase for a Rename Preview option into the mix - GUI available, but not linked, FIX: Removed DEM from RSS scans when Public Torrents option is enabled, FIX:(#2137) When adding series, if series indicated date in a different format other than just numeric, would cause error
2018-12-10 17:27:53 +00:00
is_cnt = myDB . select ( " SELECT COUNT(*) FROM issues WHERE ComicID=? " , [ ComicID ] )
iscnt = is_cnt [ 0 ] [ 0 ]
2014-11-21 17:16:28 +00:00
for ca in fca :
i = 0
while True :
try :
cla = ca [ ' comiclist ' ] [ i ]
except ( IndexError , KeyError ) as e :
break
2019-01-14 15:54:54 +00:00
try :
2019-07-26 00:36:44 +00:00
if all ( [ booktype == ' TPB ' , iscnt > 1 ] ) or all ( [ booktype == ' One-Shot ' , iscnt == 1 , cla [ ' JusttheDigits ' ] is None ] ) :
2019-01-14 15:54:54 +00:00
if cla [ ' SeriesVolume ' ] is not None :
just_the_digits = re . sub ( ' [^0-9] ' , ' ' , cla [ ' SeriesVolume ' ] ) . strip ( )
else :
just_the_digits = re . sub ( ' [^0-9] ' , ' ' , cla [ ' JusttheDigits ' ] ) . strip ( )
else :
just_the_digits = cla [ ' JusttheDigits ' ]
except Exception as e :
logger . warn ( ' [Exception: %s ] Unable to properly match up/retrieve issue number (or volume) for this [CS: %s ] ' % ( e , cla ) )
FIX: if 32p was disabled due to inability to signon, would still attempt to retrieve cached result when match occured which would then possibly result in excessive attempted logins, IMP: Added Force Type option to comicdetails page which will allow the type of series to be forced to TPB/GN format if Mylar is unable to decipher it under normal circumstances (doing this will affect how files are parsed for the given series), IMP: TPB/GN File Parser will now account for mutiple issues of the same TPB series within the given directory, IMP: When enabling/disabling Force Type, will rename existing folder to accomodate new naming folder convention if required, IMP: Started to move some of the file rename/manipulation modules into a filer.py module for more consistency, IMP: Added a to Folder Format which when triggered will use the designated Series Type, IMP:(#2130) When importing, option now available for the final series destination path to use the imported path location instead of defaulting to the Comic Location base folder - status will be no longer be Archived in such cases. Will also not show previous imported series on subsequent import runs if the import directory has not changed (thnx @CuddleBear92), IMP: Added the codebase for a Rename Preview option into the mix - GUI available, but not linked, FIX: Removed DEM from RSS scans when Public Torrents option is enabled, FIX:(#2137) When adding series, if series indicated date in a different format other than just numeric, would cause error
2018-12-10 17:27:53 +00:00
else :
2019-01-14 15:54:54 +00:00
fcb . append ( { " ComicFilename " : cla [ ' ComicFilename ' ] ,
" ComicLocation " : cla [ ' ComicLocation ' ] ,
" ComicSize " : cla [ ' ComicSize ' ] ,
" JusttheDigits " : just_the_digits ,
" AnnualComicID " : cla [ ' AnnualComicID ' ] } )
2014-11-21 17:16:28 +00:00
i + = 1
FIX: if 32p was disabled due to inability to signon, would still attempt to retrieve cached result when match occured which would then possibly result in excessive attempted logins, IMP: Added Force Type option to comicdetails page which will allow the type of series to be forced to TPB/GN format if Mylar is unable to decipher it under normal circumstances (doing this will affect how files are parsed for the given series), IMP: TPB/GN File Parser will now account for mutiple issues of the same TPB series within the given directory, IMP: When enabling/disabling Force Type, will rename existing folder to accomodate new naming folder convention if required, IMP: Started to move some of the file rename/manipulation modules into a filer.py module for more consistency, IMP: Added a to Folder Format which when triggered will use the designated Series Type, IMP:(#2130) When importing, option now available for the final series destination path to use the imported path location instead of defaulting to the Comic Location base folder - status will be no longer be Archived in such cases. Will also not show previous imported series on subsequent import runs if the import directory has not changed (thnx @CuddleBear92), IMP: Added the codebase for a Rename Preview option into the mix - GUI available, but not linked, FIX: Removed DEM from RSS scans when Public Torrents option is enabled, FIX:(#2137) When adding series, if series indicated date in a different format other than just numeric, would cause error
2018-12-10 17:27:53 +00:00
2014-11-21 17:16:28 +00:00
fc [ ' comiclist ' ] = fcb
2012-09-13 15:27:34 +00:00
havefiles = 0
2017-10-25 18:33:56 +00:00
if mylar . CONFIG . ANNUALS_ON :
IMP: Ability to now specify search provider order (regardless of torrents or nzb) within the config.ini, IMP: (#667) Changed the db module to try to accomodate db locking errors and lowering the amount of actual write transactions that were committed along with a new scheduler system, IMP: Changed sabnzbd directory to post-processing, and included subdirs for sabnzbd & nzbget ComicRN scripts, IMP: NZBGet Post-Processing ComicRN.py script (updated for use with nzbget v11.0+)added & updated in post-processing/nzbget directory (thnx ministoat), FIX: If Issue Location was None, and status was Downloaded would cause error in GUI and break series, IMP: (#689) Minimum # of seeders added (will work with KAT), IMP: (#680) Added Boxcar 2 IO Notifications, IMP: Added PushBullet Notifications, IMP: Cleaned up some notification messages so it's not so cluttered, IMP: Added Clickable series link in History tab, IMP: Added Post-Processed as a status to History tab to show manually post-processed items, IMP: Removed log level dropdown from Logs page & added 'ThreadName' as a column, IMP: Added Force Check Availability & View Future Pull-list to Upcoming sub-tabs, IMP: Added '--safe' option to startup options which will redirect directly to Manage Comics screen incase things are broken, FIX: Added proper month conversions for manual post-processing when doing comparitive issue analysis for matches, FIX: (#613) Allow for negative issue numbers in post-processing when renaming and issue padding is enabled, FIX: File Permissions on post-processing would stop post-processing if couldn't change, now will just log the error and continue, IMP: Added Scheduler (from sickbeard) to allow for threadnaming and better scheduling, IMP: Filenames in the format of ' () ' will now get scanned in, IMP: During manual post-processing will now stop looking for matches upon a successful match, IMP: A Refresh/Weeklypull series check will now just scan in issue data, instead of series info,etc, IMP: Removed some legacy GCD code that is no longer in use, IMP: Exception/traceback handling will now be logged, FIX: Unable to grab torrents from KAT due to content-encoding detection failing, IMP: Added universal date-time conversion to allow for non-english based dates to be properly compared when checking search results against publication dates, FIX: Annuals will now get proper notification (prior was leaving out the word 'annual' from notification/logs), IMP: Improved future pull-list detection and increased retension (now ~5 months), IMP: Will now mark new issues as Wanted on a Refresh Series if autowant upcoming is enabled (was reverting to a status of None previously), IMP: Cannot change status to Downloaded if current status is Skipped or Wanted, FIX: (#704) UnSkipped will now work (X in options column on comic details page), IMP: future_check will check upcoming future issues (future pull-list) that have no series data yet (ie. #1's) and auto-add them to watchlist when the data is available and auto-want accordingly, IMP: (#706) Downloading issues to local machine (via comicdetails screen) with special characters in filename now will work, IMP: improved comparison checks during weekly pull list and improved speed abit since only refreshing issue data now instead of entire series, Other Referenced issues: (#670)(#690) and some others....
2014-05-25 18:32:11 +00:00
an_cnt = myDB . select ( " SELECT COUNT(*) FROM annuals WHERE ComicID=? " , [ ComicID ] )
2013-09-09 04:39:08 +00:00
anncnt = an_cnt [ 0 ] [ 0 ]
else :
anncnt = 0
2014-11-21 17:16:28 +00:00
fccnt = comiccnt #int(fc['comiccount'])
2012-09-13 15:27:34 +00:00
issnum = 1
fcnew = [ ]
2012-10-25 06:44:51 +00:00
fn = 0
2013-01-03 10:01:07 +00:00
issuedupechk = [ ]
2013-07-01 05:19:15 +00:00
annualdupechk = [ ]
2013-04-07 18:06:36 +00:00
issueexceptdupechk = [ ]
IMP: (#844)Annuals now deleted from table on a Delete Series command, IMP: (#837)Better volume detection when searching for issues, IMP: (#842)(#808) Added some handling for issue titles when searching (it's not perfect, but it works for most), IMP: (#836)Added .cb7 for filechecking purposes only, IMP: (#830)issue numbers that are -1 can now be renamed, FIX: 'bi-annual' now fixed along with some other improvements for annual handling, IMP: Added version check for autoProcessComics and ComicRN - in order to help warn users of changes and the need to update these files since they are usually outside of the mylar git directory when being used, FIX:(#840) Boxcar2 should be working again, FIX: (#845) If search returned a filename that had no distinction between issue number and issue title and/or extra information, would error out if any of the words contained NOW, IMP: (#823) Added some better detection for NZBGet parameters when using ComicRN.py
2014-10-06 18:10:36 +00:00
mc_issue = [ ]
mc_issuenumber = [ ]
multiple_check = myDB . select ( ' SELECT * FROM issues WHERE ComicID=? GROUP BY Int_IssueNumber HAVING (COUNT(Int_IssueNumber) > 1) ' , [ ComicID ] )
2015-05-22 08:32:51 +00:00
if len ( multiple_check ) == 0 :
IMP: (#844)Annuals now deleted from table on a Delete Series command, IMP: (#837)Better volume detection when searching for issues, IMP: (#842)(#808) Added some handling for issue titles when searching (it's not perfect, but it works for most), IMP: (#836)Added .cb7 for filechecking purposes only, IMP: (#830)issue numbers that are -1 can now be renamed, FIX: 'bi-annual' now fixed along with some other improvements for annual handling, IMP: Added version check for autoProcessComics and ComicRN - in order to help warn users of changes and the need to update these files since they are usually outside of the mylar git directory when being used, FIX:(#840) Boxcar2 should be working again, FIX: (#845) If search returned a filename that had no distinction between issue number and issue title and/or extra information, would error out if any of the words contained NOW, IMP: (#823) Added some better detection for NZBGet parameters when using ComicRN.py
2014-10-06 18:10:36 +00:00
logger . fdebug ( ' No issues with identical issue numbering were detected for this series ' )
mc_issuenumber = None
else :
logger . fdebug ( ' Multiple issues with identical numbering were detected. Attempting to accomodate. ' )
for mc in multiple_check :
mc_issuenumber . append ( { " Int_IssueNumber " : mc [ ' Int_IssueNumber ' ] } )
if not mc_issuenumber is None :
for mciss in mc_issuenumber :
mchk = myDB . select ( ' SELECT * FROM issues WHERE ComicID=? AND Int_IssueNumber=? ' , [ ComicID , mciss [ ' Int_IssueNumber ' ] ] )
for mck in mchk :
mc_issue . append ( { " Int_IssueNumber " : mck [ ' Int_IssueNumber ' ] ,
" IssueYear " : mck [ ' IssueDate ' ] [ : 4 ] ,
" IssueID " : mck [ ' IssueID ' ] } )
2017-05-04 18:41:05 +00:00
mc_annual = [ ]
mc_annualnumber = [ ]
2017-10-25 18:33:56 +00:00
if mylar . CONFIG . ANNUALS_ON :
2017-05-04 18:41:05 +00:00
mult_ann_check = myDB . select ( ' SELECT * FROM annuals WHERE ComicID=? GROUP BY Int_IssueNumber HAVING (COUNT(Int_IssueNumber) > 1) ' , [ ComicID ] )
if len ( mult_ann_check ) == 0 :
logger . fdebug ( ' [ANNUAL-CHK] No annuals with identical issue numbering across annual volumes were detected for this series ' )
mc_annualnumber = None
else :
logger . fdebug ( ' [ANNUAL-CHK] Multiple issues with identical numbering were detected across multiple annual volumes. Attempting to accomodate. ' )
for mc in mult_ann_check :
mc_annualnumber . append ( { " Int_IssueNumber " : mc [ ' Int_IssueNumber ' ] } )
if not mc_annualnumber is None :
for mcann in mc_annualnumber :
achk = myDB . select ( ' SELECT * FROM annuals WHERE ComicID=? AND Int_IssueNumber=? ' , [ ComicID , mcann [ ' Int_IssueNumber ' ] ] )
for ack in achk :
mc_annual . append ( { " Int_IssueNumber " : ack [ ' Int_IssueNumber ' ] ,
" IssueYear " : ack [ ' IssueDate ' ] [ : 4 ] ,
" IssueID " : ack [ ' IssueID ' ] ,
" ReleaseComicID " : ack [ ' ReleaseComicID ' ] } )
IMP: (#844)Annuals now deleted from table on a Delete Series command, IMP: (#837)Better volume detection when searching for issues, IMP: (#842)(#808) Added some handling for issue titles when searching (it's not perfect, but it works for most), IMP: (#836)Added .cb7 for filechecking purposes only, IMP: (#830)issue numbers that are -1 can now be renamed, FIX: 'bi-annual' now fixed along with some other improvements for annual handling, IMP: Added version check for autoProcessComics and ComicRN - in order to help warn users of changes and the need to update these files since they are usually outside of the mylar git directory when being used, FIX:(#840) Boxcar2 should be working again, FIX: (#845) If search returned a filename that had no distinction between issue number and issue title and/or extra information, would error out if any of the words contained NOW, IMP: (#823) Added some better detection for NZBGet parameters when using ComicRN.py
2014-10-06 18:10:36 +00:00
2017-09-21 19:01:11 +00:00
#logger.fdebug('mc_issue:' + str(mc_issue))
#logger.fdebug('mc_annual:' + str(mc_annual))
IMP: (#844)Annuals now deleted from table on a Delete Series command, IMP: (#837)Better volume detection when searching for issues, IMP: (#842)(#808) Added some handling for issue titles when searching (it's not perfect, but it works for most), IMP: (#836)Added .cb7 for filechecking purposes only, IMP: (#830)issue numbers that are -1 can now be renamed, FIX: 'bi-annual' now fixed along with some other improvements for annual handling, IMP: Added version check for autoProcessComics and ComicRN - in order to help warn users of changes and the need to update these files since they are usually outside of the mylar git directory when being used, FIX:(#840) Boxcar2 should be working again, FIX: (#845) If search returned a filename that had no distinction between issue number and issue title and/or extra information, would error out if any of the words contained NOW, IMP: (#823) Added some better detection for NZBGet parameters when using ComicRN.py
2014-10-06 18:10:36 +00:00
2013-08-06 00:53:10 +00:00
issID_to_ignore = [ ]
2013-08-06 03:20:09 +00:00
issID_to_ignore . append ( str ( ComicID ) )
IMP: Ability to now specify search provider order (regardless of torrents or nzb) within the config.ini, IMP: (#667) Changed the db module to try to accomodate db locking errors and lowering the amount of actual write transactions that were committed along with a new scheduler system, IMP: Changed sabnzbd directory to post-processing, and included subdirs for sabnzbd & nzbget ComicRN scripts, IMP: NZBGet Post-Processing ComicRN.py script (updated for use with nzbget v11.0+)added & updated in post-processing/nzbget directory (thnx ministoat), FIX: If Issue Location was None, and status was Downloaded would cause error in GUI and break series, IMP: (#689) Minimum # of seeders added (will work with KAT), IMP: (#680) Added Boxcar 2 IO Notifications, IMP: Added PushBullet Notifications, IMP: Cleaned up some notification messages so it's not so cluttered, IMP: Added Clickable series link in History tab, IMP: Added Post-Processed as a status to History tab to show manually post-processed items, IMP: Removed log level dropdown from Logs page & added 'ThreadName' as a column, IMP: Added Force Check Availability & View Future Pull-list to Upcoming sub-tabs, IMP: Added '--safe' option to startup options which will redirect directly to Manage Comics screen incase things are broken, FIX: Added proper month conversions for manual post-processing when doing comparitive issue analysis for matches, FIX: (#613) Allow for negative issue numbers in post-processing when renaming and issue padding is enabled, FIX: File Permissions on post-processing would stop post-processing if couldn't change, now will just log the error and continue, IMP: Added Scheduler (from sickbeard) to allow for threadnaming and better scheduling, IMP: Filenames in the format of ' () ' will now get scanned in, IMP: During manual post-processing will now stop looking for matches upon a successful match, IMP: A Refresh/Weeklypull series check will now just scan in issue data, instead of series info,etc, IMP: Removed some legacy GCD code that is no longer in use, IMP: Exception/traceback handling will now be logged, FIX: Unable to grab torrents from KAT due to content-encoding detection failing, IMP: Added universal date-time conversion to allow for non-english based dates to be properly compared when checking search results against publication dates, FIX: Annuals will now get proper notification (prior was leaving out the word 'annual' from notification/logs), IMP: Improved future pull-list detection and increased retension (now ~5 months), IMP: Will now mark new issues as Wanted on a Refresh Series if autowant upcoming is enabled (was reverting to a status of None previously), IMP: Cannot change status to Downloaded if current status is Skipped or Wanted, FIX: (#704) UnSkipped will now work (X in options column on comic details page), IMP: future_check will check upcoming future issues (future pull-list) that have no series data yet (ie. #1's) and auto-add them to watchlist when the data is available and auto-want accordingly, IMP: (#706) Downloading issues to local machine (via comicdetails screen) with special characters in filename now will work, IMP: improved comparison checks during weekly pull list and improved speed abit since only refreshing issue data now instead of entire series, Other Referenced issues: (#670)(#690) and some others....
2014-05-25 18:32:11 +00:00
issID_to_write = [ ]
IMP: 32P legacy mode (rss feed only) and Auth mode (user/pass) available now. With Auth mode, will monitor all of personal notification feeds, IMP: Changed access methods to 32P for retrieving links, FIX:(#1007) Fixed error for one-off downloading via weekly pull list, FIX: (#1004) Fix for post-processing errors (missing files), FIX: (#1001) Extra scripts & pre-scripts will now accept bash scripts, FIX: Annuals were being incorrectly tallied in Have total as well as not having status correctly updated, IMP: Fixed some discrepencies with the provider_order generation line, IMP: Made some additional adjustments to the file-checking to account for some additional variations in naming, FIX: Fixed traceback errors when folder monitor was running and wouldn't be able to complete the post-processing properly, FIX: Cleanup code and some other small fixes....
2015-05-09 02:39:28 +00:00
ANNComicID = None
IMP: Ability to now specify search provider order (regardless of torrents or nzb) within the config.ini, IMP: (#667) Changed the db module to try to accomodate db locking errors and lowering the amount of actual write transactions that were committed along with a new scheduler system, IMP: Changed sabnzbd directory to post-processing, and included subdirs for sabnzbd & nzbget ComicRN scripts, IMP: NZBGet Post-Processing ComicRN.py script (updated for use with nzbget v11.0+)added & updated in post-processing/nzbget directory (thnx ministoat), FIX: If Issue Location was None, and status was Downloaded would cause error in GUI and break series, IMP: (#689) Minimum # of seeders added (will work with KAT), IMP: (#680) Added Boxcar 2 IO Notifications, IMP: Added PushBullet Notifications, IMP: Cleaned up some notification messages so it's not so cluttered, IMP: Added Clickable series link in History tab, IMP: Added Post-Processed as a status to History tab to show manually post-processed items, IMP: Removed log level dropdown from Logs page & added 'ThreadName' as a column, IMP: Added Force Check Availability & View Future Pull-list to Upcoming sub-tabs, IMP: Added '--safe' option to startup options which will redirect directly to Manage Comics screen incase things are broken, FIX: Added proper month conversions for manual post-processing when doing comparitive issue analysis for matches, FIX: (#613) Allow for negative issue numbers in post-processing when renaming and issue padding is enabled, FIX: File Permissions on post-processing would stop post-processing if couldn't change, now will just log the error and continue, IMP: Added Scheduler (from sickbeard) to allow for threadnaming and better scheduling, IMP: Filenames in the format of ' () ' will now get scanned in, IMP: During manual post-processing will now stop looking for matches upon a successful match, IMP: A Refresh/Weeklypull series check will now just scan in issue data, instead of series info,etc, IMP: Removed some legacy GCD code that is no longer in use, IMP: Exception/traceback handling will now be logged, FIX: Unable to grab torrents from KAT due to content-encoding detection failing, IMP: Added universal date-time conversion to allow for non-english based dates to be properly compared when checking search results against publication dates, FIX: Annuals will now get proper notification (prior was leaving out the word 'annual' from notification/logs), IMP: Improved future pull-list detection and increased retension (now ~5 months), IMP: Will now mark new issues as Wanted on a Refresh Series if autowant upcoming is enabled (was reverting to a status of None previously), IMP: Cannot change status to Downloaded if current status is Skipped or Wanted, FIX: (#704) UnSkipped will now work (X in options column on comic details page), IMP: future_check will check upcoming future issues (future pull-list) that have no series data yet (ie. #1's) and auto-add them to watchlist when the data is available and auto-want accordingly, IMP: (#706) Downloading issues to local machine (via comicdetails screen) with special characters in filename now will work, IMP: improved comparison checks during weekly pull list and improved speed abit since only refreshing issue data now instead of entire series, Other Referenced issues: (#670)(#690) and some others....
2014-05-25 18:32:11 +00:00
2017-05-04 18:41:05 +00:00
reissues = myDB . select ( ' SELECT * FROM issues WHERE ComicID=? ' , [ ComicID ] )
2015-05-22 08:32:51 +00:00
while ( fn < fccnt ) :
2012-10-25 06:44:51 +00:00
haveissue = " no "
2013-01-03 10:01:07 +00:00
issuedupe = " no "
IMP: Cleaned up interface for StoryArcs / Story Arc Details, IMP: Cleaned up interface for Reading list Management, IMP: Added better reading list management - new status (added, downloaded, read), IMP: Added sync option for use with another device for reading list transfer (ie. tablet) Android only, IMP: Autopopulate new weekly pull releases to reading list, IMP: 'Watch' option in weekly pull list now fully functional. Will watch CV for series' that do not have any series data yet as they are new starting series. Will auto-add once available, IMP: Auto-watch check is run after every refresh/recreate of the weeklypull list, IMP: Improved the Add a Series option where it will now look for issues that are 'new' or 'wanted' during add sequence, IMP: Main page interface now has coloured have/total bars to denote series completion, IMP: New scheduler / threading locks in place in an attempt to avoid database locks, FIX: Removed some erroneous locking that was going on when importing a directory was being run, IMP: Stat counter now present when post-processing multiple issues in sequence, FIX: for issue number error when post-processing and issue number was a non-alphanumeric, FIX: for metatagging: when original file was .cbz, would try to convert and fail, FIX: for issues that were negative and were preceeded by a # in the filename (filechecker), FIX: for publisher having non-alphanumeric character in name when attempting to determine publisher, FIX: if annuals enabled, would incorrectly show as being 'already in library' when viewing search results if results constained annuals, FIX:(#944) for incorrect nzbname being used when post-processing was being performed from an nzb client (experimental mainly), IMP: Turned off logging for ComicVine API counter, FIX: Added retry attempts when connecting to ComicVine in order to avoid errors when adding a series, IMP:(#963) Added ability to add snatched to filter when viewing Wanted issues on Wanted tab, FIX: When importing and then selecting a series to import via the select screen, will now flip back to the importresults and add the selected series in the background, IMP:(#952) Main page is now sorted in ascending order by Continuing/Ended status (and subbed by whether is Active/Paused).Custom sorting is still available, FIX: Dupecheck will now automatically assume existing 0-byte files are to be overwritten when performing post-processing, FIX: If publication date for series contained a '?' (usually with brand new series) will force to 'Present' to allow for pull-list comparisons to take place, FIX: Mylar will now disallow search results which have 'covers only' or 'variant' in the filename, IMP: Better nzbname generation/retrieval (will check inside nzb for possible names) to be used when post-processing, IMP: DB Update will now perform update to all active comics in descending order by Latest Date (instead of random order), FIX: Enforce the 5hr limit rule when running DB update (will only update series that haven't been updated in >5 hours), FIX: Annuals will now have/retain the proper status upon doing DB Update, FIX: Have totals will now be updated when doing a recheck files (sometimes wouldn't get updated depending on various states of status'), FIX:(#966) Added urllib2.URLError exeception trap when attempting to check Git for updates, IMP: Removed the individual sqlite calls for weeklypull, and brought them into line with using the db module (which will minimize concurrent access, which seemed to be causing db locks), IMP: Cleaned up some code and shuffled some functions so they are in more appropriate locations
2015-03-27 17:27:59 +00:00
annualdupe = " no "
2012-09-24 05:17:29 +00:00
try :
2012-10-25 06:44:51 +00:00
tmpfc = fc [ ' comiclist ' ] [ fn ]
2012-09-24 05:17:29 +00:00
except IndexError :
IMP:(#753) Better handling of cbz files when using metatagging option, IMP: Added option for MetaTagging to enable/disable overwriting of existing metadata on cbz files, IMP: Post-Processing will now delete folders that are identical to filenames after successful post-processing, FIX:(#756) Fix for Manual Search for an Issue which resulted in traceback errors upon not finding any matches (Windows), IMP: Added module-name to logging for some modules to help troubleshooting (eventually all will be done), IMP: If a filename contained repeated characters for spaces (ie. ___ or ...), it would fail on post-processing due to the nzbname being off, IMP: Monitoring of CV API will now be persitent on restarts/shutdowns/etc, IMP: Added a maximum CV API global so that it can be adjusted for monitoring more easily should API requirements change, IMP: When Adding new series, will now only mark issues that are actually upcoming as Wanted (based on Store Date)
2014-06-22 08:32:45 +00:00
logger . fdebug ( module + ' Unable to properly retrieve a file listing for the given series. ' )
logger . fdebug ( module + ' Probably because the filenames being scanned are not in a parseable format ' )
2015-05-22 08:32:51 +00:00
if fn == 0 :
IMP: Added ForceRSS Check and Test SABnzbd Connection buttons in Config, FIX: If Annuals not enabled, would error on home screen, IMP: updated mylar.init.d (thnx Kalinon), FIX: Manual Post-Processing fix for Manual Run (thnx Kalinon), IMP: Library Monitor working (check folder every X minutes and Post-Process), IMP: Future Upcoming introduction, IMP: Experimental search better handling of year inclusions, FIX: Filechecker will now pick up series with years in the series title accordingly, FIX: Torrent seedbox sending would lockup occassionally when attempting to send torrent file, FIX: malformed image url on some series, IMP: Moved issue updating to a seperate function, IMP: When series was refreshed, would download the last issue (or few issues depending on date), regardless of status, IMP: When series is volume 1 or volume label doesn't exist, either assume V1 or remove volume requirements to improve matching hits, IMP: StoryArcs will now check in StoryArc folder for existing issues and change status in StoryArc accordingly...
2013-11-28 15:48:59 +00:00
return
else :
break
2019-01-15 02:52:53 +00:00
2019-01-14 15:54:54 +00:00
if tmpfc [ ' JusttheDigits ' ] is not None :
temploc = tmpfc [ ' JusttheDigits ' ] . replace ( ' _ ' , ' ' )
temploc = re . sub ( ' [ \ # \' ] ' , ' ' , temploc )
logger . fdebug ( ' temploc: %s ' % temploc )
else :
2019-01-15 02:52:53 +00:00
#assume 1 if not given
if any ( [ booktype == ' TPB ' , booktype == ' One-Shot ' ] ) :
temploc = ' 1 '
else :
temploc = None
FIX:(#2179) Post-processing item would fail if match would occur on story-arc check, FIX: Fixed some sub-directory problems when doing various types of scans, IMP: Added booktype to filechecker parsing results, FIX: When downloading via DDL, would not adhere to the booktype as a restraint, IMP: Pack support added for DDL (available as a per series option), IMP: Added BookType & Aliases to the arc's section which will impact how issues/series are searched/post-processed/cheked when they're an issue from an arc, IMP: Initial codebase for the a queue manager section, IMP: Write DDL-Queue data to the sql table so that stalled/broken downloads can be resumed/deleted etc eventually, FIX: If a filename didn't have a valid issue number and it is a Print Edition, will now throw a warning indicating other options to try instead of causing a traceback, IMP: Updated snatch notifications so the notification header will just say 'Issue Snatched' with a brief description, FIX: Removed multiple import db lines from the helpers module, IMP: cleanup_cache variable (true/false) added to config section which will initiate a cleanup of items in the cache directory on startup which will remove items that are no longer needed, IMP: Changed some logging string concatenation lines to try and avoid traceback errors due to logging
2019-02-01 21:25:24 +00:00
logger . warn ( ' The filename [ %s ] does not have a valid issue number, and the Edition of the series is %s . You might need to Forcibly Mark the Series as TPB/GN and try this again. ' % ( tmpfc [ ' ComicFilename ' ] , rescan [ ' Type ' ] ) )
return
2019-01-14 15:54:54 +00:00
FIX: (#2006)(#1986) Auto-Want series were not being auto-added, and even after manually added would result in count error after sucessful post-processing, FIX: When filechecking, if a 'special' was included as part of the annuals of a given series, it would not be detected, IMP: (#1982) When ComicRN & Completed Download Handling (CDH) are both enabled, will now check to see if ComicRN is active for the given category in the respective client. If it is, CDH will not post-process and issue a warning and let ComicRN handle the post-processing, FIX: Fixed a utf-8 encoding error when 'Run script AFTER snatch' was enabled, FIX: (#2004) Possible fix for 500 error on Wanted tab, when series had an annual/special attached but was located in the incorrect table, FIX: Removed some logging lines when CDH is monitoring downloads
2018-06-30 18:35:52 +00:00
if all ( [ ' annual ' not in temploc . lower ( ) , ' special ' not in temploc . lower ( ) ] ) :
2013-02-06 19:55:23 +00:00
#remove the extension here
2015-05-22 08:32:51 +00:00
extensions = ( ' .cbr ' , ' .cbz ' , ' .cb7 ' )
2013-02-06 19:55:23 +00:00
if temploc . lower ( ) . endswith ( extensions ) :
2016-04-17 17:17:40 +00:00
logger . fdebug ( module + ' Removed extension for issue: ' + temploc )
2013-07-30 04:57:37 +00:00
temploc = temploc [ : - 4 ]
fcnew_af = re . findall ( ' [^ \ ()]+ ' , temploc )
fcnew = shlex . split ( fcnew_af [ 0 ] )
2012-10-25 06:44:51 +00:00
fcn = len ( fcnew )
n = 0
2015-01-16 19:40:08 +00:00
while True :
2012-10-25 06:44:51 +00:00
try :
reiss = reissues [ n ]
2019-01-16 19:23:04 +00:00
int_iss = None
2012-10-25 06:44:51 +00:00
except IndexError :
break
2013-07-01 05:19:15 +00:00
int_iss = helpers . issuedigits ( reiss [ ' Issue_Number ' ] )
2012-10-25 06:44:51 +00:00
issyear = reiss [ ' IssueDate ' ] [ : 4 ]
old_status = reiss [ ' Status ' ]
2013-07-01 05:19:15 +00:00
issname = reiss [ ' IssueName ' ]
2013-01-07 19:58:34 +00:00
2013-04-02 08:56:24 +00:00
fnd_iss_except = ' None '
2015-05-22 08:32:51 +00:00
2019-01-15 02:52:53 +00:00
if temploc is not None :
fcdigit = helpers . issuedigits ( temploc )
elif any ( [ booktype == ' TPB ' , booktype == ' One-Shot ' ] ) and temploc is None :
fcdigit = helpers . issuedigits ( ' 1 ' )
2015-01-16 19:40:08 +00:00
if int ( fcdigit ) == int_iss :
logger . fdebug ( module + ' [ ' + str ( reiss [ ' IssueID ' ] ) + ' ] Issue match - fcdigit: ' + str ( fcdigit ) + ' ... int_iss: ' + str ( int_iss ) )
2013-07-01 05:19:15 +00:00
2015-01-16 19:40:08 +00:00
if ' - ' in temploc and temploc . find ( reiss [ ' Issue_Number ' ] ) > temploc . find ( ' - ' ) :
logger . fdebug ( module + ' I have detected a possible Title in the filename ' )
logger . fdebug ( module + ' the issue # has occured after the -, so I assume that it is part of the Title ' )
break
2013-07-01 05:19:15 +00:00
2015-01-16 19:40:08 +00:00
#baseline these to default to normal scanning
multiplechk = False
issuedupe = " no "
foundchk = False
#check here if muliple identical numbering issues exist for the series
if len ( mc_issue ) > 1 :
for mi in mc_issue :
if mi [ ' Int_IssueNumber ' ] == int_iss :
if mi [ ' IssueID ' ] == reiss [ ' IssueID ' ] :
logger . fdebug ( module + ' IssueID matches to multiple issues : ' + str ( mi [ ' IssueID ' ] ) + ' . Checking dupe. ' )
logger . fdebug ( module + ' miISSUEYEAR: ' + str ( mi [ ' IssueYear ' ] ) + ' -- issyear : ' + str ( issyear ) )
if any ( mi [ ' IssueID ' ] == d [ ' issueid ' ] for d in issuedupechk ) :
logger . fdebug ( module + ' IssueID already within dupe. Checking next if available. ' )
2015-05-22 08:32:51 +00:00
multiplechk = True
2015-01-16 19:40:08 +00:00
break
if ( mi [ ' IssueYear ' ] in tmpfc [ ' ComicFilename ' ] ) and ( issyear == mi [ ' IssueYear ' ] ) :
logger . fdebug ( module + ' Matched to year within filename : ' + str ( issyear ) )
multiplechk = False
break
else :
logger . fdebug ( module + ' Did not match to year within filename : ' + str ( issyear ) )
multiplechk = True
if multiplechk == True :
n + = 1
continue
#this will detect duplicate filenames within the same directory.
for di in issuedupechk :
2019-05-09 14:36:29 +00:00
if di [ ' fcdigit ' ] == fcdigit and di [ ' issueid ' ] == reiss [ ' IssueID ' ] :
2015-01-16 19:40:08 +00:00
#base off of config - base duplication keep on filesize or file-type (or both)
logger . fdebug ( ' [DUPECHECK] Duplicate issue detected [ ' + di [ ' filename ' ] + ' ] [ ' + tmpfc [ ' ComicFilename ' ] + ' ] ' )
2017-10-25 18:33:56 +00:00
# mylar.CONFIG.DUPECONSTRAINT = 'filesize' / 'filetype-cbr' / 'filetype-cbz'
logger . fdebug ( ' [DUPECHECK] Based on duplication preferences I will retain based on : ' + mylar . CONFIG . DUPECONSTRAINT )
2015-01-16 19:40:08 +00:00
removedupe = False
2017-10-25 18:33:56 +00:00
if ' cbr ' in mylar . CONFIG . DUPECONSTRAINT or ' cbz ' in mylar . CONFIG . DUPECONSTRAINT :
if ' cbr ' in mylar . CONFIG . DUPECONSTRAINT :
2015-01-16 19:40:08 +00:00
#this has to be configured in config - either retain cbr or cbz.
if tmpfc [ ' ComicFilename ' ] . endswith ( ' .cbz ' ) :
#keep di['filename']
logger . fdebug ( ' [DUPECHECK-CBR PRIORITY] [# ' + reiss [ ' Issue_Number ' ] + ' ] Retaining currently scanned in file : ' + di [ ' filename ' ] )
issuedupe = " yes "
break
else :
#keep tmpfc['ComicFilename']
logger . fdebug ( ' [DUPECHECK-CBR PRIORITY] [# ' + reiss [ ' Issue_Number ' ] + ' ] Retaining newly scanned in file : ' + tmpfc [ ' ComicFilename ' ] )
removedupe = True
2017-10-25 18:33:56 +00:00
elif ' cbz ' in mylar . CONFIG . DUPECONSTRAINT :
2015-01-16 19:40:08 +00:00
if tmpfc [ ' ComicFilename ' ] . endswith ( ' .cbr ' ) :
#keep di['filename']
logger . fdebug ( ' [DUPECHECK-CBZ PRIORITY] [# ' + reiss [ ' Issue_Number ' ] + ' ] Retaining currently scanned in filename : ' + di [ ' filename ' ] )
issuedupe = " yes "
break
else :
#keep tmpfc['ComicFilename']
logger . fdebug ( ' [DUPECHECK-CBZ PRIORITY] [# ' + reiss [ ' Issue_Number ' ] + ' ] Retaining newly scanned in filename : ' + tmpfc [ ' ComicFilename ' ] )
removedupe = True
2017-10-25 18:33:56 +00:00
if mylar . CONFIG . DUPECONSTRAINT == ' filesize ' :
2015-02-09 18:44:10 +00:00
if tmpfc [ ' ComicSize ' ] < = di [ ' filesize ' ] :
logger . fdebug ( ' [DUPECHECK-FILESIZE PRIORITY] [# ' + reiss [ ' Issue_Number ' ] + ' ] Retaining currently scanned in filename : ' + di [ ' filename ' ] )
issuedupe = " yes "
2015-01-16 19:40:08 +00:00
break
2015-02-09 18:44:10 +00:00
else :
logger . fdebug ( ' [DUPECHECK-FILESIZE PRIORITY] [# ' + reiss [ ' Issue_Number ' ] + ' ] Retaining newly scanned in filename : ' + tmpfc [ ' ComicFilename ' ] )
removedupe = True
if removedupe :
#need to remove the entry from issuedupechk so can add new one.
#tuple(y for y in x if y) for x in a
issuedupe_temp = [ ]
tmphavefiles = 0
for x in issuedupechk :
IMP: Cleaned up interface for StoryArcs / Story Arc Details, IMP: Cleaned up interface for Reading list Management, IMP: Added better reading list management - new status (added, downloaded, read), IMP: Added sync option for use with another device for reading list transfer (ie. tablet) Android only, IMP: Autopopulate new weekly pull releases to reading list, IMP: 'Watch' option in weekly pull list now fully functional. Will watch CV for series' that do not have any series data yet as they are new starting series. Will auto-add once available, IMP: Auto-watch check is run after every refresh/recreate of the weeklypull list, IMP: Improved the Add a Series option where it will now look for issues that are 'new' or 'wanted' during add sequence, IMP: Main page interface now has coloured have/total bars to denote series completion, IMP: New scheduler / threading locks in place in an attempt to avoid database locks, FIX: Removed some erroneous locking that was going on when importing a directory was being run, IMP: Stat counter now present when post-processing multiple issues in sequence, FIX: for issue number error when post-processing and issue number was a non-alphanumeric, FIX: for metatagging: when original file was .cbz, would try to convert and fail, FIX: for issues that were negative and were preceeded by a # in the filename (filechecker), FIX: for publisher having non-alphanumeric character in name when attempting to determine publisher, FIX: if annuals enabled, would incorrectly show as being 'already in library' when viewing search results if results constained annuals, FIX:(#944) for incorrect nzbname being used when post-processing was being performed from an nzb client (experimental mainly), IMP: Turned off logging for ComicVine API counter, FIX: Added retry attempts when connecting to ComicVine in order to avoid errors when adding a series, IMP:(#963) Added ability to add snatched to filter when viewing Wanted issues on Wanted tab, FIX: When importing and then selecting a series to import via the select screen, will now flip back to the importresults and add the selected series in the background, IMP:(#952) Main page is now sorted in ascending order by Continuing/Ended status (and subbed by whether is Active/Paused).Custom sorting is still available, FIX: Dupecheck will now automatically assume existing 0-byte files are to be overwritten when performing post-processing, FIX: If publication date for series contained a '?' (usually with brand new series) will force to 'Present' to allow for pull-list comparisons to take place, FIX: Mylar will now disallow search results which have 'covers only' or 'variant' in the filename, IMP: Better nzbname generation/retrieval (will check inside nzb for possible names) to be used when post-processing, IMP: DB Update will now perform update to all active comics in descending order by Latest Date (instead of random order), FIX: Enforce the 5hr limit rule when running DB update (will only update series that haven't been updated in >5 hours), FIX: Annuals will now have/retain the proper status upon doing DB Update, FIX: Have totals will now be updated when doing a recheck files (sometimes wouldn't get updated depending on various states of status'), FIX:(#966) Added urllib2.URLError exeception trap when attempting to check Git for updates, IMP: Removed the individual sqlite calls for weeklypull, and brought them into line with using the db module (which will minimize concurrent access, which seemed to be causing db locks), IMP: Cleaned up some code and shuffled some functions so they are in more appropriate locations
2015-03-27 17:27:59 +00:00
#logger.fdebug('Comparing x: ' + x['filename'] + ' to di:' + di['filename'])
2015-02-09 18:44:10 +00:00
if x [ ' filename ' ] != di [ ' filename ' ] :
IMP: Cleaned up interface for StoryArcs / Story Arc Details, IMP: Cleaned up interface for Reading list Management, IMP: Added better reading list management - new status (added, downloaded, read), IMP: Added sync option for use with another device for reading list transfer (ie. tablet) Android only, IMP: Autopopulate new weekly pull releases to reading list, IMP: 'Watch' option in weekly pull list now fully functional. Will watch CV for series' that do not have any series data yet as they are new starting series. Will auto-add once available, IMP: Auto-watch check is run after every refresh/recreate of the weeklypull list, IMP: Improved the Add a Series option where it will now look for issues that are 'new' or 'wanted' during add sequence, IMP: Main page interface now has coloured have/total bars to denote series completion, IMP: New scheduler / threading locks in place in an attempt to avoid database locks, FIX: Removed some erroneous locking that was going on when importing a directory was being run, IMP: Stat counter now present when post-processing multiple issues in sequence, FIX: for issue number error when post-processing and issue number was a non-alphanumeric, FIX: for metatagging: when original file was .cbz, would try to convert and fail, FIX: for issues that were negative and were preceeded by a # in the filename (filechecker), FIX: for publisher having non-alphanumeric character in name when attempting to determine publisher, FIX: if annuals enabled, would incorrectly show as being 'already in library' when viewing search results if results constained annuals, FIX:(#944) for incorrect nzbname being used when post-processing was being performed from an nzb client (experimental mainly), IMP: Turned off logging for ComicVine API counter, FIX: Added retry attempts when connecting to ComicVine in order to avoid errors when adding a series, IMP:(#963) Added ability to add snatched to filter when viewing Wanted issues on Wanted tab, FIX: When importing and then selecting a series to import via the select screen, will now flip back to the importresults and add the selected series in the background, IMP:(#952) Main page is now sorted in ascending order by Continuing/Ended status (and subbed by whether is Active/Paused).Custom sorting is still available, FIX: Dupecheck will now automatically assume existing 0-byte files are to be overwritten when performing post-processing, FIX: If publication date for series contained a '?' (usually with brand new series) will force to 'Present' to allow for pull-list comparisons to take place, FIX: Mylar will now disallow search results which have 'covers only' or 'variant' in the filename, IMP: Better nzbname generation/retrieval (will check inside nzb for possible names) to be used when post-processing, IMP: DB Update will now perform update to all active comics in descending order by Latest Date (instead of random order), FIX: Enforce the 5hr limit rule when running DB update (will only update series that haven't been updated in >5 hours), FIX: Annuals will now have/retain the proper status upon doing DB Update, FIX: Have totals will now be updated when doing a recheck files (sometimes wouldn't get updated depending on various states of status'), FIX:(#966) Added urllib2.URLError exeception trap when attempting to check Git for updates, IMP: Removed the individual sqlite calls for weeklypull, and brought them into line with using the db module (which will minimize concurrent access, which seemed to be causing db locks), IMP: Cleaned up some code and shuffled some functions so they are in more appropriate locations
2015-03-27 17:27:59 +00:00
#logger.fdebug('Matched.')
2015-02-09 18:44:10 +00:00
issuedupe_temp . append ( x )
tmphavefiles + = 1
issuedupechk = issuedupe_temp
FIX: When manually post-processing files, if files were in more than one subdirectory level deep would use incorrect path resulting in error, FIX:(#1261, #1262) Traceback error when searching and returned value had volume, FIX: If filenames had the Volume label after the issue number, would fail to recognize the series title properly resulting in errors during scanning, FIX: Removing imported series from the Import List via dropdowns, IMP: Improved upon imports matching up to series that have multiple volumes by comparing issue counts & years, IMP: If an import is determined to need manual intervention, will display only the results that have passed the issue counts/years/name matching as opposed to showing all the results, FIX: When moving files during an import will now only move those files that are attached to the specific grouping by series title/volume, FIX: Import Manual Intervention results sortable by Year column now, FIX: Fixed an errant callback to home during configuration saves that caused web browser error when used with reverse proxy, FIX: During story arc searching if a story arc had a '-' in the arc name, would not be able to find it, FIX: When adding a story-arc, if files pre-existed in the directory attached to the arc would error, FIX: When adding a series, if was used for Folder Format and the volume label didn't exist would label directory with 'noversion', FIX: Selecting to metatag annuals via the manual metatag option on the series detail page, FIX: When retrieving from KAT and response returned was a 404 would ignore it, now will retry alternate torcache site
2016-04-25 15:35:12 +00:00
havefiles = tmphavefiles + len ( annualdupechk )
2015-02-09 18:44:10 +00:00
foundchk = False
break
IMP: (#844)Annuals now deleted from table on a Delete Series command, IMP: (#837)Better volume detection when searching for issues, IMP: (#842)(#808) Added some handling for issue titles when searching (it's not perfect, but it works for most), IMP: (#836)Added .cb7 for filechecking purposes only, IMP: (#830)issue numbers that are -1 can now be renamed, FIX: 'bi-annual' now fixed along with some other improvements for annual handling, IMP: Added version check for autoProcessComics and ComicRN - in order to help warn users of changes and the need to update these files since they are usually outside of the mylar git directory when being used, FIX:(#840) Boxcar2 should be working again, FIX: (#845) If search returned a filename that had no distinction between issue number and issue title and/or extra information, would error out if any of the words contained NOW, IMP: (#823) Added some better detection for NZBGet parameters when using ComicRN.py
2014-10-06 18:10:36 +00:00
2015-01-16 19:40:08 +00:00
if issuedupe == " no " :
2015-05-22 08:32:51 +00:00
if foundchk == False :
2015-01-16 19:40:08 +00:00
logger . fdebug ( module + ' Matched...issue: ' + rescan [ ' ComicName ' ] + ' # ' + reiss [ ' Issue_Number ' ] + ' --- ' + str ( int_iss ) )
havefiles + = 1
haveissue = " yes "
2016-10-05 14:12:58 +00:00
isslocation = helpers . conversion ( tmpfc [ ' ComicFilename ' ] )
2015-01-16 19:40:08 +00:00
issSize = str ( tmpfc [ ' ComicSize ' ] )
2015-05-27 05:40:38 +00:00
logger . fdebug ( module + ' .......filename: ' + isslocation )
logger . fdebug ( module + ' .......filesize: ' + str ( tmpfc [ ' ComicSize ' ] ) )
# to avoid duplicate issues which screws up the count...let's store the filename issues then
2015-01-16 19:40:08 +00:00
# compare earlier...
issuedupechk . append ( { ' fcdigit ' : fcdigit ,
' filename ' : tmpfc [ ' ComicFilename ' ] ,
' filesize ' : tmpfc [ ' ComicSize ' ] ,
' issueyear ' : issyear ,
' issueid ' : reiss [ ' IssueID ' ] } )
IMP: (#844)Annuals now deleted from table on a Delete Series command, IMP: (#837)Better volume detection when searching for issues, IMP: (#842)(#808) Added some handling for issue titles when searching (it's not perfect, but it works for most), IMP: (#836)Added .cb7 for filechecking purposes only, IMP: (#830)issue numbers that are -1 can now be renamed, FIX: 'bi-annual' now fixed along with some other improvements for annual handling, IMP: Added version check for autoProcessComics and ComicRN - in order to help warn users of changes and the need to update these files since they are usually outside of the mylar git directory when being used, FIX:(#840) Boxcar2 should be working again, FIX: (#845) If search returned a filename that had no distinction between issue number and issue title and/or extra information, would error out if any of the words contained NOW, IMP: (#823) Added some better detection for NZBGet parameters when using ComicRN.py
2014-10-06 18:10:36 +00:00
break
2015-01-16 19:40:08 +00:00
if issuedupe == " yes " :
logger . fdebug ( module + ' I should break out here because of a dupe. ' )
break
2013-07-01 05:19:15 +00:00
if haveissue == " yes " or issuedupe == " yes " : break
n + = 1
else :
FIX: undefined on filter box on startup, IMP: Added Meta-Tagging options on a series / issue basis on comic details screen, IMP: Issue Information is now available per issue and is extracted currently from the cbz file to display (if no cbz is present, the option isn't available), IMP: Failed Download handling is implemented and available in GUI - required to replace existing autoProcessComics.py and ComicRN.py scripts, IMP: Added ability to specify post-processing script instead of ComicRN.py, IMP: Added abilty to edit the issue date for a given issue by simply clicking on it - this will help to avoid dates that have incorrect or 0000-00-00, IMP: Story Arc searching is working (not adding yet), IMP: Added Archived/Ignored options to Upcoming/Wanted tab, IMP: Fixed some alignment and display issues on the Upcoming section, IMP: Added better directory handling for Updating Comic Location when it needs to get changed for all existing series (locmove in config.ini), IMP: Added better handling for unicode characters in series titles when searching / filechecking, IMP: When adding a new series with no data, Mylar would error out (now will add and just retain 0 issues), FIX: When year was fuzzied, search would still attempt to do a date-check comparison and say everything failed, IMP: Better handling of nzb names when retaining for post-processing comparisons, IMP: Future Upcoming now will use actual shipping date of issue if available in order to see if issue is available, FIX: If annuals were enabled, refreshing a series would put some issues into an Archived status because the actual counts would be off, IMP: When file checking, Alternate Naming would be searched last which resulted in matching incorrectly to the series title or other alternate naming for the given series, now will check the Alternate Naming first for a match, then drop back down to the series name itself otherwise, IMP: Improved Annual detection when integrated with a given series, IMP: Improved the checking of the future Upcoming list for issues marked as Wanted but not available yet and then auto-adding, IMP: Improved upon story arc checking for missing issues / searching for wanted, IMP: Enabling Annuals support now within Configuration GUI, bunch of other things....
2014-07-28 19:28:09 +00:00
if tmpfc [ ' AnnualComicID ' ] :
ANNComicID = tmpfc [ ' AnnualComicID ' ]
logger . fdebug ( module + ' Forcing ComicID to ' + str ( ANNComicID ) + ' in case of duplicate numbering across volumes. ' )
reannuals = myDB . select ( ' SELECT * FROM annuals WHERE ComicID=? AND ReleaseComicID=? ' , [ ComicID , ANNComicID ] )
else :
reannuals = myDB . select ( ' SELECT * FROM annuals WHERE ComicID=? ' , [ ComicID ] )
2014-08-08 06:54:16 +00:00
ANNComicID = ComicID
2016-12-24 07:24:14 +00:00
if len ( reannuals ) == 0 :
#it's possible if annual integration is enabled, and an annual series is added directly to the wachlist,
#not as part of a series, that the above won't work since it's looking in the wrong table.
reannuals = myDB . select ( ' SELECT * FROM issues WHERE ComicID=? ' , [ ComicID ] )
ANNComicID = None #need to set this to None so we write to the issues table and not the annuals
2013-07-01 05:19:15 +00:00
# annual inclusion here.
#logger.fdebug("checking " + str(temploc))
fcnew = shlex . split ( str ( temploc ) )
fcn = len ( fcnew )
n = 0
2014-01-27 04:23:11 +00:00
reann = None
IMP: Cleaned up interface for StoryArcs / Story Arc Details, IMP: Cleaned up interface for Reading list Management, IMP: Added better reading list management - new status (added, downloaded, read), IMP: Added sync option for use with another device for reading list transfer (ie. tablet) Android only, IMP: Autopopulate new weekly pull releases to reading list, IMP: 'Watch' option in weekly pull list now fully functional. Will watch CV for series' that do not have any series data yet as they are new starting series. Will auto-add once available, IMP: Auto-watch check is run after every refresh/recreate of the weeklypull list, IMP: Improved the Add a Series option where it will now look for issues that are 'new' or 'wanted' during add sequence, IMP: Main page interface now has coloured have/total bars to denote series completion, IMP: New scheduler / threading locks in place in an attempt to avoid database locks, FIX: Removed some erroneous locking that was going on when importing a directory was being run, IMP: Stat counter now present when post-processing multiple issues in sequence, FIX: for issue number error when post-processing and issue number was a non-alphanumeric, FIX: for metatagging: when original file was .cbz, would try to convert and fail, FIX: for issues that were negative and were preceeded by a # in the filename (filechecker), FIX: for publisher having non-alphanumeric character in name when attempting to determine publisher, FIX: if annuals enabled, would incorrectly show as being 'already in library' when viewing search results if results constained annuals, FIX:(#944) for incorrect nzbname being used when post-processing was being performed from an nzb client (experimental mainly), IMP: Turned off logging for ComicVine API counter, FIX: Added retry attempts when connecting to ComicVine in order to avoid errors when adding a series, IMP:(#963) Added ability to add snatched to filter when viewing Wanted issues on Wanted tab, FIX: When importing and then selecting a series to import via the select screen, will now flip back to the importresults and add the selected series in the background, IMP:(#952) Main page is now sorted in ascending order by Continuing/Ended status (and subbed by whether is Active/Paused).Custom sorting is still available, FIX: Dupecheck will now automatically assume existing 0-byte files are to be overwritten when performing post-processing, FIX: If publication date for series contained a '?' (usually with brand new series) will force to 'Present' to allow for pull-list comparisons to take place, FIX: Mylar will now disallow search results which have 'covers only' or 'variant' in the filename, IMP: Better nzbname generation/retrieval (will check inside nzb for possible names) to be used when post-processing, IMP: DB Update will now perform update to all active comics in descending order by Latest Date (instead of random order), FIX: Enforce the 5hr limit rule when running DB update (will only update series that haven't been updated in >5 hours), FIX: Annuals will now have/retain the proper status upon doing DB Update, FIX: Have totals will now be updated when doing a recheck files (sometimes wouldn't get updated depending on various states of status'), FIX:(#966) Added urllib2.URLError exeception trap when attempting to check Git for updates, IMP: Removed the individual sqlite calls for weeklypull, and brought them into line with using the db module (which will minimize concurrent access, which seemed to be causing db locks), IMP: Cleaned up some code and shuffled some functions so they are in more appropriate locations
2015-03-27 17:27:59 +00:00
while True :
2013-07-01 05:19:15 +00:00
try :
reann = reannuals [ n ]
except IndexError :
break
IMP: Cleaned up interface for StoryArcs / Story Arc Details, IMP: Cleaned up interface for Reading list Management, IMP: Added better reading list management - new status (added, downloaded, read), IMP: Added sync option for use with another device for reading list transfer (ie. tablet) Android only, IMP: Autopopulate new weekly pull releases to reading list, IMP: 'Watch' option in weekly pull list now fully functional. Will watch CV for series' that do not have any series data yet as they are new starting series. Will auto-add once available, IMP: Auto-watch check is run after every refresh/recreate of the weeklypull list, IMP: Improved the Add a Series option where it will now look for issues that are 'new' or 'wanted' during add sequence, IMP: Main page interface now has coloured have/total bars to denote series completion, IMP: New scheduler / threading locks in place in an attempt to avoid database locks, FIX: Removed some erroneous locking that was going on when importing a directory was being run, IMP: Stat counter now present when post-processing multiple issues in sequence, FIX: for issue number error when post-processing and issue number was a non-alphanumeric, FIX: for metatagging: when original file was .cbz, would try to convert and fail, FIX: for issues that were negative and were preceeded by a # in the filename (filechecker), FIX: for publisher having non-alphanumeric character in name when attempting to determine publisher, FIX: if annuals enabled, would incorrectly show as being 'already in library' when viewing search results if results constained annuals, FIX:(#944) for incorrect nzbname being used when post-processing was being performed from an nzb client (experimental mainly), IMP: Turned off logging for ComicVine API counter, FIX: Added retry attempts when connecting to ComicVine in order to avoid errors when adding a series, IMP:(#963) Added ability to add snatched to filter when viewing Wanted issues on Wanted tab, FIX: When importing and then selecting a series to import via the select screen, will now flip back to the importresults and add the selected series in the background, IMP:(#952) Main page is now sorted in ascending order by Continuing/Ended status (and subbed by whether is Active/Paused).Custom sorting is still available, FIX: Dupecheck will now automatically assume existing 0-byte files are to be overwritten when performing post-processing, FIX: If publication date for series contained a '?' (usually with brand new series) will force to 'Present' to allow for pull-list comparisons to take place, FIX: Mylar will now disallow search results which have 'covers only' or 'variant' in the filename, IMP: Better nzbname generation/retrieval (will check inside nzb for possible names) to be used when post-processing, IMP: DB Update will now perform update to all active comics in descending order by Latest Date (instead of random order), FIX: Enforce the 5hr limit rule when running DB update (will only update series that haven't been updated in >5 hours), FIX: Annuals will now have/retain the proper status upon doing DB Update, FIX: Have totals will now be updated when doing a recheck files (sometimes wouldn't get updated depending on various states of status'), FIX:(#966) Added urllib2.URLError exeception trap when attempting to check Git for updates, IMP: Removed the individual sqlite calls for weeklypull, and brought them into line with using the db module (which will minimize concurrent access, which seemed to be causing db locks), IMP: Cleaned up some code and shuffled some functions so they are in more appropriate locations
2015-03-27 17:27:59 +00:00
int_iss = helpers . issuedigits ( reann [ ' Issue_Number ' ] )
2017-09-19 15:41:48 +00:00
#logger.fdebug(module + ' int_iss:' + str(int_iss))
IMP: Cleaned up interface for StoryArcs / Story Arc Details, IMP: Cleaned up interface for Reading list Management, IMP: Added better reading list management - new status (added, downloaded, read), IMP: Added sync option for use with another device for reading list transfer (ie. tablet) Android only, IMP: Autopopulate new weekly pull releases to reading list, IMP: 'Watch' option in weekly pull list now fully functional. Will watch CV for series' that do not have any series data yet as they are new starting series. Will auto-add once available, IMP: Auto-watch check is run after every refresh/recreate of the weeklypull list, IMP: Improved the Add a Series option where it will now look for issues that are 'new' or 'wanted' during add sequence, IMP: Main page interface now has coloured have/total bars to denote series completion, IMP: New scheduler / threading locks in place in an attempt to avoid database locks, FIX: Removed some erroneous locking that was going on when importing a directory was being run, IMP: Stat counter now present when post-processing multiple issues in sequence, FIX: for issue number error when post-processing and issue number was a non-alphanumeric, FIX: for metatagging: when original file was .cbz, would try to convert and fail, FIX: for issues that were negative and were preceeded by a # in the filename (filechecker), FIX: for publisher having non-alphanumeric character in name when attempting to determine publisher, FIX: if annuals enabled, would incorrectly show as being 'already in library' when viewing search results if results constained annuals, FIX:(#944) for incorrect nzbname being used when post-processing was being performed from an nzb client (experimental mainly), IMP: Turned off logging for ComicVine API counter, FIX: Added retry attempts when connecting to ComicVine in order to avoid errors when adding a series, IMP:(#963) Added ability to add snatched to filter when viewing Wanted issues on Wanted tab, FIX: When importing and then selecting a series to import via the select screen, will now flip back to the importresults and add the selected series in the background, IMP:(#952) Main page is now sorted in ascending order by Continuing/Ended status (and subbed by whether is Active/Paused).Custom sorting is still available, FIX: Dupecheck will now automatically assume existing 0-byte files are to be overwritten when performing post-processing, FIX: If publication date for series contained a '?' (usually with brand new series) will force to 'Present' to allow for pull-list comparisons to take place, FIX: Mylar will now disallow search results which have 'covers only' or 'variant' in the filename, IMP: Better nzbname generation/retrieval (will check inside nzb for possible names) to be used when post-processing, IMP: DB Update will now perform update to all active comics in descending order by Latest Date (instead of random order), FIX: Enforce the 5hr limit rule when running DB update (will only update series that haven't been updated in >5 hours), FIX: Annuals will now have/retain the proper status upon doing DB Update, FIX: Have totals will now be updated when doing a recheck files (sometimes wouldn't get updated depending on various states of status'), FIX:(#966) Added urllib2.URLError exeception trap when attempting to check Git for updates, IMP: Removed the individual sqlite calls for weeklypull, and brought them into line with using the db module (which will minimize concurrent access, which seemed to be causing db locks), IMP: Cleaned up some code and shuffled some functions so they are in more appropriate locations
2015-03-27 17:27:59 +00:00
2013-07-01 05:19:15 +00:00
issyear = reann [ ' IssueDate ' ] [ : 4 ]
2015-05-22 08:32:51 +00:00
old_status = reann [ ' Status ' ]
IMP: Cleaned up interface for StoryArcs / Story Arc Details, IMP: Cleaned up interface for Reading list Management, IMP: Added better reading list management - new status (added, downloaded, read), IMP: Added sync option for use with another device for reading list transfer (ie. tablet) Android only, IMP: Autopopulate new weekly pull releases to reading list, IMP: 'Watch' option in weekly pull list now fully functional. Will watch CV for series' that do not have any series data yet as they are new starting series. Will auto-add once available, IMP: Auto-watch check is run after every refresh/recreate of the weeklypull list, IMP: Improved the Add a Series option where it will now look for issues that are 'new' or 'wanted' during add sequence, IMP: Main page interface now has coloured have/total bars to denote series completion, IMP: New scheduler / threading locks in place in an attempt to avoid database locks, FIX: Removed some erroneous locking that was going on when importing a directory was being run, IMP: Stat counter now present when post-processing multiple issues in sequence, FIX: for issue number error when post-processing and issue number was a non-alphanumeric, FIX: for metatagging: when original file was .cbz, would try to convert and fail, FIX: for issues that were negative and were preceeded by a # in the filename (filechecker), FIX: for publisher having non-alphanumeric character in name when attempting to determine publisher, FIX: if annuals enabled, would incorrectly show as being 'already in library' when viewing search results if results constained annuals, FIX:(#944) for incorrect nzbname being used when post-processing was being performed from an nzb client (experimental mainly), IMP: Turned off logging for ComicVine API counter, FIX: Added retry attempts when connecting to ComicVine in order to avoid errors when adding a series, IMP:(#963) Added ability to add snatched to filter when viewing Wanted issues on Wanted tab, FIX: When importing and then selecting a series to import via the select screen, will now flip back to the importresults and add the selected series in the background, IMP:(#952) Main page is now sorted in ascending order by Continuing/Ended status (and subbed by whether is Active/Paused).Custom sorting is still available, FIX: Dupecheck will now automatically assume existing 0-byte files are to be overwritten when performing post-processing, FIX: If publication date for series contained a '?' (usually with brand new series) will force to 'Present' to allow for pull-list comparisons to take place, FIX: Mylar will now disallow search results which have 'covers only' or 'variant' in the filename, IMP: Better nzbname generation/retrieval (will check inside nzb for possible names) to be used when post-processing, IMP: DB Update will now perform update to all active comics in descending order by Latest Date (instead of random order), FIX: Enforce the 5hr limit rule when running DB update (will only update series that haven't been updated in >5 hours), FIX: Annuals will now have/retain the proper status upon doing DB Update, FIX: Have totals will now be updated when doing a recheck files (sometimes wouldn't get updated depending on various states of status'), FIX:(#966) Added urllib2.URLError exeception trap when attempting to check Git for updates, IMP: Removed the individual sqlite calls for weeklypull, and brought them into line with using the db module (which will minimize concurrent access, which seemed to be causing db locks), IMP: Cleaned up some code and shuffled some functions so they are in more appropriate locations
2015-03-27 17:27:59 +00:00
fcdigit = helpers . issuedigits ( re . sub ( ' annual ' , ' ' , temploc . lower ( ) ) . strip ( ) )
2018-07-21 17:15:14 +00:00
if fcdigit == 999999999999999 :
fcdigit = helpers . issuedigits ( re . sub ( ' special ' , ' ' , temploc . lower ( ) ) . strip ( ) )
IMP: Cleaned up interface for StoryArcs / Story Arc Details, IMP: Cleaned up interface for Reading list Management, IMP: Added better reading list management - new status (added, downloaded, read), IMP: Added sync option for use with another device for reading list transfer (ie. tablet) Android only, IMP: Autopopulate new weekly pull releases to reading list, IMP: 'Watch' option in weekly pull list now fully functional. Will watch CV for series' that do not have any series data yet as they are new starting series. Will auto-add once available, IMP: Auto-watch check is run after every refresh/recreate of the weeklypull list, IMP: Improved the Add a Series option where it will now look for issues that are 'new' or 'wanted' during add sequence, IMP: Main page interface now has coloured have/total bars to denote series completion, IMP: New scheduler / threading locks in place in an attempt to avoid database locks, FIX: Removed some erroneous locking that was going on when importing a directory was being run, IMP: Stat counter now present when post-processing multiple issues in sequence, FIX: for issue number error when post-processing and issue number was a non-alphanumeric, FIX: for metatagging: when original file was .cbz, would try to convert and fail, FIX: for issues that were negative and were preceeded by a # in the filename (filechecker), FIX: for publisher having non-alphanumeric character in name when attempting to determine publisher, FIX: if annuals enabled, would incorrectly show as being 'already in library' when viewing search results if results constained annuals, FIX:(#944) for incorrect nzbname being used when post-processing was being performed from an nzb client (experimental mainly), IMP: Turned off logging for ComicVine API counter, FIX: Added retry attempts when connecting to ComicVine in order to avoid errors when adding a series, IMP:(#963) Added ability to add snatched to filter when viewing Wanted issues on Wanted tab, FIX: When importing and then selecting a series to import via the select screen, will now flip back to the importresults and add the selected series in the background, IMP:(#952) Main page is now sorted in ascending order by Continuing/Ended status (and subbed by whether is Active/Paused).Custom sorting is still available, FIX: Dupecheck will now automatically assume existing 0-byte files are to be overwritten when performing post-processing, FIX: If publication date for series contained a '?' (usually with brand new series) will force to 'Present' to allow for pull-list comparisons to take place, FIX: Mylar will now disallow search results which have 'covers only' or 'variant' in the filename, IMP: Better nzbname generation/retrieval (will check inside nzb for possible names) to be used when post-processing, IMP: DB Update will now perform update to all active comics in descending order by Latest Date (instead of random order), FIX: Enforce the 5hr limit rule when running DB update (will only update series that haven't been updated in >5 hours), FIX: Annuals will now have/retain the proper status upon doing DB Update, FIX: Have totals will now be updated when doing a recheck files (sometimes wouldn't get updated depending on various states of status'), FIX:(#966) Added urllib2.URLError exeception trap when attempting to check Git for updates, IMP: Removed the individual sqlite calls for weeklypull, and brought them into line with using the db module (which will minimize concurrent access, which seemed to be causing db locks), IMP: Cleaned up some code and shuffled some functions so they are in more appropriate locations
2015-03-27 17:27:59 +00:00
2017-09-19 15:41:48 +00:00
if int ( fcdigit ) == int_iss and ANNComicID is not None :
IMP: Cleaned up interface for StoryArcs / Story Arc Details, IMP: Cleaned up interface for Reading list Management, IMP: Added better reading list management - new status (added, downloaded, read), IMP: Added sync option for use with another device for reading list transfer (ie. tablet) Android only, IMP: Autopopulate new weekly pull releases to reading list, IMP: 'Watch' option in weekly pull list now fully functional. Will watch CV for series' that do not have any series data yet as they are new starting series. Will auto-add once available, IMP: Auto-watch check is run after every refresh/recreate of the weeklypull list, IMP: Improved the Add a Series option where it will now look for issues that are 'new' or 'wanted' during add sequence, IMP: Main page interface now has coloured have/total bars to denote series completion, IMP: New scheduler / threading locks in place in an attempt to avoid database locks, FIX: Removed some erroneous locking that was going on when importing a directory was being run, IMP: Stat counter now present when post-processing multiple issues in sequence, FIX: for issue number error when post-processing and issue number was a non-alphanumeric, FIX: for metatagging: when original file was .cbz, would try to convert and fail, FIX: for issues that were negative and were preceeded by a # in the filename (filechecker), FIX: for publisher having non-alphanumeric character in name when attempting to determine publisher, FIX: if annuals enabled, would incorrectly show as being 'already in library' when viewing search results if results constained annuals, FIX:(#944) for incorrect nzbname being used when post-processing was being performed from an nzb client (experimental mainly), IMP: Turned off logging for ComicVine API counter, FIX: Added retry attempts when connecting to ComicVine in order to avoid errors when adding a series, IMP:(#963) Added ability to add snatched to filter when viewing Wanted issues on Wanted tab, FIX: When importing and then selecting a series to import via the select screen, will now flip back to the importresults and add the selected series in the background, IMP:(#952) Main page is now sorted in ascending order by Continuing/Ended status (and subbed by whether is Active/Paused).Custom sorting is still available, FIX: Dupecheck will now automatically assume existing 0-byte files are to be overwritten when performing post-processing, FIX: If publication date for series contained a '?' (usually with brand new series) will force to 'Present' to allow for pull-list comparisons to take place, FIX: Mylar will now disallow search results which have 'covers only' or 'variant' in the filename, IMP: Better nzbname generation/retrieval (will check inside nzb for possible names) to be used when post-processing, IMP: DB Update will now perform update to all active comics in descending order by Latest Date (instead of random order), FIX: Enforce the 5hr limit rule when running DB update (will only update series that haven't been updated in >5 hours), FIX: Annuals will now have/retain the proper status upon doing DB Update, FIX: Have totals will now be updated when doing a recheck files (sometimes wouldn't get updated depending on various states of status'), FIX:(#966) Added urllib2.URLError exeception trap when attempting to check Git for updates, IMP: Removed the individual sqlite calls for weeklypull, and brought them into line with using the db module (which will minimize concurrent access, which seemed to be causing db locks), IMP: Cleaned up some code and shuffled some functions so they are in more appropriate locations
2015-03-27 17:27:59 +00:00
logger . fdebug ( module + ' [ ' + str ( ANNComicID ) + ' ] Annual match - issue : ' + str ( int_iss ) )
#baseline these to default to normal scanning
multiplechk = False
annualdupe = " no "
foundchk = False
#check here if muliple identical numbering issues exist for the series
2017-05-04 18:41:05 +00:00
if len ( mc_annual ) > 1 :
for ma in mc_annual :
if ma [ ' Int_IssueNumber ' ] == int_iss :
if ma [ ' IssueID ' ] == reann [ ' IssueID ' ] :
logger . fdebug ( module + ' IssueID matches to multiple issues : ' + str ( ma [ ' IssueID ' ] ) + ' . Checking dupe. ' )
logger . fdebug ( module + ' maISSUEYEAR: ' + str ( ma [ ' IssueYear ' ] ) + ' -- issyear : ' + str ( issyear ) )
if any ( ma [ ' IssueID ' ] == d [ ' issueid ' ] for d in annualdupechk ) :
IMP: Cleaned up interface for StoryArcs / Story Arc Details, IMP: Cleaned up interface for Reading list Management, IMP: Added better reading list management - new status (added, downloaded, read), IMP: Added sync option for use with another device for reading list transfer (ie. tablet) Android only, IMP: Autopopulate new weekly pull releases to reading list, IMP: 'Watch' option in weekly pull list now fully functional. Will watch CV for series' that do not have any series data yet as they are new starting series. Will auto-add once available, IMP: Auto-watch check is run after every refresh/recreate of the weeklypull list, IMP: Improved the Add a Series option where it will now look for issues that are 'new' or 'wanted' during add sequence, IMP: Main page interface now has coloured have/total bars to denote series completion, IMP: New scheduler / threading locks in place in an attempt to avoid database locks, FIX: Removed some erroneous locking that was going on when importing a directory was being run, IMP: Stat counter now present when post-processing multiple issues in sequence, FIX: for issue number error when post-processing and issue number was a non-alphanumeric, FIX: for metatagging: when original file was .cbz, would try to convert and fail, FIX: for issues that were negative and were preceeded by a # in the filename (filechecker), FIX: for publisher having non-alphanumeric character in name when attempting to determine publisher, FIX: if annuals enabled, would incorrectly show as being 'already in library' when viewing search results if results constained annuals, FIX:(#944) for incorrect nzbname being used when post-processing was being performed from an nzb client (experimental mainly), IMP: Turned off logging for ComicVine API counter, FIX: Added retry attempts when connecting to ComicVine in order to avoid errors when adding a series, IMP:(#963) Added ability to add snatched to filter when viewing Wanted issues on Wanted tab, FIX: When importing and then selecting a series to import via the select screen, will now flip back to the importresults and add the selected series in the background, IMP:(#952) Main page is now sorted in ascending order by Continuing/Ended status (and subbed by whether is Active/Paused).Custom sorting is still available, FIX: Dupecheck will now automatically assume existing 0-byte files are to be overwritten when performing post-processing, FIX: If publication date for series contained a '?' (usually with brand new series) will force to 'Present' to allow for pull-list comparisons to take place, FIX: Mylar will now disallow search results which have 'covers only' or 'variant' in the filename, IMP: Better nzbname generation/retrieval (will check inside nzb for possible names) to be used when post-processing, IMP: DB Update will now perform update to all active comics in descending order by Latest Date (instead of random order), FIX: Enforce the 5hr limit rule when running DB update (will only update series that haven't been updated in >5 hours), FIX: Annuals will now have/retain the proper status upon doing DB Update, FIX: Have totals will now be updated when doing a recheck files (sometimes wouldn't get updated depending on various states of status'), FIX:(#966) Added urllib2.URLError exeception trap when attempting to check Git for updates, IMP: Removed the individual sqlite calls for weeklypull, and brought them into line with using the db module (which will minimize concurrent access, which seemed to be causing db locks), IMP: Cleaned up some code and shuffled some functions so they are in more appropriate locations
2015-03-27 17:27:59 +00:00
logger . fdebug ( module + ' IssueID already within dupe. Checking next if available. ' )
multiplechk = True
break
2017-05-04 18:41:05 +00:00
if ( ma [ ' IssueYear ' ] in tmpfc [ ' ComicFilename ' ] ) and ( issyear == ma [ ' IssueYear ' ] ) :
IMP: Cleaned up interface for StoryArcs / Story Arc Details, IMP: Cleaned up interface for Reading list Management, IMP: Added better reading list management - new status (added, downloaded, read), IMP: Added sync option for use with another device for reading list transfer (ie. tablet) Android only, IMP: Autopopulate new weekly pull releases to reading list, IMP: 'Watch' option in weekly pull list now fully functional. Will watch CV for series' that do not have any series data yet as they are new starting series. Will auto-add once available, IMP: Auto-watch check is run after every refresh/recreate of the weeklypull list, IMP: Improved the Add a Series option where it will now look for issues that are 'new' or 'wanted' during add sequence, IMP: Main page interface now has coloured have/total bars to denote series completion, IMP: New scheduler / threading locks in place in an attempt to avoid database locks, FIX: Removed some erroneous locking that was going on when importing a directory was being run, IMP: Stat counter now present when post-processing multiple issues in sequence, FIX: for issue number error when post-processing and issue number was a non-alphanumeric, FIX: for metatagging: when original file was .cbz, would try to convert and fail, FIX: for issues that were negative and were preceeded by a # in the filename (filechecker), FIX: for publisher having non-alphanumeric character in name when attempting to determine publisher, FIX: if annuals enabled, would incorrectly show as being 'already in library' when viewing search results if results constained annuals, FIX:(#944) for incorrect nzbname being used when post-processing was being performed from an nzb client (experimental mainly), IMP: Turned off logging for ComicVine API counter, FIX: Added retry attempts when connecting to ComicVine in order to avoid errors when adding a series, IMP:(#963) Added ability to add snatched to filter when viewing Wanted issues on Wanted tab, FIX: When importing and then selecting a series to import via the select screen, will now flip back to the importresults and add the selected series in the background, IMP:(#952) Main page is now sorted in ascending order by Continuing/Ended status (and subbed by whether is Active/Paused).Custom sorting is still available, FIX: Dupecheck will now automatically assume existing 0-byte files are to be overwritten when performing post-processing, FIX: If publication date for series contained a '?' (usually with brand new series) will force to 'Present' to allow for pull-list comparisons to take place, FIX: Mylar will now disallow search results which have 'covers only' or 'variant' in the filename, IMP: Better nzbname generation/retrieval (will check inside nzb for possible names) to be used when post-processing, IMP: DB Update will now perform update to all active comics in descending order by Latest Date (instead of random order), FIX: Enforce the 5hr limit rule when running DB update (will only update series that haven't been updated in >5 hours), FIX: Annuals will now have/retain the proper status upon doing DB Update, FIX: Have totals will now be updated when doing a recheck files (sometimes wouldn't get updated depending on various states of status'), FIX:(#966) Added urllib2.URLError exeception trap when attempting to check Git for updates, IMP: Removed the individual sqlite calls for weeklypull, and brought them into line with using the db module (which will minimize concurrent access, which seemed to be causing db locks), IMP: Cleaned up some code and shuffled some functions so they are in more appropriate locations
2015-03-27 17:27:59 +00:00
logger . fdebug ( module + ' Matched to year within filename : ' + str ( issyear ) )
multiplechk = False
2017-05-04 18:41:05 +00:00
ANNComicID = ack [ ' ReleaseComicID ' ]
IMP: Cleaned up interface for StoryArcs / Story Arc Details, IMP: Cleaned up interface for Reading list Management, IMP: Added better reading list management - new status (added, downloaded, read), IMP: Added sync option for use with another device for reading list transfer (ie. tablet) Android only, IMP: Autopopulate new weekly pull releases to reading list, IMP: 'Watch' option in weekly pull list now fully functional. Will watch CV for series' that do not have any series data yet as they are new starting series. Will auto-add once available, IMP: Auto-watch check is run after every refresh/recreate of the weeklypull list, IMP: Improved the Add a Series option where it will now look for issues that are 'new' or 'wanted' during add sequence, IMP: Main page interface now has coloured have/total bars to denote series completion, IMP: New scheduler / threading locks in place in an attempt to avoid database locks, FIX: Removed some erroneous locking that was going on when importing a directory was being run, IMP: Stat counter now present when post-processing multiple issues in sequence, FIX: for issue number error when post-processing and issue number was a non-alphanumeric, FIX: for metatagging: when original file was .cbz, would try to convert and fail, FIX: for issues that were negative and were preceeded by a # in the filename (filechecker), FIX: for publisher having non-alphanumeric character in name when attempting to determine publisher, FIX: if annuals enabled, would incorrectly show as being 'already in library' when viewing search results if results constained annuals, FIX:(#944) for incorrect nzbname being used when post-processing was being performed from an nzb client (experimental mainly), IMP: Turned off logging for ComicVine API counter, FIX: Added retry attempts when connecting to ComicVine in order to avoid errors when adding a series, IMP:(#963) Added ability to add snatched to filter when viewing Wanted issues on Wanted tab, FIX: When importing and then selecting a series to import via the select screen, will now flip back to the importresults and add the selected series in the background, IMP:(#952) Main page is now sorted in ascending order by Continuing/Ended status (and subbed by whether is Active/Paused).Custom sorting is still available, FIX: Dupecheck will now automatically assume existing 0-byte files are to be overwritten when performing post-processing, FIX: If publication date for series contained a '?' (usually with brand new series) will force to 'Present' to allow for pull-list comparisons to take place, FIX: Mylar will now disallow search results which have 'covers only' or 'variant' in the filename, IMP: Better nzbname generation/retrieval (will check inside nzb for possible names) to be used when post-processing, IMP: DB Update will now perform update to all active comics in descending order by Latest Date (instead of random order), FIX: Enforce the 5hr limit rule when running DB update (will only update series that haven't been updated in >5 hours), FIX: Annuals will now have/retain the proper status upon doing DB Update, FIX: Have totals will now be updated when doing a recheck files (sometimes wouldn't get updated depending on various states of status'), FIX:(#966) Added urllib2.URLError exeception trap when attempting to check Git for updates, IMP: Removed the individual sqlite calls for weeklypull, and brought them into line with using the db module (which will minimize concurrent access, which seemed to be causing db locks), IMP: Cleaned up some code and shuffled some functions so they are in more appropriate locations
2015-03-27 17:27:59 +00:00
break
else :
logger . fdebug ( module + ' Did not match to year within filename : ' + str ( issyear ) )
multiplechk = True
if multiplechk == True :
n + = 1
continue
#this will detect duplicate filenames within the same directory.
for di in annualdupechk :
IMP: 32P legacy mode (rss feed only) and Auth mode (user/pass) available now. With Auth mode, will monitor all of personal notification feeds, IMP: Changed access methods to 32P for retrieving links, FIX:(#1007) Fixed error for one-off downloading via weekly pull list, FIX: (#1004) Fix for post-processing errors (missing files), FIX: (#1001) Extra scripts & pre-scripts will now accept bash scripts, FIX: Annuals were being incorrectly tallied in Have total as well as not having status correctly updated, IMP: Fixed some discrepencies with the provider_order generation line, IMP: Made some additional adjustments to the file-checking to account for some additional variations in naming, FIX: Fixed traceback errors when folder monitor was running and wouldn't be able to complete the post-processing properly, FIX: Cleanup code and some other small fixes....
2015-05-09 02:39:28 +00:00
if di [ ' fcdigit ' ] == fcdigit and di [ ' issueid ' ] == reann [ ' IssueID ' ] :
IMP: Cleaned up interface for StoryArcs / Story Arc Details, IMP: Cleaned up interface for Reading list Management, IMP: Added better reading list management - new status (added, downloaded, read), IMP: Added sync option for use with another device for reading list transfer (ie. tablet) Android only, IMP: Autopopulate new weekly pull releases to reading list, IMP: 'Watch' option in weekly pull list now fully functional. Will watch CV for series' that do not have any series data yet as they are new starting series. Will auto-add once available, IMP: Auto-watch check is run after every refresh/recreate of the weeklypull list, IMP: Improved the Add a Series option where it will now look for issues that are 'new' or 'wanted' during add sequence, IMP: Main page interface now has coloured have/total bars to denote series completion, IMP: New scheduler / threading locks in place in an attempt to avoid database locks, FIX: Removed some erroneous locking that was going on when importing a directory was being run, IMP: Stat counter now present when post-processing multiple issues in sequence, FIX: for issue number error when post-processing and issue number was a non-alphanumeric, FIX: for metatagging: when original file was .cbz, would try to convert and fail, FIX: for issues that were negative and were preceeded by a # in the filename (filechecker), FIX: for publisher having non-alphanumeric character in name when attempting to determine publisher, FIX: if annuals enabled, would incorrectly show as being 'already in library' when viewing search results if results constained annuals, FIX:(#944) for incorrect nzbname being used when post-processing was being performed from an nzb client (experimental mainly), IMP: Turned off logging for ComicVine API counter, FIX: Added retry attempts when connecting to ComicVine in order to avoid errors when adding a series, IMP:(#963) Added ability to add snatched to filter when viewing Wanted issues on Wanted tab, FIX: When importing and then selecting a series to import via the select screen, will now flip back to the importresults and add the selected series in the background, IMP:(#952) Main page is now sorted in ascending order by Continuing/Ended status (and subbed by whether is Active/Paused).Custom sorting is still available, FIX: Dupecheck will now automatically assume existing 0-byte files are to be overwritten when performing post-processing, FIX: If publication date for series contained a '?' (usually with brand new series) will force to 'Present' to allow for pull-list comparisons to take place, FIX: Mylar will now disallow search results which have 'covers only' or 'variant' in the filename, IMP: Better nzbname generation/retrieval (will check inside nzb for possible names) to be used when post-processing, IMP: DB Update will now perform update to all active comics in descending order by Latest Date (instead of random order), FIX: Enforce the 5hr limit rule when running DB update (will only update series that haven't been updated in >5 hours), FIX: Annuals will now have/retain the proper status upon doing DB Update, FIX: Have totals will now be updated when doing a recheck files (sometimes wouldn't get updated depending on various states of status'), FIX:(#966) Added urllib2.URLError exeception trap when attempting to check Git for updates, IMP: Removed the individual sqlite calls for weeklypull, and brought them into line with using the db module (which will minimize concurrent access, which seemed to be causing db locks), IMP: Cleaned up some code and shuffled some functions so they are in more appropriate locations
2015-03-27 17:27:59 +00:00
#base off of config - base duplication keep on filesize or file-type (or both)
logger . fdebug ( ' [DUPECHECK] Duplicate issue detected [ ' + di [ ' filename ' ] + ' ] [ ' + tmpfc [ ' ComicFilename ' ] + ' ] ' )
2017-10-25 18:33:56 +00:00
# mylar.CONFIG.DUPECONSTRAINT = 'filesize' / 'filetype-cbr' / 'filetype-cbz'
logger . fdebug ( ' [DUPECHECK] Based on duplication preferences I will retain based on : ' + mylar . CONFIG . DUPECONSTRAINT )
IMP: Cleaned up interface for StoryArcs / Story Arc Details, IMP: Cleaned up interface for Reading list Management, IMP: Added better reading list management - new status (added, downloaded, read), IMP: Added sync option for use with another device for reading list transfer (ie. tablet) Android only, IMP: Autopopulate new weekly pull releases to reading list, IMP: 'Watch' option in weekly pull list now fully functional. Will watch CV for series' that do not have any series data yet as they are new starting series. Will auto-add once available, IMP: Auto-watch check is run after every refresh/recreate of the weeklypull list, IMP: Improved the Add a Series option where it will now look for issues that are 'new' or 'wanted' during add sequence, IMP: Main page interface now has coloured have/total bars to denote series completion, IMP: New scheduler / threading locks in place in an attempt to avoid database locks, FIX: Removed some erroneous locking that was going on when importing a directory was being run, IMP: Stat counter now present when post-processing multiple issues in sequence, FIX: for issue number error when post-processing and issue number was a non-alphanumeric, FIX: for metatagging: when original file was .cbz, would try to convert and fail, FIX: for issues that were negative and were preceeded by a # in the filename (filechecker), FIX: for publisher having non-alphanumeric character in name when attempting to determine publisher, FIX: if annuals enabled, would incorrectly show as being 'already in library' when viewing search results if results constained annuals, FIX:(#944) for incorrect nzbname being used when post-processing was being performed from an nzb client (experimental mainly), IMP: Turned off logging for ComicVine API counter, FIX: Added retry attempts when connecting to ComicVine in order to avoid errors when adding a series, IMP:(#963) Added ability to add snatched to filter when viewing Wanted issues on Wanted tab, FIX: When importing and then selecting a series to import via the select screen, will now flip back to the importresults and add the selected series in the background, IMP:(#952) Main page is now sorted in ascending order by Continuing/Ended status (and subbed by whether is Active/Paused).Custom sorting is still available, FIX: Dupecheck will now automatically assume existing 0-byte files are to be overwritten when performing post-processing, FIX: If publication date for series contained a '?' (usually with brand new series) will force to 'Present' to allow for pull-list comparisons to take place, FIX: Mylar will now disallow search results which have 'covers only' or 'variant' in the filename, IMP: Better nzbname generation/retrieval (will check inside nzb for possible names) to be used when post-processing, IMP: DB Update will now perform update to all active comics in descending order by Latest Date (instead of random order), FIX: Enforce the 5hr limit rule when running DB update (will only update series that haven't been updated in >5 hours), FIX: Annuals will now have/retain the proper status upon doing DB Update, FIX: Have totals will now be updated when doing a recheck files (sometimes wouldn't get updated depending on various states of status'), FIX:(#966) Added urllib2.URLError exeception trap when attempting to check Git for updates, IMP: Removed the individual sqlite calls for weeklypull, and brought them into line with using the db module (which will minimize concurrent access, which seemed to be causing db locks), IMP: Cleaned up some code and shuffled some functions so they are in more appropriate locations
2015-03-27 17:27:59 +00:00
removedupe = False
2017-10-25 18:33:56 +00:00
if ' cbr ' in mylar . CONFIG . DUPECONSTRAINT or ' cbz ' in mylar . CONFIG . DUPECONSTRAINT :
if ' cbr ' in mylar . CONFIG . DUPECONSTRAINT :
IMP: Cleaned up interface for StoryArcs / Story Arc Details, IMP: Cleaned up interface for Reading list Management, IMP: Added better reading list management - new status (added, downloaded, read), IMP: Added sync option for use with another device for reading list transfer (ie. tablet) Android only, IMP: Autopopulate new weekly pull releases to reading list, IMP: 'Watch' option in weekly pull list now fully functional. Will watch CV for series' that do not have any series data yet as they are new starting series. Will auto-add once available, IMP: Auto-watch check is run after every refresh/recreate of the weeklypull list, IMP: Improved the Add a Series option where it will now look for issues that are 'new' or 'wanted' during add sequence, IMP: Main page interface now has coloured have/total bars to denote series completion, IMP: New scheduler / threading locks in place in an attempt to avoid database locks, FIX: Removed some erroneous locking that was going on when importing a directory was being run, IMP: Stat counter now present when post-processing multiple issues in sequence, FIX: for issue number error when post-processing and issue number was a non-alphanumeric, FIX: for metatagging: when original file was .cbz, would try to convert and fail, FIX: for issues that were negative and were preceeded by a # in the filename (filechecker), FIX: for publisher having non-alphanumeric character in name when attempting to determine publisher, FIX: if annuals enabled, would incorrectly show as being 'already in library' when viewing search results if results constained annuals, FIX:(#944) for incorrect nzbname being used when post-processing was being performed from an nzb client (experimental mainly), IMP: Turned off logging for ComicVine API counter, FIX: Added retry attempts when connecting to ComicVine in order to avoid errors when adding a series, IMP:(#963) Added ability to add snatched to filter when viewing Wanted issues on Wanted tab, FIX: When importing and then selecting a series to import via the select screen, will now flip back to the importresults and add the selected series in the background, IMP:(#952) Main page is now sorted in ascending order by Continuing/Ended status (and subbed by whether is Active/Paused).Custom sorting is still available, FIX: Dupecheck will now automatically assume existing 0-byte files are to be overwritten when performing post-processing, FIX: If publication date for series contained a '?' (usually with brand new series) will force to 'Present' to allow for pull-list comparisons to take place, FIX: Mylar will now disallow search results which have 'covers only' or 'variant' in the filename, IMP: Better nzbname generation/retrieval (will check inside nzb for possible names) to be used when post-processing, IMP: DB Update will now perform update to all active comics in descending order by Latest Date (instead of random order), FIX: Enforce the 5hr limit rule when running DB update (will only update series that haven't been updated in >5 hours), FIX: Annuals will now have/retain the proper status upon doing DB Update, FIX: Have totals will now be updated when doing a recheck files (sometimes wouldn't get updated depending on various states of status'), FIX:(#966) Added urllib2.URLError exeception trap when attempting to check Git for updates, IMP: Removed the individual sqlite calls for weeklypull, and brought them into line with using the db module (which will minimize concurrent access, which seemed to be causing db locks), IMP: Cleaned up some code and shuffled some functions so they are in more appropriate locations
2015-03-27 17:27:59 +00:00
#this has to be configured in config - either retain cbr or cbz.
if tmpfc [ ' ComicFilename ' ] . endswith ( ' .cbz ' ) :
#keep di['filename']
logger . fdebug ( ' [DUPECHECK-CBR PRIORITY] [# ' + reann [ ' Issue_Number ' ] + ' ] Retaining currently scanned in file : ' + di [ ' filename ' ] )
annualdupe = " yes "
break
else :
#keep tmpfc['ComicFilename']
logger . fdebug ( ' [DUPECHECK-CBR PRIORITY] [# ' + reann [ ' Issue_Number ' ] + ' ] Retaining newly scanned in file : ' + tmpfc [ ' ComicFilename ' ] )
removedupe = True
2017-10-25 18:33:56 +00:00
elif ' cbz ' in mylar . CONFIG . DUPECONSTRAINT :
IMP: Cleaned up interface for StoryArcs / Story Arc Details, IMP: Cleaned up interface for Reading list Management, IMP: Added better reading list management - new status (added, downloaded, read), IMP: Added sync option for use with another device for reading list transfer (ie. tablet) Android only, IMP: Autopopulate new weekly pull releases to reading list, IMP: 'Watch' option in weekly pull list now fully functional. Will watch CV for series' that do not have any series data yet as they are new starting series. Will auto-add once available, IMP: Auto-watch check is run after every refresh/recreate of the weeklypull list, IMP: Improved the Add a Series option where it will now look for issues that are 'new' or 'wanted' during add sequence, IMP: Main page interface now has coloured have/total bars to denote series completion, IMP: New scheduler / threading locks in place in an attempt to avoid database locks, FIX: Removed some erroneous locking that was going on when importing a directory was being run, IMP: Stat counter now present when post-processing multiple issues in sequence, FIX: for issue number error when post-processing and issue number was a non-alphanumeric, FIX: for metatagging: when original file was .cbz, would try to convert and fail, FIX: for issues that were negative and were preceeded by a # in the filename (filechecker), FIX: for publisher having non-alphanumeric character in name when attempting to determine publisher, FIX: if annuals enabled, would incorrectly show as being 'already in library' when viewing search results if results constained annuals, FIX:(#944) for incorrect nzbname being used when post-processing was being performed from an nzb client (experimental mainly), IMP: Turned off logging for ComicVine API counter, FIX: Added retry attempts when connecting to ComicVine in order to avoid errors when adding a series, IMP:(#963) Added ability to add snatched to filter when viewing Wanted issues on Wanted tab, FIX: When importing and then selecting a series to import via the select screen, will now flip back to the importresults and add the selected series in the background, IMP:(#952) Main page is now sorted in ascending order by Continuing/Ended status (and subbed by whether is Active/Paused).Custom sorting is still available, FIX: Dupecheck will now automatically assume existing 0-byte files are to be overwritten when performing post-processing, FIX: If publication date for series contained a '?' (usually with brand new series) will force to 'Present' to allow for pull-list comparisons to take place, FIX: Mylar will now disallow search results which have 'covers only' or 'variant' in the filename, IMP: Better nzbname generation/retrieval (will check inside nzb for possible names) to be used when post-processing, IMP: DB Update will now perform update to all active comics in descending order by Latest Date (instead of random order), FIX: Enforce the 5hr limit rule when running DB update (will only update series that haven't been updated in >5 hours), FIX: Annuals will now have/retain the proper status upon doing DB Update, FIX: Have totals will now be updated when doing a recheck files (sometimes wouldn't get updated depending on various states of status'), FIX:(#966) Added urllib2.URLError exeception trap when attempting to check Git for updates, IMP: Removed the individual sqlite calls for weeklypull, and brought them into line with using the db module (which will minimize concurrent access, which seemed to be causing db locks), IMP: Cleaned up some code and shuffled some functions so they are in more appropriate locations
2015-03-27 17:27:59 +00:00
if tmpfc [ ' ComicFilename ' ] . endswith ( ' .cbr ' ) :
#keep di['filename']
logger . fdebug ( ' [DUPECHECK-CBZ PRIORITY] [# ' + reann [ ' Issue_Number ' ] + ' ] Retaining currently scanned in filename : ' + di [ ' filename ' ] )
annualdupe = " yes "
break
else :
#keep tmpfc['ComicFilename']
logger . fdebug ( ' [DUPECHECK-CBZ PRIORITY] [# ' + reann [ ' Issue_Number ' ] + ' ] Retaining newly scanned in filename : ' + tmpfc [ ' ComicFilename ' ] )
removedupe = True
2015-05-22 08:32:51 +00:00
2017-10-25 18:33:56 +00:00
if mylar . CONFIG . DUPECONSTRAINT == ' filesize ' :
IMP: Cleaned up interface for StoryArcs / Story Arc Details, IMP: Cleaned up interface for Reading list Management, IMP: Added better reading list management - new status (added, downloaded, read), IMP: Added sync option for use with another device for reading list transfer (ie. tablet) Android only, IMP: Autopopulate new weekly pull releases to reading list, IMP: 'Watch' option in weekly pull list now fully functional. Will watch CV for series' that do not have any series data yet as they are new starting series. Will auto-add once available, IMP: Auto-watch check is run after every refresh/recreate of the weeklypull list, IMP: Improved the Add a Series option where it will now look for issues that are 'new' or 'wanted' during add sequence, IMP: Main page interface now has coloured have/total bars to denote series completion, IMP: New scheduler / threading locks in place in an attempt to avoid database locks, FIX: Removed some erroneous locking that was going on when importing a directory was being run, IMP: Stat counter now present when post-processing multiple issues in sequence, FIX: for issue number error when post-processing and issue number was a non-alphanumeric, FIX: for metatagging: when original file was .cbz, would try to convert and fail, FIX: for issues that were negative and were preceeded by a # in the filename (filechecker), FIX: for publisher having non-alphanumeric character in name when attempting to determine publisher, FIX: if annuals enabled, would incorrectly show as being 'already in library' when viewing search results if results constained annuals, FIX:(#944) for incorrect nzbname being used when post-processing was being performed from an nzb client (experimental mainly), IMP: Turned off logging for ComicVine API counter, FIX: Added retry attempts when connecting to ComicVine in order to avoid errors when adding a series, IMP:(#963) Added ability to add snatched to filter when viewing Wanted issues on Wanted tab, FIX: When importing and then selecting a series to import via the select screen, will now flip back to the importresults and add the selected series in the background, IMP:(#952) Main page is now sorted in ascending order by Continuing/Ended status (and subbed by whether is Active/Paused).Custom sorting is still available, FIX: Dupecheck will now automatically assume existing 0-byte files are to be overwritten when performing post-processing, FIX: If publication date for series contained a '?' (usually with brand new series) will force to 'Present' to allow for pull-list comparisons to take place, FIX: Mylar will now disallow search results which have 'covers only' or 'variant' in the filename, IMP: Better nzbname generation/retrieval (will check inside nzb for possible names) to be used when post-processing, IMP: DB Update will now perform update to all active comics in descending order by Latest Date (instead of random order), FIX: Enforce the 5hr limit rule when running DB update (will only update series that haven't been updated in >5 hours), FIX: Annuals will now have/retain the proper status upon doing DB Update, FIX: Have totals will now be updated when doing a recheck files (sometimes wouldn't get updated depending on various states of status'), FIX:(#966) Added urllib2.URLError exeception trap when attempting to check Git for updates, IMP: Removed the individual sqlite calls for weeklypull, and brought them into line with using the db module (which will minimize concurrent access, which seemed to be causing db locks), IMP: Cleaned up some code and shuffled some functions so they are in more appropriate locations
2015-03-27 17:27:59 +00:00
if tmpfc [ ' ComicSize ' ] < = di [ ' filesize ' ] :
logger . fdebug ( ' [DUPECHECK-FILESIZE PRIORITY] [# ' + reann [ ' Issue_Number ' ] + ' ] Retaining currently scanned in filename : ' + di [ ' filename ' ] )
annualdupe = " yes "
break
else :
logger . fdebug ( ' [DUPECHECK-FILESIZE PRIORITY] [# ' + reann [ ' Issue_Number ' ] + ' ] Retaining newly scanned in filename : ' + tmpfc [ ' ComicFilename ' ] )
removedupe = True
if removedupe :
#need to remove the entry from issuedupechk so can add new one.
#tuple(y for y in x if y) for x in a
annualdupe_temp = [ ]
tmphavefiles = 0
for x in annualdupechk :
logger . fdebug ( ' Comparing x: ' + x [ ' filename ' ] + ' to di: ' + di [ ' filename ' ] )
if x [ ' filename ' ] != di [ ' filename ' ] :
annualdupe_temp . append ( x )
tmphavefiles + = 1
annualdupechk = annualdupe_temp
FIX: When manually post-processing files, if files were in more than one subdirectory level deep would use incorrect path resulting in error, FIX:(#1261, #1262) Traceback error when searching and returned value had volume, FIX: If filenames had the Volume label after the issue number, would fail to recognize the series title properly resulting in errors during scanning, FIX: Removing imported series from the Import List via dropdowns, IMP: Improved upon imports matching up to series that have multiple volumes by comparing issue counts & years, IMP: If an import is determined to need manual intervention, will display only the results that have passed the issue counts/years/name matching as opposed to showing all the results, FIX: When moving files during an import will now only move those files that are attached to the specific grouping by series title/volume, FIX: Import Manual Intervention results sortable by Year column now, FIX: Fixed an errant callback to home during configuration saves that caused web browser error when used with reverse proxy, FIX: During story arc searching if a story arc had a '-' in the arc name, would not be able to find it, FIX: When adding a story-arc, if files pre-existed in the directory attached to the arc would error, FIX: When adding a series, if was used for Folder Format and the volume label didn't exist would label directory with 'noversion', FIX: Selecting to metatag annuals via the manual metatag option on the series detail page, FIX: When retrieving from KAT and response returned was a 404 would ignore it, now will retry alternate torcache site
2016-04-25 15:35:12 +00:00
havefiles = tmphavefiles + len ( issuedupechk )
IMP: Cleaned up interface for StoryArcs / Story Arc Details, IMP: Cleaned up interface for Reading list Management, IMP: Added better reading list management - new status (added, downloaded, read), IMP: Added sync option for use with another device for reading list transfer (ie. tablet) Android only, IMP: Autopopulate new weekly pull releases to reading list, IMP: 'Watch' option in weekly pull list now fully functional. Will watch CV for series' that do not have any series data yet as they are new starting series. Will auto-add once available, IMP: Auto-watch check is run after every refresh/recreate of the weeklypull list, IMP: Improved the Add a Series option where it will now look for issues that are 'new' or 'wanted' during add sequence, IMP: Main page interface now has coloured have/total bars to denote series completion, IMP: New scheduler / threading locks in place in an attempt to avoid database locks, FIX: Removed some erroneous locking that was going on when importing a directory was being run, IMP: Stat counter now present when post-processing multiple issues in sequence, FIX: for issue number error when post-processing and issue number was a non-alphanumeric, FIX: for metatagging: when original file was .cbz, would try to convert and fail, FIX: for issues that were negative and were preceeded by a # in the filename (filechecker), FIX: for publisher having non-alphanumeric character in name when attempting to determine publisher, FIX: if annuals enabled, would incorrectly show as being 'already in library' when viewing search results if results constained annuals, FIX:(#944) for incorrect nzbname being used when post-processing was being performed from an nzb client (experimental mainly), IMP: Turned off logging for ComicVine API counter, FIX: Added retry attempts when connecting to ComicVine in order to avoid errors when adding a series, IMP:(#963) Added ability to add snatched to filter when viewing Wanted issues on Wanted tab, FIX: When importing and then selecting a series to import via the select screen, will now flip back to the importresults and add the selected series in the background, IMP:(#952) Main page is now sorted in ascending order by Continuing/Ended status (and subbed by whether is Active/Paused).Custom sorting is still available, FIX: Dupecheck will now automatically assume existing 0-byte files are to be overwritten when performing post-processing, FIX: If publication date for series contained a '?' (usually with brand new series) will force to 'Present' to allow for pull-list comparisons to take place, FIX: Mylar will now disallow search results which have 'covers only' or 'variant' in the filename, IMP: Better nzbname generation/retrieval (will check inside nzb for possible names) to be used when post-processing, IMP: DB Update will now perform update to all active comics in descending order by Latest Date (instead of random order), FIX: Enforce the 5hr limit rule when running DB update (will only update series that haven't been updated in >5 hours), FIX: Annuals will now have/retain the proper status upon doing DB Update, FIX: Have totals will now be updated when doing a recheck files (sometimes wouldn't get updated depending on various states of status'), FIX:(#966) Added urllib2.URLError exeception trap when attempting to check Git for updates, IMP: Removed the individual sqlite calls for weeklypull, and brought them into line with using the db module (which will minimize concurrent access, which seemed to be causing db locks), IMP: Cleaned up some code and shuffled some functions so they are in more appropriate locations
2015-03-27 17:27:59 +00:00
foundchk = False
2013-07-01 05:19:15 +00:00
break
IMP: Cleaned up interface for StoryArcs / Story Arc Details, IMP: Cleaned up interface for Reading list Management, IMP: Added better reading list management - new status (added, downloaded, read), IMP: Added sync option for use with another device for reading list transfer (ie. tablet) Android only, IMP: Autopopulate new weekly pull releases to reading list, IMP: 'Watch' option in weekly pull list now fully functional. Will watch CV for series' that do not have any series data yet as they are new starting series. Will auto-add once available, IMP: Auto-watch check is run after every refresh/recreate of the weeklypull list, IMP: Improved the Add a Series option where it will now look for issues that are 'new' or 'wanted' during add sequence, IMP: Main page interface now has coloured have/total bars to denote series completion, IMP: New scheduler / threading locks in place in an attempt to avoid database locks, FIX: Removed some erroneous locking that was going on when importing a directory was being run, IMP: Stat counter now present when post-processing multiple issues in sequence, FIX: for issue number error when post-processing and issue number was a non-alphanumeric, FIX: for metatagging: when original file was .cbz, would try to convert and fail, FIX: for issues that were negative and were preceeded by a # in the filename (filechecker), FIX: for publisher having non-alphanumeric character in name when attempting to determine publisher, FIX: if annuals enabled, would incorrectly show as being 'already in library' when viewing search results if results constained annuals, FIX:(#944) for incorrect nzbname being used when post-processing was being performed from an nzb client (experimental mainly), IMP: Turned off logging for ComicVine API counter, FIX: Added retry attempts when connecting to ComicVine in order to avoid errors when adding a series, IMP:(#963) Added ability to add snatched to filter when viewing Wanted issues on Wanted tab, FIX: When importing and then selecting a series to import via the select screen, will now flip back to the importresults and add the selected series in the background, IMP:(#952) Main page is now sorted in ascending order by Continuing/Ended status (and subbed by whether is Active/Paused).Custom sorting is still available, FIX: Dupecheck will now automatically assume existing 0-byte files are to be overwritten when performing post-processing, FIX: If publication date for series contained a '?' (usually with brand new series) will force to 'Present' to allow for pull-list comparisons to take place, FIX: Mylar will now disallow search results which have 'covers only' or 'variant' in the filename, IMP: Better nzbname generation/retrieval (will check inside nzb for possible names) to be used when post-processing, IMP: DB Update will now perform update to all active comics in descending order by Latest Date (instead of random order), FIX: Enforce the 5hr limit rule when running DB update (will only update series that haven't been updated in >5 hours), FIX: Annuals will now have/retain the proper status upon doing DB Update, FIX: Have totals will now be updated when doing a recheck files (sometimes wouldn't get updated depending on various states of status'), FIX:(#966) Added urllib2.URLError exeception trap when attempting to check Git for updates, IMP: Removed the individual sqlite calls for weeklypull, and brought them into line with using the db module (which will minimize concurrent access, which seemed to be causing db locks), IMP: Cleaned up some code and shuffled some functions so they are in more appropriate locations
2015-03-27 17:27:59 +00:00
if annualdupe == " no " :
if foundchk == False :
IMP:(#753) Better handling of cbz files when using metatagging option, IMP: Added option for MetaTagging to enable/disable overwriting of existing metadata on cbz files, IMP: Post-Processing will now delete folders that are identical to filenames after successful post-processing, FIX:(#756) Fix for Manual Search for an Issue which resulted in traceback errors upon not finding any matches (Windows), IMP: Added module-name to logging for some modules to help troubleshooting (eventually all will be done), IMP: If a filename contained repeated characters for spaces (ie. ___ or ...), it would fail on post-processing due to the nzbname being off, IMP: Monitoring of CV API will now be persitent on restarts/shutdowns/etc, IMP: Added a maximum CV API global so that it can be adjusted for monitoring more easily should API requirements change, IMP: When Adding new series, will now only mark issues that are actually upcoming as Wanted (based on Store Date)
2014-06-22 08:32:45 +00:00
logger . fdebug ( module + ' Matched...annual issue: ' + rescan [ ' ComicName ' ] + ' # ' + str ( reann [ ' Issue_Number ' ] ) + ' --- ' + str ( int_iss ) )
2013-07-01 05:19:15 +00:00
havefiles + = 1
haveissue = " yes "
2016-10-05 14:12:58 +00:00
isslocation = helpers . conversion ( tmpfc [ ' ComicFilename ' ] )
2013-07-01 05:19:15 +00:00
issSize = str ( tmpfc [ ' ComicSize ' ] )
FIX: When retrieving feeds from 32p and in Auth mode, personal notification feeds contained some invalid html entries that weren't removed properly resulting in no results when attempting to match for downloading, FIX: When searching 32P, if title had a '/' within the title - Mylar would mistakingly skip it due to some previous exceptions that were made for CBT, FIX: Main page would quickly display & hide the have% column instead of always being hidden, FIX: Adjusted some incorrect spacing for non-alphanumeric characters when comparing search results (should result in better matching hopefully), FIX: When adding a series and the most recent issue was present on the weekly-pull list, it would sometimes not mark it as Wanted and auto-attempt to search for it (if auto mark Upcoming enabled), FIX: Added Test Connection button for 32P where it will test logon credentials as well as if Captcha is present, IMP: If captcha is enabled for 32p and signon is required because keys are stale, will not send authentication information and will just bypass as a provider, IMP: Test Connection button added for SABnzbd, IMP: Added ability to directly add torrents to rtorrent and apply label + download directory options (config.ini only atm), FIX: If a search result had a 'vol.' label in it, depending on how the format of the label was mylar would refuse to remove the volume which resulted in failed matches (also fixed a similar issue with failing to remove the volume label when comparing search results), FIX: When filechecking, if a series had a - in the title, will now account for it properly, IMP: Completely redid the filecheck module which allows for integration into other modules as well as more detailed failure logs, IMP: Added Dynamic handder integration into filechecker and subsequent modules that use it which allows for special characters to be replaced with any other type of character, IMP: Manual post-processing speed improved greatly due to new usage of filecheck module, IMP: Importer backend code redone to include new filecheck module, IMP: Added status/counter to import process, IMP: Added force unlock option to importer for failed imports, IMP: Added new status to Import labelled as 'Manual Intervention' for imports that need the user to manually select an option from an available list, FIX: When import said there were search results to view, but none available - would blank screen, IMP: Added a failure log entry showing all the failed files that weren't able to be scanned in during an import (will be in GUI eventually), IMP: if only partial metadata is available during import, Mylar will attempt to use what's available from the metatagging instead of picking all of one/other, IMP: Better grouping of series/volumes when viewing the import results page as well as now indicating if annuals are present within the files, IMP: Added a file-icon beside each imported item on the import result page which allows the user to view the files that are associated with the given series grouping, IMP: Added a blacklisted_publishers option to config.ini which will blacklist specific publishers from being returned during search / import results, FIX: If duplicate dump folder had a value, but duplicate dump wasn't enabled - would still use the duplicate dump folder during post-processing runs, FIX: (#1194) Patch to allow for fixed H1 elements for title (thnx chazlarson), FIX: Removed UnRAR dependency checks in cmtagmylar since not being used anymore, FIX: Fixed a problem with non-ascii characters being recognized during a file-check in certain cases, IMP: Attmept by Mylar to grab an alternate jpg from file when viewing the issue details if it complies with the naming conventions, FIX: Fixed some metatagging issues with ComicBookLover tags not being handled properly if they didn't exist, IMP: Dupecheck now has a failback if it's comparing a cbr/cbr, cbz/cbz and cbr/cbz-priority is enabled, FIX: Quick check added for when adding/refreshing a comic that if a cover already existed, it would delete the cover prior to the attempt to retrieve it, IMP: Added some additional handling for when searching/adding fails, FIX: If a story arc didn't have proper issue dates (or invalid ones) would error out on loading the story arc main page - usually when arcs were imported using a cbl file.
2016-04-07 17:09:06 +00:00
logger . fdebug ( module + ' .......filename: ' + isslocation )
IMP:(#753) Better handling of cbz files when using metatagging option, IMP: Added option for MetaTagging to enable/disable overwriting of existing metadata on cbz files, IMP: Post-Processing will now delete folders that are identical to filenames after successful post-processing, FIX:(#756) Fix for Manual Search for an Issue which resulted in traceback errors upon not finding any matches (Windows), IMP: Added module-name to logging for some modules to help troubleshooting (eventually all will be done), IMP: If a filename contained repeated characters for spaces (ie. ___ or ...), it would fail on post-processing due to the nzbname being off, IMP: Monitoring of CV API will now be persitent on restarts/shutdowns/etc, IMP: Added a maximum CV API global so that it can be adjusted for monitoring more easily should API requirements change, IMP: When Adding new series, will now only mark issues that are actually upcoming as Wanted (based on Store Date)
2014-06-22 08:32:45 +00:00
logger . fdebug ( module + ' .......filesize: ' + str ( tmpfc [ ' ComicSize ' ] ) )
2013-07-01 05:19:15 +00:00
# to avoid duplicate issues which screws up the count...let's store the filename issues then
# compare earlier...
FIX: undefined on filter box on startup, IMP: Added Meta-Tagging options on a series / issue basis on comic details screen, IMP: Issue Information is now available per issue and is extracted currently from the cbz file to display (if no cbz is present, the option isn't available), IMP: Failed Download handling is implemented and available in GUI - required to replace existing autoProcessComics.py and ComicRN.py scripts, IMP: Added ability to specify post-processing script instead of ComicRN.py, IMP: Added abilty to edit the issue date for a given issue by simply clicking on it - this will help to avoid dates that have incorrect or 0000-00-00, IMP: Story Arc searching is working (not adding yet), IMP: Added Archived/Ignored options to Upcoming/Wanted tab, IMP: Fixed some alignment and display issues on the Upcoming section, IMP: Added better directory handling for Updating Comic Location when it needs to get changed for all existing series (locmove in config.ini), IMP: Added better handling for unicode characters in series titles when searching / filechecking, IMP: When adding a new series with no data, Mylar would error out (now will add and just retain 0 issues), FIX: When year was fuzzied, search would still attempt to do a date-check comparison and say everything failed, IMP: Better handling of nzb names when retaining for post-processing comparisons, IMP: Future Upcoming now will use actual shipping date of issue if available in order to see if issue is available, FIX: If annuals were enabled, refreshing a series would put some issues into an Archived status because the actual counts would be off, IMP: When file checking, Alternate Naming would be searched last which resulted in matching incorrectly to the series title or other alternate naming for the given series, now will check the Alternate Naming first for a match, then drop back down to the series name itself otherwise, IMP: Improved Annual detection when integrated with a given series, IMP: Improved the checking of the future Upcoming list for issues marked as Wanted but not available yet and then auto-adding, IMP: Improved upon story arc checking for missing issues / searching for wanted, IMP: Enabling Annuals support now within Configuration GUI, bunch of other things....
2014-07-28 19:28:09 +00:00
annualdupechk . append ( { ' fcdigit ' : int ( fcdigit ) ,
IMP: Cleaned up interface for StoryArcs / Story Arc Details, IMP: Cleaned up interface for Reading list Management, IMP: Added better reading list management - new status (added, downloaded, read), IMP: Added sync option for use with another device for reading list transfer (ie. tablet) Android only, IMP: Autopopulate new weekly pull releases to reading list, IMP: 'Watch' option in weekly pull list now fully functional. Will watch CV for series' that do not have any series data yet as they are new starting series. Will auto-add once available, IMP: Auto-watch check is run after every refresh/recreate of the weeklypull list, IMP: Improved the Add a Series option where it will now look for issues that are 'new' or 'wanted' during add sequence, IMP: Main page interface now has coloured have/total bars to denote series completion, IMP: New scheduler / threading locks in place in an attempt to avoid database locks, FIX: Removed some erroneous locking that was going on when importing a directory was being run, IMP: Stat counter now present when post-processing multiple issues in sequence, FIX: for issue number error when post-processing and issue number was a non-alphanumeric, FIX: for metatagging: when original file was .cbz, would try to convert and fail, FIX: for issues that were negative and were preceeded by a # in the filename (filechecker), FIX: for publisher having non-alphanumeric character in name when attempting to determine publisher, FIX: if annuals enabled, would incorrectly show as being 'already in library' when viewing search results if results constained annuals, FIX:(#944) for incorrect nzbname being used when post-processing was being performed from an nzb client (experimental mainly), IMP: Turned off logging for ComicVine API counter, FIX: Added retry attempts when connecting to ComicVine in order to avoid errors when adding a series, IMP:(#963) Added ability to add snatched to filter when viewing Wanted issues on Wanted tab, FIX: When importing and then selecting a series to import via the select screen, will now flip back to the importresults and add the selected series in the background, IMP:(#952) Main page is now sorted in ascending order by Continuing/Ended status (and subbed by whether is Active/Paused).Custom sorting is still available, FIX: Dupecheck will now automatically assume existing 0-byte files are to be overwritten when performing post-processing, FIX: If publication date for series contained a '?' (usually with brand new series) will force to 'Present' to allow for pull-list comparisons to take place, FIX: Mylar will now disallow search results which have 'covers only' or 'variant' in the filename, IMP: Better nzbname generation/retrieval (will check inside nzb for possible names) to be used when post-processing, IMP: DB Update will now perform update to all active comics in descending order by Latest Date (instead of random order), FIX: Enforce the 5hr limit rule when running DB update (will only update series that haven't been updated in >5 hours), FIX: Annuals will now have/retain the proper status upon doing DB Update, FIX: Have totals will now be updated when doing a recheck files (sometimes wouldn't get updated depending on various states of status'), FIX:(#966) Added urllib2.URLError exeception trap when attempting to check Git for updates, IMP: Removed the individual sqlite calls for weeklypull, and brought them into line with using the db module (which will minimize concurrent access, which seemed to be causing db locks), IMP: Cleaned up some code and shuffled some functions so they are in more appropriate locations
2015-03-27 17:27:59 +00:00
' anncomicid ' : ANNComicID ,
' filename ' : tmpfc [ ' ComicFilename ' ] ,
' filesize ' : tmpfc [ ' ComicSize ' ] ,
' issueyear ' : issyear ,
' issueid ' : reann [ ' IssueID ' ] } )
2013-07-01 05:19:15 +00:00
break
IMP: Cleaned up interface for StoryArcs / Story Arc Details, IMP: Cleaned up interface for Reading list Management, IMP: Added better reading list management - new status (added, downloaded, read), IMP: Added sync option for use with another device for reading list transfer (ie. tablet) Android only, IMP: Autopopulate new weekly pull releases to reading list, IMP: 'Watch' option in weekly pull list now fully functional. Will watch CV for series' that do not have any series data yet as they are new starting series. Will auto-add once available, IMP: Auto-watch check is run after every refresh/recreate of the weeklypull list, IMP: Improved the Add a Series option where it will now look for issues that are 'new' or 'wanted' during add sequence, IMP: Main page interface now has coloured have/total bars to denote series completion, IMP: New scheduler / threading locks in place in an attempt to avoid database locks, FIX: Removed some erroneous locking that was going on when importing a directory was being run, IMP: Stat counter now present when post-processing multiple issues in sequence, FIX: for issue number error when post-processing and issue number was a non-alphanumeric, FIX: for metatagging: when original file was .cbz, would try to convert and fail, FIX: for issues that were negative and were preceeded by a # in the filename (filechecker), FIX: for publisher having non-alphanumeric character in name when attempting to determine publisher, FIX: if annuals enabled, would incorrectly show as being 'already in library' when viewing search results if results constained annuals, FIX:(#944) for incorrect nzbname being used when post-processing was being performed from an nzb client (experimental mainly), IMP: Turned off logging for ComicVine API counter, FIX: Added retry attempts when connecting to ComicVine in order to avoid errors when adding a series, IMP:(#963) Added ability to add snatched to filter when viewing Wanted issues on Wanted tab, FIX: When importing and then selecting a series to import via the select screen, will now flip back to the importresults and add the selected series in the background, IMP:(#952) Main page is now sorted in ascending order by Continuing/Ended status (and subbed by whether is Active/Paused).Custom sorting is still available, FIX: Dupecheck will now automatically assume existing 0-byte files are to be overwritten when performing post-processing, FIX: If publication date for series contained a '?' (usually with brand new series) will force to 'Present' to allow for pull-list comparisons to take place, FIX: Mylar will now disallow search results which have 'covers only' or 'variant' in the filename, IMP: Better nzbname generation/retrieval (will check inside nzb for possible names) to be used when post-processing, IMP: DB Update will now perform update to all active comics in descending order by Latest Date (instead of random order), FIX: Enforce the 5hr limit rule when running DB update (will only update series that haven't been updated in >5 hours), FIX: Annuals will now have/retain the proper status upon doing DB Update, FIX: Have totals will now be updated when doing a recheck files (sometimes wouldn't get updated depending on various states of status'), FIX:(#966) Added urllib2.URLError exeception trap when attempting to check Git for updates, IMP: Removed the individual sqlite calls for weeklypull, and brought them into line with using the db module (which will minimize concurrent access, which seemed to be causing db locks), IMP: Cleaned up some code and shuffled some functions so they are in more appropriate locations
2015-03-27 17:27:59 +00:00
if annualdupe == " yes " :
logger . fdebug ( module + ' I should break out here because of a dupe. ' )
break
if haveissue == " yes " or annualdupe == " yes " : break
2012-10-25 06:44:51 +00:00
n + = 1
2013-07-01 05:19:15 +00:00
IMP: Cleaned up interface for StoryArcs / Story Arc Details, IMP: Cleaned up interface for Reading list Management, IMP: Added better reading list management - new status (added, downloaded, read), IMP: Added sync option for use with another device for reading list transfer (ie. tablet) Android only, IMP: Autopopulate new weekly pull releases to reading list, IMP: 'Watch' option in weekly pull list now fully functional. Will watch CV for series' that do not have any series data yet as they are new starting series. Will auto-add once available, IMP: Auto-watch check is run after every refresh/recreate of the weeklypull list, IMP: Improved the Add a Series option where it will now look for issues that are 'new' or 'wanted' during add sequence, IMP: Main page interface now has coloured have/total bars to denote series completion, IMP: New scheduler / threading locks in place in an attempt to avoid database locks, FIX: Removed some erroneous locking that was going on when importing a directory was being run, IMP: Stat counter now present when post-processing multiple issues in sequence, FIX: for issue number error when post-processing and issue number was a non-alphanumeric, FIX: for metatagging: when original file was .cbz, would try to convert and fail, FIX: for issues that were negative and were preceeded by a # in the filename (filechecker), FIX: for publisher having non-alphanumeric character in name when attempting to determine publisher, FIX: if annuals enabled, would incorrectly show as being 'already in library' when viewing search results if results constained annuals, FIX:(#944) for incorrect nzbname being used when post-processing was being performed from an nzb client (experimental mainly), IMP: Turned off logging for ComicVine API counter, FIX: Added retry attempts when connecting to ComicVine in order to avoid errors when adding a series, IMP:(#963) Added ability to add snatched to filter when viewing Wanted issues on Wanted tab, FIX: When importing and then selecting a series to import via the select screen, will now flip back to the importresults and add the selected series in the background, IMP:(#952) Main page is now sorted in ascending order by Continuing/Ended status (and subbed by whether is Active/Paused).Custom sorting is still available, FIX: Dupecheck will now automatically assume existing 0-byte files are to be overwritten when performing post-processing, FIX: If publication date for series contained a '?' (usually with brand new series) will force to 'Present' to allow for pull-list comparisons to take place, FIX: Mylar will now disallow search results which have 'covers only' or 'variant' in the filename, IMP: Better nzbname generation/retrieval (will check inside nzb for possible names) to be used when post-processing, IMP: DB Update will now perform update to all active comics in descending order by Latest Date (instead of random order), FIX: Enforce the 5hr limit rule when running DB update (will only update series that haven't been updated in >5 hours), FIX: Annuals will now have/retain the proper status upon doing DB Update, FIX: Have totals will now be updated when doing a recheck files (sometimes wouldn't get updated depending on various states of status'), FIX:(#966) Added urllib2.URLError exeception trap when attempting to check Git for updates, IMP: Removed the individual sqlite calls for weeklypull, and brought them into line with using the db module (which will minimize concurrent access, which seemed to be causing db locks), IMP: Cleaned up some code and shuffled some functions so they are in more appropriate locations
2015-03-27 17:27:59 +00:00
if issuedupe == " yes " or annualdupe == " yes " : pass
2013-07-01 05:19:15 +00:00
else :
#we have the # of comics, now let's update the db.
#even if we couldn't find the physical issue, check the status.
2013-12-09 19:44:26 +00:00
#-- if annuals aren't enabled, this will bugger out.
writeit = True
IMP: (#844)Annuals now deleted from table on a Delete Series command, IMP: (#837)Better volume detection when searching for issues, IMP: (#842)(#808) Added some handling for issue titles when searching (it's not perfect, but it works for most), IMP: (#836)Added .cb7 for filechecking purposes only, IMP: (#830)issue numbers that are -1 can now be renamed, FIX: 'bi-annual' now fixed along with some other improvements for annual handling, IMP: Added version check for autoProcessComics and ComicRN - in order to help warn users of changes and the need to update these files since they are usually outside of the mylar git directory when being used, FIX:(#840) Boxcar2 should be working again, FIX: (#845) If search returned a filename that had no distinction between issue number and issue title and/or extra information, would error out if any of the words contained NOW, IMP: (#823) Added some better detection for NZBGet parameters when using ComicRN.py
2014-10-06 18:10:36 +00:00
try :
2017-10-25 18:33:56 +00:00
if mylar . CONFIG . ANNUALS_ON :
FIX: (#2006)(#1986) Auto-Want series were not being auto-added, and even after manually added would result in count error after sucessful post-processing, FIX: When filechecking, if a 'special' was included as part of the annuals of a given series, it would not be detected, IMP: (#1982) When ComicRN & Completed Download Handling (CDH) are both enabled, will now check to see if ComicRN is active for the given category in the respective client. If it is, CDH will not post-process and issue a warning and let ComicRN handle the post-processing, FIX: Fixed a utf-8 encoding error when 'Run script AFTER snatch' was enabled, FIX: (#2004) Possible fix for 500 error on Wanted tab, when series had an annual/special attached but was located in the incorrect table, FIX: Removed some logging lines when CDH is monitoring downloads
2018-06-30 18:35:52 +00:00
if any ( [ ' annual ' in temploc . lower ( ) , ' special ' in temploc . lower ( ) ] ) :
IMP: (#844)Annuals now deleted from table on a Delete Series command, IMP: (#837)Better volume detection when searching for issues, IMP: (#842)(#808) Added some handling for issue titles when searching (it's not perfect, but it works for most), IMP: (#836)Added .cb7 for filechecking purposes only, IMP: (#830)issue numbers that are -1 can now be renamed, FIX: 'bi-annual' now fixed along with some other improvements for annual handling, IMP: Added version check for autoProcessComics and ComicRN - in order to help warn users of changes and the need to update these files since they are usually outside of the mylar git directory when being used, FIX:(#840) Boxcar2 should be working again, FIX: (#845) If search returned a filename that had no distinction between issue number and issue title and/or extra information, would error out if any of the words contained NOW, IMP: (#823) Added some better detection for NZBGet parameters when using ComicRN.py
2014-10-06 18:10:36 +00:00
if reann is None :
FIX: (#2006)(#1986) Auto-Want series were not being auto-added, and even after manually added would result in count error after sucessful post-processing, FIX: When filechecking, if a 'special' was included as part of the annuals of a given series, it would not be detected, IMP: (#1982) When ComicRN & Completed Download Handling (CDH) are both enabled, will now check to see if ComicRN is active for the given category in the respective client. If it is, CDH will not post-process and issue a warning and let ComicRN handle the post-processing, FIX: Fixed a utf-8 encoding error when 'Run script AFTER snatch' was enabled, FIX: (#2004) Possible fix for 500 error on Wanted tab, when series had an annual/special attached but was located in the incorrect table, FIX: Removed some logging lines when CDH is monitoring downloads
2018-06-30 18:35:52 +00:00
logger . fdebug ( module + ' Annual/Special present in location, but series does not have any annuals attached to it - Ignoring ' )
IMP: (#844)Annuals now deleted from table on a Delete Series command, IMP: (#837)Better volume detection when searching for issues, IMP: (#842)(#808) Added some handling for issue titles when searching (it's not perfect, but it works for most), IMP: (#836)Added .cb7 for filechecking purposes only, IMP: (#830)issue numbers that are -1 can now be renamed, FIX: 'bi-annual' now fixed along with some other improvements for annual handling, IMP: Added version check for autoProcessComics and ComicRN - in order to help warn users of changes and the need to update these files since they are usually outside of the mylar git directory when being used, FIX:(#840) Boxcar2 should be working again, FIX: (#845) If search returned a filename that had no distinction between issue number and issue title and/or extra information, would error out if any of the words contained NOW, IMP: (#823) Added some better detection for NZBGet parameters when using ComicRN.py
2014-10-06 18:10:36 +00:00
writeit = False
else :
iss_id = reann [ ' IssueID ' ]
2014-01-27 04:23:11 +00:00
else :
IMP: (#844)Annuals now deleted from table on a Delete Series command, IMP: (#837)Better volume detection when searching for issues, IMP: (#842)(#808) Added some handling for issue titles when searching (it's not perfect, but it works for most), IMP: (#836)Added .cb7 for filechecking purposes only, IMP: (#830)issue numbers that are -1 can now be renamed, FIX: 'bi-annual' now fixed along with some other improvements for annual handling, IMP: Added version check for autoProcessComics and ComicRN - in order to help warn users of changes and the need to update these files since they are usually outside of the mylar git directory when being used, FIX:(#840) Boxcar2 should be working again, FIX: (#845) If search returned a filename that had no distinction between issue number and issue title and/or extra information, would error out if any of the words contained NOW, IMP: (#823) Added some better detection for NZBGet parameters when using ComicRN.py
2014-10-06 18:10:36 +00:00
iss_id = reiss [ ' IssueID ' ]
2013-08-11 05:31:41 +00:00
else :
FIX: (#2006)(#1986) Auto-Want series were not being auto-added, and even after manually added would result in count error after sucessful post-processing, FIX: When filechecking, if a 'special' was included as part of the annuals of a given series, it would not be detected, IMP: (#1982) When ComicRN & Completed Download Handling (CDH) are both enabled, will now check to see if ComicRN is active for the given category in the respective client. If it is, CDH will not post-process and issue a warning and let ComicRN handle the post-processing, FIX: Fixed a utf-8 encoding error when 'Run script AFTER snatch' was enabled, FIX: (#2004) Possible fix for 500 error on Wanted tab, when series had an annual/special attached but was located in the incorrect table, FIX: Removed some logging lines when CDH is monitoring downloads
2018-06-30 18:35:52 +00:00
if any ( [ ' annual ' in temploc . lower ( ) , ' special ' in temploc . lower ( ) ] ) :
logger . fdebug ( module + ' Annual support not enabled, but annual/special issue present within directory. Ignoring issue. ' )
IMP: (#844)Annuals now deleted from table on a Delete Series command, IMP: (#837)Better volume detection when searching for issues, IMP: (#842)(#808) Added some handling for issue titles when searching (it's not perfect, but it works for most), IMP: (#836)Added .cb7 for filechecking purposes only, IMP: (#830)issue numbers that are -1 can now be renamed, FIX: 'bi-annual' now fixed along with some other improvements for annual handling, IMP: Added version check for autoProcessComics and ComicRN - in order to help warn users of changes and the need to update these files since they are usually outside of the mylar git directory when being used, FIX:(#840) Boxcar2 should be working again, FIX: (#845) If search returned a filename that had no distinction between issue number and issue title and/or extra information, would error out if any of the words contained NOW, IMP: (#823) Added some better detection for NZBGet parameters when using ComicRN.py
2014-10-06 18:10:36 +00:00
writeit = False
else :
iss_id = reiss [ ' IssueID ' ]
except :
logger . warn ( module + ' An error occured trying to get the relevant issue data. This is probably due to the series not having proper issue data. ' )
logger . warn ( module + ' you should either Refresh the series, and/or submit an issue on github in regards to the series and the error. ' )
return
2013-12-09 19:44:26 +00:00
2016-04-13 03:19:37 +00:00
if writeit == True and haveissue == ' yes ' :
#logger.fdebug(module + ' issueID to write to db:' + str(iss_id))
controlValueDict = { " IssueID " : str ( iss_id ) }
2013-12-09 19:44:26 +00:00
#if Archived, increase the 'Have' count.
IMP: (#1122) Added provider OMGWTFNZBS (results untested), FIX:(#1124) Experimental search results had a 1/9 (or simliar) parts as part of the header, would error out trying to create the nzb due to invalid path, FIX: When importing a series, if Move Files wasn't enabled would error out during the import process, FIX: When on Import page, if selecting Save Changes without Scanning Library would leave pop-up notification up forever, FIX: When searching for series (like A-Force), results would return 16000+ which would look like it locked up due to the excessive number of results - will now better handle such titles, and overall search results are limited to the first 1000 hits, FIX: When clicking on 'Add Series' from the pull-list, if a series was a new series with no issues published would return 0 results from CV even though the series would exist, FIX:(#1120) Story Arcs with special characters would cause error during manual post-processing run, FIX:(#1125) If year is fuzzied for a given series when searching, would error out with invalid date comparisons
2015-09-11 15:49:56 +00:00
if archive :
issStatus = " Archived "
else :
2013-12-09 19:44:26 +00:00
issStatus = " Downloaded "
IMP: (#1122) Added provider OMGWTFNZBS (results untested), FIX:(#1124) Experimental search results had a 1/9 (or simliar) parts as part of the header, would error out trying to create the nzb due to invalid path, FIX: When importing a series, if Move Files wasn't enabled would error out during the import process, FIX: When on Import page, if selecting Save Changes without Scanning Library would leave pop-up notification up forever, FIX: When searching for series (like A-Force), results would return 16000+ which would look like it locked up due to the excessive number of results - will now better handle such titles, and overall search results are limited to the first 1000 hits, FIX: When clicking on 'Add Series' from the pull-list, if a series was a new series with no issues published would return 0 results from CV even though the series would exist, FIX:(#1120) Story Arcs with special characters would cause error during manual post-processing run, FIX:(#1125) If year is fuzzied for a given series when searching, would error out with invalid date comparisons
2015-09-11 15:49:56 +00:00
2016-04-13 03:19:37 +00:00
newValueDict = { " Location " : isslocation ,
" ComicSize " : issSize ,
" Status " : issStatus
}
issID_to_ignore . append ( str ( iss_id ) )
2016-10-11 15:08:30 +00:00
2016-04-13 03:19:37 +00:00
if ANNComicID :
myDB . upsert ( " annuals " , newValueDict , controlValueDict )
ANNComicID = None
else :
myDB . upsert ( " issues " , newValueDict , controlValueDict )
else :
ANNComicID = None
2012-10-25 06:44:51 +00:00
fn + = 1
2012-09-13 15:27:34 +00:00
2013-08-06 00:53:10 +00:00
#here we need to change the status of the ones we DIDN'T FIND above since the loop only hits on FOUND issues.
update_iss = [ ]
IMP: Added ALT_PULL=2 method for weeklypull management. Will now update against an external site and pull down already populated ComicID/IssueID's for the series that exist on the pullist. Alternate Search Names aren't needed with this option to match on pullist, as well as CV API Hits are also not used since it is all populated. Also allows for future viewing of pullists (up to 4 weeks in advance), FIX: Alternate search names now will be searched against when doing manual post-processing, FIX: When manually post-processing, if series volume wasn't specified would fail to match against v1 (by default), IMP:(#1309) Https_chain option now allowed within config.ini, IMP: 32P pack support on a per series basis (will search individual torrents first before packs), IMP: When pack is snatched, will mark all issues within pack that are not in a Downloaded status as Snatched within Mylar (annuals currently don't work), IMP: Removed unnecessary config spamming on startup when verbose mode was enabled, IMP: Allow for searching on 32p against series+publisher for some titles that distinguish between series by different publisher, IMP: Better series matching when trying to find series matches on 32P, FIX: When metatagging, if volume label is not provided within Mylar would default to None (now will be v1), IMP:(#1304) Attempt at better file parsing when utf-8 filenames are being parsed, FIX: Proper handling of Infinity issue number when file-checking, FIX: When adding series and annuals were enabled, if annual was a new release, would not be shown on the annual subset table for the given series (and subsequently wouldn't be auto-marked as Wanted), FIX:(#1306) Correct handling of the imported value when doing an import and moving files was selected (would previously error out during moving for some imports), FIX: When cbz files were being imported and were attempted to being auto-imported, would fail due to improper handling of the imported variable, FIX: Manage issues will now default the dropdown to the correct selected option, FIX: Manage Comics - fixed dropdown options for multiple selection of series - delete/pause/resume, IMP: Added 'delete stragglers' option to Story Arcs when deleting an arc to ensure that all traces of the arc are removed from the db, FIX: Manual/group metatagging would not tag properly if the START_YEAR_AS_VOLUME option was enabled, FIX: (#1313) NzbHydra wouldn't set the nzbid properly when using Failed Download handling/Retrying
2016-07-10 22:28:14 +00:00
#break this up in sequnces of 200 so it doesn't break the sql statement.
2016-10-11 15:08:30 +00:00
cnt = 0
for genlist in helpers . chunker ( issID_to_ignore , 200 ) :
tmpsql = " SELECT * FROM issues WHERE ComicID=? AND IssueID not in ( {seq} ) " . format ( seq = ' , ' . join ( [ ' ? ' ] * ( len ( genlist ) - 1 ) ) )
chkthis = myDB . select ( tmpsql , genlist )
if chkthis is None :
pass
else :
for chk in chkthis :
2016-11-09 16:49:06 +00:00
if chk [ ' IssueID ' ] in issID_to_ignore :
2016-10-11 15:08:30 +00:00
continue
IMP: Added Choose specific Download option to manually select from result list on Upcoming/Details/Weekly tabs, IMP: Added Wanted storyarcs to the overall Wanted section so now will search as per global options (storyarc issues can be displayed optionally on Wanted tab), IMP: Added custom url option for image banners/posters for storyarcs, IMP: updated Cherrypy, FIX: Fixed ComicRN not working when forms authentication used - in conjunction updated autoProcessComics to 2.0 which now uses apikey instead of user/pass, IMP: Alternate Series Covers option for alternating existing series image to the most current issue image, IMP: Added overall series total to series page for reference, IMP: Search workflow completely changed to accomodate more than one indexer - now will simultaneously sumbit initial request to each provider, wait 30s submit additional as required at 30s intervals, FIX: Removed TPSE as an option and relabelled to just Public Torrents, IMP: Added direct backlog search to WWT option (pack support will follow), FIX: Removed line about configparser being required for ComicTagger usage, IMP: Test code in place for newzab testing, FIX: Fixed layout problem with torrents that are in auto-snatch status on weekly tab, IMP: backend code improvements to allow for better alias usage and annual linking directly from WS, IMP: Updated systemd init-scripts with read.me, IMP: When post-processing, will now check for available destination free space before actually moving files, IMP: Will copy during metatagging to cache folder instead of move being an option so cleanup is cleaner if something fails, FIX: Changed readinglist table to storyarcs for clarity, IMP: When post-processing issues, will now only update the one issue status and adjust totals accordingly (instead of doing a complete rescan of the series), FIX: Clear out empty ID's from the Failed DB on startup, IMP: Initial code-run at REST API interface, FIX: Fixed some matching problems with 32p due to case, IMP: removed apikeys from log entries that were accidentally logging, IMP: When searching 32p, if items get packed up - will now delete the cached reference so new items of the same can be located, IMP: ForceSearch option switched to scheduler so simultaneous runs should not occur, FIX: Fixed manual metatagging error that would occur if multiple destination directories existed
2018-02-16 19:57:01 +00:00
2016-10-11 15:08:30 +00:00
old_status = chk [ ' Status ' ]
2016-11-09 16:49:06 +00:00
2016-10-11 15:08:30 +00:00
if old_status == " Skipped " :
2017-10-25 18:33:56 +00:00
if mylar . CONFIG . AUTOWANT_ALL :
2016-10-11 15:08:30 +00:00
issStatus = " Wanted "
else :
issStatus = " Skipped "
IMP: Added Choose specific Download option to manually select from result list on Upcoming/Details/Weekly tabs, IMP: Added Wanted storyarcs to the overall Wanted section so now will search as per global options (storyarc issues can be displayed optionally on Wanted tab), IMP: Added custom url option for image banners/posters for storyarcs, IMP: updated Cherrypy, FIX: Fixed ComicRN not working when forms authentication used - in conjunction updated autoProcessComics to 2.0 which now uses apikey instead of user/pass, IMP: Alternate Series Covers option for alternating existing series image to the most current issue image, IMP: Added overall series total to series page for reference, IMP: Search workflow completely changed to accomodate more than one indexer - now will simultaneously sumbit initial request to each provider, wait 30s submit additional as required at 30s intervals, FIX: Removed TPSE as an option and relabelled to just Public Torrents, IMP: Added direct backlog search to WWT option (pack support will follow), FIX: Removed line about configparser being required for ComicTagger usage, IMP: Test code in place for newzab testing, FIX: Fixed layout problem with torrents that are in auto-snatch status on weekly tab, IMP: backend code improvements to allow for better alias usage and annual linking directly from WS, IMP: Updated systemd init-scripts with read.me, IMP: When post-processing, will now check for available destination free space before actually moving files, IMP: Will copy during metatagging to cache folder instead of move being an option so cleanup is cleaner if something fails, FIX: Changed readinglist table to storyarcs for clarity, IMP: When post-processing issues, will now only update the one issue status and adjust totals accordingly (instead of doing a complete rescan of the series), FIX: Clear out empty ID's from the Failed DB on startup, IMP: Initial code-run at REST API interface, FIX: Fixed some matching problems with 32p due to case, IMP: removed apikeys from log entries that were accidentally logging, IMP: When searching 32p, if items get packed up - will now delete the cached reference so new items of the same can be located, IMP: ForceSearch option switched to scheduler so simultaneous runs should not occur, FIX: Fixed manual metatagging error that would occur if multiple destination directories existed
2018-02-16 19:57:01 +00:00
#elif old_status == "Archived":
# issStatus = "Archived"
2016-10-11 15:08:30 +00:00
elif old_status == " Downloaded " :
issStatus = " Archived "
2013-08-06 00:53:10 +00:00
else :
IMP: Added Choose specific Download option to manually select from result list on Upcoming/Details/Weekly tabs, IMP: Added Wanted storyarcs to the overall Wanted section so now will search as per global options (storyarc issues can be displayed optionally on Wanted tab), IMP: Added custom url option for image banners/posters for storyarcs, IMP: updated Cherrypy, FIX: Fixed ComicRN not working when forms authentication used - in conjunction updated autoProcessComics to 2.0 which now uses apikey instead of user/pass, IMP: Alternate Series Covers option for alternating existing series image to the most current issue image, IMP: Added overall series total to series page for reference, IMP: Search workflow completely changed to accomodate more than one indexer - now will simultaneously sumbit initial request to each provider, wait 30s submit additional as required at 30s intervals, FIX: Removed TPSE as an option and relabelled to just Public Torrents, IMP: Added direct backlog search to WWT option (pack support will follow), FIX: Removed line about configparser being required for ComicTagger usage, IMP: Test code in place for newzab testing, FIX: Fixed layout problem with torrents that are in auto-snatch status on weekly tab, IMP: backend code improvements to allow for better alias usage and annual linking directly from WS, IMP: Updated systemd init-scripts with read.me, IMP: When post-processing, will now check for available destination free space before actually moving files, IMP: Will copy during metatagging to cache folder instead of move being an option so cleanup is cleaner if something fails, FIX: Changed readinglist table to storyarcs for clarity, IMP: When post-processing issues, will now only update the one issue status and adjust totals accordingly (instead of doing a complete rescan of the series), FIX: Clear out empty ID's from the Failed DB on startup, IMP: Initial code-run at REST API interface, FIX: Fixed some matching problems with 32p due to case, IMP: removed apikeys from log entries that were accidentally logging, IMP: When searching 32p, if items get packed up - will now delete the cached reference so new items of the same can be located, IMP: ForceSearch option switched to scheduler so simultaneous runs should not occur, FIX: Fixed manual metatagging error that would occur if multiple destination directories existed
2018-02-16 19:57:01 +00:00
continue
#elif old_status == "Wanted":
# issStatus = "Wanted"
#elif old_status == "Ignored":
# issStatus = "Ignored"
#elif old_status == "Snatched": #this is needed for torrents, or else it'll keep on queuing..
# issStatus = "Snatched"
#else:
# issStatus = "Skipped"
2016-10-11 15:08:30 +00:00
update_iss . append ( { " IssueID " : chk [ ' IssueID ' ] ,
" Status " : issStatus } )
2015-05-22 08:32:51 +00:00
2013-08-06 00:53:10 +00:00
if len ( update_iss ) > 0 :
i = 0
#do it like this to avoid DB locks...
for ui in update_iss :
controlValueDict = { " IssueID " : ui [ ' IssueID ' ] }
newStatusValue = { " Status " : ui [ ' Status ' ] }
myDB . upsert ( " issues " , newStatusValue , controlValueDict )
i + = 1
IMP:(#753) Better handling of cbz files when using metatagging option, IMP: Added option for MetaTagging to enable/disable overwriting of existing metadata on cbz files, IMP: Post-Processing will now delete folders that are identical to filenames after successful post-processing, FIX:(#756) Fix for Manual Search for an Issue which resulted in traceback errors upon not finding any matches (Windows), IMP: Added module-name to logging for some modules to help troubleshooting (eventually all will be done), IMP: If a filename contained repeated characters for spaces (ie. ___ or ...), it would fail on post-processing due to the nzbname being off, IMP: Monitoring of CV API will now be persitent on restarts/shutdowns/etc, IMP: Added a maximum CV API global so that it can be adjusted for monitoring more easily should API requirements change, IMP: When Adding new series, will now only mark issues that are actually upcoming as Wanted (based on Store Date)
2014-06-22 08:32:45 +00:00
logger . info ( module + ' Updated the status of ' + str ( i ) + ' issues for ' + rescan [ ' ComicName ' ] + ' ( ' + str ( rescan [ ' ComicYear ' ] ) + ' ) that were not found. ' )
2012-09-13 15:27:34 +00:00
IMP:(#753) Better handling of cbz files when using metatagging option, IMP: Added option for MetaTagging to enable/disable overwriting of existing metadata on cbz files, IMP: Post-Processing will now delete folders that are identical to filenames after successful post-processing, FIX:(#756) Fix for Manual Search for an Issue which resulted in traceback errors upon not finding any matches (Windows), IMP: Added module-name to logging for some modules to help troubleshooting (eventually all will be done), IMP: If a filename contained repeated characters for spaces (ie. ___ or ...), it would fail on post-processing due to the nzbname being off, IMP: Monitoring of CV API will now be persitent on restarts/shutdowns/etc, IMP: Added a maximum CV API global so that it can be adjusted for monitoring more easily should API requirements change, IMP: When Adding new series, will now only mark issues that are actually upcoming as Wanted (based on Store Date)
2014-06-22 08:32:45 +00:00
logger . info ( module + ' Total files located: ' + str ( havefiles ) )
IMP: Added Choose specific Download option to manually select from result list on Upcoming/Details/Weekly tabs, IMP: Added Wanted storyarcs to the overall Wanted section so now will search as per global options (storyarc issues can be displayed optionally on Wanted tab), IMP: Added custom url option for image banners/posters for storyarcs, IMP: updated Cherrypy, FIX: Fixed ComicRN not working when forms authentication used - in conjunction updated autoProcessComics to 2.0 which now uses apikey instead of user/pass, IMP: Alternate Series Covers option for alternating existing series image to the most current issue image, IMP: Added overall series total to series page for reference, IMP: Search workflow completely changed to accomodate more than one indexer - now will simultaneously sumbit initial request to each provider, wait 30s submit additional as required at 30s intervals, FIX: Removed TPSE as an option and relabelled to just Public Torrents, IMP: Added direct backlog search to WWT option (pack support will follow), FIX: Removed line about configparser being required for ComicTagger usage, IMP: Test code in place for newzab testing, FIX: Fixed layout problem with torrents that are in auto-snatch status on weekly tab, IMP: backend code improvements to allow for better alias usage and annual linking directly from WS, IMP: Updated systemd init-scripts with read.me, IMP: When post-processing, will now check for available destination free space before actually moving files, IMP: Will copy during metatagging to cache folder instead of move being an option so cleanup is cleaner if something fails, FIX: Changed readinglist table to storyarcs for clarity, IMP: When post-processing issues, will now only update the one issue status and adjust totals accordingly (instead of doing a complete rescan of the series), FIX: Clear out empty ID's from the Failed DB on startup, IMP: Initial code-run at REST API interface, FIX: Fixed some matching problems with 32p due to case, IMP: removed apikeys from log entries that were accidentally logging, IMP: When searching 32p, if items get packed up - will now delete the cached reference so new items of the same can be located, IMP: ForceSearch option switched to scheduler so simultaneous runs should not occur, FIX: Fixed manual metatagging error that would occur if multiple destination directories existed
2018-02-16 19:57:01 +00:00
2013-04-10 02:28:27 +00:00
foundcount = havefiles
arcfiles = 0
2014-04-14 17:42:42 +00:00
arcanns = 0
2013-04-10 02:28:27 +00:00
# if filechecker returns 0 files (it doesn't find any), but some issues have a status of 'Archived'
# the loop below won't work...let's adjust :)
FIX:(#1431) Fixed post-processing cleanup when using SAB + files resided in a sub-folder of the path sent from SAB, IMP: Added hardlink/softlink option for non-windows users (note that metatagging is forcibly disabled when these options are enabled), IMP: Added Enforce Permissions option to force Mylar to either force / ensure permissions are as set within Mylar, or to ignore permissions and just try to perform the given action (this is for adding series, post-processing, importing, arcs, etc), FIX: Fixed a problem where if local watchdir was set and a torrent downloader was selected, both would attempt to be used, but only the watchdir was actually utilized (now it's one or the other, never both), FIX: Mylar will now remove issues that are in Failed status that have incorrect Series Data attached, FIX:(#1438) When file-checking/post-processing if issue date in Mylar was current year and date in filename was the following year (for end-of-year issues), would cause traceback error, FIX: If series year polled from CV is incorrect or not provided, option is available in Edit Settings of series detail page that allows user to specify the correct series year, FIX:(#1427) Fixed a typo when the Scan Library option is used and ComicID's were able to be parsed from the files, FIX: When issues were archived, depending on if series had issues in a Downloaded status or not, would give different results to the Have totals, IMP: Manage tab / Manage Comics now has metatag option available to allow metatagging multiple series in a batch mode, IMP:(#1419) When 'write cvinfo' option is enabled, will now also write cvinfo into story arc directories when added / refreshed, FIX: Fixed a problem when group metatag was being used and causing errors, IMP: Manage Failed page now will link back to the series for easier reference, and the 'link' column was removed due to invalid references, FIX: Removed the 'issue not updated' from the series detail page as was not accurate, IMP: Added a Select Filtered Issues button to the series detail page to allow for selecting of all filtered issues (Show ALL MUST be enabled for it to work properly)
2016-11-02 15:00:56 +00:00
arcissues = myDB . select ( " SELECT count(*) FROM issues WHERE ComicID=? and Status= ' Archived ' " , [ ComicID ] )
if int ( arcissues [ 0 ] [ 0 ] ) > 0 :
arcfiles = arcissues [ 0 ] [ 0 ]
arcannuals = myDB . select ( " SELECT count(*) FROM annuals WHERE ComicID=? and Status= ' Archived ' " , [ ComicID ] )
if int ( arcannuals [ 0 ] [ 0 ] ) > 0 :
arcanns = arcannuals [ 0 ] [ 0 ]
2016-10-11 15:08:30 +00:00
FIX:(#1431) Fixed post-processing cleanup when using SAB + files resided in a sub-folder of the path sent from SAB, IMP: Added hardlink/softlink option for non-windows users (note that metatagging is forcibly disabled when these options are enabled), IMP: Added Enforce Permissions option to force Mylar to either force / ensure permissions are as set within Mylar, or to ignore permissions and just try to perform the given action (this is for adding series, post-processing, importing, arcs, etc), FIX: Fixed a problem where if local watchdir was set and a torrent downloader was selected, both would attempt to be used, but only the watchdir was actually utilized (now it's one or the other, never both), FIX: Mylar will now remove issues that are in Failed status that have incorrect Series Data attached, FIX:(#1438) When file-checking/post-processing if issue date in Mylar was current year and date in filename was the following year (for end-of-year issues), would cause traceback error, FIX: If series year polled from CV is incorrect or not provided, option is available in Edit Settings of series detail page that allows user to specify the correct series year, FIX:(#1427) Fixed a typo when the Scan Library option is used and ComicID's were able to be parsed from the files, FIX: When issues were archived, depending on if series had issues in a Downloaded status or not, would give different results to the Have totals, IMP: Manage tab / Manage Comics now has metatag option available to allow metatagging multiple series in a batch mode, IMP:(#1419) When 'write cvinfo' option is enabled, will now also write cvinfo into story arc directories when added / refreshed, FIX: Fixed a problem when group metatag was being used and causing errors, IMP: Manage Failed page now will link back to the series for easier reference, and the 'link' column was removed due to invalid references, FIX: Removed the 'issue not updated' from the series detail page as was not accurate, IMP: Added a Select Filtered Issues button to the series detail page to allow for selecting of all filtered issues (Show ALL MUST be enabled for it to work properly)
2016-11-02 15:00:56 +00:00
if havefiles == 0 :
2016-10-11 15:08:30 +00:00
if arcfiles > 0 or arcanns > 0 :
arcfiles = arcfiles + arcanns
havefiles = havefiles + arcfiles
FIX:(#1431) Fixed post-processing cleanup when using SAB + files resided in a sub-folder of the path sent from SAB, IMP: Added hardlink/softlink option for non-windows users (note that metatagging is forcibly disabled when these options are enabled), IMP: Added Enforce Permissions option to force Mylar to either force / ensure permissions are as set within Mylar, or to ignore permissions and just try to perform the given action (this is for adding series, post-processing, importing, arcs, etc), FIX: Fixed a problem where if local watchdir was set and a torrent downloader was selected, both would attempt to be used, but only the watchdir was actually utilized (now it's one or the other, never both), FIX: Mylar will now remove issues that are in Failed status that have incorrect Series Data attached, FIX:(#1438) When file-checking/post-processing if issue date in Mylar was current year and date in filename was the following year (for end-of-year issues), would cause traceback error, FIX: If series year polled from CV is incorrect or not provided, option is available in Edit Settings of series detail page that allows user to specify the correct series year, FIX:(#1427) Fixed a typo when the Scan Library option is used and ComicID's were able to be parsed from the files, FIX: When issues were archived, depending on if series had issues in a Downloaded status or not, would give different results to the Have totals, IMP: Manage tab / Manage Comics now has metatag option available to allow metatagging multiple series in a batch mode, IMP:(#1419) When 'write cvinfo' option is enabled, will now also write cvinfo into story arc directories when added / refreshed, FIX: Fixed a problem when group metatag was being used and causing errors, IMP: Manage Failed page now will link back to the series for easier reference, and the 'link' column was removed due to invalid references, FIX: Removed the 'issue not updated' from the series detail page as was not accurate, IMP: Added a Select Filtered Issues button to the series detail page to allow for selecting of all filtered issues (Show ALL MUST be enabled for it to work properly)
2016-11-02 15:00:56 +00:00
logger . fdebug ( module + ' Adjusting have total to ' + str ( havefiles ) + ' because of this many archive files already in Archive status : ' + str ( arcfiles ) )
else :
#if files exist in the given directory, but are in an archived state - the numbers will get botched up here.
2017-09-19 15:41:48 +00:00
if ( arcfiles + arcanns ) > 0 :
logger . fdebug ( module + ' ' + str ( int ( arcfiles + arcanns ) ) + ' issue(s) are in an Archive status already. Increasing Have total from ' + str ( havefiles ) + ' to include these archives. ' )
havefiles = havefiles + ( arcfiles + arcanns )
2013-04-10 02:28:27 +00:00
2013-07-30 04:57:37 +00:00
ignorecount = 0
2017-10-25 18:33:56 +00:00
if mylar . CONFIG . IGNORE_HAVETOTAL : # if this is enabled, will increase Have total as if in Archived Status
2018-11-19 16:49:05 +00:00
ignoresi = myDB . select ( " SELECT count(*) FROM issues WHERE ComicID=? AND Status= ' Ignored ' " , [ ComicID ] )
ignoresa = myDB . select ( " SELECT count(*) FROM annuals WHERE ComicID=? AND Status= ' Ignored ' " , [ ComicID ] )
ignorecount = int ( ignoresi [ 0 ] [ 0 ] ) + int ( ignoresa [ 0 ] [ 0 ] )
if ignorecount > 0 :
2013-07-30 04:57:37 +00:00
havefiles = havefiles + ignorecount
IMP:(#753) Better handling of cbz files when using metatagging option, IMP: Added option for MetaTagging to enable/disable overwriting of existing metadata on cbz files, IMP: Post-Processing will now delete folders that are identical to filenames after successful post-processing, FIX:(#756) Fix for Manual Search for an Issue which resulted in traceback errors upon not finding any matches (Windows), IMP: Added module-name to logging for some modules to help troubleshooting (eventually all will be done), IMP: If a filename contained repeated characters for spaces (ie. ___ or ...), it would fail on post-processing due to the nzbname being off, IMP: Monitoring of CV API will now be persitent on restarts/shutdowns/etc, IMP: Added a maximum CV API global so that it can be adjusted for monitoring more easily should API requirements change, IMP: When Adding new series, will now only mark issues that are actually upcoming as Wanted (based on Store Date)
2014-06-22 08:32:45 +00:00
logger . fdebug ( module + ' Adjusting have total to ' + str ( havefiles ) + ' because of this many Ignored files: ' + str ( ignorecount ) )
2012-09-13 15:27:34 +00:00
IMP: Added Choose specific Download option to manually select from result list on Upcoming/Details/Weekly tabs, IMP: Added Wanted storyarcs to the overall Wanted section so now will search as per global options (storyarc issues can be displayed optionally on Wanted tab), IMP: Added custom url option for image banners/posters for storyarcs, IMP: updated Cherrypy, FIX: Fixed ComicRN not working when forms authentication used - in conjunction updated autoProcessComics to 2.0 which now uses apikey instead of user/pass, IMP: Alternate Series Covers option for alternating existing series image to the most current issue image, IMP: Added overall series total to series page for reference, IMP: Search workflow completely changed to accomodate more than one indexer - now will simultaneously sumbit initial request to each provider, wait 30s submit additional as required at 30s intervals, FIX: Removed TPSE as an option and relabelled to just Public Torrents, IMP: Added direct backlog search to WWT option (pack support will follow), FIX: Removed line about configparser being required for ComicTagger usage, IMP: Test code in place for newzab testing, FIX: Fixed layout problem with torrents that are in auto-snatch status on weekly tab, IMP: backend code improvements to allow for better alias usage and annual linking directly from WS, IMP: Updated systemd init-scripts with read.me, IMP: When post-processing, will now check for available destination free space before actually moving files, IMP: Will copy during metatagging to cache folder instead of move being an option so cleanup is cleaner if something fails, FIX: Changed readinglist table to storyarcs for clarity, IMP: When post-processing issues, will now only update the one issue status and adjust totals accordingly (instead of doing a complete rescan of the series), FIX: Clear out empty ID's from the Failed DB on startup, IMP: Initial code-run at REST API interface, FIX: Fixed some matching problems with 32p due to case, IMP: removed apikeys from log entries that were accidentally logging, IMP: When searching 32p, if items get packed up - will now delete the cached reference so new items of the same can be located, IMP: ForceSearch option switched to scheduler so simultaneous runs should not occur, FIX: Fixed manual metatagging error that would occur if multiple destination directories existed
2018-02-16 19:57:01 +00:00
2014-12-08 17:48:13 +00:00
snatchedcount = 0
2017-10-25 18:33:56 +00:00
if mylar . CONFIG . SNATCHED_HAVETOTAL : # if this is enabled, will increase Have total as if in Archived Status
2014-12-08 17:48:13 +00:00
snatches = myDB . select ( " SELECT count(*) FROM issues WHERE ComicID=? AND Status= ' Snatched ' " , [ ComicID ] )
if int ( snatches [ 0 ] [ 0 ] ) > 0 :
snatchedcount = snatches [ 0 ] [ 0 ]
havefiles = havefiles + snatchedcount
logger . fdebug ( module + ' Adjusting have total to ' + str ( havefiles ) + ' because of this many Snatched files: ' + str ( snatchedcount ) )
2012-10-30 10:43:01 +00:00
#now that we are finished...
#adjust for issues that have been marked as Downloaded, but aren't found/don't exist.
#do it here, because above loop only cycles though found comics using filechecker.
2013-04-10 02:28:27 +00:00
downissues = myDB . select ( " SELECT * FROM issues WHERE ComicID=? and Status= ' Downloaded ' " , [ ComicID ] )
2014-04-14 17:42:42 +00:00
downissues + = myDB . select ( " SELECT * FROM annuals WHERE ComicID=? and Status= ' Downloaded ' " , [ ComicID ] )
2012-10-30 10:43:01 +00:00
if downissues is None :
pass
else :
2013-04-10 02:28:27 +00:00
archivedissues = 0 #set this to 0 so it tallies correctly.
2012-10-30 10:43:01 +00:00
for down in downissues :
#print "downlocation:" + str(down['Location'])
2015-05-22 08:32:51 +00:00
#remove special characters from
2012-12-31 16:52:16 +00:00
#temploc = rescan['ComicLocation'].replace('_', ' ')
#temploc = re.sub('[\#\'\/\.]', '', temploc)
#print ("comiclocation: " + str(rescan['ComicLocation']))
#print ("downlocation: " + str(down['Location']))
2013-03-29 04:02:35 +00:00
if down [ ' Location ' ] is None :
IMP:(#753) Better handling of cbz files when using metatagging option, IMP: Added option for MetaTagging to enable/disable overwriting of existing metadata on cbz files, IMP: Post-Processing will now delete folders that are identical to filenames after successful post-processing, FIX:(#756) Fix for Manual Search for an Issue which resulted in traceback errors upon not finding any matches (Windows), IMP: Added module-name to logging for some modules to help troubleshooting (eventually all will be done), IMP: If a filename contained repeated characters for spaces (ie. ___ or ...), it would fail on post-processing due to the nzbname being off, IMP: Monitoring of CV API will now be persitent on restarts/shutdowns/etc, IMP: Added a maximum CV API global so that it can be adjusted for monitoring more easily should API requirements change, IMP: When Adding new series, will now only mark issues that are actually upcoming as Wanted (based on Store Date)
2014-06-22 08:32:45 +00:00
logger . fdebug ( module + ' Location does not exist which means file was not downloaded successfully, or was moved. ' )
2013-03-29 04:02:35 +00:00
controlValue = { " IssueID " : down [ ' IssueID ' ] }
newValue = { " Status " : " Archived " }
myDB . upsert ( " issues " , newValue , controlValue )
2013-04-10 02:28:27 +00:00
archivedissues + = 1
2012-10-30 15:40:41 +00:00
pass
2012-10-30 10:43:01 +00:00
else :
2013-03-29 04:02:35 +00:00
comicpath = os . path . join ( rescan [ ' ComicLocation ' ] , down [ ' Location ' ] )
if os . path . exists ( comicpath ) :
2014-11-21 17:16:28 +00:00
continue
FIX:(#1431) Fixed post-processing cleanup when using SAB + files resided in a sub-folder of the path sent from SAB, IMP: Added hardlink/softlink option for non-windows users (note that metatagging is forcibly disabled when these options are enabled), IMP: Added Enforce Permissions option to force Mylar to either force / ensure permissions are as set within Mylar, or to ignore permissions and just try to perform the given action (this is for adding series, post-processing, importing, arcs, etc), FIX: Fixed a problem where if local watchdir was set and a torrent downloader was selected, both would attempt to be used, but only the watchdir was actually utilized (now it's one or the other, never both), FIX: Mylar will now remove issues that are in Failed status that have incorrect Series Data attached, FIX:(#1438) When file-checking/post-processing if issue date in Mylar was current year and date in filename was the following year (for end-of-year issues), would cause traceback error, FIX: If series year polled from CV is incorrect or not provided, option is available in Edit Settings of series detail page that allows user to specify the correct series year, FIX:(#1427) Fixed a typo when the Scan Library option is used and ComicID's were able to be parsed from the files, FIX: When issues were archived, depending on if series had issues in a Downloaded status or not, would give different results to the Have totals, IMP: Manage tab / Manage Comics now has metatag option available to allow metatagging multiple series in a batch mode, IMP:(#1419) When 'write cvinfo' option is enabled, will now also write cvinfo into story arc directories when added / refreshed, FIX: Fixed a problem when group metatag was being used and causing errors, IMP: Manage Failed page now will link back to the series for easier reference, and the 'link' column was removed due to invalid references, FIX: Removed the 'issue not updated' from the series detail page as was not accurate, IMP: Added a Select Filtered Issues button to the series detail page to allow for selecting of all filtered issues (Show ALL MUST be enabled for it to work properly)
2016-11-02 15:00:56 +00:00
logger . fdebug ( ' Issue exists - no need to change status. ' )
2013-03-29 04:02:35 +00:00
else :
2017-10-25 18:33:56 +00:00
if mylar . CONFIG . MULTIPLE_DEST_DIRS is not None and mylar . CONFIG . MULTIPLE_DEST_DIRS != ' None ' :
if os . path . exists ( os . path . join ( mylar . CONFIG . MULTIPLE_DEST_DIRS , os . path . basename ( rescan [ ' ComicLocation ' ] ) ) ) :
2015-02-09 18:44:10 +00:00
#logger.fdebug('Issue(s) currently exist and found within multiple destination directory location')
2014-11-21 17:16:28 +00:00
continue
2013-03-29 04:02:35 +00:00
#print "Changing status from Downloaded to Archived - cannot locate file"
controlValue = { " IssueID " : down [ ' IssueID ' ] }
newValue = { " Status " : " Archived " }
2013-04-10 02:28:27 +00:00
myDB . upsert ( " issues " , newValue , controlValue )
2015-05-22 08:32:51 +00:00
archivedissues + = 1
2017-09-19 15:41:48 +00:00
if archivedissues > 0 :
logger . fdebug ( module + ' I have changed the status of ' + str ( archivedissues ) + ' issues to a status of Archived, as I now cannot locate them in the series directory. ' )
FIX:(#1431) Fixed post-processing cleanup when using SAB + files resided in a sub-folder of the path sent from SAB, IMP: Added hardlink/softlink option for non-windows users (note that metatagging is forcibly disabled when these options are enabled), IMP: Added Enforce Permissions option to force Mylar to either force / ensure permissions are as set within Mylar, or to ignore permissions and just try to perform the given action (this is for adding series, post-processing, importing, arcs, etc), FIX: Fixed a problem where if local watchdir was set and a torrent downloader was selected, both would attempt to be used, but only the watchdir was actually utilized (now it's one or the other, never both), FIX: Mylar will now remove issues that are in Failed status that have incorrect Series Data attached, FIX:(#1438) When file-checking/post-processing if issue date in Mylar was current year and date in filename was the following year (for end-of-year issues), would cause traceback error, FIX: If series year polled from CV is incorrect or not provided, option is available in Edit Settings of series detail page that allows user to specify the correct series year, FIX:(#1427) Fixed a typo when the Scan Library option is used and ComicID's were able to be parsed from the files, FIX: When issues were archived, depending on if series had issues in a Downloaded status or not, would give different results to the Have totals, IMP: Manage tab / Manage Comics now has metatag option available to allow metatagging multiple series in a batch mode, IMP:(#1419) When 'write cvinfo' option is enabled, will now also write cvinfo into story arc directories when added / refreshed, FIX: Fixed a problem when group metatag was being used and causing errors, IMP: Manage Failed page now will link back to the series for easier reference, and the 'link' column was removed due to invalid references, FIX: Removed the 'issue not updated' from the series detail page as was not accurate, IMP: Added a Select Filtered Issues button to the series detail page to allow for selecting of all filtered issues (Show ALL MUST be enabled for it to work properly)
2016-11-02 15:00:56 +00:00
2013-04-10 15:59:12 +00:00
havefiles = havefiles + archivedissues #arcfiles already tallied in havefiles in above segment
2013-04-10 02:28:27 +00:00
2015-05-22 08:32:51 +00:00
#combined total for dispay total purposes only.
2018-11-19 17:20:18 +00:00
combined_total = iscnt + anncnt
if mylar . CONFIG . IGNORE_TOTAL : # if this is enabled, will increase Have total as if in Archived Status
ignoresa = myDB . select ( " SELECT count(*) FROM issues WHERE ComicID=? AND Status= ' Ignored ' " , [ ComicID ] )
ignoresb = myDB . select ( " SELECT count(*) FROM annuals WHERE ComicID=? AND Status= ' Ignored ' " , [ ComicID ] )
ignorecnt = ignoresa [ 0 ] [ 0 ] + ignoresb [ 0 ] [ 0 ]
if ignorecnt > 0 :
combined_total - = ignorecnt
logger . fdebug ( ' %s Reducing total comics in series from %s to %s because of %s ignored files. ' % ( module , ( iscnt + anncnt ) , combined_total , ignorecnt ) )
IMP: Cleaned up interface for StoryArcs / Story Arc Details, IMP: Cleaned up interface for Reading list Management, IMP: Added better reading list management - new status (added, downloaded, read), IMP: Added sync option for use with another device for reading list transfer (ie. tablet) Android only, IMP: Autopopulate new weekly pull releases to reading list, IMP: 'Watch' option in weekly pull list now fully functional. Will watch CV for series' that do not have any series data yet as they are new starting series. Will auto-add once available, IMP: Auto-watch check is run after every refresh/recreate of the weeklypull list, IMP: Improved the Add a Series option where it will now look for issues that are 'new' or 'wanted' during add sequence, IMP: Main page interface now has coloured have/total bars to denote series completion, IMP: New scheduler / threading locks in place in an attempt to avoid database locks, FIX: Removed some erroneous locking that was going on when importing a directory was being run, IMP: Stat counter now present when post-processing multiple issues in sequence, FIX: for issue number error when post-processing and issue number was a non-alphanumeric, FIX: for metatagging: when original file was .cbz, would try to convert and fail, FIX: for issues that were negative and were preceeded by a # in the filename (filechecker), FIX: for publisher having non-alphanumeric character in name when attempting to determine publisher, FIX: if annuals enabled, would incorrectly show as being 'already in library' when viewing search results if results constained annuals, FIX:(#944) for incorrect nzbname being used when post-processing was being performed from an nzb client (experimental mainly), IMP: Turned off logging for ComicVine API counter, FIX: Added retry attempts when connecting to ComicVine in order to avoid errors when adding a series, IMP:(#963) Added ability to add snatched to filter when viewing Wanted issues on Wanted tab, FIX: When importing and then selecting a series to import via the select screen, will now flip back to the importresults and add the selected series in the background, IMP:(#952) Main page is now sorted in ascending order by Continuing/Ended status (and subbed by whether is Active/Paused).Custom sorting is still available, FIX: Dupecheck will now automatically assume existing 0-byte files are to be overwritten when performing post-processing, FIX: If publication date for series contained a '?' (usually with brand new series) will force to 'Present' to allow for pull-list comparisons to take place, FIX: Mylar will now disallow search results which have 'covers only' or 'variant' in the filename, IMP: Better nzbname generation/retrieval (will check inside nzb for possible names) to be used when post-processing, IMP: DB Update will now perform update to all active comics in descending order by Latest Date (instead of random order), FIX: Enforce the 5hr limit rule when running DB update (will only update series that haven't been updated in >5 hours), FIX: Annuals will now have/retain the proper status upon doing DB Update, FIX: Have totals will now be updated when doing a recheck files (sometimes wouldn't get updated depending on various states of status'), FIX:(#966) Added urllib2.URLError exeception trap when attempting to check Git for updates, IMP: Removed the individual sqlite calls for weeklypull, and brought them into line with using the db module (which will minimize concurrent access, which seemed to be causing db locks), IMP: Cleaned up some code and shuffled some functions so they are in more appropriate locations
2015-03-27 17:27:59 +00:00
FIX:(#1431) Fixed post-processing cleanup when using SAB + files resided in a sub-folder of the path sent from SAB, IMP: Added hardlink/softlink option for non-windows users (note that metatagging is forcibly disabled when these options are enabled), IMP: Added Enforce Permissions option to force Mylar to either force / ensure permissions are as set within Mylar, or to ignore permissions and just try to perform the given action (this is for adding series, post-processing, importing, arcs, etc), FIX: Fixed a problem where if local watchdir was set and a torrent downloader was selected, both would attempt to be used, but only the watchdir was actually utilized (now it's one or the other, never both), FIX: Mylar will now remove issues that are in Failed status that have incorrect Series Data attached, FIX:(#1438) When file-checking/post-processing if issue date in Mylar was current year and date in filename was the following year (for end-of-year issues), would cause traceback error, FIX: If series year polled from CV is incorrect or not provided, option is available in Edit Settings of series detail page that allows user to specify the correct series year, FIX:(#1427) Fixed a typo when the Scan Library option is used and ComicID's were able to be parsed from the files, FIX: When issues were archived, depending on if series had issues in a Downloaded status or not, would give different results to the Have totals, IMP: Manage tab / Manage Comics now has metatag option available to allow metatagging multiple series in a batch mode, IMP:(#1419) When 'write cvinfo' option is enabled, will now also write cvinfo into story arc directories when added / refreshed, FIX: Fixed a problem when group metatag was being used and causing errors, IMP: Manage Failed page now will link back to the series for easier reference, and the 'link' column was removed due to invalid references, FIX: Removed the 'issue not updated' from the series detail page as was not accurate, IMP: Added a Select Filtered Issues button to the series detail page to allow for selecting of all filtered issues (Show ALL MUST be enabled for it to work properly)
2016-11-02 15:00:56 +00:00
#quick check
if havefiles > combined_total :
logger . warn ( module + ' It looks like you have physical issues in the series directory, but are forcing these issues to an Archived Status. Adjusting have counts. ' )
havefiles = havefiles - arcfiles
FIX: (#2006)(#1986) Auto-Want series were not being auto-added, and even after manually added would result in count error after sucessful post-processing, FIX: When filechecking, if a 'special' was included as part of the annuals of a given series, it would not be detected, IMP: (#1982) When ComicRN & Completed Download Handling (CDH) are both enabled, will now check to see if ComicRN is active for the given category in the respective client. If it is, CDH will not post-process and issue a warning and let ComicRN handle the post-processing, FIX: Fixed a utf-8 encoding error when 'Run script AFTER snatch' was enabled, FIX: (#2004) Possible fix for 500 error on Wanted tab, when series had an annual/special attached but was located in the incorrect table, FIX: Removed some logging lines when CDH is monitoring downloads
2018-06-30 18:35:52 +00:00
thetotals = totals ( ComicID , havefiles , combined_total , module , recheck = recheck )
IMP: Added Choose specific Download option to manually select from result list on Upcoming/Details/Weekly tabs, IMP: Added Wanted storyarcs to the overall Wanted section so now will search as per global options (storyarc issues can be displayed optionally on Wanted tab), IMP: Added custom url option for image banners/posters for storyarcs, IMP: updated Cherrypy, FIX: Fixed ComicRN not working when forms authentication used - in conjunction updated autoProcessComics to 2.0 which now uses apikey instead of user/pass, IMP: Alternate Series Covers option for alternating existing series image to the most current issue image, IMP: Added overall series total to series page for reference, IMP: Search workflow completely changed to accomodate more than one indexer - now will simultaneously sumbit initial request to each provider, wait 30s submit additional as required at 30s intervals, FIX: Removed TPSE as an option and relabelled to just Public Torrents, IMP: Added direct backlog search to WWT option (pack support will follow), FIX: Removed line about configparser being required for ComicTagger usage, IMP: Test code in place for newzab testing, FIX: Fixed layout problem with torrents that are in auto-snatch status on weekly tab, IMP: backend code improvements to allow for better alias usage and annual linking directly from WS, IMP: Updated systemd init-scripts with read.me, IMP: When post-processing, will now check for available destination free space before actually moving files, IMP: Will copy during metatagging to cache folder instead of move being an option so cleanup is cleaner if something fails, FIX: Changed readinglist table to storyarcs for clarity, IMP: When post-processing issues, will now only update the one issue status and adjust totals accordingly (instead of doing a complete rescan of the series), FIX: Clear out empty ID's from the Failed DB on startup, IMP: Initial code-run at REST API interface, FIX: Fixed some matching problems with 32p due to case, IMP: removed apikeys from log entries that were accidentally logging, IMP: When searching 32p, if items get packed up - will now delete the cached reference so new items of the same can be located, IMP: ForceSearch option switched to scheduler so simultaneous runs should not occur, FIX: Fixed manual metatagging error that would occur if multiple destination directories existed
2018-02-16 19:57:01 +00:00
totalarc = arcfiles + archivedissues
2013-09-09 04:39:08 +00:00
2015-07-01 22:52:26 +00:00
#enforce permissions
2017-10-25 18:33:56 +00:00
if mylar . CONFIG . ENFORCE_PERMS :
FIX:(#1431) Fixed post-processing cleanup when using SAB + files resided in a sub-folder of the path sent from SAB, IMP: Added hardlink/softlink option for non-windows users (note that metatagging is forcibly disabled when these options are enabled), IMP: Added Enforce Permissions option to force Mylar to either force / ensure permissions are as set within Mylar, or to ignore permissions and just try to perform the given action (this is for adding series, post-processing, importing, arcs, etc), FIX: Fixed a problem where if local watchdir was set and a torrent downloader was selected, both would attempt to be used, but only the watchdir was actually utilized (now it's one or the other, never both), FIX: Mylar will now remove issues that are in Failed status that have incorrect Series Data attached, FIX:(#1438) When file-checking/post-processing if issue date in Mylar was current year and date in filename was the following year (for end-of-year issues), would cause traceback error, FIX: If series year polled from CV is incorrect or not provided, option is available in Edit Settings of series detail page that allows user to specify the correct series year, FIX:(#1427) Fixed a typo when the Scan Library option is used and ComicID's were able to be parsed from the files, FIX: When issues were archived, depending on if series had issues in a Downloaded status or not, would give different results to the Have totals, IMP: Manage tab / Manage Comics now has metatag option available to allow metatagging multiple series in a batch mode, IMP:(#1419) When 'write cvinfo' option is enabled, will now also write cvinfo into story arc directories when added / refreshed, FIX: Fixed a problem when group metatag was being used and causing errors, IMP: Manage Failed page now will link back to the series for easier reference, and the 'link' column was removed due to invalid references, FIX: Removed the 'issue not updated' from the series detail page as was not accurate, IMP: Added a Select Filtered Issues button to the series detail page to allow for selecting of all filtered issues (Show ALL MUST be enabled for it to work properly)
2016-11-02 15:00:56 +00:00
logger . fdebug ( module + ' Ensuring permissions/ownership enforced for series: ' + rescan [ ' ComicName ' ] )
filechecker . setperms ( rescan [ ' ComicLocation ' ] )
2014-12-08 17:48:13 +00:00
logger . info ( module + ' I have physically found ' + str ( foundcount ) + ' issues, ignored ' + str ( ignorecount ) + ' issues, snatched ' + str ( snatchedcount ) + ' issues, and accounted for ' + str ( totalarc ) + ' in an Archived state [ Total Issue Count: ' + str ( havefiles ) + ' / ' + str ( combined_total ) + ' ] ' )
2012-10-30 10:43:01 +00:00
FIX: (#2006)(#1986) Auto-Want series were not being auto-added, and even after manually added would result in count error after sucessful post-processing, FIX: When filechecking, if a 'special' was included as part of the annuals of a given series, it would not be detected, IMP: (#1982) When ComicRN & Completed Download Handling (CDH) are both enabled, will now check to see if ComicRN is active for the given category in the respective client. If it is, CDH will not post-process and issue a warning and let ComicRN handle the post-processing, FIX: Fixed a utf-8 encoding error when 'Run script AFTER snatch' was enabled, FIX: (#2004) Possible fix for 500 error on Wanted tab, when series had an annual/special attached but was located in the incorrect table, FIX: Removed some logging lines when CDH is monitoring downloads
2018-06-30 18:35:52 +00:00
def totals ( ComicID , havefiles = None , totalfiles = None , module = None , issueid = None , file = None , recheck = False ) :
IMP: Added Choose specific Download option to manually select from result list on Upcoming/Details/Weekly tabs, IMP: Added Wanted storyarcs to the overall Wanted section so now will search as per global options (storyarc issues can be displayed optionally on Wanted tab), IMP: Added custom url option for image banners/posters for storyarcs, IMP: updated Cherrypy, FIX: Fixed ComicRN not working when forms authentication used - in conjunction updated autoProcessComics to 2.0 which now uses apikey instead of user/pass, IMP: Alternate Series Covers option for alternating existing series image to the most current issue image, IMP: Added overall series total to series page for reference, IMP: Search workflow completely changed to accomodate more than one indexer - now will simultaneously sumbit initial request to each provider, wait 30s submit additional as required at 30s intervals, FIX: Removed TPSE as an option and relabelled to just Public Torrents, IMP: Added direct backlog search to WWT option (pack support will follow), FIX: Removed line about configparser being required for ComicTagger usage, IMP: Test code in place for newzab testing, FIX: Fixed layout problem with torrents that are in auto-snatch status on weekly tab, IMP: backend code improvements to allow for better alias usage and annual linking directly from WS, IMP: Updated systemd init-scripts with read.me, IMP: When post-processing, will now check for available destination free space before actually moving files, IMP: Will copy during metatagging to cache folder instead of move being an option so cleanup is cleaner if something fails, FIX: Changed readinglist table to storyarcs for clarity, IMP: When post-processing issues, will now only update the one issue status and adjust totals accordingly (instead of doing a complete rescan of the series), FIX: Clear out empty ID's from the Failed DB on startup, IMP: Initial code-run at REST API interface, FIX: Fixed some matching problems with 32p due to case, IMP: removed apikeys from log entries that were accidentally logging, IMP: When searching 32p, if items get packed up - will now delete the cached reference so new items of the same can be located, IMP: ForceSearch option switched to scheduler so simultaneous runs should not occur, FIX: Fixed manual metatagging error that would occur if multiple destination directories existed
2018-02-16 19:57:01 +00:00
if module is None :
module = ' [FILE-RESCAN] '
myDB = db . DBConnection ( )
IMP: When comic location is changed for a given series, will rename the directory if possible as opposed to creating a new directory, IMP:(#1920) Ability to directly input a 32p series ID for more accurate search results via the series details page/edit tab, IMP: Better error messages regarding: no CV API key being set when searching, and no Comic Location being set when trying to add a series, FIX: When using test SABnzbd button, if SABnzbd was not enabled previously (meaning config saved after the option selected), the test would return a fail status, FIX: When post-processing annuals, would fail to update the correct table resulting in a null table entry that would cause further problems, IMP: Added 1 day grace to the date check when comparing search results to allow for some leniency with regards to timezone differences, FIX: Fixed slight display issue when 32p provider was enabled, but torrents were turned off (pack option was being shown in series page when it shouldn't have been)
2018-04-26 15:27:51 +00:00
filetable = ' issues '
IMP: Added Choose specific Download option to manually select from result list on Upcoming/Details/Weekly tabs, IMP: Added Wanted storyarcs to the overall Wanted section so now will search as per global options (storyarc issues can be displayed optionally on Wanted tab), IMP: Added custom url option for image banners/posters for storyarcs, IMP: updated Cherrypy, FIX: Fixed ComicRN not working when forms authentication used - in conjunction updated autoProcessComics to 2.0 which now uses apikey instead of user/pass, IMP: Alternate Series Covers option for alternating existing series image to the most current issue image, IMP: Added overall series total to series page for reference, IMP: Search workflow completely changed to accomodate more than one indexer - now will simultaneously sumbit initial request to each provider, wait 30s submit additional as required at 30s intervals, FIX: Removed TPSE as an option and relabelled to just Public Torrents, IMP: Added direct backlog search to WWT option (pack support will follow), FIX: Removed line about configparser being required for ComicTagger usage, IMP: Test code in place for newzab testing, FIX: Fixed layout problem with torrents that are in auto-snatch status on weekly tab, IMP: backend code improvements to allow for better alias usage and annual linking directly from WS, IMP: Updated systemd init-scripts with read.me, IMP: When post-processing, will now check for available destination free space before actually moving files, IMP: Will copy during metatagging to cache folder instead of move being an option so cleanup is cleaner if something fails, FIX: Changed readinglist table to storyarcs for clarity, IMP: When post-processing issues, will now only update the one issue status and adjust totals accordingly (instead of doing a complete rescan of the series), FIX: Clear out empty ID's from the Failed DB on startup, IMP: Initial code-run at REST API interface, FIX: Fixed some matching problems with 32p due to case, IMP: removed apikeys from log entries that were accidentally logging, IMP: When searching 32p, if items get packed up - will now delete the cached reference so new items of the same can be located, IMP: ForceSearch option switched to scheduler so simultaneous runs should not occur, FIX: Fixed manual metatagging error that would occur if multiple destination directories existed
2018-02-16 19:57:01 +00:00
if any ( [ havefiles is None , havefiles == ' +1 ' ] ) :
if havefiles is None :
hf = myDB . selectone ( " SELECT Have, Total FROM comics WHERE ComicID=? " , [ ComicID ] ) . fetchone ( )
havefiles = int ( hf [ ' Have ' ] )
totalfiles = int ( hf [ ' Total ' ] )
else :
hf = myDB . selectone ( " SELECT a.Have, a.Total, b.Status as IssStatus FROM comics AS a INNER JOIN issues as b ON a.ComicID=b.ComicID WHERE b.IssueID=? " , [ issueid ] ) . fetchone ( )
if hf is None :
hf = myDB . selectone ( " SELECT a.Have, a.Total, b.Status as IssStatus FROM comics AS a INNER JOIN annuals as b ON a.ComicID=b.ComicID WHERE b.IssueID=? " , [ issueid ] ) . fetchone ( )
IMP: When comic location is changed for a given series, will rename the directory if possible as opposed to creating a new directory, IMP:(#1920) Ability to directly input a 32p series ID for more accurate search results via the series details page/edit tab, IMP: Better error messages regarding: no CV API key being set when searching, and no Comic Location being set when trying to add a series, FIX: When using test SABnzbd button, if SABnzbd was not enabled previously (meaning config saved after the option selected), the test would return a fail status, FIX: When post-processing annuals, would fail to update the correct table resulting in a null table entry that would cause further problems, IMP: Added 1 day grace to the date check when comparing search results to allow for some leniency with regards to timezone differences, FIX: Fixed slight display issue when 32p provider was enabled, but torrents were turned off (pack option was being shown in series page when it shouldn't have been)
2018-04-26 15:27:51 +00:00
filetable = ' annuals '
IMP: Added Choose specific Download option to manually select from result list on Upcoming/Details/Weekly tabs, IMP: Added Wanted storyarcs to the overall Wanted section so now will search as per global options (storyarc issues can be displayed optionally on Wanted tab), IMP: Added custom url option for image banners/posters for storyarcs, IMP: updated Cherrypy, FIX: Fixed ComicRN not working when forms authentication used - in conjunction updated autoProcessComics to 2.0 which now uses apikey instead of user/pass, IMP: Alternate Series Covers option for alternating existing series image to the most current issue image, IMP: Added overall series total to series page for reference, IMP: Search workflow completely changed to accomodate more than one indexer - now will simultaneously sumbit initial request to each provider, wait 30s submit additional as required at 30s intervals, FIX: Removed TPSE as an option and relabelled to just Public Torrents, IMP: Added direct backlog search to WWT option (pack support will follow), FIX: Removed line about configparser being required for ComicTagger usage, IMP: Test code in place for newzab testing, FIX: Fixed layout problem with torrents that are in auto-snatch status on weekly tab, IMP: backend code improvements to allow for better alias usage and annual linking directly from WS, IMP: Updated systemd init-scripts with read.me, IMP: When post-processing, will now check for available destination free space before actually moving files, IMP: Will copy during metatagging to cache folder instead of move being an option so cleanup is cleaner if something fails, FIX: Changed readinglist table to storyarcs for clarity, IMP: When post-processing issues, will now only update the one issue status and adjust totals accordingly (instead of doing a complete rescan of the series), FIX: Clear out empty ID's from the Failed DB on startup, IMP: Initial code-run at REST API interface, FIX: Fixed some matching problems with 32p due to case, IMP: removed apikeys from log entries that were accidentally logging, IMP: When searching 32p, if items get packed up - will now delete the cached reference so new items of the same can be located, IMP: ForceSearch option switched to scheduler so simultaneous runs should not occur, FIX: Fixed manual metatagging error that would occur if multiple destination directories existed
2018-02-16 19:57:01 +00:00
totalfiles = int ( hf [ ' Total ' ] )
2018-03-07 15:02:48 +00:00
logger . fdebug ( ' totalfiles: %s ' % totalfiles )
logger . fdebug ( ' status: %s ' % hf [ ' IssStatus ' ] )
IMP: Added Choose specific Download option to manually select from result list on Upcoming/Details/Weekly tabs, IMP: Added Wanted storyarcs to the overall Wanted section so now will search as per global options (storyarc issues can be displayed optionally on Wanted tab), IMP: Added custom url option for image banners/posters for storyarcs, IMP: updated Cherrypy, FIX: Fixed ComicRN not working when forms authentication used - in conjunction updated autoProcessComics to 2.0 which now uses apikey instead of user/pass, IMP: Alternate Series Covers option for alternating existing series image to the most current issue image, IMP: Added overall series total to series page for reference, IMP: Search workflow completely changed to accomodate more than one indexer - now will simultaneously sumbit initial request to each provider, wait 30s submit additional as required at 30s intervals, FIX: Removed TPSE as an option and relabelled to just Public Torrents, IMP: Added direct backlog search to WWT option (pack support will follow), FIX: Removed line about configparser being required for ComicTagger usage, IMP: Test code in place for newzab testing, FIX: Fixed layout problem with torrents that are in auto-snatch status on weekly tab, IMP: backend code improvements to allow for better alias usage and annual linking directly from WS, IMP: Updated systemd init-scripts with read.me, IMP: When post-processing, will now check for available destination free space before actually moving files, IMP: Will copy during metatagging to cache folder instead of move being an option so cleanup is cleaner if something fails, FIX: Changed readinglist table to storyarcs for clarity, IMP: When post-processing issues, will now only update the one issue status and adjust totals accordingly (instead of doing a complete rescan of the series), FIX: Clear out empty ID's from the Failed DB on startup, IMP: Initial code-run at REST API interface, FIX: Fixed some matching problems with 32p due to case, IMP: removed apikeys from log entries that were accidentally logging, IMP: When searching 32p, if items get packed up - will now delete the cached reference so new items of the same can be located, IMP: ForceSearch option switched to scheduler so simultaneous runs should not occur, FIX: Fixed manual metatagging error that would occur if multiple destination directories existed
2018-02-16 19:57:01 +00:00
if hf [ ' IssStatus ' ] != ' Downloaded ' :
FIX: (#2006)(#1986) Auto-Want series were not being auto-added, and even after manually added would result in count error after sucessful post-processing, FIX: When filechecking, if a 'special' was included as part of the annuals of a given series, it would not be detected, IMP: (#1982) When ComicRN & Completed Download Handling (CDH) are both enabled, will now check to see if ComicRN is active for the given category in the respective client. If it is, CDH will not post-process and issue a warning and let ComicRN handle the post-processing, FIX: Fixed a utf-8 encoding error when 'Run script AFTER snatch' was enabled, FIX: (#2004) Possible fix for 500 error on Wanted tab, when series had an annual/special attached but was located in the incorrect table, FIX: Removed some logging lines when CDH is monitoring downloads
2018-06-30 18:35:52 +00:00
try :
havefiles = int ( hf [ ' Have ' ] ) + 1
if havefiles > totalfiles and recheck is False :
recheck = True
return forceRescan ( ComicID , recheck = recheck )
except TypeError :
if totalfiles == 1 :
havefiles = 1
else :
2019-03-14 21:15:12 +00:00
logger . warn ( ' Total issues for this series [ComiciD: %s /IssueID: %s ] is not 1 when it should be. This is probably a mistake and the series should be refreshed. ' % ( ComicID , issueid ) )
FIX: (#2006)(#1986) Auto-Want series were not being auto-added, and even after manually added would result in count error after sucessful post-processing, FIX: When filechecking, if a 'special' was included as part of the annuals of a given series, it would not be detected, IMP: (#1982) When ComicRN & Completed Download Handling (CDH) are both enabled, will now check to see if ComicRN is active for the given category in the respective client. If it is, CDH will not post-process and issue a warning and let ComicRN handle the post-processing, FIX: Fixed a utf-8 encoding error when 'Run script AFTER snatch' was enabled, FIX: (#2004) Possible fix for 500 error on Wanted tab, when series had an annual/special attached but was located in the incorrect table, FIX: Removed some logging lines when CDH is monitoring downloads
2018-06-30 18:35:52 +00:00
havefiles = 0
2018-03-07 15:02:48 +00:00
logger . fdebug ( ' incremented havefiles: %s ' % havefiles )
IMP: Added Choose specific Download option to manually select from result list on Upcoming/Details/Weekly tabs, IMP: Added Wanted storyarcs to the overall Wanted section so now will search as per global options (storyarc issues can be displayed optionally on Wanted tab), IMP: Added custom url option for image banners/posters for storyarcs, IMP: updated Cherrypy, FIX: Fixed ComicRN not working when forms authentication used - in conjunction updated autoProcessComics to 2.0 which now uses apikey instead of user/pass, IMP: Alternate Series Covers option for alternating existing series image to the most current issue image, IMP: Added overall series total to series page for reference, IMP: Search workflow completely changed to accomodate more than one indexer - now will simultaneously sumbit initial request to each provider, wait 30s submit additional as required at 30s intervals, FIX: Removed TPSE as an option and relabelled to just Public Torrents, IMP: Added direct backlog search to WWT option (pack support will follow), FIX: Removed line about configparser being required for ComicTagger usage, IMP: Test code in place for newzab testing, FIX: Fixed layout problem with torrents that are in auto-snatch status on weekly tab, IMP: backend code improvements to allow for better alias usage and annual linking directly from WS, IMP: Updated systemd init-scripts with read.me, IMP: When post-processing, will now check for available destination free space before actually moving files, IMP: Will copy during metatagging to cache folder instead of move being an option so cleanup is cleaner if something fails, FIX: Changed readinglist table to storyarcs for clarity, IMP: When post-processing issues, will now only update the one issue status and adjust totals accordingly (instead of doing a complete rescan of the series), FIX: Clear out empty ID's from the Failed DB on startup, IMP: Initial code-run at REST API interface, FIX: Fixed some matching problems with 32p due to case, IMP: removed apikeys from log entries that were accidentally logging, IMP: When searching 32p, if items get packed up - will now delete the cached reference so new items of the same can be located, IMP: ForceSearch option switched to scheduler so simultaneous runs should not occur, FIX: Fixed manual metatagging error that would occur if multiple destination directories existed
2018-02-16 19:57:01 +00:00
else :
havefiles = int ( hf [ ' Have ' ] )
2018-03-07 15:02:48 +00:00
logger . fdebug ( ' untouched havefiles: %s ' % havefiles )
IMP: Added Choose specific Download option to manually select from result list on Upcoming/Details/Weekly tabs, IMP: Added Wanted storyarcs to the overall Wanted section so now will search as per global options (storyarc issues can be displayed optionally on Wanted tab), IMP: Added custom url option for image banners/posters for storyarcs, IMP: updated Cherrypy, FIX: Fixed ComicRN not working when forms authentication used - in conjunction updated autoProcessComics to 2.0 which now uses apikey instead of user/pass, IMP: Alternate Series Covers option for alternating existing series image to the most current issue image, IMP: Added overall series total to series page for reference, IMP: Search workflow completely changed to accomodate more than one indexer - now will simultaneously sumbit initial request to each provider, wait 30s submit additional as required at 30s intervals, FIX: Removed TPSE as an option and relabelled to just Public Torrents, IMP: Added direct backlog search to WWT option (pack support will follow), FIX: Removed line about configparser being required for ComicTagger usage, IMP: Test code in place for newzab testing, FIX: Fixed layout problem with torrents that are in auto-snatch status on weekly tab, IMP: backend code improvements to allow for better alias usage and annual linking directly from WS, IMP: Updated systemd init-scripts with read.me, IMP: When post-processing, will now check for available destination free space before actually moving files, IMP: Will copy during metatagging to cache folder instead of move being an option so cleanup is cleaner if something fails, FIX: Changed readinglist table to storyarcs for clarity, IMP: When post-processing issues, will now only update the one issue status and adjust totals accordingly (instead of doing a complete rescan of the series), FIX: Clear out empty ID's from the Failed DB on startup, IMP: Initial code-run at REST API interface, FIX: Fixed some matching problems with 32p due to case, IMP: removed apikeys from log entries that were accidentally logging, IMP: When searching 32p, if items get packed up - will now delete the cached reference so new items of the same can be located, IMP: ForceSearch option switched to scheduler so simultaneous runs should not occur, FIX: Fixed manual metatagging error that would occur if multiple destination directories existed
2018-02-16 19:57:01 +00:00
#let's update the total count of comics that was found.
#store just the total of issues, since annuals gets tracked seperately.
controlValueStat = { " ComicID " : ComicID }
newValueStat = { " Have " : havefiles ,
" Total " : totalfiles }
myDB . upsert ( " comics " , newValueStat , controlValueStat )
2018-03-07 15:02:48 +00:00
if file is not None :
controlValueStat = { " IssueID " : issueid ,
" ComicID " : ComicID }
newValueStat = { " ComicSize " : os . path . getsize ( file ) }
IMP: When comic location is changed for a given series, will rename the directory if possible as opposed to creating a new directory, IMP:(#1920) Ability to directly input a 32p series ID for more accurate search results via the series details page/edit tab, IMP: Better error messages regarding: no CV API key being set when searching, and no Comic Location being set when trying to add a series, FIX: When using test SABnzbd button, if SABnzbd was not enabled previously (meaning config saved after the option selected), the test would return a fail status, FIX: When post-processing annuals, would fail to update the correct table resulting in a null table entry that would cause further problems, IMP: Added 1 day grace to the date check when comparing search results to allow for some leniency with regards to timezone differences, FIX: Fixed slight display issue when 32p provider was enabled, but torrents were turned off (pack option was being shown in series page when it shouldn't have been)
2018-04-26 15:27:51 +00:00
myDB . upsert ( filetable , newValueStat , controlValueStat )