IMP: Changed configuration completely. New config processing, global/config variables changed over entire application. All changes in the configuration GUI take effect immediately after saving - no more need to restart Mylar, IMP: Added provider order sequence header to the top of the Providers tab for improved visibilty, IMP: Added completed download handling for both SABnzbd and NZBGet - will monitor active queue for downloads sent by Mylar and then post-process them accordingly (no more ComicRN), FIX: When recreating a pullist for a week that's not the current week, will now refresh series that are still missing data in case of late population, IMP: Removed loose/explict search options, and search results will now match more accurately to the terms entered, as well as being much quicker, IMP: Changed colour-codes on search results screen, green now indicates that the series is a print series, non-green indicates that it is either a HC/TPB/digital series and will also be indicated as such, IMP: Fixed weeklypull not refreshing some series due to legacy numbering on series deemed as Ended by Mylar, or the pull not refreshing propeprly and thereby were not elgible to be considered as a valid series for matching, IMP: Fixed problem with the scheduler of the weekly pull not running at the designated time, which causes problems when attempting to match series to the pullist for issues, IMP: Changed the autosnatch script so that the get.conf is no longer required to be maintained seperately. The get.conf config items are now located in the main config.ini and have to be repopulated initially, FIX: Fixed problem with the import count not resetting to 0 on subsequent runs of the Import A Directory option, IMP: Changed the sab api key from using the nzbkey to the full api key as is required for completed download handling, FIX: Fixed the retry option for newznab entries so that it should work across the board now for newznabs as opposed to only some providers, IMP: some minor code cleanup
2017-11-10 19:25:14 +00:00
#!/usr/bin/python
# This file is part of Harpoon.
#
# Harpoon is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# Harpoon is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with Harpoon. If not, see <http://www.gnu.org/licenses/>.
import optparse
import xmlrpclib
from base64 import standard_b64encode
from xml . dom . minidom import parseString
import os
import sys
import re
import time
import mylar
import logger
class NZBGet ( object ) :
def __init__ ( self ) :
if mylar . CONFIG . NZBGET_HOST [ : 5 ] == ' https ' :
protocol = " https "
nzbget_host = mylar . CONFIG . NZBGET_HOST [ 8 : ]
elif mylar . CONFIG . NZBGET_HOST [ : 4 ] == ' http ' :
protocol = " http "
nzbget_host = mylar . CONFIG . NZBGET_HOST [ 7 : ]
2018-05-22 17:01:48 +00:00
url = ' %s :// '
val = ( protocol , )
if mylar . CONFIG . NZBGET_USERNAME is not None :
url = url + ' %s : '
val = val + ( mylar . CONFIG . NZBGET_USERNAME , )
if mylar . CONFIG . NZBGET_PASSWORD is not None :
url = url + ' %s '
val = val + ( mylar . CONFIG . NZBGET_PASSWORD , )
if any ( [ mylar . CONFIG . NZBGET_USERNAME , mylar . CONFIG . NZBGET_PASSWORD ] ) :
url = url + ' @ %s : %s /xmlrpc '
else :
url = url + ' %s : %s /xmlrpc '
val = val + ( nzbget_host , mylar . CONFIG . NZBGET_PORT , )
self . nzb_url = ( url % val )
self . server = xmlrpclib . ServerProxy ( self . nzb_url ) #,allow_none=True)
IMP: Changed configuration completely. New config processing, global/config variables changed over entire application. All changes in the configuration GUI take effect immediately after saving - no more need to restart Mylar, IMP: Added provider order sequence header to the top of the Providers tab for improved visibilty, IMP: Added completed download handling for both SABnzbd and NZBGet - will monitor active queue for downloads sent by Mylar and then post-process them accordingly (no more ComicRN), FIX: When recreating a pullist for a week that's not the current week, will now refresh series that are still missing data in case of late population, IMP: Removed loose/explict search options, and search results will now match more accurately to the terms entered, as well as being much quicker, IMP: Changed colour-codes on search results screen, green now indicates that the series is a print series, non-green indicates that it is either a HC/TPB/digital series and will also be indicated as such, IMP: Fixed weeklypull not refreshing some series due to legacy numbering on series deemed as Ended by Mylar, or the pull not refreshing propeprly and thereby were not elgible to be considered as a valid series for matching, IMP: Fixed problem with the scheduler of the weekly pull not running at the designated time, which causes problems when attempting to match series to the pullist for issues, IMP: Changed the autosnatch script so that the get.conf is no longer required to be maintained seperately. The get.conf config items are now located in the main config.ini and have to be repopulated initially, FIX: Fixed problem with the import count not resetting to 0 on subsequent runs of the Import A Directory option, IMP: Changed the sab api key from using the nzbkey to the full api key as is required for completed download handling, FIX: Fixed the retry option for newznab entries so that it should work across the board now for newznabs as opposed to only some providers, IMP: some minor code cleanup
2017-11-10 19:25:14 +00:00
FIX:(#1905) Unicode errors when logging and system language is not a subset of english, IMP: Added post-processing queue (currently just for forceProcess api, but will be used by default for all post-processing actions soon), IMP: Multiple Torznab support added, IMP: Logging values for number of files and size/file added to config, IMP: Pushover notifications can be sent to specific device now, IMP: Added nzbget, newznab, nzb.su and dognzb test options, IMP: Added graphical checkmark/x when performing various tests (providers, downloaders, notifiers) within configuration, IMP: Removed some more unnecessary logging lines from searches
2018-04-06 17:46:39 +00:00
def sender ( self , filename , test = False ) :
IMP: Changed configuration completely. New config processing, global/config variables changed over entire application. All changes in the configuration GUI take effect immediately after saving - no more need to restart Mylar, IMP: Added provider order sequence header to the top of the Providers tab for improved visibilty, IMP: Added completed download handling for both SABnzbd and NZBGet - will monitor active queue for downloads sent by Mylar and then post-process them accordingly (no more ComicRN), FIX: When recreating a pullist for a week that's not the current week, will now refresh series that are still missing data in case of late population, IMP: Removed loose/explict search options, and search results will now match more accurately to the terms entered, as well as being much quicker, IMP: Changed colour-codes on search results screen, green now indicates that the series is a print series, non-green indicates that it is either a HC/TPB/digital series and will also be indicated as such, IMP: Fixed weeklypull not refreshing some series due to legacy numbering on series deemed as Ended by Mylar, or the pull not refreshing propeprly and thereby were not elgible to be considered as a valid series for matching, IMP: Fixed problem with the scheduler of the weekly pull not running at the designated time, which causes problems when attempting to match series to the pullist for issues, IMP: Changed the autosnatch script so that the get.conf is no longer required to be maintained seperately. The get.conf config items are now located in the main config.ini and have to be repopulated initially, FIX: Fixed problem with the import count not resetting to 0 on subsequent runs of the Import A Directory option, IMP: Changed the sab api key from using the nzbkey to the full api key as is required for completed download handling, FIX: Fixed the retry option for newznab entries so that it should work across the board now for newznabs as opposed to only some providers, IMP: some minor code cleanup
2017-11-10 19:25:14 +00:00
if mylar . CONFIG . NZBGET_PRIORITY :
if any ( [ mylar . CONFIG . NZBGET_PRIORITY == ' Default ' , mylar . CONFIG . NZBGET_PRIORITY == ' Normal ' ] ) :
nzbgetpriority = 0
elif mylar . CONFIG . NZBGET_PRIORITY == ' Low ' :
nzbgetpriority = - 50
elif mylar . CONFIG . NZBGET_PRIORITY == ' High ' :
nzbgetpriority = 50
#there's no priority for "paused", so set "Very Low" and deal with that later...
elif mylar . CONFIG . NZBGET_PRIORITY == ' Paused ' :
nzbgetpriority = - 100
else :
#if nzbget priority isn't selected, default to Normal (0)
nzbgetpriority = 0
in_file = open ( filename , ' r ' )
nzbcontent = in_file . read ( )
in_file . close ( )
nzbcontent64 = standard_b64encode ( nzbcontent )
try :
logger . fdebug ( ' sending now to %s ' % self . nzb_url )
2018-05-22 17:01:48 +00:00
if mylar . CONFIG . NZBGET_CATEGORY is None :
nzb_category = ' '
else :
nzb_category = mylar . CONFIG . NZBGET_CATEGORY
sendresponse = self . server . append ( filename , nzbcontent64 , nzb_category , nzbgetpriority , False , False , ' ' , 0 , ' SCORE ' )
IMP: Changed configuration completely. New config processing, global/config variables changed over entire application. All changes in the configuration GUI take effect immediately after saving - no more need to restart Mylar, IMP: Added provider order sequence header to the top of the Providers tab for improved visibilty, IMP: Added completed download handling for both SABnzbd and NZBGet - will monitor active queue for downloads sent by Mylar and then post-process them accordingly (no more ComicRN), FIX: When recreating a pullist for a week that's not the current week, will now refresh series that are still missing data in case of late population, IMP: Removed loose/explict search options, and search results will now match more accurately to the terms entered, as well as being much quicker, IMP: Changed colour-codes on search results screen, green now indicates that the series is a print series, non-green indicates that it is either a HC/TPB/digital series and will also be indicated as such, IMP: Fixed weeklypull not refreshing some series due to legacy numbering on series deemed as Ended by Mylar, or the pull not refreshing propeprly and thereby were not elgible to be considered as a valid series for matching, IMP: Fixed problem with the scheduler of the weekly pull not running at the designated time, which causes problems when attempting to match series to the pullist for issues, IMP: Changed the autosnatch script so that the get.conf is no longer required to be maintained seperately. The get.conf config items are now located in the main config.ini and have to be repopulated initially, FIX: Fixed problem with the import count not resetting to 0 on subsequent runs of the Import A Directory option, IMP: Changed the sab api key from using the nzbkey to the full api key as is required for completed download handling, FIX: Fixed the retry option for newznab entries so that it should work across the board now for newznabs as opposed to only some providers, IMP: some minor code cleanup
2017-11-10 19:25:14 +00:00
except Exception as e :
logger . warn ( ' uh-oh: %s ' % e )
return { ' status ' : False }
else :
if sendresponse < = 0 :
logger . warn ( ' Invalid response received after sending to NZBGet: %s ' % sendresponse )
return { ' status ' : False }
else :
#sendresponse is the NZBID that we use to track the progress....
return { ' status ' : True ,
' NZBID ' : sendresponse }
def processor ( self , nzbinfo ) :
nzbid = nzbinfo [ ' NZBID ' ]
try :
logger . fdebug ( ' Now checking the active queue of nzbget for the download ' )
queueinfo = self . server . listgroups ( )
2018-06-08 19:42:12 +00:00
except Exception as e :
IMP: Changed configuration completely. New config processing, global/config variables changed over entire application. All changes in the configuration GUI take effect immediately after saving - no more need to restart Mylar, IMP: Added provider order sequence header to the top of the Providers tab for improved visibilty, IMP: Added completed download handling for both SABnzbd and NZBGet - will monitor active queue for downloads sent by Mylar and then post-process them accordingly (no more ComicRN), FIX: When recreating a pullist for a week that's not the current week, will now refresh series that are still missing data in case of late population, IMP: Removed loose/explict search options, and search results will now match more accurately to the terms entered, as well as being much quicker, IMP: Changed colour-codes on search results screen, green now indicates that the series is a print series, non-green indicates that it is either a HC/TPB/digital series and will also be indicated as such, IMP: Fixed weeklypull not refreshing some series due to legacy numbering on series deemed as Ended by Mylar, or the pull not refreshing propeprly and thereby were not elgible to be considered as a valid series for matching, IMP: Fixed problem with the scheduler of the weekly pull not running at the designated time, which causes problems when attempting to match series to the pullist for issues, IMP: Changed the autosnatch script so that the get.conf is no longer required to be maintained seperately. The get.conf config items are now located in the main config.ini and have to be repopulated initially, FIX: Fixed problem with the import count not resetting to 0 on subsequent runs of the Import A Directory option, IMP: Changed the sab api key from using the nzbkey to the full api key as is required for completed download handling, FIX: Fixed the retry option for newznab entries so that it should work across the board now for newznabs as opposed to only some providers, IMP: some minor code cleanup
2017-11-10 19:25:14 +00:00
logger . warn ( ' Error attempting to retrieve active queue listing: %s ' % e )
return { ' status ' : False }
else :
logger . fdebug ( ' valid queue result returned. Analyzing... ' )
queuedl = [ qu for qu in queueinfo if qu [ ' NZBID ' ] == nzbid ]
if len ( queuedl ) == 0 :
2018-06-17 01:10:06 +00:00
logger . warn ( ' Unable to locate NZBID %s in active queue. Could it be finished already ? ' % nzbid )
IMP: Updated CT to a newer version - changed to use rarfile, fixes size invalid errors with some cbr's encountered during conversions, IMP: Search and Post-Processing Queues now added in order to queue up all searches/post-processing in sequence instead of loading lists and iterating through it and encountering various lock-related errors, IMP: In most cases, will now post-process directly against comicid/issueid/storyarcid instead of relying on relationship via nzb/torrent, FIX: realigned some checkboxes on series detail page, FIX: in configuration, indicated that ComicRN cannot be used when Completed Download Handling is enabled for a client now, FIX: Fix for issues named as 'special' that are part of the annuals section, but would not work in various portions (also will allow for naming differences in the future hopefully), FIX: Will now trap error properly when file cannot be located during dupecheck so that post-processing can continue without error, FIX: Fixed newznab test option on configuration screen so that it will return the correct response (previous was an invalid api endpoint), FIX: When retrieving image for storyarcs would error when no image was present and stop the process, FIX: accounted for some incorrect newznab responses that would return a non-api endpoint when retrieving via rss, which resulted in invalid nzbs, FIX: in some cases, weekly pull snatches would not update properly on the weekly page due to some db values not being written
2018-06-14 17:07:39 +00:00
return self . historycheck ( nzbinfo )
IMP: Changed configuration completely. New config processing, global/config variables changed over entire application. All changes in the configuration GUI take effect immediately after saving - no more need to restart Mylar, IMP: Added provider order sequence header to the top of the Providers tab for improved visibilty, IMP: Added completed download handling for both SABnzbd and NZBGet - will monitor active queue for downloads sent by Mylar and then post-process them accordingly (no more ComicRN), FIX: When recreating a pullist for a week that's not the current week, will now refresh series that are still missing data in case of late population, IMP: Removed loose/explict search options, and search results will now match more accurately to the terms entered, as well as being much quicker, IMP: Changed colour-codes on search results screen, green now indicates that the series is a print series, non-green indicates that it is either a HC/TPB/digital series and will also be indicated as such, IMP: Fixed weeklypull not refreshing some series due to legacy numbering on series deemed as Ended by Mylar, or the pull not refreshing propeprly and thereby were not elgible to be considered as a valid series for matching, IMP: Fixed problem with the scheduler of the weekly pull not running at the designated time, which causes problems when attempting to match series to the pullist for issues, IMP: Changed the autosnatch script so that the get.conf is no longer required to be maintained seperately. The get.conf config items are now located in the main config.ini and have to be repopulated initially, FIX: Fixed problem with the import count not resetting to 0 on subsequent runs of the Import A Directory option, IMP: Changed the sab api key from using the nzbkey to the full api key as is required for completed download handling, FIX: Fixed the retry option for newznab entries so that it should work across the board now for newznabs as opposed to only some providers, IMP: some minor code cleanup
2017-11-10 19:25:14 +00:00
stat = False
FIX: (#2006)(#1986) Auto-Want series were not being auto-added, and even after manually added would result in count error after sucessful post-processing, FIX: When filechecking, if a 'special' was included as part of the annuals of a given series, it would not be detected, IMP: (#1982) When ComicRN & Completed Download Handling (CDH) are both enabled, will now check to see if ComicRN is active for the given category in the respective client. If it is, CDH will not post-process and issue a warning and let ComicRN handle the post-processing, FIX: Fixed a utf-8 encoding error when 'Run script AFTER snatch' was enabled, FIX: (#2004) Possible fix for 500 error on Wanted tab, when series had an annual/special attached but was located in the incorrect table, FIX: Removed some logging lines when CDH is monitoring downloads
2018-06-30 18:35:52 +00:00
double_pp = False
IMP: Tiered wanted list searching - anything older than 2 weeks will only by searched for using RSS and/or manual means (scheduled searches only), IMP: Tiered option viewable in Upcoming tab, FIX: Fixed API searching against WWT option (RSS is still broken however, as that is due to WWT), IMP: Removed cherrypy logging in some instances, IMP: Updated cfscrape library to most recent, FIX: Fixed problem with simliar titles being incorrectly post-processed due to multiple matches when manual post-processing, FIX: Fixed notifications not firing off on a successful manual post-process/folder monitor due to some previous changes, FIX: If grabbag_dir was not set, would not correctly set the location, FIX:(#2051) When using nzbget, if nzbToMylar is enabled along with CDH - warning will be issued within Mylar to indicate as such
2018-08-31 16:03:54 +00:00
double_type = None
IMP: Changed configuration completely. New config processing, global/config variables changed over entire application. All changes in the configuration GUI take effect immediately after saving - no more need to restart Mylar, IMP: Added provider order sequence header to the top of the Providers tab for improved visibilty, IMP: Added completed download handling for both SABnzbd and NZBGet - will monitor active queue for downloads sent by Mylar and then post-process them accordingly (no more ComicRN), FIX: When recreating a pullist for a week that's not the current week, will now refresh series that are still missing data in case of late population, IMP: Removed loose/explict search options, and search results will now match more accurately to the terms entered, as well as being much quicker, IMP: Changed colour-codes on search results screen, green now indicates that the series is a print series, non-green indicates that it is either a HC/TPB/digital series and will also be indicated as such, IMP: Fixed weeklypull not refreshing some series due to legacy numbering on series deemed as Ended by Mylar, or the pull not refreshing propeprly and thereby were not elgible to be considered as a valid series for matching, IMP: Fixed problem with the scheduler of the weekly pull not running at the designated time, which causes problems when attempting to match series to the pullist for issues, IMP: Changed the autosnatch script so that the get.conf is no longer required to be maintained seperately. The get.conf config items are now located in the main config.ini and have to be repopulated initially, FIX: Fixed problem with the import count not resetting to 0 on subsequent runs of the Import A Directory option, IMP: Changed the sab api key from using the nzbkey to the full api key as is required for completed download handling, FIX: Fixed the retry option for newznab entries so that it should work across the board now for newznabs as opposed to only some providers, IMP: some minor code cleanup
2017-11-10 19:25:14 +00:00
while stat is False :
time . sleep ( 10 )
queueinfo = self . server . listgroups ( )
queuedl = [ qu for qu in queueinfo if qu [ ' NZBID ' ] == nzbid ]
if len ( queuedl ) == 0 :
logger . fdebug ( ' Item is no longer in active queue. It should be finished by my calculations ' )
stat = True
else :
FIX: (#2006)(#1986) Auto-Want series were not being auto-added, and even after manually added would result in count error after sucessful post-processing, FIX: When filechecking, if a 'special' was included as part of the annuals of a given series, it would not be detected, IMP: (#1982) When ComicRN & Completed Download Handling (CDH) are both enabled, will now check to see if ComicRN is active for the given category in the respective client. If it is, CDH will not post-process and issue a warning and let ComicRN handle the post-processing, FIX: Fixed a utf-8 encoding error when 'Run script AFTER snatch' was enabled, FIX: (#2004) Possible fix for 500 error on Wanted tab, when series had an annual/special attached but was located in the incorrect table, FIX: Removed some logging lines when CDH is monitoring downloads
2018-06-30 18:35:52 +00:00
if ' comicrn ' in queuedl [ 0 ] [ ' PostInfoText ' ] . lower ( ) :
double_pp = True
IMP: Tiered wanted list searching - anything older than 2 weeks will only by searched for using RSS and/or manual means (scheduled searches only), IMP: Tiered option viewable in Upcoming tab, FIX: Fixed API searching against WWT option (RSS is still broken however, as that is due to WWT), IMP: Removed cherrypy logging in some instances, IMP: Updated cfscrape library to most recent, FIX: Fixed problem with simliar titles being incorrectly post-processed due to multiple matches when manual post-processing, FIX: Fixed notifications not firing off on a successful manual post-process/folder monitor due to some previous changes, FIX: If grabbag_dir was not set, would not correctly set the location, FIX:(#2051) When using nzbget, if nzbToMylar is enabled along with CDH - warning will be issued within Mylar to indicate as such
2018-08-31 16:03:54 +00:00
double_type = ' ComicRN '
elif ' nzbtomylar ' in queuedl [ 0 ] [ ' PostInfoText ' ] . lower ( ) :
double_pp = True
double_type = ' nzbToMylar '
FIX: (#2006)(#1986) Auto-Want series were not being auto-added, and even after manually added would result in count error after sucessful post-processing, FIX: When filechecking, if a 'special' was included as part of the annuals of a given series, it would not be detected, IMP: (#1982) When ComicRN & Completed Download Handling (CDH) are both enabled, will now check to see if ComicRN is active for the given category in the respective client. If it is, CDH will not post-process and issue a warning and let ComicRN handle the post-processing, FIX: Fixed a utf-8 encoding error when 'Run script AFTER snatch' was enabled, FIX: (#2004) Possible fix for 500 error on Wanted tab, when series had an annual/special attached but was located in the incorrect table, FIX: Removed some logging lines when CDH is monitoring downloads
2018-06-30 18:35:52 +00:00
if all ( [ len ( queuedl [ 0 ] [ ' ScriptStatuses ' ] ) > 0 , double_pp is False ] ) :
for x in queuedl [ 0 ] [ ' ScriptStatuses ' ] :
if ' comicrn ' in x [ ' Name ' ] . lower ( ) :
double_pp = True
IMP: Tiered wanted list searching - anything older than 2 weeks will only by searched for using RSS and/or manual means (scheduled searches only), IMP: Tiered option viewable in Upcoming tab, FIX: Fixed API searching against WWT option (RSS is still broken however, as that is due to WWT), IMP: Removed cherrypy logging in some instances, IMP: Updated cfscrape library to most recent, FIX: Fixed problem with simliar titles being incorrectly post-processed due to multiple matches when manual post-processing, FIX: Fixed notifications not firing off on a successful manual post-process/folder monitor due to some previous changes, FIX: If grabbag_dir was not set, would not correctly set the location, FIX:(#2051) When using nzbget, if nzbToMylar is enabled along with CDH - warning will be issued within Mylar to indicate as such
2018-08-31 16:03:54 +00:00
double_type = ' ComicRN '
break
elif ' nzbtomylar ' in x [ ' Name ' ] . lower ( ) :
double_pp = True
double_type = ' nzbToMylar '
FIX: (#2006)(#1986) Auto-Want series were not being auto-added, and even after manually added would result in count error after sucessful post-processing, FIX: When filechecking, if a 'special' was included as part of the annuals of a given series, it would not be detected, IMP: (#1982) When ComicRN & Completed Download Handling (CDH) are both enabled, will now check to see if ComicRN is active for the given category in the respective client. If it is, CDH will not post-process and issue a warning and let ComicRN handle the post-processing, FIX: Fixed a utf-8 encoding error when 'Run script AFTER snatch' was enabled, FIX: (#2004) Possible fix for 500 error on Wanted tab, when series had an annual/special attached but was located in the incorrect table, FIX: Removed some logging lines when CDH is monitoring downloads
2018-06-30 18:35:52 +00:00
break
if all ( [ len ( queuedl [ 0 ] [ ' Parameters ' ] ) > 0 , double_pp is False ] ) :
for x in queuedl [ 0 ] [ ' Parameters ' ] :
if all ( [ ' comicrn ' in x [ ' Name ' ] . lower ( ) , x [ ' Value ' ] == ' yes ' ] ) :
double_pp = True
IMP: Tiered wanted list searching - anything older than 2 weeks will only by searched for using RSS and/or manual means (scheduled searches only), IMP: Tiered option viewable in Upcoming tab, FIX: Fixed API searching against WWT option (RSS is still broken however, as that is due to WWT), IMP: Removed cherrypy logging in some instances, IMP: Updated cfscrape library to most recent, FIX: Fixed problem with simliar titles being incorrectly post-processed due to multiple matches when manual post-processing, FIX: Fixed notifications not firing off on a successful manual post-process/folder monitor due to some previous changes, FIX: If grabbag_dir was not set, would not correctly set the location, FIX:(#2051) When using nzbget, if nzbToMylar is enabled along with CDH - warning will be issued within Mylar to indicate as such
2018-08-31 16:03:54 +00:00
double_type = ' ComicRN '
FIX: (#2006)(#1986) Auto-Want series were not being auto-added, and even after manually added would result in count error after sucessful post-processing, FIX: When filechecking, if a 'special' was included as part of the annuals of a given series, it would not be detected, IMP: (#1982) When ComicRN & Completed Download Handling (CDH) are both enabled, will now check to see if ComicRN is active for the given category in the respective client. If it is, CDH will not post-process and issue a warning and let ComicRN handle the post-processing, FIX: Fixed a utf-8 encoding error when 'Run script AFTER snatch' was enabled, FIX: (#2004) Possible fix for 500 error on Wanted tab, when series had an annual/special attached but was located in the incorrect table, FIX: Removed some logging lines when CDH is monitoring downloads
2018-06-30 18:35:52 +00:00
break
IMP: Tiered wanted list searching - anything older than 2 weeks will only by searched for using RSS and/or manual means (scheduled searches only), IMP: Tiered option viewable in Upcoming tab, FIX: Fixed API searching against WWT option (RSS is still broken however, as that is due to WWT), IMP: Removed cherrypy logging in some instances, IMP: Updated cfscrape library to most recent, FIX: Fixed problem with simliar titles being incorrectly post-processed due to multiple matches when manual post-processing, FIX: Fixed notifications not firing off on a successful manual post-process/folder monitor due to some previous changes, FIX: If grabbag_dir was not set, would not correctly set the location, FIX:(#2051) When using nzbget, if nzbToMylar is enabled along with CDH - warning will be issued within Mylar to indicate as such
2018-08-31 16:03:54 +00:00
elif all ( [ ' nzbtomylar ' in x [ ' Name ' ] . lower ( ) , x [ ' Value ' ] == ' yes ' ] ) :
double_pp = True
double_type = ' nzbToMylar '
break
FIX: (#2006)(#1986) Auto-Want series were not being auto-added, and even after manually added would result in count error after sucessful post-processing, FIX: When filechecking, if a 'special' was included as part of the annuals of a given series, it would not be detected, IMP: (#1982) When ComicRN & Completed Download Handling (CDH) are both enabled, will now check to see if ComicRN is active for the given category in the respective client. If it is, CDH will not post-process and issue a warning and let ComicRN handle the post-processing, FIX: Fixed a utf-8 encoding error when 'Run script AFTER snatch' was enabled, FIX: (#2004) Possible fix for 500 error on Wanted tab, when series had an annual/special attached but was located in the incorrect table, FIX: Removed some logging lines when CDH is monitoring downloads
2018-06-30 18:35:52 +00:00
if double_pp is True :
IMP: Tiered wanted list searching - anything older than 2 weeks will only by searched for using RSS and/or manual means (scheduled searches only), IMP: Tiered option viewable in Upcoming tab, FIX: Fixed API searching against WWT option (RSS is still broken however, as that is due to WWT), IMP: Removed cherrypy logging in some instances, IMP: Updated cfscrape library to most recent, FIX: Fixed problem with simliar titles being incorrectly post-processed due to multiple matches when manual post-processing, FIX: Fixed notifications not firing off on a successful manual post-process/folder monitor due to some previous changes, FIX: If grabbag_dir was not set, would not correctly set the location, FIX:(#2051) When using nzbget, if nzbToMylar is enabled along with CDH - warning will be issued within Mylar to indicate as such
2018-08-31 16:03:54 +00:00
logger . warn ( ' %s has been detected as being active for this category & download. Completed Download Handling will NOT be performed due to this. ' % double_type )
logger . warn ( ' Either disable Completed Download Handling for NZBGet within Mylar, or remove %s from your category script in NZBGet. ' % double_type )
FIX: (#2006)(#1986) Auto-Want series were not being auto-added, and even after manually added would result in count error after sucessful post-processing, FIX: When filechecking, if a 'special' was included as part of the annuals of a given series, it would not be detected, IMP: (#1982) When ComicRN & Completed Download Handling (CDH) are both enabled, will now check to see if ComicRN is active for the given category in the respective client. If it is, CDH will not post-process and issue a warning and let ComicRN handle the post-processing, FIX: Fixed a utf-8 encoding error when 'Run script AFTER snatch' was enabled, FIX: (#2004) Possible fix for 500 error on Wanted tab, when series had an annual/special attached but was located in the incorrect table, FIX: Removed some logging lines when CDH is monitoring downloads
2018-06-30 18:35:52 +00:00
return { ' status ' : ' double-pp ' , ' failed ' : False }
IMP: Changed configuration completely. New config processing, global/config variables changed over entire application. All changes in the configuration GUI take effect immediately after saving - no more need to restart Mylar, IMP: Added provider order sequence header to the top of the Providers tab for improved visibilty, IMP: Added completed download handling for both SABnzbd and NZBGet - will monitor active queue for downloads sent by Mylar and then post-process them accordingly (no more ComicRN), FIX: When recreating a pullist for a week that's not the current week, will now refresh series that are still missing data in case of late population, IMP: Removed loose/explict search options, and search results will now match more accurately to the terms entered, as well as being much quicker, IMP: Changed colour-codes on search results screen, green now indicates that the series is a print series, non-green indicates that it is either a HC/TPB/digital series and will also be indicated as such, IMP: Fixed weeklypull not refreshing some series due to legacy numbering on series deemed as Ended by Mylar, or the pull not refreshing propeprly and thereby were not elgible to be considered as a valid series for matching, IMP: Fixed problem with the scheduler of the weekly pull not running at the designated time, which causes problems when attempting to match series to the pullist for issues, IMP: Changed the autosnatch script so that the get.conf is no longer required to be maintained seperately. The get.conf config items are now located in the main config.ini and have to be repopulated initially, FIX: Fixed problem with the import count not resetting to 0 on subsequent runs of the Import A Directory option, IMP: Changed the sab api key from using the nzbkey to the full api key as is required for completed download handling, FIX: Fixed the retry option for newznab entries so that it should work across the board now for newznabs as opposed to only some providers, IMP: some minor code cleanup
2017-11-10 19:25:14 +00:00
logger . fdebug ( ' status: %s ' % queuedl [ 0 ] [ ' Status ' ] )
logger . fdebug ( ' name: %s ' % queuedl [ 0 ] [ ' NZBName ' ] )
logger . fdebug ( ' FileSize: %s MB ' % queuedl [ 0 ] [ ' FileSizeMB ' ] )
logger . fdebug ( ' Download Left: %s MB ' % queuedl [ 0 ] [ ' RemainingSizeMB ' ] )
logger . fdebug ( ' health: %s ' % ( queuedl [ 0 ] [ ' Health ' ] / 10 ) )
logger . fdebug ( ' destination: %s ' % queuedl [ 0 ] [ ' DestDir ' ] )
2018-05-24 15:36:37 +00:00
IMP: Changed configuration completely. New config processing, global/config variables changed over entire application. All changes in the configuration GUI take effect immediately after saving - no more need to restart Mylar, IMP: Added provider order sequence header to the top of the Providers tab for improved visibilty, IMP: Added completed download handling for both SABnzbd and NZBGet - will monitor active queue for downloads sent by Mylar and then post-process them accordingly (no more ComicRN), FIX: When recreating a pullist for a week that's not the current week, will now refresh series that are still missing data in case of late population, IMP: Removed loose/explict search options, and search results will now match more accurately to the terms entered, as well as being much quicker, IMP: Changed colour-codes on search results screen, green now indicates that the series is a print series, non-green indicates that it is either a HC/TPB/digital series and will also be indicated as such, IMP: Fixed weeklypull not refreshing some series due to legacy numbering on series deemed as Ended by Mylar, or the pull not refreshing propeprly and thereby were not elgible to be considered as a valid series for matching, IMP: Fixed problem with the scheduler of the weekly pull not running at the designated time, which causes problems when attempting to match series to the pullist for issues, IMP: Changed the autosnatch script so that the get.conf is no longer required to be maintained seperately. The get.conf config items are now located in the main config.ini and have to be repopulated initially, FIX: Fixed problem with the import count not resetting to 0 on subsequent runs of the Import A Directory option, IMP: Changed the sab api key from using the nzbkey to the full api key as is required for completed download handling, FIX: Fixed the retry option for newznab entries so that it should work across the board now for newznabs as opposed to only some providers, IMP: some minor code cleanup
2017-11-10 19:25:14 +00:00
logger . fdebug ( ' File has now downloaded! ' )
time . sleep ( 5 ) #wait some seconds so shit can get written to history properly
IMP: Updated CT to a newer version - changed to use rarfile, fixes size invalid errors with some cbr's encountered during conversions, IMP: Search and Post-Processing Queues now added in order to queue up all searches/post-processing in sequence instead of loading lists and iterating through it and encountering various lock-related errors, IMP: In most cases, will now post-process directly against comicid/issueid/storyarcid instead of relying on relationship via nzb/torrent, FIX: realigned some checkboxes on series detail page, FIX: in configuration, indicated that ComicRN cannot be used when Completed Download Handling is enabled for a client now, FIX: Fix for issues named as 'special' that are part of the annuals section, but would not work in various portions (also will allow for naming differences in the future hopefully), FIX: Will now trap error properly when file cannot be located during dupecheck so that post-processing can continue without error, FIX: Fixed newznab test option on configuration screen so that it will return the correct response (previous was an invalid api endpoint), FIX: When retrieving image for storyarcs would error when no image was present and stop the process, FIX: accounted for some incorrect newznab responses that would return a non-api endpoint when retrieving via rss, which resulted in invalid nzbs, FIX: in some cases, weekly pull snatches would not update properly on the weekly page due to some db values not being written
2018-06-14 17:07:39 +00:00
return self . historycheck ( nzbinfo )
2017-12-01 22:10:20 +00:00
IMP: Updated CT to a newer version - changed to use rarfile, fixes size invalid errors with some cbr's encountered during conversions, IMP: Search and Post-Processing Queues now added in order to queue up all searches/post-processing in sequence instead of loading lists and iterating through it and encountering various lock-related errors, IMP: In most cases, will now post-process directly against comicid/issueid/storyarcid instead of relying on relationship via nzb/torrent, FIX: realigned some checkboxes on series detail page, FIX: in configuration, indicated that ComicRN cannot be used when Completed Download Handling is enabled for a client now, FIX: Fix for issues named as 'special' that are part of the annuals section, but would not work in various portions (also will allow for naming differences in the future hopefully), FIX: Will now trap error properly when file cannot be located during dupecheck so that post-processing can continue without error, FIX: Fixed newznab test option on configuration screen so that it will return the correct response (previous was an invalid api endpoint), FIX: When retrieving image for storyarcs would error when no image was present and stop the process, FIX: accounted for some incorrect newznab responses that would return a non-api endpoint when retrieving via rss, which resulted in invalid nzbs, FIX: in some cases, weekly pull snatches would not update properly on the weekly page due to some db values not being written
2018-06-14 17:07:39 +00:00
def historycheck ( self , nzbinfo ) :
nzbid = nzbinfo [ ' NZBID ' ]
2018-06-20 18:18:49 +00:00
history = self . server . history ( True )
2018-05-24 15:36:37 +00:00
found = False
2018-06-20 18:18:49 +00:00
destdir = None
FIX: (#2006)(#1986) Auto-Want series were not being auto-added, and even after manually added would result in count error after sucessful post-processing, FIX: When filechecking, if a 'special' was included as part of the annuals of a given series, it would not be detected, IMP: (#1982) When ComicRN & Completed Download Handling (CDH) are both enabled, will now check to see if ComicRN is active for the given category in the respective client. If it is, CDH will not post-process and issue a warning and let ComicRN handle the post-processing, FIX: Fixed a utf-8 encoding error when 'Run script AFTER snatch' was enabled, FIX: (#2004) Possible fix for 500 error on Wanted tab, when series had an annual/special attached but was located in the incorrect table, FIX: Removed some logging lines when CDH is monitoring downloads
2018-06-30 18:35:52 +00:00
double_pp = False
IMP: Tiered wanted list searching - anything older than 2 weeks will only by searched for using RSS and/or manual means (scheduled searches only), IMP: Tiered option viewable in Upcoming tab, FIX: Fixed API searching against WWT option (RSS is still broken however, as that is due to WWT), IMP: Removed cherrypy logging in some instances, IMP: Updated cfscrape library to most recent, FIX: Fixed problem with simliar titles being incorrectly post-processed due to multiple matches when manual post-processing, FIX: Fixed notifications not firing off on a successful manual post-process/folder monitor due to some previous changes, FIX: If grabbag_dir was not set, would not correctly set the location, FIX:(#2051) When using nzbget, if nzbToMylar is enabled along with CDH - warning will be issued within Mylar to indicate as such
2018-08-31 16:03:54 +00:00
hq = [ hs for hs in history if hs [ ' NZBID ' ] == nzbid and ( ' SUCCESS ' in hs [ ' Status ' ] or ( ' COPY ' in hs [ ' Status ' ] and ' DELETED ' not in hq [ 0 ] [ ' Status ' ] ) ) ]
2018-05-24 15:36:37 +00:00
if len ( hq ) > 0 :
logger . fdebug ( ' found matching completed item in history. Job has a status of %s ' % hq [ 0 ] [ ' Status ' ] )
FIX: (#2006)(#1986) Auto-Want series were not being auto-added, and even after manually added would result in count error after sucessful post-processing, FIX: When filechecking, if a 'special' was included as part of the annuals of a given series, it would not be detected, IMP: (#1982) When ComicRN & Completed Download Handling (CDH) are both enabled, will now check to see if ComicRN is active for the given category in the respective client. If it is, CDH will not post-process and issue a warning and let ComicRN handle the post-processing, FIX: Fixed a utf-8 encoding error when 'Run script AFTER snatch' was enabled, FIX: (#2004) Possible fix for 500 error on Wanted tab, when series had an annual/special attached but was located in the incorrect table, FIX: Removed some logging lines when CDH is monitoring downloads
2018-06-30 18:35:52 +00:00
if len ( hq [ 0 ] [ ' ScriptStatuses ' ] ) > 0 :
for x in hq [ 0 ] [ ' ScriptStatuses ' ] :
if ' comicrn ' in x [ ' Name ' ] . lower ( ) :
double_pp = True
break
if all ( [ len ( hq [ 0 ] [ ' Parameters ' ] ) > 0 , double_pp is False ] ) :
for x in hq [ 0 ] [ ' Parameters ' ] :
if all ( [ ' comicrn ' in x [ ' Name ' ] . lower ( ) , x [ ' Value ' ] == ' yes ' ] ) :
double_pp = True
break
if double_pp is True :
logger . warn ( ' ComicRN has been detected as being active for this category & download. Completed Download Handling will NOT be performed due to this. ' )
logger . warn ( ' Either disable Completed Download Handling for NZBGet within Mylar, or remove ComicRN from your category script in NZBGet. ' )
return { ' status ' : ' double-pp ' , ' failed ' : False }
2018-08-31 21:22:38 +00:00
if all ( [ ' SUCCESS ' in hq [ 0 ] [ ' Status ' ] , ( hq [ 0 ] [ ' FileSizeMB ' ] * .95 ) < = hq [ 0 ] [ ' DownloadedSizeMB ' ] < = ( hq [ 0 ] [ ' FileSizeMB ' ] * 1.05 ) ] ) :
2018-05-24 15:36:37 +00:00
logger . fdebug ( ' %s has final file size of %s MB ' % ( hq [ 0 ] [ ' Name ' ] , hq [ 0 ] [ ' DownloadedSizeMB ' ] ) )
if os . path . isdir ( hq [ 0 ] [ ' DestDir ' ] ) :
2018-06-20 18:18:49 +00:00
destdir = hq [ 0 ] [ ' DestDir ' ]
logger . fdebug ( ' location found @ %s ' % destdir )
elif all ( [ ' COPY ' in hq [ 0 ] [ ' Status ' ] , int ( hq [ 0 ] [ ' FileSizeMB ' ] ) > 0 , hq [ 0 ] [ ' DeleteStatus ' ] == ' COPY ' ] ) :
config = self . server . config ( )
cDestDir = None
for x in config :
if x [ ' Name ' ] == ' TempDir ' :
cTempDir = x [ ' Value ' ]
elif x [ ' Name ' ] == ' DestDir ' :
cDestDir = x [ ' Value ' ]
if cDestDir is not None :
break
2018-05-24 15:36:37 +00:00
2018-06-20 18:18:49 +00:00
if cTempDir in hq [ 0 ] [ ' DestDir ' ] :
destdir2 = re . sub ( cTempDir , cDestDir , hq [ 0 ] [ ' DestDir ' ] ) . strip ( )
if not destdir2 . endswith ( os . sep ) :
destdir2 = destdir2 + os . sep
destdir = os . path . join ( destdir2 , hq [ 0 ] [ ' Name ' ] )
logger . fdebug ( ' NZBGET Destination dir set to: %s ' % destdir )
else :
logger . warn ( ' no file found where it should be @ %s - is there another script that moves things after completion ? ' % hq [ 0 ] [ ' DestDir ' ] )
return { ' status ' : False }
if mylar . CONFIG . NZBGET_DIRECTORY is not None :
destdir2 = mylar . CONFIG . NZBGET_DIRECTORY
if not destdir2 . endswith ( os . sep ) :
destdir = destdir2 + os . sep
destdir = os . path . join ( destdir2 , hq [ 0 ] [ ' Name ' ] )
logger . fdebug ( ' NZBGet Destination folder set via config to: %s ' % destdir )
if destdir is not None :
return { ' status ' : True ,
' name ' : re . sub ( ' .nzb ' , ' ' , hq [ 0 ] [ ' Name ' ] ) . strip ( ) ,
' location ' : destdir ,
' failed ' : False ,
' issueid ' : nzbinfo [ ' issueid ' ] ,
' comicid ' : nzbinfo [ ' comicid ' ] ,
' apicall ' : True }
2018-05-24 15:36:37 +00:00
else :
2018-06-16 22:00:18 +00:00
logger . warn ( ' Could not find completed NZBID %s in history ' % nzbid )
2018-05-24 15:36:37 +00:00
return { ' status ' : False }