2012-09-13 15:27:34 +00:00
# This file is part of Mylar.
#
# Mylar is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# Mylar is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with Mylar. If not, see <http://www.gnu.org/licenses/>.
from __future__ import with_statement
FIX: undefined on filter box on startup, IMP: Added Meta-Tagging options on a series / issue basis on comic details screen, IMP: Issue Information is now available per issue and is extracted currently from the cbz file to display (if no cbz is present, the option isn't available), IMP: Failed Download handling is implemented and available in GUI - required to replace existing autoProcessComics.py and ComicRN.py scripts, IMP: Added ability to specify post-processing script instead of ComicRN.py, IMP: Added abilty to edit the issue date for a given issue by simply clicking on it - this will help to avoid dates that have incorrect or 0000-00-00, IMP: Story Arc searching is working (not adding yet), IMP: Added Archived/Ignored options to Upcoming/Wanted tab, IMP: Fixed some alignment and display issues on the Upcoming section, IMP: Added better directory handling for Updating Comic Location when it needs to get changed for all existing series (locmove in config.ini), IMP: Added better handling for unicode characters in series titles when searching / filechecking, IMP: When adding a new series with no data, Mylar would error out (now will add and just retain 0 issues), FIX: When year was fuzzied, search would still attempt to do a date-check comparison and say everything failed, IMP: Better handling of nzb names when retaining for post-processing comparisons, IMP: Future Upcoming now will use actual shipping date of issue if available in order to see if issue is available, FIX: If annuals were enabled, refreshing a series would put some issues into an Archived status because the actual counts would be off, IMP: When file checking, Alternate Naming would be searched last which resulted in matching incorrectly to the series title or other alternate naming for the given series, now will check the Alternate Naming first for a match, then drop back down to the series name itself otherwise, IMP: Improved Annual detection when integrated with a given series, IMP: Improved the checking of the future Upcoming list for issues marked as Wanted but not available yet and then auto-adding, IMP: Improved upon story arc checking for missing issues / searching for wanted, IMP: Enabling Annuals support now within Configuration GUI, bunch of other things....
2014-07-28 19:28:09 +00:00
import re
2012-09-13 15:27:34 +00:00
import time
import threading
2015-05-13 02:21:22 +00:00
import platform
2014-01-29 05:06:00 +00:00
import urllib , urllib2
2012-09-13 15:27:34 +00:00
from xml . dom . minidom import parseString , Element
import mylar
from mylar import logger , db , cv
2015-11-23 08:25:59 +00:00
from mylar . helpers import multikeysort , replace_all , cleanName , listLibrary
2015-05-13 01:35:40 +00:00
import httplib
2012-09-13 15:27:34 +00:00
mb_lock = threading . Lock ( )
2015-05-13 01:35:40 +00:00
def patch_http_response_read ( func ) :
def inner ( * args ) :
try :
return func ( * args )
except httplib . IncompleteRead , e :
return e . partial
return inner
httplib . HTTPResponse . read = patch_http_response_read ( httplib . HTTPResponse . read )
2012-09-13 15:27:34 +00:00
2015-05-13 02:21:22 +00:00
if platform . python_version ( ) == ' 2.7.6 ' :
httplib . HTTPConnection . _http_vsn = 10
httplib . HTTPConnection . _http_vsn_str = ' HTTP/1.0 '
2015-05-22 08:32:51 +00:00
def pullsearch ( comicapi , comicquery , offset , explicit , type ) :
2014-01-29 05:06:00 +00:00
u_comicquery = urllib . quote ( comicquery . encode ( ' utf-8 ' ) . strip ( ) )
2014-02-26 19:48:50 +00:00
u_comicquery = u_comicquery . replace ( " " , " % 20 " )
IMP: (#1122) Added provider OMGWTFNZBS (results untested), FIX:(#1124) Experimental search results had a 1/9 (or simliar) parts as part of the header, would error out trying to create the nzb due to invalid path, FIX: When importing a series, if Move Files wasn't enabled would error out during the import process, FIX: When on Import page, if selecting Save Changes without Scanning Library would leave pop-up notification up forever, FIX: When searching for series (like A-Force), results would return 16000+ which would look like it locked up due to the excessive number of results - will now better handle such titles, and overall search results are limited to the first 1000 hits, FIX: When clicking on 'Add Series' from the pull-list, if a series was a new series with no issues published would return 0 results from CV even though the series would exist, FIX:(#1120) Story Arcs with special characters would cause error during manual post-processing run, FIX:(#1125) If year is fuzzied for a given series when searching, would error out with invalid date comparisons
2015-09-11 15:49:56 +00:00
if ' - ' in u_comicquery :
#cause titles like A-Force will return 16,000+ results otherwise
u_comicquery = ' % 22 ' + u_comicquery + ' % 22 '
2014-02-26 19:48:50 +00:00
2014-06-04 18:53:33 +00:00
if explicit == ' all ' or explicit == ' loose ' :
IMP: When scanning a folder using the Import a Directory option, upon completion a button to the Import Results Management page will be displayed to allow for easier access / acknowledgement of completion of scan, FIX: Fixed some issues with the search results page displaying too much information, IMP: Improved the ability to see alternate covers within issues when viewing selected issue details via the 'i' icon on the issue details page, IMP: Fixed the problem with issues that had 2015 dates and wouldn't scan in via manual post-processing, IMP: Added duplicate issue detection (beta) - enabled, when post-processing (manually or otherwise), will retain the issue that is bigger in filesize and won't post-process if it's smaller in size, instead of hammering the oldest copy, FIX: bunch of small fixes, and getting groundwork done for other things.
2014-12-18 18:30:57 +00:00
PULLURL = mylar . CVURL + ' search?api_key= ' + str ( comicapi ) + ' &resources= ' + str ( type ) + ' &query= ' + u_comicquery + ' &field_list=id,name,start_year,first_issue,site_detail_url,count_of_issues,image,publisher,deck,description&format=xml&page= ' + str ( offset )
2014-02-26 19:48:50 +00:00
2014-06-02 19:02:28 +00:00
else :
2014-06-04 18:53:33 +00:00
# 02/22/2014 use the volume filter label to get the right results.
FIX: undefined on filter box on startup, IMP: Added Meta-Tagging options on a series / issue basis on comic details screen, IMP: Issue Information is now available per issue and is extracted currently from the cbz file to display (if no cbz is present, the option isn't available), IMP: Failed Download handling is implemented and available in GUI - required to replace existing autoProcessComics.py and ComicRN.py scripts, IMP: Added ability to specify post-processing script instead of ComicRN.py, IMP: Added abilty to edit the issue date for a given issue by simply clicking on it - this will help to avoid dates that have incorrect or 0000-00-00, IMP: Story Arc searching is working (not adding yet), IMP: Added Archived/Ignored options to Upcoming/Wanted tab, IMP: Fixed some alignment and display issues on the Upcoming section, IMP: Added better directory handling for Updating Comic Location when it needs to get changed for all existing series (locmove in config.ini), IMP: Added better handling for unicode characters in series titles when searching / filechecking, IMP: When adding a new series with no data, Mylar would error out (now will add and just retain 0 issues), FIX: When year was fuzzied, search would still attempt to do a date-check comparison and say everything failed, IMP: Better handling of nzb names when retaining for post-processing comparisons, IMP: Future Upcoming now will use actual shipping date of issue if available in order to see if issue is available, FIX: If annuals were enabled, refreshing a series would put some issues into an Archived status because the actual counts would be off, IMP: When file checking, Alternate Naming would be searched last which resulted in matching incorrectly to the series title or other alternate naming for the given series, now will check the Alternate Naming first for a match, then drop back down to the series name itself otherwise, IMP: Improved Annual detection when integrated with a given series, IMP: Improved the checking of the future Upcoming list for issues marked as Wanted but not available yet and then auto-adding, IMP: Improved upon story arc checking for missing issues / searching for wanted, IMP: Enabling Annuals support now within Configuration GUI, bunch of other things....
2014-07-28 19:28:09 +00:00
# add the 's' to the end of type to pluralize the caption (it's needed)
if type == ' story_arc ' :
u_comicquery = re . sub ( " % 20AND % 20 " , " % 20 " , u_comicquery )
IMP: When scanning a folder using the Import a Directory option, upon completion a button to the Import Results Management page will be displayed to allow for easier access / acknowledgement of completion of scan, FIX: Fixed some issues with the search results page displaying too much information, IMP: Improved the ability to see alternate covers within issues when viewing selected issue details via the 'i' icon on the issue details page, IMP: Fixed the problem with issues that had 2015 dates and wouldn't scan in via manual post-processing, IMP: Added duplicate issue detection (beta) - enabled, when post-processing (manually or otherwise), will retain the issue that is bigger in filesize and won't post-process if it's smaller in size, instead of hammering the oldest copy, FIX: bunch of small fixes, and getting groundwork done for other things.
2014-12-18 18:30:57 +00:00
PULLURL = mylar . CVURL + str ( type ) + ' s?api_key= ' + str ( comicapi ) + ' &filter=name: ' + u_comicquery + ' &field_list=id,name,start_year,site_detail_url,count_of_issues,image,publisher,deck,description&format=xml&offset= ' + str ( offset ) # 2012/22/02 - CVAPI flipped back to offset instead of page
2012-09-13 15:27:34 +00:00
#all these imports are standard on most modern python implementations
FIX: One-off Failed Download handling will now work, IMP: Some better error handling when detecting one-off's during post-processing, FIX:(#1143) When series contained a digit preceded by a dash, would incorrectly assume it as a negative issue number, FIX: Improved being able to detect corresponding annuals on CV when refreshing/importing a series when the annuals are new (ie. no data on CV), FIX: Alt_Pull method for weekly pull list retrieval working again, FIX: Fixed nzbid detection for experimental search, IMP: Mass Import button now available on Import Results screen (will attempt to import all series that are in a 'Not Imported' status), IMP: When searching for arc issues using the 'Search for Missing' option, the call is now threaded so it runs in the background, IMP: Changed IssueYear to IssueDate for Story Arc Issues on the arc details page, FIX:(#1156) Typo that caused error when attempting to view cbz comics in the series detail page, FIX:(#1145) Select All option via top checkbox (on series detail page), FIX: Auto-Want feature via weeklypull will better match to titles that contain 'the' and have hypens in differing character positions, FIX:(#1160) Would incorrectly take the length of a decimal placed issue when searching and never complete the issue number cylcing search, FIX:(#1161) When annuals not enabled, and series in watchlist and series annual in pullist, would error out trying to link series to pullist.
2015-11-18 06:32:40 +00:00
#logger.info('MB.PULLURL:' + PULLURL)
2015-11-23 08:25:59 +00:00
#new CV API restriction - one api request / second.
if mylar . CVAPI_RATE is None or mylar . CVAPI_RATE < 2 :
time . sleep ( 2 )
else :
time . sleep ( mylar . CVAPI_RATE )
2012-09-13 15:27:34 +00:00
#download the file:
2013-03-27 07:03:10 +00:00
try :
file = urllib2 . urlopen ( PULLURL )
except urllib2 . HTTPError , err :
2014-01-29 05:06:00 +00:00
logger . error ( ' err : ' + str ( err ) )
2013-03-27 07:03:10 +00:00
logger . error ( " There was a major problem retrieving data from ComicVine - on their end. You ' ll have to try again later most likely. " )
2015-05-22 08:32:51 +00:00
return
2012-09-13 15:27:34 +00:00
#convert to string:
data = file . read ( )
#close file because we dont need it anymore:
file . close ( )
#parse the xml you downloaded
dom = parseString ( data )
return dom
FIX: undefined on filter box on startup, IMP: Added Meta-Tagging options on a series / issue basis on comic details screen, IMP: Issue Information is now available per issue and is extracted currently from the cbz file to display (if no cbz is present, the option isn't available), IMP: Failed Download handling is implemented and available in GUI - required to replace existing autoProcessComics.py and ComicRN.py scripts, IMP: Added ability to specify post-processing script instead of ComicRN.py, IMP: Added abilty to edit the issue date for a given issue by simply clicking on it - this will help to avoid dates that have incorrect or 0000-00-00, IMP: Story Arc searching is working (not adding yet), IMP: Added Archived/Ignored options to Upcoming/Wanted tab, IMP: Fixed some alignment and display issues on the Upcoming section, IMP: Added better directory handling for Updating Comic Location when it needs to get changed for all existing series (locmove in config.ini), IMP: Added better handling for unicode characters in series titles when searching / filechecking, IMP: When adding a new series with no data, Mylar would error out (now will add and just retain 0 issues), FIX: When year was fuzzied, search would still attempt to do a date-check comparison and say everything failed, IMP: Better handling of nzb names when retaining for post-processing comparisons, IMP: Future Upcoming now will use actual shipping date of issue if available in order to see if issue is available, FIX: If annuals were enabled, refreshing a series would put some issues into an Archived status because the actual counts would be off, IMP: When file checking, Alternate Naming would be searched last which resulted in matching incorrectly to the series title or other alternate naming for the given series, now will check the Alternate Naming first for a match, then drop back down to the series name itself otherwise, IMP: Improved Annual detection when integrated with a given series, IMP: Improved the checking of the future Upcoming list for issues marked as Wanted but not available yet and then auto-adding, IMP: Improved upon story arc checking for missing issues / searching for wanted, IMP: Enabling Annuals support now within Configuration GUI, bunch of other things....
2014-07-28 19:28:09 +00:00
def findComic ( name , mode , issue , limityear = None , explicit = None , type = None ) :
2012-09-13 15:27:34 +00:00
2015-05-27 05:40:38 +00:00
#with mb_lock:
2012-09-13 15:27:34 +00:00
comicResults = None
2014-10-08 06:45:51 +00:00
comicLibrary = listLibrary ( )
2015-05-27 05:40:38 +00:00
comiclist = [ ]
arcinfolist = [ ]
2012-09-13 15:27:34 +00:00
chars = set ( ' !?* ' )
if any ( ( c in chars ) for c in name ) :
2015-05-22 08:32:51 +00:00
name = ' " ' + name + ' " '
2013-02-06 19:55:23 +00:00
2015-05-22 08:32:51 +00:00
#print ("limityear: " + str(limityear))
2013-02-06 19:55:23 +00:00
if limityear is None : limityear = ' None '
2015-05-22 08:32:51 +00:00
2014-02-26 19:48:50 +00:00
comicquery = name
2014-01-29 05:06:00 +00:00
#comicquery=name.replace(" ", "%20")
2014-06-02 19:02:28 +00:00
if explicit is None :
2014-06-04 18:53:33 +00:00
#logger.fdebug('explicit is None. Setting to Default mode of ALL search words.')
FIX: undefined on filter box on startup, IMP: Added Meta-Tagging options on a series / issue basis on comic details screen, IMP: Issue Information is now available per issue and is extracted currently from the cbz file to display (if no cbz is present, the option isn't available), IMP: Failed Download handling is implemented and available in GUI - required to replace existing autoProcessComics.py and ComicRN.py scripts, IMP: Added ability to specify post-processing script instead of ComicRN.py, IMP: Added abilty to edit the issue date for a given issue by simply clicking on it - this will help to avoid dates that have incorrect or 0000-00-00, IMP: Story Arc searching is working (not adding yet), IMP: Added Archived/Ignored options to Upcoming/Wanted tab, IMP: Fixed some alignment and display issues on the Upcoming section, IMP: Added better directory handling for Updating Comic Location when it needs to get changed for all existing series (locmove in config.ini), IMP: Added better handling for unicode characters in series titles when searching / filechecking, IMP: When adding a new series with no data, Mylar would error out (now will add and just retain 0 issues), FIX: When year was fuzzied, search would still attempt to do a date-check comparison and say everything failed, IMP: Better handling of nzb names when retaining for post-processing comparisons, IMP: Future Upcoming now will use actual shipping date of issue if available in order to see if issue is available, FIX: If annuals were enabled, refreshing a series would put some issues into an Archived status because the actual counts would be off, IMP: When file checking, Alternate Naming would be searched last which resulted in matching incorrectly to the series title or other alternate naming for the given series, now will check the Alternate Naming first for a match, then drop back down to the series name itself otherwise, IMP: Improved Annual detection when integrated with a given series, IMP: Improved the checking of the future Upcoming list for issues marked as Wanted but not available yet and then auto-adding, IMP: Improved upon story arc checking for missing issues / searching for wanted, IMP: Enabling Annuals support now within Configuration GUI, bunch of other things....
2014-07-28 19:28:09 +00:00
#comicquery=name.replace(" ", " AND ")
2014-06-04 18:53:33 +00:00
explicit = ' all '
#OR
FIX: included version of comictagger should now work with both Windows and *nix based OS' again, IMP: Global Copy/Move option available when performing post-processing, IMP: Added a verbose file-checking option (FOLDER_SCAN_LOG_VERBOSE) - when enabled will log as it currently does during manual post-processing/file-checking runs, when disabled it will not spam the log nearly as much resulting in more readable log files, IMP: Added Verbose debug logging both via startup option(-v), as well as toggle button in Log GUI (from headphones), as well as per-page loading of log file(s) in GUI, FIX: When doing manual post-processing on issues that were in story arcs, will now indicate X story-arc issues were post-processed for better visibility, FIX: Fixed an issue with deleting from the nzblog table when story arc issues were post-processed, IMP: Added WEEKFOLDER_LOC to the config.ini to allow for specification of where the weekly download directories will default to (as opposed to off of ComicLocation root), IMP: Better handling of some special character references in series titles when looking for series on the auto-wanted list, IMP: 32P will now auto-disable provider if logon returns invalid credentials, FIX: When using alt_pull on weekly pull list, xA0 unicode character caused error, FIX: If title had invalid character in filename that was replaced with a character that already existed in the title, would not scan in during file-checking, FIX: When searching for a series (weeklypull-list/add a series), if the title contained 'and' or '&' would return really mixed up results, FIX: When Post-Processing, if filename being processed had special characters (ie. comma) and was different than nzbname, in some cases would fail to find/move issues, IMP: Utilize internal comictagger to convert from cbr/cbz, IMP: Added more checks when post-processing to ensure files are handled correctly, IMP: Added meta-tag reading when importing series/issues - if previously tagged with CT, will reverse look-up the provided IssueID to reference the correct ComicID, IMP: If scanned directory during import contins cvinfo file, use that and force the ComicID to entire directory when importing a series, IMP: Manual meta-tagging issues will no longer create temporary directories and/or create files in the Comic Location root causing problems for some users, FIX: Annuals weren't properly sorted upon loading of comic details page for some series, IMP: Added some extra checks when validating/creating directories, FIX: Fixed a problem when displaying some covers of .cbz files on the comic details page
2016-01-26 07:49:56 +00:00
if ' and ' in comicquery . lower ( ) or ' & ' in comicquery :
logger . fdebug ( ' Enforcing exact naming match due to operator in title (and) ' )
explicit = ' all '
elif explicit == ' loose ' :
2014-06-04 18:53:33 +00:00
logger . fdebug ( ' Changing to loose mode - this will match ANY of the search words ' )
comicquery = name . replace ( " " , " OR " )
elif explicit == ' explicit ' :
logger . fdebug ( ' Changing to explicit mode - this will match explicitly on the EXACT words ' )
2014-06-02 19:02:28 +00:00
comicquery = name . replace ( " " , " AND " )
else :
2014-06-04 18:53:33 +00:00
logger . fdebug ( ' Default search mode - this will match on ALL search words ' )
comicquery = name . replace ( " " , " AND " )
explicit = ' all '
2014-06-09 07:55:05 +00:00
2014-06-09 08:03:10 +00:00
if mylar . COMICVINE_API == ' None ' or mylar . COMICVINE_API is None or mylar . COMICVINE_API == mylar . DEFAULT_CVAPI :
2014-06-09 07:55:05 +00:00
logger . warn ( ' You have not specified your own ComicVine API key - alot of things will be limited. Get your own @ http://api.comicvine.com. ' )
comicapi = mylar . DEFAULT_CVAPI
else :
comicapi = mylar . COMICVINE_API
2012-09-13 15:27:34 +00:00
FIX: undefined on filter box on startup, IMP: Added Meta-Tagging options on a series / issue basis on comic details screen, IMP: Issue Information is now available per issue and is extracted currently from the cbz file to display (if no cbz is present, the option isn't available), IMP: Failed Download handling is implemented and available in GUI - required to replace existing autoProcessComics.py and ComicRN.py scripts, IMP: Added ability to specify post-processing script instead of ComicRN.py, IMP: Added abilty to edit the issue date for a given issue by simply clicking on it - this will help to avoid dates that have incorrect or 0000-00-00, IMP: Story Arc searching is working (not adding yet), IMP: Added Archived/Ignored options to Upcoming/Wanted tab, IMP: Fixed some alignment and display issues on the Upcoming section, IMP: Added better directory handling for Updating Comic Location when it needs to get changed for all existing series (locmove in config.ini), IMP: Added better handling for unicode characters in series titles when searching / filechecking, IMP: When adding a new series with no data, Mylar would error out (now will add and just retain 0 issues), FIX: When year was fuzzied, search would still attempt to do a date-check comparison and say everything failed, IMP: Better handling of nzb names when retaining for post-processing comparisons, IMP: Future Upcoming now will use actual shipping date of issue if available in order to see if issue is available, FIX: If annuals were enabled, refreshing a series would put some issues into an Archived status because the actual counts would be off, IMP: When file checking, Alternate Naming would be searched last which resulted in matching incorrectly to the series title or other alternate naming for the given series, now will check the Alternate Naming first for a match, then drop back down to the series name itself otherwise, IMP: Improved Annual detection when integrated with a given series, IMP: Improved the checking of the future Upcoming list for issues marked as Wanted but not available yet and then auto-adding, IMP: Improved upon story arc checking for missing issues / searching for wanted, IMP: Enabling Annuals support now within Configuration GUI, bunch of other things....
2014-07-28 19:28:09 +00:00
if type is None :
type = ' volume '
2015-05-22 08:32:51 +00:00
#let's find out how many results we get from the query...
searched = pullsearch ( comicapi , comicquery , 0 , explicit , type )
2015-06-22 03:38:10 +00:00
if searched is None :
return False
2012-09-13 15:27:34 +00:00
totalResults = searched . getElementsByTagName ( ' number_of_total_results ' ) [ 0 ] . firstChild . wholeText
2014-02-26 19:48:50 +00:00
logger . fdebug ( " there are " + str ( totalResults ) + " search results... " )
2012-09-13 15:27:34 +00:00
if not totalResults :
return False
IMP: (#1122) Added provider OMGWTFNZBS (results untested), FIX:(#1124) Experimental search results had a 1/9 (or simliar) parts as part of the header, would error out trying to create the nzb due to invalid path, FIX: When importing a series, if Move Files wasn't enabled would error out during the import process, FIX: When on Import page, if selecting Save Changes without Scanning Library would leave pop-up notification up forever, FIX: When searching for series (like A-Force), results would return 16000+ which would look like it locked up due to the excessive number of results - will now better handle such titles, and overall search results are limited to the first 1000 hits, FIX: When clicking on 'Add Series' from the pull-list, if a series was a new series with no issues published would return 0 results from CV even though the series would exist, FIX:(#1120) Story Arcs with special characters would cause error during manual post-processing run, FIX:(#1125) If year is fuzzied for a given series when searching, would error out with invalid date comparisons
2015-09-11 15:49:56 +00:00
if int ( totalResults ) > 1000 :
logger . warn ( ' Search returned more than 1000 hits [ ' + str ( totalResults ) + ' ]. Only displaying first 2000 results - use more specifics or the exact ComicID if required. ' )
totalResults = 1000
2012-09-13 15:27:34 +00:00
countResults = 0
2013-03-29 11:16:55 +00:00
while ( countResults < int ( totalResults ) ) :
2014-02-26 19:48:50 +00:00
#logger.fdebug("querying " + str(countResults))
2012-12-12 22:15:06 +00:00
if countResults > 0 :
2015-05-22 08:32:51 +00:00
#2012/22/02 - CV API flipped back to offset usage instead of page
2014-06-11 18:39:50 +00:00
if explicit == ' all ' or explicit == ' loose ' :
#all / loose uses page for offset
2015-05-22 08:32:51 +00:00
offsetcount = ( countResults / 100 ) + 1
2014-06-11 18:39:50 +00:00
else :
#explicit uses offset
offsetcount = countResults
2015-05-22 08:32:51 +00:00
searched = pullsearch ( comicapi , comicquery , offsetcount , explicit , type )
FIX: undefined on filter box on startup, IMP: Added Meta-Tagging options on a series / issue basis on comic details screen, IMP: Issue Information is now available per issue and is extracted currently from the cbz file to display (if no cbz is present, the option isn't available), IMP: Failed Download handling is implemented and available in GUI - required to replace existing autoProcessComics.py and ComicRN.py scripts, IMP: Added ability to specify post-processing script instead of ComicRN.py, IMP: Added abilty to edit the issue date for a given issue by simply clicking on it - this will help to avoid dates that have incorrect or 0000-00-00, IMP: Story Arc searching is working (not adding yet), IMP: Added Archived/Ignored options to Upcoming/Wanted tab, IMP: Fixed some alignment and display issues on the Upcoming section, IMP: Added better directory handling for Updating Comic Location when it needs to get changed for all existing series (locmove in config.ini), IMP: Added better handling for unicode characters in series titles when searching / filechecking, IMP: When adding a new series with no data, Mylar would error out (now will add and just retain 0 issues), FIX: When year was fuzzied, search would still attempt to do a date-check comparison and say everything failed, IMP: Better handling of nzb names when retaining for post-processing comparisons, IMP: Future Upcoming now will use actual shipping date of issue if available in order to see if issue is available, FIX: If annuals were enabled, refreshing a series would put some issues into an Archived status because the actual counts would be off, IMP: When file checking, Alternate Naming would be searched last which resulted in matching incorrectly to the series title or other alternate naming for the given series, now will check the Alternate Naming first for a match, then drop back down to the series name itself otherwise, IMP: Improved Annual detection when integrated with a given series, IMP: Improved the checking of the future Upcoming list for issues marked as Wanted but not available yet and then auto-adding, IMP: Improved upon story arc checking for missing issues / searching for wanted, IMP: Enabling Annuals support now within Configuration GUI, bunch of other things....
2014-07-28 19:28:09 +00:00
comicResults = searched . getElementsByTagName ( type ) #('volume')
2012-09-13 15:27:34 +00:00
body = ' '
2015-05-22 08:32:51 +00:00
n = 0
2012-09-13 15:27:34 +00:00
if not comicResults :
2015-05-22 08:32:51 +00:00
break
2012-09-13 15:27:34 +00:00
for result in comicResults :
#retrieve the first xml tag (<tag>data</tag>)
#that the parser finds with name tagName:
2014-11-21 17:16:28 +00:00
arclist = [ ]
IMP: (#844)Annuals now deleted from table on a Delete Series command, IMP: (#837)Better volume detection when searching for issues, IMP: (#842)(#808) Added some handling for issue titles when searching (it's not perfect, but it works for most), IMP: (#836)Added .cb7 for filechecking purposes only, IMP: (#830)issue numbers that are -1 can now be renamed, FIX: 'bi-annual' now fixed along with some other improvements for annual handling, IMP: Added version check for autoProcessComics and ComicRN - in order to help warn users of changes and the need to update these files since they are usually outside of the mylar git directory when being used, FIX:(#840) Boxcar2 should be working again, FIX: (#845) If search returned a filename that had no distinction between issue number and issue title and/or extra information, would error out if any of the words contained NOW, IMP: (#823) Added some better detection for NZBGet parameters when using ComicRN.py
2014-10-06 18:10:36 +00:00
if type == ' story_arc ' :
#call cv.py here to find out issue count in story arc
try :
logger . fdebug ( ' story_arc ascension ' )
2015-05-22 08:32:51 +00:00
names = len ( result . getElementsByTagName ( ' name ' ) )
IMP: (#844)Annuals now deleted from table on a Delete Series command, IMP: (#837)Better volume detection when searching for issues, IMP: (#842)(#808) Added some handling for issue titles when searching (it's not perfect, but it works for most), IMP: (#836)Added .cb7 for filechecking purposes only, IMP: (#830)issue numbers that are -1 can now be renamed, FIX: 'bi-annual' now fixed along with some other improvements for annual handling, IMP: Added version check for autoProcessComics and ComicRN - in order to help warn users of changes and the need to update these files since they are usually outside of the mylar git directory when being used, FIX:(#840) Boxcar2 should be working again, FIX: (#845) If search returned a filename that had no distinction between issue number and issue title and/or extra information, would error out if any of the words contained NOW, IMP: (#823) Added some better detection for NZBGet parameters when using ComicRN.py
2014-10-06 18:10:36 +00:00
n = 0
logger . fdebug ( ' length: ' + str ( names ) )
2015-01-04 09:17:49 +00:00
xmlpub = None #set this incase the publisher field isn't populated in the xml
2015-05-22 08:32:51 +00:00
while ( n < names ) :
IMP: (#844)Annuals now deleted from table on a Delete Series command, IMP: (#837)Better volume detection when searching for issues, IMP: (#842)(#808) Added some handling for issue titles when searching (it's not perfect, but it works for most), IMP: (#836)Added .cb7 for filechecking purposes only, IMP: (#830)issue numbers that are -1 can now be renamed, FIX: 'bi-annual' now fixed along with some other improvements for annual handling, IMP: Added version check for autoProcessComics and ComicRN - in order to help warn users of changes and the need to update these files since they are usually outside of the mylar git directory when being used, FIX:(#840) Boxcar2 should be working again, FIX: (#845) If search returned a filename that had no distinction between issue number and issue title and/or extra information, would error out if any of the words contained NOW, IMP: (#823) Added some better detection for NZBGet parameters when using ComicRN.py
2014-10-06 18:10:36 +00:00
logger . fdebug ( result . getElementsByTagName ( ' name ' ) [ n ] . parentNode . nodeName )
if result . getElementsByTagName ( ' name ' ) [ n ] . parentNode . nodeName == ' story_arc ' :
logger . fdebug ( ' yes ' )
try :
xmlTag = result . getElementsByTagName ( ' name ' ) [ n ] . firstChild . wholeText
xmlTag = xmlTag . rstrip ( )
logger . fdebug ( ' name: ' + str ( xmlTag ) )
except :
logger . error ( ' There was a problem retrieving the given data from ComicVine. Ensure that www.comicvine.com is accessible. ' )
return
elif result . getElementsByTagName ( ' name ' ) [ n ] . parentNode . nodeName == ' publisher ' :
logger . fdebug ( ' publisher check. ' )
xmlpub = result . getElementsByTagName ( ' name ' ) [ n ] . firstChild . wholeText
2013-07-01 05:19:15 +00:00
IMP: (#844)Annuals now deleted from table on a Delete Series command, IMP: (#837)Better volume detection when searching for issues, IMP: (#842)(#808) Added some handling for issue titles when searching (it's not perfect, but it works for most), IMP: (#836)Added .cb7 for filechecking purposes only, IMP: (#830)issue numbers that are -1 can now be renamed, FIX: 'bi-annual' now fixed along with some other improvements for annual handling, IMP: Added version check for autoProcessComics and ComicRN - in order to help warn users of changes and the need to update these files since they are usually outside of the mylar git directory when being used, FIX:(#840) Boxcar2 should be working again, FIX: (#845) If search returned a filename that had no distinction between issue number and issue title and/or extra information, would error out if any of the words contained NOW, IMP: (#823) Added some better detection for NZBGet parameters when using ComicRN.py
2014-10-06 18:10:36 +00:00
n + = 1
except :
logger . warn ( ' error retrieving story arc search results. ' )
return
2015-05-22 08:32:51 +00:00
siteurl = len ( result . getElementsByTagName ( ' site_detail_url ' ) )
IMP: (#844)Annuals now deleted from table on a Delete Series command, IMP: (#837)Better volume detection when searching for issues, IMP: (#842)(#808) Added some handling for issue titles when searching (it's not perfect, but it works for most), IMP: (#836)Added .cb7 for filechecking purposes only, IMP: (#830)issue numbers that are -1 can now be renamed, FIX: 'bi-annual' now fixed along with some other improvements for annual handling, IMP: Added version check for autoProcessComics and ComicRN - in order to help warn users of changes and the need to update these files since they are usually outside of the mylar git directory when being used, FIX:(#840) Boxcar2 should be working again, FIX: (#845) If search returned a filename that had no distinction between issue number and issue title and/or extra information, would error out if any of the words contained NOW, IMP: (#823) Added some better detection for NZBGet parameters when using ComicRN.py
2014-10-06 18:10:36 +00:00
s = 0
logger . fdebug ( ' length: ' + str ( names ) )
xmlurl = None
2015-05-22 08:32:51 +00:00
while ( s < siteurl ) :
IMP: (#844)Annuals now deleted from table on a Delete Series command, IMP: (#837)Better volume detection when searching for issues, IMP: (#842)(#808) Added some handling for issue titles when searching (it's not perfect, but it works for most), IMP: (#836)Added .cb7 for filechecking purposes only, IMP: (#830)issue numbers that are -1 can now be renamed, FIX: 'bi-annual' now fixed along with some other improvements for annual handling, IMP: Added version check for autoProcessComics and ComicRN - in order to help warn users of changes and the need to update these files since they are usually outside of the mylar git directory when being used, FIX:(#840) Boxcar2 should be working again, FIX: (#845) If search returned a filename that had no distinction between issue number and issue title and/or extra information, would error out if any of the words contained NOW, IMP: (#823) Added some better detection for NZBGet parameters when using ComicRN.py
2014-10-06 18:10:36 +00:00
logger . fdebug ( result . getElementsByTagName ( ' site_detail_url ' ) [ s ] . parentNode . nodeName )
if result . getElementsByTagName ( ' site_detail_url ' ) [ s ] . parentNode . nodeName == ' story_arc ' :
try :
xmlurl = result . getElementsByTagName ( ' site_detail_url ' ) [ s ] . firstChild . wholeText
except :
logger . error ( ' There was a problem retrieving the given data from ComicVine. Ensure that www.comicvine.com is accessible. ' )
return
s + = 1
xmlid = result . getElementsByTagName ( ' id ' ) [ 0 ] . firstChild . wholeText
if xmlid is not None :
2015-05-27 05:40:38 +00:00
arcinfolist = storyarcinfo ( xmlid )
comiclist . append ( {
' name ' : xmlTag ,
' comicyear ' : arcinfolist [ ' comicyear ' ] ,
' comicid ' : xmlid ,
' cvarcid ' : xmlid ,
' url ' : xmlurl ,
' issues ' : arcinfolist [ ' issues ' ] ,
' comicimage ' : arcinfolist [ ' comicimage ' ] ,
' publisher ' : xmlpub ,
' description ' : arcinfolist [ ' description ' ] ,
' deck ' : arcinfolist [ ' deck ' ] ,
' arclist ' : arcinfolist [ ' arclist ' ] ,
' haveit ' : arcinfolist [ ' haveit ' ]
} )
else :
comiclist . append ( {
' name ' : xmlTag ,
' comicyear ' : arcyear ,
' comicid ' : xmlid ,
' url ' : xmlurl ,
' issues ' : issuecount ,
' comicimage ' : xmlimage ,
' publisher ' : xmlpub ,
' description ' : xmldesc ,
' deck ' : xmldeck ,
' arclist ' : arclist ,
' haveit ' : haveit
} )
logger . fdebug ( ' IssueID \' s that are a part of ' + xmlTag + ' : ' + str ( arclist ) )
IMP: (#844)Annuals now deleted from table on a Delete Series command, IMP: (#837)Better volume detection when searching for issues, IMP: (#842)(#808) Added some handling for issue titles when searching (it's not perfect, but it works for most), IMP: (#836)Added .cb7 for filechecking purposes only, IMP: (#830)issue numbers that are -1 can now be renamed, FIX: 'bi-annual' now fixed along with some other improvements for annual handling, IMP: Added version check for autoProcessComics and ComicRN - in order to help warn users of changes and the need to update these files since they are usually outside of the mylar git directory when being used, FIX:(#840) Boxcar2 should be working again, FIX: (#845) If search returned a filename that had no distinction between issue number and issue title and/or extra information, would error out if any of the words contained NOW, IMP: (#823) Added some better detection for NZBGet parameters when using ComicRN.py
2014-10-06 18:10:36 +00:00
else :
xmlcnt = result . getElementsByTagName ( ' count_of_issues ' ) [ 0 ] . firstChild . wholeText
#here we can determine what called us, and either start gathering all issues or just limited ones.
if issue is not None and str ( issue ) . isdigit ( ) :
#this gets buggered up with NEW/ONGOING series because the db hasn't been updated
#to reflect the proper count. Drop it by 1 to make sure.
limiter = int ( issue ) - 1
else : limiter = 0
2014-11-21 17:16:28 +00:00
#get the first issue # (for auto-magick calcs)
try :
xmlfirst = result . getElementsByTagName ( ' issue_number ' ) [ 0 ] . firstChild . wholeText
if ' \xbd ' in xmlfirst :
xmlfirst = " 1 " #if the first issue is 1/2, just assume 1 for logistics
except :
xmlfirst = ' 1 '
#logger.info('There are : ' + str(xmlcnt) + ' issues in this series.')
#logger.info('The first issue started at # ' + str(xmlfirst))
2015-05-22 08:32:51 +00:00
2014-11-21 17:16:28 +00:00
cnt_numerical = int ( xmlcnt ) + int ( xmlfirst ) # (of issues + start of first issue = numerical range)
#logger.info('The maximum issue number should be roughly # ' + str(cnt_numerical))
#logger.info('The limiter (issue max that we know of) is # ' + str(limiter))
if cnt_numerical > = limiter :
2015-05-22 08:32:51 +00:00
cnl = len ( result . getElementsByTagName ( ' name ' ) )
2014-11-21 17:16:28 +00:00
cl = 0
xmlTag = ' None '
xmlimage = " cache/blankcover.jpg "
while ( cl < cnl ) :
if result . getElementsByTagName ( ' name ' ) [ cl ] . parentNode . nodeName == ' volume ' :
xmlTag = result . getElementsByTagName ( ' name ' ) [ cl ] . firstChild . wholeText
#break
if result . getElementsByTagName ( ' name ' ) [ cl ] . parentNode . nodeName == ' image ' :
xmlimage = result . getElementsByTagName ( ' super_url ' ) [ 0 ] . firstChild . wholeText
cl + = 1
IMP: (#844)Annuals now deleted from table on a Delete Series command, IMP: (#837)Better volume detection when searching for issues, IMP: (#842)(#808) Added some handling for issue titles when searching (it's not perfect, but it works for most), IMP: (#836)Added .cb7 for filechecking purposes only, IMP: (#830)issue numbers that are -1 can now be renamed, FIX: 'bi-annual' now fixed along with some other improvements for annual handling, IMP: Added version check for autoProcessComics and ComicRN - in order to help warn users of changes and the need to update these files since they are usually outside of the mylar git directory when being used, FIX:(#840) Boxcar2 should be working again, FIX: (#845) If search returned a filename that had no distinction between issue number and issue title and/or extra information, would error out if any of the words contained NOW, IMP: (#823) Added some better detection for NZBGet parameters when using ComicRN.py
2014-10-06 18:10:36 +00:00
if ( result . getElementsByTagName ( ' start_year ' ) [ 0 ] . firstChild ) is not None :
xmlYr = result . getElementsByTagName ( ' start_year ' ) [ 0 ] . firstChild . wholeText
else : xmlYr = " 0000 "
2014-11-21 17:16:28 +00:00
#logger.info('name:' + str(xmlTag) + ' -- ' + str(xmlYr))
IMP: (#844)Annuals now deleted from table on a Delete Series command, IMP: (#837)Better volume detection when searching for issues, IMP: (#842)(#808) Added some handling for issue titles when searching (it's not perfect, but it works for most), IMP: (#836)Added .cb7 for filechecking purposes only, IMP: (#830)issue numbers that are -1 can now be renamed, FIX: 'bi-annual' now fixed along with some other improvements for annual handling, IMP: Added version check for autoProcessComics and ComicRN - in order to help warn users of changes and the need to update these files since they are usually outside of the mylar git directory when being used, FIX:(#840) Boxcar2 should be working again, FIX: (#845) If search returned a filename that had no distinction between issue number and issue title and/or extra information, would error out if any of the words contained NOW, IMP: (#823) Added some better detection for NZBGet parameters when using ComicRN.py
2014-10-06 18:10:36 +00:00
if xmlYr in limityear or limityear == ' None ' :
xmlurl = result . getElementsByTagName ( ' site_detail_url ' ) [ 0 ] . firstChild . wholeText
2015-05-22 08:32:51 +00:00
idl = len ( result . getElementsByTagName ( ' id ' ) )
2014-11-21 17:16:28 +00:00
idt = 0
xmlid = None
while ( idt < idl ) :
if result . getElementsByTagName ( ' id ' ) [ idt ] . parentNode . nodeName == ' volume ' :
xmlid = result . getElementsByTagName ( ' id ' ) [ idt ] . firstChild . wholeText
break
idt + = 1
if xmlid is None :
logger . error ( ' Unable to figure out the comicid - skipping this : ' + str ( xmlurl ) )
2015-05-22 08:32:51 +00:00
continue
2014-11-21 17:16:28 +00:00
#logger.info('xmlid: ' + str(xmlid))
IMP: (#844)Annuals now deleted from table on a Delete Series command, IMP: (#837)Better volume detection when searching for issues, IMP: (#842)(#808) Added some handling for issue titles when searching (it's not perfect, but it works for most), IMP: (#836)Added .cb7 for filechecking purposes only, IMP: (#830)issue numbers that are -1 can now be renamed, FIX: 'bi-annual' now fixed along with some other improvements for annual handling, IMP: Added version check for autoProcessComics and ComicRN - in order to help warn users of changes and the need to update these files since they are usually outside of the mylar git directory when being used, FIX:(#840) Boxcar2 should be working again, FIX: (#845) If search returned a filename that had no distinction between issue number and issue title and/or extra information, would error out if any of the words contained NOW, IMP: (#823) Added some better detection for NZBGet parameters when using ComicRN.py
2014-10-06 18:10:36 +00:00
publishers = result . getElementsByTagName ( ' publisher ' )
if len ( publishers ) > 0 :
pubnames = publishers [ 0 ] . getElementsByTagName ( ' name ' )
if len ( pubnames ) > 0 :
xmlpub = pubnames [ 0 ] . firstChild . wholeText
else :
xmlpub = " Unknown "
else :
xmlpub = " Unknown "
try :
IMP: When scanning a folder using the Import a Directory option, upon completion a button to the Import Results Management page will be displayed to allow for easier access / acknowledgement of completion of scan, FIX: Fixed some issues with the search results page displaying too much information, IMP: Improved the ability to see alternate covers within issues when viewing selected issue details via the 'i' icon on the issue details page, IMP: Fixed the problem with issues that had 2015 dates and wouldn't scan in via manual post-processing, IMP: Added duplicate issue detection (beta) - enabled, when post-processing (manually or otherwise), will retain the issue that is bigger in filesize and won't post-process if it's smaller in size, instead of hammering the oldest copy, FIX: bunch of small fixes, and getting groundwork done for other things.
2014-12-18 18:30:57 +00:00
xmldesc = result . getElementsByTagName ( ' description ' ) [ 0 ] . firstChild . wholeText
IMP: (#844)Annuals now deleted from table on a Delete Series command, IMP: (#837)Better volume detection when searching for issues, IMP: (#842)(#808) Added some handling for issue titles when searching (it's not perfect, but it works for most), IMP: (#836)Added .cb7 for filechecking purposes only, IMP: (#830)issue numbers that are -1 can now be renamed, FIX: 'bi-annual' now fixed along with some other improvements for annual handling, IMP: Added version check for autoProcessComics and ComicRN - in order to help warn users of changes and the need to update these files since they are usually outside of the mylar git directory when being used, FIX:(#840) Boxcar2 should be working again, FIX: (#845) If search returned a filename that had no distinction between issue number and issue title and/or extra information, would error out if any of the words contained NOW, IMP: (#823) Added some better detection for NZBGet parameters when using ComicRN.py
2014-10-06 18:10:36 +00:00
except :
xmldesc = " None "
IMP: When scanning a folder using the Import a Directory option, upon completion a button to the Import Results Management page will be displayed to allow for easier access / acknowledgement of completion of scan, FIX: Fixed some issues with the search results page displaying too much information, IMP: Improved the ability to see alternate covers within issues when viewing selected issue details via the 'i' icon on the issue details page, IMP: Fixed the problem with issues that had 2015 dates and wouldn't scan in via manual post-processing, IMP: Added duplicate issue detection (beta) - enabled, when post-processing (manually or otherwise), will retain the issue that is bigger in filesize and won't post-process if it's smaller in size, instead of hammering the oldest copy, FIX: bunch of small fixes, and getting groundwork done for other things.
2014-12-18 18:30:57 +00:00
#this is needed to display brief synopsis for each series on search results page.
try :
xmldeck = result . getElementsByTagName ( ' deck ' ) [ 0 ] . firstChild . wholeText
except :
xmldeck = " None "
2014-10-08 06:45:51 +00:00
if xmlid in comicLibrary :
2014-10-08 18:17:44 +00:00
haveit = comicLibrary [ xmlid ]
2014-10-08 06:45:51 +00:00
else :
haveit = " No "
IMP: (#844)Annuals now deleted from table on a Delete Series command, IMP: (#837)Better volume detection when searching for issues, IMP: (#842)(#808) Added some handling for issue titles when searching (it's not perfect, but it works for most), IMP: (#836)Added .cb7 for filechecking purposes only, IMP: (#830)issue numbers that are -1 can now be renamed, FIX: 'bi-annual' now fixed along with some other improvements for annual handling, IMP: Added version check for autoProcessComics and ComicRN - in order to help warn users of changes and the need to update these files since they are usually outside of the mylar git directory when being used, FIX:(#840) Boxcar2 should be working again, FIX: (#845) If search returned a filename that had no distinction between issue number and issue title and/or extra information, would error out if any of the words contained NOW, IMP: (#823) Added some better detection for NZBGet parameters when using ComicRN.py
2014-10-06 18:10:36 +00:00
comiclist . append ( {
2014-11-21 17:16:28 +00:00
' name ' : xmlTag ,
' comicyear ' : xmlYr ,
' comicid ' : xmlid ,
' url ' : xmlurl ,
' issues ' : xmlcnt ,
' comicimage ' : xmlimage ,
IMP: (#844)Annuals now deleted from table on a Delete Series command, IMP: (#837)Better volume detection when searching for issues, IMP: (#842)(#808) Added some handling for issue titles when searching (it's not perfect, but it works for most), IMP: (#836)Added .cb7 for filechecking purposes only, IMP: (#830)issue numbers that are -1 can now be renamed, FIX: 'bi-annual' now fixed along with some other improvements for annual handling, IMP: Added version check for autoProcessComics and ComicRN - in order to help warn users of changes and the need to update these files since they are usually outside of the mylar git directory when being used, FIX:(#840) Boxcar2 should be working again, FIX: (#845) If search returned a filename that had no distinction between issue number and issue title and/or extra information, would error out if any of the words contained NOW, IMP: (#823) Added some better detection for NZBGet parameters when using ComicRN.py
2014-10-06 18:10:36 +00:00
' publisher ' : xmlpub ,
2014-10-08 06:45:51 +00:00
' description ' : xmldesc ,
IMP: When scanning a folder using the Import a Directory option, upon completion a button to the Import Results Management page will be displayed to allow for easier access / acknowledgement of completion of scan, FIX: Fixed some issues with the search results page displaying too much information, IMP: Improved the ability to see alternate covers within issues when viewing selected issue details via the 'i' icon on the issue details page, IMP: Fixed the problem with issues that had 2015 dates and wouldn't scan in via manual post-processing, IMP: Added duplicate issue detection (beta) - enabled, when post-processing (manually or otherwise), will retain the issue that is bigger in filesize and won't post-process if it's smaller in size, instead of hammering the oldest copy, FIX: bunch of small fixes, and getting groundwork done for other things.
2014-12-18 18:30:57 +00:00
' deck ' : xmldeck ,
2014-11-21 17:16:28 +00:00
' haveit ' : haveit
IMP: (#844)Annuals now deleted from table on a Delete Series command, IMP: (#837)Better volume detection when searching for issues, IMP: (#842)(#808) Added some handling for issue titles when searching (it's not perfect, but it works for most), IMP: (#836)Added .cb7 for filechecking purposes only, IMP: (#830)issue numbers that are -1 can now be renamed, FIX: 'bi-annual' now fixed along with some other improvements for annual handling, IMP: Added version check for autoProcessComics and ComicRN - in order to help warn users of changes and the need to update these files since they are usually outside of the mylar git directory when being used, FIX:(#840) Boxcar2 should be working again, FIX: (#845) If search returned a filename that had no distinction between issue number and issue title and/or extra information, would error out if any of the words contained NOW, IMP: (#823) Added some better detection for NZBGet parameters when using ComicRN.py
2014-10-06 18:10:36 +00:00
} )
2014-11-21 17:16:28 +00:00
#logger.fdebug('year: ' + str(xmlYr) + ' - constraint met: ' + str(xmlTag) + '[' + str(xmlYr) + '] --- 4050-' + str(xmlid))
IMP: (#844)Annuals now deleted from table on a Delete Series command, IMP: (#837)Better volume detection when searching for issues, IMP: (#842)(#808) Added some handling for issue titles when searching (it's not perfect, but it works for most), IMP: (#836)Added .cb7 for filechecking purposes only, IMP: (#830)issue numbers that are -1 can now be renamed, FIX: 'bi-annual' now fixed along with some other improvements for annual handling, IMP: Added version check for autoProcessComics and ComicRN - in order to help warn users of changes and the need to update these files since they are usually outside of the mylar git directory when being used, FIX:(#840) Boxcar2 should be working again, FIX: (#845) If search returned a filename that had no distinction between issue number and issue title and/or extra information, would error out if any of the words contained NOW, IMP: (#823) Added some better detection for NZBGet parameters when using ComicRN.py
2014-10-06 18:10:36 +00:00
else :
2015-01-16 19:40:08 +00:00
pass
2015-05-22 08:32:51 +00:00
#logger.fdebug('year: ' + str(xmlYr) + ' - contraint not met. Has to be within ' + str(limityear))
n + = 1
2013-03-27 07:03:10 +00:00
#search results are limited to 100 and by pagination now...let's account for this.
countResults = countResults + 100
2015-05-22 08:32:51 +00:00
2014-06-04 18:53:33 +00:00
return comiclist , explicit
2015-05-27 05:40:38 +00:00
def storyarcinfo ( xmlid ) :
comicLibrary = listLibrary ( )
arcinfo = { }
if mylar . COMICVINE_API == ' None ' or mylar . COMICVINE_API is None or mylar . COMICVINE_API == mylar . DEFAULT_CVAPI :
logger . warn ( ' You have not specified your own ComicVine API key - alot of things will be limited. Get your own @ http://api.comicvine.com. ' )
comicapi = mylar . DEFAULT_CVAPI
else :
comicapi = mylar . COMICVINE_API
#respawn to the exact id for the story arc and count the # of issues present.
ARCPULL_URL = mylar . CVURL + ' story_arc/4045- ' + str ( xmlid ) + ' /?api_key= ' + str ( comicapi ) + ' &field_list=issues,name,first_appeared_in_issue,deck,image&format=xml&offset=0 '
logger . fdebug ( ' arcpull_url: ' + str ( ARCPULL_URL ) )
2015-11-23 08:25:59 +00:00
#new CV API restriction - one api request / second.
if mylar . CVAPI_RATE is None or mylar . CVAPI_RATE < 2 :
time . sleep ( 2 )
else :
time . sleep ( mylar . CVAPI_RATE )
2015-05-27 05:40:38 +00:00
try :
file = urllib2 . urlopen ( ARCPULL_URL )
except urllib2 . HTTPError , err :
logger . error ( ' err : ' + str ( err ) )
logger . error ( ' There was a major problem retrieving data from ComicVine - on their end. ' )
return
arcdata = file . read ( )
file . close ( )
arcdom = parseString ( arcdata )
try :
logger . fdebug ( ' story_arc ascension ' )
issuecount = len ( arcdom . getElementsByTagName ( ' issue ' ) )
issuedom = arcdom . getElementsByTagName ( ' issue ' )
isc = 0
arclist = ' '
2015-08-11 21:56:22 +00:00
ordernum = 1
2015-05-27 05:40:38 +00:00
for isd in issuedom :
zeline = isd . getElementsByTagName ( ' id ' )
isdlen = len ( zeline )
isb = 0
while ( isb < isdlen ) :
if isc == 0 :
2015-08-11 21:56:22 +00:00
arclist = str ( zeline [ isb ] . firstChild . wholeText ) . strip ( ) + ' , ' + str ( ordernum )
2015-05-27 05:40:38 +00:00
else :
2015-08-11 21:56:22 +00:00
arclist + = ' | ' + str ( zeline [ isb ] . firstChild . wholeText ) . strip ( ) + ' , ' + str ( ordernum )
ordernum + = 1
2015-05-27 05:40:38 +00:00
isb + = 1
isc + = 1
except :
logger . fdebug ( ' unable to retrive issue count - nullifying value. ' )
issuecount = 0
try :
firstid = None
arcyear = None
fid = len ( arcdom . getElementsByTagName ( ' id ' ) )
fi = 0
while ( fi < fid ) :
if arcdom . getElementsByTagName ( ' id ' ) [ fi ] . parentNode . nodeName == ' first_appeared_in_issue ' :
if not arcdom . getElementsByTagName ( ' id ' ) [ fi ] . firstChild . wholeText == xmlid :
logger . fdebug ( ' hit it. ' )
firstid = arcdom . getElementsByTagName ( ' id ' ) [ fi ] . firstChild . wholeText
break # - dont' break out here as we want to gather ALL the issue ID's since it's here
fi + = 1
logger . fdebug ( ' firstid: ' + str ( firstid ) )
if firstid is not None :
firstdom = cv . pulldetails ( comicid = None , type = ' firstissue ' , issueid = firstid )
logger . fdebug ( ' success ' )
arcyear = cv . GetFirstIssue ( firstid , firstdom )
except :
logger . fdebug ( ' Unable to retrieve first issue details. Not caclulating at this time. ' )
if ( arcdom . getElementsByTagName ( ' image ' ) [ 0 ] . childNodes [ 0 ] . nodeValue ) is None :
xmlimage = arcdom . getElementsByTagName ( ' super_url ' ) [ 0 ] . firstChild . wholeText
else :
xmlimage = " cache/blankcover.jpg "
try :
xmldesc = arcdom . getElementsByTagName ( ' desc ' ) [ 0 ] . firstChild . wholeText
except :
xmldesc = " None "
try :
xmldeck = arcdom . getElementsByTagName ( ' deck ' ) [ 0 ] . firstChild . wholeText
except :
xmldeck = " None "
if xmlid in comicLibrary :
haveit = comicLibrary [ xmlid ]
else :
haveit = " No "
arcinfo = {
#'name': xmlTag, #theese four are passed into it only when it's a new add
#'url': xmlurl, #needs to be modified for refreshing to work completely.
#'publisher': xmlpub,
' comicyear ' : arcyear ,
' comicid ' : xmlid ,
' issues ' : issuecount ,
' comicimage ' : xmlimage ,
' description ' : xmldesc ,
' deck ' : xmldeck ,
' arclist ' : arclist ,
' haveit ' : haveit
}
return arcinfo
FIX: One-off Failed Download handling will now work, IMP: Some better error handling when detecting one-off's during post-processing, FIX:(#1143) When series contained a digit preceded by a dash, would incorrectly assume it as a negative issue number, FIX: Improved being able to detect corresponding annuals on CV when refreshing/importing a series when the annuals are new (ie. no data on CV), FIX: Alt_Pull method for weekly pull list retrieval working again, FIX: Fixed nzbid detection for experimental search, IMP: Mass Import button now available on Import Results screen (will attempt to import all series that are in a 'Not Imported' status), IMP: When searching for arc issues using the 'Search for Missing' option, the call is now threaded so it runs in the background, IMP: Changed IssueYear to IssueDate for Story Arc Issues on the arc details page, FIX:(#1156) Typo that caused error when attempting to view cbz comics in the series detail page, FIX:(#1145) Select All option via top checkbox (on series detail page), FIX: Auto-Want feature via weeklypull will better match to titles that contain 'the' and have hypens in differing character positions, FIX:(#1160) Would incorrectly take the length of a decimal placed issue when searching and never complete the issue number cylcing search, FIX:(#1161) When annuals not enabled, and series in watchlist and series annual in pullist, would error out trying to link series to pullist.
2015-11-18 06:32:40 +00:00