Commit Graph

82 Commits

Author SHA1 Message Date
chibidev 057df28d1b Fixing a null exception when searching AnimeTosho
AnimeTosho failed to populate the Origin field of the results when
searching thereby creating an issue upon creating the proxy link. This
was mainly because an IEnumerable can contain a deferred LINQ query as
well (e.g. in the case of AnimeTosho a Select) that will re-execute
every single time. So when iterating over IEnumerables we cannot really
pose any assumption on what we are dealing with, so either explicitly
force an execution (e.g. via ToList), or use LINQ queries as well. Since
the second is probably more performant, let's stick with that.
2017-07-29 13:20:10 +02:00
chibidev 0d25ed2921 #1575 - Fix invalid URL handling 2017-07-19 19:24:32 +02:00
chibidev 17f544be36 Bugfix/1532 1539 fix manual search circular reference (#1543)
* Line endings...

* Encoding Encoding in a reasonable way

Sadly Encoding contains a self-reference somewhere, which makes it
really hard for the json serializer to automatically encode it...
Probably there's no value in sending it over especially since no one is
using it, however just for the sake of the argument, let's just
serialize it in a reasonable way. Maybe someday there will be someone
expecting this. Or we clearly separate DTOs and models...

- Fixes Jackett/Jackett#1532
- Fixes Jackett/Jackett#1539

* Seriously... Please port this to dotnet Core so that I can use something other than @drunkvs on my MacBook

* Fix autofac registration after merge

* "Implement" new function of interface in test project
2017-07-11 22:32:56 +02:00
chibidev 9e3076dde6 I have a feeling I'm breaking the world and bringing the apocalypse
It's quite hard to encapsulate something this large. This refactor
contains multiple attacks on the current architecture and is changing
things that were probably created quite a while back then. Luckily this
was done in increments so it mustn't be that impossible to recall what
has been done. I just need to relax my memory a little bit.

So the basic idea was quite simple. Let's distingush metas and normal
indexers a little bit more. Both of them were originating from
BaseIndexer, however very little of the functionality was actually
shared between them. Actually quite a few things made it even harder to
implement a different kind of indexer, especially for a newcomer for
both Jackett and C#.

Then in order to further reduce whatever was encapsulated in
any kind of, a couple things had to be changed. Like CardigannIndexer,
which probably had quite a mindshift change. IndexerManager and the
configuration management were also encapsulated and refactored, and now
I have a feeling that although the code could be improved, at least the
responsibilities of services and what they actually do is now clearer.

Anyhow, it would be safe to assume that I will not be able to go
step-by-step and define everything that has been changed. I'm sorry.
2017-07-11 21:53:46 +02:00
chibidev 4a0d2dcc57 Feature/metaindexer torrent download improvement (#1517)
* Line endings...

* Improve download handling of torrents in metas

Until now, downloads were handled by metas, however they had to "guess"
which indexer the result was originating from and resolve to that
indexer. While this has been working without an issue, it really
shouldn't be considered stable.

Therefore indexers now link themselves to the results they provide. In
order to keep my sanity and automate this as much as possible, I had to
slightly modify the interface (sorry, everyone).
2017-07-03 07:15:47 +02:00
chibidev 11fd2db5a5 Bugfix/cannot add cardigann indexers (#1511)
* Line endings...

* Fix invalid ID for cardigann indexers (refactoring regression - my bad)
2017-06-29 20:46:36 +02:00
chibidev 345602926e Feature/cleaning up aggregate indexer fallback (#1507)
* Line endings...

* Refactoring how MetaIndexers handle fallbacks

Originally this modification was part of a much larger refactoring,
however for the sake of reviewability I split it into smaller chunks.
Sadly it is still quite large.
I wanted to split it even more, however after a certain point there was
really no value in creating smaller chunks. The biggest part of this
modification would be still huge.

So all in all, there're 3 aspects of this modification
- It modifies BaseIndexer so that it now implements IIndexer (will be
very useful later on)
- Resolving most of the warnings currently in Jackett (the only ones
remaining are related to Autofac, however if I could I would just burn
Autofac altogether rather than fix the warnings. Will open discussion on
this.)
- Biggest part: refactoring how MetaIndexers handle fallbacks and how
they provide the final result set

MetaIndexers now accept any kind of fallback and filtering mechanism
that implements the necessary interface, so that in the future IMDB
fallback and filtering won't be the only one. I know there are not a lot
of unit tests around Jackett at the moment, however this renders the
class much more unittestable as well.

* Autofac started complaining... I don't understand...
2017-06-28 07:31:38 +02:00
chibidev 3b4eceed87 Feature/improved aggregate results (#1432)
* Line endings...

* Add fallback query for meta indexers

In cases where multiple indexers are configured under one
metaindexer if any of them supports IMDB search the meta
will support IMDB search as well. However the actual query will
then only be performed by those supporting IMDB search, because
others refuse it (see CanHandleQuery implementation).
- This adds support of a fallback mechanism for other indexers
- Adds first implementation of result improvement (necessary for
  fallback queries as they might produce irrelevant results)
- Some minor fixes encountered while debugging/coding

Known issue:
- Configuring nCore and IsoHunt together will render results
  from nCore unusuable. Don't know why.
2017-06-03 15:04:51 +02:00
chibidev 27d4f2108e Feature/aggregate performance improvement (#1349)
* Checking capabilities before executing a query

* Introduce base class for meta indexers

* Build fix - I seriously do not know how I missed that

* Moving things to the appropriate place

* Simplifying things as much as possible and moving once again

* Build fix?
2017-05-14 18:55:36 +02:00
kaso17 6b35da8ad0 SaveConfig() after IsConfigured 2017-05-06 19:39:02 +02:00
chibidev b942689dad Fixing a bunch of issues with the newly added AggregateIndexer (#1324)
This patch improves the general stability of the aggregate.
- Result is generated even if some trackers failed to answer
  or some other issue were encountered
- Fixes Jackett/Jackett#1316
2017-05-04 21:46:39 +02:00
flightlevel f2def1f615 Normalise Line endings (#1284)
* Add .gitattributes

* Normalise line endings
2017-04-15 18:45:10 +10:00
kaso17 b226304214 BaseIndexer: use referer for Downloads 2017-03-14 13:30:03 +01:00
kaso17 b1453157bd allow TrackerCategory == null 2017-03-10 09:34:43 +01:00
kaso17 8e9f463bb5 Throw exception 2017-03-07 12:27:05 +01:00
kaso17 d18cc7575c improve error logging 2017-03-06 13:59:39 +01:00
kaso17 cdd9411d5d Add support for self signed certificates 2017-02-28 20:05:57 +01:00
kaso17 87d3fd9443 Add initial support for multiple categories and 1:1 category mapping 2017-02-21 13:07:54 +01:00
kaso17 d6bcfa7e2e RequestBytesWithCookies: add support for headers 2017-02-15 20:47:06 +01:00
kaso17 ef75684c0e allow cookies with empty value 2017-02-07 17:05:29 +01:00
kaso17 18908be38f Add HD-Only 2017-02-02 14:53:28 +01:00
kaso17 e088f1e960 Add GetAllTrackerCategories() 2017-02-02 12:58:23 +01:00
kaso17 50c3a5fb3a Add support for tracker type (private/public/semi-private) 2017-01-27 16:57:32 +01:00
kaso17 f1fa8c131a Cardigann: add Keywordsfilters and before download feature 2017-01-27 12:14:49 +01:00
kaso17 7c0d619f2f Cardigann: set referer on captcha request 2017-01-17 12:32:11 +01:00
kaso17 af1aa408c9 Add support for movie-search 2017-01-16 17:23:19 +01:00
kaso17 175bea9269 Automatically load the backup config if the primary one is corrupted 2017-01-05 18:05:14 +01:00
kaso17 03ab411d68 Cardigann: make download method configurable 2017-01-03 13:19:23 +01:00
kaso17 f99e104755 Make site link configurable for all indexers 2017-01-02 21:39:28 +01:00
kaso17 5fc7fca5c6 CategoryMapping: Add support for the tracker category description 2016-12-17 16:42:50 +01:00
kaso17 9a66289a26 Add more debug output on Download error 2016-12-15 06:56:41 +01:00
kaso17 d499c291b9 WebRequest: make encoding configurable (#816) 2016-12-06 14:56:47 +01:00
kaso17 7835da2ea8 Indexer Download: UrlEncode some special chars 2016-12-06 11:13:18 +01:00
kaso17 bb6f56bfa6 Add LastError to indexer API (#802) 2016-12-05 11:43:07 +01:00
kaso17 e56ff204f8 Add language field to indexers (#788) 2016-12-02 18:58:10 +01:00
kaso17 7cfdaf4b41 Make cookie merging more robust (#669) 2016-11-10 08:37:52 +01:00
kaso17 809362e08c Dump DL response if there's an error (#653) 2016-11-06 18:14:15 +01:00
kaso17 21cffe2d35 Add basic support for Cardigann definitions (#571)
* Add basic support for Cardigann definitions

* Add HDME definition

* Fix tests

* support split with negative indexes

* allow FromTimeAgo formats without spaces betwen value and unit

Example: 2h 3m

* Add basic support for Cardigann definitions

* Add HDME definition

* Fix tests

* support split with negative indexes

* allow FromTimeAgo formats without spaces betwen value and unit

Example: 2h 3m
2016-10-27 18:30:03 +11:00
kaso17 7eaff55955 Support CloudFlare challenges with mono/libcurl (#538)
* Add CloudFlare support for the libcurl WebClient

* Save config if cookies are updated

If the cookieheader/config isn't updated with e.g. the cf_clearance cookie jackett has to recompute the challenge on every request.
2016-10-07 08:50:15 +02:00
Azerelat 4f5d7a3d54 Make category mapping a little less confusing #255 2016-02-24 18:40:16 +00:00
Azerelat 2901cceea9 Fix cache items not expiring and change redirect to a temp one as we can now change the base path 2016-01-17 13:19:18 +00:00
twistedpear a50ab9c49b Improved cookie accumulation to reduce roundtrips to the server. 2015-12-28 11:16:53 -05:00
garreth.jeremiah@gmail.com f4129dc4a0 Added simple (non-auth) proxy support including processing the second set of headers (server) vs first (proxy). New command line option (-j 127.0.0.1:8888) to set the proxy and port. unfortunatly both -p and -x were already taken
extended refresh header handling to libcurl and safecurl
also some minor tweaks needed to have Curl 'behave' with certain servers by adding accept-language header to Curl
Added ability/template to allow the user to select their own SiteURL (see BitSoup implementation) with a minor update to ConfigurationData class to prevent the "display text" from disappearing on errors.
XSpeeds indexer - updated to handle a case where the returned XML occasionally is null terminated resulting in XML parsing exceptions
BitSoup indexer - added user customizable url selection, including some url validation
BaseIndexer - cleaned up some of my earlier implementation for accumulating cookies.
2015-12-24 22:26:39 -05:00
garreth.jeremiah@gmail.com b5ff430e2d RequestLoginAndFollowRedirect - added "accumulateCookies" which adds cookies from redirection responses. This was needed for xspeeds processing of cloudflare redirection
FollowIfRedirect - during redirects this can keep the cookies
2015-12-18 11:55:53 -05:00
flightlevel eb6e178c2f Fix Missing Url Redirect Override 2015-10-04 12:26:25 +01:00
KZ 982300ec6e Complete BTNv1 and Fix Cloudfare cookies 2015-09-30 21:21:43 +01:00
KZ 53e680ea87 Complete Shazbat, fix category picker on web ui and start btn 2015-09-24 19:55:15 +01:00
KZ 9437765697 Implement Shazbat #151 2015-09-22 21:29:44 +01:00
KZ 66b8448186 Torrent download bugfix 2015-09-21 20:54:24 +01:00
KZ e8bb1680f3 #181 Fix links are not cleaned before being sent to the webui. Fix failed requests not being forwarded to the caller. Add size display to web ui. 2015-09-21 18:48:53 +01:00