In simple words, when you make a request in Jackett, the results are saved in memory (cache). The next request will return results form the cache improving response time and making fewer requests to the sites.
* We assume all indexers/sites are stateless, the same request return the same response. If you change the search term, categories or something in the query Jackett has to make a live request to the indexer.
* There are some situations when we don't want to use the cache:
** When we are testing the indexers => if query.IsTest results are not cached
** When the user updates the configuration of one indexer => We call CleanIndexerCache to remove cached results before testing the configuration
** When there is some error/exception in the indexer => The results are not cached so we can retry in the next request
* We want to limit the memory usage, so we try to remove elements from cache ASAP:
** Each indexer can have a maximum number of results in memory. If the limit is exceeded we remove old results
** Cached results expire after some time
* Users can configure the cache or even disable it
* When category ids in the indexer are "strings" we create a unique hash to make it compatible with Torznab
* This PR also fix several issues related to custom cats
* Core: Categories are stored in a real tree
* Sorting: First Torznab categories sorted by Id and then custom cats sorted by Name
* Filtering: Results with child category are not removed when searching by parent category. Details in #8049
* Jacket UI: Add parent category when at least one child category exists
* Torznab (caps): Remove non existent children categories. Remove duplicated categories. Details in #10006
core: allow renaming & merging of trackers with config (part 1 #8355) (#8608)
* rename site field to id in yaml indexers
* add id field in c# indexers
* use id field instead of class name in c#
* core: fix cookie parsing (part 2)
After fixing cookie storage in #8133 I noticed that I still have a lot of '.json.bak' files in the Jackett configuration folder. After deleting them they were created again in each request. The cause was we were parsing bad the cookies with '=' character in the value. Most Cloudflare cookies include if so we were sending bad cookies and solving the callenge in each request.
This PR should increase performance in several ways: we are not solving the challenge again (it takes time), we are not making extra requests and we are not updating the Jackett configuration in each request (both files '.json' and '.json.bak').
Tested with the client HttpWebClient2NetCore only. Please do some tests with the site 1337x.
Diego Heras