Updated Implementing a Torznab indexer (markdown)

Taloth 2016-08-06 16:53:18 +02:00
parent 3f4605f08d
commit 91e4699b45
1 changed files with 44 additions and 36 deletions

@ -1,67 +1,73 @@
##Introduction##
Since version 2.0.0.3000 Sonarr supports a Newznab-like api dubbed **Torznab**.
This api offers a standardized recent/search api for both tv and movies (of which Sonarr only uses tv).
Sonarr supports a Newznab-like api dubbed **Torznab**. This api offers a standardized recent/search api for both tv and movies (of which Sonarr only uses tv).
The Newznab api specification is built around a simple rss feed with filtering and paging capabilities. It was designed for usenet primarily, but it can be easily adapted for torrents.
Obviously, Torznab is only implemented on a limited number of torrent trackers, but that will change and when it does Sonarr will be ready for it.
The purpose of this wiki page is to describe the differences with Newznab and some recommendations to implement a custom proxy/indexer.
##Introduction for Private Trackers##
Many torrent trackers thrive on the community and may have rules in place that mandate site visits, karma, votes, comments and all. Having an API doesn't need to be detrimental to this goal.
A site can integrate the api into the existing karma/bonus system to keep users engaged in the community. There are quite a few creative ways of doing this.
**Note:** If your torrent tracker is based on a generic tracker software, please coordinate to get the api implemented in the generic base code so everyone can benefit and to avoid double work.
It's also imperative to think about how to integrate this in your existing karma/bonus system to keep your users engaged in your community.
##Differences with Newznab##
Torznab extends on Newznab on a couple of points to make it more useful. However, it is in principle backward compatible.
###Torznab queries###
###Newznab/Torznab queries###
Newznab defines the query params `q,rid,season,ep` for `t=tvsearch` and `q` for `t=search`, however not all sites support tvrage ids, and therefore would have to be queries with the q= param.
Torznab implementations should return the supported query params in their `t=caps` response. (more on `caps` later)
Implementations should also take care in returning appropriate responses when a query includes an unsupported newznab parameter, for example, if rid is not supported, return an empty result set.
This is to remain backward compatibility with clients that do not query the caps.
Originally Newznab defined the query params `q,rid,season,ep` for `t=tvsearch` and `q` for `t=search`, however not all sites support tvrage ids, and therefore would have to be queries with the q= param.
In later versions, Newznab returns a list of the supported query params in the `t=caps` response via a `supportedParams` attribute.
~~Newznab does not define `tvdbid=` and `imdbid=`, but implementors are recommended to support these if possible.~~
The latest Newznab version support `tvdbid=` and `tvmazeid=` as well. That newznab version also uses the `supportedParams` mentioned later, and supports specifying multiple id parameters simultaneously (of which only one needs to match).
Sites should specify the parameters they support using `supportedParams`, allowing clients to perform the most optimal query, limiting the number of api calls and reducing database load.
Clients should default `supportedParams` to `q,rid,season,ep` if a site doesn't provide it, thus remaining backward compatible. Clients should also query by `*id` as much as possible to avoid database keyword searches.
Multiple `*id` query params can be specified simultaneously (of which only one needs to match, so a `OR` operation on the db side).
```
t=tvsearch:
Newznab query params: q,rid,season,ep
Torznab query params: tvdbid
Default query params: q,rid,season,ep
Known query params : q,rid,tvdbid,tvmazeid,season,ep
t=search:
Newznab query params: q
Default query params: q
Known query params : q
t=movie:
Newznab query params: q
Torznab query params: imdbid
Default query params: q
Known query params : q,imdbid
```
Examples:
```
?t=tvsearch&cat=..,..&rid=..&season=1&episode=1
?t=tvsearch&cat=..,..&q=..&season=1&episode=1
/api?t=tvsearch&cat=..,..&rid=..&season=1&ep=1
/api?t=tvsearch&cat=..,..&q=..&season=1&ep=1
?t=tvsearch&cat=..,..&rid=..&season=2016&ep=12/20
?t=tvsearch&cat=..,..&q=..&season=2016&ep=12/20
/api?t=tvsearch&cat=..,..&rid=..&season=2016&ep=12/20
/api?t=tvsearch&cat=..,..&q=..&season=2016&ep=12/20
?t=search&cat=..,..&q=Anime+Title+123
?t=search&cat=..,..&q=Fantastic+Series+With+Episode+Title+For+Special
/api?t=search&cat=..,..&q=Anime+Title+123
/api?t=search&cat=..,..&q=Fantastic+Series+With+Episode+Title+For+Special
```
###Torznab caps endpoint###
`/api?t=tvsearch&cat=..,..&rid=143&tvdbid=1232&season=1&ep=2` should perform a database query like `SELECT * FROM Releases WHERE (rid=143 OR tvdbid=1232) AND season=1 AND ep=2`.
So `q AND (rid OR tvdbid OR tvmazeid) AND season AND ep`.
###Caps endpoint###
The mode `t=caps` is supposed to return the capabilities and categories of the newznab/torznab indexer.
Implementing this mode is mandatory.
The caps response for torznab is mostly the same as newznab, except for a few points:
The caps response for torznab is mostly the same as newznab. However, newznab specifies a list of default categories.
It's recommended to keep those categories intact, Client applications can then query one of the defaults, instea
The `searching` element is normally used to specify what kind of modes (`tvsearch`,`search`,`movie`) the api supports. But torznab adds an optional `supportedParams` attribute allowing the api to specify which query params are supported.
If the `supportedParams` attribute is not found, then it defaults to the original newznab params as specified earlier.
Many torrent sites have their own categories starting at '1', it's recommended NOT to map these 1 to 1 on newznab categories but instead add 100000 to the number.
That way it falls in the site specific range and does not interfere with the original newznab categories.
Many torrent sites have their own rss categories starting at '1', it's recommended not to map these 1 to 1 on newznab categories but instead add 100000 to the number. That way it falls in the site specific range and does not interfere with the original newznab categories.
The api should then alias all site-specific categories in one of the original newznab category.
For example, if you have category 10 as Blu-Ray 1080p TV, then releases in that category would use the torznab api cats 100010,5040,5000 (Site-specific, TV/HD, TV).
Any query that includes any of those numbers in the `cat=` parameter would return those releases as results.
Of course it's also allowed to omit the site-specific categories entirely, and map only on newznab categories.
A full list of default categories can be found in the References at the bottom of this page.
Example `t=caps` response:
```
@ -77,7 +83,7 @@ Example `t=caps` response:
<searching>
<search available="yes" supportedParams="q" />
<tv-search available="yes" supportedParams="q,rid,season,ep" />
<tv-search available="yes" supportedParams="q,rid,tvmazeid,season,ep" />
<movie-search available="no" supportedParams="q" />
</searching>
@ -105,7 +111,7 @@ Example `t=caps` response:
###Torznab results###
First and foremost: Torznab indexers should return application/x-bittorrent as enclosure type:
First and foremost: Torznab indexers should return `application/x-bittorrent` as enclosure type:
```
<enclosure url="https://yoursite.com/download.php?torrent=123&amp;passkey=123456"
length="1460985071" type="application/x-bittorrent" />
@ -120,8 +126,9 @@ All these attributes are optional, the bold ones are recommended.
| Name | Type | Description |
|:---------------------|:--------|:-----------------------------------------------|
| type | string | series,movie,music,book (if unknown just omit) |
| tvdbid | integer | id for thetvdb.com |
| **rageid** | integer | id for tvrage.com |
| **tvdbid** | integer | id for thetvdb.com |
| rageid | integer | id for tvrage.com |
| **tvmazeid** | integer | id for tvmaze.com |
| imdb | integer | id for imdb.com |
| bannerurl | url | Url to a banner image |
| **infohash** | string | Torrent infohash |
@ -139,7 +146,8 @@ These two attributes are included to allow a torrent tracker to specify the seed
These values should be set to the long term requirements for the torrent. So if it's freeleech for a decade, you can set minimumratio to 0.0, but if it's only for a day, keep it at 1.0. (0.0 is a bit drastic, but you get the point)
The idea is that if one of the two criteria is satisfied, the torrent client can put the torrent at a lower priority or stop seeding entirely.
Specifying one or both of these attributes is recommended, because it allows future clients to appropriately adjust the seeding configuration of the torrent client.
Specifying one or both of these attributes is recommended, because it allows future clients to appropriately adjust the seeding configuration of the torrent client and/or warn the user to avoid HnRs.
##Hints and gotta's##
@ -150,10 +158,10 @@ Specifying one or both of these attributes is recommended, because it allows fut
- Return the appropriate newznab errors on apilimits and such.
- Support the ```offset=``` paging option.
- Ensure you have a ```<guid>...</guid>``` item in your rss results.
- Implement the ```t=caps``` endpoint, feel free to put it behind an apikey check. It provides useful information about which categories you support.
- Implement the ```t=caps``` endpoint, feel free to put it behind an apikey check.
##Some References##
Newznab specification: http://newznab.readthedocs.org/en/latest/misc/api/
nZEDb specification: https://github.com/nZEDb/nZEDb/blob/master/docs/newznab_api_specification.txt
Torznab xml attributes: https://github.com/Sonarr/Sonarr/blob/develop/schemas/torznab.xsd
Specific example: https://github.com/Sonarr/Sonarr/blob/develop/src/NzbDrone.Core.Test/Files/RSS/torznab_hdaccess_net.xml
Specific (older) example: https://github.com/Sonarr/Sonarr/blob/develop/src/NzbDrone.Core.Test/Files/RSS/torznab_hdaccess_net.xml