General warning
Note: This PR is the first step in my desire to provide a way for the user to request a specific number of result on endpoint allowing pagination, as discussed with several developers on Discord and completing the work previosuly started in #276 (and thus use the internal pagination of the request function, which is currently not used by high-level functions!). This feature is not implemented in this PR.
However, the features implemented in this PR can be considered independent and already offer new cool stuff, so I propose it for review.
In this PR, I make a difference between the first
parameter, which is the parameter sent to Twitch API when performing the HTTP request, and the first
argument exposed by some methods as User.fetch_pools()
Changes
In the current version of TwitchIO, the number of elements requested to endpoints when pagination is enabled is 100 by default, this is set by the get_limit() subfunction:
def get_limit():
if limit is None:
return "100"
to_get = limit - len(data)
return str(to_get) if to_get < 100 else "100"
This leads to two main issues:
- For endpoints where the maximum value allowed for
first
is less than 100, we have to do a little trick by adding a "first" parameter when calling the fetch_
method, this is currently the case for 4 methods: get_reward_redemptions
, get_polls
, get_predictions
, get_channel_schedule
. These functions are the ONLY functions exposing a first
parameter. All the other ones suppose by default that first=100
due to the get_limit() function.
- For endpoints allowing higher values than 100, you will receive 100 data and no more. Endpoints like entitlements/drops allows 1000 results per page.
This PR fix all of this by specifying the maximum value allowed by first
when requesting the endpoint. This is done by adding a new parameter, max_elements_per_page
to the request() method, containing this information.
Below, I listed all endpoint using the "first" parameter, with their min, default and max value. For each endpoint, the value of the max_elements_per_page
argument is the maximum value allowed to first
by the documentation.
| Method | Endpoint | min | default | max | Implemented in TwitchIO |
|--------|-------------------------------------------|-----|---------|------|----------------------------|
| GET | analytics/extensions | 1 | 20 | 100 | X |
| GET | analytics/games | 1 | 20 | 100 | X |
| GET | extensions/transactions | 1 | 20 | 100 | get_extension_transactions |
| GET | channel_points/custom_rewards/redemptions | 1 | 20 | 50 | get_reward_redemptions |
| GET | chat/chatters | 1 | 100 | 1000 | X, BETA |
| GET | clips | 1 | 20 | 100 | get_clips |
| GET | entitlements/drops | 1 | 20 | 1000 | get_entitlements |
| GET | extensions/live | 1 | 20 | 100 | X |
| GET | games/top | 1 | 20 | 100 | get_top_games |
| GET | hypetrain/events | 1 | 1 | 100 | get_hype_train |
| GET | moderation/banned | 1 | 20 | 100 | get_channel_bans |
| GET | moderation/blocked_terms | 1 | 20 | 100 | X |
| GET | moderation/moderators | 1 | 20 | 100 | get_channel_moderators |
| GET | channels/vips | 1 | 20 | 100 | get_channel_vips |
| GET | polls | 1 | 20 | 20 | get_polls |
| GET | predictions | 1 | 20 | 25 | get_predictions |
| GET | schedule | 1 | 20 | 25 | get_channel_schedule |
| GET | search/categories | 1 | 20 | 100 | get_search_categories |
| GET | search/channels | 1 | 20 | 100 | get_search_channels |
| GET | soundtrack/playlist | 1 | 20 | 50 | X |
| GET | soundtrack/playlists | 1 | 20 | 50 | X |
| GET | streams | 1 | 20 | 100 | get_streams |
| GET | streams/followed | 1 | 100 | 100 | X |
| GET | streams/markers | 1 | 20 | 100 | get_stream_markers |
| GET | subscriptions | 1 | 20 | 100 | get_channel_subscriptions |
| GET | tags/streams | 1 | 20 | 100 | get_stream_tags |
| GET | users/follows | 1 | 20 | 100 | get_user_follows |
| GET | users/blocks | 1 | 20 | 100 | X |
| GET | videos | 1 | 20 | 100 | get_videos |
(Note that for /videos, we can specify first
parameter only if we specify the game_id or user_id query parameter. This condition has been implemented in this PR)
In fetch_
methods exposing a first
argument, it has been rerouted to the limit
attribute of request() to avoid any breaking change.
Moreover, the following functions:
- get_predictions
- get_channel_schedule
- get_polls
Can implement pagination, but were delivered with pagination=False.
Activate the pagination in these functions doesn't change or break anything and allow them to benefit from this PR. So I did it.
Finally, I disabled the assertion breaking the request when full_body was True as the same time as paginate, since the body is returned before the evaluation of paginate. This will prevent any futher bug in the future.
I tested all helix endpoints with both app token and user token to validate this PR.
Checklist
- [X] If code changes were made then they have been tested.
- [X] I have updated the documentation to reflect the changes.
- [X] I have updated the changelog with a quick recap of my changes.
- [ ] This PR fixes an issue.
- [X] This PR adds something new (e.g. new method or parameters).
- [ ] This PR is a breaking change (e.g. methods or parameters removed/renamed)
- [ ] This PR is not a code change (e.g. documentation, README, ...)