Class: TokenPool

core/token-pooling/token-pool~TokenPool(batchSize)

Encapsulate a collection of rate-limited tokens and choose the next available token when one is needed.

Designed for the Github API, though may be also useful with other rate- limited APIs.

Constructor

new TokenPool(batchSize)

TokenPool Constructor

Parameters:
Name Type Description
batchSize number

The maximum number of times we use each token before moving on to the next one.

Source:

Methods

add(id, data, usesRemaining, nextReset) → {boolean}

Add a token with user-provided ID and data.

Parameters:
Name Type Description
id string

token string

data *

reserved for future use

usesRemaining number

Number of uses remaining until the token is exhausted

nextReset number

Time when the token can be used again even if it's exhausted (unix timestamp)

Source:
Returns:

Was the token added to the pool?

Type
boolean

forEach(callback)

Iterate over all valid tokens.

Parameters:
Name Type Description
callback function

function to execute on each valid token

Source:

next() → {module:core/token-pooling/token-pool~Token}

Obtain the next available token, returning null if no tokens are available.

Tokens are initially pulled from a FIFO queue. The first token is used for a batch of requests, then returned to the queue to give those requests the opportunity to complete. The next token is used for the next batch of requests.

This strategy allows a token to be used for concurrent requests, not just sequential request, and simplifies token recovery after errored and timed out requests.

By the time the original token re-emerges, its requests should have long since completed. Even if a couple them are still running, they can reasonably be ignored. The uses remaining won't be 100% correct, but that's fine, because Shields uses only 75%

The process continues until an exhausted token is pulled from the FIFO queue. At that time it's placed in the priority queue based on its scheduled reset time. To ensure the priority queue works as intended, the scheduled reset time is frozen then.

After obtaining a token using next(), invoke update() on it to set a new use-remaining count and next-reset time. Invoke invalidate() to indicate it should not be reused.

Source:
Returns:

token

Type
module:core/token-pooling/token-pool~Token

(static) compareTokens(first, second) → {module:core/token-pooling/token-pool~Token}

compareTokens

Parameters:
Name Type Description
first module:core/token-pooling/token-pool~Token

first token to compare

second module:core/token-pooling/token-pool~Token

second token to compare

Source:
Returns:

The token whose current rate allotment is expiring soonest.

Type
module:core/token-pooling/token-pool~Token