RateLimiter

[sumake99]

Data structures
Application – logical grouping of controllers grouped by similar functionalities or business needs into one web api service
Controller – controller
Endpoint - http methods exposed by Controller
Client – API consumer, authenticated through OAuth2 protocol
These are assumed to be configured in database, for sake of simplicity, I skipped Controller entity.

Goal of Rate limiting can be control of performance to avoid system overload or business requirements, for example, limiting free-trial customers
There are 2 main challenges

1. Efficient request tracking implementation
2. Rate limiting implementation that can be configurable, expandable and maintainable

Request tracking
For the purpose of request tracking in this project, I selected local cache.
If Request is allowed to proceed, Request will be cached with key patterns built from {ClientApplicationEndpointID}.
ConcurrentDictionary<string, List> will be used.

Pros of this approach:
• Fast implementation for startup project
Cons:
• This can work on low to medium traffic systems, possible performance issues on high traffic systems
• Will not work with horizontally scaled service, in this case different implementation of the interface can be done using database, Redis, Elastic etc.

Rate Limiting
For Rate limiting I decided to use Json Rules Engine (https://microsoft.github.io/RulesEngine/#create-a-workflow-file-with-rules)

Here are some pros:
• Flexible Json based rules definition
• Rules can be loaded dynamically from any other source (database)
• Dynamic object input support, it’s agnostic of business object’s structure
• C# Expression support
• Extending expression via custom class/type injection

Cons:
• It’s not compiled code, it’s physical file which require deployment
• Json file can become malformed
• Can eventually grow to become big and cumbersome
• Require some technical expertise to maintain

One of the ways to find balance between maintainability and flexibility can be limiting expressions to operate on well-defined sets of properties. For example, classify API Clients into some kind of groups, tiers etc.
Groups can be stored in database with Client record and maintained through Backoffice UI. Changes to rules would be done and deployed by engineers. Tests can be added to validate rules files and run as part of deployment process.

Projects
RulesEngine – implementation of rules service, uses provider pattern. Currently only Json based provider is implemented. RulesEngine service is agnostic of its consumers
RulesEngine.Models – models for RuleEngine
RateLimiter – consumer of RulesEngine,
RateLimiter.Models – models for RateLimiter
Utilities – includes Cache implementation
RequestTracking – service, abstracting request tracking storage, uses local cache, but different provider can be implemented without major disruption to other projects
RequestTracking.Models – models for RequestTracking
RateLimiter.Tests – tests for RateLimiterService.
RequestTracking.Tests – CacheTrackingStorageProvider desewrved it’s own tests.

In TODO list, each of the services should have its own tests
The choice of implementation should eventually depend on multiple objective and subjective factors – technology stack, expertise, existing application structure, infrastructure, development and deployment process.