I would like to create a database to store how often access tokens in my system are being used.

I was thinking of a data model like so -

  • Access Token
  • from (basically I will store this hourly, so this will be the timestamp of the start of the hour)
  • Number of times used

What would be the ideal database for this? I am conflicted with several choices.

Some more details to help out:

  1. We get around 5k requests per minute, so the DB will have to be quick enough to update that.
  2. We will not often be querying the database, probably only be looking for daily summaries and month end aggregations for accounting and auditing purposes.
  3. We would prefer this to be on AWS as a service, but we can also spin up instances on our own.
  • 1
    Please help me reconcile "basically I will store this hourly" with "5k requests per minute." Does that mean your data set is growing by 300,000 new summary records per hour, or did you mean you'd be creating or atomically incrementing some smaller number of hourly records at an average of 5k/min (84 trx/sec) (and if so, how many hourly records per hour would you expect to represent the growth rate? – Michael - sqlbot May 31 '18 at 10:58
  • Sure. It basically means that I will monitor a set of access tokens and keep incrementing the number of times used fields per hour. So that means that if I am tracking 5 access tokens. Every hour, there will be 5 new records created. From our current system, we are seeing the usage of around 10k access tokens, so 10k new records per hour. – Michael Victor May 31 '18 at 11:35
  • 1
    So each token's counter would be updated approximately every 2 minutes on average (10k unique/hr, 5k trx/min)? – Michael - sqlbot May 31 '18 at 12:11
  • Yes, you are right! – Michael Victor May 31 '18 at 12:12
  • so better use a managed service if you are worried about scale, availability, performance and resiliency, a non-relational database such as DynamoDB, even better you might look at the serverless DynamoDB for better scalability and cost optimization. – Walid May 31 '18 at 22:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.