Thanks to a weekly Rust workshop hosted by one of my colleagues at QuestDB, along with some extra time spent in an airport lounge during a long layover, I was finally able to hack together some Rust that does something useful!

questdb-retention

The project I put together is questdb-retention, a small program that allows you to manage data retention in your own QuestDB instance.

Even though QuestDB does not support a traditional SQL DELETE statement (as of v7.0), you can still purge stale data from you database by using the DROP PARTITION command. Check out the docs for more information about the specifics around data retention.

The questdb-retention package allows you to drop old questdb partitions either interactively (on the command line) or through a yaml config file. Only tables that are partitioned are supported, for obvious reasons.

In my view, the best way to use this package is to write a yaml config, compile the executable, and add a cron job that runs the command at regular intervals. Here's an example of a yaml file that you can use:

---
conn_str: host=localhost user=admin password=quest port=8812
tables:
  my_partitioned_table_by_month: 5
  my_partitioned_table_by_day: 5

The config only specifies the number of partitions to keep per table, but since QuestDB supports partitioning by anywhere from HOUR to YEAR, it's difficult to tell how much data is being retained by just inspecting the config file. This is something that's worth improving in the future. Also, once the package reaches stability, I'll take a look at adding it to crates.io to make it official.

All in all, once I got the hang of some Rust basics, I really enjoyed working in the language! I find Rust's matching to be incredibly useful, and I'm starting to get comfortable with borrows, mutability, and common Rust types like Option and Result. I'm excited to learn more concepts like async and lifetimes, as well as starting to write some lower-level code that really can take advantage of all Rust has to offer.