For example, AWS announced their new "RDS Proxy" service at re:Invent 2019 and it is now in preview. How can I find out when/if Terraform will support this (for planning purposes)?
Terraform is an open source project that is managed on GitHub. You can follow existing issues or open a new one if your enhancement request isn't found. See their issue list at: https://github.com/hashicorp/terraform/issues
As Martin Atkins points out in the comments, the plugin specific features are each developed and tracked in their respective repositories. Here's the one for AWS: https://github.com/terraform-providers/terraform-provider-aws
Hashicorp is usually quite quick in rolling out support for new AWS services. You can also follow their blog regarding announcements for more such services in the future. Although it does not mention support for RDS Proxy yet, For other services announced during AWS:ReInvent 2019, they announced support during the reinvent week itself on Dec 04th, 2019. https://www.hashicorp.com/blog/hashicorp-re-invent-19-terraform-supports-newly-announced-aws-services