As of August 2022, the solution for this is to load the "Authentication" product of Firebase in the console ("Authentication" is under the "Build" product category on the left side or can be found in "All products"):

Then select the "Templates" tab in the Authentication screen:

Select any of the Email templates (like "Password reset") and click on the Pencil icon to edit:

Click on the "Customize domain" link under the disabled domain name field:

Add your custom domain name (which you already own and have DNS control over) in the dialog box that appears:

(If the "Continue" button is disabled after typing in the domain name, make sure you click outside of the text input box. The "Continue" button should then be active.)
The next dialog box will provide DNS records that need to be added/updated. There should be two TXT records (a v=SPF and a firebase record) and two CNAME records (two mail records). Update those on your DNS and then return to the Firebase Console to click on "Verify". Google may be able to see and verify the new records immediately or it may take some time (they claim up to 48 hours, but if it takes that long, it is possible a mistake was made).