How do I reference favicons in my /public-folder?

This appears to be the same issue as this very recent one:

This article may be interesting:
https://www.sitepoint.com/all-you-need-to-know-about-the-new-wordpress-site-icon-api/
WordPress already adds HTML for the site icon, as with
https://developer.wordpress.org/reference/functions/wp_site_icon/

So just using the get_site_icon_url may already result in the HTML you need.