If I use one of your instrumental covers, do I still have to get permission to use the song that is being covered?

If you use the same lyrics and you want to publish them for commercial use (a use not covered by "Fair Use"), then yes you need permission from the owner of those lyrics.  If you change the lyrics and do your own song using the instrumental, then no you don't need permission, since the production is an original rendition and your lyrics would be original.

What is Fair Use?  See Baylor's definition (scroll down to "What is Fair Use") - https://www.baylor.edu/copyright/index.php?id=56543#:~:text=Fair%20use%20allows%20limited%20use,use%20is%20a%20fair%20one.