I’m trying to customize the look of my posts page (blog page).
I have no problem changing the template (look) of my front page, which is actually a regular page, but no luck with my posts page. My posts page is different from my front page.
Should I create a new template (*.php file)?
Not sure how to handle this.
This isn’t the place for general WordPress questions or a beginners guide to WordPress. Try WordPress Answers or one of its equivalents.
In future always start by consulting the codex:
This shows if you duplicate index.php to home.php, then it will be used for your blog posts page.
If you need to change the general markup of the page (header, footer etc) duplicate base.php to base-home.php if needed. More on that here.