Players who do extra stuff generally get higher level and more items. They become more powerful.
These players are also generally the players with more patience, more skill, more interest in the game, more willingness to put time into the game, and who learn more about how to play well as they go along (due to doing more fights from the extra stuff).
So these people who do the optional content are in general the ones who'd most appreciate the game being hard. They will on average be better players.
But instead they get an easier game. Why? Because doing the optional stuff makes their characters more powerful (more levels, more items).
The best players tend to get the easier RPG game experiences due powering up from completeing optional content. This is very unfortunate. They were doing the optional content for fun and for additional challenge, not to try to make the game easier.
The guy who likes the game and wants to do everything – which includes most of the best and most serious players – will accidentally, just by trying to complete every challenge in the game, make the game a lot easier.
One note: the optional content does serve the purpose of letting bad players, who get stuck, have a way to power up. Some of the worst players need to get strong to make progress on the game, so it helps them. If the optional content didn't let you gain any more power on your characters (no experience or item rewards), that would not just annoy the large majority of players (who irrationally seek virtual rewards to motivate them), it could also result in some especially bad playrs getting stuck.
I personally had this problem with Pillars of Eternity. By completeing everything I got too powerful. When I got to the later stages of the game, it was so easy that I got bored and stopped playing. I mostly liked the game quite a bit, but I just couldn't deal with how easy it was (and didn't want to take extensive steps to artificially handicap myself).
On a related note, games often have several difficulty modes. Pillars of Eternity did. I was already playing on the hardest when one the game became painfully, boringly easy. It would be very easy for them to have made another harder mode. Or just let me choose my own difficulty. Let me input two numbers: damage and hp multipliers for the enemies. If I want the enemies to have 10x the hp and 2x the damage (compared to their hardest mode) – which honestly sounds about right to me – then why not let me? People literally soloed the hardest mode using one character when it's supposed to be a game where you have a party of 6 characters...
Why do they make difficulty modes ranging from extremely easy to, at the top end, moderately easy? I think it's to protect the self-esteem of bad players who don't want to admit how much better some people are. They don't want a mode that only the best players can beat because a lot of other people will try it and fail and find failure frustrating and say the game is "unfair" and blame the game designers for allowing a hard difficult mode to exist at all.
I find that basically every single game makes the maximum difficult mode way too easy (often in objectively measureable ways, e.g. the game is beatable on maximum difficulty without any party members, meaning that with a full party you have at least a multiple like 6x of the combat power needed to beat the maximum difficulty). This is one of the many ways that being better at thinking and learning can set one apart from other people and create some incompatibilities.