With (one's) head held high phrase
If you do something "with your head held high", you are unashamed, you display your confidence and pride, often after a failure.
If you do your best, you can come back home with your head held high.
After the bankruptcy, Mr. David started his business again with his head held high.
Despite his failure in the first round, he walked into the next round with his head held high.
This expression alludes to a posture of pride. It originates from the second half of the 1500s.
Try to avoid to change something because doing something else could make things worse
I could have changed my answer, but I decided to leave well enough alone.