You are held responsible for the oaths and promises you have made up in your mind. The verse 225 in Surah al-Baqarah explicitly describes this:
“God does not hold you responsible for your unintentional oaths, but He holds you responsible for what (the oaths) your hearts have earned. God is forgiver and He treats gently.” (al-Baqarah 2:225)
According to the verse, speaking out the oath is not essential. It is enough that you have made a firm intention in your heart. Although you do not make it verbally, God knows what is inside your heart. So, we advise that you keep on acting according to your promise.
If you have to break your promise because there is a better way now or there is another vital reason, then you should expiate for it. Please see the following about the expiation for breaking an oath or vow: