Weight gain - Feeling like I've already failed

I'm a 36 year-old FTM, 5'3" and my starting weight was 92kgs (202lbs, BMI of 35). I am now in Week 23, and I've already gained 6.5kgs (14lbs), despite trying to be mindful of how much I eat and motivating myself to walk several times a week and also just starting swimming once a week...

I've read that women with my starting BMI shouldn't gain more than 20lbs overall, so I now pretty sure I'll fail at doing this. I'm feeling embarrassed and am getting really worried about the consequences for my baby daughter and the birth and recovery.

I will try and do even more exercise but am scared of changing my food intake as I've been careful to get all the nutrients that my baby needs (my baby seemed absolutely fine at my 20-week scan).

Will gaining more than the target 20lbs have serious consequences on the both of us? The midwife isn't particularly helpful or understanding, the guidelines are putting a lot of pressure on me, and I'm just feeling very down and ashamed right now.

I'd love to hear from you guys' experience and advice on that matter and understand what might happen if this trend continues... Thank you!