The problem is iPhone still thinks the handsfree is plugged in so it turns off the speaker.
My iPhone has got this issue as well. I fixed by keep plugging and unplugging until it worked.
That probably got the dust out from the jack.
If that does not fix, let’s try..
You could probably use alcohol to do this as well.
Warning: do not drink Windex