When I am on the phone with an automated answering system (press “1” to do this, “2” to do that…) and then get an incoming phone call from someone else, the number pad (that I am using on the display for the first call) disappears until I take care of the 2nd call.
Although, I have not experienced it…as it is a unique scenario…and likely a behavior of the default dialer app. There are other third party dialers available on the Google play store. The one that I like is
I am not sure how it might handle your specific scenario…but that would be worth exploring…as the default dialer…to the best of my knowledge…offers very little configurability.