Former top Pence aide says White House officials wanted to get rid of the coronavirus task force as the pandemic raged in April