A lot of the dollar’s value comes from its demand across the world. Countries simply need to have a stockpile if they’re to pay for their imports etc. If some of those countries begin to leave it behind in favor of another currency, demand for the dollar along with its value will drop. This, of course, would have disastrous consequences for the U.S. economy.