There's one error in your logic...you are assuming that the reason you have doubled your money is because you made the right moves. What you fail to realize is that the Dow and the S&P are both up like 50% from the recent bottom, meaning your performance has not been anything truly amazing.
Plus, placing all of your money into one asset is the worst move in the books. Haven't you ever heard the most important rule of investing? Diversify, Diversify, Diversify. Plus, silver is over valued due to its safe nature and will be really fucked when the inflation talk does not come to fruition due to timely rate hikes and investors begin to move their money out of safe investments and into the markets.