The Know Nothings collapsed when they dropped their anti-Kansas-Nebraska position, they lost all Northern support, even though they did so to keep a united front. Also, the decline of immigration and failure of Know-Nothing dominated legislatures to make good their campaign promises enabled critics to say they were useless. Their greatest problem however was that by 1856, many Know Nothings were old guard politicians, however at the start, one of their biggest appeals was that they were a new people's party.
The Republicans meanwhile, had gone from strength to strength, with the Speaker of the House now a Republican, and experienced politicians like Charles Sumner, William Seward and David Wilmot now part of the party. The Republicans also had a clearer platform, they were against slavery expansion and the idea of a Slave Power conspiracy was central to their beliefs.
The 1856 elections pit Fillmore, standing for the Know Nothings, against Buchanan, standing for the Democrats, against Fremont, standing for the Republicans.
In the South, the competition was basically between Fillmore and Buchanan, in the North it was between Fremont and Buchanan. Buchanan was from Pennsylvania, regarded as a battleground state,and was also acceptable to Southerners, with the Democrats painting themselves as the party of the status quo and unity. Republicans meanwhile, voted for Fremont because of the romance surrounding his career, he was a young, successful military leader who made the Republicans look young and energetic.
The American Party lost badly, only gaining 8 electoral votes. Buchanan won, carrying all but one Southern state along with Pennsylvania, New Jersey, Illinois and California. Freemont won elsewhere.
The results were good for Republicans. They had decimated the competition, the Know Nothings, and were only two states away from victory. Many were confident that they would win in 1860, especially with how much support the Democarts had lost in the North.