When Mr. King resigned in 1843, Richard Ellis became principal. He was followed by Calvin Littlefield in 1845. Mr. Ellis was again head of the school in 1848. The building became in adequate for the larger number of new pupils coming continually and a new structure was constructed in front of and adjoining the old stone school. This addition cost approximately $4000. It was ready for use in October 1857, Mr. J. Dunbar Houghton, principal. He and his family moved into the wooden structure which was provided with a suite or rooming space for him and his family, teachers and a number of pupils.

All went well and the school prospered until the time of the Civil War when a number of the faculty and a large number of the pupils enlisted, and only with great effort did the school continue to the end of the period. In 1852 there were 188 pupils; 1862--266 pupils and in 1866 there were 342.

Mrs. William Mather and Mrs. George Mather endowed the Academy with $10,000 for the establishment of a course in Agriculture, 1901, the first in any secondary school in the United States. The School Park was donated by Mr. Fredrick Williams.

The wooden portion of Union Academy was remodeled in 1918. On January 29th, 1923, the entire set of buildings was burned to the ground with a total loss.