The folly castle in the park of Hagley Hall is a Grade II* listed building and the largest building in Hagley Park. It was designed by Sanderson Miller for George Lyllelton in the middle of the 18th century to look like a small ruined medieval castle.
The building of the castle (and another folly called the rotunda), was started in 1747 while Sir Thomas Lyttelton was still alive (he died in 1751) so he was not opposed to the modernisation of his park with suitable fashionable ornamental follies, but the credit for its creation is usually given to his son and heir George Lyttelton (the future 1st Lord Lyttelton).
The Gothic Revival castle is rectangular with a round tower at each corner. Only the tower in the north-west is complete to a height of four storeys and an attached adjoining taller stair turret both topped with battlements. The top floor of the completed tower has large pointed windows, it has a "domed ceiling with Gothic plaster decoration, cusped pointedniches alternating with the windows and above the door is a quatrefoil containing a coat of arms". The floors below have smaller rectangular and Lancet windows (some of them cusped).
The other towers are one or two storeys high with small rectangular windows and built to look as if they are ruined, (this effect was achieved by building the parts of the walls furthers from the castle's bailey higher than those closest allowing sloped roofs invisible to an observer outside the bailey).
The curtain walls are also built to look as if they are ruins. The west wall is the most complete with three tall windows two of which are complete enough to contain Gothic arches. Below the middle window is a Gothic arched doorway which are three shield reliefs.
The towers and curtain walls are faced with sandstone some of which may well have been taken from the ruins of Halesowen Abbey.
In the nineteenth century a single story building with a hipped slate roof was built next to the north-west tower. On the south side the entrance is through a gabled rustic timber porch; to its right there is a three-light casement window with four-centered head and latticed lights.