Set on a rocky outcrop overlooking the River Rye, the ruins of Helmsley Castle still dominate the town. The castle was first established in the 1120s by Walter Espec, one of the most prominent nobles in England at the time. The castle was extended and enhanced by its various owners throughout the Middle Ages, most notably in the 14th century, when alterations are likely to have been made from King Edward III's visit to the castle in 1334. By the 16th century, the medieval buildings were inadequate for the ambitious Edward Manners, and he converted the chamber block into a fine Tudor mansion, the remains of which can still be seen today.