Before the manor was built it is thought that a Roman fort occupied the same site. The manor, which is the oldest house in Portmouth, has its earliest known records in the Doomsday book. Some of its earlier owners are King Edward the Confessor in 1042 and from 1066 till 1084 King William the Conqueror. Throughout the years it has been home to many private owners. Only change being in the later years as for 4 decades it was owned by Portsmouth City Council and was leased to a Youth Hostel Association, before once again becoming privately owned.
Many apparitions have been reported at the manor, also the sound of childish whispers, sudden changes in temperature, many orbs and strange lights have been seen and people have reported feeling scared and intimidated by some of the spirit energies.