When the Normans conquered Rhossili in the 12th century, it was a small village and church tucked at the foot of Rhossili Downs, on a strip of ground close to the beach, known today as the Warren. The church here dated back to the 6th Century and was dedicated to St. Sulien or St. Sili. The name St. Sili together with the Welsh word for moorland, 'Rhos', gives Rhossili its name. It is claimed that during the 13th century huge storms blew up which lashed the west coast of Gower with forceful winds and rain. This weather caused the powerful action of the Atlantic waves to send mountains of sand ashore, engulfing both village and church. Evidence that this village had existed and succumb to sandstorms was revealed at the end of 1979 when another severe storm exposed some of the old buildings. An archeological dig on the site in 1980 confirmed that these buildings were indeed the fabric of the original Rhossili settlement and the area is now protected by law as an Ancient Monument.

As a result of the unpredictable, harsh weather found at this exposed headland, the Rhossili inhabitants rebuilt their village on the clifftop above, away from the sand and sea that forever threatens the lowland. Rhossili village is characteristically void of large specimen trees. With the constant salt-thick winds from the Atlantic scouring all that dares to bare to its unmitigated mastery, the only notable tree with some history that managed to survive here is an ancient ash, demoted by the wind to grovel at the ground on its side. Sadly, this tree is no more but reminders are still evident at the farm where it was rooted - named Ash Tree Farm. Currently, the closest resemblance of a tree in Rhossili is a stunted, hedge-bound Sycamore which crouches in the shadows of the Worm's Head Hotel.

Positioned on an ancient raised shoreline plateau from the beach, halfway between the villages of Rhossili and Llangennith, is the Old Rectory, an isolated cottage complete with many tales of smuggling and mysterious ghostly apparitions.

Marking the western end of Rhossili's long beach and also the most westerly tip of the whole of Gower, is the strange serpent-like promontory, Worm's Head. At low tide, a rocky causeway out to 'The Worm' can be seen, but visitors must be warned that the sea only allows access to the causeway for a few hours each tide cycle, prefering to keep 'The Worm' to itself.

The Vile is an ancient strip field system stretching from the church to the headland. This was a system of sharing out the best land in the parish.

At the towering height of 193 metres (632ft.) above sea level, the pinnacle of Rhossili Downs, 'The Beacon', is the highest point in Gower allowing unparalleled 360° views of the peninsula. Rhossili Downs are regularly used as a dramatic launching site for hang-glider enthusiasts, when the weather permits. Scattering the landscape at this altitude, are six cairns close to the main footpath and further north, towards Hillend, lay Sweyne Howes, two ruined cairns with possible Viking connections.

On days when the seasonal elements peel away the layers of outbound poise, adoration for Rhossili's landscape (along with a certain smugness) can be regained, from the welcoming warmth and clemency of The Worm's Head public house and hotel!