Long before Jan van Riebeeck landed at the Cape, Portuguese explorers charted the bay in the 15th and 16th centuries, the first being Bartholomew Dias in 1487. Ninety years later Manuel da Perestrello aptly called it Bahia Formosa or the Beautiful Bay. The first European inhabitants were 100 Portuguese sailors marooned here for 9 months when the San Gonzales sank in the bay in 1630. The survivors built two small boats which they used to link with a passing vessel. A stone that they left behind on Beacon Island, known as the Van Plettenberg Stone, is now in the Cape Town museum. In 1763, the first European settlers in the Bay were stock farmers, hunters and frontiersmen from the Western Cape.

A stinkwood navigational beacon was first erected on Beacon Island in 1771. The original was a square block of stinkwood, inscribed with the latitude and longitude of Plettenberg Bay and erected to enable mariners to check their location. It was replaced by a stone one by Captain Sewell in 1881.

The bay housed a barracks for the Dutch East India Company in 1776. The Governor of the Cape, Baron Joachim van Plettenberg, renamed the town Plettenberg Bay in 1779. In 1869 the barracks was bought by St Peters Church and used as a rectory for the next 70 years. Today it is privately owned.

In 1787/88 Johann Jerling erected a timber shed commissioned by the Dutch East India Company; the remains can still be seen and are preserved as a provincial heritage site.

In the 1830’s, Plettenberg Bay was a stopping point for ships travelling through during the winter time, heading to India.

As of 1833, the population of Plettenberg Bay was estimated around 400, with half being white, according to diplomat Edmund Roberts who visited the area. Edmund Roberts noted that the bay had a large abundance of cattle and sheep. He also recognized "the excellence of its butter" and a timber industry.