The Rockport Town Hall opened in the winter of 1869. In the year that followed, a series of concerts and lectures—including one by Mark Twain—raised $250 to establish a town library. The Town accepted the donation and approved matching funds for the project in 1871.

The library opened in the Town Hall, with subscribers paying 50 cents/year. By 1886 the library was open two days a week, and the collection numbered 2,400 books.

By 1890 the Library Committee was anxious to obtain larger quarters, and another library was started in Pigeon Cove. In 1892 the subscription fee was terminated, and the number of patrons rose to 500. In 1893 the Library Committee was replaced by a three-member elected Board of Trustees, and the hours of service were expanded.

In the early 1900s, Leander M. Haskins began negotiating with Andrew Carnegie, who was giving libraries to towns that could not afford them. On November 11, 1903, a special town meeting accepted Carnegie’s offer to provide $10,000 to build a free public library building for Rockport. In return, the Town agreed to provide a suitable site for the building and to spend at least $1,000 a year to maintain a free public library. In 1904 town meeting voted 50–7 to purchase land at the junction of Jewett and Cleaves streets for $2,000.

The Carnegie Library was built in 1907. The 4,369-square-foot structure was built of locally quarried bi-color granite with Greek Revival features, interior paneling, and a terrazzo floor under the dome. The building functioned as a library until the fall of 1993. It is now a private residence.

By the 1980s, the Carnegie building was bursting at the seams. Franz Denghausen (1911-1987), a local sculptor and benefactor, left a one-million-dollar bequest to provide a new library. The Smithsonian Institution mounted a three-year challenge to that bequest, but in 1990 the funds were received.

On Oct. 28, 1989, the Town had voted to transfer the site once known as the Tarr School to the Library Trustees for conversion to a library. The Rockport Public Library opened in its present location at 17 School Street in December 1993. The ashlar granite building was built in 1864 as part of a mill complex. In 1904 it was converted to a primary school, and in 1938 it was expanded with an addition along Broadway. The building is 15,392 square feet and is part of a National Historic District.