To view the interior of the Old Statehouse at 84 Broad Street, enter from Courthouse Square around the back of the building and go through security. Although the Probate Court is on the third floor, the entire building is solemn and quiet. Enjoy the respite from the heat; there is air conditioning and also a public restroom.
If you like early governmental history, 84 Broad Street is a treasure chest. When you enter, on the opposite side of the staircase is a small touch screen where you can access a narrated film about the building and some of its contents. If you've got the time, I would suggest you briefly explore the building first, then watch the film, and afterwards, explore again.