From the thirteenth century onward, the district grew around a naval dockyard on what was originally the Isole Gemini. The land in the district was dominated by the Arsenale of the Republic of Venice, then the largest naval complex in Europe. A Greek mercantile community numbering around 5,000 in the Renaissance and late Middle Ages was based in this district, with the Flanginian School and the Greek Orthodox Church of San Giorgio dei Greci being located here, of which the former comprises the Hellenic Institute of Byzantine and Post-Byzantine Studies in Venice and the latter is now the seat of the Greek Orthodox Archdiocese of Italy.
Other significant structures were by the monasteries in the north of the quarter.
Prominent sites and buildings in Castello include:
- Church of Santi Giovanni e Paolo (San Zanipoli)
- Church of San Giorgio dei Greci
- Campo Santa Maria Formosa and Church of Santa Maria Formosa
- Church and Ospedale della Pietà
- Church of San Zaccaria
- Church of San Giovanni di Malta
- Scuola of San Giorgio degli Schiavoni
- Scuola di San Marco
- Venetian Arsenal
- Church of Sant'Elena
- Palazzo Malipiero-Trevisan
- Palazzo Vitturi
- Palazzo Zorzi Bon