Many coastal communities in developing countries depend on mangrove ecosystem services (ES). A combination of anthropogenic and environmental stresses threatens mangroves globally. This study at the Ankobra catchment communities in Ghana focused on the relation between ES utilization and mangrove forest structure. Through vegetation survey, we observed significant effects of selective logging, branch cutting, density of Acrostichum aureum, and water stress on tree stocking and sapling densities. We observed through interviews in five communities that about 98% and 88% of mangrove wood harvested are used for fuelwood and construction respectively. The vegetation structure of the forest areas receiving high harvesting pressures was less complex, with lower tree and sapling density, as well as lower seed-bearing trees than less-disturbed areas. Existing mangrove harvesting regulations are compromised to accommodate the needs of the surrounding communities. Recognizing these impacts is important to improve management decisions, address community needs, and reduce pressure on mangroves.