To modify the header, locate the template named "buildHeader" inside the structural.xsl. Copy this template into your theme's local template.xsl and modify the XSLT as needed.
<xsl:template name="buildHeader">
....
</xsl:template>
To modify the footer, locate the template named "buildFooter" inside the structural.xsl. Copy this template into your theme's local template.xsl and modify the XSLT as needed.