What do I need to back up in order to be able to rebuild my repository?

All you absolutely need are the datastream and object directories, wherever you configured them to be.
The locations are the values of the "object_store_base" and "datastream_store_base" params in $FEDORA_HOME/sever/config/fedora.fcfg.
As per default they point to the following directories:
$FEDORA_HOME/data/objects (contains all FOXML files)
$FEDORA_HOME/data/datastreams (contains all binary files from the managed datastreams)
Anything within these directories must be backed up.

NOTE: This assumes you're using the default standard filesystem storage, and haven't configured another storage layer implementation.
Optionally, you can also backup your custom XACML policies, should you have any. That location is configured in the param "REPOSITORY-POLICIES-DIRECTORY". The default points to $FEDORA_HOME/data/fedora-xacml-policies/repository-policies.

If you have an active FeSL installation, the backup should also include $FEDORA_HOME/pdp/conf and $FEDORA_HOME/pdp/policies. Also, you should find a $FEDORA_HOME/install directory containing an install.properties file. It's useful to also back up that file, as when you re-install you can use that file during the installation to reinstall with the same options (without having to enter them manually) with:

java -jar fedora-installer.jar install.properties

(substituting the actual name of the fedora installer jar which will depend on which release you're using)