You can use a custom CSS to make the HTML preview look prettier. To configure
this, edit the configuration option texinfo.preview.customCSS. The CSS
file can either be an online or a local (starting with file://)
resource.
A good example is https://www.gnu.org/software/gnulib/manual.css, which is popular among manuals of GNU projects. (Note: May require some tinkering when used with darker editor themes)