This might seem really obvious, but I couldn’t find any information about how to do this, so here’s a post in case someone else gets stuck.

When you install Umbraco with nuget and then run it, you are presented with an installer. There is a link next to the Install button “Customize” which you need to click on if you don’t want to use SQL Server CE:

Umbraco customize

From there you get a lot more options about how your umbraco is set up, including database type and connection string.