You can use agenix for secrets management. You can store all secrets encrypted with your SSH key in public config. The only non functional part is installing your SSH key, which is probably not a bad thing.