I'm going to talk about general process here, rather than addressing Magneto specifically, so that it's useful to more people and situations.
How do I automate this Magento2 installation process from dockerfile. Like setting up database and the other installation process.
Run through the wizard process once. Track down the files it generates, which isn't straightforward and may require a few tries. Pull those configuration files into your app configuration, and try a fresh container that just uses those.
That is, reproduce what the installation wizard is doing in your own configuration tool. This sounds difficult, but often the wizard is complex because it needs to address many situations, and you are fine hard-coding many things for your specific situation.
As a final note, since we're talking about Docker, presumably the database resides on another machine. Therefore, you only need to set it up once, and that process will be part of its provisioning, not this container's.