How to configure http-proxy-middleware without using Create-React-App

Question

I have a cumtom react startkit package, and I need to dynamically configure the domain for front-end API calls. I plan to use the http-proxy-middleware plugin, but all the examples I can find are under the Create-React-App package, setting setupProxy.js will be loaded automatically, which confuses me, isn't there a way to configure http-proxy-middleware to webapck without using Create-React-App?

I also tried axios to configure cross-domain, but the same error, like this

axios.defaults.headers.common['Access-Control-Allow-Origin'] = '*';

Refer