NGINX TLS Proxy Add-on config to allow simultaneous HTTPS and HTTP access required by ecowitt integration

del13r · July 7, 2023, 10:55am

This lengthy guide focuses mainly on getting the NGINX TLS Proxy Add-on to serve HTTPS while leaving Home Assistant to directly receive HTTP webhook data at the same time.

Why did I do this?
I recently bought an ecowitt GW1100 WiFi Gateway and WH31 temperature and humidity sensor.
https://www.ecowitt.com/shop/goodsDetail/136

I checked for official integrations and found the matching integration here
https://www.home-assistant.io/integrations/ecowitt/

At the bottom of the page, I noticed the following text:

Ecowitt doesn’t support TLS/SSL, you can use the NGINX TLS Proxy Add-on to support HTTPS and HTTP at the same time.

I thought that looked a bit odd and a bit of a challenge so I decided to give it a try.

Up until now, my home assistant was setup exclusively for https access only via the default port 8123.
The address I was using was https://MYHOST.duckdns.org:8123

To get to that point, I was using:

port forwarding on my router just for port 8123 externally to Home Assistant port 8123 internally.
DuckDNS addon for both Dynamic DNS with LetsEncrypt support which generates the 2 .pem files.
Added the following code to the http config block which promotes the home assistant core default port 8123 from a http server to a https server:

http:
  ssl_certificate: /ssl/fullchain.pem
  ssl_key: /ssl/privkey.pem

Original Port Forwarding table

Service Name	External Port	Internal Port	Internal IP Address	Protocol
HomeAssistant	8123		192.168.1.205	TCP

Early on, without the NGINX proxy addon, I could only get either http or https access working, but never both at the same time.

What I wanted to acheive was:

HTTPS for my web browser to access Home Assistant securely using signed certificates.
HTTP for the ecowitt device to transmit data to Home Assistant.

After many hours and a lot of frustrated attempts, here is how I finally got it to work:

I decided to start fresh by removing both the NGINX addon and the DuckDNS addon.

After stopping and uninstalling the add-ons, the next thing I did was demote/remove https access:
To do this, I opened up /config/configuration.yaml and removed the following config block and restarted home assistant

http:
  ssl_certificate: /ssl/fullchain.pem
  ssl_key: /ssl/privkey.pem

After a restart, I could now only access Home Assistant using http to either:
http://192.168.1.205:8123
http://MYHOST.duckdns.org:8123

I opened up a terminal and deleted all files in the /ssl folder.

➜  /ssl ls -lahs
total 16K
   4.0K drwxr-xr-x    2 root     root        4.0K Jul  7 15:46 .
   4.0K drwxr-xr-x    1 root     root        4.0K Jul  2 10:41 ..
   4.0K -rw-r--r--    1 root     root        3.5K Jul  7 15:46 fullchain.pem
   4.0K -rw-------    1 root     root        1.7K Jul  7 15:46 privkey.pem

➜  /ssl rm -r *
zsh: sure you want to delete all 2 files in /ssl [yn]? y

I next re-installed DuckDNS (which has Let’s Encrypt support built in)
https://github.com/home-assistant/addons/tree/master/duckdns

And configured DuckDNS using the following settings:

domains:
  - MYHOST.duckdns.org
token: 36-CHARACTER-TOKEN
aliases: []
lets_encrypt:
  accept_terms: true
  algo: secp384r1
  certfile: fullchain.pem
  keyfile: privkey.pem
seconds: 300

I deliberately chose to not follow the DuckDNS documentation past this point and IGNORED the following section:

Additionally, you’ll need to configure the Home Assistant Core to pick up the SSL certificates. This is done by setting the following configuration for the HTTP integration configuration in your configuration.yaml:

http:
ssl_certificate: /ssl/fullchain.pem
ssl_key: /ssl/privkey.pem

The reason why I deliberately ignored that section of the DuckDNS documentation was that I am now going to use those certificates with the NGINX Home Assistant SSL proxy instead of directly by Home Assistant.

I then started DuckDNS addon which checked and updated my IP as well as generated my Privacy Enhanced Mail (PEM) files.

Here is the DuckDNS log:

+ Generating account key...
+ Registering account key with ACME server...
+ Fetching account URL...
+ Done!
[19:22:53] INFO: OK
MY.PUBLIC.IP.ADDRESS
NOCHANGE
[19:22:53] INFO: Renew certificate for domains: MYHOST.duckdns.org and aliases: 
# INFO: Using main config file /data/workdir/config
 + Creating chain cache directory /data/workdir/chains
Processing MYHOST.duckdns.org
 + Creating new directory /data/letsencrypt/MYHOST.duckdns.org ...
 + Signing domains...
 + Generating private key...
 + Generating signing request...
 + Requesting new certificate order from CA...
 + Received 1 authorizations URLs from the CA
 + Handling authorization for MYHOST.duckdns.org
 + 1 pending challenge(s)
 + Deploying challenge tokens...
OK + Responding to challenge for MYHOST.duckdns.org authorization...
 + Challenge is valid!
 + Cleaning challenge tokens...
OK + Requesting certificate...
 + Checking certificate...
 + Done!
 + Creating fullchain.pem...
 + Done!

I then went back to my terminal and checked the contents of the /ssl folder

➜  /ssl ls -lahs
total 20K
   4.0K drwxr-xr-x    2 root     root        4.0K Jul  7 19:23 .
   4.0K drwxr-xr-x    1 root     root        4.0K Jul  2 10:41 ..
   8.0K -rw-------    1 root     root        5.2K Jul  7 19:23 fullchain.pem
   4.0K -rw-------    1 root     root         359 Jul  7 19:23 privkey.pem

The newly generated Privacy Enhanced Mail (PEM) files are now in the /ssl folder.

At this point, Home Assistant is still only accessible using either:
http://MYHOST.duckdns.org:8123
http://homeassistant.local:8123

This was due to me deliberately ignoring the instructions to modify the http config of home assistant.

Again, the reason why I deliberately ignored that section of the DuckDNS documentation was that we are now going to use those certificates with the NGINX Home Assistant SSL proxy instead of directly by Home Assistant.

I installed NGINX Home Assistant SSL proxy
https://github.com/home-assistant/addons/tree/master/nginx_proxy

I noticed that this addon exposes port 443 so I figured I would forward that port on the router as well.

Service Name	External Port	Internal Port	Internal IP Address	Protocol
NGINX HTTPS	443		192.168.1.205	TCP

In the NGINX addon, I configured only my MYHOST.duckdns.org domain and left everything else as default and pressed save.

domain: MYHOST.duckdns.org
hsts: max-age=31536000; includeSubDomains
certfile: fullchain.pem
keyfile: privkey.pem
cloudflare: false
customize:
  active: false
  default: nginx_proxy_default*.conf
  servers: nginx_proxy/*.conf

I did not start the addon yet as I plan to do this after I restart Home Assistant.

I read the NGINX addon documentation and noticed that I need to add the following config block to the /config/configuration.yaml

http:
  use_x_forwarded_for: true
  trusted_proxies:
    - 172.30.33.0/24

VERY IMPORTANT Note: Insert this text as it is. Do not change any of this to suit your network.

I pressed save on /config/configuration.yaml and then restarted home assistant.

After Home Assistant came back up, I went to the NGINX addon and pressed start, then I looked at the log:

[19:40:35] INFO: Generating dhparams (this will take some time)...
Generating DSA parameters, 4096 bit long prime
...+........+............+.+..+...+..+....+.+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*
..+..+..........+....+...............+...+.........+.....+..........+.......+...+..................+.+....................+...+......+................+.+...............+......+...........+.....+...+.....+..+................+...+.+.....+.+.........+....+........+.+................+....................+...............+..................+......+.+....................................................+.....+..............+.+.......+..+..............+.+........+........................+..................................+.....+.............+....+.......+.+............+.........+.+.+....+....+.+..+............+............+.........................+..+...................+.........+............+.+.......+....+....+........+...............+.......+.......+.......+.....................+..............+......................+...+.+......+..............+...+.......+.............+.......+.....+.+.+......+...+......+..+..........+............+..+..+....................+........+....+......+..........+.....+..............+..............+...............+.....+...........+....+..............+....................+........+..+........................+.....+............+..............+....+........+.............+..+....+...........+...+........+.....+........+..............................................+.......+..+...+...+.........+............+......+...+.....+............+...........................+.............+.......+.......+..............+......+....................+....+.........+...............................+..+........+.........+................+.+...........+...+..+.+....................+.......+.................+..................+............................+...............+......+.............+...+.+....+................+..+.........+............+.+.......................+.+......+.+......+.....+...........+.........+.....+.....+.....+.................+.+..+.....+.......+....................+...+.......+........+..+.+.......................+...+..+.......+......+......+..............+.+...+.....+................+.....+..........................................+......+........+.....+...+....+..+...........+....+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*
[19:40:53] INFO: Running nginx...

I then decided to try out my https access to
https://MYHOST.duckdns.org/
and it finally worked and showed as secure in the browser!

Lastly, I configured my ecowitt device to send data direct to my Home Assistant via http using:
http://192.168.1.205:8123/api/webhook/32CHARACTERSTRING

Now my ecowitt weather sensor data is visible in Home Assistant

AND

I can access my Home Assistant securely using HTTPS.

Problem solved!

Here is my final port forwarding table:

Service Name	External Port	Internal Port	Internal IP Address	Protocol
NGINX HTTPS	443		192.168.1.205	TCP

Currently I am only exposing port 443 publicly.
2 months later, my certificate was automatically renewed by only exposing/forwarding port 443. This was enough that was needed for Let’s Encrypt to renew the certificate.

My hope in documenting all of this is that it helps someone else in the same situation as this was really frustrating to figure out but also rewarding once I solved it.

Cheers.

del13r · July 10, 2023, 11:49pm

The best way to visualise how the NGINX proxy works is:

example of http access direct to home assistant core (no addons involved):

curl -v http://homeassistant.local:8123
*   Trying [my.local.ipv4.address]:8123...
* Connected to homeassistant.local (my.local.ipv4.address) port 8123 (#0)

< Server: Python/3.11 aiohttp/3.8.4

which tells us that we are talking directly with the home assistant server.
NGINX is NOT involved when accessing http on port 8123.

Now we contrast that with an
example of https access via NGINX proxy:

curl -v https://MYHOST.duckdns.org
*   Trying my.external.ipv4.address:443...
* Connected to MYHOST.duckdns.org (my.external.ipv4.address) port 443 (#0)

< Server: nginx

which tells us the https website on port 443 is being served/provided by the NGINX proxy addon.

Loeffel · July 14, 2023, 11:10pm

Hello,

I tried this way. I can access my HA via HTTP and HTTPS at port 8123 without any problems.
I have tried to set up the Ecowitt HP2553 via IP-address and via domain name.
As you, I entered port 8123.
But it doesn’t matters, the integration doesn’t show the device. It just says “No devices or entities”. It doesn’t matter if I wait the 60s interval or even an hour it doesn’t change. I even tried to delete the device and reinstall it and copy the new api path.

I really have no idea where to look. I have no entries in any log, so I can’t make out an error.

del13r · July 14, 2023, 11:32pm

It is not possible to have both http and https on the same port.
Your will only be able to access via either: http://yourhost:8123
OR
https://yourhost:8123
But never both on the same port.
I believe you were misled into thinking that http and https were both active at the same time due to your browser cache.

If you can’t reach http://yourhost:8123/, then neither will the ecowitt device be able to reach the http url to send webhook data.

Loeffel · July 14, 2023, 11:39pm

I just had it set up as it is said in the NGinx docu. Afterwards I could use http://myserver:8123 and https://myserver:8123
Now after a server restart HA isn’t accessible at all. I’ve lost it.

Strange enough all controls via Alexa still work. I can access it via nabu.casa it seems.

del13r · July 14, 2023, 11:46pm

Are you using duckdns?
Do you have port forwarding enabled?
The default port for Nginx proxy is 443.
Did you change this port from the default 443?

Loeffel · July 14, 2023, 11:48pm

Yes, I use duckdns and no I haven’t changed the 443 port.

del13r · July 14, 2023, 11:53pm

After setting up Nginx proxy with the default port 443, you will only be able to access via
https://yourhost.duckdns.org
Once you have setup the port forwarding properly to allow port 443 on your public ip to map to port 443 on your home assistant local ip.

The Nginx https server will no longer allow you to directly access via https to the local ip anymore as it is now configured to only allow https access via the duckdns host you configured here

Alternatively, the only way to access home assistant core locally is via http is to the default port using http://homeassistant.local:8123

Loeffel · July 15, 2023, 12:06am

I can reach HA via the .local address, even though I don’t like it, but now my HA isn’t accessible via internet at all, unless I access it via nabu.casa

del13r · July 15, 2023, 12:07am

Ah, I’m not sure how nabu casa works, but it’s presence might affect things.

del13r · July 15, 2023, 12:13am

If you are paying for nabu casa, why would you also use duckdns and let’s encrypt certificates? Just seems a bit redundant.
Excuse my ignorance as I have no experience with nabu casa, but doesn’t nabu casa do all that stuff for you?

Loeffel · July 15, 2023, 12:14am

If I try to access from external via the normal 443, I get no page load at all.
Then I get an error “400: Bad Request”

If I try to access via 8123 I got:

This site can’t be reached

The webpage at **https://xxx.duckdns.org:8123/auth/authorize?bla might be temporarily down or it may have moved permanently to a new web address.

ERR_FAILED

Loeffel · July 15, 2023, 12:24am

If you wan’t to access via nabu casa you always need to login on their webpage and then press connect to get there. Additionally, when I access HA from internal, he is always going via the internet.
So with nabu casa you always need to login twice. First nabu, then on your HA.

With duckdns I just rewrite the IPs for the duckdns address and I can access it without problems. Especially when I use my notebook I don’t need to use two different links to access HA. I just access the duckdns address and I either go directly to my HA or when I’m not in my network the normal way via internet.

del13r · July 15, 2023, 12:36am

At this point, I would use curl -v commands to see which server has control of the tcp port.

See Post 2 for examples of how to tell the difference between which server (ha core or Nginx) has control of the tcp port.

If you are able get a response from https://yourhost:8123, that indicates you did not demote home assistant core from https to http by removing the ssl related lines under the http section of configuration.yaml

If you are able to get a response from http://yourhost:8123, then that indicates, you did remove the ssl related lines from the http section of configuration.yaml

del13r:

the next thing I did was demote/remove https access:
To do this, I opened up /config/configuration.yaml and removed the following config block and restarted home assistant
http:
  ssl_certificate: /ssl/fullchain.pem
  ssl_key: /ssl/privkey.pem
After a restart, I could now only access Home Assistant using http to either:
http://192.168.1.205:8123
http://MYHOST.duckdns.org:8123

Loeffel · July 15, 2023, 12:47am

Those lines are commented out. When I try to access them I got the HA logo with the “Unable to connect to Home Assistant.” message and the countdown.

del13r · July 15, 2023, 12:49am

The browser might have cached pages which could be misleading.

Did you try the curl -v commands from a command prompt or terminal?

If so, what were the results?

Loeffel · July 15, 2023, 12:49am

Thats my HTTP section:

http:
  use_x_forwarded_for: true
  trusted_proxies:
    - 172.30.33.0
#  ssl_certificate: /ssl/fullchain.pem
#  ssl_key: /ssl/privkey.pem
  ip_ban_enabled: true
  login_attempts_threshold: 5

del13r · July 15, 2023, 12:51am

I noticed a missing subnet mask

Loeffel · July 15, 2023, 12:53am

If I put in the /24 HA tells me that my configuration file is wrong.

del13r · July 15, 2023, 12:53am

Did you want to try commenting out these lines too just incase you somehow inadvertently banned yourself.