Try TX to TX and RX to RX on the ESP/RS485-module connection
And TRY USING SOFTWARE UART! As JFK344 did, that finally fixed it for me:
uart:
id: mod_uart
tx_pin: D2 #this line
rx_pin: D1 #and this one, use these pins (others could work, but these are fine)
baud_rate: 9600
stop_bits: 1