I have been able to fix the issue by following the guide here :
By using the console on chrome + a bluetooth dongle (BLE4.0) I am able to add matter devices without any router.