Augmented board gaming

Background and idea

I’ve always been passionate about blending technology with traditional hobbies to create unique experiences. Recently, I embarked on a project that aimed to combine my interests in smart home technology with my love for board gaming. The concept was intriguing, and I was eager to explore innovative ways to enhance our board gaming sessions through smart technology.

My initial step was to conduct a web search to see if anyone had already ventured into this niche intersection. Surprisingly, my search results were quite limited, but I did stumble upon a podcast episode (Link) that caught my attention. The podcast discussed an inventive approach to selecting board games for game nights. The hosts described a system where an NFC reader was used in conjunction with specially prepared game boxes. This setup allowed the NFC reader to automatically determine which board game would be played during a game night, based on the proximity of the box to the reader. Then the light setting would be adopted and a theming playlist would be started.

This concept was fascinating, but there was a slight hitch in adopting it directly for my project. Our gaming group tends to lean towards playing extensive and complex board games that can span weeks, months, or even up to a year. At the time of my research, we were all eagerly anticipating the release of the German version of “Frosthaven,” a game known for its depth and complexity. Given the long-term commitment required for games like “Frosthaven,” the podcast’s solution seemed less applicable to our needs. The idea of using technology to enhance our board game nights was still very appealing, but it was clear I needed to adapt the approach to better suit the nature of the games we enjoy.

Frosthaven

Frosthaven is a popular tactical combat and dungeon-crawling board game that features a unique element infusion system which plays a crucial role in its gameplay. This system is used to enhance certain abilities or actions within the game. Here’s an overview of how the element system works in Frosthaven:

Elements

Frosthaven uses six elements:

  • Fire

  • Ice

  • Air (also known as Wind)

  • Earth (Nature)

  • Light

  • Darkness

These elements can be infused into the environment during a player’s turn or by certain monster actions. Once infused, these elements can be consumed by players or monsters on subsequent turns to power up abilities or trigger special effects.

Infusing Elements

Elements are typically infused using specific actions on ability cards. These actions will have symbols representing one of the six elements, indicating that performing the action infuses that element into the environment. When an element is infused, it is moved from the “Inert” section of the element board to the “Strong” section.

Consuming Elements

On subsequent turns, players can consume an element that has been infused into the environment to enhance an ability on one of their cards. This usually grants additional effects or increases the power of the ability. Consuming an element moves it from the “Strong” section to the “Waning” section or from “Waning” to “Inert” if it was already in the “Waning” section at the start of the turn, indicating it cannot be used until it is infused again.

Elemental Timing

The timing of element infusion and consumption is critical. Elements infused during a player’s turn do not become available for use until the beginning of the next player’s turn, preventing the player who infused the element from immediately consuming it. This encourages strategic planning and teamwork, as players can set up powerful combos by infusing elements for their teammates to use.

Strategic Importance

The element system adds a layer of strategy to Frosthaven, as players must carefully plan their actions and coordinate with their teammates to make the most of the elements. Choosing when to infuse or consume an element, as well as which abilities to enhance, can be crucial to overcoming the game’s challenges.

Interaction with Monsters

Monsters can also infuse and consume elements, as dictated by their ability cards. This adds another layer of strategy, as players may need to consider how their actions will affect the elemental availability for both their team and their enemies.

In summary, the element system in Frosthaven is a dynamic and strategic component of the game that encourages careful planning and teamwork. Managing elements effectively can be key to success in the game’s challenging scenarios.

My take on environmental/augmented board gaming

It was apparent that our game nights for the foreseeable future would be dominated by Frosthaven. One point of the Home Assistant Podcast caught my attention. They talked about the use of reed switches connected to an ESP32 microcontroller to determine the absence of boardgame packages. Using this setup you can check the state of multiple items at a cheap price point.

Inspired, I envisioned a project that would intertwine the physical and digital realms of our board gaming experience. The core idea was to modify the elemental board of Frosthaven, employing reed switches to digitally capture its state. Initially, the concept was straightforward—based on the board’s status, various lamps around the room would illuminate in different colors. However, as often happens with projects born from a spark of inspiration, the scope expanded significantly.

Driven by enthusiasm and the possibilities that lay ahead, I embarked on this ambitious endeavor. The project evolved to not only change the lighting based on the game’s elemental board but also to integrate further into our smart home system, creating an immersive environment that responded dynamically to our gameplay.

The system in action

Video
Unfortunately, the video doesn’t accurately showcase how impressive the lighting system truly is. The camera struggles with maintaining focus and continuously attempts to adjust to the changing lighting conditions. This is particularly noticeable with the Light and Darkness elements, where the video barely shows any change. In reality, however, the room’s lighting becomes warmer or darker, significantly enhancing the atmosphere in ways that the video fails to capture.

For this video, I used only two lights to demonstrate the system. Normally, the room includes the main light and six additional lights distributed throughout the space.

Thanks

I owe immense gratitude to Anthony Jestin, whose project presentation inspired my venture. Jestin’s innovative blend of technology and board gaming not only provided a foundation for my project but also served as a guiding light, leading to a significantly enhanced Frosthaven gaming experience. Through his pioneering efforts, our game nights have transformed, merging the excitement of board gaming with the enchantment of smart technology for an unparalleled experience.

Hardware/Sensors

The board

The elemental board of Frosthaven features six different types of elements, each with three possible states: “inert,” “waning,” and “strong.” This results in a total of 18 states that need to be monitored. However, in my project, I decided to forego checking the “inert” states directly, operating under the assumption that if both “waning” and “strong” are inactive, the “inert” state must be active by default.

The board included with the game does not specify exact positions for where the elemental markers should be placed. This lack of precise positioning makes it challenging to accurately determine the state of each element. However, we already played „Gloomhaven" using a third-party board made by Laserox. This aftermarket board features rails that guide the placement of the markers, thereby defining the positions of the markers relative to each state with sufficient precision.

To trigger the reed switches, magnets had to be attached to the markers on the board. I was reluctant to stick magnets onto the back of the cardboard markers included in the game. I generally preferred the wooden markers from Gloomhaven, but I also didn’t want to stick magnets onto them or drill holes into their backs to insert the magnets.

I decided to print my own markers instead. I found suitable templates at Element Tracker for Frosthaven Gloomhaven JOTL by ZippyZip - Thingiverse. Using OpenSCAD, I cut a circular recess into the backs of the 3D models to directly glue in the magnets.

Reed switches

Reed switches are electromechanical devices that operate using a pair of ferromagnetic reeds encapsulated within a glass or plastic tube. These reeds act as contacts that come together and close a circuit when exposed to a magnetic field. When the magnetic field is removed, the reeds separate, opening the circuit. Reed switches are known for their reliability, longevity, and ability to switch high voltages and currents. They are commonly used in various applications, including sensing, automotive, telecommunications, and home appliances, due to their simplicity and effectiveness in detecting magnetic fields.

Initially, I utilized small glass reed switches for my project. Glass reed switches are delicate and require careful handling to prevent damage. During the processing and installation phase, I encountered challenges with their fragility, resulting in several switches breaking. This setback prompted me to search for a more robust solution. Consequently, I transitioned to using reed switches made from plastic. Plastic reed switches offer a higher degree of durability and resistance to physical stress, significantly reducing the risk of breakage during handling and installation. This switch to plastic reed switches improved the resilience and reliability of my setup, ensuring smoother project execution.

To interface with the reed switches, I employed an ESP32 microcontroller, which easily accommodated the twelve sensors required for my project. The ESP32’s inputs were perfectly suited for this application, allowing for a straightforward connection of the necessary sensors without any issues. Furthermore, I integrated the microcontroller into my Home Assistant environment using ESPhome. This integration not only facilitated seamless communication between the sensors and my smart home setup but also allowed for easy management and monitoring of the system’s status through Home Assistant’s user-friendly interface. The use of the ESP32 and ESPhome proved to be an effective solution for incorporating the physical interactions detected by the reed switches into my digital home automation system.

Enclosure

Naturally, the entire electronic setup needed to be completely invisible to the players to maintain the aesthetic and immersive experience of the board game. To achieve this, I had to construct a housing to conceal the components. Utilizing Fusion 360, I designed two parts that were specifically tailored to encapsulate the electronics while blending seamlessly with the element board. These parts were then produced using a 3D printer, ensuring a perfect fit and finish that matched my specifications.

The reed switches were inserted into the custom-designed enclosure, strategically positioned to interact with the game elements without being noticeable. The electronics, including the microcontroller and wiring, were placed loosely underneath this housing. This arrangement not only kept the electronics hidden from view but also allowed for easy access should adjustments or maintenance be required. The result was a sleek and unobtrusive integration of technology that enhanced the gaming experience without detracting from the game’s visual appeal or mechanics.

1 Like

Sound

Although I started with the lighting aspect of the system, the final version was first realized in the sound component. Therefore, I’ll describe the sound part first here.

In my man cave, I have an Echo Show and an iMac at my disposal for audio playback. These devices can be utilized as sound generators in my project.

First version

In the first iteration of my project, I utilized the Echo Show to play an existing Frosthaven Spotify playlist. Additionally, I selected suitable sound effects for each element from the Amazon sound library. Whenever an element’s state changed to “strong,” the corresponding sound effect for that element was played. However, a significant drawback was the substantial delay of approximately 2 seconds. This delay greatly impacted the immediacy and effectiveness of the audio cues, detracting from the immersive experience I aimed to create.

Unity version

After being dissatisfied with the first version of my project, particularly the delay in sound effects, I started contemplating ways to enhance the audio experience. My goal was to accurately reflect the active elemental states in the room not just visually, but audibly as well.

In my quest for a solution, I sought a system capable of streaming multiple sound effects to a Home Assistant Media Player. During this search, I stumbled upon the project Amniotic (link). Unfortunately, I encountered difficulties getting it to work promptly. Additionally, the lack of a streaming-capable speaker in my playroom led me to abandon this approach.

Having experience with game development in Unity3d, I knew that this game engine inherently offers straightforward options for mixing sounds and music. Motivated by this knowledge, I set out to create a small app using the engine. This app, utilizing WebSocket (link), receives the state of the element board and accordingly adjusts the audio playback. This method allowed me to directly control the sound environment in response to the game’s elemental states, providing a much more immediate and immersive auditory experience.

My background in game development enabled me to quickly devise distinct auditory experiences for each elemental state. Here are the details:

  • Fire: Upon transitioning to the “Strong” state, a soft explosion sound is played, followed by a sound loop of intensely burning fire. This then changes to a campfire sound loop when transitioning to the “Waning” state.

  • Ice: Imagining sounds for Ice was challenging. (Snowfall doesn’t have a compelling sound, and the sound of a blizzard overlaps with the Wind element.) Therefore, I decided to utilize rain and thunderstorm sounds for Ice. Depending on the element’s intensity, sound loops of varying rainfall intensity are played, with thunder sounds occurring at random intervals. Concurrently, the background color of the app changes from black to white for 0.3 seconds during thunder, illuminating the wall lit by the Mac.

  • Wind: For both strength states of the Wind element, simple sound loops of varying wind intensities are played.

  • Earth/Nature: When this element is active, sounds like bird chirping, cricket chirping, etc., are played at random intervals and in the volume corresponding to the intensity of the element intensity.

In addition to these specific sound effects, music fitting the setting is played throughout. Similar to how the Light and Dark elements received special treatment in the lighting domain, they were uniquely addressed in the audio aspect as well. When the Light element is active, the normal music fades out, and something akin to choir singing is played. For the Dark element, mystical music is played. If Light and Dark are active simultaneously, they cancel each other out, and the standard music resumes.

Light

First version

The first iteration of the light-component was controlled through a Node-Red flow. In the room, there is a main light along with several additional light sources. All of these lights are either mood lights from Lidl or Zigbee-RGB bulbs, allowing for a wide range of color and intensity settings.

When there is a change in the state on the elemental board, specific actions are triggered to reflect these changes in the room’s lighting environment:

For changes in the status of the Fire, Earth/Nature, Ice, or Wind elements, one of the additional light sources is randomly selected and turned on in the color corresponding to the element. The brightness of the light is adjusted to match the intensity of the element, whether it’s in a “waning” or “strong” state.

For the other two elements, Darkness and Light, the main room light is manipulated. If Darkness is active, the overall brightness of the room is dimmed to create a more subdued atmosphere. Conversely, when Light is active, the “temperature” of the light is shifted to a warmer range, enhancing the ambiance to reflect the element’s presence.

This dynamic lighting setup not only adds an immersive layer to the gaming experience but also visually represents the elemental changes happening within the game, creating a more engaging and thematic environment for players.

Unity version

A few years ago, I came across a video by Jason Keyser and a colleague, where they dissected various game video effects from League of Legends, breaking them down into their component parts and explaining the concepts behind each effect’s components. I believe the video was from GDC, but regrettably, I’ve been unable to find it since. However, one can find something similar on the VFX Apprentice page (link). From this video, I grasped that different elements can manifest a variety of effects.

Having already devoted considerable effort to my Augmented Board Gaming System, I was eager to integrate this understanding of effects. The system’s first version featured lights that would only shine in various colors.

For the second version, I managed the light control through Unity. Being a software developer, I found programming the controls more to my liking than implementing them via Node-Red.

I eventually implemented the following effects:

  • Fire: This effect was meant to represent a blaze. Hence, a light source is obscured as long as the Fire element is in a “strong” or “waning” state. Throughout, random shades of orange and red, along with random brightness levels, are generated, causing the light to fade into these colors.

  • Ice: Since I had already transitioned the Ice element to rain and thunder, it made sense to me to let lightning flashes coincide with thunder sounds for the ice effect.

  • Wind: Representing wind through light effects is challenging; it requires movement within the light, which is not achievable with simple bulbs. Perhaps someday I’ll acquire a WLED system, and then it might be possible to incorporate a light component for Wind. Currently, no light effect is produced for Wind.

  • Nature/Earth: At random intervals, green lights in the room are gently faded in and out. I imagine wisps or orbs flitting about in nature.

  • Light/Dark: I didn’t conceive a special effect for Light and Dark. I was satisfied with the main room light changing its color temperature and brightness depending on the state of these two elements.

Like in the first version, a light source in the room is randomly selected for each effect.

1 Like

Special

To be continued…

Thanks for the interesting post - mixing in hardware to board game mechanics is something in an early stage, but I particularly like that your implementation looks better than the original and probably plays better too.

I regularly play Mansions of Madness v2 as it allows all payers to be equally involved with out a DM/GM as a tablet assists with a lot of the drudgery of HP counting and turn timers. The tablet also plays background music, so I can definitely see the benefit to set the ambiance.

There are a load of 3d-printable items for MoM, ranging from player organisers (small play area with token slots, card stands, etc), and some 3D physical token replacements, but nothing “haptic” nor “physically augmented”.

The cost of adding something like NFC tags to tokens is probably prohibitive, but I remember a Microsoft demo with an overhead camera/ projector system for gaming which probably used simple QR codes to read the playing field state.

During C19, I ran a physical table top game via conferencing and several cameras - it worked, but having a system “understand” the state might have helped remote players as the card/token graphics are too small for a camera/ codec/ WAN link to pass on.

Using reed switches and magnets to read state is simple and cheap but from a playing perspective, and probably also helps prevent a misplaced elbow knocking the lot and causing a 20 minute discussion about the previous state!

MoM has a re-implementation of the software GM for player generated content, so technically it might be able to integrate physical state with software, but the game mechanics don’t rely on physical states (beyond player stats).

Augmentation doesn’t need to be complex, though. One game with “augmentation” was particularly simple - each turn was a player co-op session with a cassette tape soundtrack giving a specific time pressure for the round. I can see flashing red lights and even a smoke machine turning a board game into something closer to an escape room!

I suspect any augmentation / HA integration needs a game if not designed for it, one at least with something central that needs managing. It looks like you’ve worked out a nice way to make your Frosthaven gaming sessions more immersive - noice!

Look into MCP230xx or MAX 6956 port expanders using i2c. You’ll be able to squeeze out much more from a single ESP :wink:

Tag stickers themselves cost less than 20c on Ali. I’d agree with you that tag readers, on the other hand, are slightly more expensive but still relatively cheap, especially when you consider you’d only need 1 reader per token

Thank you. Yes, you are correct. You would need one central aspect of the game that can be augmented.

I think MoM is just too different in every mission/campaign. We played several games of MoM. We played lots of Arkham Horror LCG and this also would have the same problem. You could try to register low sanity or health and react to this, but is it worth the effort?

Thanks a lot. This has been my first project using ESP32s. I switched last summer to Home Assistant from OpenHAB and got my first ESP32 in January.

But I got infected by the ESP32 fever. They are just so flexible. I think I will integrate many more in my home for presence detection and voice assistants.