Weeks 14 & 15: Heart Sync

Project Description
Visualize your heartbeats and see if you can sync them with others! Heart Sync pictures the heartbeats of two people and encourages them to match their heart rates. Participants may also adjust the color of their pulse information to co-create the shared color field between them. Depending on their moods or personalities, it's either a calming, meditative moment or a playful, competitive game for two. All ages welcome.

Materials, Tools, and Process
Frequent playtesting at ITP provided valuable feedback, and since Week 13 I replaced the knob potentiometers with sliders, programmed the ellipses to pulse, and added rings to emerge if the beats per minutes (BPMs) aligned. I also built an enclosure for all the components. Detailed descriptions of all follows.

The Physical Components

FritzingHeartSync.png

1 Laptop
1 Eizo ColorEdge CG222W Display (or another monitor that lies flat)
1 Arduino Uno
1 Half-size Breadboard
2x Pulse Sensors
2x 10K Ohm Slide Potentiometers
2x Slide Potentiometer Knobs
2x Phillips Rounded Head Screws, M3 x 0.5 mm Thread, 6 mm Long
Stranded Core Wire
Soldering Iron
Solder
Heat Shrink Tubing
Black Spike Tape 1/2" Thick

The physical guts of the project are really quite few. The pictures above show how the sensors attach to the Arduino (schematic created with Fritzing) and the hand panels. I soldered the slide potentiometers to stranded core wire long enough to stretch from the panels to the microcontroller which, when all assembled within the final enclosure, tucks into the most perfect opening underneath the monitor near the monitor's power and VGA ports.

01-171203IMG_8098.jpg

2x 9x12" Clear Acrylic 1/8" Thick Sheets
2x Round Stickers 5/16" Diameter
Blue Painter's Tape
Black Spray Paint Matte
White Spray Paint Glossy
Digital Caliper
Adobe Illustrator
Laser Cutter
Heartbeat Icon (by Deemak Daksina S from the Noun Project)

For the hand panels I started with clear acrylic sheets. For each one I measured the location of the pulse sensor and covered that area with a round sticker. Then, I sprayed the entire sheet with black spray paint. Once dried, I covered the panel with blue painter's tape and headed to the 75 Watt Epilog laser cutter. There I cut the mounting holes and an opening for the slide potentiometer (four passes at speed 15, power 100, frequency 100) and etched away the blue tape and black spray paint to reveal the hand print and heart pulse icon (four-five passes at speed 100, power 40). Next, I sprayed the panel with glossy white spray paint and once dry, removed the blue painter's tape. I also removed the round sticker, revealing a clear spot under which to attach the pulse sensor. 

30 x 40" Black Foam Board 3/16" Thick Sheet
30 x 40" Black Foam Board 1/2"Thick Sheet
Utility Knife (and many fresh blades)
12" and 24" Metal Rulers
Cutting Mat
Paper
Hot Glue Gun
Glue Sticks

After determining the length of the monitor and my hand panels I measured and first cut the four sides of the box from the 1/2" thick foam board. Then, as practice for the top and before using the thinner foam board, I measured and cut a piece of white paper. The paper template helped me to determined the spacing I wanted between the hand panels, the monitor, and the surrounding border edge. With that set, I measured and cut the top from the 3/16" thick foam board. Next, I cut supports for the hand panels to run parallel to the monitor; instead of securing the panels to the enclosure they will rest on top of these supports. (Later I filled the entirety of these cavities with foam board for extra support.) After cutting out a notch on one of the long sides to allow for the monitor and Arduino cables to exit, I finally glued everything together for a very snug fit around the monitor.

The Code
Arduino Sketch 
As noted in my previous project post, for I modified the original sketch, TwoPulseSensors_On_OneArduino, from the Pulse Sensor Playground. This sketch provided crucial pieces of information via asynchronous serial communication for my Processing sketch from each pulse sensor: the raw signal data, the heart rate or beats per minutes (BPMs), and the moment of each beat. The latter I added myself to increase the size (strokeWeight) of each "heart" visualization and to create essential timers within my Processing sketch. I also added code to capture changing values from two slide potentiometers; in Processing, I mapped these changing values to a sliding range of colors such that each participant might change the color of their pulse information. Finally, I commented out the code to light and and pulse LEDs on the breadboard. They were useful during the preliminary testing phases of the project, but no longer so.

Processing Sketch 
My Processing sketch also modifies an original sketch by Joel Murphy, PulseSensorAmped_2_Sensors, and visualized the incoming pulse sensor data from the Arduino in several ways, some of which I detailed in earlier posts. Here's a summary of everything.

From each sensor, the raw signal data is used to draw pulse waveforms across the screen. As well, it is mapped to the radii of ellipses for each participant's "heart." This is a change from Week 13, and the result is that the ellipses appear to "breathe" as they pulse (instead of turning on and off abruptly due to changes in strokeWeight only).

The moment of each beat is still used to increase the strokeWeight of the ellipses in each heart. In the original code, though, this moment was set by the BPM value, which seemed off. However, I learned from this portion of the sketch how to set timers for events elsewhere in my sketch. For example, I also use beat detection from each sensor to determine if the ellipses (hearts) should move across the screen. If the program senses BPM values from each user, then it compares the rates and moves the respective hearts closer or farther away from one another: the smaller the difference in rates, the closer hearts to the center of the screen. The moment of syncing actually occurs within a threshold BPM difference of 0-3. (At one point I also used the BPM timers to display BPM values on the screen while testing the above but since removed that data in favor of a cleaner and less game-like display.)

For the changing radii and x-coordinate locations of the ellipses, I continued to play with the lerp function to create smoother transitions within the animations.  

As I mentioned above, I mapped the incoming values from the slide potentiometers to changes in hue to allow each participant to alter the color of their pulse information. 

Finally, I wished for participants to received a stronger, more gratifying visual indicator at the moment of syncing. I envisioned rings emerging at that moment and learned that I could achieve this with a simple particle system. Thanks to Dan Shiffman for pointing me in that direction and for his resources available here, here, and here. Now when the ellipses align in the center, concentric rings ripple and fade out across the screen. These rings are actually ellipses themselves but with no fill and thick strokeWeights. Over time their radii increase as their strokeWeight and alpha values decrease, eventually disappearing altogether.

Conclusions
What did I learn? Wow, I learned SO MUCH this semester from how to work with sensors and output data, to how to program more extensively in P5 and Processing, to how to fabricate enclosures that communicate the functionality of my projects. Along the way I tried to note major progress updates on my blog. It's impossible to capture everything, but here are some highlights.

I learned to work with the pulse sensors by breaking down the code in both the existing Arduino and Processing sketches. It was difficult, and it took a long time. But I as stayed with it, I eventually saw that the data from both sensors was wrapped up in an array. As I worked out the visualizations in Processing, I uncoupled each sensor from this array because it was easier for me to visualize the variables as separate entities and consider how I wanted them to interact. Along the way, I learned more about syntax and how to use new functions and create my own. I tried to organize my code at the end (and where I could, re-wrap the sensor data back into an array), but it still feels quite messy to me. However, so was my learning throughout this process. I need to keep practicing to to communicate my ideas succinctly in this medium. (After all, it's only been 15 weeks since school started!)

More importantly I discovered that I really enjoy creating open-ended, collaborative, and playful experiences for people in which participant interaction is the content. I'm interested in the conversations and relationships that are initiated or supported by the technology. In this regard, Heart Sync is similar to my Social Drawing Sol LeWitt shared drawing canvas and to some extent, to my Multiples of Five Modular Sculpture, and I'm excited to explore these ideas further at ITP. 

The most fun aspect of realizing this project has been watching people come up with different strategies--completely unprompted by me--to match their heart rates, including: sitting still and quietly, breathing deeply in unison, running in place, holding hands, and listening to music. Several times I've heard the word, play. Can we play?! I want to play. I had so fun much playing! These moments are the most gratifying.

Thanks!
A huge thanks to ITP faculty Ayodamola Okunseinde, Dan Shiffman, and Ben Light, and ITP residents Lisa Jamhoury, Aarón Montoya-Moraga, and Davíd Lockard, for all of their advice and technical help. And of course to all my fabulous classmates who provided ongoing invaluable feedback on the floor. I can't wait to see what we all create in the next year!