Week 2: Expressive Output

We played with a variety of traditional (pens, paints, markers) and non-traditional (egg shells, hot sauce, toothpaste) art materials in class this week to collaboratively create a very colorful and messy mural. It was a fun lead into a discussion about the degree and range of our expression with the tools at hand. How might we capture the analog data of our physical gestures and express it in digital spaces? Afterwards, our in-class code examples incorporated ranging values from our phone sensors (rotation, acceleration, pressure) and from our laptops (the speed of typing) and opened the door to different ways of capturing and translating more expressive types of user input. 

For the assignment to make one of our Hello World examples more expressive, I decided to combine my mouseClicked example with one of the in-class examples, pollacky. This week’s examples had both input and output screens: in pollacky, a user visits the input screen and tilts their phone to change the direction of a line drawn on the output screen. In addition, the tilt speed modify’s the line’s stroke weight and shaking the phone triggers ellipses to be drawn. Called blobs in the sketch, these mimic the drips and drops of paint, and the greater the force of the shake, the larger they are in size. For my expressive sketch, I set my goal to replace the action of clicking the mouse to render ellipses with the act of shaking a phone (by taking advantage of my mobile’s accelerometer). I also wanted increases in force to increase ellipse size and opacity.

Combining sketches proved to be very useful because I’m still wrapping my head around the syntax and functionality of the server and socket code in our example sketches, specifically how and where the data is communicated between the input, server, and output files. I was forced to go through each line of the pollacky example to figure out what code was necessary for my goal and the reasons why. It was very helpful to understand it all by stripping away what I did not need. 

It was also good practice to break down the process into small steps. Here’s a summary:

Step 1: First, for the input display, I figured out how to replace the four quadrants that had previously lit up upon tilting the phone with an empty ellipse that moved about the screen accordingly. The were no changes to the server file in this step, and the only change to the output file was to slow down the velocity because I realized data was streaming in way too fast. 

Step 2: I realized that if the user was going to generate ellipses on the output screen, they would need some way to orient themselves within the space of the output screen. So next, on the output file, I replaced the line with another empty ellipse. Of course, now it looks like a line of of ellipses moving across the screen. 

Step 3: For step three, I figured out how to increase the size and opacity of the blobs when a device is shaken with greater force. Similar to my sketch from last week, ellipses are filled with random hues. I noticed that multiple ellipses are rendered in the same location for every shake.

Code on GitHub
Remix on Glitch
Play here Input (use mobile) & Output

That was it! Except that it looks weird with the line of tiny ellipses all over the canvas between the bright, colorful blobs. Ideally, I would love for the empty ellipse on the output screen to ditch it’s never-ending tail and act like a mouse. To do that I would have to clear the background repeatedly (as I do in the draw() function in the input file), but that would also immediately clear out any drawn blobs. So I experimented with two different graphics objects with different background settings—drawing the empty ellipse to one and the blobs to the other. I never got it working quite right and the colors shifted in a wonky way, but I did realize that the socket.on event listeners, for example the “tilt” and “shake” listeners, act like draw() loops because they are always actively listening for and serving up incoming data. 

To further aid my understanding I outlined the events in the main files of the program, although I still have gaps with regards to how the users' registry works:

Input (input/sketch.js)

  • Open and connect input socket
  • Listen for confirmation of connection
  • In setup(), create canvas according to windowWidth & windowHeight
  • In draw()
    • Calculate rotation based on tilt of device (ignoring if the phone is turned over)
    • Map rotation to current window’s dimension
    • Draw ellipse at a position based on tilt of device
    • Emit tilt angle to the server (‘tilt’)
  • In deviceShaken()
    • Calculate blop/ellipse size based on changes in acceleration
    • Emit blop size to the server (‘shake’)

Server (server.js)

  • Create an Express server listening at a particular port
  • Tell server to look for files in the public directory
  • Create a socket connection using the socket.io library
  • Declare the input namespace
  • Listen for input browser clients to connect
    • Once a client connects, listen for tilt data (’tilt’), and if this data arrives,
      • emit it to output clients the tilt data along with the ID of the socket through which it arrived
    • Also listen for blop data (‘shake’), and if this data arrives
      • emit it to output clients the blop data along with the ID of the socket through which it arrived
    • Listen for input clients to disconnect and if a client disconnects, emit this to output clients (‘disconnect’)
  • Declare the output namespace
    • Listen for output clients to connect
    • Listen for output clients to disconnect

Output (output/sketch.js)

  • Open and connects output socket
  • Listen for confirmation of connection
  • Create an empty users’ object for key-value pairs to keep track of input user data (user registry)
  • In setup()
    • Create canvas according to windowWidth & windowHeight
    • Listen for movement (’tilt’) data from server, and if data arrives,
      • Create a new user in the registry if the socket.id for that user is not currently in the users’ registry
      • Using tilt data, set the velocity
      • Update user data (I'm sure how this operates here)
      • Add velocity to current position to get new position
      • Draw an empty ellipse at the new position
    • Listen for blop data from server (‘shake’), and if data arrives,
      • Update user data or create a new user
      • Draw a blop/ellipse at user position according to blop size
  • Listen for input clients to disconnect (‘disconnect’) and remove disconnected users from the registry