Final Project – Documentation Catch-up

Oh boy, I’ve done a lot and haven’t written anything down.  So here’s an attempt to catch up on everything that’s been going on in final project land.

The proposal and first mock-ups can be seen here.

Missing pieces

The protoype shown during the proposal presentation was already using a database.  Parse is amazing.  It gives you a database without having to write a single line of server side code.  In fact, you don’t have to worry about the server at all.  All the effort is on the front end, and Parse gives you really simple ways to interact with the data you want to store.  This is what I was using for data storage since I had no experience with backend and parse provided everything I needed.  Or so I thought.  More on this later.

What happened next?

The flocking was working, but the next steps were to get it flock by type of emotion and or location.  The idea being that patterns would emerge given enough data and time to allow for significant re-flocking.  Without real data (so far it’s all been dummy data input by me), it’s hard to get a feel for how this feature would really work.  So I put aside this for now and took a diversion:

Diversion into 3D and Three.js

This was a bit of wormhole – but I wanted to have a more immersive environment.  Since the idea of this project was to exist in someone else’s mind, however briefly, I thought having these thoughts and/or emotions flying around you in 3D might be more powerful.  So I decided to begin wrestling with Three.js.  I found a great example of 3D flocking, that fortunately was on github.  I forked it and started modifying it.  I tied in my Parse code, fiddled with Three.js for a few days, trying to wrap my head around renderers, cameras, 3D movement, and a handful of other not-so-trivial aspects of working with 3D.  But after banging my head against the wall for a couple of days, I emerged with a not so pretty prototype of emotions floating in 3D.  Unfortunately I don’t have a working prototype anymore so I can’t show it here.  I really need to get better at documentation.  But alas, it was a fruitless diversion as I quickly realized that going 3D muddied the story of what I was trying to get across.  It felt gimmicky, and detracted from the real point.  So back to canvas we go.

Canvas next steps

Based on feedback I got from the class, and from Crit Group, I realized a few main things: 1. asking people directly how they feel is a recipe for disaster.  It’s a super hard question. 2. Jumping off the last point –  giving people a dropdown list of emotions is reductive.  People are multifaceted, and hard to pin down.  So I had to figure out a way to probe more deeply but also give people the freedom to respond how they want.  3. The flocking wasn’t adding as much as I thought to the project.  Maybe I just got bored of it after seeing it for so long, but like the 3D version I felt that it was detracting, rather than adding, to the experience.

To address points 1 and 2, I decided to get rid of the direct emotion aspect of things, and jumping off a few resources that were shown to me, as well as a few great suggestions from the folks in Crit Group, and create a public journal for people to honestly share their inner thoughts.  The idea being that by asking direct questions about all sorts of topics, ranging from mundane, to hyper personal, and having all this information in one public place could create some real powerful interpretations of other people’s experiences.  Obviously the heart of this is having a large amount of data from a diverse set of people.  But that’s a problem for another day – for right now if I can collect data locally to get across the point of the project, that’s enough for me.  But to make things more interesting, I’d need a way to find trends among answers.  To qualitatively describe them somehow.  This is where I stumbled on a Node.js module called Sentiment.

Sentiment Analysis, Node.js, and a world of headaches

Ugh.  Node.js.  That means servers.  That means databases.  I’m way in over my head.  How the hell do I even deploy a node.js app?  What the hell is express?  And what’s this Jade stuff?


Oh my goodness! Now I know what all that stuff is! After three or four long, long, frustrating, and long days of doing nothing but reading, following tutorials, fucking up, reading some more, fucking up some more, I finally was able to build a node.js app using Express, tie it to a MongoDB database, utilizing Mongoose as a wrapper for MongoDB drivers, and deploy the app on Heroku so I could start gathering data.  That sounds crazy, and yes it was crazy.  I cannot express the frustration and feeling of pure confusion.  The fact that when I first deployed to Heroku, everything worked fine, then I realized that Heroku doesn’t serve your app on a stationary server.  They can move your stuff around (they call them dynos, and when you deploy an app you get some space on a dyno, but which dyno you’re on can change).  This mean that the database methods I was using (a node module called diskDB that writes directly to the disk) no longer worked.  If you write to a dyno, there’s no guarentee that you’ll ever see that data again.  Ugh.  So I had to start investigating MongoDB.  That took about 5 hours of reading, and trying things, just to get basic database functionality.  Not ot mention the second I updated my heroku deployment, everything broke and I spend about three hours just trying to get it back to life.  Eventually I realized you can roll back your deployment (it’s just using git commits, after all).  I rolled back to v1 and somehow all was well again.  In the beginning, the way Express routing worked was totally non-intuitive.  I spent almost a day just trying to understand how an AJAX GET request was handled on the server side by Express.  Basically you have to define a route (a url) that matches your GET, then you can access the request (the info being passed from client to server), do some stuff, then pass back the response (the info being passed back from server to client).  All in all, I probably spent 20 or 30 hours just working through this Node.js stuff, and for what?  Just so I could access this Sentiment analysis module.  Was it worth it?


But now why you might think.  The sentiment analysis module is okay.  It takes the string you give it, chops it up into individual words, then checks a database for those words.  The database contains a large list of words that are either considered positive or negative, and assigns them a value based on how extreme they are.  E.g. hate has a value of -3, while pensive has a value of -1.  Overjoyed has a value of +4.  It then just adds up all those values and spits back a number at you.  Not great.  It only gives a vague sense of positive or negative, nothing more.  And if you have a mixture of positive and negative words, you’ll get roughly zero sentiment, which isn’t exactly right.

So what was it worth it?  Because I learned a TON in the process.  I feel comfortable now that I could learn to create a full fledged web app off Node.  A week ago I thought it was all magic.

Where I am now. And where to next?

My project currently exists HERE

Screen Shot 2015-11-25 at 8.02.11 PM

What you see is bunch of hard to see circles arranged in a line.  If you click on the button in the upper right, you’ll be promped with a question (you’ll get a different question every single day).

Screen Shot 2015-11-25 at 8.03.40 PM

Next Steps

The colors theoretically match with the sentiment score of each entry.  But I need to fine tune the settings that there’s a wide range of colors represented, otherwise it will just look bland.  To do this properly, I need real data.

Other than collecting data, I’d like to make the database searchable and filterable by sentiment score, location, date, and question.  But we’ll see if I can get there by next thursday.

Leave a Reply

Your email address will not be published. Required fields are marked *