Details

Smaug was built over winter break of my senior year for my uncle who own’s a smog check business (hence the pun). It takes scanned pdf images and extracts the fields into a database for easy retrieval. My uncle was having a hard time remembering which customer’s visited and what they came in for so he requested that I make Smaug to help replace his room full of cabinents and paper invoices.

Gallery

Details

The whole goal of this project was to master the idea that images, along with their color, edges, and geometry, can be treated like any other mathematical object. That is, they can be interpolated and averaged. Though this is a oversimplification, we can use this idea take warp faces together, create the “mean” face of a demographic, extrapolate faces to make caricaturs, and even extrapolate faces through gender, race, and age. Now let’s go in more detail. For exaple, morphing was done by:

1. Manually choosing control points on the source and destination images.
2. Computing an weighted average geometry between the soure and destination image. This was done by taking the sum of (1-t)*source_pixels and t*destination_pixels. The weights are normalized to sum to t.
3. Finding the Delaunay triangulation for one of the set of xy control points and using the same triangluation for the other set of control points.
4. Calculating the affine transformation needed to warp all triangles in the weighted average image their corresponding triangles in the source image. Likewise, affines were found to map weight average triangles to the destination image.
5. Using t-search to categorize every pixel in the average image under the one of the triangles in step 3. As we do this we can look up the affine transformations we found in step 4 to color this pixel using inverse warping.
6. Repeat this for a bunch of t's in a continuous range, lets say [0, 1] to create a smooth animation from source to destination image.

The results of face morphing are shown below. If you’d like to know more the full project page can be seen at my original project page .

Gallery

Details

Below, in the Gallery, are examples of images that have had their widths reduced by 50%. The originals along with the 20% intermediate can be seen here. using gradient magnitude seam carving energy function. The gradient magnitude function effectively computes a differential between adjacent pixel values so it allows our dynamic programming algorithm to avoid edges- regions considered to of high energy and particularly noticable to human eyes. As such, low energy seams, that is seams without many edges, will be removed as the image rescales.

Interestingly, we can extrapolate this idea even further. The idea of low and high energy seams can be utilized to expand images and remove objects. All we have to do is define an appropiate energy function. The results and methodology can be seen in the original project page .

Gallery

Details

I animated a virtual skeleton arm to move along a path using C++ and inverse kinematics. Inverse kinematics basically takes in a point in space and calcuates the rotations neeeded to move the all the joints in an arm to the given point (as opposed to forward kinematics which uses user specified angles to move a joint). The rotations were calculated using a numerical method consisting of calculating the pseudo inverse of the Jacobian matrix. Out of range targets were also handled along with arbitrary number of joints and length.

Gallery

Details

The goal was to take a given set of vertices, specified as a .obj or .bez file, and render it as an object via subdivision. Two subdivision techniques are supported: adaptive tessalation and uniform subdivision. Basically, the vertices specified by the input are grouped as control points for Bezier curves which are in turn grouped into Bezier surfaces. 4 control points specify a Bezier curve and 4 Bezier curves specify a Bezier surface.

The challenging part was making these surfaces look smooth and continuous. If we are given round, curve objects we should make it look smooth by breaking each Bezier surface into smaller Bezier surfaces. This is called subdivision. The more we subdivide the smoother and prettier the rendered object is. The catch is that the more we subdivide the more expensive it becomes (exponential, actually). The two techniques we used were:

• Uniform subdivision: break surfaces into same size patches regardless input curvature, basically split the object into a mesh/grid-like fashion
• Adaptive tessalation: recursively break surfaces into smaller and smaller triangles untill the each triangle passes the flatness test (flatness testing is to ensure there is no cracking between adjacent surfaces)

Gallery

Details

So what is ray tracing? Ray tracing, in a nutshell, is a recursive technique used to color an image and make pretty pictures. The basic algorithm goes like this: for every pixel, shoot out an “eye ray” and let it bounce around your virtual scene- once the eye ray is done bouncing, color the pixel according to the colors the eye ray picks up along the way. If that doesn’t make sense, think of the eye ray as a snowball. For every pixel, you throw a snowball. This is a elastic snowball so it’ll bounce around the scene- whenever it hits something the snowball gets slightly stained by the objects color. For example, if the snowball hits a red object and then a blue object- it should be colored magenta. Once the snowball finishes bouncing- that’s the color I decide to color my pixel. This is a really rough analogy- there are many caveats but many simplifications were made for explanation purposes. You might notice that this is actually “backwards” to how real life works:

• In real life, light strikes an object and bounces around a scene until it reaches your eye.
• In ray tracing, your eye sends out a pretend ray and it bounces around until it reaches a light source.

Gallery

Details

Yelp Wannabe was the first personal project I did outside of school and outside of a hackathon. I wanted to see if it was possible to rank businesses, specifically restaurants, based on their written reviews. I was curious to see if they matched up with Yelp’s top 10 ranking that weighs the star score, location, and cost.

I do not remember my motivation for using Python’s Beautiful Soup module to manually scrape the website food reviews but it was probably due to the fact that I didn’t have access to the Yelp API at the time. Or I was just an idiot- which probably the actual reason. Anyways, for every businesses (up to a total of 200, I think), I parsed all their reviews and outputed a ranking based on the written review. My first attempt was the simplest- map every word to a sentiment score from a massive sentiment dicitonary I found online.

This naiive approach actually didn’t do that badly but it got easly confused by words of neutral or nuanced connotation. For example, words such as “sloppy” could indicate that a restaurant was messy or unclean but if the business specialized in sloppy Joe’s they would get unfairly penalized. Basically, the naiive method sucks for obvious reasons- it didn’t look at context, parts of speech, or structure. Pretty much, it didn’t perform sentiment analysis at all.

At this point, I was really lazy so I decided to use Python NLTK for sentiment analysis. Python NLTK utilizes Twitter sentiment and movie reviews in order to train a Bayes’ classifier. At the time, I didn’t know anything about ML so it boiled down to me reading learning a lot of new concepts. Python NLTK turned out to be quite powerful out of the box and pretty much negated all the effort I put into find and stiching together massive sentiment dictionaries. At the end of the day, Yelp Wannabe actually was able to match 6 out of 10 Yelp’s top 10 list. Though this is not that impressive, it was pretty fun to learn about classifiers and scraping in the process.

Details

Viewfinder was built at Facebook Hackathon with my good friend and housemate Jason Won. It was my first hackathon and the first time I ever tried web development. Neither of us did not even know how to use Flask, Heroku, or Javascript. But it was so fun that this hackathon ended up being one of the primary reasons I decided to switch from Bioengineering to EECS. This amazing experience made me truly understand how powerful and easy it is for a person with a computer and internet access to build amazing, fun apps.

Suprising the Heroku app is still running now- you can go check it out here! And if you’d like to see our horrendous code (there’s seriously a git conflict in the README, no joke) take a look here.

Gallery