HW5 - 3101 Matlab -- (INSERT NAME and UNI HERE)
Due 11:59pm on Tuesday, April 15th, 2008
Contents
Goals
Practice with reading in a dataset, analyzing it and visualizing results. Basic dimensionality rediuction, clustering, and classification using machine learning.
Directions
Please fill in your name at the top of this page and write your answers under their corresponding question. When you are done, publish this file to html using File > Publish To HTML (or the command publish('hw1.m', 'html') and create a new zip file that is labeled with your UNI and homework number, in this format: bs2018_hw1.zip, that contains this original m file, and the html directory created from publishing the file. Make sure the html file adequately shows your work, (has images, etc.), and then email this file to cs3101@gmail.com, making sure to include your name, and uni in the subject.
Problem 1 - Loading USPS digit data
1.1
Download the file hw5digits.mat from the class website, and load this file into your workspace. List the variables in your workspace.
1.2
The variable X contains 1000 images of handwritten digits from 0-9 collected by the United States Postal Service. Each digit is represented as a 64x64 pixel black and white image. However, each image has been vectorized, meaning each row of the images has been concatenated together to make a flat vector representation of the image. Get the 5th image in X as a vector and store that image vector in a variable called v. v should be of length 256x1 Do not display the output.
1.3
Use the reshape command to make v into a 16x16 matrix, and plot this matrix as an image in figure(1) in black and white.
Problem 2 - Dimensionality reduction
Currently each image can be thought of as a single point in a 256 dimensional space, where each dimension corresponds to the intensity of the image at that single pixel location. If we want to plot all of these 1000 points using only 2 dimensions, so that images that are similar are placed next to each other, we need to reduce the dimensionality of the data from 1000 to 2 without losing a significant ammount of information.
2.1
Let's start by computing the similarity between all of the 1000 images in X. The vectorized representation of X makes it simple to do this. We can use the matrix inner product between two image vectors as a measure of similarity. What is the inner product between the 1st digit and the 2nd? What is the inner product between the 1st and 3rd?
2.2
Use matrix multiplcation to compute the inner product of X's transpose and X, so that you get a matrix A which is of size 1000x1000. A can be thought of as an affinity matrix. Each value of A corresponds to computing the inner product between two vectors in X, and also represents the similarity between those two vectors; big values in A correspond to a pair of images that are very similar. You should be able to compute A with one single matrix multiply in one line.
2.3
Plot A as an image, use the jet colormap
2.4
Principal Component Analysis (PCA) allows us to take this high-dimensional data: 1000 points in 256 dimensions, and reduce the dimensionality while preserving as much information as possible. Let's start by computing the eigenvalues and eigenvectors of A, call the eigenvalues D, and th eigenvectors V.
2.5
Set d equal to the diagonal of D, then show a bar plot of the last 10 elements of d. These are the 10 biggest eigenvalues of A.
2.6
Each column of V is an Eigenvector of the matrix A, meaning that if we want to plot the data using only 2 dimensions, the best 2 dimensions to choose correspond to these last 2 eigenvectors (the ones with the biggest eigenvalues). Store the last column of V in the variable x, and the second to last column of V in the variable Y
2.7
Now we have 2 numbers to describe every image instead of 256, so we can plot the data in a 2D plane with one point per digit image by using the variables x and y. Display a scatter plot of x,y. Set the scatter plot so each dot is filled in, and not empty.
Problem 3 - Clustering
3.1
We know that the data has 10 clusters, each cluster corresponding to a digit between 0-9. The goal of clustering is to give each point a label such that all the points with a certain label are more similar then points with different labels. We can use matlab's built-in clustering tool called K-means. Apply K-means to X transpose, to get a list called C, where C labels each point in X with a value from 1 to numClusters representing which cluster that point is in. numClusters should equal 10.
3.2
Plot a new scatter plot, where each point has coordinates in x and y computed from PCA above, has size 50, and has the color determined from C. Use the colormap jet. You should be able to see how different clusters of points are grouped in similar areas on the map.
Problem 4 - Classification
Now assume that someone gives you the ground truth lables of all the images in X, can we classify new images based on what we have seen before? You will write a simple 1-nearest-neighbor classifier to accomplish this.
4.1
Download the file hw5trainlabels.mat from the website and load it into the workspace.
4.2
The variable Y contains all of the labels for each image in X. Use Y to find all images of 3s, average those images together, and plot the average 3 in figure 3 as a 16x16 black and white image.
4.3
Download the file hw5classify.mat from the class website and load it into the workspace
4.4
Write your own function called getDists, this means making a separate file called getDists.m and proplery structuring that file as a function. getDists should accept as input a single point as a vector of size 1xD, and a set of points as a matrix of size NxD, where D can be any number of dimenions, and N can be any number of points. getDists() should return a new vector of size 1xN that contains the euclidean distance from the single point to N other points. Make sure getDists is properly formatted and commented, and make sure to include getDists.m file in your submission zip file.
4.5
Xtest contains 20 new digit images that we haven't seen before. For each image in Xtest, find the image in X with the closest euclidean distance by using your getDists function. Use the label from this closest point as your prediction label. Create a new variable called predictions that contains your predicted labels for the 20 new images. Display the predictions variable.
4.6
Download from the class website and load the file hw5labels.mat. Use the variable Ytest to determine the accuracy of your prediction as a percent.