Technical Journal - Dataset

       Ramya Korlakai Vinayak, a graduate from Caltech under Prof. Babak Hassibi, graciously offered us a dataset that was labeled using MTurk (Amazon Mechanical Turk), a crowdsourcing marketplace. The folder contained four MatLab files: Birds5EdgeQuery300workers, Birds5TriangleQuery285workers, Dogs3EdgeQuery300workers, and Dogs3TriangleQuery320workers. The naming convention is as follows: Animal Classified + Number of Breeds + Comparison Query Method + Number of workers. It is still vague as to which one we will use for our project but they are all very similar in some sense. Each file has 6 variables:
  • Adj: Adjacency matrix under standard rules (and -1 = unobserved)
  • AdjWithMultiples: Adj with entries observed more than once due to random queries
  • CAdj: logical, mapping matrix to tell if an edge is observed (1) or not (0)
  • count: number of edges observed (no multiples)
  • groundtruth: actual breed of the animal
  • m: number of nodes in the graph

Image result for mturk
Above demonstrates how MTurk works

       This dataset is useful because it does a lot for us already. Rather than waiting for god knows how long, 14773 edges are labeled through comparison (either by pairs or triangles). One small thing we will use on the dataset is making unobserved edges into no-edges. We do an entrywise multiplication of Adj with CAdj which leaves us with a cleaned adjacency matrix. This way, we avoid -1's and assume they are 0.

Image of my MatLab window. The Command Window displays the calculation we would need to do to clean the dataset.

       A big problem at hand is knowing what file we should work on. The four are offered but I would rightfully assume that one will be focussed on. It is a bit unclear as to what are the pros and cons of using an edge query and triangle query which factors into our final decision. My team and I are also fairly new to MatLab so much of the steps we need to take to run spectral clustering or some algorithm are unclear.

       In conclusion, the presented dataset consists of adjacency matrixes with a total of 342 (for birds) or 473 (for dogs) nodes. They form graphs that can be clustered into 5 (for birds) or 3 (for dogs) classes. We plan to implement semi-crowdsourced clustering as our method of choice.

Comments

Post a Comment

Popular posts from this blog

Technical Journal - Research Papers

Image
On the Efficiency of Data Collection for Crowdsourced Classification :        The paper aims to explain the accuracy gaps between popular collection policies (state-action mapping) on crowdsourced data. The quality of crowdsourced data is often highly variable so it is suspected that it is the cause. However, studies show that the policies used to collect such data have a strong impact on the accuracy of the system. The first theoretical explanation of the accuracy gaps is that the non-adaptive uniform allocation, and the adaptive uncertainty sampling and information gain maximization. Done with the representation of the collection process in terms of random walks in the log-odds (log of odds of guessing correctly/incorrectly) domain, the author derives lower and upper bounds on the accuracy of the policies. The bounds accurately tell us by how much do two adaptive policies trump a non-adaptive one. It is believed that techniques used by them are applicable to addi...
Read more

Journal

Image
On Tuesday, March 24th, we had a meeting with Professor Hassibi over Zoom. Prior to our talk, we had issues with our code regarding the fact that it didn't produce the results expected. He told us to find and graph the eigenvalues of the graph Laplacian, as that would indicate the number of clusters found and value you can refer other datapoints to. He noted that graphing the corresponding eigenvector would indicate whether a cluster had been isolated or not. On that note, it was up to us to tinker with values for a bit and try different algorithms like K-means. Below is a video of the meeting with Professor Hassibi: This week has been very interesting, to say the least. As you may know, COVID-19 has just taken the U.S. for a turn and has put a pause on daily life, forcing businesses to adapt, overcome, or fail in some cases. For us high schoolers, classes have migrated to the Zoom platform. With it followed teachers with bad internet service, leading to indecipherable dialect...
Read more

First Month

Image
The first four weeks of this Caltech Stem class have been interesting, to say the least. Much of the material we went into have been all over the place with various objectives. We started off with teamwork exercises, watching college students present their research, replicating one for ourselves, writing a concept map, and going to Caltech to meet Dr. Hassibi. We had two, teamwork workshops for the first two weeks. One of which was the dominoes problem where each student was given two dominoes and we were instructed to get three dominoes per person by exchanging dominoes for chips and chips for dominoes. Instructions were very vague other than that everyone must have a domino to "survive" a round. We struggled in the first few attempts but through several days of discussion and trial & error, we were able to find a way to accomplish our goal. Some key underlying goals of this exercise were the importance of leadership, communication, and open-mindedness...
Read more